Swiftly.org

Download Sweeper V0.3.1 With Support for Twitter Streaming API here

Any of you keen Swiftriver lovers out there will remember the recent launch of our 0.3 version of the Sweeper app. The release went well and we got loads of feedback.

We were able to identify a couple of bugs in the release that were causing headaches for you guys and girls out there trying to set Sweeper up. So we fixed them …

Fixed the mysql table engine bug, thanks to everyone on the Swiftriver google group for finding and helping to fix this!

Fixed the link back to original content in the content source popup.

So why didn’t we give you these fixes a week ago … well we wanted to hold off as we were sooooo close to finished two other pieces of work that it didn’t seem right to give you the bug fixes without a little ‘we’re sorry’ present, so here it comes:

1. Swiftriver and Sweeper now support collecting content from the Twitter Streaming API. This is a massive win for our software and we know that there will be loads of you who want to check it out. Its still early doors on this at the moment and there are a few things that we haven’t quite got right – for example we cant tag and translate all the content coming from a stream yet – but we didn’t feel that this was reason enough to hold you dudes back. So if you LOVE data and want to see a whole lot of it, download the latest V0.3.1 release of Sweeper and get cracking. 
2. Sweeper can be now be deployed behind a proxy. This is great news for any of you wanting to use Sweeper on your corporate networks and we know you have had issues with this in the past, well they are now truly in the past!!!

We hope that you love this latest mini release as much as we do and as always, we love your comments, questions and criticisms (yes we really do!)

Just a small note for those of you who are already running a version of Sweeper but who want to take advantage of the new features in this release … while Sweeper does not yet officially support upgrades, it is possible. Basically all you have to do is dump the new code over the existing install then run through the installer (http://[where-you-installed-sweeper]/installer) and when you get the ‘create admin user section’, just skip it (just click on ‘6:Proxy Server Setup’ at the top of the page) without entering a new admin password. Hope this helps, and we are working on an official upgrade path to make adoption of the latest versions of Sweeper easier in the future!

So that’s all for now folks, till next time,

Matthew Griffiths,

Director of Platform

SwiftRiver

Download Sweeper V0.3.1 With Support for Twitter Streaming API here

Sweeper has been around for just over a year now and although the code is still changing quite rapidly. However, the core platform is now stable enough that we’re releasing this User’s Guide. It’s certainly long overdue. Sweeper is the a free and opensource multimedia curation platform that we’ve been developing over the past few months. It’s a quick and easy way to create a social media monitoring dashboard. You can find the User Manual embedded below or at http://swiftly.org/userguide

So what are some of the things one can perform with the Sweeper application?

• Combine many feeds into one: then curate, filter and translate
• Count and archive Twitter mentions
• Mashup content from Email, Twitter, Blogs, Flickr, News sites, RSS/ATOM etc.
• Automate the addition of context to data: Location, priority, influence, reputation, tags etc.
• Structure unstructured data
• Buffering against an excess of crowdsourced data
• Set-up pipes of conditional logic for automating data processes
• Collect and manage realtime data from SMS while completely offline
• Translate real-time content from social media on the fly
• Datamine real-time content aggregated by you or your team
• Keyword monitoring from Twitter, Posterous, Blogger, Google News and Wordpress
• Native integration with Ushahidi and Crowdmap

Sweeper can be downloaded today from Swiftly.org.

Sweeper User Guide v0.3 View more documents from Ushahidi

The Google Summer of Code is drawing to a close. This year SwiftRiver was a mentoring organization and we wanted to give our GSoCers some ‘face’ time on the blog by interviewing them.  Mang-Git is a developer who worked on our influence analytic application, Reverberations.  

Reverberations is a pretty simple app, it’s a RESTful solution for determining the influence of content online.  Mang worked on the Twitter portion, where an algorithm returns the number of retweets an item has, and then the number of retweets those retweets have as well.

Reverberations is an open source project which can be found here: http://github.com/appfrica/Reverberations

Meet Mang-Git

Q: What is your educational (or professional) background?

I am a recent graduate of the University of Michigan, with an undergraduate degree in Computer Engineering. Over the past couple summers I have interned at Cisco Systems, as a Software Development intern. During the school year I worked as a computer technician for the School of Social Work at U of M.

Q: What was the project you worked on for us? Why did you select that as your GSoC project and what did you learn from working on it?

The specific project I worked on was called Reverberations, it is an add on to the SwiftRiver tool set. I chose to work on SwiftRiver, because of the far reaching social implications the software had. From my time working on Reverberations I have learned a lot about social networking, the REST architecture, as well as how to use the twitter API to gather the information I needed.

Q: What challenges did you run into during and how did you overcome them?

The largest challenge I came across was determining how to create a retweet tree. Twitter does not provide information on whether a retweet is a retweet of another retweet.

example…

All retweets on twitter are displayed as retweets of the original tweet. To make matters even more difficult, I did not notice that the status_retweets API call returned all retweets as a retweet of the orignal tweet until the last week of GSoC. Eventually with some help from the Twitter developer forum I came up with an algorithm to create a retweet tree, using retweet timestamp information and info on the followers of a user. The most difficult part about developing this “algorithm” was figuring out what the implications are for each piece of information, as well has how to handle situations when it is impossible to determine who the likely parent tweet is. In the end I’ve created an algorithm, that although cannot always be a 100% sure of whether the retweet tree is correct, but is better than any other solution I have seen, especially since I have yet to find an application that even attempts to create a retweet tree using information form the Twitter API.

Q: GSoCers get to choose the organizations they work with, why did you choose to work with SwiftRiver?

SwiftRiver, along with Ushahidi, not only provides crucial information to rescuers during a crises, but it can also provide realtime, crowd sourced news to the general public. An example of this would be using Ushahidi and Reverberations in a situation such as the riots in Iran. Allowing reporters to filter and manage the huge amounts of user generated content flowing out of Iran, and thus helping spread the news of Iran even when western reporters were not allowed in Iran.

Q: Any closing remarks?

GSoC is a great program, and provides an excellent opportunity for students like myself to participate in open source projects, and be rewarded for giving back to the open source community. My participation with SwiftRiver has taught me a lot about the software development process. I have had to do everything from managing my time, to defining a spec from a concept idea, to learning how to use the tools needed for developing my program, and finally actual development and testing of my code. Working with Jon and GSoC has really been an excellent experience, I feel that I have learned a lot and grown as a developer.

Thanks, Mang-Git!

On August 30th we’ll release the first public beta of the SwiftRiver platform, an open source toolkit of semantic web technologies. It’s been a busy few months as we’ve been working round the clock to bring you a solid product.

One of the questions I’m asked frequently is “What can I use SwiftRiver for?” Here are a few examples:

1. Monitoring Real-Time Conversations

The most obvious use of Swift, what organizations like Ushahidi will use it for, is to help manage large streams of real-time content. Whether from blogs, twitter, email, SMS or other means, when something happens (ex. the Haiti Earth Quakes or Kanye West and Taylor Swift at the VMAs) there’s a flurry of activity immediately following the event. For someone collecting news on that subject, for whatever reason, we envision that they would download an instance of Swift, and begin monitoring a number of sources discussing said event.

2. Brand Monitoring

Similar to the scenario above, a PR firm or advertising agency might use Swift to monitor mentions of company’s brand online. This would of course include mentions on Twitter, but might also include SMS and Email campaigns.

3. Data Collection / Research

As a research tool, Swift’s veracity algorithms can be used to curate sources and content that the user trusts to offer more accurate information.

4. Sweeping through Email

If you don’t understand anything about the real-time web or the aforementioned ideas. One of the things almost everyone can relate to, is the need for ways to improve email filtering. Swift is something anyone can set up to help them sort their email by ranking the people you are likely to want to hear from higher than the people you don’t. Meanwhile, users can apply language processing tools to automatically sort email by subject, category or sender.

5. Sweeping Through SMS

Even users who don’t deal with the web at all may find use in SwiftRiver. For one, not all real-time data is online. If you’re on a closed network, you can use it to process text messages received from a local gateway. This useful for users of tools like Frontline SMS or Kannell.

6. Creating a Public Aggregator

One of our pilot partners used Swift to create public ‘planet’ style aggregator and news portal. This required some custom work from our end but we’re excited for their launch.

7. Monitoring Hundreds of Blogs/Sources

Perhaps you’re just a person, a blogger or journalists, who consumes large amounts of information on a number of subjects, like me. I currently follow about 2000 blogs in Google Reader. Reader is extremely useful because I can aggregate whatever I want. From the aggregated datasets, I can then choose to read and share whatever I want. Likewise, in Swift having too much information is actually a good thing, there are still serendipitous ways for navigating content (using tags), as well as a number of filters for viewings items in a more structured manner.

8. Building Apps on SWS

A few days a go we received a number of tweets about an app called FlipBoard, asking if Swift was anything like it. SwiftRiver is actually a very different animal. We’re more like the stack that something like FlipBoard would be built on. We offer several advanced tools (social graph mining, natural language processing, location servers, twitter analytics) for free use via our open API platform Swift Web Services. Anyone can use them and thus anyone can build applications on top of them.

We’ve been working with large media organizations around the world to customize such tools for their needs but because our stack is open, so can you!

9. Dashboard and Shared History Across Media Channels

The most basic feature that makes many of the above possible is that Swift allows you to create dashboard that includes messages from a number of sources and lets you sort, search and curate them all any way you want. This might include videos, tweets, email, text messages, blogs. All of the content you have a need to mine for information, for any reason is also possible.

10. Improving Your Blog

In addition to using Swift to collect research, bloggers are using Swift Web Services for their blogs. Users of Wordpress or Drupal can add features like auto-tagging and more using Swift Web Services.

---

These are just some of the ways our alpha testers have been using Swift, there are many more possibilities and we look forward to exploring after our Beta! To find out more about Swift, try these recent posts from Robert Scoble, the BBC and GigaOM.

On April 15th at this year’s Twitter developer conference, Tim O’Reilly moderated a panel with Katie Stanton (White House Director of Citizen Participation), Anil Dash (Expert Labs), and Patrick Meier (Ushahidi) called “Twitter as a Force for Good”. They discussed how Twitter was an incredible platform for information gathering for emergency response organizations during the Haiti earthquakes. Other topics of conversation ranged from how the US government keeps up with the latest web technologies, tech innovations originating outside of Silicon Valley and the U.S., and leveraging crowd-sourcing as a way to improve policy decisions. Check out video of the full panel below…


Watch live video from Twitter Chirp Conference on Justin.tv