Thanks to all who came out this weekend for our first-ever hackathon! For those who missed it, our theme challenge was DataSF: find a dataset from the City of SF's online repo of public data and do something cool + useful with it.

How To Improve DataSF?

Before diving in, we had some great discussion on how to make this platform--and the open gov initiative everywhere--work better. Jay Nath (@Jay_Nath), the open-data pioneer who works at the City of SF's Department of Technology and helped spearhead the creation of DataSF, kicked things off Saturday morning with a great backgrounder on how SF got its open-data platform and what the future holds. It was interesting to learn that the city is moving DataSF over to Socrata, the cloud-based open-data platform that already stores public data for Chicago, Seattle, the federal data site Data.Gov, and recently landed its first national customer: Kenya.

Jay readily acknowledges that while launching DataSF was a good first step in the direction of opening up government in San Francisco, the platform could be way better. The biggest challenge seems to be encouraging participation from city workers: DataSF is only useful if the city's best, juiciest, most valuable numbers are constantly being uploaded to the site (and of course, some in city government stand to benefit if that never happens).

Some city officials hope to make it mandatory for public agencies and officials to upload their data to DataSF, and here at The Bay Citizen, we wanted to demonstrate that there is deep civic interest in working with this stuff. So, we teamed up with Burt Herman and Hacks/Hackers and got about 40 people pulling down data and hacking on DataSF visualizations for a weekend.

Some highlights from the weekend:

Lisa Zhang's excellent and creative viz of tags applied to datasets uploaded to DataSF. You can see that the Planning Department is clearly leading in # of datasets uploaded (perhaps unsurprising, since they're used to working with structured data by virtue of their geodata-heavy jobs.)

William Gunn, Mason Wolf and others hacked on a great project about emergency response times. Basically, which SF neighborhoods get their emergencies responded to quickest? This is a project with a lot of promise and civic usefulness, especially if mashed up with Census data on neighborhood demographics, and we're hoping to keep working on it! Stay tuned!

We also had three great tutorial sessions on useful data tools:

TBC's own Aurelio Tinio did a session on Google Fusion, the mapping service we've used for geodata on several of our news apps (see his 5-minute tutorial here!).

Shawn Simister from the Google Refine team showed how to clean and analyze data with that immensely useful tool--Refine has been a huge timesaver for several of our bigger web apps, and we love this tool.

And Al Nevarez (Product Manager of Allegiance.com) and William Gunn (Head of Academic Outreach at Mendeley) did a fantastic two parter on the open-source stats package R, going from the absolute basics to gorgeous advanced visualizations.