The Daily Q: How much digital data has the city already unleashed?

Yesterday, the New York City Council passed the most comprehensive municipal open data bill in the country. Intro 29A will require all city agencies to publish public datasets online in machine-readable formats and provide APIs. The first datasets will be released a year from now and the process will continue through 2018.

But this isn’t the first open data initiative the city has undertaken. The city launched the NYC DataMine in 2009 to host datasets for its first BigApps competition, aimed at producing mobile applications that could save the city money and possibly make money for developers. The portal saw a refresh last year with a new name, NYC OpenData, and hundreds of new datasets.

As we reported, the new open data bill relies heavily on public scrutiny to make sure agencies release as much data as possible. The New York World wants to
know, how many datasets have New York City agencies already disclosed through the NYC OpenData portal and how many times have they been downloaded?

What we found

The NYC OpenData portal lists 898 datasets but it’s light on information beyond a name and description. The New York City Department of Information Technology & Telecommunications (DoITT) says to expect identification soon on the site which agency uploaded each file.

As currently categorized, “public safety” has the most with 113 datasets, followed closely by “education” with 112 and “transportation” with 80. “Women’s rights” contains the fewest, just two, while “construction” and “housing” each have six and “media” and “events” are tied at ten. The most-viewed dataset is wifi hotspots (more than 7,800 views) followed by 311 service requests and parks maps (both at 3,744 views).

So how many more sets are out there? The answer to that question is supposed to be public, but has not been for more than a decade. Since 1989 the city charter has required an annual Public Data directory listing all names and descriptions of all public databases and a specific contact person for each, and city published its first edition in 1993. DoITT published a similar inventory in 2001. The Commission on Public Information and Communication, led by the Public Advocate, is now in charge of the data directory but has never released an edition.

Data Tools


Our work has appeared in…

About TNYW

The New York World focuses on producing data-driven investigative projects.