Amid Controversy, Public NYPD Data Answers Few Questions

The homepage for New York City’s sleek Open Data portal has a section titled Public Safety with a police officer as an icon. Once clicked, a list of datasets opens up and in the first slot, deemed the most relevant, is “NYC Tornado,” data on complaints about tornadoes.

In a city where the debate over police/community relations — from stop and frisk to broken windows — is raging in the streets, data on tornadoes in New York City seems less than relevant. The NYC Open Data website, created after a landmark transparency bill in 2012, includes very little information that is relevant to the current debate on police/community relations and controversial police tactics. It’s not surprising. Of the 145 datasets in that Public Safety section, the NYPD is responsible for uploading four of them.

This is revealing in a number of ways, but mostly shows that a year after Mayor Bill de Blasio and Commissioner Bill Bratton took office, after both promising more transparency in government, the public still has very little access to data from the NYPD.

This has never been more evident than the past few weeks, where crime statistics are dominating the news cycle. Papers are reporting that officers are writing less tickets and making fewer arrests in what appears to be a protest against the mayor following the murder of two police officers in December. On Monday, the mayor said the reported slowdown was due to “a few very aberrant days.” But after seeing all the yearly crime stats during his slideshow, it is evident that the public has very little information on the day-to-day, or even weekly, operations of the NYPD.

There are signs that things will be different this year, but when it comes to openness, 2014 wasn’t much different than the opaque years of Mayor Bloomberg and Commissioner Ray Kelly.

Soon after taking the helm of the NYPD, Bratton said all the right things. At a Citizens Crime Commission breakfast in February, he said, “There should be no secrets in the NYPD,” according to multiple news outlets. “We are going to do more to open up the organization, to make it more inclusive, to make our information more readily available to the public, and to try and format it in a way that is more easily retrievable.”

A week later at a symposium at John Jay College, Bratton told this reporter (for a story for The Crime Report) that legislation wouldn’t be needed to make that happen.

“I don’t think we need City Council legislation requiring it, we’re more than happy to do it,” he said. When pressed on what the NYPD would release, he said, “Eventually we could be in a position to release just about everything we do once we are able to find a way to format it appropriately and push it out.”

That hasn’t happened. In fact, the NYPD has still not released two datasets that were due last year, according to the city’s Open Data Plan that was created after 2012’s Open Data bill passed.

Due on Sept. 1 was a dataset on firearms discharge. Due on Nov. 1 was a dataset on reasonable suspicion stops. Neither has been released yet. Part of the problem is that the law does not have accountability measures, something Manhattan Borough President Gale Brewer, who spearheaded the bill while in the Council, said was something she worked tirelessly to make happen but in the end proved impossible.

“If an agency is not complying, it’s a big problem. You have to have pressure from the outside and inside,” she said.

The NYPD’s has a 2017 deadline to release weekly crime statistics through the portal. The department has also released historical data on its website and through the portal, but the data on its website is more than a year old and in the portal, more than four years old.

Plus, there are problems with the data that the NYPD does release.

Top of that list is the weekly CompStat reports, which are stats compiled by the NYPD and released by precinct, borough and citywide on a weekly basis. Aside from the larger problem of relying on the so-called “seven major” crimes to determine citywide safeness, especially after evidence of downgrading crimes, the information is released as a PDF.

Nothing gets good government groups and transparency advocates more riled up than those three letters, PDF. PDFs are the enemy of open data because numbers on a PDF are not easily downloaded raw so anyone can analyze, chart and map the information as they please.

Councilman Ben Kallos says he’d like to see the NYPD this year release CompStat in a “machine readable format,” and archived. On a weekly basis, the most recent CompStat report replaces last week’s and the NYPD does not keep that archive online.

Kallos introduced two bills last year to get the NYPD to release more data to the public. One (Intro 362) would add violations to the NYPD’s public crime map, as well as exact locations using GPS; the other (Intro 364) would require the city to release the data sets behind city maps (like the crime map) so the public can create its own apps.

“Every New Yorker has a moment where we see a problem and we feel alone,” Kallos said in a recent interview. “I want to provide every New Yorker with the tools to make their case to improve their city.”

The one dataset the NYPD did release last year as part of the city’s open data initiative, “Stop, Question and Frisk Reports,” is troubling. Instead of readable data within the city’s portal like hundreds of other datasets from other agencies, the report links to the NYPD’s website with a link to a zip file. That zip file unpacks a .por file and in order to read the data, requires an SPSS program, which is not standard on personal computers and requires a download from a third-party site, to run. It’s hardly open data.

The NYPD also launched a public crime map in late 2013 following legislation from the Council . But the map has little information beyond CompStat numbers and is often outdated. Since its launch, the data is often more than a month old despite the law clearly stating that it should be updated with data “in no case more than one month after a crime complaint has been filed.” On Thursday, Jan. 8, the most recent data was 38 days old. Sure, it’s minor, but it’s more evidence that even when required by a law to release data, the NYPD doesn’t follow through.

Despite a handful of bills introduced last year to increase NYPD transparency (only one on crime statistics in parks and playgrounds passed), the only major step forward was the release of traffic collision data. While many saw that release as a win for transparency, others had problems with the data for being incomplete and missing important data on the exact location of incidents. And it seems that the release only happened because of pressure from the mayor’s office due to Vision Zero.

“You have to have a lot of pressure from the top. That’s what I believe,” said Brewer.

Brewer credited the NYPD under the new administration with having internal discussions about being more transparent. She said the NYPD has met with her and members of the NYC Transparency Working Group to ask what data they would like to see released. When asked why there seems to be such a resistance by the NYPD to release data, she said she doesn’t know.

“If you asked the [NYPD] that two years ago, it’s because they just don’t put anything out. But we’ve been to a lot of meetings at the NYPD and they’ve been asking the Transparency Working Group, ‘What do you want to see in the portal?'” she said.

“We’re gonna push,” she added.

Kallos said it’s important to have more information as the city debates public safety policies.

“As people are trying to figure out a solution to move forward as a city, we can’t do that without having the information we need,” said Kallos.

The mayor’s office and NYPD did not return requests for comment.

Follow City Limits’ other investigations. Get our free, weekly newsletter.