How Big Data Is Helping the NYPD Solve Crimes Faster
On December 4, 2015, NYPD officers in New York’s 73rd precinct received alerts on their mobile phones from a new high technology “shot-spotter” system: Eight shots had been fired near 409 Saratoga Avenue in the Bedford-Stuyvesant neighborhood of Brooklyn.
What happened next, according to an account from the local news channel NY1, showed how far technology had come as a policing tool.
When they searched the building’s roof, they found bullet casings. Using their phones again, cops discovered a woman in the building had an outstanding arrest warrant. They got a search warrant over their phones for her apartment, where they found two guns, and made three arrests.
The NYPD officers were able to make these arrests so quickly and easily through the help of a situational awareness (SA) system called DAS, for Domain Awareness System.
Being aware of your situation is not a new idea, of course. In the 1970s and 1980s, there was a theory of leadership called situational leadership that argued for the use of different leadership approaches under different circumstances. Similarly, situational ethics, a philosophy dating back to the 19th century, even prescribes different moral standards for different circumstances.
It’s hard to disagree that people and organizations should vary their behaviors depending on the situation. But in order to do that effectively, they need to know what situation they are in. And that requires some sort of systematic attempt to gather and display information about the environment. Companies have been doing that for decades with their internal environments, but few have built systems to manage information about their external situations—opportunities, threats, competitors, and so forth.
Now, however, there is a budding movement to measure and monitor external situational awareness. Given the proliferation of sensors and signals in the world, it makes absolute sense to bring data on the external environment to one place for monitoring and analysis. Who wouldn’t want to know what is happening in the relevant domains of the outside world?
Well, thus far it appears that public sector organizations are the most interested. While they are not the only ones that need to understand their external situations, they seem to be the only ones developing situational awareness systems.
A group of Canadian government agencies, the city of Chicago, and the New York Police Department (NYPD) are three examples of SA from which private sector organizations can learn. One key lesson gleaned from their experiences is that the more targeted a system is, the better.
MASAS, the Multi-Agency Situational Awareness System, is run by the Canadian Public Safety Operations Organization (CanOps) and is intended to monitor and display information that is relevant to public safety. Thus it includes information about fires, earthquakes, bad weather, traffic problems, road outages, large crowds, shelter locations and status, border crossings, and so on.
The breadth of MASAS is noble, but it seems to limit its value. For example, as the CanOps website notes, because agencies are reticent to share sensitive information with other agencies, all the information shared was non-sensitive (i.e. not terribly useful).
A city is more focused than a country, and the third biggest U.S. city, Chicago, was one of the first to adopt an SA system back in 2012. Called WindyGrid, it is, in the words of Sean Thornton of Harvard University’s Ash Center for Democratic Governance and Innovation, a geographic information system that “presents a unified view of city operations—past and present—across a map of Chicago, giving key personnel access to all of the city’s spatial data, historically and in real time.”
The WindyGrid system contains information on 911 and 311 service calls, transit and mobile asset locations, building status, tweets by geographical origin, and so forth. It’s focused only on spatial data, so it’s a bit narrower than the Canadian system. But given the city’s struggles with violent crime, one might argue that an SA system tightly focused on that should be a higher priority. WindyGrid was developed not by the mayor or some other powerful city leader, but by the city’s Chief Information Officer. It’s a system driven more by information efficiency than strategic priorities.
In New York, terrorism and crime prevention are the clear strategic priority of the NYPD’s DAS system. It was initially developed by the Counterterrorism Bureau, and now is used extensively in daily policing. It collects and analyzes data from sensors—including 9,000 closed circuit TV cameras, 500 license plate readers with over 2 billion plate reads, 600 fixed and mobile radiation and chemical sensors, and a network of ShotSpotter audio gunshot detectors covering 24 square miles—as well as 54 million 911 calls from citizens. The system also can draw from NYPD crime records, including 100 million summonses.
The DAS project started in 2008 and has been refined ever since. In 2010 analytics were added to the system. In 2011 an automated pattern recognition capability was added, and 2014 saw the introduction of “predictive policing” functions (the other SA systems mentioned provide purely descriptive analytics). And in 2015 cops were able to get real-time 911 information.
The primary interface with the system is the smartphone; all 35,000 NY police officers have them now. Over 10,000 cops use DAS every day. The system is a technical marvel, and it came about because of strong leadership and priorities.
NYPD Commissioner Bill Bratton has been perhaps the most aggressive advocate of data-based policing in the world. He introduced the CompStat performance management system to the NYPD in his first term there in 1994, and also employed it in his Los Angeles commissioner stint. Jessica Tisch, the NYPD’s Deputy Commissioner of Information Technology, helped to introduce the DAS idea when she was head of policy and planning at the Counterterrorism Bureau. And Evan Levine, an astrophysics Ph.D. who is head of analytics for Counterterrorism, supplied the data science expertise for the predictive analytics algorithms in DAS. The NYPD worked with Microsoft (MSFT) to implement DAS.
The focused nature of the NYPD system is a big reason for its success. Of course it’s risky to attribute outcomes to a single cause, but crime in New York City is down (below the U.S. national average) and homicide clearance rates are up. I’m sure that DAS is a factor, but probably more important is the overall culture of evidence-based policing that characterizes the NYPD.
Should your organization implement some form of situational awareness system? Absolutely. It’s increasingly possible to know what’s going on outside the walls of your organization, and that might ultimately affect its success. But given the breadth and complexity of that outside world, you should probably focus your SA system on something like customers, competitors, or regulators. Eventually you may be able to understand your entire situation, but make progress on a specific aspect first.