Jörg Dräger - We Humans and the Intelligent Machines

In New York City, algorithms play the same role that the three mutants do for Dick and Spielberg: They provide crime forecasts. Yet with one decisive difference: The computer does not predict who will commit a crime in the near future but where it will take place. The term for this is “predictive policing.”

And it works like this: Software evaluates the history of crime for each district of New York in recent years and compares the identified patterns with daily police reports. Crime may seem random at first glance, but in fact certain crimes such as burglary or theft adhere to patterns that can be worked out. These patterns depend on demographics, the day of the week, the time of day and other conditions. Just as earthquakes occur at the edges of tectonic plates, crime takes place around certain hot spots, such as supermarket parking lots, bars and schools. The predictive policing software marks small quadrants of 100 to 200 meters in length, where thefts, drug trafficking or violent crimes have recently taken place, which – according to the analysis – are often followed by other crimes.

Since law enforcement officers started using predictive policing, their day-to-day work has changed. In the past, they were only called when a crime had already been committed and needed to be solved. Today, the computer tells them where the next crime is most likely to occur. In the past, they often took the same route every day, but now the software determines so-called crime hotspots where they need to be present to monitor what is going on. The police can thus better plan and deploy their resources and work more preventively. “The hope is the holy grail of law enforcement – preventing crime before it happens,” says Washington law professor Andrew G. Ferguson. 6New York Mayor Bill de Blasio sees this in a more pragmatic and less poetic way: Algorithmic systems, he argues, have made police work more effective and more trustworthy. The city is now safer and more livable. 7In fact, within 20 years the number of murders in New York City has fallen by 80 percent to only about 350 per year. Thefts and robberies also fell by 85 percent. It is not possible to determine exactly how much predictive policing has contributed to this. In any case, the software enables policemen to be where they are needed most.

The specific functioning of the algorithms, however, remains hidden from the public: How do these programs work? What data do they collect? There are lawsuits pending against the New York police for violating the Freedom of Information Act. People have just as little knowledge about where the algorithms are used, the plaintiffs argue, as they do about how the calculations take place. The first court to hear the case ruled in favor of the plaintiffs. Nevertheless, the police continue to refuse to publish detailed information about their predictive policing.

The New York Fire Department also prefers preventing fires to extinguishing them. 8But like the police, it struggles with limited resources. Not all of the 330,000 buildings in New York can be inspected every year. The firefighters must therefore set priorities and identify the buildings most at risk. But which ones are they? This selection process alone used to occupy an entire department. For a few years now, the firefighters have been using a computer program that algorithmically calculates the risk of each building catching fire. Taking into account the size, age, building material, pest infestation and inhabitant density as well as the history of fires in the neighborhood, the algorithm creates an inspection list for the next day (see Chapter 10).

“Smaller, safer, fairer.” 9Using this motto, Mayor de Blasio presented his plan to close New York’s largest prison in June 2017. 10In the 1990s, most of the city’s then 20,000 prisoners were incarcerated on Rikers Island, once known as the new Alcatraz. By now, less than 10,000 New Yorkers are imprisoned and Rikers Island, which costs $800 million a year to run, is partly empty. Moreover, the prison has recently been shaken by a scandal about the mistreatment of a juvenile detainee. De Blasio therefore has several reasons for wanting to close the facility. He also would like to further reduce the number of prisoners: to 7,000 in five years and to 5,000 in the long term.

His biggest lever: algorithms. They are supposed to help New York’s judges better assess risks, for example, whether pre-trial detention is necessary or whether an early release is adequate. The probabilities to be assessed here are, in the first case, the danger that the alleged person will flee before the trial and, in the second case, the threat of recidivism. These probabilities depend on so many factors that a judge can hardly be expected to evaluate all of them adequately in the time allotted for each case.

COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) is the software that calculates the risk of flight and recidivism. While the company that developed the program refuses to publish the algorithm behind it, research by ProPublica, a non-profit organization for investigative journalism, has shown that such systems collect and analyze a large amount of data, such as age, gender, residential address, and type and severity of previous convictions. They even gather information on the family environment and on existing telephone services. All in all, COMPAS collects answers to 137 such questions.

The potential for providing algorithmic support to judges is huge. In a study in New York City, researchers calculated that if prisoners with a low probability of recidivism were released, the total number of detainees could be reduced by 42 percent without increasing the crime rate. 11In Virginia, several courts tested the use of algorithms. They ordered detainment only in half as many cases as when judges issued a ruling without such software. Despite that, there was no increase in the rate of people who did not show up for their trial or who committed a crime in the interim.

Algorithmically supported decisions improve forecasts even if they do not offer 100-percent accuracy. In addition, they could also reduce variations in the sentences handed down. In New York City, for example, the toughest judge requires bail more than twice as often as the most lenient of his colleagues. The fluctuations may be due to the attitude of the judges but also to their workload, since they only have a few minutes to decide what bail to set.

What promises advantages for society can, however, result in tangible disadvantages for the individual. Hardly anyone knows this better than Eric Loomis, a resident of the state of Wisconsin. In 2013, he was sentenced to six years in prison for a crime that usually draws a suspended sentence. The COMPAS algorithm had predicted a high probability of recidivism, contributing to the judge’s decision in favor of a long prison sentence. The discrimination that can result from the use of algorithms will be discussed in more detail in Chapter 4.

Every autumn in New York City, the application phase for high school begins. 12For many parents this is a time of stress and uncertainty because there are too few places at the popular schools known for getting their students into good colleges and thus providing better career prospects. The teenagers and their parents research secondary schools for months, and some have taken admission tests or gone in for interviews. The right high school should be academically challenging, have good sports facilities and ideally be located in the neighborhood. Naturally, it would also have a high graduation rate and be seen as competitive. Approximately 80,000 young people and their parents have until December 1 to choose 12 schools from over 400 options on the application form. The following March, the Department of Education will tell them which school they can attend.