Anthony M. Townsend - Smart Cities - Big Data, Civic Hackers, and the Quest for a New Utopia

We shouldn’t be surprised that wireless has won us over. Almost a century ago, at the very dawn of the untethered age, Nicola Tesla saw clearly the world into which we are now moving. A visionary pioneer of electricity and radio technology, Tesla laid the future bare in 1926 in Colliers magazine: “When wireless is perfectly applied the whole earth will be converted into a huge brain, which in fact it is, all things being particles of a real and rhythmic whole.”

Representatives and direct Taxes,” read the new government’s charter in 1787, “shall be apportioned among the several States which may be included within this Union, according to their respective Numbers....” This single sentence of the US Constitution begat the Census, the Census begat IBM, and then IBM begat the modern world. An oversimplification of biblical scale, but allow me to explain.

At less than five thousand words, America’s Constitution is one of the world’s shortest government charters. But despite its brevity, its authors didn’t leave important details to chance. On the very first page, they laid out not only the formula for divvying up seats in the new legislature, the House of Representatives but also the process by which data to feed the calculations should be collected. “Enumeration shall be made within three Years after the first Meeting of the Congress of the United States,” the Constitution reads, “and within every subsequent Term of ten Years, in such Manner as they shall by Law direct.”And so, the census was born.

The first count began on Monday, August 2,1790, just over a year after President George Washington’s inauguration.2 In 1793 the full results were published. In fifty-six pages of elegantly typeset tabulations, Return of the Whole Number of Persons Within the Several Districts of the United States described a land of villagers and farmers—barely one in twenty Americans lived in cities and towns in 1790.3 In New York City, already the nation’s largest settlement, dwelled a mere 32,328 persons. This pattern would hold for decades. As late as 1840, just 10.8 percent of the nation’s population were city dwellers. But the industrial revolution would change all that. From just 2 million townspeople in 1840, America’s urban population grew to over 50 million in 1920, when for the first time they outnumbered the country folk.

As the nation grew, the census grew in scale too. The first census, conducted house by house in 1790, found slightly fewer than 4 million souls in the land. By the tenth count in 1880, some 50 million persons were enumerated.

The scope of data points gathered on each person expanded dramatically as well. Despite the ravages of war, America remained a magnet for immigrants, who showed up in astonishing numbers. From 1850 to 1880, an average of nearly 1.5 million arrived each decade.3 Alarmed at the unprecedented growth of immigrant ghettos in the cities, a Congress still mostly dominated by rural landowners authorized an expansion of the census’s demographic data collection. General Francis Amasa Walker, the economist who had overseen the 1870 census, was tapped again to plan the new survey. He added questions about marital status, birthplace of parents, and length of residence in the United States, and two questions on mental health (Question 18, “Was the person idiotic?” and Question 19, “Was the person insane?” explored apparently obvious contemporary distinctions). More importantly, for the first time the census included a massive survey of the economy, tallying its manufacturing, mining, agriculture, and railroad sectors. The 1870 census reported back in just three volumes; the 1880 report swelled to twenty-two.

The broadened scope of its inquiry overwhelmed the Census Office, then still a temporary group in the Department of the Interior. (The permanent Bureau of the Census would not be established until 1902). Despite tripling the number of staff from the 1870 effort to over 1,500 workers, the full tabulation of the 1880 census lasted seven years.9 By the time it was completed in 1887, the next count was just three years away. While plans for the 1890 census called for even more staff, many feared that given the accelerating pace of change in the population and economy, “the 1890 figures would be obsolete before they could be completely analyzed.”10 This decoupling of the nation’s demographic and economic reality from what could be measured was yet another dimension of the “control revolution” of the late nineteenth century we saw in chapter 1, when “innovations in information-processing and communications technologies lagged behind those of energy and its application to manufacturing and transportation,” according to sociologist James Beniger.11 We were building cities faster than we could count the people pouring into them.

The solution to the young nations counting problems would come from a former Census Office clerk turned engineer and entrepreneur. Hired to work on the 1880 census, Herman Hollerith hailed from Buffalo, New York. At the Census Office, Hollerith befriended John Shaw Billings, who headed the Division of Vital Statistics. Hollerith and Billings often discussed new approaches to the problem of tabulating the massive piles of data being collected. As Hollerith later recalled, “One Sunday evening at Dr. Billings’ tea table, he said to me there ought to be a machine for doing the purely mechanical work of tabulating population and similar statistics. We alked the matter over.... He thought of using cards with the description of the individual shown by notches punched in the edge of the card.” Punch cards had been used to control machinery since the 1801 invention of the Jacquard loom, a French machine that used thousands of cards to weave extremely complicated patterns in textiles. Their application to data processing held tantalizing potential.

When General Walker left the Census Bureau in 1881 to take over as the president of MIT, he invited Hollerith to join him as an instructor in mechanical engineering. Hollerith soon tired of teaching, however, and found his way back to Washington to work as a patent examiner. Over the course of a busy year, he familiarized himself with the art of patent writing and prior art in punch-card technologies. Supporting himself as a consultant to other aspiring inventors, during the next several years Hollerith began building the tabulating machine first imagined in his conversations with Billings.

Hollerith’s machine was remarkably simple to operate. To process a card, which had been punched to record the characteristics of a single individual captured by the Census, the operator simply placed it on a rubber pad, beneath which lay dozens of tiny cups of mercury, and with a handle lowered a swinging array of metal pins. As pins passed through the punched-out holes in the card, they would contact the mercury and close a circuit with a tiny electric motor. On a panel facing the operator, four rows of ten clocklike dials represented the various data items encoded on the card—race, gender, age, and so on. With each pulse, the indicator would advance. Using two rotating hands and a circumference marked off into 100 ticks, it could track sums up to 9,999. From time to time, the operator would read the dials, jot down the totals and reset them to zero.

Compared to manual tabulation, this system was blazingly fast. In June 1890, ninety-six of Holleriths machines were put to task processing the results of the new census, taken on the second of that month. By the end of the summer, the machines’ impact was clear—the raw population count of Washington, DC, was announced on June 28, and New York City a few weeks later on July 18. By the end of August, the full tally of every state, which enumerated over 60 million Americans, was completed. A full statistical breakdown was published in 1892.15 Hollerith boasted that the bureau could now process a stack of forms the height of the Washington Monument in a single day.