Using Data Science Methods to Explore the History, and Future, of State Secrecy

29 November 2023

Hands on History lecture with Matthew Connelly, Columbia University/University of Cambridge.



For its first one hundred and fifty years, the U.S. government had relatively few secrets, and subsidized the free flow of information to ensure democratic accountability. But all that changed after Pearl Harbor, with the rise of the dark state. The last two decades have witnessed particularly dramatic growth in government secrecy, at the same time that declassification has all but collapsed. Now that so much is secret, it has become all but impossible to protect truly dangerous information. Connelly explores the policies and practices that gave rise to the current crisis, above all presidents’ determination to preserve sovereign power over “national security information.” The digital archiving methods that were supposed to stop unauthorized leaks instead gave rise to Wikileaks, and the wholesale destruction of the documentary record has made it all but impossible even for historians to hold public officials to account. But new data science methods now make it possible to systematically analyze patterns and anomalies in state secrecy, and show not only what the government did not want us to know, but why.


Matthew Connelly


Matthew Connelly is a professor of international and global history at Columbia University, and director of the Centre for the Study of Existential Risk at the University of Cambridge. Connelly is also the principal investigator of History Lab, a project that uses data science to analyze state secrecy. His publications include A Diplomatic Revolution: Algeria's Fight for Independence and the Origins of the Post-Cold War Era, Fatal Misconception: The Struggle to Control World Population, and The Declassification Engine: What History Reveals about America’s Top Secrets. Connelly received his B.A. from Columbia in 1990 and earned his Ph.D. from Yale in 1997. 


Wednesday, 29 November 2023

14.00 - 15.00