Disappearing state web archives prompt questions about preservation

Tappan Zee Old Site

The original site for the Tappan Zee Bridge Project as it currently appears after not being available for roughly two weeks.

A roller coaster of changes to the information-packed website for the rebuilding of the Tappan Zee Bridge has frustrated area residents and transit advocates. At the same time, it has raised questions about how New York preserves its digital public records in light as the Cuomo administration moves to expand the state’s online information offerings.

Before October 11, 2011, the web homepage for the nine-year-long Tappan Zee Bridge Environmental Review billed itself as part of the “extensive public outreach program that is at the center” of the project, one of the state’s largest public works underway. Offering research reports, minutes or slides from nearly 300 public hearings and comments from the public, the site documented an almost decade-long environmental review and planning process that weighed four construction options to widen the bridge and add mass-transit alternatives.

Two weeks ago, however, Governor Cuomo and the Federal Highway Administration announced that the project was being fast-tracked and moving forward in a new, less ambitious form. At the same time, the project’s website — archives of almost a decade’s worth of taxpayer-funded work — was replaced by a bare-bones site featuring a handful of documents that briefly announced the change in direction. A document on the transformed website also noted that transit would not be considered, a major point of contention, as first reported by blogger Cap’n Transit.

Cuomo’s New York City spokesman, Matt Wing, said Monday that the website was “undergoing a transition.”

Tuesday evening, however, following complaints from transit advocates and elected officials, the agencies in charge of the site — the Federal Highway Administration, the New York State Department of Transportation and New York State Thruway Authority — restored the original website after it had inaccessible for roughly two weeks. The agencies have since held public meetings at which Department of Transportation Commissioner Joan McDonald said that mass transit could be added when “financially feasible,” the Lower Hudson Journal reported.

The Cuomo administration is now moving to make more government documents available online, yet nothing compels state agencies to disclose materials online, according to Bob Freeman at the NYS Committee on Open Government. In other words, the Tappan Zee public outreach was purely voluntary.

Even 20th-century technology for managing and retaining documents remains a challenge for some state agencies, says Brian Burke, senior program associate at the Center for Technology in Government at SUNY Albany. “Government never got the paper process perfect,” he said. “In translating it to the electronic process, there’s still a problem of understanding the scope and process.” The New York State Archives, for instance, depend on state agencies and local governments to deliver documents for archiving. In the paper document days, state agencies “were very inconsistent about giving all of the records they should be handing over,” said Burke, adding that the State Archives didn’t have the ability to follow up with every local agency to ensure they had what they needed.

The challenge is even more complex with the proliferation of electronic records that arrive in unstandardized formats, from PDFs to databases. “The technology has changed so quickly that the standards of long-term preservation of certain kinds of record created in electronic format don’t exist,” said New York State Archivist Christine Ward. Her office is currently working with colleagues in other states to develop best practices.

The Washington State Digital Archives have been on the forefront of digital record keeping since the early 2000s. Through the archives’ website, citizens can download birth records, census records, maps, minutes, historical photographs and other documents that save citizens and the state considerable time and money since state employees no longer spend their time fetching records. Washington State learned a vital lesson in the late 1990s when it won a $4.5 billion settlement against the tobacco industry in part due to its ability to retrieve records from the 1940s and before.

Washington also has the advantage that its Secretary of State has a constitutional mandate to preserve the state’s historical records and the ability to independently create protocols. The New York State Archives operates under the Department of Education and often requires approval from the Attorney General, Office of the State Comptroller and Commissioner of Education to perform certain tasks.

The New York State Archives computers “crawl” each government agency’s website annually, creating a snapshot for the records, said Ward. The snapshots are available through the State Archives’ website by searching for the agency name and “web site” in quotes. (In many cases, including the Tappan Zee project, government archives can be found via the Internet Archive’s Wayback Machine.)

When The New York World tried searching for the Tappan Zee Environmental Review site, however, the results turned up with nothing. The help desk responded to an email saying that the site was not crawled during its nine-year existence due to “technical limitations” with sites run by the Department of Transportation.

Data Tools


Our work has appeared in…

About TNYW

The New York World focuses on producing data-driven investigative projects.