The Electronic Records Archives (ERA) Program National Archives and Records Administration 1 ERA...

National Archives and Records AdministrationNational Archives and Records Administration 11

The Electronic Records The Electronic Records Archives (ERA) ProgramArchives (ERA) Program

ERA Program and Geospatial Records Research UpdateERA Program and Geospatial Records Research Update

Map and Geographic Information Collections in Transitions ConferenceMap and Geographic Information Collections in Transitions Conference

The Cartographic Users Advisory CouncilThe Cartographic Users Advisory Council&&

The Library of Congress Geography and Map DivisionThe Library of Congress Geography and Map Division12 May 200512 May 2005

Robert ChadduckRobert Chadduck

Director of ResearchDirector of Research ERA ProgramERA Program


Background: Background: Federal Geospatial Federal Geospatial

RecordsRecords

A Federal RecordA Federal Record Any piece of information, regardless of form or characteristics, made or received by an Any piece of information, regardless of form or characteristics, made or received by an

agency in course of businessagency in course of business That is, or should be, keptThat is, or should be, kept

As evidence of that activityAs evidence of that activity Because it contains valuable dataBecause it contains valuable data

44 U.S.C. 330144 U.S.C. 3301

NARA NARA Provides direction, guidance, and assistance to all agencies in managing their records, Provides direction, guidance, and assistance to all agencies in managing their records,

and and Preserves records with “enduring” value in the National ArchivesPreserves records with “enduring” value in the National Archives Through lifecycle management of recordsThrough lifecycle management of records


Background:Background:

Applicable Federal Directives and GuidanceApplicable Federal Directives and Guidance

36 CFR Subchapter B - Records Management 36 CFR Subchapter B - Records Management Part 1228 -- Disposition of Federal RecordsPart 1228 -- Disposition of Federal Records

Scheduling Records Scheduling Records Transfer of Records to Records Storage FacilitiesTransfer of Records to Records Storage Facilities Transfer of Records to the National Archives of the United States Transfer of Records to the National Archives of the United States

1228.268 Cartographic and architectural records 1228.268 Cartographic and architectural records 1228.270 Electronic records1228.270 Electronic records

Guidance (examples)Guidance (examples) Disposition of Federal RecordsDisposition of Federal Records Documenting Your Public ServiceDocumenting Your Public Service Managing Audiovisual RecordsManaging Audiovisual Records Managing X-Ray Films as Federal RecordsManaging X-Ray Films as Federal Records Moving? Consolidating? Reorganizing? Moving? Consolidating? Reorganizing? Vital Records and Records Disaster Mitigation and RecoveryVital Records and Records Disaster Mitigation and Recovery


Background:Background:Types of Cartographic Records PreservedTypes of Cartographic Records Preserved

in the National Archives in the National Archives

Exploration and Scientific Surveys Exploration and Scientific Surveys Public Land Surveys and Settlement Public Land Surveys and Settlement Indian Affairs Indian Affairs Hydrography and Navigation Hydrography and Navigation Topography and Natural Resources Topography and Natural Resources Census Mapping Census Mapping Urban Development Urban Development Maps of Foreign Countries Maps of Foreign Countries Military Campaigns Military Campaigns Aerial Photographs Aerial Photographs


Background Background Digital Spatial Data Records Digital Spatial Data Records

in the National Archivesin the National Archives

Records of The National Agricultural Statistics Service Records of The National Agricultural Statistics Service Crop Estimates: County-level Estimates Of Crop Yields, 1972-1989 Crop Estimates: County-level Estimates Of Crop Yields, 1972-1989

Records of The Bureau of The Census Records of The Bureau of The Census Annual Housing Surveys/American Housing Surveys (AHS), Standard Annual Housing Surveys/American Housing Surveys (AHS), Standard

Metropolitan Statistical Areas Metropolitan Statistical Areas Census of Population And Housing, 1950, Public Use Sample, 1/100 Sample Census of Population And Housing, 1950, Public Use Sample, 1/100 Sample

Records of The U.S. Joint Chiefs of Staff Records of The U.S. Joint Chiefs of Staff Combat Air Activities File (Cacta), Oct. 1965-dec. 1970 Combat Air Activities File (Cacta), Oct. 1965-dec. 1970

Records of The National Institute of EducationRecords of The National Institute of Education National Assessment of Educational Progress, Study Year 1, 1969-1970 National Assessment of Educational Progress, Study Year 1, 1969-1970

Records of The Historic American Buildings SurveyRecords of The Historic American Buildings Survey Historic American Buildings Survey/ Historic American Engineering Records Historic American Buildings Survey/ Historic American Engineering Records

DatabaseDatabase Records of The Law Enforcement Assistance Administration Records of The Law Enforcement Assistance Administration

Capital Punishment In The United States, 1973-1987 Capital Punishment In The United States, 1973-1987 Records of The Environmental Protection AgencyRecords of The Environmental Protection Agency

Toxic Chemical Release Inventory SystemToxic Chemical Release Inventory System


Current Standards for Current Standards for Transfer of GIS RecordsTransfer of GIS Records

Electronic Records Management (ERM) E-Gov Electronic Records Management (ERM) E-Gov Initiative. Initiative. One of the 24 E-Gov initiatives under the President's One of the 24 E-Gov initiatives under the President's

Management AgendaManagement Agenda Geospatial data records from GIS are a priority electronic records Geospatial data records from GIS are a priority electronic records

formatformat

NARA accepts digital geospatial data fully NARA accepts digital geospatial data fully implementing a non-proprietary, published, open implementing a non-proprietary, published, open standard. standard.


Current Standards for Current Standards for Transfer of GIS RecordsTransfer of GIS Records

Examples include, but are not limited to:Examples include, but are not limited to:

Spatial Data Transfer StandardSpatial Data Transfer Standard http://www.fgdc.gov/standards/status/textstatus.http://www.fgdc.gov/standards/status/textstatus.html)html)

Geography Markup Language, versions 2 Geography Markup Language, versions 2 and 3and 3 (Open GIS Consortium). (Open GIS Consortium). (http://www.opengis.org/docs/02-023r4.pdf)(http://www.opengis.org/docs/02-023r4.pdf)


Genesis of the ERA ProgramGenesis of the ERA ProgramSome ObservationsSome Observations

While NARA has more than three decadesmore than three decades of practical experience in electronic records preservation, there is no single unifying model or applicable technical benchmark for formal technical digital object preservation into perpetuity

Facets of technical issues generally associated with the federal government’s electronic records challenge actually constitute domain specific outcroppings of general problems under active research in computer science and engineering in the operational federal computing context.


ERA Program VisionERA Program Vision

“The Electronic Records Archives will authentically preserve and provide access to any kind of electronic record, free from dependency on any specific hardware or software, enabling NARA to carry

out its mission into the future.”


NARA’s Electronic Records NARA’s Electronic Records Archives StrategyArchives Strategy

Define the requirements in terms of the lifecycle Define the requirements in terms of the lifecycle management of recordsmanagement of records

Find solutions in commercially viable, Find solutions in commercially viable, mainstream technologies being developed to mainstream technologies being developed to support e-commerce, e-government and the next support e-commerce, e-government and the next generation national information infrastructuregeneration national information infrastructure

Align with overall direction of IT in the U.S. Align with overall direction of IT in the U.S. GovernmentGovernment


ERA Program Update:ERA Program Update:Major System Acquisition Activity Major System Acquisition Activity

NARA ERA ProgramNARA ERA Program

FY2003 $11.8M appropriation FY2003 $11.8M appropriation (P.L. 108-7)(P.L. 108-7)

FY2004 $35.9M “Line Item” appropriation FY2004 $35.9M “Line Item” appropriation (P.L. 108-199)(P.L. 108-199)

FY2005 $35.9M “Line Item” appropriation in Consolidated FY2005 $35.9M “Line Item” appropriation in Consolidated

Appropriations Act, Appropriations Act, {P.L.108-447}{P.L.108-447}

FY2006 President’s Budget includes $35.9M FY2006 President’s Budget includes $35.9M

“Line Item” appropriation “Line Item” appropriation


Select Single Developer

Initial Initial Operating Operating CapabilityCapability

Awarded Contracts Full Operating Full Operating

CapabilityCapability

FY 2007FY 200712/2003 8/2004 4th

Qtr.2005

NARA Released the RFP

Increment 1

2 years

Increment 2

(1 year)

20112011

Increment 3

(1 year)

Increment 4

(1 year)

Increment 5

(1 year)

ERA TimelineERA Timeline


ERA Program UpdateERA Program UpdateThe Competitive AcquisitionThe Competitive Acquisition

Lockheed Martin Corporation:Lockheed Martin Corporation: Transportation and Security Solutions Division Transportation and Security Solutions Division Firm-fixed price design contract value: Firm-fixed price design contract value:

$9.5 million$9.5 million Harris Corporation:Harris Corporation:

Government Communications Systems DivisionGovernment Communications Systems Division Firm-fixed price design contract value: Firm-fixed price design contract value:

$10.6 million$10.6 million


ERA Program UpdateERA Program UpdateThe Competitive AcquisitionThe Competitive Acquisition

At the end of the one-year design At the end of the one-year design competition, one of these two contractors competition, one of these two contractors will be selected to build the ERA Systemwill be selected to build the ERA System

Over the life of the contract, total value is Over the life of the contract, total value is potentially worth hundreds of millions of potentially worth hundreds of millions of dollarsdollars


ERA Research MissionERA Research Mission

Attack the critical preservation problemAttack the critical preservation problem

Contribute to mitigation of risks associated with ERA Contribute to mitigation of risks associated with ERA Program, especially with respect to emerging technologies, Program, especially with respect to emerging technologies, technology enabled processes and potential coststechnology enabled processes and potential costs

Contribute to ERA Program Acquisitions, Capital and Contribute to ERA Program Acquisitions, Capital and Strategic Planning Decisions Strategic Planning Decisions

Serve as catalyst in continuing innovation and technology Serve as catalyst in continuing innovation and technology transfer in areas responsive to federal electronic records transfer in areas responsive to federal electronic records preservation, management and accesspreservation, management and access

Serve as foundation for continuing collaborationServe as foundation for continuing collaboration


Build on Existing Foundations Build on Existing Foundations

Eliminate any reliance on presumed clairvoyanceEliminate any reliance on presumed clairvoyance

Establish technical testbeds & analytic demonstrations Establish technical testbeds & analytic demonstrations where NARA contributes electronic records or records where NARA contributes electronic records or records surrogate test corpora, domain knowledge and funding surrogate test corpora, domain knowledge and funding to ongoing technology research responsive to core to ongoing technology research responsive to core interests interests

Collaboration ModelCollaboration Model


NationalScienceFoundationSan Diego

Supercomputer Center

NISTNIST

Our PartnersOur Partners

Global Grid Forum

National Computational Science Alliance

National Partnership for Advanced Computational Infrastructure

Army ResearchArmy ResearchLaboratoryLaboratory


Collaboration with Collaboration with the Library of Congressthe Library of Congress

In late 2000, NARA joined with the Library of Congress to develop a joint In late 2000, NARA joined with the Library of Congress to develop a joint presentation on critical issues that both institutions face in the area of presentation on critical issues that both institutions face in the area of digital preservation digital preservation

This presentation to the Congress’s Joint Committee on the Library This presentation to the Congress’s Joint Committee on the Library resulted in creation of the National Digital Information Infrastructure and resulted in creation of the National Digital Information Infrastructure and Preservation (NDIIP) ProgramPreservation (NDIIP) Program

NARA continues this collaboration with the Library on the NDIIP NARA continues this collaboration with the Library on the NDIIP

NARA currently participates in the NDIIP Steering Committee and NARA currently participates in the NDIIP Steering Committee and contributes to the exploration of issues and opportunities that the Library contributes to the exploration of issues and opportunities that the Library

of Congress and other libraries face in this area.of Congress and other libraries face in this area.


Data intensive application of interconnected computational Data intensive application of interconnected computational engines, appropriately scalable data repositories & engines, appropriately scalable data repositories & advanced network technology enabled applicationsadvanced network technology enabled applications

Understanding all applied processes in terms of I/O, storage Understanding all applied processes in terms of I/O, storage & file systems, very large mass storage systems enabling & file systems, very large mass storage systems enabling scalable & extensible “write biased” federated repositoriesscalable & extensible “write biased” federated repositories

Information assurance/Cybersecurity mechanismsInformation assurance/Cybersecurity mechanisms

Scalable collaborative decision support environments Scalable collaborative decision support environments

Data structure optimization (especially for “Input”)Data structure optimization (especially for “Input”)

Core Technology Research Interests Core Technology Research Interests


ERA ResearchERA ResearchCurrent ActivitiesCurrent Activities

““Transcontinental”Transcontinental”Persistent Archives PrototypePersistent Archives Prototype

>> >> Over 3.0 million filesOver 3.0 million files across nationally distributed sites (SDSC, across nationally distributed sites (SDSC, UMd, NARA, Stanford/SLAC) UMd, NARA, Stanford/SLAC)

Users can Users can access filesaccess files without knowing where or how without knowing where or how they are stored they are stored

Grid Technology EnabledGrid Technology Enabled

Applications of Applications of advanced data management and I/O techniquesadvanced data management and I/O techniques – NCSA, U of IL – Urbana-Champaign– NCSA, U of IL – Urbana-Champaign

Knowledge-based technologiesKnowledge-based technologies enabling preservation of enabling preservation of electronic records collectionselectronic records collections


ERA ResearchERA ResearchTest CollectionsTest Collections

Executive Office of the President EmailExecutive Office of the President Email OMB E-gov Digital Photographs, Document Images & OMB E-gov Digital Photographs, Document Images &

Geospatial Records (OMB & multiple agencies)Geospatial Records (OMB & multiple agencies) CAD/CAE Solid Model Design Records CAD/CAE Solid Model Design Records Columbia Accident Investigation Board RecordsColumbia Accident Investigation Board Records Motion Video & Virtual Reality Records Motion Video & Virtual Reality Records Presidential Libraries Records Presidential Libraries Records Federal Agencies Web Records Federal Agencies Web Records Patent Records Patent Records Technology Research Records (DoE)Technology Research Records (DoE)


ERA Geospatial ResearchERA Geospatial Research

National Center for Supercomputing National Center for Supercomputing ApplicationsApplications

Storage and Retrieval Efficiency Evaluations of Boundry Data Representations Storage and Retrieval Efficiency Evaluations of Boundry Data Representations for LLS, TIGER and DLG Data Structures, Technical Report, 8 December for LLS, TIGER and DLG Data Structures, Technical Report, 8 December 2004, The national Center for Supercomputing Applications 2004, The national Center for Supercomputing Applications David Clutter and Peter BajcsyDavid Clutter and Peter Bajcsy

Scientific formats for geospatial data preservation -- A study of suitability and Scientific formats for geospatial data preservation -- A study of suitability and performance performance Mike Folk and Vailin ChoiMike Folk and Vailin Choi

Attributes of File Formats for Long-Term Preservation of Scientific and Attributes of File Formats for Long-Term Preservation of Scientific and Engineering Data in Digital Libraries Engineering Data in Digital Libraries Mike Folk and Bruce BarkstromMike Folk and Bruce Barkstrom


ERA Geospatial ResearchERA Geospatial ResearchSan Diego Supercomputer CenterSan Diego Supercomputer Center

Mapping Herbicide Missions in Vietnam

Ilya Zavlaskyhttp://ngda.org/wkshop%20presentations/4_Zaslavsky.ppt

Studying Redlining in San Diego

Richard Marcianohttp://sdtj.regionalworkbench.org/UCSD/RedliningCentral.htm

The Electronic Records Archives (ERA) Program National Archives and Records Administration 1 ERA...

Documents

Transcript of The Electronic Records Archives (ERA) Program National Archives and Records Administration 1 ERA...