Cabrinety-NIST ProjectLarge-scale digital preservation of a legacy software collection

AMIA Digital Asset Symposium November 19, 2015Charlotte C. ThaiCabrinety Project Archivistcthai2@stanford.edu |@cabrinety

1

Outline

Part 1: Cabrinety-NIST Project BackgroundStephen M. Cabrinety Collection in the History of Microcomputing, ca. 1975-1995Who is Stephen M. Cabrinety?NSRL Grant Details (2012-2014; 2014-2016)

Part 2: Cabrinety-NIST Project workflowGeneral overviewDeep dive into key sections

Part 3: Next steps

2

15,000+ software packages27 operating systems400 pieces computer hardware200 pieces computer-related realiaDocuments, ephemera, periodicals

Stephen M. Cabrinety collection in the History of Microcomputing, ca. 1975-1995

3

Stephen M. Cabrinety

4

NSRL GrantCabrinety-NIST ProjectFunding agencyNational Software Reference Library

Project PartnersStanford University LibrariesNIST (National Institute of Standards and Technology)

5

NSRL GrantStanford Digital RepositoryHydra Technology StackFedora Commons Ruby on RailsApache SolrBlacklight

http://projecthydra.org

6

7

STARTRequest boxProcessingRegistrationCatalogingPhotographyAuditingNIST receives boxCreate disk imageShip from SUL to NISTVerify imageCreate Box Archive ingest packageShip box from NIST to SULReceive box at StanfordDownload Box ArchivePerform physical QCAccession files and metadata into Stanford Digital RepositoryEndCreate PURLReturn box

Perform digital QC

Cabrinety-NIST Project WorkflowStage 1Stage 3Stage 2

8

SAL3 (Stanford auxiliary library 3)3,925,000 barcoded items170 average requests per day22 machine carts1 truck

9

SearchWorksSirsiDynix WorkFlows

10

SAL3 to Stanford Campus

11

Going the distanceLeg 1: SAL3 -> Stanford (40 miles)Leg 2: Stanford -> Redwood City (9 miles)Leg 3: Redwood City -> NIST (2,817 miles)Leg 4: NIST -> Stanford (2, 827 miles)Leg 5: Stanford -> Redwood City (9 miles)Leg 6: Redwood City -> SAL3 (40 miles)

Approx. distance each box travels: 5,742 miles

12

Wait for it

13

This Fruit Roll-Up is 22 years old.

14

Argo registration

15

Argo tracking sheet

16

DatastreamsDCRELS-EXTdescMetadatacontentMetadatatechnicalMetadata

17

18

19

20

21

1 Game = 55 Photographs

22

23

24

25

DisclaimerTrade names and company products are mentioned in the text or identified. In no case does such identification imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the products are necessarily the best available for the purpose.

2626

Perl media imager (custom, wraps dcfldd, etc.)

dcfldd (or, digital computer forensics lab dd)

27

Media Image Listingzy394kt3697-1/0b1a57009bb039c4f4dc87030d5179c0e4d3acd5:

-rw-r--r-- 1 admin wheel 0 Jul 22 15:14 0b1a57009bb039c4f4dc87030d5179c0e4d3acd5.dderr-rw-r--r-- 1 admin wheel 737280 Jul 22 15:15 0b1a57009bb039c4f4dc87030d5179c0e4d3acd5.img-rw-r--r-- 1 admin wheel 173 Jul 22 15:15 0b1a57009bb039c4f4dc87030d5179c0e4d3acd5.img.sha-rw-r--r-- 1 admin wheel 0 Jul 22 15:13 0b1a57009bb039c4f4dc87030d5179c0e4d3acd5.mountstr

zy394kt3697-2/aef4055512d90778c10a2b70f7232cf237e67023:

-rw-r--r-- 1 admin wheel 0 Jul 22 15:16 aef4055512d90778c10a2b70f7232cf237e67023.dderr-rw-r--r-- 1 admin wheel 737280 Jul 22 15:17 aef4055512d90778c10a2b70f7232cf237e67023.img-rw-r--r-- 1 admin wheel 173 Jul 22 15:17 aef4055512d90778c10a2b70f7232cf237e67023.img.sha-rw-r--r-- 1 admin wheel 0 Jul 22 15:15 aef4055512d90778c10a2b70f7232cf237e67023.mountstr

zy394kt3697-3/caad5a2e928551421fb567eeba0509f7f7f9be8a:

-rw-r--r-- 1 admin wheel 0 Jul 22 15:18 caad5a2e928551421fb567eeba0509f7f7f9be8a.dderr-rw-r--r-- 1 admin wheel 737280 Jul 22 15:19 caad5a2e928551421fb567eeba0509f7f7f9be8a.img-rw-r--r-- 1 admin wheel 173 Jul 22 15:19 caad5a2e928551421fb567eeba0509f7f7f9be8a.img.sha-rw-r--r-- 1 admin wheel 0 Jul 22 15:17 caad5a2e928551421fb567eeba0509f7f7f9be8a.mountstr

zy394kt3697-4/a7e8b599626bb1cf5de832ba5ef011aedbe40f31:

-rw-r--r-- 1 admin wheel 0 Jul 22 15:20 a7e8b599626bb1cf5de832ba5ef011aedbe40f31.dderr-rw-r--r-- 1 admin wheel 737280 Jul 22 15:20 a7e8b599626bb1cf5de832ba5ef011aedbe40f31.img-rw-r--r-- 1 admin wheel 173 Jul 22 15:21 a7e8b599626bb1cf5de832ba5ef011aedbe40f31.img.sha-rw-r--r-- 1 admin wheel 0 Jul 22 15:19 a7e8b599626bb1cf5de832ba5ef011aedbe40f31.mountstr

2828

5.25 inch floppy disk captureDevice Side Data FC5025 USB 5.25 inch floppy controller5.25 inch floppy drive

5.25 inch floppy disks

29

How is the data captured from cartridge media?

Retrode

30

SNES and Sega Genesis Checksums

http://www.stanford.edu/group/htgg/cgi-bin/drupal/?q=node/1179

31

CopyNES

32

Mappers

33

CRC32 (cyclic redundancy check)

34

GameBoy + Arduino

35

Computer CassetteEcasoundAudacity

36

Computer CassetteWAV2TAPVICE (VIC-20) Emulator

37

Box Archives

38

Accessioning: JIRA

39

40

Next steps?

41

EaaS: Emulation as a Service

42

Emulation: jsMESS

43

Questions?Contacts:

Stanford University LibrariesCharlotte C. Thai (cthai2@stanford.edu, 650.497-7020)

National Institute of Standards and TechnologyDouglas White (dwhite@nist.gov, 301.642.8596)

44

LinksArduino: https://www.arduino.cc/

Audacity screen capture: Doug White presentation at Stanford Capture Lab Unconference 2015

CopyNES: www.retrousb.com

DCFLDD: http://dcfldd.sourceforge.net/

Emulation as a Service: http://bw-fla.uni-freiburg.de/

gphoto2: http://www.gphoto.org/

Hydra: http://projecthydra.org/

Image Science Associates: http://www.imagescienceassociates.com

Internet Archive Oregon Trail: https://archive.org/details/msdos_Oregon_Trail_The_1990

JMESS Github: https://github.com/jsmess/jsmess

Maze Commodore VIC-20 screen capture: Doug White presentation at Stanford Capture Lab Unconference 2015

NSRL: http://www.nsrl.nist.gov/

Pokemon Red Game Boy cartridge, Game Boy Cart Shield, Arduino: Christopher Fox blog How to Give Cartridge-Based Video Game Data an Extra Life (Part 2): http://web.stanford.edu/group/htgg/cgi-bin/drupal/?q=node/1205

Retrode: http://www.retrode.org/

Stanford Digital Repository: https://library.stanford.edu/research/stanford-digital-repository

Stanford Digital Repository Sprite PURL: https://purl.stanford.edu/kg712zz5745

45