Foott & Wojtkowski - Monash University...(E xtensible Markup Language) using an XML editor. An XSLT...

Preserving History: Digitising WWII Terrain Studies of the Southwest Pacific Area (SWPA)

Journal of the Australian and New Zealand Map Society Inc.

43

PRESERVING HISTORY: DIGITISING WWII TERRAIN STUDIESOF THE SOUTHWEST PACIFIC AREA (SWPA)Bronwyn Foott & Barbara Wojtkowski1

Abstract: In 1942, after a series of directives from General MacArthur’s Headquarters, the Allied GeographicalSection (AGS) was formed with the task of compiling all the available geographical intelligence on the largelyunmapped and underexplored south-west Pacific area. During its lifespan (1942-45), the AGS produced 3 major typesof publications: Terrain Studies; Special Reports; and Terrain Handbooks. The Terrain Studies were the mostimportant and complex works, designed to give the most comprehensive information. Thus, they contained text,diagrams, photographs and various sized and coloured maps. In 2012, Monash University Library initiated a project todigitise its substantial collection of Terrain Studies and make them available via the Monash University ResearchRepository (http://arrow.monash.edu) to complement its online collections for the ANZAC Centenary.

This paper explores a number of aspects of the project: its genesis; challenges, given the format of the material; thescanning process; metadata creation and changing geography (role of gazetteers); how it complements existing WWIexhibits; and methods of access.

December, 1941 saw the Japanese attack Pearl Harbour. By August 1942 much of South- EastAsia was under Japanese control and the threat to Australia very real. In response to this, theSouthwest Pacific Area (SWPA) was established in Australia under General MacArthur’s

command on 18 April 1942 (Willoughby et al., 1983). This area included the tropical and island areasnorth of Australia and extended toward mainland Asia (Fig. 1.).

Unlike the European theatre of war which waswell known and had been extensively mappedfor hundreds of years, the SWPA was largelyunknown and unmapped – an intolerablesituation given the pressures of war.

To address this critical lack of geographic,anthropologic and hydrographic information, theAllied Geographical Section (AGS) wasestablished under the command of Lt.-Col. (laterCol.) Evan Mander-Jones in August 1942, andsubsequently, Maj. (later Col.) William Jardine-Blake. The Section’s main objective was tocollect, collate, verify and distribute all relevantavailable intelligence to assist military planningand activity in this area. Thus, the publicationsincluded maps, texts, photographs, climatic andanthropologic data; in fact, any informationwhich could be of assistance to militarypersonnel, such as landing spots, vegetation,native foods and the dangers which could beencountered in these unknown regions.

1 Bronwyn Foott is the Repository Content Officer at the Monash University Research Repository,Monash University Library. Barbara Wojtkowski is the Map Librarian at Monash University’s Hargrave-Andrew Library. Contact: [email protected] ; [email protected]

7

Figure 1. Pacific Theatre commands in 1942 and AGSarea of interest for which reports were produced.

The Globe, Number 78, 2015


44

Before it was dissolved in November 1945, the AGS produced 3 major types of publications: 110Terrain Studies, 62 Terrain Handbooks and 101 Special Reports. The most important of these were theTerrain Studies, the purpose of which was to cover as completely as possible the geographic conditionsof an area from a military perspective.

Monash University Library holds in its Maps Collection the majority of the AGS publications (seeAppendix). Due to their uncommon nature, fragility and worldwide appeal, the decision was made in2012 to digitise and distribute via the Monash University Research Repository as much of thiscollection as possible. At the time of this decision, it was hoped to have the project coincide with theANZAC centenary commemorations in 2015.

In October, 2012, the Terrain Studies were sent to CAVAL1 for scanning using their Kirtas 2400 bookscanner (Fig. 2.). This particular scanner, although powerful enough to function autonomously, was runin manual mode with one individual taking overall responsibility for the quality control of the scanningprocess. Consistency was thus achieved across the process along with a sense of “ownership” for theresultant product, which included images in LZW TIF, JPEG and PDF formats.

Figure 2. Kirtas 2400 book scanner at CAVAL

Given the fragile, often damaged state of the physical items, staff took care to delicately dismantle theitems, removing the metal bindings and carefully unfolding often large maps (Fig. 3.) and scanning inminimum light to maintain the true colour of each page. Checking for quality, missing pages, order,etc., was conducted at regular stages of the scanning and PDF creation process. Each textual documentwas processed using ABBYY Finereader OCR (Optical Character Recognition) software. This wouldenable patrons at the end of the process to search the full text of each document once it had been loadedinto the Repository – a truly powerful and useful feature.

Despite the care taken throughout the project, staff at CAVAL reported their fair share of challenges,which were compensated for by the fascination the items themselves exerted on the staff. Among the



45

challenges, the most obvious was the nature of the items – their fragile, damaged nature and the extremecare which needed to be taken in handling and actual scanning. On the positive side, the intellectualcontent, maps, images and sheer detail contained in the Studies was at times “mind blowing” as reportedby one staff member. Staff were also impressed by the relevance of the Studies to a range of modernresearch – historical studies and also to disciplines as disparate as climate change, social andanthropological studies, engineering and genealogy.

Figure 3. Large format map from Terrain Study no. 23.

The early months of 2013, saw CAVAL and Monash corresponding regularly and testing possibilities asto the optimum method and final display of the Studies. Foremost in our thoughts was the desire to finda way to present the most “true” and “least distorted” representation of each physical Study and itscomponent parts in a digital environment. Of equal importance, was the need to make this user-friendly, in terms of access, speed, simplicity and searchability.

After receiving the final files for the first quarter of the collection in November of the same year, ourstaff began the arduous task of final quality control and creation of metadata to describe each item andthus enhance its efficient retrieval from its final home in the online digital Repository.

The metadata for each item, be it a report, photograph, map, terrain study, etc., was based on twometadata schemas: MODS (Metadata Object Description Schema) for the rich detailed metadata andDC (Dublin Core) for the access and dissemination metadata. Why two schemas? The MODSmetadata is more complex and allows for thorough description and detail at a fine level; this is ideal formanipulation of the way the metadata and items themselves are displayed and ordered within theRepository public display (as illustrated in the comparison shown in Fig. 4.). Dublin Core metadata,based on 15 core elements2 although simpler, allows for wider application, and it is this metadata that isharvested by search engines and other aggregators such as Google and Trove. The two together providethe end-searcher with a very powerful “behind-the-scenes” tool to rely upon.



46

Figure 4. MODS xml file, DC xml file and final DC display in the Research Repository.

When creating metadata for any object, the reality institutions face is that it is always a careful balancebetween time versus quality. While striving to achieve the most complete metadata possible, this mustalways be weighed against the time it takes (i.e. costs such as staff wages) to create a record, especiallygiven the enormous size of this particular project.

Metadata creation can be a complex process and during the Terrain Study metadata project, there weremany difficulties faced by our staff. Even something one might think to be quite straight forward suchas assigning a title to a map or other items was complicated by the multiple titles present and the varyinglanguages that were used in many of the studies, such as Japanese, German, English or Dutch.

The changing geography and politics of the areas also played a key role in deciding which geographicalkeywords to assign to each item. Should an historical name be assigned, the current name, or both? Inaddressing this issue a glossary of geographical terms was created to maintain consistency in both thenames assigned and their spelling.

At Monash University, the practice is to use a csv to xml based metadata creation process (Fig. 5.).This involves repository staff creating a csv spreadsheet version of the metadata which is then convertedto an xml file.

Figure 5. The CSV (comma-separated values) metadata created by repository staff is converted into XML(Extensible Markup Language) using an XML editor. An XSLT (Extensible Stylesheet Language Transformation)is then executed over this XML to output the MODS metadata. A second XSLT is then executed on this metadata

to produce the Dublin Core metadata.

In very simplistic terms, one may say that this is a “bulk automatic conversion”. The reasons for takingthis approach are that it results in a greater rate of consistency and accuracy plus a faster work rate,resulting in fewer human errors.

Once the MODS metadata has been created, a series of backend instruction files are written. These filesautomatically govern the loading process – how each file is displayed, what metadata is associated with



47

it, which collection it belongs to and which files to attach to each record. Again, since the process isautomatic it minimises human error and maximises speed. However, quality control must be exercised.In this case, the process is simplified by the creation of log files for each load, listing the success orfailure of each of these instructions. These have proven very effective, even registering typographicalerrors in the initial entry.

To the delight, and no small relief, of the whole team working on this project, the first lot of some 450items were ready for public display in the Repository in time for ANZAC Day (at time of writing, theproject has been completed). 2015 marks the ANZAC Centenary, which Monash has beencommemorating in a number of ways. Of relevance to this digitisation project, is our on-line exhibition“A new nation goes to war” (http:/monash.edu.au/library/collections/exhibitions/world-war-one/), aselection of items from our Rare Books Collection which include books, pamphlets, ephemera,photographs and manuscripts representing contemporary themes of soldiers’ experiences, varyingpolitical views, conscription and home life. Although the Terrain Studies are from a different historicalperiod, they are of a more “immediate past”, and so potentially closer to our consciousness, especially tothat of survivors and their relatives. The Studies add to a fuller picture of our military and culturalhistory; not to mention their geographic and research applications. Together, the two collections offerresearchers and the general public a pool of lesser-known and otherwise difficult to access material.

One must not omit the preservation aspect of the project - both of the physical items themselves andtheir intellectual content. Since the collection had not been catalogued, it was difficult to “discover” orfind. Our extensive metadata and OCR functionality built into each record and/or text item hasenhanced the “findability” of not only each Study itself, but also its component parts.

Figure 6. Examples of types of damage



48

With the new online presence, the project has ensured that handling has been dramatically reduced, ifnot eliminated. The physical copies of the Terrain Studies were in a generally poor state; a number hadbeen damaged by water, many had pages stuck together and rust stains throughout as a result of thebindings (Fig. 6.). Having the digitised copies available has reduced physical handling of the items andexposed has them to a broader worldwide audience.

In the few months since the initial batch went live it has been gratifying to receive personalcommunication and requests from various researchers around the globe. This is not includingunmediated visits to the website, which was one of the prime purposes of the project. After almost threeyears, the Monash collection of Terrain Studies is available to a global audience.

NOTES1 Formerly Cooperative Action by Victorian Academic Libraries, a member-based for-benefit company offering

reciprocal borrowing, shared collection storage and other economy-of-scale services to member libraries.2 Dublin Core elements: Title, Creator, Subject, description, publisher; contributor; date; type; format; identifier;

source; language; relation; coverage; rights

BIBLIOGRAPHY

BOWD, R.R.E., (2004). “The Allied Geographical Section, 1942-46: forgotten by history”. AustralianDefence Force Journal, 165:35-48.

———, (2005). A basis for victory : the Allied Geographical Section 1942-1946. Canberra Papaers onStrategy & Defence 157, Strategic and Defence Studies Centre, ANU, Canberra.

DUBLIN CORE METADATA INITIATIVE (DCMI), Metadata Innovation, http://dublincore.org/, viewed 15March, 2015.

IND, A. (1958). Spy ring Pacific : the story of the Allied Intelligence Bureau in South East Asia.Weidenfeld and Nicolson, London.

LIBRARY OF CONGRESS, MODS: METADATA OBJECT DESCRIPTION SCHEMA,http://www.loc.gov/standards/mods/, viewed 15 March 2015,

RICHMOND, K., (2005). “Geographical and technical intelligence in the Southwest Pacific 1942-1945”Sabretache, 46(4):21.

ROSE, E.P.F. & CLATWORTHY, J.C., (2008). “Terrain evaluation for allied military operations in Europeand the Far East during World War II: ‘secret’ British reports and specialist maps generated by theGeological Section, Inter-Service Topographical Department”, Quarterly Journal of EngineeringGeology and Hydrogeology, 41:237-256.

WILLOUGHBY, CHARLES A. (ed.), (1983). History of intelligence activities under General DouglasMacarthur, 1942-1950 the intelligence Series, G-2 USAFFE-SWPA-AFPAC-FEC-SCAP, ScholarlyResources, Wilmington (Del.).



49

APPENDIX

Terrain Studies list, highlighting Monash University Library holdings

No. Description1 * Rabaul, Gasmata, New Ireland. Superseded by

52,60, 742 * Babo. Superseded by 373 * Manus. Superseded by 674 * Buna. Superseded by 12, 275 * Lae, Salamaua, Madang, Wewak. Superseded

by 32, 33, 59, 69, 766 * Tulagi. Superseded by 117 * Rabaul, Gasmata. Superseded by 60, 748 * New Ireland. Superseded by 529 * Lae & Salamaua. Superseded by 32 & 3310 * South New Guinea11 * Guadalcanal12 * Buna. Superseded by 2713 * Kai Islands. Superseded by 5614 * Fak Fak. Superseded by 3715 * Mandated Solomons. Superseded by 4116 * Salamaua. Superseded by 3317 * Lae. Superseded by 3218 * Choiseul. Superseded by4819 * New Georgia. Superseded by 5420 * Ysabel. Superseded by 3921 * Wissel Lakes. Superseded by 6822 * Gazelle Peninsula & Rabaul. Superseded by 7423 D’Entrecasteaux & Trobriand Islands24 * Talasea. Superseded by 6425 * Cape Gloucester. Superseded by 6326 * Gasmata. Superseded by 6027 * Buna (Revised)28 * Main Routes Across New Guinea29 * South West Papua & The Fly River.

Superseded by SR 2830 * Tual Subdivision Tanimbar. Superseded by 8731 * Locality Study of Madang. Superseded by 59

& 6932 Locality Study of Lae (Revised) & The

Lower Markham Valley33 Locality Study of Salamaua (revised)34 Louisiade Archipelago35 Woodlark Island & other islands36 Finschhafen & the Huon Peninsula37 Bomerai Peninsula (Netherlands New

Guinea)38 * The Aroe Islands. Superseded by SR 4239 Ysabel (revised)40 * New Georgia (revised). Superseded by 5441 The Mandated Solomons & the Shortland

Islands42 Marobe - Waria Valley & Adjacent

Catchment Areas

43 * Manus & Western Islands. Superseded by 6744 * Merauke (Locality Study)(preliminary).

Superseded by 5545 Ambon Island46 Vogelkop & Salawati Island - Netherlands

New Guinea47 Russel Islands48 Choiseul (revised)49 The Markham Catchment50 Portuguese Timor51 Eastern New Britain Excluding Gazelle

Peninsula. Superseded by SR 6352 New Ireland53 * Tanimbar Islands - Netherlands East

Indies. Superseded by SR 3754 New Georgia Group (2nd revision)55 South-west New Guinea56 * Kai Islands - Netherlands East Indies.

Superseded by SR 4957 Western New Britain, excluded Cape

Gloucester58 South-western Islands (Islands N.E. of

Timor)59 Madang. v.1 (text & maps); v.2

(photographs)60 Gasmata (revised)61 Northeastern Netherlands New Guinea &

Geelvink Bay ; 61B Photographic Annex -Sarmi Village to Masi Masi Village ; 61CPhotographic Annex - Impedi Village toMenoerwar Village

62 New Hanover & Islands North & East ofNew Ireland

63 Cape Gloucester (revised)64 Talasea (revised)65 The Sepik District. v.1 (text & maps); v.2

(photographs)66 Ramu - Markham Valley with link to

Madang67 The Admiralty Islands68 Central Dutch New Guinea69 Madang70 Dutch Timor, Netherlands East Indies71 Northern Molukkas72 Hansa Bay. v.1 (text & map); v.2

(photographs)73 Seran (Ceram) & Adjacent Islands;74 Gazelle Peninsula. v.1 (text & maps); v.2

(photographs),75 Kavieng76 Wewak



50

77 Aitape - Wanimo78 Hollandia (Dutch New Guinea); 78B

Hollandia - Photographic Annex79 Radja Ampat Group80 ‡ Mindanao Island. v.1 (text & maps);

v.2 (photographs)82 Talaud & Sangihe Islands. Superseded by

TH 3283 Menado (Celebes)84 ‡ Leyte Province85 ‡ S.E. Luzon [Bicols]86 ‡ Cagayan Valley ; Cagayan Valley (map

supplement)87 Tanimbar Islands - Netherlands East Indies88 ‡ Samar Province89 Sarawak & Brunei. v.1 (text & maps); v.2

(photographs)90 North Borneo91 ‡ Tayabas Province92 ‡ Davao (Gulf) - Sarangani (Bay) Mindanao

Island93 ‡ Lingayen

94 ‡ Central Luzon. v.1 (text & maps); v.2(photographs); v.3 (Manila Directory)

95 ‡ Batangas - Lucena (Luzon Island)96 ‡ Mindoro Province97 ‡ N.W. Luzon98 ‡ Central Mindanao ; Central Mindanao

(map supplement)99 ‡ Negros Island & Siquijor Island100 Cebu & Bohol Islands101 ‡ Panay Island

Sulu Archipelago103 ‡ Palawan Province. v.1 (text & maps); v.2

(appendices & photographs)107 Kendari (S.E. Celebes)109 S.E. Borneo112 Batvia - Bandoeng (Java Series)130 § S.E. Kyushu ; S.E. Kyushu Map

Supplement132 § Tokyo & Kwanto Plain134 § Nagoya Area136 § Osaka

Total 110. Monash University does not hold 27 titles.* = out of print and superseded as of 15th September, 1945.‡ = Philippine series§ = Japan seriesBold = held by Monash University Library and digitised

For a complete listing of Monash holdings of the Allied Geographical Section’s publications, please alsorefer to the following lists (Note: These are currently NOT digitised):Special Reports:

http://resources.lib.monash.edu.au/inventories/maps/specials/allied-pacific-special-rep.htmlTerrain Handbooks:

http://resources.lib.monash.edu.au/inventories/maps/specials/allied-pacific-terrain-hndbks.htmlAssorted other publications:

http://resources.lib.monash.edu.au/inventories/maps/specials/allied-pacific-miscellaneous.html

Foott & Wojtkowski - Monash University...(E xtensible Markup Language) using an XML editor. An XSLT...

Documents

Transcript of Foott & Wojtkowski - Monash University...(E xtensible Markup Language) using an XML editor. An XSLT...