Format Specification for DREA .DAT32 Files - Version 1.0 · 2012. 8. 3. · Format Specification...

Defence R&D Canada – Atlantic

DEFENCE DÉFENSE&

Format Specification for DREA .DAT32

Files - Version 1.0.a

C. Calnanxwave

xwave36 Solutions DriveHalifax, NS B3S 1N2

Project Manager: J. Theriault, 902-426-3100 x376

Contract Number: W7707-03-2425

Contract Scientific Authority: J. Theriault, 902-426-3100 x376

xwave Contract Number: 1004920 DAT32

Contract Report

DRDC Atlantic CR 2004-072

April 2004

Copy No.________

Defence Research andDevelopment Canada

Recherche et développementpour la défense Canada

This page intentionally left blank.

Format Specification for DREA .DAT32 Files - Version 1.0.a

C. Calnan xwave

xwave 36 Solutions Drive Halifax, N.S. B3S 1N2

Project Manager: J. Theriault 902-426-3100 Ext 376

Contract Number: W7707-03-2425

Contract Scientific Authority: J. Theriault 902-426-3100 Ext 376

Defence R&D Canada – Atlantic

Contract Report


April 2004

Abstract An earlier report [Caldwell, 1987] described the specifications for a binary file format in use at DREA at the time. That report described a format that consisted of a file containing a header, with information on the file and its contents, that was followed by the file’s data. Eventually this format, familiarly know as the “DREA .DAT” format, proved to be insufficient to contain data recorded by the evolving nature of the instrumentation and the requirements of researchers. That format became enhanced, becoming the higher resolution “DREA .DAT32” format. Over time programs were written to produce data in DREA .DAT32 format, as well as to read and use the contents of these files. However no analogue to Caldwell’s report was ever produced, which meant that user’s were essentially free to do what they wanted with the format, within limits. This report sets down the requirements and specifications of the DREA .DAT32 format based on data gathered from programs that write and read the files, as well as from users of DREA .DAT32 files and the programs that use them.

Résumé Un rapport antérieur [Caldwell, 1987] définissait les spécifications du format de fichier binaire en usage à ce moment-là au CRDA. Ce rapport décrivait un format constitué d’un fichier contenant un en-tête, où figuraient des renseignements sur le fichier et son contenu, et qui était suivi des données du fichier. Ultérieurement, ce format, familièrement appelé format « DREA .DAT », est devenu insuffisant pour contenir les données enregistrées en fonction de la nature dynamique de l’instrumentation et des exigences des chercheurs. On a donc amélioré ce format qui est alors devenu le format « DREA.DAT32 » à haute définition. Avec le temps, on a commencé à écrire des logiciels capables de produire des données en format DREA.DAT32, ainsi que de lire et d’utiliser leur contenu. Mais aucun rapport semblable à celui de Caldwell n’a jamais été rédigé, ce qui signifiait que l’utilisateur était pratiquement libre de faire ce qu’il voulait du format, sans restrictions.

Le présent rapport établit les exigences et les spécifications du format DREA .DAT32 d’après les données recueillies à partir de programmes qui écrivent et lisent des fichiers, auprès des utilisateurs de fichiers DREA .DAT32 et de programmes qui utilisent ces fichiers.

DRDC Atlantic CR2004-072 i


ii DRDC Atlantic CR2004-072

Executive summary

Background DRDC Atlantic has had a long history of using towed array data. The Towed Integrated Active-Passive Sonar (TIAPS) technology project put new demands on the data formats previously in use. The combination of higher dynamic range and more channels made it difficult to use the previous “.DAT” format. The “.dat” format was extended to address it’s short-comings. An earlier report [Caldwell, 1987] described the specifications for the “.DAT” binary file format. That report described a format that consisted of a file containing a header, with information on the file and its contents, that was followed by the file’s data. Over time programs were written to produce data in the new .DAT32 format, as well as to read and use the contents of these files. However no analogue to Caldwell’s report was ever produced, which meant that user’s were essentially free to do what they wanted with the format, within limits. This report sets down the requirements and specifications of the DRDC Atlantic .DAT32 format based on data gathered from programs that write and read the files, as well as from users of DRDC Atlantic .DAT32 files and the programs that use them.

Results and Significance

This document provides a baseline specification for the .DAT32 format. Through documenting the standard, it is expected that further software tool development may be accomplished in a much more cost-effective manner.

Future Plans This file format specification should not be taken as the final state. File formats should evolve, to meet an ever growing need. Enhancements to the format must be documented in subsequent releases of the specification.

Calnan, C. 2004. Format Specification for .DAT32 Files DRDC Atlantic CR 2004-072. Defence R&D Canada – Atlantic.

DRDC Atlantic CR2004-072 iii

Sommaire

Contexte RDDC Atlantique utilise depuis longtemps les données de réseau remorqué. Le projet de technologie du sonar remorqué intégré actif et passif (TIAPS) a donné lieu à de nouvelles demandes quant aux formats de données déjà utilisés. La combinaison de la portée dynamique plus étendue combinée et du nombre accru de canaux a rendu difficile l’utilisation des anciens formats « .DAT ». Le format « .dat » a été élargi afin de corriger ses lacunes. Un rapport antérieur [Caldwell, 1987] décrivait les spécifications du format de fichier binaire « .DAT ». Ce rapport décrivait un format constitué d’un fichier contenant un en-tête, où figuraient des renseignements sur le fichier et son contenu, et qui était suivi des données du fichier. Avec le temps, on a commencé à écrire des logiciels capables de produire des données en format DREA.DAT32, ainsi que de lire et d’utiliser leur contenu. Mais aucun rapport semblable à celui de Caldwell n’a jamais été rédigé, ce qui signifiait que l’utilisateur était pratiquement libre de faire ce qu’il voulait du format, sans restrictions. Le présent rapport établit les exigences et les spécifications du format DREA .DAT32 d’après les données recueillies à partir de programmes qui écrivent et lisent des fichiers, auprès des utilisateurs de fichiers DREA .DAT32 et de programmes qui utilisent ces fichiers.

Résultats et importance

Le présent document contient les spécifications de base pour le format .DAT32. Grâce à l’établissement de la documentation sur la norme, on s’attend à ce que le développement d’outil logiciel puisse se faire ultérieurement de manière plus rentable.

Plans La spécification de ce format de fichier ne doit pas être considérée comme le point final. Les formats de fichier doivent évoluer afin de répondre aux besoins sans cesse croissants. Les améliorations apportées au format doivent être documentées dans ces versions ultérieures de la spécification.

Calnan, C. 2004. Spécifications des fichiers DAT32 RDDC Atlantique DCD 2004-072. R & D pour la défense Canada – Atlantique.

iv DRDC Atlantic CR2004-072

Table of Contents

Abstract........................................................................................................................................ i

Résumé ........................................................................................................................................ i

Executive Summary................................................................................................................... iii

Sommaire................................................................................................................................... iv

Table of Contents ....................................................................................................................... v

List of Tables............................................................................................................................. xi

Acknowledgements .................................................................................................................. xii

1. Introduction ................................................................................................................... 1

2. Report Versions ............................................................................................................. 2

3. DREA .DAT32 File Description ................................................................................... 3

3.1 Data Endianess ................................................................................................. 3

3.2 File Header ....................................................................................................... 4

3.3 File Body .......................................................................................................... 5

3.4 File Types and Differentiation.......................................................................... 5

3.5 Potential Problem ............................................................................................. 6

4. Tape Storage.................................................................................................................. 8

5. Time Series.................................................................................................................... 9

5.1 Integer Block .................................................................................................. 11

5.1.1 Block Size [Required] ....................................................................... 11

5.1.2 Record Size [Optional] ..................................................................... 11

5.1.3 Number of Records [Optional].......................................................... 11

5.1.4 Repetition Rate [Optional] ................................................................ 11

5.1.5 Number Type [Required] .................................................................. 12

5.1.6 Bytes per Number [Required] ........................................................... 12

DRDC Atlantic CR 2004-072 v

5.1.7 Number of Channels [Required] ....................................................... 12

5.1.8 Multiplex Length [Required]............................................................. 12

5.1.9 Number of Averages [Required] ....................................................... 13

5.1.10 X Axis Type [Required]................................................................... 13

5.1.11 Y Axis Type [Required]................................................................... 13

5.1.12 Sequence Number [Optional]........................................................... 13

5.1.13 Block Scaling Factor [Required]...................................................... 13

5.1.14 [Unused]........................................................................................... 14

5.1.15 [Unused]........................................................................................... 14

5.1.16 [Unused]........................................................................................... 14

5.2 Floating Point Block....................................................................................... 14

5.2.1 Sampling Frequency [Required]........................................................ 14

5.2.2 Heterodyning Frequency [Required]................................................. 14

5.2.3 Reference Level [Required] .............................................................. 15

5.2.4 Maximum Magnitude [Optional] ...................................................... 15

5.2.5 Gain Correction [Required]............................................................... 16

5.2.6 [Unused] ............................................................................................ 16

5.2.7 [Unused] ............................................................................................ 16

5.2.8 [Unused] ............................................................................................ 16

5.3 Text Block ...................................................................................................... 16

5.3.1 Date of Record 1 [Required] ............................................................. 16

5.3.2 Time of Record 1 [Required] ............................................................ 17

5.3.3 Y Axis Units [Required].................................................................... 17

5.3.4 User Message [Optional]................................................................... 17

5.4 Channel Block ................................................................................................ 17

5.5 Data File to User Unit Conversion ................................................................. 18

6. Fourier Transforms...................................................................................................... 20

6.1 Integer Block .................................................................................................. 22




6.1.4 Number of Sequential Transforms [Optional]................................... 22

vi DRDC Atlantic CR 2004-072




6.1.8 Number of Frequency Bins [Required] ............................................. 23






6.1.14 Window Type [Required] ................................................................ 24

6.1.15 Number of Zeros [Required] ............................................................ 25

6.1.16 [Unused]........................................................................................... 25


6.2.1 Start Frequency [Required] ............................................................... 25


6.2.3 Frequency Resolution [Required]...................................................... 26


6.2.5 Y axis Calibration Factor [Required] ................................................ 26

6.2.6 [Unused] ............................................................................................ 26

6.2.7 Percent Overlap [Required]............................................................... 27

6.2.8 Time Interval of a Single Fourier Transform [Required] .................. 27

6.3 Text Block ...................................................................................................... 27





6.4 Channel Block ................................................................................................ 28


7. Power Spectra.............................................................................................................. 30

7.1 Integer Block .................................................................................................. 32



DRDC Atlantic CR 2004-072 vii


7.1.4 Number of Sequential Spectra [Optional] ......................................... 32




7.1.8 Number of Frequency Bins [Required] ............................................. 33





7.1.13 [Unused]........................................................................................... 34

7.1.14 Window Type [Required] ................................................................ 34

7.1.15 Number of Zeros [Required] ............................................................ 35

7.1.16 [Unused]........................................................................................... 35


7.2.1 Start Frequency [Required] ............................................................... 35


7.2.3 Frequency Resolution [Required]...................................................... 36


7.2.5 [Unused] ............................................................................................ 36

7.2.6 Time Interval Between Spectra [Required] ....................................... 36

7.2.7 Percent Overlap [Required]............................................................... 36

7.2.8 Time Interval of a Single Fourier Transform [Required] .................. 37

7.3 Text Block ...................................................................................................... 37





7.4 Channel Block ................................................................................................ 38


8. Staggered Time Series ................................................................................................. 40

8.1 Integer Block .................................................................................................. 43

viii DRDC Atlantic CR 2004-072




8.1.4 Repetition Rate [Optional] ................................................................ 43




8.1.8 Multiplex Length [Required]............................................................. 44






8.1.14 [Unused]........................................................................................... 46

8.1.15 [Unused]........................................................................................... 46

8.1.16 [Unused]........................................................................................... 46


8.2.1 Sampling Frequency [Required]........................................................ 46


8.2.3 Reference Level [Required] .............................................................. 47


8.2.5 Gain Correction [Required]............................................................... 48

8.2.6 [Unused] ............................................................................................ 48

8.2.7 [Unused] ............................................................................................ 48

8.2.8 [Unused] ............................................................................................ 48

8.3 Text Block ...................................................................................................... 48

8.3.1 Reference Date [Required] ................................................................ 48

8.3.2 Reference Time [Required] ............................................................... 49



8.4 Channel Block ................................................................................................ 49


DRDC Atlantic CR 2004-072 ix

Bibliography ............................................................................................................................. 52

Initialisms and Acronyms......................................................................................................... 53

Distribution List........................................................................................................................ 54

x DRDC Atlantic CR 2004-072

List of Tables

Table 1. Report Versions ........................................................................................................... 2

Table 2. File Type Differentiation............................................................................................. 6

Table 3. Time Series File Header Description......................................................................... 10

Table 4. Fourier Transform File Header Description .............................................................. 21

Table 5. Power Spectra File Header Description..................................................................... 31

Table 6. Staggered Time Series File Header Description........................................................ 42

DRDC Atlantic CR 2004-072 xi

Acknowledgements The author would like to thank: Bruce Skinner for information on the arcane of tape and file formats and for copies of code that produces and reads DREA .DAT32 files.

xii DRDC Atlantic CR 2004-072

1. Introduction

For several years DRDC Atlantic has been producing binary data files in “DREA .DAT32” format. This format is an extension of the “DREA .DAT” format, the specifications for which were set down in A Standard for the DREA Data Descriptor Block [Caldwell, 1987]. That earlier standard worked well for its time, but since it was developed the equipment capabilities and researchers’ requirements have outstripped the capabilities of that format. The name of the format derives from its development at DREA (later renamed DRDC Atlantic) for in-house use. That name was kept since it is by now well established and because renaming the format “DRDC .DAT32” would imply that it is a format developed by and for all DRDC locations. The DREA .DAT32 format that evolved from the DREA .DAT format is basically a high resolution version of the earlier format. That is, 4-byte integers and 8-byte floats replaced the 2-byte integers and 4-byte floats in the files’ headers. (The use of 4-byte integers, i.e. 32-bit integers, in DREA .DAT32 files’ headers is what gave this format its familiar name.) This change allowed for the greater number sizes and finer resolutions currently needed. Two other major changes from the earlier data format should be mentioned. The first is that whereas the earlier standard had no endianess requirements, the current format specifies that the data be stored in big endian form. The last major change is that the size DREA .DAT32 file header blocks has been changed from 512 bytes to 1024 bytes. As part of this change, some of the contents of the header have been moved around a bit to make it easier to read the contents of both the header block and any supplemental header blocks that the files may have. This topic is explained in more detail in Sections 3.2 for general information and 5 on for specific file types. Since the general format of DREA .DAT32 files is essentially that of DREA .DAT files, and since Caldwell’s report was widely circulated and used as a programming reference, that report was used as a guide to the writing of this report. In fact a significant portion of the text in this report was taken directly from that report since the information is still valid. Section 1 of that report may also be consulted for background information on the development of the original DREA .DAT format.

DRDC Atlantic CR 2004-072 1

2. Report Versions

This document is not intended to be exclusionary in nature, preventing further changes to the DREA .DAT32 format. Rather it is intended that as that format is extended (without causing conflicts with existing format capabilities) this document be added to and updated. Or put otherwise: this document is not the last word in the definition of the DREA .DAT32 format, but just the latest word. At the time of this writing (V. 1.0.a) it is known that there are several variants of the DREA .DAT32 format in use at DRDC Atlantic. These have been created to solve particular problems and store data in ways not covered by the original DREA .DAT32 formats. Programs have been written to write and read these almost-DREA .DAT32 formats, but the formats’ specifications are usually only known to small groups of personnel. Anyone outside these groups would probably be quite confused when coming across a non-standard format file. It is hoped that these variants can be codified and standardized, and their specifications added to this document where they could become known to and appreciated by a wider audience. It is for this reason that a version number is included in the title of this report. Its version numbers is in the form “Ma.Mi.L”, where three parts are separated by periods. The parts are:

Ma - a major version number, which would be incremented whenever major changes are made to the DREA .DAT32 format.

Mi - a minor version number, which would be incremented whenever minor changes are made to the DREA .DAT32 format.

L - a letter code that is incremented when changes are made to this document (error corrections, clarifications, etc.) but not to the DREA .DAT32 standard which it describes. However, whenever changes are made to Ma or Mi, L should be reset to “a”, indicating that it’s the first report version for the current DREA .DAT32 standard.

The definition of the terms “major change” and “minor change” are left to the author updating this reference, hopefully in consultation with the various users of the format. The following table lists the versions of this report along with notes on them.

Table 1. Report Versions

Version Notes 1.0.a Original release

2 DRDC Atlantic CR 2004-072

3. DREA .DAT32 File Description

DREA .DAT32 files are binary data files modelled on DREA .DAT files. As such the files were designed to record data arriving simultaneously from multiple devices at a constant time interval. Seen in this light, the major parts of the file are the time at the start of the data, the time or frequency interval between values in a channel, the number of devices, and the data themselves. Since the file’s format lent itself to others uses, the generic term “channel” was adopted for each data stream. In this way data from various devices, processed results, or equally timed model values could be placed into the file and be referred to without semantic confusion. Therefore in order to provide the required data, DREA .DAT32 files consist of two parts:

• a header that describes the composition of the file, and • the body of the file, containing the data.

There are DREA .DAT32 file format descriptions for several different types of data. Section 3.3 lists the types and contains information on how to differentiate between the three file types. As new file types, or variations on existing file types, are added to the DREA .DAT32 family, their descriptions and the ways in which they can be identified will be added to later versions of this document.

3.1 Data Endianess One requirement that is new to the DREA .DAT32 format is that the entire contents of all data files must be stored in big endian form. All devices that, by default, create data files in little endian format have the ability or procedures to produce big endian files, usually with insignificant overhead. A simple way to check if a file is “endian-compliant” is to look at the value of a known long integer in the header block, something that can be done quite easily on disk files but not quite as simply on tape files. For example, bytes 61-64 comprise the 16th long integer of the header, which is unused in the data formats specified in Sections 5 through 8 and so has the value of –999. If this integer has the value 436,011,007 instead, then its bytes are stored in reverse order and the data must be byte swapped before they can be used. Similarly, the 10th long integer (bytes 37-40) contain the X axis type. This can have the values 1, 2, or 3 for the file types of Sections 5, 6 and 7, and 8, respectively. By checking these items a user can determine both the type of data file (see Section 3.4


for more details) and whether the data need to be byte-swapped or not. The actual values of items that may be checked and their byte-swapped versions are: 1 16,777,216 2 33,554,432 3 50,331,648 -999 436,011,007

3.2 File Header A DREA .DAT32 file header block is 1024 bytes long. It may happen that the number of channels for a file’s type requires more space for channel data than can be provided in 1024 bytes. When this happens extra blocks of 1024 bytes are allocated to hold the additional information. Whether or not this is the case for a particular file can be determined by the contents of the first 1024 bytes. The “.4” subsections of Sections 5 on indicate when this happens and indicates how to determine from the number of channels if this is the case. This is particularly important when reading DREA .DAT32 files from tapes, where data are read in blocks of bytes of a fixed size. This allows an initial 1024 byte read, the determination of how many more 1024 byte blocks there are, if any, and then allows them to be read. The data in the file is also read from tapes in blocks, as is described in Section 5.1.1, for example. In the initial 1024 bytes, the first 512 bytes are used for general file information and the last 512 bytes comprise a channel block. This file header is independent of the number of channels of data in the file with the single exception of an item that contains that information. The channel related information is stored in the channel block, and if the amount of space required for this information exceeds 512 bytes then further blocks of 1024 bytes are added to the channel block until the block’s size can accommodate the data is was designed to hold. From this it can be seen that a DREA .DAT32 format data file will have a complete header, including all channel information, that is a multiple of 1024 bytes large. The file header has several distinct parts:

• an integer block with 16 long (4-byte) values • a floating point value block with 8 double (8-byte) values • a text block of 384 bytes, split among 4 variables, and • a channel block that is intended to contain information about the data

channels in the file; its exact size and contents depend on the type of file in which it appears

The exact contents of the header are discussed in Sections 5, 6, 7, and 8 for, respectively, time series, Fourier transform, power spectrum, and staggered time series data. Depending on the type of data stored in the file some of the header values


are required, some are optional, and some are not used. These will be mentioned in the explanations of the headers.

3.3 File Body Immediately following the header is the body of the file. If the file contains N_CHAN channels of data, then the N_CHAN values immediately following the header are the data from the data start time specified in the header. The next N_CHAN values are the data from the next time step, and so on. As an example, assume that:

• the data consist of 4-byte floating point numbers, • there are 7 channels, • the data start at 2004-08-23 19:22:31, and • the data are recorded 2 seconds apart.

In this case the first 28 bytes of the file’s body are the data from channels 1 through 7, and all were recorded at (or produced for) 2004-08-23 19:22:31. The next 28 bytes are the data from channels 1 through 7, and all were recorded at (or produced for) 2004-08-23 19:22:33. The only exception to this layout is if the file type has a multiplex value, and that value is greater than 1. This is discussed elsewhere in this document; in Section 5.1.8, for example. The only timestamp in the header is that of the first record. The time of the last record is not stored anywhere in the file and the number of records is optional and so may very well be missing. If the user desires either of these pieces of information, it must either be calculated from the file’s size and the above data or determined by reading the file until the end-of-file is encountered.

3.4 File Types and Differentiation Originally DREA .DAT32 files were used to store three types of data:

• Time Series data – header description in Section 5 • Fourier Transform data – header description in Section 6 • Power Spectra data – header description in Section 7

According to Caldwell [1987] the names of the data files containing DREA .DAT format data would indicate the type of data contained in these files. Time series (a.k.a. “raw”) data filenames were supposed to end with .DAT, Fourier transform data filenames were supposed to end in .FTR, and power spectra data filenames were supposed to end in .PWR. This rule may have been followed initially, but the


practice eventually slipped away. In fact, the majority of DREA .DAT format files obtained by this author ended in none of those file-defining endings. It seems reasonable to assume that if the same scheme were to be applied to DREA .DAT32 format files, the same thing would happen. Unfortunately the DREA .DAT header formats were not originally designed with a way of explicitly identifying the type of contents they contain. However the headers of the three original types of DREA .DAT files were examined to determine if anything in them could provide clues as to the files’ contents, and luckily a way was found. The following table lists the DREA .DAT32 format file types that are described in this document (the three original data types and one new type) and identifies the smallest number of items that can be use to differentiate the file types. The file differentiation items for the first three were discovered in their headers and the item for the fourth file type was decided upon arbitrarily. Since the run of data in DREA .DAT32 files is, essentially, along the X axis it did not seem unreasonable to use that parameter as a way to identify new data type files. This document requires that the X axis type of DREA .DAT32 file headers shall be used as a way of identifying the contents of the files. Besides the reason suggested above, the fact that at present the X axis type has only very few legal values makes it easy to check and validate. The columns in the table and their contents are:

File Type - the type of file being identified Item (Section) - the header block item(s) used as an identifier and, in

parentheses, the document section that describes the item Value - the values specific to the file type

Table 2. File Type Differentiation

File Type Item (Section) Value Time series X Axis Type (5.1.10) 1 Fourier Transform Number Type (6.1.5)

X Axis Type (6.1.10)9 or 10

2 Power Spectra Number Type (7.1.5)

X Axis Type (7.1.10)1 or 2

2 Staggered time series X Axis Type (8.1.10) 3

3.5 Potential Problem Some DREA .DAT32 files are created on two different computers: the header is created on one and data are collected and output into DREA .DAT32 format on another. Users should try to determine when this happens because different types of computers may store floating point data using different schemes.


This problem mainly affects files of time series data recorded aboard the CFAV Quest (UAT files, for example), but a wary user would be advised to inquire about the way in which unfamiliar files were created. The specific problem with the UAT files is that the headers are not created on VAX computers but that the bodies of the files are. VAXs do not store floating point data in the commonly used IEEE 754 standard, but in a proprietary, non standard format. Files created in this manner are, strictly speaking, non-compliant with DREA .DAT32 standards since different methods must be used to read the different parts of the files. However, there exists at DRDC Atlantic a number of routines that will translate files created in the combined format to be totally in the IEEE standard, and thus make them fully DRDC .DAT32 compliant.


4. Tape Storage

The byte order in DREA .DAT32 files is network byte order as defined by the TCP/IP standards, which is effectively big endian. The tape file system used for DREA .DAT32 data files is a subset of the “ANSI standard for file structure and labelling of magnetic tapes for information interchange”. The DREA .DAT32 tape data file format follows that of the original DREA .DAT tape format, which was developed just before ANSI X3.27-1978 became a standard. The DREA .DAT32 format essentially follows version 4 of the ANSI standard, with the only difference between the two being that the DREA .DAT32 format does not have the ANSI standard’s HDR2 and EOF2 labels.


5. Time Series

The main use of DREA .DAT32 files is the storage of time series data. Data are usually acoustic in nature, although non-acoustic data may also be stored in this format. Acoustic data are normally collected from a series of receivers, with all available receivers being sampled simultaneously and usually at a rate of several thousand values per second per device. Time series data may also be generated by models, in which case the channels may refer not to different devices on a sensor array but to different locations, depths, etc. In either case the data for all channels must be produced at simultaneous times and at a constant rate. The following table presents the time series file header block contents. It is followed by a more detailed description of each parameter. The columns in the table are:

Block Type - This lists the four types of data block in the header. Item - This is the sequential item number within its block. Size - This is the item’s size in bytes. Parameter - A description of the item is presented here. Each entry is flagged

with a character in brackets. The meanings of the flags are: R - required O - optional U - unused for this data type

Values - For the various Parameter flags this contains: R - allowed values when they are restricted O - values to use when the file’s creator opts not to fill in a valid

value for the parameter U - value to use


Table 3. Time Series File Header Description

Block Type Item Size

(bytes) Parameter [R/O/U] Values

Integer 1 4 Block size [R] 2 4 Record size [O] -1 if not used 3 4 Number of records [O] -1 if unknown 4 4 Repetition rate [O] -1 5 4 Number type [R] 1 – integers

2 – floating point numbers 9 – complex integers 10 – complex floating point

numbers 6 4 Bytes per number [R] 2 – short integers

4 – long integers, single precision floats or complex

8 – double precision floats or complex

7 4 Number of channels [R] ≥ 1 8 4 Multiplex length [R] ≥ 1 9 4 Number of averages [R] ≥ 1 10 4 X axis type [R] 1 11 4 Y axis type [R] 1 – linear

2 – RMS amplitude squared 4 – logarithmic

12 4 Sequence number [O] -1 if not used 13 4 Block scaling factor [R] 14 4 Unused [U] -999 15 4 Unused [U] -999 16 4 Unused [U] -999

Float 1 8 Sampling frequency [R] > 0.0 2 8 Heterodyning frequency [R] 3 8 Reference level [R] 4 8 Max. magnitude of data [O] -1.0 if not used 5 8 Gain correction [R] 1.0 or 1.00343332 6 8 Unused [U] -999.0 7 8 Unused [U] -999.0 8 8 Unused [U] -999.0

Text 1 10 Date of first record [R] 2 10 Time of first record [R] 3 44 Y axis units [R] 4 320 User message [O] blank if not used

Channel Block

1 4/channel Channel gain [R] 0 – for unused elements


5.1 Integer Block

5.1.1 Block Size [Required]

Block sizes are something required for tape processing of files, either in their creation to or reading from tape. This is because tape machines process data in blocks, and to be most efficient the reads and writes should maximize the block sizes, and so minimize the number of reads and writes. The minimum size of a block should be one record, as defined in the next section. The optimum size would be an integral multiple of one record up to a size that is less than the capabilities of the tape unit being used. As one example, TIAPS time series data blocks have a maximum size of 64512 bytes. However since the maximum is machine dependant, some knowledge of the capabilities of the tape unit being used is required. A not uncommon restriction on some devices is based on a 16 bit I/O limitation, meaning that the maximum block size allowed is 216-1, or 65535 bytes. DREA .DAT32 files on hard drives or other randomly accessible media are basically oblivious to block sizes since files on these devices can be read in whatever size portions as a user desires.

5.1.2 Record Size [Optional]

This is the number of bytes required to store a record’s worth of data and originally was the number of bytes transferred at a time between a computer and a sensor. The record size is found by multiplying the number of channels times the number of bytes required for a single datum from a channel times the multiplex length. See Section 5.1.8 for more on this topic. If this value is not calculated for the file then a value of –1 is to be used.

5.1.3 Number of Records [Optional]

If the number of records in the file is known that value may be stored here. If the value is unknown the default value of –1 is to be stored.

5.1.4 Repetition Rate [Optional]

This item is not normally used in time series files and must have –1 for its value.


5.1.5 Number Type [Required]

This describes the type of number used to represent the data in the body of the file. At present four types of numbers may be represented with the code values: 1 - integer 2 - floating point 9 - complex integer 10 - complex floating point

5.1.6 Bytes per Number [Required]

This parameter contains the number of eight-bit bytes used to represent a channel’s value in a record. The options, followed by non-exclusive examples, are 2 - short integers 4 - long integers, single precision floats, single precision complex floats 8 - double precision floats, double precision complex floats This value must be consistent with the number type.

5.1.7 Number of Channels [Required]

When data are sampled from a number of sensors, the data from each is transferred to the acquisition system via a channel. In this case the number of channels describes the number of active sensors having data recorded. However since DREA .DAT32 files are not constricted to containing sensor data, this parameter simply indicates the number of coincident data streams recorded in the file’s body, regardless of their origin.

5.1.8 Multiplex Length [Required]

This parameter is the number of consecutive samples recorded from one channel before samples are received from another channel. This parameter, combined with the number of channels, data type, and number of bytes per number, is sufficient to decode any time series data file. Normally one value from one channel is stored in the file, followed by one value from the next channel, and so on until one value from each channel is stored. Then the second value from each channel, in order, is stored. In this case the multiplex length is 1. If the multiplex length is 3, for example, the file’s data block starts with 3 values from the first channel. These are followed by 3 values from the second channel, 3


from the third channel, and so on. In all cases the first datum of each group of 3 was recorded at the same time, as were the second of each group and the third of each group. Since a record is defined as having the same amount of data from each channel, it consists N bytes, where:

N = number of channels × multiplex length × bytes/number

5.1.9 Number of Averages [Required]

The data sampled at each sensor or produced by a model or analysis routine may be averaged before being written to a DREA .DAT32 file. When this is the case this value is used to indicate how many values were averaged to produce the file’s value. If no averaging was done this should have the value 1.

5.1.10 X Axis Type [Required]

This must have a value of 1 for time series data.

5.1.11 Y Axis Type [Required]

This is used to indicate the type of Y axis for which the data have been prepared. The usual value is 1, but the allowable options are: 1 - linear 2 - RMS amplitude squared 4 - logarithmic

5.1.12 Sequence Number [Optional]

This is a user defined file identification number that may be used to determine where in a sequence a file occurs. This can be useful when continuously recorded data are saved over a number of contiguous files. This could happen if files are being saved to a storage device (e.g. data tape, CD, DVD) that becomes filled and is replaced with a new one. A value of –1 indicates that sequence numbering was not used or is unknown.

5.1.13 Block Scaling Factor [Required]


The block scaling factor is an integer used as an exponent of 2, the result of which is used to convert data file values to absolute values in user’s units. See Section 5.5 for an illustration of how this value is used.

5.1.14 [Unused]

This item is unused and must have –999 for its value.

5.1.15 [Unused]


5.1.16 [Unused]


5.2 Floating Point Block

5.2.1 Sampling Frequency [Required]

The sampling frequency is the rate at which the data for the channels are acquired. By design this is the same for all channels, but it may be affected by postprocessing with items like heterodyning and decimation. A negative value for a sampling frequency indicates that the data have been heterodyned while a positive value indicates that this is a straight-through data rate with no heterodyning. The standard unit of sampling frequency is samples per second, or Hertz, where a sample is one value, the type and size of which are defined in the integer block of the header.

5.2.2 Heterodyning Frequency [Required]

In a typical data acquisition system the sampling frequency determines the frequency bandwidth of the data. Often a large bandwidth is not needed, nor is the large amount of data associated with it.


Heterodyning is a process that frequency-translates incoming time series data. After heterodyning, results are usually filtered and decimated, typically by factors of 20 to 100 with a similar reduction in the final bandwidth. The heterodyning frequency is the frequency around which this reduced band is centred. If the data are not heterodyned a value 0.0 Hz is normally assigned to this parameter. Real or complex modes of heterodyning can be done. Real mode is indicated by negating the heterodyning frequency and complex mode is indicated by a positive value.

5.2.3 Reference Level [Required]

The reference level is another multiplicative scaling factor required to convert the raw data sample amplitudes to absolute units. As an example of how to calculate this value, for a 12 bit A/D system with maximum signal levels of +5 volts DC, the reference level would be: RLEVEL = 5 V / 2(12 bits – 1)

= 5 / 2048 = 0.00244140625 volts This is the quantization level, the level corresponding to the least significant bit of the digitized number. See Section 5.5 for an illustration of how this value is used.

5.2.4 Maximum Magnitude [Optional]

This parameter is set according to the dynamic range of the acquisition system, a function of the number of output bits in the analogue to digital (A/D) conversion. For example, in a 12 bit A/D system the absolute value of the largest integer number representable is 2048 (i.e. 212-1), whereas in a 16 bit system the maximum magnitude is 32768 (i.e. 216-1). Note that this value is not necessarily the actual maximum value recorded in the data file, but the maximum possible value based on the equipment used. A value of –1.0 indicates that this value is unknown or was not used.


5.2.5 Gain Correction [Required]

If the gain values in the channel block (Section 5.4) are presented in dB then this must have a value of 1.0, but if the gains are powers of 2 then the gain correction must be 1.00343332. See Section 5.5 for an illustration of how this value is used.

5.2.6 [Unused]

This item is unused and must have –999.0 for its value.

5.2.7 [Unused]


5.2.8 [Unused]


5.3 Text Block

5.3.1 Date of Record 1 [Required]

This 10 byte string contains the date on which the first record in the file was produced. The date is stored as a series of ASCII characters representing the day of the month, the month name, and the year, in a fixed format in that order. An example is: “□29-Dec-84”. The string contains:

• □, a blank space (ASCII character 32) for the first character • day of the month in positions 2 and 3; if the day is less than 10 then character

2 must be either a blank space or a zero • hyphens in spaces 4 and 8 • a three letter abbreviation for the month’s name, which may be in any mixture

of cases (i.e. “DEC”, “Dec”, “dec”, “dEc”, etc. are all valid abbreviations for December) in positions 5 through 7

• the last two digits of the year, in positions 9 and 10, are interpreted as being in the 1900s if the values are from “70” to “99” and in the 2000s otherwise.


5.3.2 Time of Record 1 [Required]

This 10 byte string contains the time at which the first record in the file was produced. The time is stored to the nearest second as a series of ASCII characters representing the hour, minute, and second, in a fixed format in that order. As an example: “□16:07:47φ”. The string contains:

• □, a blank space (ASCII character 32) for the first character • colon separators in spaces 4 and 7 • φ, a null character (ASCII character 0) for the last character • characters from “0” to “9” for all the other characters

Hours run from 0 through 23 and both minutes and seconds run from 0 through 59.

5.3.3 Y Axis Units [Required]

Forty-four bytes are allocated to this value, which is entered as a string of ASCII characters starting at the first character. This parameter indicates the units of the calibrated data. As an example, if the units are to be volts, this information would be placed in the first six bytes of the variable as “voltsφ”. The null character is always used to indicate the end of the units label.

5.3.4 User Message [Optional]

Space is available for an optional user message that may assist in decoding or explaining data. When a message is included in the 320 bytes allocated for it, it should be terminated by the null character. The acquisition program usually inserts this message.

5.4 Channel Block The channel block has two adjacent 2-byte integers reserved for each channel of data in the file. The first is an unsigned short integer and the second is a signed short integer. The data placed in this area consists of pairs of channel names and their corresponding gains. The names are user defined, with values ranging from 0 through 65535. The associated gain values may range from –128 through +128 dB.


As an example of how this area is used, if a file has 12 channels with gains of –23 for the first three channels, -19 for the next five, and +21 for the last four, then the first 24 2-byte integer values in this area would be: 0 -23 1 -23 2 -23 3 -19 4 -19 5 -19 6 -19 7 -19 8 21 9 21 10 21 11 21 The channels are named 0 through 11, values that appear as the first, third, fifth, etc. locations in the run. These channel names are followed by their gains. Should there be any space remaining in the channel block after the file’s gains are written, these remaining locations should be given the value of zero. In the case of the above example there would be 232 remaining 2-byte integers, all of which would be set to zero. The default header block size is 1024 bytes, of which the last 512 bytes are allocated for the channel block. At 4 bytes/channel, this allows for 128 channels. If there are more than 128 channels in the file (Section 5.1.7), then extra space is allocated for the header in blocks of 1024 bytes. Each additional 1024-byte block has space for information on 256 more channels. The following table indicates the header size as a function of the number of data channels in the file.

Number of channels Total header size 1 – 128 1024 bytes

129 – 384 2048 bytes 385 – 640 3072 bytes 641 – 896 4096 bytes

etc. etc.

5.5 Data File to User Unit Conversion This section shows how the data read from the file are converted to user units, as specified in the Section 5.3.3 Y axis units. The conversion is performed via the following equation:

OUT = IN × 2BSF × RLEVEL × GAINCOR

where: OUT - is the absolute level in absolute units (Section 5.3.3) IN - is a value read from the DREA .DAT32 file BSF - is the block scaling factor (Section 5.1.13) RLEVEL - is the reference level for the acquisition system (Section 5.2.3) GAINCOR - is the gain correction (Section 5.2.5)


As an example, with: • a DREA .DAT32 file value of 333.0 • a block scale value of –2 • the reference level of Section 5.2.3 • a gain correction of 1.00 • a stated Y axis units of “volts”

The conversion produces a voltage of: OUT = IN × 2BSF × RLEVEL × GAINCOR = 333.0 × 2-2 × 0.0024414 × 1.00 = 0.2032 volts


6. Fourier Transforms

The calculation of Fourier transforms is an intermediate step in determining many parameters about a time series. These include power spectral densities, cross power spectral densities, transfer functions, convolutions and filtering, correlations, and interpolation. Therefore it is often useful to retain results of the raw Fourier transform so as to make them available to later analyses. Also, since subsets of the full transform can be kept, it is a means of reducing the amount of time series data. Given these requirements, a set of specifications was produced that would allow Fourier transform results to be stored in DREA .DAT32 format files. Generally the relation between a time series DREA .DAT32 file and its Fourier counterpart is fairly straightforward: the channel N data in the time series file was put through a Fourier transform and the results stored in channel N of a Fourier transform file. However this is not always the case, and programs exist that can produce Fourier transform DREA .DAT32 files without starting from DREA .DAT32 time series files. Regardless of their origin, this section describes the DREA .DAT32 specifications for Fourier transform files. The following table presents the header block contents for this file type. It is followed by a more detailed description of each parameter. The columns in the table are:






Table 4. Fourier Transform File Header Description



Integer 1 4 Block size [R] 2 4 Record size [O] -1 if not used 3 4 Number of records [O] -1 if unknown 4 4 Number of sequential

transforms [O]-1 if unknown

5 4 Number type [R] 9 – complex integers 10 – complex floating point

numbers 6 4 Bytes per number [R] 2 – short complex integers

4 – long complex integers or single precision complex floats

8 – double precision complex floats

7 4 Number of channels [R] ≥ 1 8 4 Number of frequency

bins [R]≥ 1

9 4 Number of averages [R] ≥ 1 10 4 X axis type [R] 2 11 4 Y axis type [R] 1 – linear (i.e. amplitude)


12 4 Sequence number [O] -1 if not used 13 4 Block scaling factor [R] 14 4 Window type [R] 1 – No window

2 – Hanning 3 – Hamming 4 – Kaiser-Bessel (Taylor)

15 4 Number of zeros [R] ≥ 0 16 4 Unused [U] -999

Float 1 8 Start frequency [R] > 0.0 2 8 Heterodyning frequency [R] 3 8 Frequency resolution [R] > 0.0 4 8 Max. magnitude of data [O] -1.0 if not used 5 8 Calibration factor [R] 6 8 Unused [U] -999.0 7 8 Percent overlap [R] 0.0 through 100.0 8 8 Time interval of one

Fourier transform (hrs) [R]> 0.0

Text 1 10 Date of first record [R] 2 10 Time of first record [R] 3 44 Y axis units [R] 4 320 User message [O] blank if not used

Channel Block



6.1 Integer Block




This is the number of bytes required to store a record’s worth of data. The record size is found by multiplying the number of channels times the number of bytes required for a single datum from a channel. If this value is not calculated for the file then a value of –1 is to be used.



6.1.4 Number of Sequential Transforms [Optional]


Transform results are located sequentially in the data file. If known, the number of transforms can be stored at this location in the header. The value –1 indicates that the number of transforms in the file is unknown.


The results of Fourier analysis are complex numbers. Therefore only two number types are possible: 9 - complex integer 10 - complex floating point




This parameter equals the number of channels of Fourier transform data that are located in the file. If the data in a Fourier transform file were calculated from a DREA .DAT32 file of time series data then this value may be the same as the number of channels in the time series file if all channels were transformed. This value may be less than the number of time series channels if not all time series channels were transformed. The channel block (Section 6.4) will contain information on which time series channels were processed, should the Fourier transform data have been produced in this way.

6.1.8 Number of Frequency Bins [Required]

A number of frequency bins of equal width are calculated when a Fourier transform is calculated. For example, a 1024-point complex Fourier transform calculates 1024 complex frequency bins. This number is to be placed in the file. If the full bandwidth of the Fourier transform is not retained then the number of frequency bins equals the number of bins in the subset.



Normally Fourier transforms are not averaged, and so this value is usually 1; however this is not a rule and this item permits a way of indicating that transform averaging was performed.


This must have a value of 2 for Fourier transform data, indicating frequency domain data.


Fourier transforms usually maintain the units of amplitude; therefore 1 is the normal value for this parameter. However all the valid options are: 1 - linear (i.e. amplitude) 2 - RMS amplitude squared 4 - logarithmic


This is a user defined file identification number that may be used to determine where in a sequence a file occurs. When Fourier transform files are produced from time series files the sequence numbers of the time series originals are usually carried over to the Fourier transform files. A value of –1 indicates that sequence numbering was not used or is unknown.


The block scaling factor is an integer used as an exponent of 2, the result of which is multiplied by the individual sample magnitudes. When Fourier transform files are produced from time series files, this value is usually carried over from the original time series files. See Section 6.5 for an illustration of how this value is used.

6.1.14 Window Type [Required]


When viewed from the frequency domain, multiplying a time series by a “data window” suppresses large sidelobes in the effective filter obtained from the raw transform. In the time domain this equates to “rounding off” potential discontinuities at each end of the finite segment of the time history being analysed. There are several windows from which to select, each of which has its own characteristic sidelobe reduction. The trade-off to sidelobe reduction is a loss of frequency resolution. For further details see Harris [1978], Kaiser and Schafer [1980], and Nuttall [1981]. This parameter’s value indicates which window, if any, has been applied to the time series data. The options are:

1 - no window 2 - Hanning window 3 - Hamming window 4 - Kaiser-Bessel (sometimes called a Taylor) window

6.1.15 Number of Zeros [Required]

It is possible to compute Fourier transform points at a finer frequency spacing by adding zeros to the time series. These points add no information but result in interpolation in the frequency domain. This value in the file is simply the total number of zeros inserted in each time series block.

6.1.16 [Unused]



6.2.1 Start Frequency [Required]

With time series data digitized at a sampling rate of S Hz, there is a resultant frequency bandwidth of

F = S / 2 where: F - is the resulting folding (or Nyquist) frequency which changes whenever

the sampling frequency is changed – through decimation, for example. Full bandwidth spectra have 0.0 Hz as a start frequency. It is not necessary to retain the full frequency spectrum for recording in the file. A subset of the spectrum starts at a specified frequency and spans a specified number of frequency bins. The start frequency is the frequency of the first bin.


The standard unit of the start frequency is Hertz (Hz).


If the original time series data were heterodyned it is useful to carry over the heterodyning frequency recorded in that file. The standard unit of the heterodyning frequency is Hertz (Hz).

6.2.3 Frequency Resolution [Required]

The frequency resolution is the frequency width of each bin of the Fourier transform, a function of the Fourier analysis size. Combined with the start frequency and the number of frequency bins, the frequency of the upper frequency bin in the file can be found. The standard unit of frequency resolution is Hertz (Hz).


When Fourier transforms are calculated it is possible to obtain the maximum value and when all calculations are done for than file, insert that value into the header. The units of the maximum magnitude are consistent with that defined in the Y axis units parameter of the text block (Section 6.3.3). A value of –1.0 indicates that this value is unknown or was not used.

6.2.5 Y axis Calibration Factor [Required]

This calibration factor is a multiplicative scaling factor required to convert the transform amplitudes to absolute units. See Section 6.5 for an illustration of how this value is used.

6.2.6 [Unused]



6.2.7 Percent Overlap [Required]

This is the fraction of the time series for one Fourier transform which overlaps the time series for the previous Fourier transform. Overlapping is done to produce Fourier transforms at closer time intervals and to reduce the data processing losses due to windowing and other effects.

6.2.8 Time Interval of a Single Fourier Transform [Required]

This parameter is the duration of a time series used to produce a single Fourier transform. For example, a 1024 point Fourier transform with the time series sampled at 2048 samples/second (i.e. 0.5 seconds worth of data) has a time interval of 0.0001388889 hours (1024 pts ÷ 2048 pts/sec ÷ 3600 sec/hr). The parameter’s units are hours.

6.3 Text Block


This 10 byte string contains the date on which the first record of the time series data used for Fourier transforming was recorded. The date is stored as a series of ASCII characters representing the day of the month, the month name, and the year, in a fixed format in that order. An example is: “□29-Dec-84”. The string contains:






This 10 byte string contains the time at which the first record of the time series data used for Fourier transforming was recorded. The time is stored to the nearest second


as a series of ASCII characters representing the hour, minute, and second, in a fixed format in that order. As an example: “□16:07:47φ”. The string contains:




Forty-four bytes are allocated to this value, which is entered as a string of ASCII characters starting at the first character. This parameter indicates the units of the calibrated data. As an example, if the units are to be volts, this information would be placed in the first six bytes of the variable as “voltsφ”. The null character is always used to indicate the end of the units label.


Space is available for an optional user message that may assist in decoding or explaining data. When a message is included in the 320 bytes allocated for it, it should be terminated by the null character. All information from the time series file (if any) is usually retained. Any further information pertaining to the Fourier transform calculations may be inserted here.

6.4 Channel Block The channel block has two adjacent 2-byte integers reserved for each channel of data in the file. The first is an unsigned short integer and the second is a signed short integer. The data placed in this area consists of the names of data channels from a time series file that have been Fourier transformed and their associated gain values. The names are user defined, with values ranging from 0 through 65535. The associated gain values may range from –128 through +128 dB. As an example of how this area is used, if a time series file has 12 channels with gains of –23 for the first three channels, -19 for the next five, and +21 for the last four, and


channels 0, 4, 7, and 8 were Fourier transformed, then the following would appear in the Fourier transform file’s channel block: 0 -23 4 -19 7 -19 8 21 Should there be any space remaining in the channel block after the gains are written, these remaining locations should be given the value of zero. In the case of the above example there would be 248 remaining 4-byte integers, all of which would be set to zero. The default header block size is 1024 bytes, of which the last 512 bytes are allocated for the channel block. At 4 bytes/channel, this allows for 128 channels. If there are more than 128 channels in the file (Section 6.1.7) , then extra space is allocated for the header in blocks of 1024 bytes. Each additional 1024-byte block has space for information on 256 more channels. The following table indicates the header size as a function of the number of data channels in the file.



etc. etc.


OUT = IN × 2BSF × CALFAC

where: OUT - is the absolute level in absolute units (Section 6.3.3) IN - is a value read from the DREA .DAT32 file BSF - is the block scaling factor (Section 6.1.13) CALFAC - is the Y axis calibration factor (Section 6.2.5)


7. Power Spectra

A power spectrum is the distribution of variance (mean square values) of a time series over frequency. At DRDC Atlantic the power spectrum of a time series may be obtained by a number of programs. In the normal course of events a time series file is processed to obtain its Fourier transform. Those results may be windowed, averaged, or overlapped, before being used to produce a power spectrum. The resultant spectrum, usually as floating point numbers, may then be put into a DREA .DAT32 format file. This section describes the header block of that power spectra file. The header block has a number of parameters that are designed to hold data carried over from DREA .DAT32 time series and Fourier transform files, based on the assumption that the analysis stream made use of them. However this is not necessarily the case. It is quite possible that neither the original time series or Fourier transform files were DREA .DAT32 files. In fact it’s possible that the production of a power spectra file began with a Fourier transform file minus an original time series. It’s also possible that a power spectra file was generated directly by a program without using either time series or Fourier transform files. In these cases the “carried data” parameters would be filled with default values. Regardless of the origin of a DREA .DAT32 power spectra file, this section describes the specifications for that type of file. The following table presents the header block contents for this file type. It is followed by a more detailed description of each parameter. The columns in the table are:






Table 5. Power Spectra File Header Description



Integer 1 4 Block size [R] 2 4 Record size [O] -1 if not used 3 4 Number of records [O] -1 if unknown 4 4 Number of sequential

spectra [O]-1 if unknown

5 4 Number type [R] 1 – integers 2 – floating point numbers

6 4 Bytes per number [R] 2 – short integers 4 – long integers or single

precision floats 8 – double precision floats

7 4 Number of channels [R] ≥ 1 8 4 Number of frequency

bins [R]≥ 1

9 4 Number of averages [R] ≥ 1 10 4 X axis type [R] 2 11 4 Y axis type [R] 1 – linear (i.e. amplitude)


12 4 Sequence number [O] -1 if not used 13 4 Unused [U] -999 14 4 Window type [R] 1 – No window

2 – Hanning 3 – Hamming 4 – Kaiser-Bessel (Taylor)

15 4 Number of zeros [R] ≥ 0 16 4 Unused [U] -999

Float 1 8 Start frequency of first bin [R]

> 0.0

2 8 Heterodyning frequency [R] 3 8 Frequency resolution [R] > 0.0 4 8 Max. magnitude of data [O] -1.0 if not used 5 8 Unused [U] -999.0 6 8 Time interval between

spectra [R]

7 8 Percent overlap [R] 0.0 through 100.0 -1.0 if unknown

8 8 Time interval of one Fourier transform (hrs) [R]

> 0.0 -1.0 if unknown

Text 1 10 Date of record 1 [R] 2 10 Time of record 1 [R] 3 44 Y axis units [R] 4 320 User message [O] blank if not used

Channel Block



7.1 Integer Block




This is the number of bytes required to store a record’s worth of data. The record size is found by multiplying the number of channels times the number of bytes required for a single datum from a channel. If this value is not calculated for the file then a value of –1 is to be used.


If the number of records in the file is known that value may be stored here. If the value is unknown the default value of –1 is to be used.

7.1.4 Number of Sequential Spectra [Optional]


Most spectral analysis programs allow the user to choose the number of spectral estimates (frames) to calculate. If this option is utilized the number of estimates should be put into the header block. If the value is unknown the default value of –1 is to be used.


Power spectra can be written in one of two number types: 1 - integers 2 - floating point numbers


This parameter contains the number of eight-bit bytes used to represent a channel’s value in a record. The options, followed by non-exclusive examples, are 2 - short integers 4 - long integers, single precision floats 8 - double precision floats This value must be consistent with the number type.


This parameter equals the number of channels of power spectra data that are located in the file. If the data in a power spectra file were calculated from a DREA .DAT32 file of time series data then this value may be the same as the number of channels in the time series file if all channels were processed. This value may be less than the number of time series channels if not all time series channels were processed. The channel block (Section 7.4) will contain information on which time series channels were processed, should the power spectra data have been produced in this way.

7.1.8 Number of Frequency Bins [Required]

A number of frequency bins of equal width are calculated when a Fourier transform is calculated. When the power spectra are calculated from the transform data, the same number of frequency bins is retained. If the full bandwidth of the power spectra is not retained then the number of frequency bins is the number of bins in the subset.



Frequency band averaging (averaging together neighbouring raw spectral estimates) results in better estimates of power spectra. This reduces the noise levels to give a better signal to noise ratio, thus resulting in smoother individual spectra. This value is the number of bands that were averaged.


This must have a value of 2 for power spectra data, indicating frequency domain data.


Power spectra are calibrated in one of three units: 1 - RMS 2 - magnitude squared 4 - logarithmic magnitude


This is a user defined file identification number that may be used to determine where in a sequence a file occurs. When power spectra files are derived from DREA .DAT32 time series files the sequence numbers of the time series originals are usually carried over to the power spectra files. A value of –1 indicates that sequence numbering was not used or is unknown.

7.1.13 [Unused]


7.1.14 Window Type [Required]

When viewed from the frequency domain multiplying a time series by a “data window” suppresses large sidelobes in the effective filter obtained from the raw transform. In the time domain this equates to “rounding off” potential discontinuities at each end of the finite segment of the time history being analysed. There are several windows from which to select, each of which has its own characteristic sidelobe reduction. The trade-off to sidelobe reduction is a loss of frequency resolution. For further details see Harris [1978], Kaiser and Schafer [1980], and Nuttall [1981].


This parameter’s value indicates which window has been applied to the time series data, if any. The options are:

1 - no window 2 - Hanning window 3 - Hamming window 4 - Kaiser-Bessel (sometimes called a Taylor) window

7.1.15 Number of Zeros [Required]

It is possible to compute Fourier transform (and so power spectra) points at a finer frequency spacing by adding zeros to the time series. These points add no information but result in interpolation in the frequency domain. This value in the file is simply the total number of zeros inserted in each time series block.

7.1.16 [Unused]



7.2.1 Start Frequency [Required]

It is not necessary to retain full frequency spectra for insertion into a DREA .DAT32 file. A subset of the spectrum starts at a specified frequency and spans a specified number of frequency bins. The start frequency is the frequency of the first bin. The standard unit of the start frequency is Hertz (Hz).


If the original time series data were heterodyned it is useful to carry over the heterodyning frequency recorded in that file. (Hopefully if the data went through a Fourier transform file the heterodyning frequency would have been copied there as well.) The standard unit of the heterodyning frequency is Hertz (Hz).


7.2.3 Frequency Resolution [Required]

The frequency resolution is the frequency width of each bin of the power spectra, a function of the Fourier analysis size. Combined with the start frequency and the number of frequency bins, the frequency of the upper frequency bin in the spectra can be found. The standard unit of frequency resolution is Hertz (Hz).


When power spectral densities are calculated it is possible to find and retain the maximum value for insertion into the header block. For power spectra files the most common practice is to insert the absolute maximum value of 350 dB, calculated from 10log10-35, where 10-35 is the closest approximation to zero in 16 bits. The units of the maximum magnitude are consistent with that defined in the Y axis units parameter of the text block (as described in Section 7.3.3). A value of –1.0 indicates that this value is unknown or was not used.

7.2.5 [Unused]


7.2.6 Time Interval Between Spectra [Required]

The length of the Fourier transform, number of zeros, overlapping, and averaging are factors when determining the amount of time series data is required to calculate one power spectrum. This calculation is done in the spectral analysis program. The standard unit for this parameter is hours.

7.2.7 Percent Overlap [Required]

This is the fraction of the time series for one Fourier transform which overlaps the time series for the previous Fourier transform. Overlapping is done to produce Fourier transforms at closer time intervals and to reduce the data processing losses due to windowing and other effects.


This value is carried over from the DREA .DAT32 Fourier transform file used to produce this power spectra file, if any. If this file did not originate with a file of that type and the percent overlap is not known, a value of –1.0 must be used.

7.2.8 Time Interval of a Single Fourier Transform [Required]

This parameter is the elapsed time of a single Fourier transform. For example, a 1024 point Fourier transform with the time series sampled at 2048 samples/second has a time interval of 0.0001388889 hours (1024 pts ÷ 2048 pts/sec ÷ 3600 sec/hr). This value is carried over from the DREA .DAT32 Fourier transform file used to produce this power spectra file, if any. If this file did not originate with a file of that type and the percent overlap is not known, a value of –1.0 must be used. The parameter’s units are hours.

7.3 Text Block


This 10 byte string contains the date on which the first record of the data used to calculate power spectra was recorded. The date is stored as a series of ASCII characters representing the day of the month, the month name, and the year, in a fixed format in that order. An example is: “□29-Dec-84”. The string contains:






This 10 byte string contains the time at which the first record of the data used to calculate power spectra was recorded. The time is stored to the nearest second as a


series of ASCII characters representing the hour, minute, and second, in a fixed format in that order. As an example: “□16:07:47φ”. The string contains:




Forty-four bytes are allocated to this value, which is entered as a string of ASCII characters starting at the first character. This parameter indicates the units of the calibrated data. For acoustic logarithmic estimates, the Y axis label is “dB//uPa**2/Hzφ”. The null character is always used to indicate the end of the units label.


Space is available for an optional user message that may assist in decoding or explaining data. When a message is included in the 320 bytes allocated for it, it should be terminated by the null character. All information from the time series file (if any) is usually retained. Any further information pertaining to the Fourier transform calculations may be inserted here.

7.4 Channel Block The channel block has two adjacent 2-byte integers reserved for each channel of data in the file. The first is an unsigned short integer and the second is a signed short integer. The data placed in this area consists of the names of data channels from a time series file that have been Fourier transformed and their associated gain values. The names are user defined, with values ranging from 0 through 65535. The associated gain values may range from –128 through +128 dB. For power spectra estimates gains have normally been incorporated in the power spectrum calculations. If this is the case the gains for all channels are zero.


The default header block size is 1024 bytes, of which the last 512 bytes are allocated for the channel block. At 4 bytes/channel, this allows for 128 channels. If there are more than 128 channels in the file (Section 7.1.7), then extra space is allocated for the header in blocks of 1024 bytes. Each additional 1024-byte block has space for information on 256 more channels. The following table indicates the header size as a function of the number of data channels in the file.



etc. etc.

7.5 Data File to User Unit Conversion The data in a power spectra file are stored in the units specified in Section 7.3.3 and so no conversion is required.


8. Staggered Time Series

Consider a model that, in effect, sends an acoustic pulse through a medium in which are located a number of hydrophones at various distances from the source. The modelled data are produced simultaneously for each hydrophone at the same constant rate. If the data were stored in a standard time series DREA .DAT32 file format, as described in Section 5, with each hydrophone’s data recorded in a separate channel, the channels’ data would all have a similar appearance. Each channel would be seen to have a run of zeros for the period leading up to the pulse’s arrival, a burst of non-zero values as the pulse passes the hydrophone, and finally a run of post-pulse zeros that goes to the end of the file. Because the channels’ data all start at the same time, the location of each channel’s non-zero data burst would differ in its run of data. Depending on the pulse’s duration and hydrophones’ distances this non-zero burst might only be a very small portion of the total amount of space taken up by a channel’s data. The staggered time series format is a method that can be used to reduce the size of the data file. Instead of containing all the data from a sensor, it only has data from a period of interest in each channel. The majority of leading and trailing zeros are omitted, except for enough of them to make each channel’s sample the same duration. If each hydrophone in the model received a number of pulses they could be dealt with in one of two ways.

• If the pulses are close enough together then each channel could have the entire string of pulses for a hydrophone. Only the zeros preceding the first and following the last pulse would be clipped from the data.

• Widely spaced pulses from the same hydrophone could each be given their own channel, thus eliminating all the zeros between pulses.

In either case each channel would have a different starting time. Section 8.4 describes how this is handled, as is the many-channels/hydrophone case. The data would still be restricted to having exactly the same number of values per channel. This is most easily accomplished by determining the length of each channel’s pulse, identifying the longest run of data, and then zero padding any channel that has less than that number of values. The staggered time series format can also be used for actual recorded data if pulses or areas of interest in a time series can be isolated and extracted from a longer run. The major differences between the staggered time series format and the original time series format are the contents of the X axis type (Section 8.1.10) and the channel block (Section 8.4). The minor difference is the definition of the file’s date and time (Sections 8.3.1 and 8.3.2).


The following table presents the time series file header block contents. It is followed by a more detailed description of each parameter. The columns in the table are:






Table 6. Staggered Time Series File Header Description



Integer 1 4 Block size [R] 2 4 Record size [O] -1 if not used 3 4 Number of records [O] -1 if unknown 4 4 Repetition rate [O] -1 5 4 Number type [R] 1 – integers

2 – floating point numbers 9 – complex integers 10 – complex floating point

numbers 6 4 Bytes per number [R] 2 – short integers

4 – long integers, single precision floats or complex

8 – double precision floats or complex

7 4 Number of channels [R] ≥ 1 8 4 Multiplex length [R] ≥ 1 9 4 Number of averages [R] ≥ 1 10 4 X axis type [R] 3 11 4 Y axis type [R] 1 – linear


12 4 Sequence number [O] -1 if not used 13 4 Block scaling factor [R] 14 4 Unused [U] -999 15 4 Unused [U] -999 16 4 Unused [U] -999

Float 1 8 Sampling frequency [R] > 0.0 2 8 Heterodyning frequency [R] 3 8 Reference level [R] 4 8 Max. magnitude of data [O] -1.0 if not used 5 8 Gain correction [R] 1.0 or 1.00343332 6 8 Unused [U] -999.0 7 8 Unused [U] -999.0 8 8 Unused [U] -999.0

Text 1 10 Reference date [R] 2 10 Reference time [R] 3 44 Y axis units [R] 4 320 User message [O] blank if not used

Channel Block

1 8/channel Channel gain and time offsets [R]

0 – for unused elements


8.1 Integer Block




This is the number of bytes required to store a record’s worth of data and originally was the number of bytes transferred at a time between a computer and a sensor. The record size is found by multiplying the number of channels times the number of bytes required for a single datum from a channel times the multiplex length. See Section 8.1.8 for more on this topic. If this value is not calculated for the file then a value of –1 is to be used.



8.1.4 Repetition Rate [Optional]

This item is not normally used in time series files and must have –1 for its value.



This describes the type of number used to represent the data in the body of the file. At present four types of numbers may be represented with the code values: 1 - integer 2 - floating point 9 - complex integer 10 - complex floating point




When data are sampled from a number of sensors, the data from each is transferred to the acquisition system via a channel. In this case the number of channels describes the number of active sensors having data recorded. However since DREA .DAT32 files are not constricted to containing sensor data, this parameter simply indicates the number of data streams recorded in the file’s body, regardless of their origin.

8.1.8 Multiplex Length [Required]

This parameter is the number of consecutive samples recorded from one channel before samples are received from another channel. This parameter, combined with the number of channels, data type, and number of bytes per number, is sufficient to decode any time series data file. Normally one value from one channel is stored in the file, followed by one value from the next channel, and so on until one value from each channel is stored. Then the second value from each channel, in order, is stored. In this case the multiplex length is 1. If the multiplex length is 3, for example, the file’s data block starts with 3 values from the first channel. These are followed by 3 values from the second channel, 3 from the third channel, and so on. In all cases the first datum of each group of 3 was


recorded at the same time, as were the second of each group and the third of each group. Since a record is defined as having the same amount of data from each channel, it consists N bytes, where:

N = number of channels × multiplex length × bytes/number


The data sampled at each sensor or produced by a model or analysis routine may be averaged before being written to a DREA .DAT32 file. When this is the case this value is used to indicate how many values were averaged to produce the file’s value. If no averaging was done this should have the value 1.


This must have a value of 3 for staggered time series data.


This is used to indicate the type of Y axis for which the data have been prepared. The usual value is 1, but the allowable options are: 1 - linear 2 - RMS amplitude squared 4 - logarithmic


This is a user defined file identification number that may be used to determine where in a sequence a file occurs. This can be useful when continuously recorded data are saved over a number of contiguous files. This could happen if files are being saved to a storage device (e.g. data tape, CD, DVD) that becomes filled and is replaced with a new one. A value of –1 indicates that sequence numbering was not used or is unknown.


The block scaling factor is an integer used as an exponent of 2, the result of which is used to convert data file values to absolute values in user’s units.


See Section 8.5 for an illustration of how this value is used.

8.1.14 [Unused]


8.1.15 [Unused]


8.1.16 [Unused]



8.2.1 Sampling Frequency [Required]

The sampling frequency is the rate at which the data for the channels are acquired. By design this is the same for all channels, but it may be affected by postprocessing with items like heterodyning and decimation. A negative value for a sampling frequency indicates that the data have been heterodyned while a positive value indicates that this is a straight-through data rate with no heterodyning. The standard unit of sampling frequency is samples per second, or Hertz, where a sample is one value, the type and size of which are defined in the integer block of the header.


In a typical data acquisition system the sampling frequency determines the frequency bandwidth of the data. Often a large bandwidth is not needed, nor is the large amount of data associated with it.


Heterodyning is a process that frequency-translates incoming time series data. After heterodyning, results are usually filtered and decimated, typically by factors of 20 to 100 with a similar reduction in the final bandwidth. The heterodyning frequency is the frequency around which this reduced band is centred. If the data are not heterodyned a value 0.0 Hz is normally assigned to this parameter. Real or complex modes of heterodyning can be done. Real mode is indicated by negating the heterodyning frequency and complex mode is indicated by a positive value.

8.2.3 Reference Level [Required]

The reference level is another multiplicative scaling factor required to convert the raw data sample amplitudes to absolute units. As an example of how to calculate this value, for a 12 bit A/D system with maximum signal levels of +5 volts DC, the reference level would be: RLEVEL = 5 V / 2(12 bits – 1)

= 5 / 2048 = 0.00244140625 volts This is the quantization level, the level corresponding to the least significant bit of the digitized number. See Section 8.5 for an illustration of how this value is used.


This parameter is set according to the dynamic range of the acquisition system, a function of the number of output bits in the analogue to digital (A/D) conversion. For example, in a 12 bit A/D system the absolute value of the largest integer number representable is 2048 (i.e. 212-1), whereas in a 16 bit system the maximum magnitude is 32768 (i.e. 216-1). Note that this value is not necessarily the actual maximum value recoded in the data file, but the maximum possible value based on the equipment used. A value of –1.0 indicates that this value is unknown or was not used.


8.2.5 Gain Correction [Required]

If the gain values in the channel block (Section 8.4) are presented in dB then this must have a value of 1.0, but if the gains are powers of 2 then the gain correction must be 1.00343332. See Section 8.5 for an illustration of how this value is used.

8.2.6 [Unused]


8.2.7 [Unused]


8.2.8 [Unused]


8.3 Text Block

8.3.1 Reference Date [Required]

This 10 byte string contains a reference date used to assign dates to the time series data. The reference date is stored as a series of ASCII characters representing the day of the month, the month name, and the year, in a fixed format in that order. An example is: “□29-Dec-84”. The string contains:






8.3.2 Reference Time [Required]

This 10 byte string contains a reference time used to assign times to the time series data. The reference time is stored to the nearest second as a series of ASCII characters representing the hour, minute, and second, in a fixed format in that order. An example is: “□16:07:47φ”. The string contains:




Forty-four bytes are allocated to this value, which is entered as a string of ASCII characters starting at the first character. This parameter indicates the units of the calibrated data. As an example, if the units are to be volts, this information would be placed in the first six bytes of the variable as “voltsφ”. The null character (ASCII character 0) is always used to indicate the end of the units label.


Space is available for an optional user message that may assist in decoding or explaining data. When a message is included in the 320 bytes allocated for it, it should be terminated by the null character. The acquisition program usually inserts this message.

8.4 Channel Block The channel block uses 8 contiguous bytes for each channel in the file. This is split into one 2-byte unsigned short integer, one 2-byte signed integer, and one 4-byte signed long integer. The data placed in this area consists of a channel source code, its corresponding gain, and the offset of the start of the channel from the file’s reference date and time.


This is considerably different from the channel block layout for normal time series, as described in Section 5.4, and is best illustrated by an example. The following example lists the channel block data for the first 5 channels of a staggered time series data file (remembering that the three columns represent a short unsigned, a short signed, and a long signed integer): 1 -23 14524 2 19 -2322 3 20 0 1 -23 265542 4 12 -12134 Decoding this information shows the following:

• The first channel’s reference code is 1, its gain is –23, and it starts 14524 records after the header’s reference date and time. To convert a number of records to a time in seconds, divide the time offset in number of records by the data sampling frequency (Section 8.2.1).

• The second channel’s reference code is 2, its gain is 19, and it starts 2322 records before the reference date and time.

• The third channel’s code is 3, its gain is 20, and it starts at the reference date and time.

• The fourth channel’s code is 1 (meaning that it has the same source as the first channel), its gain is –23, and it starts 265542 records after the reference date and time.

• The fifth channel’s code is 4, its gain is 12, and it starts 12134 records before the reference date and time.

As can be seen from the example:

• more than one channel may have the same source, • channels’ data may start before, at, or after the reference date and time, and • channels with the same source (one and four in this example) would generally

be expected to have the same gain, but that may not always be the case. Should there be any space remaining in the channel block after the channel information is written, these remaining values should be given the value of zero. For example, if a file has five channels then 40 bytes would be used for the required data, leaving 472 bytes remaining. These bytes would consist of 59 groups of three integers (two short and one long), all of which would be given the value zero. The default header block size is 1024 bytes, of which the last 512 bytes are allocated for the channel block. At 8 bytes/channel, this allows for 64 channels. If there are more than 64 channels in the file (Section 8.1.7), then extra space is allocated for the header in blocks of 1024 bytes. Each additional 1024-byte block has space for information on 128 more channels. The following table indicates the header size as a function of the number of data channels in the file.




etc. etc.


OUT = IN × 2BSF × RLEVEL × GAINCOR

where: OUT - is the absolute level in absolute units (Section 8.3.3) IN - is a value read from the DREA .DAT32 file BSF - is the block scaling factor (Section 8.1.13) RLEVEL - is the reference level for the acquisition system (Section 8.2.3) GAINCOR - is the gain correction (Section 8.2.5) As an example, with:

• a DREA .DAT32 file value of 333.0 • a block scale value of –2 • the reference level of Section 8.2.3 • a gain correction of 1.00 • a stated Y axis units of “volts”

The conversion produces a voltage of: OUT = IN × 2BSF × RLEVEL × GAINCOR = 333.0 × 2-2 × 0.0024414 × 1.00 = 0.2032 volts


Bibliography

Caldwell, D.A. (1987). A Standard for the DREA Data Descriptor Block (DREA TC

87/302, January 1987) Harris, F.J. (1978). On the use of windows for harmonic analysis with the discrete

Fourier transform (Proc. IEEE 66:51, 1978) Kaiser, J.F, and Schafer, R.W. (1980). On the use of the I0-Sinh window for spectrum

analysis (IEEE Trans. ASSP 28:105, 1980) Nuttall, A.H. (1981). Some windows with very good sidelobe behaviour (IEEE Trans.

ASSP 29:84, 1981)


Initialisms and Acronyms

ASCII American Standard Code for Information Interchange DRDC Defence Research and Development Canada DREA Defence Research Establishment Atlantic IEEE Institute of Electrical and Electronics Engineers


Distribution list

Internal List

6 DRDC Atlantic LIBRARY FILE COPIES7 James A. Theriault1 Dave Hazen1 Sean Pecknold1 Gavin Hemphill1 Bruce Skinner1 Doug Caldwell1 John Bottomley1 Mario Boutin1 Brian Maranda1 Marcel LeFrancois1 Gordon Murray1 Bill Roger1 Nicole Collison1 Joe Maksym1 Dale Ellis____________________27 TOTAL LIST PART 1

LIST PART 2: DISTRIBUTED BY DRDKIM 2-2-5

1 NDHQ/DRDKIM2-2-5

1 Vance CroweNATO Undersea Research CenterViale San Bartolomeo 40019138 La SpeziaItaly

1 Joe HoodMacDonald, Dettwiler, and AssociatesSuite 60, 1000 Windmill RoadDartmouth, NS B3B 1L7

1 Gary InglisGD Canada629 Windmill Road,Dartmouth, NS B3B 1B7

1 G. HaralabusNATO Undersea Research CenterViale San Bartolomeo 40019138 La SpeziaItaly

___________________5 TOTAL LIST PART 2

32 TOTAL COPIES REQUIRED


DRDC Atlantic mod. May 02

DOCUMENT CONTROL DATA(Security classification of title, body of abstract and indexing annotation must be entered when the overall document is classified)

1. ORIGINATOR (the name and address of the organization preparing the document..Organizations for whom the document was prepared, e.g. Centre sponsoring acontractor's report, or tasking agency, are entered in section 8.)

xwave36 Solutions Drive, Halifax, NS B3S 1N2

2. SECURITY CLASSIFICATION !!(overall security classification of the document including special warning terms if applicable).

UNCLASSIFIED

3. TITLE (the complete document title as indicated on the title page. Its classification should be indicated by the appropriate abbreviation (S,C,R or U) in parentheses after the title).

Format Specification for DREA .DAT32 Files - Version 1.0.a

4. AUTHORS (Last name, first name, middle initial. If military, show rank, e.g. Doe, Maj. John E.)

Colin Calnan

5. DATE OF PUBLICATION (month and year of publication ofdocument)

April 2004

6a. NO. OF PAGES (totalcontaining information IncludeAnnexes, Appendices, etc).68 (approx.)

6b. NO. OF REFS (total citedin document)

4

7. DESCRIPTIVE NOTES (the category of the document, e.g. technical report, technical note or memorandum. If appropriate, enter thetype of report, e.g. interim, progress, summary, annual or final. Give the inclusive dates when a specific reporting period is covered).

CONTRACT REPORT 8. SPONSORING ACTIVITY (the name of the department project office or laboratory sponsoring the research and development. Include address).

Defence R&D Canada – AtlanticPO Box 1012Dartmouth, NS, Canada B2Y 3Z7

9a. PROJECT OR GRANT NO. (if appropriate, the applicable researchand development project or grant number under which the documentwas written. Please specify whether project or grant).

11cm01

9b. CONTRACT NO. (if appropriate, the applicable number underwhich the document was written).

W7707-03-2425 10a ORIGINATOR'S DOCUMENT NUMBER (the official document

number by which the document is identified by the originatingactivity. This number must be unique to this document.)


10b OTHER DOCUMENT NOs. (Any other numbers which may beassigned this document either by the originator or by thesponsor.)

11. DOCUMENT AVAILABILITY (any limitations on further dissemination of the document, other than those imposedby security classification)( x ) Unlimited distribution( ) Defence departments and defence contractors; further distribution only as approved( ) Defence departments and Canadian defence contractors; further distribution only as approved( ) Government departments and agencies; further distribution only as approved( ) Defence departments; further distribution only as approved( ) Other (please specify):

12. DOCUMENT ANNOUNCEMENT (any limitation to the bibliographic announcement of this document. This will normally correspond to theDocument Availability (11). However, where further distribution (beyond the audience specified in (11) is possible, a wider announcementaudience may be selected).

DRDC Atlantic mod. May 02

13. ABSTRACT (a brief and factual summary of the document. It may also appear elsewhere in the body of the document itself. Itis highly desirable that the abstract of classified documents be unclassified. Each paragraph of the abstract shall begin with anindication of the security classification of the information in the paragraph (unless the document itself is unclassified) representedas (S), (C), (R), or (U). It is not necessary to include here abstracts in both official languages unless the text is bilingual).

An earlier report [Caldwell, 1987] described the specifications for a binary file format inuse at DREA at the time. That report described a format that consisted of a file containinga header, with information on the file and its contents, that was followed by the file’s data.Eventually this format, familiarly know as the “DREA .DAT” format, proved to beinsufficient to contain data recorded by the evolving nature of the instrumentation and therequirements of researchers. That format became enhanced, becoming the higherresolution “DREA .DAT32” format.

Over time programs were written to produce data in DREA .DAT32 format, as well as toread and use the contents of these files. However no analogue to Caldwell’s report wasever produced, which meant that user’s were essentially free to do what they wanted withthe format, within limits.

This report sets down the requirements and specifications of the DREA .DAT32 formatbased on data gathered from programs that write and read the files, as well as fromusers of DREA .DAT32 files and the programs that use them.

14. KEYWORDS, DESCRIPTORS or IDENTIFIERS (technically meaningful terms or short phrases that characterize adocument and could be helpful in cataloguing the document. They should be selected so that no security classification isrequired. Identifiers, such as equipment model designation, trade name, military project code name, geographic location mayalso be included. If possible keywords should be selected from a published thesaurus. e.g. Thesaurus of Engineering andScientific Terms (TEST) and that thesaurus-identified. If it not possible to select indexing terms which are Unclassified, theclassification of each should be indicated as with the title).

AcousticData formatUnderwater AcousticsDat32

Format Specification for DREA .DAT32 Files - Version 1.0 · 2012. 8. 3. · Format Specification...

Documents

Transcript of Format Specification for DREA .DAT32 Files - Version 1.0 · 2012. 8. 3. · Format Specification...