10thNovember 2009

High-definition video recording in defenceand aerospace applicationsAndrew Haylett

2

Outline

• introduction and platforms• technology review – analogue and digital• ‘hard’ versus ‘soft’ recording• video interfaces and metadata• raw video bit rates and recording time• video compression technologies• storage media• recording file formats• the replay/debrief facility• digital video distribution

3

Introduction – the need

• acquisition and storage of video signals a key feature of modern defence applications

• new sensor/camera technology brings higher resolution images

• need to capture complex displays as well as multiple sensors and complex metadata

• application areas include training, debrief and forensics

• solution must be rugged, cost-effective and scaleable, minimizing size, weight and power

4

Typical platforms

• US air: Joint Strike Fighter, P-8A Poseidon

• UK air: Nimrod MRA4• US land: Bradley Fighting Vehicle,

Abrams Tank, Stryker family• UK land: Warrior armoured vehicle• Many platforms undergoing

modernization and capability enhancement

5

Analogue video recording

• mechanical ruggedization a challenge• limitations on media capacity and recording time• limited video resolution and quality• difficult to make acceptable copies• vulnerable to electrical interference• access is sequential rather than random• cannot easily scale to multiple video streams• cannot achieve comprehensive data fusion

6

‘Soft’ digital recording

‘soft’ digital recording system: uses graphics card to capture framestorewith simple software compression

7

‘Hard’ digital recording

uses custom hardware tocapture, compress and record

multiple video streams

8

Soft recording

• No extra hardware required

• Loads processor/graphics system

• Localized; not readily scalable across multiple sources

• Focuses on display video capture

Hard recording

• Hardware acquisition and compression

• Independent of rest of system

• Flexible scalable architecture includes entire installation

• Captures sensors, displays, metadata

‘Soft’ vs. ‘hard’ digital recording

9

Video interfaces

Analogue• Composite/RGB TV (NTSC/PAL), including

STANAG 3350• High-resolution analogue RGB – VGA to WUXGA

(1920 x 1200)

Serial digital• DVI/HDMI• Firewire (IEEE 1394), CameraLink, USB• SD-SDI, HD-SDI (SMPTE 292M)• GigE Vision – video over GbE

10

Metadata and audio

Need to capture events or data streams from multiple sources

• Operator-initiated signalling of events• Navigational data (GPS)• Time source (IRIG-B)• MIL-STD-1553/1773, MilCAN, ARINC 429, GbE• Data streams should be reconstituted or stored at

debrief station• Recordings should be searchable by metadata• Support multiple audio channels

11

Raw video bit rates

TV

HD

WSXGA

250Mbps110GB/hour

1400Mbps615GB/hour

2400Mbps1TB/hour

GbE = 1000Mbps

12

Raw video recording time

TV

HD

WSXGA

140 minutes

25 minutes

15 minutes

recording times assuming use of256GB memory cartridge

13

Video compression technologies – JPEG

• JPEG (ISO/IEC IS 10918-1), M-JPEGWidely used, performs well for photographic images, suffers from artefacts at high compression

• JPEG2000 (ISO/IEC 15444), Motion JPEG2000Improved compression and reduced artefacts, enhanced feature set, adopted by DoD NITF for highest quality storage

14

Video compression technologies – MPEG

• MPEG-2 (ISO/IEC 13818)Widely used in consumer applications including broadcast TV and DVD

• MPEG-4 (ISO/IEC 14496)Adds extra coding complexity to deliver improved compression

• MPEG-4 Part 10 Advanced Video Coding (ITU H.264)Used on high-definition DVD; current standard for low bitrate encoding

15

Frame-by-frame vs. inter-frame encoding

Intra-frame coding – uses only spatial redundancy

Inter-frame coding – uses spatial and temporal redundancy

I P P P PI

16

M-JPEG2000 vs. MPEG-4 AVC

JPEG2000• symmetric: encoding and

decoding computationally demanding

• intra-frame coding only• optimum for very high-

resolution images and synthetics

• decimation supported

• visually or mathematically lossless

MPEG-4 AVC• asymmetric: decoding

less computationally demanding

• inter- / intra-frame coding• optimum for medium to

high-resolution images

• decimation non-optimal• visually lossless only

JPEG-2000 vs. MPEG-4 AVC

17

Rugged implementations

HD JPEG2000 AC XMC card

TVJPEG2000 CC PMC card

18

Compressed video bit rates

TV 12Mbps5.5GB/hour

HD70Mbps

30GB/hour

WSXGA120Mbps

50GB/hour

GbE = 1000Mbps

TV 250Mbps110GB/hour

HD1400Mbps

615GB/hour

WSXGA2400Mbps1TB/hour

assuming 20:1 compression ratio

19

Compressed video recording time

46 hoursTV

HD 8 hours

WSXGA 5 hours

TV 140 minutes

HD 25 minutes

WSXGA 15 minutes

recording times assuming use of256GB memory cartridge

20

Storage media – requirements

Requirements for military / aerospace applications

• mechanically robust• withstand extreme environments: shock,

vibration, temperature, humidity• high reliability and long life• maximum storage capacity• easily transportable• security of recorded data

21

Storage media – alternatives

Conventional magnetic disks• single units up to 2TB capacity• need careful system design to protect from

environment• not ideal for transport between locations

Solid state drives• ruggedisable, transportable, reliable, low mass• available as PCMCIA, CompactFlash,

PCIexpress, FiberChannel array up to 5TB• single rugged unit up to 512GB capacity

22

PCIexpress storage

Solid-state media

256GB removable cartridge

512GB VPX3 module

CompactFlash module

23

Fixed or removable?

Fixed• Potentially greater capacity available• Easier to design for rugged environment• Streaming off recorder time-consuming• Problems of security if sensitive data left on

platform

Removable• Can be swapped during operation if necessary• Convenient to transfer data to debrief station

24

Recording file formats

• Common formats include AVI (Windows standard), MPG (MPEG-2/4) – support for video and audio

• Ideal format will encompass video, audio and metadata

• Open file formats support any video encoding standard; e.g. the Matroska MKV format is codec-neutral and allows arbitrary metadata attachments to recorded files

• Key design elements are random access with rapid search, jump to event/time

25

Open container file

Header Video 1 Video 2 Video 1 Video 2

Audio1 Audio2 Video 1 Video 2 Metadata Metadata

Video 1 Video 2 Video 1 Audio1 Audio2

26

Replay / debrief facility

• Typically based on COTS equipment, e.g. desktop PC

• May use software decoder or hardware accelerator depending on compression asymmetry and graphics card capability

• Will provide scaled multi-window presentation with jump to arbitrary time and metadata search

• Will accept removable media from recording system and optionally support archival to long-term media (e.g. Blu-Ray)

27

Digital video distribution

• Video recording and video distribution closely related• Video streams transferred to digital domain may be

easily sent point-to-point or broadcast over standard network infrastructure

• Video over IP is readily scalable to emerging technologies such as 10GbE

• Video recorder becomes node on digital video distribution network

• Standard video distribution protocols such as RTP provide quality of service and encapsulation of various compression formats

28

Example architecture

codervideo

networkswitch

GbE

recorder

GbE

GbE

decoder/display

GPS IRIG-B

29

Conclusions

Digital video recording offers:• Environmentally robust solution with emphasis on

cost, size, weight, power• Enhanced recording time and video quality• Advanced features such as play-while-recording,

record only last N hours of mission• Integrates sensor video, display video, audio,

events and metadata into single stream• Scalable to high sensor density• Part of comprehensive video distribution system

30

Any questions?

Andrew Haylettandrew.haylett@curtisswright.com

01462 472537