(BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS...

Post on 11-Jul-2015

258 views 2 download

Tags:

Transcript of (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS...

Archive:

Data retained for the

long term – for

compliance or potential

future reference

Backup:

Data retained to

support near-term

business continuity

Archive:

Data retained for the

long term – for

compliance or potential

future reference

Archive:

Data retained for the

long term – for

compliance or potential

future reference

What is archival?: Traditional archival approaches

Metered usage:

Pay as you go

No capital investment

No commitment

No risky capacity planning

Avoid opex and risks of

physical media

handling

Control your

geographic locality for

performance and

compliance

Amazon

Glacier

Application

HTTP(S) /

REST APIs

Data collection Local index

Compression

Raw data

Amazon

Glacier

Lifecycle policies

Amazon

S3

HTTP(S) /

REST APIs

Raw data

Create vault (films)

1

Configure access policies

2

ArchiveApp user policy

Effect:Allow

Resource:

arn:aws:glacier:<accountId>:vaults/Films

Action: glacier:UploadArchive

3

Upload archivesUploadArchive(data) ->

Archive ID

Initiate job

ArchiveId: AE99F…

Vault: Films -> Job ID

1

3-5 hours for job completion

2

3

Job completion

notification

4

Download output

1. Primary storage hardware (primary / remote site)

2. DR / remote site storage hardware

3. Raw to utilized storage (both primary and DR)

4. Storage growth (cost of upgrades)

5. Storage management software and 3rd party tools

6. Professional services

7. Hardware maintenance

8. Software maintenance

9. Backup software

10.Backup hardware (primary / remote site)

11.Offsite tape storage / vault

12.Archive software

13.Archive hardware

14.Power

15.Cooling

16.Space

17.Labor

18.Cost of capital

19.Training

20.Asset depreciation

21.Migration

22.Decommission / remove

23.Recycle

Checksum 2

Checksum 1

File 2

Checksum 3

. . .

Local index

File 1 offset

File 1

File 2 offset

File 3 offset

Index/directory

Checksum & metadata

Checksum & metadata

Checksum & metadata

Archive

AWS Direct

ConnectDedicated bandwidth between

your site and AWS

InternetTransfer data in a secure SSL tunnel over

the public Internet

AWS Import/ExportPhysical transfer of media into and

out of AWS

Arc

hiv

e

Parallel Uploads

Parts

Media Cloud ServicesOn-demand cloud-based solutions designed to capture,

collect, create, manage, share, produce, and archive high-

value, high-resolution content.

What is Sony MCS?

Your professional, all-in-one media production

workspace.Instantly connect your team to your media files, hardware,

and applications for streamlined communication, faster

turnaround times, and reduced production costs.

Ci

TELEVISION EDITORIAL & LEGAL REVIEW

MAJOR MOTION PICTURE DAILIES PREVIEWING

SMALL BUDGET PRODUCTIONS & ORIGINAL CONTENT

SHARE CONTENT BETWEEN STATIONS

ARCHIVECONTENT

STOCK FOOTAGE STORAGE & OPERATIONS

THOUSANDS OF ASSETS

DISTRIBUTED USERS

TRACKING/ MONITORING

Same challengeCREATIVE DRIVERS BUSINESS DRIVERS

Better Image QualityHigh dynamic range

High resolution

More Raw FootageHigher shooting ratios

High frame rate

3D / 4K

More ContentGreater diversity

More collateral marketing

Greater Distribution NeedsMulti-screen expectation

Physical, satellite, Internet

0

2

4

6

8

10

12

14

FY13 FY14 FY15 FY16 FY17 FY18

Storage (PB)

Storage (PB)

Growth projections

Cost comparison

$-

$1,000,000.00

$2,000,000.00

$3,000,000.00

$4,000,000.00

$5,000,000.00

$6,000,000.00

$7,000,000.00

FY13 FY14 FY15 FY16 FY17 FY18

On Prem

Ci/AWS

Transfer to

S3

Copy to

archive

bucket

Ingest

pipeline

Register in

Ci

Worker

updates Ci

status

Custom

app calls

Ci API

Ci calls S3

API

Worker

monitors

Ci status

updated

Update Ci

status

Please give us your feedback on this session.

Complete session evaluations and earn re:Invent swag.

http://bit.ly/awsevals