Implementation Review1 Deriving Architecture Requirements March 14, 2003.

Implementation Review 1

Deriving Architecture Requirements

March 14, 2003


Architectures Homogeneous Heterogeneous

Centralized computing Distributed computing Centralized storage Locally attached disks

One system with many CPUs

Many systems with one CPU

Dynamic server Monolithic mainframe

Unix Windows


Requirements

Architecture requirements determined by Reliability, Availability and

Serviceability (RAS) requirements, programmatic goals and constraints (budget, resources), software architecture

System requirements (CPU, memory, throughput, storage) determined by performance and data volume

requirements (covered in presentations after coffee break)

System requirements can be met by many Architectures


Overview

Recap requirements from Rodger’s presentation Derive the Architecture requirements Translate the Architecture requirements in to an

Architecture design


DADS Downtime Statistics

0.00%

2.00%

4.00%

6.00%

8.00%

10.00%

12.00%

14.00%

16.00%

18.00%

20.00%

May-02 Jun-02 Jul-02 Aug-02 Sep-02 Oct-02 Nov-02

% D

ow

nti

me

Goal: <5%

(1 day/week = 14%)

RAS:•Reliability •Availability •Serviceability

Recap


Recap

While current system runs on average! at ~50% capacity, it does not meet requirements because of system failures and inefficient use of resources

AND: Distribution volume and user demand will continue to grow 20%

per year, with jump this summer (ACS) Ingest volume will continue to grow (COS, WFC3, SM4) Calibration algorithms increasingly powerful Need to migrate from OpenVMS and Tru64 Need for Integration and Test (I&T) environment for realistic load

testing While HST budget at STScI declining


150 GB

493 GB

ZEPPO

CATLOG

ARCHC

76 GB

108 GB

1 TB

210 GB

305 GB

HST

FROM: ScienceCommunity(thru StarView)

FUSEEngineering Team

TO: ScienceCommunity

HST1

Tape DrivesCD Burners

Jukeboxes

Network AttachedStorageOMS

SDIP

ARCHB

STDATU

HALFMOON

To Test andDevelopmentEnvironment

ServerW DVDW DVDW DVDDVD

Drives

1 TB

ACDSDOPS Cluster

Server

Network AttachedStorage

Pipeline Processing IngestDistribution DB

Science Pipelines

Archive User

Interfaces

CCS Data ReceiptEngineering

DADS-OPUStransfer

ExportMedia

Current Operational Architecture

Archive Ingest

Archive Ingest DADS 9.*

Distribution

Science Pipelines

OTFR

Users Users

Archive CatalogBrowsing DB

HST

Pipeline Processing IngestDistribution DB


Nomad

Robbie

Scarab

Development & TestTeams

TO: InstituteCommunity

CD Burners

John (EDP-Dev)

1 TB

ODO Cluster

ToOperations

Jukebox

Barge

Corsair

Aardvark

Artichoke

Liner

Jukeboxes

Tape drive

Jboat

Canoe

Paul (IDR-Beta)

Ringo (EDP-Ops)

George (IDR-Dev)

Alpha/OpenVMS

Alpha/OpenVMS

Alpha/OpenVMS

150 GB

493 GB

ZEPPO

CATLOG

ARCHC

76 GB

108 GB

1 TB

210 GB

305 GB

HST




HST1


Jukeboxes


SDIP

ARCHB

STDATU

HALFMOON



Drives

1 TB

ACDSDOPS Cluster

Server



Nomad

Robbie

Scarab

Development & TestTeams

TO: InstituteCommunity

CD Burners

John (EDP-Dev)

1 TB

ODO Cluster

ToOperations

Jukebox

Barge

Corsair

Aardvark

Artichoke

Liner

Jukeboxes

Tape drive

Jboat

Canoe

Paul (IDR-Beta)

Ringo (EDP-Ops)

George (IDR-Dev)

Alpha/OpenVMS

Alpha/OpenVMS

Alpha/OpenVMS

150 GB

493 GB

ZEPPO

CATLOG

ARCHC

76 GB

108 GB

1 TB

210 GB

305 GB

HST




HST1


Jukeboxes


SDIP

ARCHB

STDATU

HALFMOON



Drives

1 TB

ACDSDOPS Cluster

Server


150 GB

493 GB

ZEPPO

CATLOG

ARCHC

76 GB

108 GB

1 TB

210 GB

305 GB

HST




HST1


Jukeboxes


SDIP

ARCHB

STDATU

HALFMOON



Drives

1 TB

ACDSDOPS Cluster

Server



Current Architecture – Issues

Complex: Three operating systems: OpenVMS, Tru64, Solaris Many systems, disks, cross-mounts, dependencies Hard to model, capture performance data, metrics

Fragile: Single points of failure, no redundancy HW used in higher-failure mode (jukeboxes) Failures are expensive because of complexity

Not scaleable: – growth will increase complexity Not flexible: – cannot handle peak loads Not testable: – does not support realistic I&T environment

Low RASExpensive


Architecture Requirements

Simple: One operating system Few systems, disks, cross-mounts, dependencies Easy to model, capture performance data, metrics

Robust: No single points of failure: redundancy HW use within specs Easy to recover

Scaleable: - grow without increasing complexity Flexible: - handle peak loads Testable: - provide I&T environment

High RASAffordable


Storage Requirements

Storage Solution must be high RAS: Fully redundant, mirrored Provide business continuity features:

Live backups Allow snapshot backups of databases

Easy to manage And:

Allow future growth within the box Allow flexible allocation of space Support I&T environment

Within budget, time constraints

Enterprise-class Storage EMC SAN


Computing requirements

Computing solution must be high RAS Fully redundant Failure isolation Easy to manage

And: Easy to model, capture performance data, metrics Allow growth within the box Allow flexible allocation of resources Provide an I&T environment

Within budget, time constraints

Enterprise-class Server SunFire 15K


Computing requirements - cont.

Avoid locking in Operating System versions on Server by vendor specific drivers: jukeboxes, CD/DVD/Tape writers Do not connect peripherals directly to Server but only

to stand-alone, dedicated systems Isolate the server from external users

Do not host web-interfaces (Archive, MAST) or application interfaces (StarView) directly on Server but only on stand-alone dedicated systems


Architecture design:

Implementation Review 16New Architecture

SUN FIRE15K

SUN FIRE 15K Domain Config

Opus/Archive


7 Dynamically Re-Configurable Domains

EMC

EMC

EMC

EMC

EMC

EMC

OPUS/Archive OPS

EMC

Databases OPS

Databases OPS

Code Development

System Test

Database Test

OS/Security Test


Architecture is more than sum of the parts: Optimal use of the EMC SAN requires a consolidated

computing environment It is the combination of the Computing and Storage

solutions that ensures the high RAS architecture The high RAS Data Processing System is sum of:

High RAS Architecture High RAS Software Systems (other presentations) High RAS use models (other presentations)

Architecture summary:


Concluding remarks

Showed how Rodger’s requirements and goals translated in to high RAS requirements, which determined the new Architecture design.

Next presentations go into system requirements: SunFire 15K Enterprise-Grade server EMC Centralized Storage

Implementation Review1 Deriving Architecture Requirements March 14, 2003.

Documents

Transcript of Implementation Review1 Deriving Architecture Requirements March 14, 2003.