What's Next with Government Big Data

Post on 18-Nov-2014

917 views 1 download

description

Big Data continues to be a hot topic in government. Now it's time to take the discussion to the next level. Most agencies understand Big Data and are collecting large amounts of data, but the challenge of how to manage it still remains.

Transcript of What's Next with Government Big Data

Government Big Data:What’s Next

March 21, 2013Brought to you by:

Today’s SpeakersSteve ResslerFounder and PresidentGovLoop

Marina MartinEntrepreneur-in-Residence & Head of the Education Data InitiativeU.S. Department of Education

Gary NewgaardDirector of Federal SolutionsEMC Isilon

Shawn KingsberryCIO Recovery Accountability and Transparency Board

Housekeeping

o Twitter Hash Tag: #gltrain

o If you would like to submit a question, just look for the "Ask a question" console. The presenters will field your questions at the end.

o If you have any technical difficulties during the training click on the Help button located below the slide window.

o We will be e-mailing you a link to the archived version of this training, so you can view it again or share it with a colleague, and a GovLoop training certificate.

February 22, 2013

On Premises or Hosted Service

Public Transparency Website

Fraud Analytics as a Service

Big Data as a Service

Public Transparency Website

Fraud Analytics as a Service

Big Data as a Service

RATB Cloud ServicesHigh Level Technical BriefingRATB Cloud ServicesHigh Level Technical Briefing

RATB

CLO

UD

SER

VICE

RATB

CLO

UD

SER

VICE

LOG

ICAL

ARC

HIT

ECTU

RELO

GIC

AL A

RCH

ITEC

TURE

RATB Logical System Diagram

Logical RATB System Design Capabilities• Public and Private Cloud providing separate

and distinct websites running off of a common software, system, and data warehouse infrastructure

• Elasticity to support millions of concurrent users

• Content and Design team to support layout and design requirements

• Secured access to sensitive data providing virtual desktop as a service.

• Data automation providing scheduled retrieval of required data sets.

• Risk framework providing streamlined matching against risk databases.

• Link analysis systems and highly skilled analysts.

• Partners with key industry companies providing rapid development level integration services.

RATB High Level Technologies

19

Social Media

Web Infrastructure

Visualization, Analysis, and Reporting

Data Layer

Infrastructure

Disclaimer of Endorsement:Reference herein to any specific commercial products, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government, and shall not be used for advertising or product endorsement purposes.

Note: There are extensive products in this “infrastructure layer” These are the key components. A more comprehensive list can be made available by request.

Recovery Accountability and Transparency Board Enterprise Architecture of the Future

20

Data Governance

21

Advanced Analytics CloudWhat is FederalAccountability.gov

• The portal allows Federal agencies and Inspectors General the ability to review and evaluate the risk assessment of entities, companies, and universities receiving Federal Funds.

22

HIGHLIGHTS• Deployed Security: FIPS 140-2• Infrastructure: Secured Private Cloud

U.S. Department of Defense U.S. Environmental Protection Agency, OIG

U.S. Department of Education, OIG U.S. Department of Justice, OIG / Civil Division

U.S. Department of Homeland Security, OIG

U.S. Army

National Science Foundation, OIG U.S. Social Security Administration, OIG

U.S. Department of Agriculture U.S. Census Bureau

Corporation for National and Community Service OIG

U.S. Department of Commerce, OIG

U.S. Department of the Interior U.S. Department of Labor

U.S. Department of Health and Human Services

U.S. Department Housing and Urban Development, OIG

Executive Office for the US States and Attorney

RATB Cloud Service Customers

Advanced Analytics CloudDesktop and Analytics As A Service

23

Structured and Unstructured Data ETL

uRevealESRI

ARC GIS Server

OracleENDECA FastAlert

Analysts, Investigators

Palantir AccountabilityScorecard

uRevealESRI

ArcGIS Server

ENDECASTORE

FastAlert SQLServer

2008

Palantir Persistence

Engine

ScoreCard SQLServer

2008

HANA In-Memory

Computing

Single Sign-On Identity and Access ManagementSecurity Layer (Netwitness, Archer, Juniper SSL VPN…)

VMWareView VDI

Stakeholders Request For Assistance

Cloud Hub Categorization

PEOPLE PROCESS TECHNOLOGY

MO

DU

LAR

LAYE

RSSIN

GLE PAN

E OF G

LASS

Note: The specific details behind the RATB Cloud Hub Categorization can be provided by request.

RATB Cloud Service WebsitesRecovery.gov

RATB Cloud Service WebsitesEducationjobsfund.gov

RATB Cloud Service WebsitesFederaltransparency.gov

RATB Cloud Service WebsitesFederalaccountability.gov

March 19, 2013

On Premises or Hosted ServiceShawn Kingsberry, Chief Information Officershawn.kingsberry@ratb.gov

30© Copyright 2012 EMC Corporation. All rights reserved.

EMC ISILON SCALE-OUT NASBig Data Storagefor the Federal Sector

Gary NewgaardDirector - EMC Isilon Federal

31© Copyright 2012 EMC Corporation. All rights reserved.

Isilon Technology

Summary

Big Data Overview

32© Copyright 2012 EMC Corporation. All rights reserved.

What Is Big Data?

Data that challenges the capabilities of a system to capture, manage, and process it within an acceptable

elapsed time~ Wikipedia ~

Data that challenges the capabilities of a system to capture, manage, and process it within an acceptable

elapsed time~ Wikipedia ~

33© Copyright 2012 EMC Corporation. All rights reserved.

Exabytes

The Big Data Challenge

By 2013, 80% of all storage capacity sold will be for file-based dataSource: “Scale Out Storage in the Content Driven Enterprise: Unleashing the Value of Information Assets,” IDC White Paper (2010 Enterprise Disk Storage Consumption Model), June 2011

File based: 61.8% CAGR Block based: 23.7% CAGR

Media & Entertainmen

t

Media & Entertainmen

t

Design & SimulationDesign & Simulation

Financial ServicesFinancial Services

Bioinformatics

Bioinformatics Oil & GasOil & GasFile Shares

& ArchivesFile Shares & Archives

34© Copyright 2012 EMC Corporation. All rights reserved.

Data growth

47%

CIOs Turning to Scale-Out to Deal with Massive File-Data Growth

84%Already using or planning to use scale-out

within the next 24 months

The#1 and#2 concerns of CIOs…

Source: Scale-out NAS Market Survey

System performance and scalability

37%

…are driving adoption

of scale-out

Source: “User Survey Analysis: Key Trends Shaping the Future of Data Center Infrastructure Through 2011,” Gartner, October 2010

35© Copyright 2012 EMC Corporation. All rights reserved.

Big Data Apps Need Big Data StorageData intensive, HPC workflows

Medical Imaging Gene Sequencing Seismic Exploration

Media & Entertainment

Product DevelopmentSatellite Images

36© Copyright 2012 EMC Corporation. All rights reserved.

Big Data Project in the Federal Sector*

*Sorce: GovWin IQ Report 2012

37© Copyright 2012 EMC Corporation. All rights reserved.

Examples of Federal Sector Big Data

• Healthcare• Life Sciences• Surveillance• Physical Security• Defense/Intelligence• Cyber

38© Copyright 2012 EMC Corporation. All rights reserved.

Sample of Government Accounts Using Isilon’s Unified Scale-out Storage…and Why?

Unlocks New Capabilities

Increases IT operating leverage

Complementary

Reduces storage costs

Speeds workflows

39© Copyright 2012 EMC Corporation. All rights reserved.

EMC Isilon Growing MomentumHealthcare and Life Sciences

UCLA

40© Copyright 2012 EMC Corporation. All rights reserved.

The EMC Isilon Difference

EMC Isilon Value Proposition for Healthcare– Eliminate silos of storage– Predictable scalability without added complexity– Consolidate active short-term and long-term data

Certification with Major Alliance Players – EMC Isilon certified with most major PACS vendors– EMC Isilon certified with most VNAs– New certifications are simple

EMC Isilon works with next wave diagnostic tools– Digital Pathology, NGS, Proteomics– Video Surveillance, Sleep Studies, Electron Microscopes

EMC Isilon Competitive Difference– “Never Migrate Again” architecture– Move away from API-built storage– Uses standard CIFS and NFS storage protocol connections– Experience the value of scale-out NAS

41© Copyright 2012 EMC Corporation. All rights reserved.

“Never Refresh Again” Architecture Meet Your Big Data Requirements with EMC Isilon

• One File System, One Volume Storage Management Simplicity

• Zero Downtime Expansion

• Greater than 80% utilization rates

• Adapt your existing storage resources

• Accommodate IT infrastructure changes

• Investment Protection: Pay As You Grow

• Eliminate Silos and Hot Spots

42© Copyright 2012 EMC Corporation. All rights reserved.

The Cost Advantage of IsilonEase of use and management simplicity

IDC: Isilon improves IT productivity by 48%, reduces OPEX*

Storage allocation

Storage provisioning

Managing capacity

Managing backup

Space reclamation

Adding new applications

Uploading of re-loading data

0.0 0.5 1.0 1.5 2.0

FTE Hours per TB in Use

Isilon

Traditional

* Source: “Quantifying the Business Benefits of Scale-Out NAS Solutions,” IDC White Paper, November 2011

43© Copyright 2012 EMC Corporation. All rights reserved.

Reduces Big Data storage costs by 40%

The Cost Advantage of Isilon

Source: “Quantifying the Business Benefits of Scale-Out NAS Solutions,” IDC White Paper, November 2011

44© Copyright 2012 EMC Corporation. All rights reserved.

Isilon Scale-Out NAS Architecture

OneFS Operating Environment

OneFS Operating Environment

Intra-cluster Communication Layer

Intra-cluster Communication Layer

Servers

Client/Application Layer

Client/Application Layer Ethernet LayerEthernet Layer

Servers

Servers

CIFSCIFSNFSNFS

FTPFTPHTTPHTTP

HDFSfor

Hadoop

HDFSfor

Hadoop

45© Copyright 2012 EMC Corporation. All rights reserved.

More scalable than traditional storage systems

Largest and Most Scalable File System

OneFS scales from 18 TB to more than 20 PB in a single file system, single volume

Under 60 seconds to scale with no downtime

World’s fastestperformance andcapacity scaling

Over 100 GB/s of throughput

46© Copyright 2012 EMC Corporation. All rights reserved.

Markets and SolutionsEMC

Isilon Federal Markets

Home Directories & Archive

Questions?