1©Cloudera,Inc.Allrightsreserved.

FredKoopmansSr.DirectorofProductManagement

EnablingSecureHadoopEnvironments

1©Cloudera,Inc.Allrightsreserved.

2©Cloudera,Inc.Allrightsreserved.©Cloudera,Inc.Allrightsreserved.

Thefutureofgovernmentisdatamanagement

What’syourstrategy?

3©Cloudera,Inc.Allrightsreserved.

Cloudera’sEnterpriseDataHubmakesitpossible

Bringallyourdatatogether

Bringallyourknowledgeworkerstogether

Bringallyourdataapplicationstogether

Runanywhere

4©Cloudera,Inc.Allrightsreserved.

But,anEDHcanalsomakeajuicytarget

Alldatainoneplace?

Provideeveryoneaccesstooneplatform?

HowdoIensuresecurity?

HowdoImaintainsecurityastheplatformgrows?

5©Cloudera,Inc.Allrightsreserved.

4focusareasforsecuringyourHadoopenvironment

AccessDefiningwhatusersandapplicationscan

dowithdata

TechnicalConcepts:PermissionsAuthorization

DataProtectingdatainthe

clusterfromunauthorizedvisibility

TechnicalConcepts:Encryption,Keymanagement,Datamasking

VisibilityReportingonwheredatacamefromandhowit’sbeingused

TechnicalConcepts:AuditingLineage

PerimeterGuardingaccesstothe

clusteritself

TechnicalConcepts:Authentication

Networkisolation

6©Cloudera,Inc.Allrightsreserved.

PerimeterSecurityRequirements

PreserveuserchoiceofHadoopservice

Conformtocentrallymanagedauthenticationpolicies

Implementwithexistingstandardsystems

ClouderaManager

PerimeterGuardingaccesstothe

clusteritself

TechnicalConcepts:Authentication

Networkisolation

7©Cloudera,Inc.Allrightsreserved.

Authentication

• Kerberosauthentication• AutomationprovidedbyClouderaManagertoleverageActiveDirectory

CDHcomponents

• LDAPandSAMLauthentication

WebUIs

• LDAPandKerberosauthentication

SQLAccess

Userauthenticates

toAD

Authenticatedusergets

KerberosTicket

TicketgrantsaccesstoServices

e.g.ImpalaUser[ssmith]Password[*****]

8©Cloudera,Inc.Allrightsreserved.

NetworkIsolation

EdgeNodes

WebServers

Onlyadmins permittedaccesstofullcluster

Mostusersonlypermittedaccesstogatewayservicesrunningonclusterperiphery

9©Cloudera,Inc.Allrightsreserved.

AccessSecurityRequirements

Keeponlyonelogicalcopyofdata

Createonlyonepermissionsruleforallapplicationsandallcomputeframeworks

Enforcepermissionsatcolumnandrowlevelgranularity

AccessDefiningwhatusersandapplicationscan

dowithdata

InfoSecConcept:Authorization

ApacheSentry,RecordService

10©Cloudera,Inc.Allrightsreserved.

FilesystemHDFS

STORA

GECO

MPU

TE

HIVE,IMPALA

SPARK,MR

EarlydaysofHadoop:Storagepermissionsonly

• Simple“AllorNothing”permissionsforeachfile/table

But...• Tablesoftencontain10s– 100sofcolumns• Notallusersareallowedtoseeallcolumnsandrows

APPS DATAMEER SASPLATFORA TABLEAU ETC...

11©Cloudera,Inc.Allrightsreserved.

Usecasesforfine-grainedaccesscontrol

Columns• Differentusergroupsneedaccesstodifferentcolumns(ex:socialsecuritynumbers)

Rows• Differentusergroupsneedaccesstodifferentrecords(ex:bysecurityclearancelevel)

12©Cloudera,Inc.Allrightsreserved.

FilesystemHDFS

STORA

GECO

MPU

TE

HIVE,IMPALA

SPARK,MR

Fewyearsago:Storagepermissions+SQLAuth.

• Addscolumnandrow-levelpermissions

But...• Createsduplicatedata,duplicatepermissionsrulestosupportSparkandMR

APPS DATAMEER SASPLATFORA TABLEAU ETC...

ApacheSentry

X

13©Cloudera,Inc.Allrightsreserved.

FilesystemHDFS

STORA

GE

APACHESENTRY,RECORDSERVICE

COMPU

TE

HIVE,IMPALA

SPARK,MR

UpNext:ApacheSentry+RecordService*workingtogether

• ColumnandRow-levelPermissions• Onecopyofdata• Onesetofpermissions

APPS DATAMEER SASPLATFORA TABLEAU ETC...

*inbeta

14©Cloudera,Inc.Allrightsreserved.

Fine-grainedaccesscontrolwithout Sentry&RecordService*

Date/time Accnt # SSN Asset Trade Country

09:33:11 16-Feb-2015

0234837823 238-23-9876

AZP Sell US

11:33:0116-Feb-2015

3947848494 329-44-9847

TBT Buy EU

14:12:3416-Feb-2015

4848367383 123-56-2345

IDI Sell UK

09:22:03 16-Feb-2015

3485739384 585-11-2345

ICBD Buy US

11:55:3316-Feb-2015

3847598390 234-11-8765

FWQ Buy US

10:22:55 16-Feb-2015

8765432176 344-22-9876

UAD Buy UK

13:45:24 16-Feb-2015

3456789012 412-22-8765

NZMA Sell EU

09:03:4416-Feb-2015

4857389329 123-44-5678

TMV Buy US

15:55:5516-Feb-2015

4756983234 234-76-9274

DRW Buy UK

Date/time Accnt # SSN Asset Trade Country

14:12:3416-Feb-2015

4848367383 123-56-2345

IDI Sell UK

10:22:55 16-Feb-2015

8765432176 344-22-9876

UAD Buy UK

15:55:5516-Feb-2015

4756983234 234-76-9274

DRW Buy UK

Date/time Accnt # SSN Asset Trade Country

11:33:0116-Feb-2015

3947848494 329-44-9847

TBT Buy EU

13:45:24 16-Feb-2015

3456789012 412-22-8765

NZMA Sell EU

Date/time Accnt # SSN Asset Trade Country

09:33:11 16-Feb-2015

0234837823 238-23-9876

AZP Sell US

09:22:03 16-Feb-2015

3485739384 585-11-2345

ICBD Buy US

11:55:3316-Feb-2015

3847598390 234-11-8765

FWQ Buy US

09:03:4416-Feb-2015

4857389329 123-44-5678

TMV Buy US

• SplittheHDFSpermissionsoriginalfile• Usetolimitaccess

15©Cloudera,Inc.Allrightsreserved.

Fine-grainedaccesscontrolwith Sentry&RecordService*

• Sentry:Definepermissionsatthetable,columnandrowlevels• Sentry+RecordService:Enforcetheseacrossallaccesspaths

Date/time Accnt # SSN Asset Trade Country

09:33:11 16-Feb-2015

0234837823 238-23-9876

AZP Sell US

11:33:0116-Feb-2015

3947848494 329-44-9847

TBT Buy EU

14:12:3416-Feb-2015

4848367383 123-56-2345

IDI Sell EU

09:22:03 16-Feb-2015

3485739384 585-11-2345

ICBD Buy US

11:55:3316-Feb-2015

3847598390 234-11-8765

FWQ Buy US

10:22:55 16-Feb-2015

8765432176 344-22-9876

UAD Buy EU

13:45:24 16-Feb-2015

3456789012 412-22-8765

NZMA Sell EU

Column-LevelControls

Row-LevelCon

trols

Date/time Accnt # SSN Asset Trade Country

09:33:11 16-Feb-2015

0234837823 238-23-9876

AZP Sell US

11:33:0116-Feb-2015

3947848494 329-44-9847

TBT Buy group2

14:12:3416-Feb-2015

4848367383 123-56-2345

IBM Sell group3

09:22:03 16-Feb-2015

3485739384 585-11-2345

ICBD Buy US

11:55:3316-Feb-2015

3847598390 234-11-8765

FWQ Buy US

10:22:55 16-Feb-2015

8765432176 344-22-9876

UA Buy group3

13:45:24 16-Feb-2015

3456789012 412-22-8765

AMZN Sell group2

Column-LevelControls

Row-LevelCon

trols

XXX-XX

XXX-XX

XXX-XX

WhatU.S.BrokersSee

Hive,Impala,MR,Spark,Pig

SingleHDFSfile:

16©Cloudera,Inc.Allrightsreserved.

VisibilitySecurityRequirements

Complywithpoliciesforaudit,dataclassification,andlineage

Centralizetheauditrepository

VisibilityReportingonwheredatacamefromandhowit’sbeingused

InfoSecConcept:AuditingLineage

ClouderaNavigator

17©Cloudera,Inc.Allrightsreserved.

Audit&LineageTrustedforproduction• 100sofcustomerdeploymentsofClouderaNavigatoroverlast3+years

Compliance-ready• OnlyHadoopdistributiontopassPCIaudit

Detailed• Columnandrowlevelaccesstrail

Playsnicelywithothers• Integratedwiththeleadingpartnersolutions

18©Cloudera,Inc.Allrightsreserved.

DataSecurityRequirements

Performanalyticsonregulateddata

Encryptdata,conformtokeymanagementpolicies,protectfromroot

IntegratewithexistingHSMaspartofkeymanagementinfrastructure

DataProtectingdatainthe

clusterfromunauthorizedvisibility

InfoSecConcept:Encryption,Keymanagement,Datamasking

NavigatorEncrypt&KeyTrustee

19©Cloudera,Inc.Allrightsreserved.

ComprehensiveDataSecurity

Manager Navigator

Impala Hive

HDFS HBase

Sentry

NavigatorKeyTrustee

LogFiles

MetadataStore

EncryptedData

EncryptionKey

Legend

IngestPaths,Temp/Spillfiles

HSM(optional)

ALLdataonthewire• ALLdataatrest:HDFS,HBase,metadatadatabases,tempfiles,ingestpaths

• Automatedkeyreplication&backup• HSMbackedkeyprotection

• Sensitivedatainlogs• Passwordsinconfig files

Encryption

KeyManagement

DataMasking

20©Cloudera,Inc.Allrightsreserved.

Cloudera’scomprehensive,compliance-readysecuritysolution

AccessDefiningwhatusersandapplicationscan

dowithdata

TechnicalConcepts:PermissionsAuthorization

DataProtectingdatainthe

clusterfromunauthorizedvisibility

TechnicalConcepts:Encryption,Keymanagement,Datamasking

VisibilityReportingonwheredatacamefromandhowit’sbeingused

TechnicalConcepts:AuditingLineage

ClouderaManager ApacheSentry&RecordService

ClouderaNavigator NavigatorEncrypt&KeyTrustee

PerimeterGuardingaccesstothe

clusteritself

TechnicalConcepts:Authentication

Networkisolation

21©Cloudera,Inc.Allrightsreserved.

Beyondtraditionalsecuritycontrols

Automateddiscoveryandtaggingofsensitivedata• Automaticallyscanforprotectedattributetypes• Automaticallyapplyauthorizationandencryptionpolicy

“Followthedata”authorizationandprotectionpolicies• Leveragelineagedatataggingenforceauthorizationandencryptionpolicy• Eliminatemanualconfigurationofsecurityforeachnewtableandcolumn

Adminsfocusedonexceptionhandlingduetoinsufficientaccess

©Cloudera,Inc.Allrightsreserved. 22

ThankyouThankYouFredKoopmans