4 Essential Steps for Managing Sensitive Data
-
Upload
hortonworks -
Category
Technology
-
view
85 -
download
1
Transcript of 4 Essential Steps for Managing Sensitive Data
1 © Hortonworks Inc. 2011 – 2017 All Rights Reserved Hortonworks Confidential. For Internal Use Only.
4ESSENTIALSTEPSFORMANAGINGSENSITIVEDATA
AGENDA▸ Hortonworks Introduction
▸ Security & Governance with Hortonworks
▸ Sensitive Data Management Challenges
▸ Hortonworks DataPlane Service
▸ Demo (Data Steward Studio)
▸ Privacera Introduction
▸ 4 steps in managing sensitive data
▸ Representative scenarios & solutions
▸ Demo (Privacera)
▸ Wrap up
© Hortonworks Inc. 2011 – 2018. All Rights Reserved
AboutHortonworks:
EnablingtheModernDataArchitecturethroughconsistentandcontinuousinnovation
© Hortonworks Inc. 2011 – 2018. All Rights Reserved
ApacheRanger
•Centralauditlocationforallaccessrequests•Supportmultipledestinationsources(HDFS,Solr,etc.)•Real-timevisualqueryinterface
AuditingAuthorization
•Storeandmanageencryptionkeys•SupportHDFSTransparentDataEncryption• IntegrationwithHSM
•SafenetLUNA
RangerKMS
•Centralizedplatformtodefine,administerandmanagesecuritypoliciesconsistentlyacrossHadoopcomponents•HDFS,Hive,HBase,YARN,Kafka,Solr,Storm,Knox,NiFi,Atlas
•ExtensibleArchitecture•Custompolicyconditions,usercontextenrichers•Easytoaddnewcomponenttypesforauthorization
6 © Hortonworks Inc. 2011 – 2018. All Rights Reserved
DynamicRowFiltering&ColumnMasking:ApacheRangerwithApacheHive
User2:IvannaLocation:EUGroup:HRUser1:Joe
Location:USGroup:Analyst
OriginalQuery:SELECTcountry, nationalid,ccnumber, mrn,nameFROM
ww_customers
Country NationalID
CCNo DOB MRN Name PolicyID
US 232323233 4539067047629850 9/12/1969 8233054331 JohnDoe nj23j424
US 333287465 5391304868205600 8/13/1979 3736885376 JaneDoe cadsd984
Germany T22000129 4532786256545550 3/5/1963 876452830A Ernie Schwarz KK-2345909
Country National ID CC No MRN
Name
US xxxxx3233 4539 xxxxxxxxxxxx null JohnDoe
US xxxxx7465 5391 xxxxxxxxxxxx null JaneDoe
RangerPolicyEnforcementQueryRewrittenbasedonDynamicRanger
Policies:Filterrowsbyregion&applyrelevantcolumnmasking
UsersfromUSAnalystgroupseedataforUSpersonswithCCandNationalID(SSN)asmaskedvaluesandMRNisnullified
Country National ID Name MRN
Germany T22000129 ErnieSchwarz
876452830A
EUHRPolicyAdminscanseeunmaskedbutarerestrictedbyrowfilteringpoliciestoseedataforEUpersonsonly
OriginalQuery:SELECTcountry, nationalid,
name,mrnFROMww_customers
AnalystsHR Marketing
© Hortonworks Inc. 2011 – 2018. All Rights Reserved
ApacheAtlas:OpenMetadata&Governance
STRUCTURED
TRADITIONALRDBMS
METADATA
MPPAPPLIANCES
Kafka Storm
Sqoop
Hive
ATLASMETADATA
Falcon
RANGERCustom
Partners
ComprehensiveEnterpriseDataCatalog• Listsallofyourdata,whereit islocated, itsorigin(lineage),owner,
structure,meaning,classification andquality• Integratebothon-premiseandcloudplatformstoprovideenterprise
wideviewOpenEnterpriseDataConnectors• Interoperableconnectorframeworktoconnecttoyourdatacatalogout
oftheboxwithmanyvendortechnologies• NoexpensivepopulationofproprietarysiloedmetadatarepositoriesDynamicMetadataDiscovery• Metadata isaddedautomatically tothecatalogasnewdataiscreatedor
dataisupdated• ExtensiblediscoveryprocessesthatcharacterizeandclassifythedataEnablingCollaboration&Workflows• Subjectmatterexperts locatethedatatheyneedquicklyandefficiently,
sharetheirknowledgeaboutthedataanditsusagetohelpothers• InterestedpartiesandprocessesarenotifiedautomaticallyAutomatedGovernanceProcesses• Metadata-drivenaccess control• Auditing,metering,andmonitoring• Qualitycontrolandexceptionmanagement• Rights(entitlement) managementPredefinedstandardsforglossaries,dataschemas,rulesandregulations
Vision:
Metadata-drivenfoundationalgovernanceservicesforenterprisedata
ecosystem
• OpenframeworksandAPIs
• Agileandsecurecollaborationarounddataandadvancedanalytics
• Reduceoperationalcostswhileextractingeconomicvalueofdata
©HortonworksInc.2011–2018.Allrightsreserved.
Hortonworksconfidentialandproprietaryinformation
NextGenerationDataProblems
DataIsSpreadAcrossMultipleClustersandData
Sources
Store&AnalyzeDataFromERP/CRM,Systems,IoT/MobileDevices,Social
Media,GeoLocationetc.
Somedataison-premise,restinthecloud.
Movingdatafromcloudtoon-premise&viceversa
Movingdatabetweendifferentclouds
HDF HDP™ ®
©HortonworksInc.2011–2018.Allrightsreserved.
Hortonworksconfidentialandproprietaryinformation
WhatIf…
IntheCloud
OnPremises
Aware ofDataSources
EnableNewServices
UnifiedSecurity&Governance
Model
Cluster2(Unstructured)
Cluster1(Structured
)
Cluster2(Unstructured)
Cluster1(Structured
)
Cluster3(Structured
)
DataCenterDublin
Cluster2(Unstructured)
Cluster1(Structured
)
Cluster3(Structured
)Cluster4
(Unstructured)
DataCenterLasVegas
Cluster2(Unstructured)
Cluster1(Structured
)
Cluster3(Structured
)
DataCenterBangkok
Cluster1(Unstructured)
Cluster2(Structured
)
©HortonworksInc.2011–2018.Allrightsreserved.
Hortonworksconfidentialandproprietaryinformation
HortonworksDataPlaneServiceaplatformwithextensibledatamanagementservicesfor:q Addressingcomplianceandregulatoryrequirementsfor
enterpriseq Providingconsistentsecurity&governanceacrossdata
landscapeq Enablingcentralizedmanagementofdataassets
q Responsibledatasharingandcollaboration
WhatisHortonworksDataPlaneService?
11 ©HortonworksInc.2011– 2018.AllRightsReserved
DataStewardStudio(DSS)Suiteofcapabilitiesthatallowsuserstounderstand, secure,andgoverndataacrossenterprisedatalakes
Ensureconsistentsecurityandgovernancefordataassetsacross tiers
• Curate,discoverandorganizedataassetsbasedonbusinessclassifications,purpose,protections,relevance,etc.
• Governproperusageandlineageofdataassetstoidentifyschema,classificationandviewlineage/datasupplychain
• Understandandauditdataassetsecurityanduseforanomalydetection,forensicaudit/compliance&propercontrolmechanisms
…allacrossmultipletypesandtiersofdata
TechnicalPreviewAvailable
HortonworksDataPlaneService:ExtensibleServices
DATASTEWARDSTUDIODSS
Discover&Fingerprint
Data
SmartEnterprise
Search
Data&MetadataSecurity
DataLineage&ImpactAnalysis
EnterpriseDataCatalog
Organize&CurateData
12 ©HortonworksInc.2011–2018.Allrightsreserved.
Hortonworksconfidentialandproprietaryinformation
CONSUMABILITY: Understand shapeofHivecolumndatawithstatisticalprofiler,example:Profile showsboxplotandhistogramfordistribution ofcolumnvalues
DataStewardStudio(DSS)
13 ©HortonworksInc.2011–2018.Allrightsreserved.
Hortonworksconfidentialandproprietaryinformation
CONSUMABILITY: Datalineageshowscompletechainofcustody anddownstream dependencies foranasset!
DataStewardStudio(DSS)
14 ©HortonworksInc.2011–2018.Allrightsreserved.
Hortonworksconfidentialandproprietaryinformation
CONSUMABILITY: AuditProfiler showsbothsummarizedviews&patternsofaccessforadataasset.
DataStewardStudio(DSS)
TEXT
PRIVACERA
DETECTMALICIOUS OR
ACCIDENTAL USE
CONTROLANONYMIZE DATA/RESTRICT ACCESS
DISCOVERWHAT TYPE OF DATA STORED AND WHERE?
REPORTSECURITY AND
COMPLIANCE REPORTING
PLATFORM TO MANAGE SENSITIVE DATA
REPRESENTATIVE SCENARIO – FINANCIAL SERVICES
DATA LAKE
Multiple systems
Multiple formats
INGESTION STORAGE AND PROCESSING DOWNSTREAM SYSTEMS
Sensitive data cannot be shared
with users
SOLUTION - PRIVACERA AUTOMATED DATA DISCOVERY
Discover and classify data
during ingest or at rest
Standard rules combined with
machine learning
Classification/ tags pushed to
Atlas
STEP 1 > STEP 2 > STEP 3 > STEP 4
REPRESENTATIVE SCENARIO – HEDGE FUND
DATA LAKEStock Info
Proprietary Confidential data
INGESTION STORAGE AND PROCESSING DOWNSTREAM SYSTEMS
Access to sensitive data is restricted Data Scientist
SOLUTION - TAG BASED ACCESS CONTROL
Simplify policies by managing at
tag level
Tag attributes such as
expiration date
Metadata updated by
Privacera
STEP 1 > STEP 2 > STEP 3 > STEP 4
REPRESENTATIVE SCENARIO - HEALTHCARE
INGESTION STORAGE AND PROCESSING DOWNSTREAM SYSTEMS
HDFS
HIVE
ETL
Tokenized sensitive
data
Select users with raw
data access
Most users see only
tokenized data
SOLUTION - PRIVACERA ANONYMIZATION
Format preserving
encryption and masking
Integrated with Ranger
infrastructure
Policy driven access
STEP 1 > STEP 2 > STEP 3 > STEP 4
REPRESENTATIVE SCENARIO – FINANCIAL SERVICES
INGESTION STORAGE AND PROCESSING DOWNSTREAM SYSTEMS
DATA LAKE
HIVE
ETL
Compliance team manually analyzing
audit logs
FTP SERVER
Where is sensitive data and where is it
moving ?
SOLUTION - PRIVACERA MONITORING
Automated monitoring of user actions
Alerts if sensitive is moved or on unusual access
Alerts if sensitive data is
discovered in restricted zones
STEP 1 > STEP 2 > STEP 3 > STEP 4
SUMMARY
▸ Understand your data before expanding your data lake
▸ Invest in automated classification and centralized metadata
▸ Manage access to user by data classification
▸ Anonymize data to reduce exposure
▸ Monitor the use of data, “trust but verify”.
▸ Data plane provides next generation for tools for hybrid data infrastructure
QUESTIONS [email protected]