DataStax & 451 Group Webinar - Real NoSQL Applications in the Enterprise Today
Building secure NoSQL applications nosqlnow_conf_2014
-
Upload
sujee-maniyam -
Category
Technology
-
view
539 -
download
0
Transcript of Building secure NoSQL applications nosqlnow_conf_2014
![Page 1: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/1.jpg)
+
Building Secure Applications With HBase / Accumulo
Sujee Maniyam [email protected]
Nosql now! 2014 Conference
Aug 2014, San Jose, CA
![Page 2: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/2.jpg)
+About This Talk…
n Some practical tips & design patterns on building secure applications using HBase and Accumulo
n A quick demo (fingers crossed!)
n Audience : technical
![Page 3: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/3.jpg)
+Who Invited This Guy?
n HI, I am Sujee Maniyam
n Founder / Principal @ Elephant Scale Consulting & Training in Big Data, NoSQL
n Co-Author of open source Hadoop book: http://hadoopilluminated.com
n Founder / Organizer of ‘Big Data Guru’ meetup http://www.meetup.com/BigDataGurus/
n Open source : http://github.com/sujee
n http://sujee.net | http://www.linkedin.com/in/sujeemaniyam
![Page 4: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/4.jpg)
+NoSQL eco-system (too many!)
![Page 5: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/5.jpg)
+HBase : Quick Intro
n Modeled after Google Big Table
n Distributed, Nosql store built on Hadoop / HDFS
n Apache project
n http://hbase.apache.org/
HDFS
HBase
![Page 6: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/6.jpg)
+Accumulo : Quick Intro
n Developed by the National Security Agency (NSA) !
n Google Big Table implementation
n Nosql store on top of HDFS
n Security is a first grade concept
HDFS
Accumulo
![Page 7: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/7.jpg)
+HBase & Accumulo
n Both are Big Table implementation
n Based on HDFS
n Written in Java
n Apache open source projects
HDFS
HBase Accumulo
![Page 8: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/8.jpg)
+Approach to Security in Hadoop Until Recently…
![Page 9: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/9.jpg)
+But Security Picture Has Improved Rapidly…
n Lot of work going on in the eco system
n Hadoop vendors (Cloudera / HortonWorks ..) have been very actively working on security features
n ‘the core’ features are in
n Ease of use improving as well
![Page 10: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/10.jpg)
+Next : Building Secure Applications
![Page 11: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/11.jpg)
+What Does It Mean to be ‘Secure’?
n 1) Control who can get in?
n 2) Verify the person’s identity
n 3) safeguard communications with user
n 4) What is allowed for this user
n 5) And finally… n Protect data at rest
![Page 12: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/12.jpg)
+1) Who can get in
n Control which machines can connect to NoSQL cluster
n Don’t expose the cluster to public n Too many open ports
n Too vulnerable
n Solutions: n Run cluster behind firewall
n Restrict which machines can connect to cluster
n Linux / Network level security
n Outside the actual NoSQL
![Page 13: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/13.jpg)
+Trusted Environment
![Page 14: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/14.jpg)
+2) User Authentication
n Wolf: Knock… Knock…
n Pig : Who is there?
n Wolf : It is me… little pig
n How can we verify the user? n Username / password (gmail)
n Or use a third person (referee) n Kerberos
Source : http://1.bp.blogspot.com/
![Page 15: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/15.jpg)
+Kerberos : Quick Primer
n Kerberos is a authentication protocol for networked machines
n Validates client to server and vice-versa
n Strong crypto algorithms (AES, 3DES…)
![Page 16: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/16.jpg)
+Kerberos Protocol for Getting a Beer in a Carnival / Fair J_
![Page 17: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/17.jpg)
+Kerberos Protocol Explained : Getting Beer @ Fair / Party
n Prove your age (identity) to wrist-band issuer n Ticket Granting Ticket
n Get a wristband à qualifies you to get beer n Service Ticket
n Go to bartender and ask for beer using your wrist-band n Service Request
n Get Beer ! J
n For technically correct explanation see : http://www.roguelynn.com/words/explain-like-im-5-kerberos/
![Page 18: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/18.jpg)
+Kerberos Integration
HBase Accumulo
Kerberos Integration yes Yes (simple authentication built-in also)
![Page 19: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/19.jpg)
+3) Secure Client Communication
n Guard client / server communication (‘on the wire’)
n Done by using SASL (certificates)
n Prevents snooping by third parties
Hbase Accumulo
Secure client communications
Yes Yes
![Page 20: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/20.jpg)
+4) What Is Allowed For This User?
n In unsecured environment users can read / write to any table n à not very secure!
n Control which data users can see..
![Page 21: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/21.jpg)
+Quick Primer on HBase Storage
n Tables have many rows
n Row has multiple columns (or qualifiers)
n They are grouped into column families
n Each cell also has a timestamp (not shown here)
info secure
Customer_id name email phone Last 4 social
Full ssn
Family1
Cell
Family2
![Page 22: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/22.jpg)
+HBase Allows Access Control At Family Level
info secure
Customer_id name email phone Last 4 social
Full ssn
First level CSR can Only access this family
Only supervisors can access this family
![Page 23: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/23.jpg)
+Need More Fine Grained Access
n We like to provide ‘cell level’ access controls
n Greater flexibility in application development
n More fine grained access controls
n Meet Accumulo’s Data Model
![Page 24: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/24.jpg)
+Accumulo Data Model
Family : info
Columns à name email Last 4 ssn Ssn Gmail password
Visibility tokens à
Level 1 Level 1 Level 1 Level 2 OR Top clearance
Top clearance
• Every thing in HBase data model • Plus each row has a ‘Visibility Token’
![Page 25: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/25.jpg)
+Users Are Assigned ‘Visibility Tokens’
User id Visibility levels
User 1 Level 1
User 2 Level 1 + Level 2
Edward Snowden Level 1 + Level 2 + Top Clearance
![Page 26: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/26.jpg)
+Accumulo only returns cells visible to user
family
Columns à name email Last 4 SSN Full SSN Gmail password
person1 Joe [email protected]
6789 123-45-6789
JoeSuperMan!
Visibility tokens à
Level 1 Level 1 Level 1 Level 2 OR Top clearance
Top clearance
![Page 27: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/27.jpg)
+What Users Can See…
User Visibility Privilage Visible Cells
User 1 Level 1 Name Email Last 4 ssn
User 2 Level 1 + Level 2
Name Email Last 4 SSN Full SSN
Edward Snowden Level 1 + Level 2 + Top Clearance
Name Email Last 4 SSN Full SSN Gmail Password
![Page 28: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/28.jpg)
+Good News For HBase
n With release 0.98 Hbase also allows cell based access controls
n Called ‘tags’
n Need to upgrade to Hfile V3 (version 3) format
![Page 29: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/29.jpg)
+Visibility / Access Controls
n Both HBase and Accumulo allow access control for the data
Hbase Accumulo
Cell Level Visibility Yes (Starting with v 0.98)
Yes
![Page 30: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/30.jpg)
+5) Final Step : Encrypt Data At Rest
n Eventually data ends up in disk
n We need to protect the ‘raw data’ on disk
n To prevent n Users going to disk directly
n Theft of hardware
![Page 31: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/31.jpg)
+Solution : Encrypt Data Transparently
n Encryption is done via keys n Uses Java Cryptography Extension (JCE)
n Data is encrypted before writing to HDFS n Does not rely on HDFS or Linux level encryption
n Per family encryption is supported
Hbase Accumulo
Encryption At Rest Yes Yes
![Page 32: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/32.jpg)
+HBase & Accumulo : Transparent Encryption
![Page 33: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/33.jpg)
+Encryption : Key Management
n The keys have to managed carefully… n Don’t loose them !
n Don’t compromise them !!
n Possible storage mechanisms n Database
n Remote file server
n Key management server
n Local file system
![Page 34: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/34.jpg)
+Summary
HBase Accumulo
Runs in a trusted environment Yes (outside configuration)
Yes (outside configuration)
User Authentication Kerberos Kerberos + Built-in
Secure client communications (via SSL)
Yes Yes
Visibility at cell level Yes (starting from v0.98)
Yes
Encrypt data at rest Yes Yes
![Page 35: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/35.jpg)
+Useful Resources
n Accumulo n http://www.slideshare.net/DonaldMiner/accumulo-
oct2013bofpresentation
n HBase n http://hbase.apache.org/book/hbase.encryption.server.html
![Page 36: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/36.jpg)
+DEMO
![Page 37: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/37.jpg)
+Demo Explained
Name email ssn Gmail_password
Person1 Joe Smith [email protected]
123-45-6789 ‘JoeDaMan!’
Visibility Level
Level 1 Level 1 Level 2 Top
Demonstrate cell level visibility feature of accumulo Here is how the data looks like:
![Page 38: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/38.jpg)
+Demo : Accumulo Users + Visibility
Accumulo user
Table1 access
Access level
Visible Columns
root yes all all
user1 yes Level 1 Name, email
user2 yes Level 1 + Level 2
Name, email + SSN
esnowden yes Level 1 + Level 2 + Top
Name, email + SSN + Gmail password J
user3 no N/A N/A
![Page 39: Building secure NoSQL applications nosqlnow_conf_2014](https://reader030.fdocuments.in/reader030/viewer/2022032420/55a5cdd31a28ab791b8b45f3/html5/thumbnails/39.jpg)
+Thanks & Questions!
http://ElephantScale.com
Expert consulting & training in Big Data (Hadoop, NoSQL, Spark)
Free, online Hadoop book ‘Hadoop illuminated’