Drupal MongoDB Integration benefits Mobile Media Site with Speed and Scalability
-
Upload
cignex-datamatics -
Category
Technology
-
view
1.286 -
download
2
description
Transcript of Drupal MongoDB Integration benefits Mobile Media Site with Speed and Scalability
CIGNEX Datamatics Confidential www.cignex.com
Scalable Mobile Media Site using Drupal & MongoDB
Presented By:
Yash Badiani, Big Data Practice Lead, CIGNEX Datamatics
Gaurav Khambhala, Technical Lead, CIGNEX Datamatics
30th July, 2013
CIGNEX Datamatics Confidential www.cignex.com
About CIGNEX Datamatics
2
CIGNEX Datamatics Confidential www.cignex.com
What Does CIGNEX Datamatics Do?
3
Since 2000, delivering Open Source
solutions for the enterprise through
adoption and integration to:
• Address business goals
• Increase business velocity
• Lower the cost of doing business
• Reduce TCO
• Gain competitive advantage
Portal Solutions Content
Solutions
Big Data Analytics Solutions
400+ Open Source Solutions
450+ Open Source Experts
200+ Open Source Integrations
12+ Books on Open Source
5000+ Community Contributions
Offices : America | India | UK | Europe | Singapore | Australia
CIGNEX Datamatics Confidential www.cignex.com
Where We Can Help You
4
SOLUTIONS
Managed Cloud Services Develop • Deploy • Manage
VAR/Annual Product Subscription Liferay • Alfresco • Cloudera Hadoop • MongoDB
Extended Development Center Center of Excellence
UI • Development • Integration • Customization • Migration • Testing • Training • Support (24*7)
User eXperience Platform
Portals Liferay, Drupal, JBoss,
ZK, HTML5,
MuleSoft
• Intranet • Extranet • EAI • SOA
• Social Collaboration
• Big Data Portal • Mobile Portal
Enterprise Content Management
Content Alfresco, Adobe CQ,
Drupal, Magento,
JBoss, Moodle, Ephesoft,
Liferay
• WCM • DM • RM • CMS • DAM
• e-Commerce • e-Learning • ERP • Imaging Solutions
SERVICES
Making Data Work
Big Data Analytics
Hadoop Ecosystem,
MongoDB, Ne04j, Pentaho,
Talend, Solr, Jaspersoft
• Data Integration
• Information Delivery
• Data Analysis
• Enterprise Search
Enterprise Mobility Strategy • Mobile UX • App Development • MEAP/MDM
CIGNEX Datamatics Confidential www.cignex.com
About the presenters
• Yash Badiani is the Big Data Practice Lead at CIGNEX Datamatics and focuses on Big Data Technologies including MongoDB & Hadoop. He has worked extensively on large Data warehousing & Business Intelligence projects with tools such as Business Objects, Microsoft SQL Server, Microstrategy, IBM Cognos.
• Gaurav Khambhala works at CIGNEX Datamatics as Technical Lead. He is the senior member of the PHP Practice at CIGNEX Datamatics and is involved on various technology initiatives like Big Data where he focuses on integration of PHP with NoSQL sources like MongoDB. He has a wide industry experience in software development & management in Open Source technologies such as Drupal, Moodle & Wordpress.
5
CIGNEX Datamatics Confidential www.cignex.com
Agenda
• The Mobile Media Use Case
• Requirements and Challenges
• Solution : Mobile Media site using Drupal & MongoDB
• Why Drupal and MongoDB?
• Demo and Solution Features
• Benefits
• Summary
6
CIGNEX Datamatics Confidential www.cignex.com
The Mobile Explosion!
7
By 2015, at least 60% of information workers will
interact with their content applications via a mobile
device
People use digital assets (videos, images) longer on
Tablets and Mobiles compared to desktops
Based on a report by a leading IT advisory firm
By 2017, 82% of all phones purchased will be
smartphones.
CIGNEX Datamatics Confidential www.cignex.com
Mobile Media Use Case
• Mobile Media site includes the following features: – Store a variety of Images & associated metadata
– Massively Scalable to store billions of images
– Access through Mobile
– Create /Edit Albums
– Add Images to the Albums
– Add / Edit Metadata of Images
– Search Images / Albums by date, metadata, albums, etc
– Social Media features – Likes, comments
8
CIGNEX Datamatics Confidential www.cignex.com
Requirements of Mobile Media sites
9
• Fast performance • Large user base • Concurrent CRUD • Access through various channels
Velocity
• Millions of digital assets • Variety of content • Complexity of data
Volume
• Rich UI features • Social features • Mobile access • Fast search
User experience
• Elastic scaling • Cost effectiveness • Centralized storage • Ease of Maintenance
Scalability
• HIGH availability • Automatic failover • User management
Security & Availability
• Easy integration • Shorter dev cycle • Faster deployment • Ease of schema design
Flexibility & Agility
CIGNEX Datamatics Confidential www.cignex.com
Standard Three Layered Data Architecture
10
File System
Metadata in RDBMS
Search
Standard Three Layered Storage
Application layer
CIGNEX Datamatics Confidential www.cignex.com
Introduction to NoSQL
• They are schema less
• Designed to support huge data volumes – Facebook 135 billion messages/month; Twitter 7TB data/day
• Scalable replication and distribution mechanism – Thousands of machines distributed around the world
• Massive write performance with asynchronous inserts and updates
• Designed to give high query performance
• Runs on commodity hardware
• Most NoSQL databases are Open Source
11
Not SQL Only
CIGNEX Datamatics Confidential www.cignex.com
NoSQL – Data Models
12
Column Families
Usage: Read/Write Intensive
Popular databases: Hbase, Cassandra
Document Store
Usage: Working with Occasionally
changing/consistent data
Popular databases: CouchDB, MongoDB
Graph Database
Usage: Spatial Data storage, Social Application
Popular databases: Neo4j
Key Value / Tulip Store
Usage: Briskly changing data and high availability
Popular databases: Riak, Redis, Azure Table storage
NoSQL Databases
• 4 broad data models • 120+ variants available in the market
CIGNEX Datamatics Confidential www.cignex.com
Requirements of Mobile Media sites - Recap
13
• Fast performance • Large user base • Concurrent CRUD • Access through various channels
• Millions of digital assets • Variety of content • Complexity of data
• Rich UI features • Social features • Mobile access • Fast search
• Elastic scaling • Cost effectiveness • Centralized storage • Ease of maintenance
• HIGH availability • Automatic failover • User management
Velocity Volume
User experience
Scalability
Security & Availability
• Easy integration • Shorter dev cycle • Faster deployment • Ease of schema design
Flexibility & Agility
Mobile Media Site
CIGNEX Datamatics Confidential www.cignex.com
Drupal with MongoDB Solution
14
Themes
Core Modules
Nodes Taxonomy User Roles
Forms & Menu
Custom Modules
Workflow Forums Comments & Ratings
Tagging
PHP
Web Services
3rd party & Internal
Applications
MongoDB Driver
Mongos Routing Process
Replica Set
MongoDB MongoDB
Replica Set
MongoDB MongoDB
CIGNEX Datamatics Confidential www.cignex.com
Why Drupal?
15
Pluggable Architecture
Data Abstraction Layer
Easy to Upgrade
Secure
Active Community Widely Adopted
Scalable
3rd Party Tools Integration
User Management & Permissions
HTML5 & CSS Support
Drupal Open Source CMS
CIGNEX Datamatics Confidential www.cignex.com
Websites using Drupal
16
Website: Whitehouse.gov Website: Data.gov.uk
Website: mtv.co.uk
Website: research.yahoo.com
Website: pdx.edu Website: EndPoverty2015.org
CIGNEX Datamatics Confidential www.cignex.com
Why MongoDB?
17
Agile and Scalable
Full Index Support
Document Oriented Storage
Replication Querying
Atomic Updates
Data Processing and aggregation
High Availability
Open Source NoSQL Solution
CIGNEX Datamatics Confidential www.cignex.com
Customers using MongoDB
• Centralized data management platform
• 2 billion+ documents
• 20 TB of photo metadata
• TV episodes and series
• Risk solutions auditing data
18
Source: http://www.10gen.com/customers
CIGNEX Datamatics Confidential www.cignex.com
Demo
19
• Media site on mobile simulator • Like & comment on an image on mobile simulator • Mobile site on web browser • Verify ‘Like’ & comment of the same image on web browser • Search images & access control
CIGNEX Datamatics Confidential www.cignex.com
Solution Features
Architecture and Design
20
CIGNEX Datamatics Confidential www.cignex.com
Architecture
21
User Metadata In
de
xes
Albums
Image Metadata
GridFS
Form API
Drupal API
Custom Module
Browser/Mobile Theme
MongoDB PHP Driver
Menu API
User Mobile Device
Image Metadata
GridFS
CIGNEX Datamatics Confidential www.cignex.com
Add Album
Flow
22
Add Image
View Album
View Individual Images
Like Image
Comment Image
Add Tags to Images
View Counter
Search Images By Tags
User Metadata Albums GridFS Image Metadata
DBRef
DBRef DBRef
DBRef
MongoDB Collections
Use
r A
ctio
ns
CIGNEX Datamatics Confidential www.cignex.com
Schema Design
23
User Metadata GridFS
Albums Image Metadata
• User ID • DBRef (Album) • Tags • Thumbnail • Likes • View Counter • Comments • Permission
• FS.Files • FS.Chunks
• User ID • Tags • Title
• Make • Model • Date Time • Aperture • Exposure • DBRef (GridFS)
CIGNEX Datamatics Confidential www.cignex.com
Schema Design
24
Image Metadata Albums
User Metadata
CIGNEX Datamatics Confidential www.cignex.com
MongoDB Monitoring Service (MMS)
25
• DB Storage
• Cursors
• Replica Sets
• Network Connections
• Non Mapped Virtual Memory
• Opcounters
CIGNEX Datamatics Confidential www.cignex.com
Benefits
Drupal MongoDB
Most advanced content management solutions
Scalability – billions of content items, millions of users
Highly customized websites Performance – FAST writes through sharding, reads through indexes
Most search friendly CMS Safety - Replication
Less coding, high on automation Centralized Data Storage - Single System
Powered by 7000+ plugins and extensions
Monitoring & Administration - MMS
Active community, real time assistance
Enterprise Support - 10gen
26
CIGNEX Datamatics Confidential www.cignex.com
Summary and Key Takeaways
• MongoDB provides the RIGHT fit for CMS applications with flexibility, scale & speed
• Drupal’s advanced & automated CMS features and tight integration with MongoDB makes it the right choice for building agile websites
• Both Drupal & MongoDB are feature rich and being Open Source, provide significant cost benefits
27
CIGNEX Datamatics Confidential www.cignex.com
To know how Open Source can be adopted and integrated into your enterprise
Sign Up for Free Assessment Workshop http://www.cignex.com/freeassessment
Thank You. Any Questions ?
Making Open Source WorkTM
Yash Badiani Big Data Practice Lead
Gaurav Khambhala Technical Lead