Developing Open Source
Solutions with NoSQL and
Other Cloud Services on Azure
Brian Benz, Sr. Technical Evangelist
Microsoft Open Technologies, Inc.
2 http://www.anglebrackets.org
Agenda
Azure
NoSQL
NoSQL on Azure
MongoDB and CouchDB Options
NoSQL on Azure IaaS, PaaS, SaaS
Azure Web Sites
Q&A
Azure Virtual Machines – OS Options
Windows Server 2008 and 2012
OpenSUSE and SUSE Enterprise
CentOS
Ubuntu
Oracle Linux
7 http://www.anglebrackets.org
NoSQL
Alternative to traditional relational DBMS
Flexible schema
Quicker/cheaper to set up
Massive scalability
Relaxed consistency = higher performance & availability
No declarative query language = more programming
Relaxed consistency = fewer guarantees
8 http://www.anglebrackets.org
NoSQL Applications
Document: JSON, XML, other semistructured formats
Operations: Insert(), Fetch(key), Update(key, doc), Delete(key)
(Key, Value) pairs
Operations: Insert(key,value), Fetch(key), Update(key), Delete(key)
Implementation: efficiency, scalability, fault-tolerance
Records distributed to nodes based on key
Replication
Single-record transactions, “eventual consistency”
9 http://www.anglebrackets.org
New Terms
RDBMS NoSQL
Database Database
Table Collection
View, Row Document
Index Index
Join Embedding, Linking
Partition Shard
Partition Key Shard Key
11 http://www.anglebrackets.org
Basic Document Structure{
"_id" : ObjectId("508d27069cc1ae293b36928d"),
"title" : "This is the title",
"body" : "This is the body text.",
"tags" : [
“tag 1"
],
"created_date" : ISODate("2012-10-28T12:41:39.110Z"),
"author_id" : ObjectId("508d280e9cc1ae293b36928e"),
"category_id" : ObjectId("508d29709cc1ae293b369295"),
"comments" : [
{
"subject" : "This is comment 1",
"body" : "This is the body of comment 1.",
"author_id" : ObjectId("508d345f9cc1ae293b369296"),
"created_date" : ISODate("2012-10-28T13:34:23.929Z")
},
]
}
12 http://www.anglebrackets.org
Less Basic Document Structure
{
"_id" : ObjectId("508d27069cc1ae293b36928d"),
"title" : "This is the title",
"body" : "This is the body text.",
"tags" : [
ObjectId("508d35349cc1ae293b36929c")
],
"created_date" : ISODate("2012-10-28T12:41:39.110Z"),
"author_id" : ObjectId("508d280e9cc1ae293b36928e"),
"category_id" : ObjectId("508d29709cc1ae293b369295"),
"comments" : [
ObjectId("508d359a9cc1ae293b3692a2")
]
}
13 http://www.anglebrackets.org
The Landscape - NoSQL on Azure
MongoDB
Also MongoLab MongoDB-as-a-Service
Also MongoDB Inc. MongoDB-as-a-Service (beta)
CouchDB
Cloudant CouchDB-as-a-Service
Cassandra
Riak
Redis
14 http://www.anglebrackets.org
What is MongoDB?
Open Source, by MongoDB, Inc.
Community and Enterprise Editions
Language: c++
Data model: Using BSON (binary JSON)
Replication and Sharding
Azure VMS, PaaS, and SaaS
15 http://www.anglebrackets.org
http://docs.mongodb.org/ecosystem/drivers/
• C, C++
• C#
• Go
• Erlang
• Java
• JavaScript
• Node.js
• Perl
• PHP
• Python
• Ruby
• Scala
16 http://www.anglebrackets.org
Support and Alerts
Custom Admin Tools
Backup and Recover
Performance Analysis – Slow Query Analyzer
Monitoring and Support
19 http://www.anglebrackets.org
What is CouchDB?
Apache Project
Language: Erlang
Data model: Using JSON and REST
Map/Reduce Views and Indexes
Replication
Azure VMs, PaaS, SaaS
20 http://www.anglebrackets.org
Managed, hosted fork of CouchDB
Incremental MapReduce engine
Horizontal scaling & built-in Lucene search
8 committers to Apache CouchDB
13,000+ users
Apache CouchDB features
BigCouch: Horizontal scaling framework based on Amazon Dynamo paper
Fauxton: Futon replacement modular CouchDB Web dashboard
live JSON editor with JavaScript hinting
20
21 http://www.anglebrackets.org
What is Cassandra?
Apache Project
Language: Java
Data model: Thrift & custom binary
Fast Writes, Memory management
Replication
Azure VMs, PaaS
22 http://www.anglebrackets.org
What is Redis?
Open Source
Language: c
Data model: RESP (REdis Serialization Protocol)
Key/Value
In Memory
Persistence via snapshotting or journaling to disk
Azure VMs, PaaS
23 http://www.anglebrackets.org
What is Riak?
Open Source
Language: Erlang & C, some JavaScript
Data model: HTTP/REST or custom binary
Key/Value
In Memory
Persistence via snapshotting or journaling to disk
Azure VMs, PaaS, SaaS
Replica set hosted
on Azure
worker roles or
virtual machines
Replica Set
DriverYour
Application
Replica Sets: High Availability
Shards, config servers and mongos (router) hosted
on Azure worker roles or virtual machines
Key range A-D
Sharding: Scalability
Key range E-G Key range G-Z
Mongos
(router)Your
Application
Config 1
Config 2
Config 3
Mongos
(router)
26 http://www.anglebrackets.org
Deploying to Virtual Machines
Compared to worker role (PaaS), VM-basedreplica sets offer more control, aswell as more responsibility
MongoDB Installer for Azure is a great learning tool. Not typically used for production deployment. Puts MongoDB on OS disk instead of attached disk
Limited to 30gb, not performant for writes, Windows-only http://www.mongodb.org/display/DOCS/MongoDB+on+Azure+VM+-
+Windows+Installer
.\deploy-mongo.ps1 <node-count> <dns-prefix> <image-name> <password> <location> <pub-settings-file-path> [replica-set-name]
For Linux VM deployment, see tutorial -http://www.mongodb.org/display/DOCS/MongoDB+on+Azure+VM+-+Linux+Tutorial
azure topic verb options
Command Line Syntax Overview
prompt>
accountaccount locationaccount affinity-groupvmvm diskvm endpointvm imageserviceservice certsiteconfig
downloadimportlistshowdeletestartrestartshutdowncapturecreateattachdetachbrowseset
usernamepassworddns-prefixvm-namelb-porttarget-image-namesource-pathdisk-image-namesize-in-gbthumbprintvalue-v-vv
28 http://www.anglebrackets.org
Example: Provisioning a Replica Set
azure vm create mongoc9 "OpenLogic__OpenLogic-CentOS-62-20120531-en-us-30GB.vhd" username password -l "West US" -e
azure vm create mongoc9 "OpenLogic__OpenLogic-CentOS-62-20120531-en-us-30GB.vhd" username password -l "West US" -e 23 -c
azure vm create mongoc9 "OpenLogic__OpenLogic-CentOS-62-20120531-en-us-30GB.vhd" username password -l "West US" -e 24 -c
azure vm endpoint create mongoc9 27017 27017
azure vm endpoint create mongoc9-2 27018 27018
azure vm endpoint create mongoc9-3 27019 27019
To set up MongoDB, SSH into each instance and:
get mongodb binaries and install
create db dir
start mongod, initialize replica set
Automation Support
Capabilities
• Fully Customize VM with Data Disks and Endpoint Configuration
• Automate Virtual Network Settings
32 http://www.anglebrackets.org
Conference Scheduler
Attendees can track sessions, vote sessions up and down
Offline Capabilities
IndexedDB
CouchDB VM for Data Storage
Web site hosting the application
Node.js
Grunt
39 http://www.anglebrackets.org
Where to learn more
Tutorial: Building a multi-tier cloud application on Windows Azure using MongoLab, node.js, and Socket.IO
http://msopentech.com/blog/2013/09/18/tutorial-building-multi-tier-cloud-application-windows-azure-using-mongolab-node-js-socket-io/
Tutorial: Using MongoDB, Windows Communication Foundation, OData, and PowerBI to build a Business Intelligence solution on Windows Azure
http://msopentech.com/blog/2014/02/19/tutorial-using-mongodb-windows-communication-foundation-odata-powerbi-build-business-intelligence-solution-windows-azure/
Tutorial – Building a Multi-Tier Windows Azure Web application use Cloudant’s Couchdb-as-a-Service, node.js, CORS, and Grunt
http://msopentech.com/blog/2013/12/19/tutorial-building-multi-tier-windows-azure-web-application-use-cloudants-couchdb-service-node-js-cors-grunt-2/
40 http://www.anglebrackets.org
Where to learn more
MongoDB on Azure VM -http://www.mongodb.org/display/DOCS/MongoDB+on+Azure+VM
Install MongoDB on a virtual machine running CentOS Linux in Azure:http://www.windowsazure.com/en-us/manage/linux/common-tasks/mongodb-on-a-linux-vm/
Install MongoDB on a virtual machine running Windows Server in Azure:http://www.windowsazure.com/en-us/manage/windows/common-tasks/install-mongodb/
Node.js Web Application with Storage on MongoDB (Virtual Machine):http://waweb.windowsazure.com/en-us/develop/nodejs/tutorials/website-with-mongodb-(mac)/
Create a Azure web site that connects to MongoDB running on a virtual machine in Azure:http://www.windowsazure.com/en-us/develop/net/tutorials/website-with-mongodb-vm/
41 http://www.anglebrackets.org
Other Resources
Microsoft Open Technologies Blog: http://blogs.msdn.com/b/interoperability VM Depot:
http://vmdepot.msopentech.com Windows Azure development:
www.WindowsAzure.com SDKs and Command Line Tools: http://www.windowsazure.com/en-us/downloads/ Windows Azure Training Kit:
http://windowsazure-trainingkit.github.com/ Microsoft Web Platform Installer: http://www.microsoft.com/web/downloads/platform.aspx Web Matrix http://www.microsoft.com/web/webmatrix/ Project Kudu https://github.com/projectkudu/kudu The application I showed today https://github.com/axemclion/conference
Top Related