Experiences using CouchDB inside Microsoft's Azure team
32
CouchDB at Microsoft We’re running it in Azure, and so can you
-
Upload
brian-benz -
Category
Internet
-
view
778 -
download
0
description
Co-presented with Will Perry (@willpe). Real-world experiences using CouchDB inside Microsoft, and also how to get started with CouchDB on Microsoft Azure.
Transcript of Experiences using CouchDB inside Microsoft's Azure team
CouchDB at MicrosoftWe’re running it in Azure, and so can you
Will Perry @willpe
Windows Azure Group
Brian Benz @bbenz
Microsoft Open Tech
An app we’ve built
How we built it. Some tips and tricks
Share some experience on helping folks get used to non-relational databases.
Open Source at Microsoft
Running CouchDB in Windows Azure
the app.
Daylight – a test automation system for the cloud
Stores metadata about every revision of every test case in our product
About 10k tests updated about 6 times a day
Stores details of test runs and results
About 5k results stored 6 times a day
100% in the cloud
the database.
Must haves:
Scale to millions/billions of documents
Fairly loose schema
Not a single point of failure
Multiple indexes – really fast access to docs
Nice to haves:
Web friendly
Aggregate oriented
Http Interface
Easily partitionable
SQL TABLE COUCH
CouchDB was the most natural fit for the data we had
and the app we were building
…and the Apache 2.0 license helped keep the lawyers happy
the architecture.
Multi-node deployment with application server and couch db running on each node and replicating across the deployment
Node 0
Load Balancer
Node 1 Node 2
VHD VHD VHD
time-partitioned data.
Mar-2013 Feb-2013Apr-2013 Jan-2013
Test data becomes exponentially less useful over time. Part of our scale strategy is deleting old data in a ‘sliding-window’ fashion.
semi-static typing.
[JsonProperty("attributes")]public JObject Attributes { get; set; }
[JsonProperty("full_name", Required = Required.Always)] public string FullName { get; set; }
[JsonProperty("priority")]public int Priority { get; set; }
Strong typing when we want it, loose typing when we don’t
http.
using (var httpClient = new HttpClient(couchBaseUri)){
return await httpClient.GetAsync(resultId);}
We didn’t feel the need to use a client library, and just extended the built-in http stack to encapsulate some of our conventions
design documents.
In the source tree
Used a directory tree to map to the json tree
Compile into a json document at start-up
Compare to existing design doc and overwrite if different
culture shock.
Map/Reduce != T-SQL
Help out SQL junkies by talking in terms of the index.
experience.
We really like CouchDB. It’s been really reliable, simple to manage and reduced our implementation costs.
Sometimes indexing can slow us down and there’s a reasonable ramp-up required for new devs on the project.
Overall, we’re confident we made the right choice.
Wholly owned subsidiary of Microsoft
To advance Microsoft’s investment in openness
Interoperability
Open source
Open standards
Windows Azure: What It Is
Flexible
Windows Azure
Virtual Machines
Windows Server 2008 R2
Windows Server 2012
OpenSUSE 12.1
CentOS 6.2
Ubuntu 12.04
SUSE Linux Enterprise Server SP2
VM Depothttp://vmdepot.msopentech.com
Call to action: Developers can use VM Depot to reach new audiences!
Windows Azure: What It Is
Deployment Complexity
OperationalResponsibility Flexibility OS Security
IaaS Some High HighLinux or Windows
Custom
PaaS Some Low HighWindows only
Built-in
SaaS Low/noneNone
Limited N/A Built-in
What’s the Difference?
Virtual Machine Sizes
VM SizeCPU Cores
MemoryBandwidth (Mbps)
# Data Disks
Extra Small
Shared 768 MB 5 1
Small 1 1.75 GB 100 2
Medium 2 3.5 GB 200 4
Large 4 7 GB 400 8
Extra Large
8 14 GB 800 16
Supported Web Frameworks
Supported Publishing Methods
Windows Azure Developer Center
Open-source SDKs forpopular programminglanguages, to get youup and running quickly
Choice of popular IDEssuch as Visual Studioand Eclipse
Consistent REST protocols and APIs across Windows Azure services
http://www.windowsazure.com/en-us/develop/overview/
.NET driver
Node.js driver
Java driver
PHP driver
Python driver
Driver download page
azure topic verb options
Command Line Syntax Overview
prompt>
accountaccount locationaccount affinity-groupvmvm diskvm endpointvm imageserviceservice certsiteconfig
downloadimportlistshowdeletestartrestartshutdowncapturecreateattachdetachbrowseset
usernamepassworddns-prefixvm-namelb-porttarget-image-namesource-pathdisk-image-namesize-in-gbthumbprintvalue-v-vv
Example: Provisioning a Cluster
azure vm create couchsd "OpenLogic__OpenLogic-CentOS-62-20120531-en-us-30GB.vhd" username password -l "West US" -e
azure vm create couchsd "OpenLogic__OpenLogic-CentOS-62-20120531-en-us-30GB.vhd" username password -l "West US" -e 23 -c
azure vm create couchsd "OpenLogic__OpenLogic-CentOS-62-20120531-en-us-30GB.vhd" username password -l "West US" -e 24 -c
azure vm endpoint create couchsd 27017 27017
azure vm endpoint create couchsd-2 27018 27018
azure vm endpoint create couchsd-3 27019 27019
Getting Started
http://www.windowsazure.com/en-us/pricing/free-trial/
http://channel9.msdn.com/Events/WindowsAzureConf/2012/KEY01
Sign up for a free trial
Learn about the latest features
Will Perry @willpe
Windows Azure Group
Brian Benz @bbenz
Microsoft Open Tech
