MongoUK 2012

© 2012 VMware, Inc. All rights reserved

Activity Streams on Cloud Foundry with MongoDB and Node.jsMongoUK - 2012

By Monica Wilkinson and Andy Piper

Wednesday, June 20, 12

CONFIDENTIAL

Andy Piper

2

social bridgebuilder, techie

Developer Advocate @ Cloud Foundry

15 years in enterprise middleware & messaging

social web enthusiast

OSS supporter / contributor

excited by “what’s next”, Internet of Things, etc.

@andypiper

[email protected]


mailto:[email protected]


CONFIDENTIAL

About Monica Wilkinson

3

Loves the web and data portability.

Developer Advocate @ Cloud Foundry12 years development experience.

Last 5 years in Social WebOpen Web Standards Advocate

Contact Me: @[email protected]




CONFIDENTIAL4

Agenda

1.Overview of Cloud Foundry2.What is Activity Streams ?3.Building an Activity Stream Engine


CONFIDENTIAL

About Cloud Foundry

5

The first Open PaaS

Multi(n) Languages, Frameworks, Services & Clouds

Open Source


Traditional web app architecture! Browser client

! Monolithic WAR / other deployment artefact

! RDBMS

Container (e.g. Tomcat)

CONFIDENTIAL

6

Desktop Browser Apache

Web Application

MySQL Database

ShippingService

AccountingService

InventoryService

StoreFront

“easy” to...

! develop

! test

! deploy

! scale ...apart from: provisioning, setup, need to replicate, shard, scale to massive numbers of mobile clients, provide polyglot support, run multiple dev teams, collaborate...


CONFIDENTIAL

Need to handle massive loads and the data explosion

7

Scalable architectures!

! Application tier:

•Replicated/clustered servers

•Modular so that components can be scaled differently

•Asynchronous architecture - communication via a message broker

! Database tier:

•Replication

•Sharding

•Polyglot persistence: Relational, NoSQL, NewSQL databases


CONFIDENTIAL

Desktop Browser

NodeJS

front-end application

Native Mobile Application

HTML5 mobile application

RabbitMQ

NodeJS

MySQL

Mongo

Redis

ShippingService

Accounting

Service

billing web application

shipping web application

InventoryService

inventory web application

StoreUI

StoreUI

StoreUI

StoreUI

8

Modern application architecture


CONFIDENTIAL9

Platform-as-a-Service is the solution

Easy deployment

Application management

Easy scaling up and down

+

SQL databases

NoSQL databases

Message Broker

Deployment Services


CONFIDENTIAL

10

“Deploy and scale applications in seconds, without locking yourself into a single cloud or vendor”

Simple, Open, Flexible,

Scalable

The Open Platform as a Service


CONFIDENTIALApplication Service Interface

Data Services

Other Services

Msg Services

Cloud Foundry open PaaS - Choice of clouds

Private(Clouds(

PublicClouds

MicroClouds

Clou

d Pr

ovid

er In

terfa

ce (C

PI)

NEW! bosh: deploy to

alternate CPIs (AWS etc)

OSS community

vFabric Postgres

vFabric RabbitMQTM

Additional partner services …


CONFIDENTIAL

12

Multi-cloud flexibility is critical to long-term success

! Make use of both public and private clouds without rewriting your applications

! Protect against vendor lock-in

! Meet different compliance and geographical needs

! Accommodate peak loads while optimizing costs

! Manage your growth and changing needs over time


CONFIDENTIAL13

Getting Started on Cloud Foundry


CONFIDENTIAL

14

http://cloudfoundry.com/signup

optional promo

Register!




CONFIDENTIAL

Ways to use Cloud Foundry

15

command line...

IDE plugin...

toolchain plugin...

Cloud IDEs


CONFIDENTIAL

Example vmc commands

16

$"gem"install"vmc

$"vmc"target"<any"cloud>$"vmc"login"<credentials>$"vmc"push"<my7app>" "">"bind"services?"Yes

$"vmc"update"<my7app>$"vmc"instances"<my7app>"+100"


CONFIDENTIAL

Logging into CloudFoundry.com

17

$"vmc"target"api.cloudfoundry.comSuccessfully"targeted"to"""""""""[http://api.cloudfoundry.com]

$"vmc"loginAttempting"login"to"""[http://api.cloudfoundry.com]Email:"[email protected]:"********Successfully"logged"into"""[http://api.cloudfoundry.com]


http://api.cloudfoundry.com








CONFIDENTIAL

Cloud Foundry Integration plugin for Eclipse

18


CONFIDENTIAL19

What is Activity Streams ?


CONFIDENTIAL20

http://activityStrea.ms! Activity Streams is a simple open specification used to describe social

actions around the web.

! The goal of Activity Streams is data portability

! Some publishers of Activity Streams: Socialcast, MySpace, Facebook and G+

! The default format is JSON but Atom is also supported. The JSON format was added after we saw some initial limitations with JSON

! Core concepts are: Actor, Verb, Object and Target


http://ActivityStrea.ms

http://ActivityStrea.ms


Actor -> Monica

Verb -> Pinned(Bookmarked)

Object -> Rosa Parks’ Bio Page

Target -> Female Leaders Board

Source -> Pinterest

Published -> MLK Day 2012



With Timeline ActionsPosting with a richer protocol like

Activity Streams allows you to do:

- Richer messages and links

- Aggregate

- Assign relevance


CONFIDENTIAL23

Example in JSON


CONFIDENTIAL24

What is an Activity Stream Engine ?

! An activity stream engine allows you to publish events and subscribe to events.

! Many activity stream engines support aggregation of events via Streams.

! Many activity stream engines support fanning out activities to subscribers.


CONFIDENTIAL25

So how did we build this Activity Stream Engine on Cloud Foundry ?


CONFIDENTIAL26

Building an Activity Streams Engine


CONFIDENTIAL27

What should we use to store Activities ?

ColumnKey-Value Document Graph

Redis, Riak

Cassandra,HBase

MongoDB Neo4J


CONFIDENTIAL28

ColumnKey-Value Document Graph

MongoDB (so easy)

Mongo DB schema-less documents are perfect for Activities


CONFIDENTIAL

About MongoDB

29

1. No-SQL database

2. Stores JSON-style documents with embedded documents

3. Horizontally scalable

4. Full Indexing Support

5. Open Source so great ORMs and drivers.

development : {

tools: many,

language_support: superb,

agility: high

},

production: {

speed: fast

fault_tolerance: true

scalability: high

=


CONFIDENTIAL

Step 1- Build the persistence Modelvar MediaLinkHash = {

duration: Number, height: Number, width: Number, url: String };

var ActivityObjectHash = { id: {type: String}, image: MediaLinkHash, displayName: {type: String}, summary: {type: String}, content: {type: String}, url: {type:String}, author: {type: ObjectId, ref: 'activityObject'}, published: {type: Date, default: Date.now}, updated: {type: Date, default: Date.now}, objectType: {type: String}, attachments: [{type: ObjectId, ref: 'activityObject'}], upstreamDuplicates: [String], downstreamDuplicates: [String] };

var ActivityObjectSchema = new Schema(ActivityObjectHash) this.ActivityObject = mongoose.model('activityObject', ActivityObjectSchema);

30


CONFIDENTIAL

var ActivitySchema = new Schema({ id: {type: String}, verb: {type: String}, url: {type: String}, title: {type: String}, content: {type: String}, icon: MediaLinkHash, object: ActivityObjectHash, actor: ActivityObjectHash, target: {type: ObjectId, ref: 'activityObject'}, published: { type: Date, default: Date.now}, updated: { type: Date, default: Date.now}, inReplyTo: {type: ObjectId, ref: 'activity'} });

this.Activity = mongoose.model('activity', ActivitySchema);

return this;};

31


CONFIDENTIAL

Step 2 - Expose helpers for the queries! The most important is to be able to list the activities in

descending order by published time.! It is also important to hydrate any objects for which we

have references

! With Mongoose you can do: ! this.getActivityStream = function(n, fx) { Activity.find().sort('published', 'descending').limit(n).populate('target').run(fx);

}! https://github.com/ciberch/activity-streams-mongoose

32


https://github.com/cloudfoundry-samples/activity-streams-mongoose

https://github.com/cloudfoundry-samples/activity-streams-mongoose

CONFIDENTIAL

How do we make our engine faster and more scalable ?! Add support for PubSub! In a PubSub model as soon as an event is published it

gets sent to all the subscribers.! The Publisher doesn’t know who the subscribers are, it

simply knows where to publish.! Redis has nice support for Pub Sub http://redis.io/topics/

pubsub

33


http://redis.io/topics/pubsub




CONFIDENTIAL

Using Socket.io

! Server Side:• Subscribe to a Stream• When there is a new event send it to the client as JSON

! Client Side• Add support to publish activities to the Stream• Change the client rendering to use ActivityStrea.ms

! Benefits• Richer messaging• Activity Syndication

34



DemoPart 1: App 1 and walkthrough http://blog.cloudfoundry.com/

2012/05/17/node-activity-streams-app-1/

Part 2: App 2 and walkthrough http://blog.cloudfoundry.com/2012/06/05/node-activity-streams-app-2/

Part 3: http://asms.cloudfoundry.com


http://node-express-monica.cloudfoundry.com

http://node-express-monica.cloudfoundry.com

http://blog.cloudfoundry.com/2012/05/17/node-activity-streams-app-1/




http://node-express-start.cloudfoundry.com/

http://node-express-start.cloudfoundry.com/





http://asms.cloudfoundry.com

http://asms.cloudfoundry.com


Thank You - Questions ?Cloud Foundry Activity Streams Libraries are at:

https://github.com/organizations/cloudfoundry-samples

http://cloudfoundry.com

Questions: @cloudfoundry, @ciberch or @andypiper








http://twitter.com/cloudfoundry

http://twitter.com/cloudfoundry

http://twitter.com/ciberch

http://twitter.com/ciberch


ResourcesPrimary Site: cloudfoundry.com

Open Source Site: cloudfoundry.org

Twitter: @cloudfoundry or #cloudfoundry

Blog: blog.cloudfoundry.com

FB: facebook.com/cloudfoundry

Documentation: docs.cloudfoundry.com

Full “bootcamp” presentation: slidesha.re/cf-bootcamp


http://cloudfoundry.com/

http://cloudfoundry.com/

http://cloudfoundry.org

http://cloudfoundry.org

https://twitter.com/%23!/search/cloudfoundry

https://twitter.com/%23!/search/cloudfoundry

http://blog.cloudfoundry.com

http://blog.cloudfoundry.com

http://facebook.com/cloudfoundry

http://facebook.com/cloudfoundry

http://docs.cloudfoundry.com

http://docs.cloudfoundry.com

http://slidesha.re/cf-bootcamp

http://slidesha.re/cf-bootcamp

MongoUK 2012

Technology

Transcript of MongoUK 2012