Post on 17-May-2015
description
© 2012 VMware, Inc. All rights reserved
Activity Streams on Cloud Foundry with MongoDB and Node.jsMongoUK - 2012
By Monica Wilkinson and Andy Piper
Wednesday, June 20, 12
CONFIDENTIAL
Andy Piper
2
social bridgebuilder, techie
Developer Advocate @ Cloud Foundry
15 years in enterprise middleware & messaging
social web enthusiast
OSS supporter / contributor
excited by “what’s next”, Internet of Things, etc.
@andypiper
apiper@vmware.com
Wednesday, June 20, 12
CONFIDENTIAL
About Monica Wilkinson
3
Loves the web and data portability.
Developer Advocate @ Cloud Foundry12 years development experience.
Last 5 years in Social WebOpen Web Standards Advocate
Contact Me: @ciberchmwilkinson@vmware.com
Wednesday, June 20, 12
CONFIDENTIAL4
Agenda
1.Overview of Cloud Foundry2.What is Activity Streams ?3.Building an Activity Stream Engine
Wednesday, June 20, 12
CONFIDENTIAL
About Cloud Foundry
5
The first Open PaaS
Multi(n) Languages, Frameworks, Services & Clouds
Open Source
Wednesday, June 20, 12
Traditional web app architecture! Browser client
! Monolithic WAR / other deployment artefact
! RDBMS
Container (e.g. Tomcat)
CONFIDENTIAL
6
Desktop Browser Apache
Web Application
MySQL Database
ShippingService
AccountingService
InventoryService
StoreFront
“easy” to...
! develop
! test
! deploy
! scale ...apart from: provisioning, setup, need to replicate, shard, scale to massive numbers of mobile clients, provide polyglot support, run multiple dev teams, collaborate...
Wednesday, June 20, 12
CONFIDENTIAL
Need to handle massive loads and the data explosion
7
Scalable architectures!
! Application tier:
•Replicated/clustered servers
•Modular so that components can be scaled differently
•Asynchronous architecture - communication via a message broker
! Database tier:
•Replication
•Sharding
•Polyglot persistence: Relational, NoSQL, NewSQL databases
Wednesday, June 20, 12
CONFIDENTIAL
Desktop Browser
NodeJS
front-end application
Native Mobile Application
HTML5 mobile application
RabbitMQ
NodeJS
MySQL
Mongo
Redis
ShippingService
Accounting
Service
billing web application
shipping web application
InventoryService
inventory web application
StoreUI
StoreUI
StoreUI
StoreUI
8
Modern application architecture
Wednesday, June 20, 12
CONFIDENTIAL9
Platform-as-a-Service is the solution
Easy deployment
Application management
Easy scaling up and down
+
SQL databases
NoSQL databases
Message Broker
Deployment Services
Wednesday, June 20, 12
CONFIDENTIAL
10
“Deploy and scale applications in seconds, without locking yourself into a single cloud or vendor”
Simple, Open, Flexible,
Scalable
The Open Platform as a Service
Wednesday, June 20, 12
CONFIDENTIALApplication Service Interface
Data Services
Other Services
Msg Services
Cloud Foundry open PaaS - Choice of clouds
Private(Clouds(
PublicClouds
MicroClouds
Clou
d Pr
ovid
er In
terfa
ce (C
PI)
NEW! bosh: deploy to
alternate CPIs (AWS etc)
OSS community
vFabric Postgres
vFabric RabbitMQTM
Additional partner services …
Wednesday, June 20, 12
CONFIDENTIAL
12
Multi-cloud flexibility is critical to long-term success
! Make use of both public and private clouds without rewriting your applications
! Protect against vendor lock-in
! Meet different compliance and geographical needs
! Accommodate peak loads while optimizing costs
! Manage your growth and changing needs over time
Wednesday, June 20, 12
CONFIDENTIAL13
Getting Started on Cloud Foundry
Wednesday, June 20, 12
CONFIDENTIAL
14
http://cloudfoundry.com/signup
optional promo
Register!
Wednesday, June 20, 12
CONFIDENTIAL
Ways to use Cloud Foundry
15
command line...
IDE plugin...
toolchain plugin...
Cloud IDEs
Wednesday, June 20, 12
CONFIDENTIAL
Example vmc commands
16
$"gem"install"vmc
$"vmc"target"<any"cloud>$"vmc"login"<credentials>$"vmc"push"<my7app>" "">"bind"services?"Yes
$"vmc"update"<my7app>$"vmc"instances"<my7app>"+100"
Wednesday, June 20, 12
CONFIDENTIAL
Logging into CloudFoundry.com
17
$"vmc"target"api.cloudfoundry.comSuccessfully"targeted"to"""""""""[http://api.cloudfoundry.com]
$"vmc"loginAttempting"login"to"""[http://api.cloudfoundry.com]Email:"apiper@vmware.comPassword:"********Successfully"logged"into"""[http://api.cloudfoundry.com]
Wednesday, June 20, 12
CONFIDENTIAL
Cloud Foundry Integration plugin for Eclipse
18
Wednesday, June 20, 12
CONFIDENTIAL19
What is Activity Streams ?
Wednesday, June 20, 12
CONFIDENTIAL20
http://activityStrea.ms! Activity Streams is a simple open specification used to describe social
actions around the web.
! The goal of Activity Streams is data portability
! Some publishers of Activity Streams: Socialcast, MySpace, Facebook and G+
! The default format is JSON but Atom is also supported. The JSON format was added after we saw some initial limitations with JSON
! Core concepts are: Actor, Verb, Object and Target
Wednesday, June 20, 12
© 2012 VMware, Inc. All rights reserved
Actor -> Monica
Verb -> Pinned(Bookmarked)
Object -> Rosa Parks’ Bio Page
Target -> Female Leaders Board
Source -> Pinterest
Published -> MLK Day 2012
Wednesday, June 20, 12
© 2012 VMware, Inc. All rights reserved
With Timeline ActionsPosting with a richer protocol like
Activity Streams allows you to do:
- Richer messages and links
- Aggregate
- Assign relevance
Wednesday, June 20, 12
CONFIDENTIAL23
Example in JSON
Wednesday, June 20, 12
CONFIDENTIAL24
What is an Activity Stream Engine ?
! An activity stream engine allows you to publish events and subscribe to events.
! Many activity stream engines support aggregation of events via Streams.
! Many activity stream engines support fanning out activities to subscribers.
Wednesday, June 20, 12
CONFIDENTIAL25
So how did we build this Activity Stream Engine on Cloud Foundry ?
Wednesday, June 20, 12
CONFIDENTIAL26
Building an Activity Streams Engine
Wednesday, June 20, 12
CONFIDENTIAL27
What should we use to store Activities ?
ColumnKey-Value Document Graph
Redis, Riak
Cassandra,HBase
MongoDB Neo4J
Wednesday, June 20, 12
CONFIDENTIAL28
ColumnKey-Value Document Graph
MongoDB (so easy)
Mongo DB schema-less documents are perfect for Activities
Wednesday, June 20, 12
CONFIDENTIAL
About MongoDB
29
1. No-SQL database
2. Stores JSON-style documents with embedded documents
3. Horizontally scalable
4. Full Indexing Support
5. Open Source so great ORMs and drivers.
development : {
tools: many,
language_support: superb,
agility: high
},
production: {
speed: fast
fault_tolerance: true
scalability: high
=
Wednesday, June 20, 12
CONFIDENTIAL
Step 1- Build the persistence Modelvar MediaLinkHash = {
duration: Number, height: Number, width: Number, url: String };
var ActivityObjectHash = { id: {type: String}, image: MediaLinkHash, displayName: {type: String}, summary: {type: String}, content: {type: String}, url: {type:String}, author: {type: ObjectId, ref: 'activityObject'}, published: {type: Date, default: Date.now}, updated: {type: Date, default: Date.now}, objectType: {type: String}, attachments: [{type: ObjectId, ref: 'activityObject'}], upstreamDuplicates: [String], downstreamDuplicates: [String] };
var ActivityObjectSchema = new Schema(ActivityObjectHash) this.ActivityObject = mongoose.model('activityObject', ActivityObjectSchema);
30
Wednesday, June 20, 12
CONFIDENTIAL
var ActivitySchema = new Schema({ id: {type: String}, verb: {type: String}, url: {type: String}, title: {type: String}, content: {type: String}, icon: MediaLinkHash, object: ActivityObjectHash, actor: ActivityObjectHash, target: {type: ObjectId, ref: 'activityObject'}, published: { type: Date, default: Date.now}, updated: { type: Date, default: Date.now}, inReplyTo: {type: ObjectId, ref: 'activity'} });
this.Activity = mongoose.model('activity', ActivitySchema);
return this;};
31
Wednesday, June 20, 12
CONFIDENTIAL
Step 2 - Expose helpers for the queries! The most important is to be able to list the activities in
descending order by published time.! It is also important to hydrate any objects for which we
have references
! With Mongoose you can do: ! this.getActivityStream = function(n, fx) { Activity.find().sort('published', 'descending').limit(n).populate('target').run(fx);
}! https://github.com/ciberch/activity-streams-mongoose
32
Wednesday, June 20, 12
CONFIDENTIAL
How do we make our engine faster and more scalable ?! Add support for PubSub! In a PubSub model as soon as an event is published it
gets sent to all the subscribers.! The Publisher doesn’t know who the subscribers are, it
simply knows where to publish.! Redis has nice support for Pub Sub http://redis.io/topics/
pubsub
33
Wednesday, June 20, 12
CONFIDENTIAL
Using Socket.io
! Server Side:• Subscribe to a Stream• When there is a new event send it to the client as JSON
! Client Side• Add support to publish activities to the Stream• Change the client rendering to use ActivityStrea.ms
! Benefits• Richer messaging• Activity Syndication
34
Wednesday, June 20, 12
© 2012 VMware, Inc. All rights reserved
DemoPart 1: App 1 and walkthrough http://blog.cloudfoundry.com/
2012/05/17/node-activity-streams-app-1/
Part 2: App 2 and walkthrough http://blog.cloudfoundry.com/2012/06/05/node-activity-streams-app-2/
Part 3: http://asms.cloudfoundry.com
Wednesday, June 20, 12
© 2012 VMware, Inc. All rights reserved
Thank You - Questions ?Cloud Foundry Activity Streams Libraries are at:
https://github.com/organizations/cloudfoundry-samples
http://cloudfoundry.com
Questions: @cloudfoundry, @ciberch or @andypiper
Wednesday, June 20, 12
© 2012 VMware, Inc. All rights reserved
ResourcesPrimary Site: cloudfoundry.com
Open Source Site: cloudfoundry.org
Twitter: @cloudfoundry or #cloudfoundry
Blog: blog.cloudfoundry.com
FB: facebook.com/cloudfoundry
Documentation: docs.cloudfoundry.com
Full “bootcamp” presentation: slidesha.re/cf-bootcamp
Wednesday, June 20, 12