Yves GoelevenCapgemini@YvesGoelevenhttp://cloudshaper.wordpress.com

Code D’Azure

Simplifying distributed applications with the Windows Azure Platform and NServiceBus

Agenda

• Introduction• What is NServiceBus and why is it so

azurey?• Communication patterns• The basics of NServiceBus

development• Azure specific implementation details• Play Time

What is a service bus ?

• An architectural pattern– Simplifies communication– Between services and their consumers

• Lousely coupled messaging– In space (contract)– And time (queues)

Still, a lot of distractions...

• Distributed application development is not that easy– Multithreaded message handling,

addressing & routing, transaction scoping, correlation, message header management, serialization & version tolerance, subscription management, persisting state of long running communication, timeouts, …

• All distractions from what really matters: The business logic

What is NServiceBus?

• A logical service bus– A framework to support this architectural

pattern– Abstract underlying infrastructure

(queues, broker)

• Ambitions– Make development of scalable, distributed

apps SIMPEL– So that developers can focus on their

business domain & logic

NServiceBus Structure

Core

TimeoutManager

DynamicHost

...

...

Host

Why is it so azurey?

• We share a common goal (but at a different level)– Make it easy to build scalable distributed

apps

• Azure provides all primitives NServiceBus needs– Durable queues: Storage, AppFabric– Storage: Table, Relational, Blob

Agenda

• Introduction• What is NServiceBus and why is it so

azurey?• Communication patterns• Azure specific implementation details• The basics of NServiceBus

development• Play Time

Moderatly Opinionated...

• Components are autonomous– Nothing shared : Own process, queue, data

store, ...

• Communication between components is– Message based : Asynchronous, Queued,

Durable

• Only a handful of patterns are supported– On purpose

One-way (Send)

Worker

Worker

Worker

Client

Client

Client

Queue

Full Duplex (Send / Reply)

Worker

Worker

Worker

Client

Client

Client

Queue

Queue

Pub / Sub (Publish)

Worker

Worker

Worker

Client

Client

Client

Queue

Queue

Queue

Saga

Worker

Worker

Worker

Client

Client

Client

Queue

Queue

Queue

DataBus

Worker

Worker

Worker

Client

Client

Client

Queue

FileStore

Agenda

• Introduction• What is NServiceBus and why is it so

azurey?• Communication patterns• The basics of NServiceBus

development• Azure specific implementation details• Play Time

Define message contracts...

• Implement IMessage– You can choose your own interface if you

like

public class ImageUploaded : IMessage { public Guid Id { get; set; } public string FileName { get; set; } public string ContentType { get; set; } public byte[] Image { get; set; } }

Handle messages

• Implement IHandleMessages<T>– In any assembly referenced by a process

that has a bus configured

public class CreateLargeThumbnail : IHandleMessages<ImageUploaded>{ public void Handle(ImageUploaded message) { // do something } }

• Using the IBus interface– Which is injected by the container

Send or publish messages

public class CreateLargeThumbnail : IHandleMessages<ImageUploaded> { private readonly IBus Bus;

public void Handle(ImageUploaded message) { // create a large thumbnail

bus.Publish(new ThumbNailCreated()); } }

• Message to endpoint mapping in config

Message Routing

<configSections> <section name="UnicastBusConfig" type="NServiceBus.Config.UnicastBusConfig, NServiceBus.Core"/></configSections>

<UnicastBusConfig> <MessageEndpointMappings> <add Messages="MyFlickr.WebSite.Events" Endpoint="processorinputqueue"/> <add Messages="MyFlickr.Sharing.Events" Endpoint="sharinginputqueue"/> </MessageEndpointMappings></UnicastBusConfig>

The core is a skeleton...

• With some default implementations– But everything is replaceable

• By default– Msmq, Autofac, Log4net,

Nhibernate/RavenDB

• We will replace– By azure’s services

Compose the bus as you like...

• Configure – What you need is defined in code– Details are defined either in config or

codeConfigure.WithWeb() // assemblies to parse .DefaultBuilder() // ioc container .Log4Net(new AzureAppender()) // diagnostics .AzureConfigurationSource() // configuration .AzureMessageQueue() // queue infrastructure .JsonSerializer() // serializer .QueuePerInstance() // addressing behavior .UnicastBus() // how the bus behaves .LoadMessageHandlers() // loads messagehandlers .IsTransactional(true) // transactional behavior.CreateBus() // create an instance of the bus.Start(); // start the instance

Or use a predefined roles & profiles

• Inherit from RoleEntryPoint

• Configure the endpoint

• Customize if required

public class Host : NServiceBus.Hosting.Azure.RoleEntryPoint { }

public class EndpointConfiguration : IConfigureThisEndpoint, AsA_Worker { }

public class SetupDataBus : IWantCustomInitialization { public void Init() { Configure.Instance.AzureDataBus(); }}

Agenda

• Introduction• What is NServiceBus and why is it so

azurey?• Communication patterns• The basics of NServiceBus

development• Azure specific implementation details• Play Time

Configuration

• AzureConfigurationsource()– Uses config section from app.config – And overrides settings from service

configuration file

• Override by convention– <Key Attribute=“value”>– Key.Attribute=“value”

Diagnostics

• Log4Net<AzureAppender>()– Directs output to diagnostics manager– Works with or without .wadcfg file

• Configuration settings– Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString– Microsoft.WindowsAzure.Plugins.Diagnostics.Level– Microsoft.WindowsAzure.Plugins.Diagnostics.Layout– Microsoft.WindowsAzure.Plugins.Diagnostics.ScheduledTransfer

Period– Microsoft.WindowsAzure.Plugins.Diagnostics.EventLogs

Queueing

• AzureMessageQueue()– Uses azure storage queues as infrastructure– Semi-transactional (Peek-lock)– Back-off (Linear)– Batching– Shared or per instance

• ConfigurationSettings– AzureQueueConfig.ConnectionString– AzureQueueConfig.MessageInvisibleTime ( x BatchSize )– AzureQueueConfig.PeekInterval– AzureQueueConfig.MaximumWaitTimeWhenIdle– AzureQueueConfig.BatchSize– AzureQueueConfig.QueuePerInstance

Queueing• AppFabricMessageQueue()

– Uses appfabric queues as infrastructure– Semi-transactional (Peek-lock)– Shared or per instance

• ConfigurationSettings– AppFabricQueueConfig.IssuerName– AppFabricQueueConfig.IssuerKey– AppFabricQueueConfig.ServiceNamespace– AppFabricQueueConfig.LockDuration– AppFabricQueueConfig.MaxSizeInMegabytes– AppFabricQueueConfig.RequiresDuplicateDetection– AppFabricQueueConfig.RequiresSession– AppFabricQueueConfig.DefaultMessageTimeToLive– AppFabricQueueConfig.EnableDeadLetteringOnMessageExpiration– AppFabricQueueConfig.DuplicateDetectionHistoryTimeWindow– AppFabricQueueConfig.MaxDeliveryCount– AppFabricQueueConfig.EnableBatchedOperations– AppFabricQueueConfig.QueuePerInstance

Subscription storage

• AzureSubcriptionStorage()– Uses Nhibernate over table storage

• ConfigurationSettings– AzureSubscriptionStorageConfig.ConnectionStri

ng– AzureSubscriptionStorageConfig.CreateSchema

Subscription storage

• DBSubcriptionStorage()– Nhibernate with connectionstring to SQL Azure

<DBSubscriptionStorageConfig> <NHibernateProperties> <add Key="connection.provider" Value="NHibernate.Connection.DriverConnectionProvider"/> <add Key="connection.driver_class" Value="NHibernate.Driver.SqlClientDriver"/> <add Key="connection.connection_string" Value="Server=tcp:{server}.database.windows.net;

Database={database}; User ID={user}@{server}; Password={password};

Trusted_Connection=False; Encrypt=True;"/> <add Key="dialect" Value="NHibernate.Dialect.MsSql2005Dialect"/> </NHibernateProperties> </DBSubscriptionStorageConfig>

Saga storage

• .Sagas().AzureSagaPersister().NHibernateUnitOfWork()– Uses Nhibernate over table storage

• ConfigurationSettings– AzureSagaPersisterConfig.ConnectionString– AzureSagaPersisterConfig.CreateSchema

Saga storage

• .Sagas().NHibernateSagaPersister().NHibernateUnitOfWork()– With connectionstring to SQL Azure

<NHibernateSagaPersisterConfig> <NHibernateProperties> <add Key="connection.provider" Value="NHibernate.Connection.DriverConnectionProvider"/> <add Key="connection.driver_class" Value="NHibernate.Driver.SqlClientDriver"/> <add Key="connection.connection_string“

Value="Server=tcp:{server}.database.windows.net; Database={database}; User ID={user}@{server}; Password={password}; Trusted_Connection=False; Encrypt=True;"/>

<add Key="dialect" Value="NHibernate.Dialect.MsSql2005Dialect"/> </NHibernateProperties> </NHibernateSagaPersisterConfig>

Databus

• .AzureDataBus()– Large properties stored as files– Using block blobs in blob storage– Blocks uploaded in parallel– Including retry on failure

• ConfigurationSettings– AzureDataBusConfig.ConnectionString– AzureDataBusConfig.MaxRetries– AzureDataBusConfig.BlockSize– AzureDataBusConfig.NumberOfIOThreads– AzureDataBusConfig.Container– AzureDataBusConfig. BasePath

Hosting

• 2 types of hosts– Role dedicated to an endpoint– Role hosts multiple endpoints in multiple

processes

• Both make sense– Limited features, lot’s of users -> Dedicated– Lot’s of features, limited users -> Shared

Hosting

• Generic Roles– AsA_Listener– AsA_Worker

• Specific Roles– AsA_Host– AsA_TimeoutManager

Hosting

• Environment Profiles– Development– Production

• Storage Profiles– OnAzureTableStorage– OnSqlAzure

• Communication Profiles– WithAzureStorageQueues– WithAppFabricQueues

Hosting

• Shared hosting– RoleEntryPoint, configured AsA_Host– Will not start a bus– But will load other NServiceBus processes from blob

storage• ConfigurationSettings

– DynamicHostControllerConfig.ConnectionString– DynamicHostControllerConfig.Container – DynamicHostControllerConfig.LocalResource – DynamicHostControllerConfig.RecycleRoleOnError– DynamicHostControllerConfig.AutoUpdate– DynamicHostControllerConfig.UpdateInterval– DynamicHostControllerConfig.TimeToWaitUntilProcessIsKilled

Agenda

• Introduction• What is NServiceBus and why is it so

azurey?• Communication patterns• The basics of NServiceBus

development• Azure specific implementation details• Play Time

Mision - Recreate Flickr

• The basics of the app– User uploads photo– System stores it and creates thumbnails of

different sizes: Original, Small, Medium, Large– System shares the photo with the world when

all thumbnails have been created (Photostream, ...)

– The system notifies the user and clears it’s caches

Architecture

Worker

Worker

Website

Website

Queue

DataBus

Queue

Queue

Worker

Worker

Queue

Upload image

Photo processing: - Store original- Create thumbnails

Sharing: - Upd. photostream

Notify User & Clear caches

SubscriptionsSaga’s

Worker

WorkerQueue

Timeout Manager

Getting started

• Begin situation– https://goeleven.blob.core.windows.net/demos/

nservicebus/Begin.zip

• Code snippets can be found on– https://gist.github.com/1164912

Yves GoelevenCapgemini@YvesGoelevenhttp://cloudshaper.wordpress.com

Questions?