Introduction to the New Asynchronous API in the .NET Driver

Introduction to the New Asynchronous API

in the .NET Driver

Robert Stam

MongoDB, Inc.

Intended audience

• Familiar with C# and .NET

• Familiar with MongoDB in general

• Probably familiar with previous versions of the .NET driver

• Maybe haven’t used async programming before

• Interested in using the 2.0 version of the .NET driver

Topics

• Quick review of async programming in C#

• Benefits of async programming

• Sample data import application using the new async API

• Sample web application using the new async API

• A few advanced async related topics

Quick overview of async programming

T result = SomeMethod();

Task<T> task = SomeMethodAsync();

New async/await keywords

public async Task<T> SomeMethodAsync()

T result = await SomeOtherMethodAsync();

return result;

Benefits of async programming

• Threads are expensive (and often limited)

• A blocked thread is not doing any work

• A blocked thread is still consuming resources

• We can get more work done with fewer threads

• We can get higher throughput with the same hardware

Async does have some overhead

• The compiler transform for async methods is not free

• Measurable, but usually insignificant

• Takeaway is to keep the granularity of async methods coarse

2.0 .NET driver API is async-only

• There is a trend toward providing async only APIs

• (e.g. HttpClient)

• Microsoft is heavily promoting async programming

Sample applications

• Using flight delay data from the Department of Transportation• http://www.transtats.bts.gov/OT_Delay/OT_DelayCause1.asp

• Console application to import the data

• Web application to query the data

• Source code in github• https://github.com/rstam/SampleMongoDBApplication

Sample data

• Flights collection

• Date, Airline, Flight Number, From, To, Scheduled Times, Delays, etc…

• Airlines collection

• Code, Description

• Airports collection

• Code, Description

Data Importer Console Application

• Using async APIs in a console application

• Dropping collections

• Loading collections

• Creating indexes

Using the async API from a console application

public static void Main(string[] args)

MainAsync(args).GetAwaiter().GetResult();

catch (Exception ex)

Console.WriteLine("Exception: {0}", ex);

The top level MainAsync method

private static IMongoClient __client;

private static IMongoDatabase __database;

private static async Task MainAsync(string[] args)

var connectionString = "mongodb://localhost";

__client = new MongoClient(connectionString);

__database = __client.GetDatabase("flights");

await DropCollectionsAsync();

await LoadCollectionsAsync();

await CreateIndexesAsync();

Dropping collections

private static async Task DropCollectionsAsync()

await __database.DropCollectionAsync("flights");

await __database.DropCollectionAsync("airlines");

await __database.DropCollectionAsync("airports");

Loading collections

private static async Task LoadCollectionsAsync()

await LoadFlightsAsync();

await LoadAirlinesAsync();

await LoadAirportsAsync();

Loading the airlines collection

private static async Task LoadAirlinesAsync(string csvFilename)

var airlines = new List<Airline>();

using (var csvReader = new CsvReader(new StreamReader(csvFilename)))

foreach (var airline in csvReader.GetRecords<Airline>())

if (__seenAirlineIds.Contains(airline.Code))

airlines.Add(airline);

var collection = __database.GetCollection<Airline>("airlines");

await collection.InsertManyAsync(airlines);

Loading the flights collection

private static async Task LoadFlightsAsync(string csvFilename){

var collection = __database.GetCollection<Flight>("flights");

var batchSize = 1000;using (var csvReader = new CsvReader(new StreamReader(csvFilename))){

var flights = new List<Flight>();

foreach (var flight in csvReader.GetRecords<Flight>()){

PreprocessFlight(flight);flights.Add(flight);

if (flights.Count == batchSize){

await collection.InsertManyAsync(flights);flights.Clear();

if (flights.Count > 0){

await collection.InsertManyAsync(flights);}

Loading the flights collection

private static async Task LoadFlightsAsync(string csvFilename){

var batchSize = 1000;using (var csvReader = new CsvReader(new StreamReader(csvFilename))){

var flights = new List<Flight>();

foreach (var flight in csvReader.GetRecords<Flight>()){

PreprocessFlight(flight);flights.Add(flight);

if (flights.Count == batchSize){

await collection.InsertManyAsync(flights);flights.Clear();

if (flights.Count > 0){

await collection.InsertManyAsync(flights);}

Preprocessing a flight document

private static void PreprocessFlight(Flight flight)

if (flight.FL_DATE.HasValue)

flight.FL_DATE = DateTime.SpecifyKind(flight.FL_DATE.Value, DateTimeKind.Utc);

if (flight.AIRLINE_ID.HasValue)

__seenAirlineIds.Add(flight.AIRLINE_ID.Value);

if (flight.ORIGIN_AIRPORT_ID.HasValue)

__seenAirportIds.Add(flight.ORIGIN_AIRPORT_ID.Value);

if (flight.DEST_AIRPORT_ID.HasValue)

__seenAirportIds.Add(flight.DEST_AIRPORT_ID.Value);

Preprocessing a flight document

private static void PreprocessFlight(Flight flight)

if (flight.FL_DATE.HasValue)

flight.FL_DATE = DateTime.SpecifyKind(flight.FL_DATE.Value, DateTimeKind.Utc);

if (flight.AIRLINE_ID.HasValue)

__seenAirlineIds.Add(flight.AIRLINE_ID.Value);

if (flight.ORIGIN_AIRPORT_ID.HasValue)

__seenAirportIds.Add(flight.ORIGIN_AIRPORT_ID.Value);

if (flight.DEST_AIRPORT_ID.HasValue)

__seenAirportIds.Add(flight.DEST_AIRPORT_ID.Value);

Creating an index

private static async Task CreateIndexesAsync()

var indexKeysBuilder = Builders<Flight>.IndexKeys;

var indexKeys = indexKeysBuilder

.Ascending(f => f.FL_DATE)

.Ascending(f => f.AIRLINE_ID)

.Ascending(f => f.ORIGIN_AIRPORT_ID)

.Ascending(f => f.DEST_AIRPORT_ID);

await collection.Indexes.CreateOneAsync(indexKeys);

Creating an index

private static async Task CreateIndexesAsync()

var indexKeysBuilder = Builders<Flight>.IndexKeys;

var indexKeys = indexKeysBuilder

.Ascending(f => f.FL_DATE)

.Ascending(f => f.AIRLINE_ID)

.Ascending(f => f.ORIGIN_AIRPORT_ID)

.Ascending(f => f.DEST_AIRPORT_ID);

await collection.Indexes.CreateOneAsync(indexKeys);

Sample Web Application

• Two pages

• Search criteria form

• Search results

Search Criteria Form

Search Results Page

MongoClient lifecycle

public class HomeController : Controller

private const string __connectionString = "mongodb://localhost";

private static Lazy<MongoClient> __client =

new Lazy<MongoClient>(() => new MongoClient(__connectionString));

// ... rest of HomeController class

Creating the Search Criteria Form

public async Task<ActionResult> Index()

var viewModel = await CreateIndexViewModelAsync();

return View(viewModel);

public class IndexViewModel

public IEnumerable<Airline> Airlines { get; set; }

public IEnumerable<Airport> Airports { get; set; }

Creating the Search Criteria Form

public async Task<ActionResult> Index()

var viewModel = await CreateIndexViewModelAsync();

return View(viewModel);

public class IndexViewModel

public IEnumerable<Airline> Airlines { get; set; }

public IEnumerable<Airport> Airports { get; set; }

Creating the IndexViewModel

private async Task<IndexViewModel> CreateIndexViewModelAsync()

// note: the two queries will be run in parallel

var findAirlinesTask = FindAirlinesAsync();

var findAirportsTask = FindAirportsAsync();

await Task.WhenAll(findAirlinesTask, findAirportsTask);

return new IndexViewModel

Airlines = findAirlinesTask.Result.OrderBy(a => a.Description),

Airports = findAirportsTask.Result.OrderBy(a => a.Description)

Creating the IndexViewModel

private async Task<IndexViewModel> CreateIndexViewModelAsync()

// note: the two queries will be run in parallel

var findAirlinesTask = FindAirlinesAsync();

var findAirportsTask = FindAirportsAsync();

await Task.WhenAll(findAirlinesTask, findAirportsTask);

return new IndexViewModel

Airlines = findAirlinesTask.Result.OrderBy(a => a.Description),

Airports = findAirportsTask.Result.OrderBy(a => a.Description)

Finding the airlines

private async Task<IEnumerable<Airline>> FindAirlinesAsync()

var client = __client.Value;

var database = client.GetDatabase("flights");

var collection = database.GetCollection<Airline>("airlines");

return await collection.Find(new BsonDocument()).ToListAsync();

The Index view

@model WebApplication.Models.IndexViewModel

@{Layout = null;ViewBag.Title = "Flight Delays";

@using (Html.BeginForm("Search", "Home")){

<div>Search flight delay data:

</div><div>

@Html.Label("From Date:")@Html.TextBox("FromDate")

</div><div>

@Html.Label("To Date:")@Html.TextBox("ToDate")

</div><div>

@Html.Label("Airline:")@Html.DropDownList("AirlineId", Model.Airlines.Select(a => new SelectListItem { Value = a.Id.ToString(), Text = a.Description }), "All")

</div><div>

@Html.Label("Origin:")@Html.DropDownList("OriginId", Model.Airports.Select(a => new SelectListItem { Value = a.Id.ToString(), Text = a.Description }), "All")

</div><div>

@Html.Label("Destination:")@Html.DropDownList("DestinationId", Model.Airports.Select(a => new SelectListItem { Value = a.Id.ToString(), Text = a.Description }), "All")

</div><div>

Creating the Search Results Page

[HttpPost]

public async Task<ActionResult> Search(SearchCriteriaModel criteriaModel)

var viewModel = await CreateSearchResultViewModelAsync(criteriaModel);

return View("SearchResultView", viewModel);

public class SearchResultViewModel

public int TotalNumberOfFlights { get; set; }

public int TotalNumberOfDelayedFlights { get; set; }

public double AverageDelayInMinutes { get; set; }

Creating the Search Results Page

[HttpPost]

public async Task<ActionResult> Search(SearchCriteriaModel criteriaModel)

var viewModel = await CreateSearchResultViewModelAsync(criteriaModel);

return View("SearchResultView", viewModel);

public class SearchResultViewModel

public int TotalNumberOfFlights { get; set; }

public int TotalNumberOfDelayedFlights { get; set; }

public double AverageDelayInMinutes { get; set; }

Creating the Search Results View Model

private async Task<SearchResultViewModel> CreateSearchResultViewModelAsync(SearchCriteriaModel criteriaModel)

{var client = __client.Value;var database = client.GetDatabase("flights");var collection = database.GetCollection<Flight>("flights");

var aggregateOfFlight = collection.Aggregate();

var filter = CreateFilter(criteriaModel);if (filter != null){

aggregateOfFlight = aggregateOfFlight.Match(filter);}

var aggregateOfSearchResultViewModel = aggregateOfFlight.Group(f => 1,g => new SearchResultViewModel{

TotalNumberOfFlights = g.Count(),TotalNumberOfDelayedFlights = g.Sum(f => f.ArrivalDelay > 0.0 ? 1 : 0),AverageDelayInMinutes =

(double)g.Average(f => f.ArrivalDelay > 0.0 ? (double?)f.ArrivalDelay : null)});

return await aggregateOfSearchResultViewModel.SingleAsync();}

A sample resulting aggregation pipeline

{ $match : {

FL_DATE : {

$gte : ISODate("2014-12-01T00:00:00Z"),

$lte : ISODate("2014-12-31T00:00:00Z") },

AIRLINE_ID : 19790, /* Delta Airlines */

ORIGIN_AIRPORT_ID : 10397, /* Atlanta Hartsfield International */

DEST_AIRPORT_ID : 12953 } /* New York La Guardia */

{ $group : {

_id : 1,

TotalNumberOfFlights : { $sum : 1 },

TotalNumberOfDelayedFlights :

{ $sum : { $cond : [{ $gt : ["$ARR_DELAY", 0.0] }, 1, 0] } },

AverageDelayInMinutes :

{ $avg : { $cond : [{ $gt : ["$ARR_DELAY", 0.0] }, "$ARR_DELAY", null] } } }

Creating the Search Results Filter

private FilterDefinition<Flight> CreateFilter(

SearchCriteriaModel criteriaModel)

var filterBuilder = Builders<Flight>.Filter;

var clauses = CreateClauses(criteriaModel, filterBuilder);

if (clauses.Count > 0)

return filterBuilder.And(clauses);

return null;

Creating the clauses

private List<FilterDefinition<Flight>> CreateClauses(SearchCriteriaModel criteriaModel,FilterDefinitionBuilder<Flight> filterBuilder)

{var clauses = new List<FilterDefinition<Flight>>();if (criteriaModel.FromDate != null){

var fromDate = DateTime.SpecifyKind(DateTime.Parse(criteriaModel.FromDate), DateTimeKind.Utc);

var clause = filterBuilder.Gte(f => f.FlightDate, fromDate);clauses.Add(clause);

}if (criteriaModel.ToDate != null){

var toDate = DateTime.SpecifyKind(DateTime.Parse(criteriaModel.ToDate), DateTimeKind.Utc);

var clause = filterBuilder.Lte(f => f.FlightDate, toDate);clauses.Add(clause);

}// code for 3 more clauses (see next slide)return clauses;

Creating the clauses (continued)

{// ... (continued from previous slide)

if (criteriaModel.AirlineId != null){

var clause = filterBuilder.Eq(f => f.AirlineId, criteriaModel.AirlineId.Value);clauses.Add(clause);

}if (criteriaModel.OriginId != null){

var clause = filterBuilder.Eq(f => f.OriginAirportId, criteriaModel.OriginId.Value);clauses.Add(clause);

}if (criteriaModel.DestinationId != null){

var clause = filterBuilder.Eq(f => f.DestinationAirportId, criteriaModel.DestinationId.Value);

clauses.Add(clause);}return clauses;

The SearchResult view

@model WebApplication.Models.SearchResultViewModel

Layout = null;

<!DOCTYPE html>

<html>

<head>

<title>SearchView</title>

</head>

<body>

Search Results:

</div>

@Model.TotalNumberOfFlights total number of flights

</div>

@Model.TotalNumberOfDelayedFlights total number of delayed flights

</div>

@Model.AverageDelayInMinutes average delay (in minutes)

</div>

@Html.ActionLink("New search", "Index")

</div>

</body>

</html>

Advanced Topics

• IAsyncCursor

• Cancellation

• Timeouts using CancellationToken

• ConfigureAwait(false)

IAsyncCursor

public interface IAsyncCursor<out TDocument> : IDisposable

IEnumerable<TDocument> Current { get; }

Task<bool> MoveNextAsync();

using (var cursor = await collection.FindAsync(filter))

while (await cursor.MoveNextAsync())

var batch = cursor.Current;

foreach (var document in batch)

// process document

IAsyncCursor

public interface IAsyncCursor<out TDocument> : IDisposable

IEnumerable<TDocument> Current { get; }

Task<bool> MoveNextAsync();

using (var cursor = await collection.FindAsync(filter))

while (await cursor.MoveNextAsync())

var batch = cursor.Current;

foreach (var document in batch)

// process document

Cancellation

using (var cancellationTokenSource = new CancellationTokenSource())

var cancellationToken = cancellationTokenSource.Token;

var task = SomeMethodAsync(cancellationToken);

// ...

cancellationTokenSource.Cancel(); // request cancellation

var result = await task;

Cancellation

// ...

Cancellation

// ...

Timeouts using CancellationToken

using (var timeoutSource = new CancellationTokenSource(timeout))

var cancellationToken = timeoutSource.Token;

var filter = ...;

var cursor = await collection.FindAsync(filter, cancellationToken);

// or: search the Web for “WithTimeout” helper method

var cursor = await collection.FindAsync(filter).WithTimeout(timeout);

Timeouts using CancellationToken

using (var timeoutSource = new CancellationTokenSource(timeout))

var cancellationToken = timeoutSource.Token;

var filter = ...;

var cursor = await collection.FindAsync(filter, cancellationToken);

// or: search the Web for “WithTimeout” helper method

var cursor = await collection.FindAsync(filter).WithTimeout(timeout);

ConfigureAwait(false)

Which thread runs the code after an await?

Depends on the captured SynchronizationContext.

We use ConfigureAwait(false) 100% of the time inside the

driver.

Improves performance. Helps prevent deadlocks.

.NET Users Birds of a Feather

Tuesday 12:20 pm to 1:50 pm

Metropolitan East Ballroom

Conclusion

Thanks for coming!

Questions?

Introduction to the New Asynchronous API in the .NET Driver

Technology

Transcript of Introduction to the New Asynchronous API in the .NET Driver