S8223: Simulating a City:

GPU Simulations of Traffic,

Crowds and BeyondDr Paul Richmond

Contributors: Peter Heywood, Robert Chisholm, Mozhgan Kabiri-Chimeh, John Charlton & Steve Maddock

Context: Everyone hates delays…

Road Travel Projections for 2040 [1]Up to 42% increase of car ownership

19 to 55% growth of UK road traffic

Demand will be greater than supply

Need to improve the supply

[1] – Department for Transport, “Road traffic forecasts 2015.” https://www.gov.uk/government/uploads/system/ uploads/attachment_data/file/260700/road-transport-forecasts-2013-extended-version.pdf, Mar. 2015.

Why City Simulations?

£709M costs for maintaining UK strategic Motorway and A-Road network in 2010/2011 [1]Changing and maintaining infrastructure is very

expensive!

Simulation allowsUnderstanding of the impact of changes to the

transport networkUnderstanding of the impact of infrastructure

investmentIntelligent management of vehicles, people and public

transportInvestigation of shocks to the system (unusual or

dangerous event)All in a safe environment with minimal cost and

disruption

© Highways England

[1] - UK Department for Transport, “Cost of maintaining the Highways Agency’s motorway and A road network per lane mile.” https://www.gov.uk/government/publications/ cost-of-maintaining-the-highways-agency-s-motorway-and-a-road-network-per-lane-mile,2011.

The Role of GPUs – Talk Outline

Acceleration of existing city simulation tools

The development of new simulation approaches

Removal of simulation bottlenecks to do new things

MacroscopicConcerned with flows and

networks

MesoscopicIn the middleCombine aspects of micro models

E.g. Junction flow delays

MicroscopicIndividual Level Modelling (Agent

Based Modelling)Car behaviours, local interactions,

pedestrian behaviourComputationally expensive

Macro, Meso, Micro: How to Simulate a City

Top

Do

wn

Bo

tto

m U

p

Macro/Mesoscopic simulation platform

Used by Highways Agency, TfL and local councils

PerformanceOptimised with OpenMP for multi-coreDemonstrates diminishing returnsRequirement to scale to regional and

national modelsLoHAM is Londan and surrounding area

0123456789

10111213

0 1 2 3 4 5 6 7 8 9 10 11 12 13

Spee

du

p c

om

pa

red

to

ser

ial

Thread Count

Single Assignment-Loop Speedup against Thread Count (LoHAM model)

i7 6850k (6 cores - 12 threads) - Average of 3 repetitions

Research Funded by ATKINS Global

Iterative convergent algorithmTypical within transport modelling

SimulationAdjusts network flow costsSimulates junction delays based off demand

Assignment (97% serial runtime)Inputs

Network and cost of links/edgesOrigin-Destination Matrix

Output Traffic flow per edge1. Single Source Shortest Path - SSSP

All-or-Nothing Path For each origin in the O-D matrix

2. Flow (Accumulation) Apply the OD value for each trip to each link on the route

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Serial

Runtime (LoHAM model)

Flow SSSP

Assignment

Simulation

Computational Bottleneck

Traffic flow per link (road) Flow delay costs

Single Source Shortest Path

For a single Origin/Source Node (N1)Find the route to each Destination Node (N7)

With the Lowest Cumulative Weight (Cost)

N1

N2

N3

N4

N5

N6

N8

N70.1

0.10.7

0.2

0.4

0.20.3

0.3

Node Back link Cost

N1 ∞

N2 N3 0.1

N3 N1 0.1

N4 N3 0.7

N5 N4 0.2

N6 N4 0.4

N7 N6 0.3

N8 N6 0.2

Calculate for all source destination pairs

Multi-GPU Origin-Vertex Frontier

Gunrock [1], NVGraph: State of the art SSSP implementationsFavour large dense graphs (e.g.

social media graphs)Transport networks are high-

diameter low density E.g. LoHAM: TODO

Origin-Vertex FrontierCombines many (each origin) SSSP

calculations to increase parallel workload

Each origin-vertex for consideration stored in a frontier.

Sorted each iteration to improve data access pattern

Co-operative thread array (CTA) used to balance workload

Multi-GPU: Balanced distribution of multiple vehicle classes

[1] – GTC 2016: ID S6374

Results

The Role of GPUs – Talk Outline

Acceleration of existing city simulation tools

The development of new simulation approaches

Removal of simulation bottlenecks to do new things

Flexible Micro-simulation with FLAME GPU

High Level Description of AgentsRepresent the properties of

Pedestrians/Vehicles. E.g. location, velocity, etc.

High Level Scripting of BehaviourThe model dynamics such as avoidance,

car following, navigation, etc.Communication abstracted as messages

to hide underlying algorithm

Automated GPU Code GenerationComplete abstraction from the GPUTemplates generate CUDA simulation

code

Road Network Model

Funded by Department for Transport (TTRIG), In collaboration with TSS UK (AimSum)

FLAME GPU vehicle model function

__FLAME_GPU_FUNC__ int checkNextJunction(xmachine_memory_car* agent, xmachine_message_junctionMessage_list* junctionMessage_messages, xmachine_message_junctionMessage_bounds* message_bounds) {

// Get the junction of current vehicleunsigned int junction = get_staticGraph_roadNetwork_edge_destination(agent->edgeId);

// Load the first junction messagexmachine_message_junctionMessage* current_message = get_first_junctionMessage_message(junctionMessage_messages,

message_bounds, junction);

// Iterate over junction messages (vehicles at this junction)while (current_message){

// Check if agent on junction is leading this vehicleif (vehicleIsLeading(agent, current_message)){

updateLeadVehicle(agent, current_message);}

// Get next junction message (vehicles at this junction)current_message = get_next_junctionMessage_message(current_message,

junctionMessage_messages, message_bounds);

}

return 0;}

Road Network Simulation Benchmark

Scalable Manhattan Grid Network

Comparative Implementation in AimSumCommercial Multi-core simulation

platform

Vehicle performing update calculation

Neighbour vehicle considered by network communication messaging

Neighbour vehicle considered by fixed radius communication messaging

Neighbour vehicle considered by all to all communication messaging

Performance results

Up to 65x Speedup

Largest model: 576k Vehicles and ~2M detectors

Real Time Ratio39x Titan V26x P100

Speedup Over Multi-core (512k vehicles)65x Titan V44x P100

Peter Heywood, Steve Maddock, Jordi Casas, David Garcia, Mark Brackstone, Paul Richmond, Data-parallel agent-based microscopic road network simulation using graphics processing units (2018),Simulation Modelling Practice and Theory, Volume 83, pp 188-200, https://doi.org/10.1016/j.simpat.2017.11.002.

The Role of GPUs – Talk Outline

Acceleration of existing city simulation tools

The development of new simulation approaches

Removal of simulation bottlenecks to do new things

Fast Accurate Pedestrian Collision Avoidance

Force based pedestrian simulation modelsFast but not accurate in dense conditions

Results in agent overlaps

ORCA[1] ModelOptimal Reciprocal Collision Avoidance

Solve set of linear velocity constraints

Unbalanced workload per agent

FLAME GPU implementation with thread block workload redistribution

Example of simulation Corresponding ORCA lines for agent A

[1] Van Den Berg, Jur, Stephen J. Guy, Ming Lin, and Dinesh Manocha. “Reciprocal N-Body Collision Avoidance.” In Robotics Research, 3–19. Springer, 2011.

SIEMENS Sheffield Advanced Multi-Model Simulator

Multi-modal

Simulations

Observe impact of city wide delays on rail network

Measure individual passenger experience through entire journey

Visualise pedestrian behaviour. E.g. station congestion

Identify improvements for public transport

Investigate major infrastructure investment projects

High Congestion Platform

Low Congestion Platform

Interactive Simulations within Virtual

Environment

Prototype new models such as Autonomous pods

Interaction within VR via OmniDeck 66m 360° treadmill16 sets active triangular rolersUser location tracking

Immersive interaction with crowdsSimulation coupled with

UNREAL engine

Available at the Transport Systems Catapult (UK)

Summary and Future Work

GPUs are a major step change in City simulation performanceExisting tools can offer orders of magnitude in performance improvements

Multi-agent simulation (with FLAME GPU) is flexible and highly efficient for vehicle and pedestrian simulations

Coupling multi-agent simulation with existing modelling adds new ways to explore infrastructure and transportation initiatives

Future Work: Simulation Implications for AICan used to generate realistic training data (from dangerous or rare events)

AI and ML can be coupled to predict simulation outcomes

Vote for our Top 5 Finalist Poster: P8215 Data-Parallel Agent-Based Microscopic Road Network Simulation using GPUs

Questions?