Two-Way Transit Signal Priority Algorithm for

Optimizing Transit Reliability and Speed A Deep Reinforcement Learning Approach

Wen Xun Hu, Chuhan Chen, Hirotaka Ishihara, Islam Taha, Amer Shalaby and Baher Abdulhai

Motivation

The transit

reliability and

speed are

performance

indicators

Transit

services are

vulnerable to

variability and

delays

No strategies can

adaptively improve

headway regularity

and reduce signal

delays

simultaneously

The conventional

TSP only aims at

reducing delays

Multiple TSP

requests from

different

approaches are

commonly

handled by

FCFS logic

Model of the

dynamics of

transportation

environment

Objectives

▪ Dual-objective TSP

– Adaptively optimize reliability (i.e., headway regularity) and

reduce signal delays simultaneously

▪ Coordination of opposite directions

– Develop an algorithm to coordinate TSP in opposite directions

of the same intersection based on real-time bus performance

Objectives

▪ High speed, poor reliability

– Pros

• Reduced in-vehicle travel time

– Cons

• Long waiting time

• Uneven bus loads (e.g., crowding)

• Operating costs

• etc.

Source: https://www.thestar.com/news/gta/2020/11/27/the-ttc-wants-to-test-

platoons-of-driverless-buses.html

https://www.thestar.com/opinion/star-columnists/2017/12/15/are-we-sure-ttc-fare-

evasion-happens-as-often-as-we-think-it-does.html

Objectives

▪ Reliability vs. Speed

– One directionScenario A

A bus arrives at the intersection with a headway (hin) > scheduled headway (hs).

• To improve speed and reduce headway deviation, the TSP system should expedite this bus.

Scenario B

hin < hs：• To improve speed or reduce signal delays, the TSP system should expedite this bus.

• To reduce headway deviation, the TSP system should hold back (i.e., slow down) this bus.

– Opposite directions• More conflicting scenarios

Model FormulationOne-Way DRL Agent

POZ

Check-out

Loop detector

Check-in

Loop

detector

• Model-free deep reinforcement learning

• Efficient for large state space

Model FormulationOne-Way DRL Agent

• Target travel time to maintain scheduled

headway at check-out

• Number of vehicles in the POZ

• Time to the end of 1st available green

reward =

𝑤ℎ ℎ𝑒𝑎𝑑𝑤𝑎𝑦 𝑖𝑚𝑝𝑟𝑜𝑣𝑒𝑚𝑒𝑛𝑡 − 𝑤𝑡(𝑡𝑖𝑚𝑒 𝑖𝑛 𝑃𝑂𝑍)

Transportation network

simulated in Aimsun

TSP control system

Green phase:

-20, -15, -10, -5, 0, +5,

+10, +15, +20 sec

←RL image:

https://shield.ai/content/2018/11/20/shield-ai-

fundamentals-on-reinforcement-learning

Source: Pawel Czerwinski,

unsplash.com

Signal cycle image:

https://winnipeg.ca/publicworks/transportati

on/trafficsignals/trafficsignaloperation.stm

Model FormulationCoordination Algorithm

RL agent X

Coordination

algorithm

𝑄 𝑠𝑡 , 𝑎; 𝜃𝑡

𝑄 𝑠𝑢 , 𝑎; 𝜃𝑢

𝐀𝐜𝐭𝐢𝐨𝐧: 𝒂𝒕′

= 𝑎𝑟𝑔𝑚𝑎𝑥𝑎

𝑤𝑥𝑄 𝑠𝑡 , 𝑎; 𝜃𝑡 + 𝑤𝑦𝑄 𝑠𝑢, 𝑎; 𝜃𝑢RL agent Y

Simulation case study

▪ Aimsun Next Microsimulation

▪ Intersection in Toronto

▪ Bus line: 36 Finch West

Finch Ave West at Kipling Ave

Results

▪ Base scenarios

– No TSP

– TSP in field (Toronto TSP)

– DRL agents + FCFS logic (FCFS TSP)

▪ Proposed scenario

– DRL agents + coordination algorithm (D2 TSP)

Source: Pawel Czerwinski,

unsplash.com

Source: Andre Furtado, unsplash.com

Results

Coefficient of Variation of Headway

0.461

0.439

0.4170.408

0.502 0.505

0.482

0.456

0.40

0.42

0.44

0.46

0.48

0.50

No TSP Toronto FCFS D2 TSP

EB WB

Results

% of Extreme Headways

3.4%

18.3%

22.1%

8.3%

3.8%

16.8%

20.8%

6.7%

1.7%

13.9%

16.9%

7.3%

1.2%

13.1%

15.4%

5.9%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

h < 0.25 hs h < 0.5 hs h > 1.5 hs h > 1.75 hs

No TSP Toronto FCFS D2 TSP

Results

Travel Time90.7

66.4

77.9

73.1

91.3

60.257.7

62.5

55.0

60.0

65.0

70.0

75.0

80.0

85.0

90.0

95.0

No TSP Toronto FCFS D2 TSP

EB WB

9 s travel time ↑,

23 s stdev of h dev↓ = 10 s waiting time↓

Conclusions

The proposed Two-Way TSP (D2 TSP)

▪ Generates the best headway performance in both directions

– Effective in reducing

headway variability and

% of extreme headways

▪ Brings noticeable reduction in travel time compared with “No TSP”

Source: Kayla Speid, unsplash.com

Future Work

▪ Coordinated TSP systems to enhance the benefit on transit reliability and speed at the route level

▪ Use connected vehicle technologies for detection and communication

▪ Integrate TSP design with other route strategies

Source: Nong Vang, unsplash.com

THANK YOU!

wenxun.hu@mail.utoronto.ca

16

Source: chuttersnap, unsplash.com