Post on 13-Jan-2016
ICUAS 2014
Towards Real Time Scheduling for Persistent UAV Service: A Rolling Horizon MILP Approach, RHTA and the STAH Heuristic
Byung Duk Song, Jonghoe Kim, and James R. Morrison*
Department of Industrial and Systems Engineering KAIST, South Korea
Thursday, May 29, 2014
ICUAS 2014 – 2
Presentation Overview
• Motivation
• Goal and ideas
• Mathematical formulation
• Solution approaches: STAH & RHTA
• Numerical examples
• Concluding remarks
ICUAS 2014 – 3
Presentation Overview
• Motivation
• Goal and ideas
• Mathematical formulation
• Solution approaches: STAH & RHTA
• Numerical examples
• Concluding remarks
ICUAS 2014 – 4
Motivation
• Increasing demand for commercial small size UAVs
• Applications: Patrol, tracking, communication relay, environmental / fire / national boundary monitoring, cartography, disaster relief
• Limitations: Flight time & mission duration
• Long duration, multiple customers and persistent service require- Collection of UAVs, refueling stations, automatic operation system- Methods to orchestrate their operations
ICUAS 2014 – 5
Motivation: Application Example (1)
• Example 1: Tracking a moving ground target
Station 1
Station 2
Station 3
ICUAS 2014 – 6
Motivation: Application Example (2)
• Example 2: UAV border patrol
Station 1
Station 2
ICUAS 2014 – 7
Presentation Overview
• Motivation
• Goal and ideas
• Mathematical formulation
• Solution approaches: STAH & RHTA
• Numerical examples
• Concluding remarks
ICUAS 2014 – 8
Goal and Ideas
• Provide uninterrupted UAV service with continuous appearance of mission requests (or unanticipated system state changes)
• Persistent UAV operation via service stations
• Real time service perspective: - Incorporate current UAV information (location, fuel level) - Efficient solving algorithms
• Uninterrupted customer service
■ Goal
■ Ideas
ICUAS 2014 – 9
Presentation Overview
• Motivation
• Goal and ideas
• Mathematical formulation
• Solution approaches: STAH & RHTA
• Numerical examples
• Concluding remarks
ICUAS 2014 – 10
Mathematical Formulation: Assumptions
■ Assumptions
1. Moving target’s path and location at specific times are known.
2. Current location and fuel levels are known.
3. Recharge time for a UAV depends on the remaining fuel(battery) amount
4. UAV travel speed is constant
ICUAS 2014 – 11
Mathematical Formulation: Split Job
Start
End
Service station 1
Service station 2
1
2
3
104
5
6 7
89
Job 1
UAV 1UAV 1
UAV 2
UAV 2
▪ Objective moves
- From point (50,250) to (950,350)
- From 13:10 to 13:20
Split jobStart point
End point
Start time
End time
1 50,250 150,250 13:10 13:11
2 150,250 250,250 13:11 13:12
3 250,250 350,250 13:12 13:13
4 350,250 450,250 13:13 13:14
5 450,250 550,250 13:14 13:15
6 550,250 650,250 13:15 13:16
7 650,250 750,250 13:16 13:17
8 750,250 850,250 13:17 13:18
9 850,250 950,250 13:18 13:19
10 950,250 950,350 13:19 13:20
■ Split job: Moving objective path is divided into set of split jobs
ICUAS 2014 – 12
Mathematical Formulation: Notation
■ Notationi, j : Indices for jobs
s : Index for stations
k : Index for UAVs
r : Index of a UAV’s rth flight
NJ: Number of split jobs
NUAV: Number of UAVs in the system
NSTA: Number of recharge stations
NR: Maximum number of flight per UAV during the time horizon
M : Large positive number
(xjs, yj
s) : Start point of split job j
(xje, yj
e) : End point of split job j
Dij: Distance from split job ith finish point to split job jth start point, Dij ≠ Dji
Ei : Start time of split job i
Pi : Processing time or split job i
qk: Maximum traveling time of UAV k
Sok: Initial location(station) of UAV k
TSk: Travel speed of UAV k
H : Required time for fully recharge(refuel) the empty fuel tank (battery)
qk,ini: Initial remaining battery (fuel) of UAV k
ICUAS 2014 – 13
Mathematical Formulation: Notation
■ Notation
ΩJ : = {1, …, NJ}, Set of split jobs
ΩINI : = Set of initial UAV location
ΩSS : = {NJ+1, NJ+3, …, NJ+2∙ NSTA-1}, set of start station
ΩSE : = {NJ+2, NJ+4, …, NJ +2∙ NSTA}, set of end station
ΩA : = (ΩD U ΩSS U ΩSE) = {1,…, NJ+2∙NSTA}, set of all jobs and recharge stations
■ Decision Variables
▪ Xijkr = 1 if UAV k processes split job j or recharges at station j after processing split job i or recharging at station i during the rth flight; 0, otherwise
▪ Cikr is job i’s start time by UAV k during its rth flight or UAV k’s recharge start time at station i; otherwise its value is 0.
▪ qkr is total fuel (battery) consumption amount for UAV k during its rth flight
ICUAS 2014 – 14
Mathematical Formulation
■ Mathematical formulation
1, 1 ( , 1... 1, )J SS J SS
iskr s ikr R SEi i
X X k K r N s
1 ( , )SE J SS
iskrs i
X k K r R
1 ( , )J INI J SE
sjkrs j
X k K r R
, 1 1 ( )ok
J SE
s jkj
X k K
Subject to
1, 1 ( , 1... 1, )skr s kr R SEC C k K r N s
1 ( )A
ijkr Jk K r R i
X j
0 ( , , )A A
ijkr jikr Jj j
X X i k K r R
0 ( , , )J SS
iskr SS INIi
X k K r R s
A A
ij ijkrk K r R i j
D X
Minimize (1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
Network flow &
job assignment
ICUAS 2014 – 15
Mathematical Formulation
/ (1 ) ( , , , )ikr i ij k jkr ijkr J SS J SEC P D TS C M X i j k K r R
( )ikr i Jk K r R
C E i
/ ( , )A A J A
ij k ijkr i ijkr kri j i j
D TS X P X q k K r R
0 ( , , )ikr AC k K r R i
{0,1} ( , , , )ijkr A AX k K r R i j
(10)
(11)
(12)
(13)
(14)
(15)
, ( , 1)kr k iniq q k K r
( , , 1)kr kq q k K r R r
0 ( , )krq k K r R
(16)
(17)
(18)
(19)
( , , )J SE
ijkr ikr J SSj
M X C i k K r R
1 ,( ( )) / / (1 ) ( , , , 2)ikr kr k k ini k ij k jkr ijkr SS J SEC H q q q q D TS C M X i j k K r
1 / / (1 ) ( , , , 2)ikr kr k ij k jkr ijkr SS J SEC H q q D TS C M X i j k K r
(20)
Start time constraints
Fuel con-straints
Decision vari-ables
ICUAS 2014 – 16
Formulation Demonstration (CPLEX)
■ Problem description : Problem map
0
50
100
150
200
250
300
0 100 200 300 400 500 600 700
customer 1
customer 2
station
initial location
Customer 1 Customer 2 Station
▲ UAVU1S1
S2
S3
U2
U3
U4U5
U6
ICUAS 2014 – 17
Formulation Demonstration (CPLEX)
■ Problem description : Split job & station & UAV information
Station x y
S1 394 126
S2 170 79
S3 72 229
UAV x ySpeed
(meter / min)qk,ini
(min)
1 560 130 240 3
2 280 180 240 8
3 170 79 240 8
4 300 100 240 6
5 394 126 240 6
6 90 200 240 8
Customer Split job
Start point End point Start time
x y x y
1
1 596 167 532 161 5
2 532 161 483 129 6
3 438 129 432 94 7
4 432 94 372 91 8
5 372 91 315 87 9
6 315 87 262 74 10
7 262 74 218 99 11
8 218 99 171 134 12
9 171 134 142 186 13
10 142 186 102 225 14
11 102 225 54 251 15
12 54 251 6 266 16
213 458 64 479 105 8
14 479 105 514 63 9
15 514 63 536 15 10
ICUAS 2014 – 18
Formulation Demonstration (CPLEX)
■ Result of example 1
UAV Scheduling qini qk1 qk2
1 Initial → 1,2,3,4,5 → Station 1 6 5.58 0
2 Initial → 13, 14, 15 → Station 1 8 4.63 0
3 Initial (Station 2) 8 0 0
4 Initial → 6,7,8,9,10,11,12 → Station 3 10 7.4 0
5 Initial (Station 1) 6 0 0
6 Initial → Station 3 8 0.14 0
- Objective value : 662.570- CPU time: 5.42 seconds
ICUAS 2014 – 19
Formulation Demonstration (CPLEX)
■ Result of example 2 : Change qk,ini
UAV Scheduling qini qk1 qk2
1 Initial location → 1,2 → Station 1 3 2.59 0
2 Initial location → Station 1 → 3,4,5,6,7,8,9,10,11,12→ Station 3 8 0.52 10.7
3 Initial location (Station 2) → Station 2 8 0 0
4 Initial location → 13,14,15 → Station 1 6 4.4 0
5 Initial location (Station 1) → Station 1 6 0 0
6 Initial location → Station 3 8 0.14 0
- Objective value : 809.067- CPU time: 9.17 seconds
ICUAS 2014 – 20
Formulation Demonstration (CPLEX)
■ These examples shows us
- Real time perspectives: Current location and fuel level of UAVs
- Access to the persistent service
- Sharing of multiple stations
ICUAS 2014 – 21
Formulation Demonstration (CPLEX)
■ Computational limit of CPLEX
System parameters CPLEX
NJ NSTA NUAVCPUtime
Obj.value
15 3 6 5.42 662.570
30 3 6 N/A N/A
45 3 6 N/A N/A
45 5 10 N/A N/A
60 3 6 N/A N/A
60 5 10 N/A N/A
Needs for alternative solution approaches to achieve real time perspective
ICUAS 2014 – 22
Presentation Overview
• Motivation
• Goal and ideas
• Mathematical formulation
• Solution approaches: STAH & RHTA
• Numerical examples
• Concluding remarks
ICUAS 2014 – 23
Solution Approach: Solution Properties
■ Analysis on solutions
1) Strict condition: Feasibility issue - Fuel - Split job start time
2) Flexible condition: Solution quality issue - Total moving distance - The number of split jobs which UAV can serve in that flight - Trade off between direct & indirect flight
ICUAS 2014 – 24
Solution Approach: Solution Properties
■ Analysis on solutions
3) Sequential assignment of split jobs
UAV Scheduling qini qk1 qk2
1 Initial → 1,2,3,4,5 → Station 1 6 5.58 0
2 Initial → 13, 14, 15 → Station 1 8 4.63 0
3 Initial (Station 2) 8 0 0
4 Initial → 6,7,8,9,10,11,12 → Station 3 10 7.4 0
5 Initial (Station 1) 6 0 0
6 Initial → Station 3 8 0.14 0
ICUAS 2014 – 25
Solution Approach: Solution Properties
■ Trade off between direct and indirect flight
Service station 1
1
2
3
104
5
6 7
89
UAV 1
< Direct flight >
- Without replenishment- Short moving distance- Serve less number of split jobs
ICUAS 2014 – 26
Solution Approach: Solution Properties
■ Trade off between direct and indirect flight
< Indirect flight >
- Replenishment on the station- Long moving distance - Serve more number of split jobs
Service station 1
1
2
3
104
5
6 7
89
UAV 1
ICUAS 2014 – 27
Solution Approach: STAH
■ Based on the analysis of solutions
1) Strict condition
2) Flexible condition
Assign UAVs which satisfy strict conditions (Ajk)
Contribution value of direct and indirect flight
VD(k) = {αw( # of split jobs) –(1-α)(total moving distance) }∙Ajk
VI(k) = {αw( # of split jobs) –(1-α)(total moving distance) }∙Ajk
Select the flight which provide highest value
1, if UAV satisfy fuel and start time constraints at split job
0, Otherwise jk
k jA
From customer 1 (earliest customer), split jobs are assigned sequentially
ICUAS 2014 – 28
Solution Approach: RHTA
■ Receding horizon task assignment algorithm (RHTA)
– Classical heuristic (petal algorithm) for vehicle routing problem
– Iterative IP model
– Generate good quality feasible solution in short time
– Classical RHTA was modified: • Eliminate the resource selection components• Modify the equation of replenishment time for a UAV
ICUAS 2014 – 29
Solution Approach: RHTA
■ Petal : Sequence of split jobs to be served by a UAV
• Step 1: Enumerate all feasible petals for each UAV k– Generate petal– Check feasibility of petal
– Evaluate of petal value
)()(/)),(()( 11 wEkTSwkcLDkat
)()(/),()()( 111 iiiii wEkTSwwDwPwE
cc ni
iinc
ni
iii krFwPsWDwwDwkcLD
1
1
111 )()(),(),()),((
Fuel limitation
Time window
ICUAS 2014 – 30
Solution Approach: RHTA
• STEP 2: Solve a single IP to select petal
• STEP 3: Assign split job of selected petal to UAVs
• STEP 4: Send exhausted UAVs to a service station - if there are remaining jobs on the list, go to STEP 1.
• STEP 5: Send any UAVs not located at a station to a station
A job can be processed by at most one UAV
At least P job should be selected
Minimize total travel distance
UAV
1 1
N
1 1
s.t. 1 ( )
kp
kp
Nk K
kip kpk p
Nk
kip kpi W k p
A Y i W
A Y P
1 1
Min kpUAV
NN
kp kpk p
S X
ICUAS 2014 – 31
Presentation Overview
• Motivation
• Goal and ideas
• Mathematical formulation
• Solution approaches: STAH & RHTA
• Numerical examples
• Concluding remarks
ICUAS 2014 – 32
Results for Various Problem Sizes
■ Results
System parameters CPLEX STAH RHTA
NJ NSTA NUAVCPUtime
Obj.value
CPUtime
Obj.value
GapCPUtime
Obj.value
Gap
15 3 6 5.42 662.570 0.004 665.261 0.406 % 0.203 664.857 0.345 %
30 3 6 N/A N/A 0.008 1010.24 - 0.327 1274.991 -
45 3 6 N/A N/A 0.015 1314.34 - 0.458 1901.816 -
45 5 10 N/A N/A 0.015 1346.96 - 0.608 1656.667 -
60 3 6 N/A N/A 0.016 1642.34 - 0.578 2677.189 -
60 5 10 N/A N/A 0.031 1565.90 - 0.765 1997.992 -
- STAH and RHTA obtain very near optimal values.
- CPLEX issues an out of memory error.
- STAH and RHTA derive feasible solution in a reasonably short time.
ICUAS 2014 – 33
Real Time Operation with New Service Request Arrival
Customer Split job
Start point End point Start time
x y x y
1
1’ 262 74 218 99 11
2’ 218 99 171 134 12
3’ 171 134 142 186 13
4’ 142 186 102 225 14
5’ 102 225 54 251 15
6’ 54 251 6 266 16
3
7’ 413 210 430 184 13
8’ 430 184 454 162 14
9’ 454 162 482 137 15
10’ 482 137 451 10 16
11’ 451 10 442 82 17
12’ 442 82 428 65 18
UAV x y qk qk,ini TSk
1 394 126 12 12 240
2 262 74 12 7.629 240
3 170 79 12 12 240
4 536 15 12 2.325 240
5 394 126 12 12 240
6 72 229 12 12 240
■ Remaining jobs and UAV information at T=11.
■ During the UAV service of example 1, new service request arrived at T=10.
ICUAS 2014 – 34
Real Time Operation with New Service Request Arrival
Initial plan
UAV Scheduling qk,iniObj.value
CPUTime (sec)
1 Initial location → 1, 2 → Station 1 3
809.067 9.17
2 Initial location → Station 1 → 3,4,5,6,7,8,9,10,11,12 → Station 3 8
3 Initial location (Station 2) 8
4 Initial location → 13, 14, 15 → Station 1 6
5 Initial location (Station 1) 6
6 Initial location → Station 3 8
New plan from time 11
UAV Scheduling qk,iniObj.value
CPUTime (sec)
1 (394,126) (Station 1) 12
411.306 4.67
2 (262,74) → 1’,2’,3’,4’,5’,6’ → Station 3 7.629
3 (170,79) (Station 2) 12
4 (536,15) → Station 1 2.325
5 (394,126) (Station 1) → 7’,8’,9’,10’,11’,12’ → Station 1 12
6 (72,229) (Station 3) 12
ICUAS 2014 – 35
0 100 200 300 400 500 600 7000
50
100
150
200
250
300
Real Time Operation with New Service Request Arrival
Station 1
Station 2
Station 3
T =
4
Customer 1
Customer 2
5678910
Customer 3
111213141516171819
ICUAS 2014 – 36
Presentation Overview
• Motivation
• Goal and ideas
• Mathematical formulation
• Solution approaches: STAH & RHTA
• Numerical examples
• Concluding remarks
ICUAS 2014 – 37
Concluding Remarks
• Fleets of UAVs conducting missions over a field of operations can achieve persistence - Supported by shared service stations distributed over the field - Real time operation to handle new arrival of service request
• MILP scheduling model - Allows a mobile robot to perform persistent service by sharing base for energy resupply - Maximize service quality by serving 100% service requests by orchestration of components - Demonstration of real time perspectives
• Develop a efficient heuristics to overcome computational limit of MILP formulation - STAH - RHTA
• Application & numerical examples
ICUAS 2014
Back-UP Materials
ICUAS 2014 – 39
Solution approach: STAH
■ STAH procedure
• STEP 1: Arrange customers in the non-decreasing order of (first) split job start time– Let P= {1,2,…,p} be the set of customers and Pt be the set of split jobs in
customer index t– Element of Pt also arranged according to the non-decreasing order of split job
start time. Set P=1.
• STEP 2: From customer P, split jobs are assigned to the UAVs.– Calculate VD(k) and VI(k)
– Among VD(k) and VI(k) for every UAV, the biggest function value and corresponding UAV schedule is selected
• STEP 3: Update parameters– Current location, fuel level, available time of UAVs– Remaining split job information for customer P is updated
ICUAS 2014 – 40
Solution approach: STAH
■ STAH procedure
• STEP 4: Repeat step 2 to 3 until every split job is assigned to UAV in customer P.
• STEP 5: Set P=P+1, and repeat step 2,3 and 4. This step is repeated until P=|P|+1.
• STEP 6: Send any UAVs not located at a station to a station
ICUAS 2014 – 41
Solution approach: STAH
■ Pseudo code (STAH)
ICUAS 2014 – 42
Feasibility
Y
Y
Y
Y
N
Solution approach: RHTA
■ Petal : sequence of split jobs to be served by a UAV
• Step 1: Enumerate all feasible petals for each UAV k– Generate petal (Limitation of maximum size of petal)– Check feasibility of petal
– Evaluate value of petal
Petal size
Instance of petal
1 {2}
1 {3}
2 {2,3}
3 {6,7,8}
4 {1,2,3,4}
)()(/)),(()( 11 wEkTSwkcLDkat
)()(/),()()( 111 iiiii wEkTSwwDwPwE
cc ni
iinc
ni
iii krFwPsWDwwDwkcLD
1
1
111 )()(),(),()),((
Fuel limitation
Time window
Travel dis-tance
6
10
(6+3)
(10+4+5)
infeasible
ICUAS 2014 – 43
Solution approach: RHTA
• STEP 2: Solve a single IP for all UAVs to minimize the travel distance – Select petals to be used
subject to
1 1
1 ( )kpNk K
kip kpk p
A Y i W
A job can be processed by at most one UAV
At least P job should be selectedUAVN
1 1
kpNk
kip kpi W k p
A Y P
Minimize total travel distance𝑀𝑖𝑛∑𝑘=1
𝑁𝑈𝐴𝑉
∑𝑝=1
𝑁 𝑘𝑝
𝑆𝑘𝑝 𝑋𝑘𝑝
ICUAS 2014 – 44
Solution approach: RHTA
• STEP 3: Assign split job to UAV– Assign UAV to a first split job of selected petal– Update system state (UAV location, fuel, available time and job information)– Remove assigned job from remaining job list
• STEP 4: Find exhausted UAVs– Send UAVs that do not have any feasible petals to a service station– Update system state – Return to STEP 1 if remaining job list is not empty
• STEP 5: Send any UAVs not located at a station to a station
ICUAS 2014 – 45
Solution approach: RHTA
■ Pseudo code (RHTA)