P4Learned Delivery Routing
10

Evaluation line · 10 of 13

Experiments

Three experiments, ten seeds per configuration. Every cell is a mean with its standard deviation; the raw CSVs ship with the source.

Seeds
10 per configuration
Script
run_experiments.py
Runtime
about 15 seconds

A* against UCS and Greedy, 6 to 10 stops

A* against UCS and Greedy, 6 to 10 stops
A* against UCS and Greedy, 6 to 10 stops
stops algorithm expanded generated cost_min gap_pct time_ms
6 UCS 95.5 ± 16.5 106.5 ± 22.2 224.8 ± 54.9 0 ± 0 0.6274 ± 0.133
6 Greedy 64.7 ± 16.4 70.8 ± 16.2 239.4 ± 56.7 6.655 ± 5.72 2.346 ± 0.699
6 A* (max-incoming h) 51.3 ± 17.9 86.1 ± 19.2 224.8 ± 54.9 0 ± 0 1.936 ± 0.411
6 A* (sum/wait h) 34 ± 20.4 63.7 ± 21.6 224.8 ± 54.9 0 ± 0 1.823 ± 0.824
7 UCS 184.3 ± 23.1 202.6 ± 31.2 258.7 ± 63.1 0 ± 0 1.447 ± 0.287
7 Greedy 136.2 ± 51.6 143.3 ± 52.1 265.2 ± 66.4 2.433 ± 3.09 5.242 ± 2.12
7 A* (max-incoming h) 123.9 ± 17.1 187.9 ± 22.2 258.7 ± 63.1 0 ± 0 4.983 ± 0.895
7 A* (sum/wait h) 92.1 ± 27.3 159.4 ± 28.3 258.7 ± 63.1 0 ± 0 5.485 ± 2.52
8 UCS 356.7 ± 53.3 425.4 ± 75.2 301.3 ± 57.4 0 ± 0 2.798 ± 0.582
8 Greedy 326.4 ± 104 333.2 ± 105 311.6 ± 55.7 3.651 ± 3.85 10.53 ± 3.23
8 A* (max-incoming h) 213.8 ± 70.7 363.6 ± 58.9 301.3 ± 57.4 0 ± 0 8.534 ± 1.51
8 A* (sum/wait h) 109.3 ± 60.8 227.4 ± 68.9 301.3 ± 57.4 0 ± 0 6.675 ± 2.21
9 UCS 777.1 ± 108 889.2 ± 145 289.9 ± 56.3 0 ± 0 6.422 ± 1.16
9 Greedy 775.2 ± 221 784.1 ± 223 304.2 ± 53.9 5.424 ± 5.1 24.59 ± 7.35
9 A* (max-incoming h) 478 ± 77.5 776.4 ± 126 289.9 ± 56.3 0 ± 0 19.02 ± 3.47
9 A* (sum/wait h) 254.3 ± 56.9 526.7 ± 73.7 289.9 ± 56.3 0 ± 0 15.07 ± 2.28
10 UCS 1519 ± 161 1717 ± 301 334.4 ± 36.6 0 ± 0 13.55 ± 1.49
10 Greedy 1312 ± 440 1322 ± 441 349.8 ± 40.5 4.612 ± 4.21 42.08 ± 13.6
10 A* (max-incoming h) 865.4 ± 180 1439 ± 184 334.4 ± 36.6 0 ± 0 36.08 ± 4.12
10 A* (sum/wait h) 368.7 ± 249 847.7 ± 293 334.4 ± 36.6 0 ± 0 24.69 ± 9.73

A* vs brute force (6-8 stops): 30/30 identical optimal costs.

Pruning: BT, BT+FC, BT+FC+AC-3 at 15, 20 and 30 orders

Pruning: BT, BT+FC, BT+FC+AC-3 at 15, 20 and 30 orders
Pruning: BT, BT+FC, BT+FC+AC-3 at 15, 20 and 30 orders
orders solver assignments assignments median backtracks checks time_ms nn_edges solved/unsat/limit
15 BT 30.1 ± 28.4 16.5 15.1 ± 28.4 53.3 ± 70.2 0.08177 ± 0.0695 2.1 ± 3.08 10/0/0
15 BT+FC 28.8 ± 27.4 16.0 13.8 ± 27.4 0 ± 0 0.07832 ± 0.0578 2.1 ± 3.08 10/0/0
15 BT+FC+AC3 28.6 ± 27.5 15.5 13.6 ± 27.5 0 ± 0 0.3828 ± 0.861 2.1 ± 3.08 10/0/0
15 BT+FC+AC3 +degree order 15 ± 0 15.0 0 ± 0 0 ± 0 0.06435 ± 0.00557 2.1 ± 3.08 10/0/0
20 BT 445.5 ± 1.23e+03 24.0 425.5 ± 1.23e+03 1038 ± 2.92e+03 1.158 ± 3.14 4.5 ± 4.13 10/0/0
20 BT+FC 192.4 ± 503 23.5 172.4 ± 503 0 ± 0 0.4694 ± 1.02 4.5 ± 4.13 10/0/0
20 BT+FC+AC3 140.1 ± 349 23.0 120.1 ± 349 0 ± 0 0.3777 ± 0.842 4.5 ± 4.13 10/0/0
20 BT+FC+AC3 +degree order 20.2 ± 0.4 20.0 0.2 ± 0.4 0 ± 0 0.1188 ± 0.0756 4.5 ± 4.13 10/0/0
30 BT 4.089e+04 ± 7.96e+04 165.5 4.087e+04 ± 7.96e+04 1.08e+05 ± 2.1e+05 99.93 ± 194 7.9 ± 6.47 8/0/2
30 BT+FC 4.051e+04 ± 7.98e+04 105.5 4.049e+04 ± 7.98e+04 0 ± 0 117.9 ± 238 7.9 ± 6.47 8/0/2
30 BT+FC+AC3 2.203e+04 ± 5.95e+04 104.5 2.2e+04 ± 5.95e+04 0 ± 0 66.14 ± 164 7.9 ± 6.47 8/1/1
30 BT+FC+AC3 +degree order 4475 ± 1.32e+04 36.5 4448 ± 1.32e+04 0 ± 0 11.56 ± 33.6 7.9 ± 6.47 9/1/0

Lateness when real trip times are replayed

Lateness when real trip times are replayed
Lateness when real trip times are replayed
scenario model replay_late replay_late_pct replay_late_minutes planned_late planned_route_min learned_rules plans ok
standard windows 30-120 min NN (MLP) 0.31 ± 0.486 1.55 ± 2.43 7.794 ± 18.3 0 ± 0 1172 ± 75.2 0 ± 0 10/10
standard windows 30-120 min hist-avg 0.3933 ± 0.499 1.967 ± 2.5 6.985 ± 11 0 ± 0 1172 ± 75.5 0 ± 0 10/10
standard windows 30-120 min distance-linear 0.3067 ± 0.46 1.533 ± 2.3 5.061 ± 9.86 0 ± 0 1159 ± 74.1 0 ± 0 10/10
tight windows 15-30 min NN (MLP) 1.043 ± 0.69 5.217 ± 3.45 9.115 ± 6.72 0 ± 0 1300 ± 71 0.6 ± 1.2 10/10
tight windows 15-30 min hist-avg 1.01 ± 1.04 5.05 ± 5.19 10.76 ± 12.7 0 ± 0 1299 ± 67.7 0.4 ± 0.917 10/10
tight windows 15-30 min distance-linear 1.617 ± 1.21 8.083 ± 6.07 17.52 ± 15.1 0.1 ± 0.3 1288 ± 70.6 0.8 ± 1.66 9/10

Replay pool back-offs: {'hour+-1': 3099, 'all-hours': 667}