International Planning Competition 2023 HTN Tracks

IPC 2023 HTN Tracks

Total Order Tracks

Satisficing Track

CSV: here

planner total Assembly Barman BDI Blocksworld HPDDL Depots Factories simple Freecell Learned Hiking Logistics Learned Minecraft Player Minecraft Regular Monroe Fully Monroe Partially Multiarm Blocksworld Robot Rover GTOHP Satellite GTOHP Towers Transport Woodworking SharpSAT lamps Snake
PandaDealer-agile-lama 15.28808 0.89 0.78 0.77 0.90 0.67 0.13 0.83 0.98 0.20 0.71 0.94 0.68 0.95 0.92 0.88 0.92 0.65 0.73 0.69 0.46 0.48 0.14
lamda-to-agl-gbfs-ao 15.15338 1.00 0.84 0.84 0.90 0.42 0.04 0.83 1.00 0.20 0.71 0.97 0.65 0.96 0.90 0.90 0.94 0.65 0.65 0.76 0.48 0.40 0.11
PandaDealer-sat-1 14.89349 0.83 0.72 0.94 0.72 0.48 0.20 0.87 0.85 0.20 0.71 0.93 0.63 0.97 0.81 0.87 0.94 0.65 0.55 0.97 0.47 0.45 0.14
lamda-to-agl-gbfs-lmc 14.87160 0.89 0.80 0.71 0.87 0.42 0.07 0.80 1.00 0.20 0.71 0.97 0.59 0.95 0.91 0.93 0.92 0.65 0.80 0.70 0.47 0.40 0.11
ppro-to-sat-gbfs-add 13.62710 0.83 0.58 0.83 0.72 0.37 0.24 0.87 0.42 0.20 0.61 0.91 0.52 0.96 0.82 0.93 0.91 0.65 0.58 0.76 0.47 0.35 0.10
PandaDealer-sat-2 11.34366 0.20 0.50 0.30 0.80 0.35 0.00 0.70 0.54 0.20 0.71 0.96 0.65 0.24 0.54 0.61 0.74 0.65 0.62 0.95 0.48 0.50 0.11
toad-io-dfad 10.92964 1.00 0.71 0.73 0.83 0.25 0.00 0.40 0.55 0.05 0.66 0.00 0.00 0.98 0.98 0.29 0.97 0.60 0.22 0.98 0.05 0.53 0.15
toad-po-dfad 10.43419 1.00 0.71 0.73 0.77 0.25 0.00 0.40 0.55 0.05 0.66 0.00 0.00 0.98 0.98 0.29 0.58 0.60 0.22 0.98 0.05 0.53 0.11
toad-io-ff 10.23798 1.00 0.65 0.70 0.85 0.25 0.00 0.40 0.50 0.05 0.66 0.00 0.00 0.98 0.98 0.28 0.54 0.60 0.17 0.98 0.05 0.50 0.12
LiftedTreePath 7.12610 0.10 0.64 0.00 0.71 0.15 0.00 0.46 0.01 0.08 0.45 0.71 0.71 0.00 0.25 0.52 0.40 0.20 0.67 0.83 0.19 0.00 0.03
SIADEX 3.45194 0.00 0.61 0.00 0.73 0.00 0.00 0.00 0.00 0.15 0.64 0.19 0.00 0.01 0.00 0.45 0.00 0.55 0.03 0.09 0.00 0.00 0.00
aries-sat 3.19394 0.07 0.15 0.00 0.40 0.05 0.00 0.03 0.01 0.00 0.02 0.40 0.40 0.00 0.20 0.23 0.30 0.20 0.40 0.00 0.00 0.33 0.00
OptiPlan 0.72308 0.00 0.00 0.00 0.03 0.05 0.00 0.00 0.00 0.00 0.03 0.00 0.00 0.00 0.00 0.12 0.14 0.00 0.31 0.03 0.00 0.00 0.00

Agile Track

CSV: here

planner total AssemblyHierarchical Barman BDI Blocksworld HPDDL Depots Factories simple Freecell Learned Hiking Logistics Learned Minecraft Player Minecraft Regular Monroe Fully Monroe Partially Multiarm Blocksworld Robot Rover GTOHP Satellite GTOHP Towers Transport Woodworking SharpSAT lamps Snake
PandaDealer-agile-1 11.74366 0.86 0.71 0.74 0.70 0.37 0.11 0.63 0.76 0.07 0.54 0.55 0.32 0.83 0.84 0.58 0.63 0.44 0.49 0.62 0.39 0.50 0.09
PandaDealer-agile-lama 11.68335 0.81 0.71 0.63 0.80 0.40 0.07 0.61 0.74 0.07 0.54 0.55 0.32 0.81 0.84 0.56 0.61 0.45 0.68 0.54 0.39 0.50 0.09
lamda-to-agl-gbfs-lmc 11.45496 0.81 0.73 0.57 0.79 0.37 0.02 0.60 0.73 0.07 0.52 0.55 0.27 0.74 0.82 0.60 0.62 0.43 0.72 0.54 0.39 0.48 0.08
PandaDealer-agile-2 11.29298 0.71 0.79 0.70 0.81 0.31 0.15 0.64 0.78 0.07 0.54 0.56 0.34 0.45 0.84 0.54 0.55 0.45 0.47 0.65 0.39 0.47 0.08
lamda-to-agl-gbfs-ao 11.15163 0.67 0.73 0.64 0.81 0.30 0.01 0.63 0.71 0.07 0.52 0.54 0.30 0.74 0.81 0.60 0.62 0.43 0.54 0.55 0.39 0.47 0.07
ppro-to-agl-gbfs-add 11.04744 0.85 0.66 0.66 0.70 0.31 0.07 0.63 0.42 0.08 0.52 0.55 0.28 0.76 0.83 0.62 0.65 0.44 0.53 0.55 0.40 0.46 0.07
toad-io-dfad 8.24055 0.92 0.52 0.56 0.66 0.18 0.00 0.27 0.41 0.01 0.46 0.00 0.00 0.73 0.81 0.23 0.61 0.43 0.18 0.68 0.02 0.49 0.06
toad-io-ff 8.05728 0.92 0.51 0.55 0.69 0.18 0.00 0.27 0.41 0.01 0.46 0.00 0.00 0.74 0.81 0.23 0.43 0.43 0.16 0.69 0.02 0.49 0.06
toad-po-dfad 7.95838 0.92 0.52 0.56 0.65 0.19 0.00 0.27 0.41 0.01 0.47 0.00 0.00 0.73 0.80 0.23 0.42 0.39 0.18 0.65 0.02 0.49 0.05
LiftedTreePath 6.65637 0.07 0.69 0.00 0.60 0.16 0.00 0.42 0.01 0.05 0.35 0.64 0.60 0.00 0.20 0.49 0.50 0.18 0.67 0.87 0.15 0.00 0.02
SIADEX 3.65923 0.00 0.91 0.00 0.69 0.00 0.00 0.00 0.00 0.12 0.34 0.24 0.00 0.01 0.00 0.76 0.00 0.45 0.03 0.10 0.00 0.00 0.00
aries-agile 1.94816 0.05 0.10 0.00 0.19 0.04 0.00 0.02 0.00 0.00 0.00 0.23 0.20 0.00 0.14 0.17 0.17 0.14 0.22 0.00 0.00 0.27 0.00
OptiPlan 0.40067 0.00 0.00 0.00 0.02 0.04 0.00 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.06 0.10 0.00 0.14 0.03 0.00 0.00 0.00

Optimal Track

CSV: here

planner total AssemblyHierarchical Barman BDI Blocksworld HPDDL Depots Factories simple Freecell Learned Hiking Logistics Learned Minecraft Player Minecraft Regular Monroe Fully Monroe Partially Multiarm Blocksworld Robot Rover GTOHP Satellite GTOHP Towers Transport Woodworking SharpSAT lamps Snake
PandaDealer-optimal 8.33212 0.13 0.50 0.17 0.60 0.30 0.00 0.23 0.35 0.10 0.63 0.95 0.60 0.15 0.55 0.27 0.30 0.65 0.25 0.57 0.43 0.50 0.11
ppro-xo-opt-as-lmc 7.82008 0.13 0.50 0.17 0.60 0.25 0.00 0.23 0.35 0.05 0.63 0.70 0.50 0.15 0.55 0.27 0.30 0.65 0.23 0.57 0.43 0.50 0.07
ppro-xo-opt-as-dof-lmc 6.76565 0.10 0.40 0.17 0.60 0.25 0.00 0.57 0.28 0.05 0.51 0.00 0.00 0.19 0.55 0.27 0.50 0.55 0.38 0.53 0.38 0.47 0.04
ppro-xo-opt-as-dof-ao 6.27969 0.10 0.35 0.00 0.60 0.20 0.00 0.60 0.28 0.05 0.56 0.00 0.00 0.00 0.55 0.27 0.50 0.55 0.38 0.50 0.33 0.43 0.04
aries-opt 0.15000 DNS 0.15 DNS DNS DNS DNS DNS DNS DNS DNS DNS DNS DNS DNS DNS DNS DNS DNS 0.00 DNS DNS DNS

Note: aries-opt does not support domains that contain recursive decomposition methods. It thus only participated on the acyclic domains in the benchmark set.

Raw Coverage

CSV: here

Partial Order Tracks

Satisficing Track

CSV: here

planner total Barman BDI Monroe Fully Monroe Partially PCP Rover Satellite Transport Ultralight Cockpit Woodworking colouring
LinearComplex-config-sat-1 7.60238 0.66 0.94 0.70 0.82 0.86 0.98 0.26 1.00 0.70 0.69
LinearComplex-config-sat-2 7.47407 0.73 0.94 0.64 0.82 0.85 1.00 0.17 1.00 0.63 0.69
LinearSimple-sat-1 7.47253 0.60 0.96 0.66 0.82 0.81 1.00 0.26 1.00 0.70 0.66
LinearSimple-sat-2 6.96385 0.54 0.84 0.56 0.82 0.83 1.00 0.15 1.00 0.57 0.66
LinearComplex-config-sat-3 6.92736 0.50 0.90 0.60 0.82 0.75 1.00 0.19 1.00 0.50 0.66
LinearSimple-sat-3 6.76902 0.50 0.84 0.52 0.82 0.73 1.00 0.20 1.00 0.50 0.66
lamda-po-agl-gas-ao 6.46739 0.15 0.91 0.75 0.82 0.43 0.99 0.33 1.00 0.42 0.66
ppro-po-sat-gas-ff 6.30870 0.19 0.96 0.72 0.82 0.27 0.97 0.32 1.00 0.39 0.66
lamda-po-agl-gas-lmc 6.28603 0.15 0.95 0.66 0.82 0.37 0.99 0.29 1.00 0.39 0.66
aries-sat 4.73026 0.15 0.44 0.52 0.59 0.79 1.00 0.32 0.66 0.00 0.27
SIADEX 2.17641 0.62 0.26 0.04 0.00 0.31 0.83 0.03 0.00 0.09 0.00
OptiPlan DSQ DSQ DSQ DSQ DSQ DSQ DSQ DSQ DSQ DSQ DSQ

Note: OptiPlan was disqualified as it produced invalid plans in two domains (Satellite [instance 1obs-2sat-1mod.hddl] and Woodworking [instances 01–p01-complete.hddl and 02–p02-part1.hddl])

Agile Track

CSV: here

planner total Barman BDI Monroe Fully Monroe Partially PCP Rover Satellite Transport Ultralight Cockpit Woodworking colouring
LinearSimple-agile-2 7.03533 0.81 0.57 0.44 0.82 1.00 1.00 0.52 0.93 0.63 0.30
LinearComplex-agile-2 7.01996 0.85 0.50 0.43 0.82 1.00 1.00 0.54 0.93 0.64 0.31
LinearComplex-agile-1 6.89761 0.75 0.49 0.34 0.82 1.00 1.00 0.63 0.93 0.62 0.31
LinearComplex-agile-3 6.89485 0.75 0.49 0.32 0.82 1.00 1.00 0.66 0.93 0.61 0.31
LinearSimple-agile-1 6.81972 0.70 0.57 0.35 0.82 0.99 1.00 0.54 0.93 0.61 0.30
LinearSimple-agile-3 6.64191 0.51 0.57 0.33 0.82 1.00 1.00 0.57 0.93 0.60 0.30
LiftedLinear-1 6.44606 0.70 0.51 0.60 0.00 0.97 1.00 0.78 0.97 0.91 0.00
LiftedLinear-2 6.24541 0.75 0.43 0.48 0.00 0.99 1.00 0.88 0.98 0.74 0.00
LiftedLinear-3 5.20113 0.74 0.38 0.40 0.00 0.58 1.00 0.64 0.85 0.60 0.00
lamda-po-agl-gas-ao 5.01524 0.08 0.50 0.36 0.82 0.36 0.99 0.34 0.92 0.36 0.29
ppro-po-agl-gas-ff 5.00565 0.10 0.53 0.36 0.82 0.32 1.00 0.30 0.91 0.36 0.30
lamda-po-agl-gas-lmc 4.95080 0.08 0.50 0.34 0.82 0.38 1.00 0.26 0.90 0.39 0.29
aries-agile 3.22657 0.05 0.21 0.26 0.38 0.44 1.00 0.20 0.58 0.00 0.12
SIADEX 3.03256 0.92 0.24 0.05 0.00 0.70 1.00 0.03 0.00 0.10 0.00
OptiPlan DSQ DSQ DSQ DSQ DSQ DSQ DSQ DSQ DSQ DSQ DSQ

Note: OptiPlan was disqualified as it produced invalid plans in two domains (Satellite [instance 1obs-2sat-1mod.hddl] and Woodworking [instances 01–p01-complete.hddl and 02–p02-part1.hddl])

Optimal Track

CSV: here

planner total Barman BDI Monroe Fully Monroe Partially PCP Rover Satellite Transport Ultralight Cockpit Woodworking colouring
ppro-xo-opt-as-lmc 5.30353 0.10 0.72 0.60 0.82 0.30 0.96 0.10 1.00 0.43 0.27
ppro-xo-opt-as-dof-lmc 3.89560 0.05 0.00 0.00 0.82 0.30 0.96 0.30 0.86 0.43 0.17
ppro-xo-opt-as-dof-ao 3.74560 0.05 0.00 0.00 0.82 0.25 0.96 0.30 0.86 0.33 0.17
aries-opt 2.10517 0.05 DNS DNS DNS 0.40 1.00 DNS 0.66 0.00 DNS

Note: aries-opt does not support domains that contain recursive decomposition methods. It thus only participated on the acyclic domains in the benchmark set.

Raw Coverage

CSV: here