IPC 2023 HTN Tracks
This is the website for the hierarchical (HTN) tracks of the IPC 2023. It is the 2nd IPC containing hierarchical tracks, after its first occurance in 2020.
Please forward the following calls to all interested parties.
| Event | Date |
|---|---|
| Call for domains | October 12, 2022 |
| Call for participation | October, 2022 |
| Domain expression of interest deadline | December 31, 2022 |
| Domain submission deadline | February 28, 2023 |
| Demo problems provided | February 10, 2023 |
| Initial planner submission | February 28, 2023 |
| Feature stop (final planner submission) | May 31, 2023, AoE |
| Planner Abstract submission deadline | May 31, 2023, AoE |
| Contest run | June, 2023 |
| Results announced | July 12, 2023 |
| Result analysis deadline | August, 2023 |
There will be (up to, provided sufficient interest) six tracks in IPC’s HTN track. These are:
The semantics of these tracks is explained in the following.
We will use the same input language as the previous HTN IPC 2020
Competitors are required to submit their code via a github repository that was set up by the track’s organisers. At the time of submission, these repositories will be private and will be made public after the competition results have been announced.
Until the feature stop (see Schedule) competitors can make any change to their planners. After the feature stop, competitors are only allowed to send pull requests with bug fixes. We will review every pull request with its accompanying description of the bug fix to make sure that no significant changes or parameter tuning are possible.
As in 2020, we will provide a continous integration system. We will run planners on sample problems and publish the resulting outputs. To faciliate the detection of anomalies, each competitor is required to also submit a script that validates the planners output and that is responsible to highlight (potential) errors that need to be investigated.
As in the HTN IPC of 2020, we will use the container software Apptainer (formerly known as Singularity) to promote reproducibility and simplify program compilation. You can find the documentation on how the Apptainer files are written here.
We will automatically extract the configurations you want to submit from your repository. We create one configuration per Apptainer file in the root directory of your repository. Apptainer files are files whose file ending is .def.
In order to participate in the HTN IPC, you need to register until February 28, 2023. To do so, please send an email to Gregor Behnke containing the following information
As the other IPC tracks, the HTN track allows for multiple submissions to use the same planner codebase. These submissions can, e.g., differ in the configuration of the planner such as the used heuristic(s) or search techniques. Each of these configurations will (most likely) be a separate apptainer definition file (formerly Singularity file).
The number of submissions per participant (natural person who participates) is limited to three per track. That means that any person may only be named as a “participant” for at most three submission per track. This limit applies to the number of configurations and not codebases. I.e. the limit is already reached if a participant submits one codebase that is used for three configurations in one track.
After some discussion we have decided to update and clarify the submission rules as follows (on 24-02-2023):
Authors are allowed to submit an arbitrary number – within reason – of different planning systems per track. Two planning systems are considered different if the relevant codebase, i.e., the parts of the code that are actually being executed, differs substantially, as judged by the organizers. (This allows, e.g., for submitting different planning approaches as separate planning systems even if they are part of a common software.)
For each planning system submitted, a maximum of three different configurations may be submitted.
A modification of somebody else’s HTN planning system is a valid submission if and only if the original author(s) have been made aware of the submission and are consenting.
The following planning systems participated in the HTN track of the IPC 2023. In this table, you can find the link to the repositories for each competitior as well as links to the apptainer containers we build based on them.
In the following table, you can find all participating planners from the HTN track of the IPC 2023. The linked github repositories contain the planners as used in the IPC evaluation. We also provide the apptainer images we build for every planner. Note: these links are currently not available as we have not uploaded the files yet Note: We cannot provide the apptainer containers for the configurations ppro-xo-opt-as-dof-ao, ppro-xo-opt-as-dof-lmc, and ppro-xo-opt-as-lmc. These require CPLEX to build – which cannot be provided as public download. If you which to run these planners, you need to obtain your own license for CPLEX and add CPLEX to the build process of the repository.
See status page.
We have used most of the domain from the previous IPC 2020. For both the Total-Order and the Partial-Order Tracks, we have added two new domains:
The full set of instances used for the HTN Track of the IPC 2023 is available on github.
The IPC 2023 has concluded and the results have been presented at ICAPS 2023 in Prague. The results presentation can be found here. The full results tables can be found here. The results website also contains links to CSV files of the results (and additionally coverage data).
You can download the raw log files of the planners and the per-instances scores for the TO Track and the PO Track.
The winners are:
| Track | Winner | Runner-Up |
|---|---|---|
| Total-Order Satisficing | PandaDealer-agile-lama Conny Olz, Daniel Höller, Pascal Bercher |
PANDApro $\lambda$ AO Daniel Höller |
| Total-Order Agile | PandaDealer-agile-1 and PandaDealer-agile-lama Conny Olz, Daniel Höller, Pascal Bercher |
PANDApro $\lambda$ lm-cut Daniel Höller |
| Total-Order Optimal | PandaDealer-optimal Conny Olz, Daniel Höller, Pascal Bercher |
PANDApro lm-cut and PANDApro DOF Daniel Höller |
| Partial-Order Satisficing | Grounded-Linear-Satisficing Ying Xian Wu, Conny Olz, Songtuan Lin, Pascal Bercher |
PANDApro $\lambda$ AO and lm-cut Daniel Höller |
| Partial-Order Agile | Grounded-Linear-Agile Ying Xian Wu, Conny Olz, Songtuan Lin, Pascal Bercher |
PANDApro $\lambda$ AO and lm-cut Daniel Höller |
| Partial-Order Optimal | PANDApro lm-cut Daniel Höller |
Aries Arthur Bit-Monnot |
The full results tables can be found here.
The preliminary versions of the planner abstracts can be found here. The participants can still submit a final version, which can also include an analysis of the behaviour of their planner in the IPC. We will publish these proceedings once we have them.
| Participant | Planner Abstract |
|---|---|
| Aries | Aries |
| SIADEX | SIADEX SIADEX is the only planner that participated with exactly the same version as in the IPC2020. We thus refer to the paper from the IPC2020 |
| Lifted Linear | Lifted Linear |
| Linear Simple | Linear Simple |
| Linear Complex | Linear Complex |
| PANDApro | PANDApro PANDApro $\lambda$ |
| TOAD | TOAD |
| PandaDealer | PandaDealer |
| PDDL4J | LiftedTreePath OptiPlan |
Contact us: ipc2023-htn@googlegroups.com