The source code of all submitted solvers must be attached to submissions and must be freely available under some open source software license. Besides the submission of the software (binary and source code) it is mandatory that an additional system description (2 pages) is provided.
We will require each solver to be submitted to the competition to be packaged in a Docker container. To do so, a submitter needs three files: (i) a Dock- erfile, (ii) a solver interface.sh file, and (iii) the solver itself. The Dockerfile defines the environment in the container. Access to resources like networking interfaces and disk drives is virtualised inside this environment. An example is shown below.
FROM alpine WORKDIR /app COPY . . ENTRYPOINT [ "./generic-interface-2019.sh" ]
It creates an Alpine Linux container (a minimal distro) and defines the environment where the solver will run. Environment variables describing the input file (i.e., the AF) and the task to be solved by the solver can be added. The command specified by CMD is run when the container is started. The file solver
interface.sh is a shell-script file with a common interface used to run each solver; this file will be provided by the organisers of ICCMA’19, and only minor changes are needed to be personalised to a specific solver.
Solvers are evaluated with respect to their performance using the IPC score which is also used for the planning competition. More precisely, for each computational problem and each semantics (called track in the following) we consider a set of argument graphs. For each graph, each solver has a fixed amount of time (currently estimated to be 10 minutes) to solve the given computational problem. The solver obtains
- 1 point, if it delivers the correct AND COMPLETE result
- a fraction of point in case of correct AND INCOMPLETE result
- −5 points, if it delivers an incorrect result;
- 0 points otherwise (NO RESULTS IN THE GIVEN TIME, or exit for not enough memory)
For each track a ranking of the solvers is determined by their number of correctly and timely classified instances.