Aggregated results by semantics

In the tables below we provide the aggregated results of the solvers for each semantics (CO-D, PR-D, ST-D and GR-D). The column Solver, #Corr, #Cra, #Inc, #Fail, #Inv, #TO, #OOM, Time, Memory, Score corresponds to: name of the solver, number of correct answers, number of crashes, number of incorrect answers, number of failures, number of invalid answers, number of timeouts, number of out of memory, running time (total), used memory (max) and final score, respectively. In detail, #Cra refers to the number of times the solver crashed due to memory issues, #Fail are all those runs that did not produce an answer (e.g., because of timeouts), #Inc refers to syntactically correct, but wrong, answers, and #Inv is the number of syntactically incorrect answers.

Results for CO-D Track: overview.

CO-D
Solver #Corr #Cra #Inc #Fail #Inv #TO #OOM Time Memory Score
coquiaas 1295 0 0 0 0 9 0 12634.792 97075200 1295
mu-toksia 1301 0 0 0 1 2 0 6949.259 75563008 1301
pyglaf 1114 0 188 0 0 2 0 7040.852 97349632 174

Results for PR-D Track: overview.

PR-D
Solver #Corr #Cra #Inc #Fail #Inv #TO #OOM Time Memory Score
coquiaas 1284 0 0 2 0 18 0 25122.808 98557952 1284
mu-toksia 1302 0 0 0 0 2 0 7011.313 80859136 1302
pyglaf 967 0 335 0 0 2 0 7218.753 85041152 -708

Results for ST-D Track: overview.

ST-D
Solver #Corr #Cra #Inc #Fail #Inv #TO #OOM Time Memory Score
coquiaas 1280 0 20 0 0 4 0 6172.906 93192192 1180
mu-toksia 1302 0 0 0 0 2 0 6282.569 80138240 1302
pyglaf 1104 0 198 0 0 2 0 6493.680 99147776 114

Results for GR-D Track: overview.

GR-D
Solver #Corr #Cra #Inc #Fail #Inv #TO #OOM Time Memory Score
coquiaas 652 0 0 0 0 0 0 439.067 3055616 652
mu-toksia 652 0 0 0 0 0 0 1894.391 409554944 652
pyglaf 408 0 244 0 0 0 0 1943.953 409067520 -812