MiniZinc - Challenge 2015 Results

MiniZinc Challenge 2015 Results

Entrants

The entrants for this year (with their descriptions, when provided):

Choco 3 (description). A Java FD solver.
Concrete (description). A CP system written in Scala.
iZplus (description).
JaCoP (description). A Java FD solver.
MinisatID (description). A solver combining techniques from the fields SAT, SMT, CP, and ASP.
MinisatID-MP (description). A variant of MinisatID using a mulit-precision library.
Mistral 2.0 (description). A C++ FD solver.
Opturion CPX (description). A FD solver using Lazy clause generation.
OR-Tools (description).
OscaR/CBLS (description). A constraint-based local search solver written in Scala.
Picat CP (description).
Picat SAT (description).
sunny-cp^— (description). A variant of sunny-cp only using Opturion CPX, OR-Tools, iZplus, Choco 3, Gecode, MinisatID, and HaifaCSP (HCSP).
YACS (description). A local search solver written in Scala.

In addition, the challenge organisers entered the following FlatZinc and MiniZinc implementations:

chuffed (description). A C++ FD solver using lazy clause generation.
MZN/CPLEX. Translates to MIP, uses CPLEX Optimizer version 12.6.1.
G12/FD. A Mercury FD solver (the G12 FlatZinc interpreter's default solver).
Gecode (description). A C++ FD solver.
MZN/Gurobi. Translates to MIP, uses Gurobi Optimizer version 6.0.4.
sunny-cp (description). A multi-threaded CP portfolio solver using 13 different CP and MIP solvers incl. chuffed, G12 solvers, Gecode.

As per the challenge rules, these entries are not eligible for prizes, but do modify the scoring results. Furthermore, entries in the FD search category (JaCoP) were automatically included in the free search category, while entries in the free search category (Concrete, chuffed, G12/FD, MinisatID, MinisatID-MP, Mistral, Opturion CPX, OscaR/CBLS, Picat CP, Picat SAT and promoted FD entries) were automatically included in the parallel search category. Lastly, all entries in the parallel search category and promoted entries into that category were automatically included in the open search category.

Summary of Results

The results for the MiniZinc Challenge 2015 were

Category	Gold	Silver	Bronze
Fixed	Opturion CPX	OR-Tools	JaCoP
Free	Opturion CPX	iZplus	OR-Tools
Parallel	OR-Tools	Opturion CPX	Choco
Open	sunny-cp——	OR-Tools	Opturion CPX

Description of Results

All times are given in milliseconds.

A score of 0.0 indicates a worse answer in quality (worse objective, no proof of optimality, or no answer for satisfaction problems), 1.0 a better solution in quality. When the quality is the same, the 1.0 purse is split with respect to time used.

If a promoted entry does not recognize an option (or states that it is just ignored), times and solutions from the previous category are used for scoring. The suffixes -fd, -free, -par or -open (for the parallel portfolio solver entered) at the end of the solvers names indicate which configuration the solvers were run with.

mzn2fzn was run with the same time and memory limits as the solvers.

In the Status column:

S indicates that a solution was found,
C indicates that the search was complete,
UNK indicates an incorrect answer OR
UNK indicates that flattening aborted (time-out or out-of-memory) OR
UNK indicates that no answer was returned in the time limit or the solver aborted.

Download all problems

All problems are available in a zipped tar-ball here.

Selection:

Select a list of solvers and benchmarks and click on “Compute Results” to score the solvers against each other on the selected benchmarks. The entrants for each of the fd search, free search and parallel search categories can be selected with the corresponding buttons.

Solver selection:	Problem selection:

Summary:

Total per problem:

Solver	Score	Score Incomplete

Problem	Solver	Score	Score Incomplete

Individual results:

Problem	Instance	Solver	Status	Time	Objective	Score	Score Incomplete

Global constraint per model

The following table lists the global constraints used by each model in this year's challenge. In addition, the columns RC and SBC respectively indicates whether the model contains redundant or/and symmetry breaking constraints.

Problem	Type	Kind	RC	SBC	MiniZinc Globals
costas_array	combi	sat	✓	✓	all_different
cvrp	real	min	✓		circuit
freepizza	real	min
gfd-schedule	combi	min		✓	nvalue, at_most
grid-colouring	combi	min
is	real	min	✓		circuit, table
largescheduling	combi	min	✓		cumulative
mapping	real	min			bin_packing_load, count, maximum, network_flow_cost
multi-knapsack	combi	max			knapsack
nmseq	combi	sat
opd	combi	min		✓	lex_greatereq
open_stacks	combi	min			all_different
p1f	combi	min		✓	all_different, inverse, lex_less
project-planning	real	min
radiation	real	min
roster	combi	min			at_least, at_most, exactly
spot5	real	min			table
tdtsp	real	min	✓		inverse
triangular	puzzle	max			inverse
zephyrus	real	min

The files on this page are for MiniZinc version 2.0.4.