MiniZinc Challenge 2011 Results

• MiniZinc Challenge
• »Challenge 2011
• »Results

Entrants

The entrants for this year (with their descriptions, when provided):

• BProlog (description). A CLP(FD) solver.
• Bumblebee (description). Translates to SAT, uses CryptoMiniSAT.
• Gecode (description). A C++ FD solver.
• JaCoP (description). A Java FD solver.
• Fzn2smt (description). Translates to SMT, uses Yices.
• SCIP. A CP/MIP solver.

In addition, the challenge organisers entered the following FlatZinc implementations:

• Chuffed (description). A C++ FD solver using Lazy clause generation.
• CPX. A C++ FD Solver using Lazy Clause Generation.
• G12/FD. A Mercury FD solver (the G12 FlatZinc interpreter's default solver).
• G12/LazyFD. A Mercury FD solver using Lazy Clause Generation.
• G12/CBC. Translates to MIP, uses Cbc version 2.6.2.
• G12/CPLEX. Translates to MIP, uses CPLEX version 12.1.
• G12/Gurobi. Translates to MIP, uses Gurobi Optimizer version 4.5.

As per the challenge rules, these entries are not eligible for prizes, but do modify the scoring results. Furthermore, entries in the FD search category (BProlog, Gecode, JaCoP, Chuffed, CPX and G12/FD) were automatically included in the free search category, while entries in the free search category (Bumblebee, Fzn2smt, SCIP, CBC, CPLEX, Gurobi and promoted FD entries) were automatically included in the parallel search category.

Summary of Results

The results for the MiniZinc Challenge 2011 were

Results Presentation

Description of Results

All times are given in seconds.

Scores of 0, 1 and 2 are used in the tables rather than 0, 0.5 and 1.

If a promoted entry does not recognize an option (or states that it is just ignored), times and solutions from the previous category are used for scoring. The suffixes -fd, -free and -par at the end of the solvers names indicate which configuration the solvers were run with.

mzn2fzn was run with the same time and memory limits as the solvers.

In the Status column:

• S and U indicates whether a solution was found (S - Solved) or not (U - Unsolved).
• C and U indicates whether the search was complete (C) or uncomplete (U).
• E indicates an error of any kind, or an incorrect answer from the solver.

Incorrect answers:

• Bumblebee returns UC (unsatisfiable) for all instances of cyclic-rcpsp.
• Bumblebee returns UC (unsatisfiable) for all instances of wwtpp-real, in around 70 seconds for each of them. We have decided to count all 5 instances results as an error (effectively disqualified bumblebee on that problem).
• SCIP completes the search on 8Ships (ship-scheduling) with a suboptimal answer.
• cpx-par finds a better-than-optimal solution for 6ShipsMixedUnconst (ship-schedule).

Errors:

• SCIP aborts on the carpet-cutting instances when parsing the generated FlatZinc.
  (We checked the FlatZinc files manually and they are valid.)
• Bumblebee aborts on the vrp, four prize-collecting instances, ship-scheduling and table layout with a compiler failed error.
• Linearisation aborts (CBC, Cplex, Gurobi) on carpet-cutting, open stacks and pentominoes.
• CBC/Cplex/Gurobi aborts on pattern set mining due to a lack of support for the FlatZinc built-in bool_le_reif/3.
• SCIP prints out the complete search terminator even for SAT instances. It did not affect the scoring.
• Other errors are mostly due to memory exhaustion.

Selection:

Individual results:

Global constraint per model

The following table lists the global constraints used by each model in this year's challenge.

The files on this page are for MiniZinc version 1.3.