This page presents the ranking of the different solvers. Solvers are ranked on the number of solved instances. Ties are broken by considering the minimum total solution time.
Solvers which gave a wrong answer in a category are excluded from the ranking in that category.
The Virtual Best Solver (VBS) is a theoretical construction which returns the best answer provided by one of the submitted solver. It corresponds to the 'Best result' column in the bench per bench view of the results. It can be considered as a meta solver which uses a perfect oracle to guess instantaneously which of the submitted solver will be the most efficient to solve the instance. Another way to look at it is to consider this VBS as a solver which would run all other solvers in parallel, bringing together all the solvers strenghs. This VBS is essentially the same notion as State Of The Art (SOTA) solver defined in "Evaluating general purpose automated theorem proving systems", Artificial Intelligence, Volume 131, Issues 1-2, September 2001, Pages 39-54 Geoff Sutcliffe, Christian Suttner. However, SOTA is a more general notion: the VBS can be seen as the least powerful SOTA solver.
Rank | Solver | Version | Number of solved instances | Detail | % of all instances | % of VBS | Cumulated CPU time on solved instances | Average CPU time per solved instance |
---|---|---|---|---|---|---|---|---|
Total number of instances in the category: 292 | ||||||||
Virtual Best Solver (VBS) | 196 | 79 SAT, 117 UNSAT | 67% | 100% | 33863.84 | 172.77 | ||
1 | precosat | 236 | 164 | 65 SAT, 99 UNSAT | 56% | 84% | 37379.67 | 227.92 |
2 | MiniSat | 2.1 (Sat-race'08 Edition) | 155 | 65 SAT, 90 UNSAT | 53% | 79% | 27011.56 | 174.27 |
3 | LySAT | i/2009-03-20 | 153 | 57 SAT, 96 UNSAT | 52% | 78% | 35271.11 | 230.53 |
4 | glucose | 1.0 | 152 | 54 SAT, 98 UNSAT | 52% | 78% | 34784.84 | 228.85 |
5 | MiniSAT 09z | 2009-03-22 | 152 | 59 SAT, 93 UNSAT | 52% | 78% | 37872.87 | 249.16 |
6 | kw | 2009-03-20 | 150 | 58 SAT, 92 UNSAT | 51% | 77% | 35080.23 | 233.87 |
7 | ManySAT 1.1 | aimd 1/2009-03-20 | 149 | 54 SAT, 95 UNSAT | 51% | 76% | 34834.19 | 233.79 |
8 | ManySAT 1.1 | aimd 0/2009-03-20 | 149 | 54 SAT, 95 UNSAT | 51% | 76% | 38639.59 | 259.33 |
9 | MXC | 2009-03-10 | 147 | 62 SAT, 85 UNSAT | 50% | 75% | 27968.90 | 190.26 |
10 | ManySAT 1.1 | aimd 2/2009-03-20 | 145 | 51 SAT, 94 UNSAT | 50% | 74% | 34242.50 | 236.16 |
11 | CircUs | 2009-03-23 | 144 | 59 SAT, 85 UNSAT | 49% | 73% | 36680.28 | 254.72 |
12 | Rsat | 2009-03-22 | 143 | 53 SAT, 90 UNSAT | 49% | 73% | 31000.89 | 216.79 |
13 | SATzilla2009_I | 2009-03-22 | 142 | 60 SAT, 82 UNSAT | 49% | 72% | 33608.36 | 236.68 |
14 | minisat_cumr | p-2009-03-18 | 141 | 58 SAT, 83 UNSAT | 48% | 72% | 29304.08 | 207.83 |
15 | picosat | 913 | 139 | 63 SAT, 76 UNSAT | 48% | 71% | 34013.47 | 244.70 |
16 | clasp | 1.2.0-SAT09-32 | 138 | 53 SAT, 85 UNSAT | 47% | 70% | 33317.37 | 241.43 |
17 | SAT07 reference solver: Rsat | 2007-02-08 | 133 | 56 SAT, 77 UNSAT | 46% | 68% | 28975.23 | 217.86 |
18 | SApperloT | base | 129 | 55 SAT, 74 UNSAT | 44% | 66% | 31762.78 | 246.22 |
19 | SAT07 reference solver: picosat | 535 | 126 | 59 SAT, 67 UNSAT | 43% | 64% | 33871.13 | 268.82 |
20 | LySAT | c/2009-03-20 | 123 | 51 SAT, 72 UNSAT | 42% | 63% | 26865.49 | 218.42 |
21 | IUT_BMB_SAT | 1.0 | 116 | 46 SAT, 70 UNSAT | 40% | 59% | 20974.40 | 180.81 |
22 | HydraSAT | 2009-03-22-Base | 116 | 53 SAT, 63 UNSAT | 40% | 59% | 26856.33 | 231.52 |
23 | HydraSAT-Flat | 2009-03-22-Flat | 115 | 51 SAT, 64 UNSAT | 39% | 59% | 26016.34 | 226.23 |
24 | VARSAT-industrial | 2009-03-22 | 110 | 49 SAT, 61 UNSAT | 38% | 56% | 22753.77 | 206.85 |
25 | SApperloT | hrp | 107 | 42 SAT, 65 UNSAT | 37% | 55% | 20954.19 | 195.83 |
26 | HydraSAT-Multi | 2009-03-22-Multi | 106 | 49 SAT, 57 UNSAT | 36% | 54% | 16308.48 | 153.85 |
27 | SATzilla2009_C | 2009-03-22 | 106 | 45 SAT, 61 UNSAT | 36% | 54% | 25974.72 | 245.04 |
28 | VARSAT-crafted | 2009-03-22 | 99 | 44 SAT, 55 UNSAT | 34% | 51% | 23553.01 | 237.91 |
29 | SAT4J CORE | 2.1 RC1 | 95 | 46 SAT, 49 UNSAT | 33% | 48% | 25380.84 | 267.17 |
30 | satake | 2009-03-22 | 92 | 40 SAT, 52 UNSAT | 32% | 47% | 18309.62 | 199.02 |
31 | CSat | 2009-03-22 | 91 | 40 SAT, 51 UNSAT | 31% | 46% | 20461.14 | 224.85 |
32 | SATzilla2009_R | 2009-03-22 | 59 | 36 SAT, 23 UNSAT | 20% | 30% | 6260.03 | 106.10 |
33 | VARSAT-random | 2009-03-22 | 59 | 25 SAT, 34 UNSAT | 20% | 30% | 16836.65 | 285.37 |
34 | march_hi | hi | 21 | 9 SAT, 12 UNSAT | 7% | 11% | 5170.80 | 246.23 |
35 | march_nn | nn | 21 | 10 SAT, 11 UNSAT | 7% | 11% | 6189.51 | 294.74 |
36 | Hybrid2 | 2009-03-22 | 12 | 11 SAT, 1 UNSAT | 4% | 6% | 3851.84 | 320.99 |
37 | adaptg2wsat2009 | 2009-03-23 | 11 | 8 SAT, 3 UNSAT | 4% | 6% | 1746.45 | 158.77 |
38 | adaptg2wsat2009++ | 2009-03-23 | 11 | 8 SAT, 3 UNSAT | 4% | 6% | 1806.37 | 164.22 |
39 | slstc | 1.0 | 10 | 9 SAT, 1 UNSAT | 3% | 5% | 2093.29 | 209.33 |
40 | tts | tts-5-0 | 10 | 6 SAT, 4 UNSAT | 3% | 5% | 2539.03 | 253.90 |
41 | NCVWr | 2009-03-22 | 10 | 9 SAT, 1 UNSAT | 3% | 5% | 2973.72 | 297.37 |
42 | iPAWS | 2009-03-22 | 8 | 8 SAT | 3% | 4% | 1400.34 | 175.04 |
43 | ttsth-5-0 | 2009-03-19 | 8 | 4 SAT, 4 UNSAT | 3% | 4% | 2937.42 | 367.18 |
44 | hybridGM7 | 7 | 7 | 7 SAT | 2% | 4% | 468.76 | 66.97 |
45 | gnovelty+ | 2009-03-22 | 7 | 7 SAT | 2% | 4% | 1586.83 | 226.69 |
46 | gNovelty+-T | 2009-03-22 | 7 | 7 SAT | 2% | 4% | 1826.46 | 260.92 |
47 | TNM | 2009-03-22 | 6 | 5 SAT, 1 UNSAT | 2% | 3% | 1157.83 | 192.97 |
48 | hybridGM | 1 | 5 | 5 SAT | 2% | 3% | 731.62 | 146.32 |
49 | hybridGM3 | 3 | 5 | 5 SAT | 2% | 3% | 1103.11 | 220.62 |
50 | gnovelty+2 | 2009-03-22 | 4 | 4 SAT | 1% | 2% | 91.85 | 22.96 |
Rank | Solver | Version | Number of solved instances | % of all instances | % of VBS | Cumulated CPU time on solved instances | Average CPU time per solved instance |
---|---|---|---|---|---|---|---|
Total number of instances in the category: 292 | |||||||
Virtual Best Solver (VBS) | 79 | 27% | 100% | 8981.63 | 113.69 | ||
1 | MiniSat | 2.1 (Sat-race'08 Edition) | 65 | 22% | 82% | 8104.11 | 124.68 |
2 | precosat | 236 | 65 | 22% | 82% | 11641.55 | 179.10 |
3 | picosat | 913 | 63 | 22% | 80% | 12779.67 | 202.85 |
4 | MXC | 2009-03-10 | 62 | 21% | 78% | 8494.77 | 137.01 |
5 | SATzilla2009_I | 2009-03-22 | 60 | 21% | 76% | 10593.32 | 176.56 |
6 | MiniSAT 09z | 2009-03-22 | 59 | 20% | 75% | 9724.50 | 164.82 |
7 | CircUs | 2009-03-23 | 59 | 20% | 75% | 13064.44 | 221.43 |
8 | SAT07 reference solver: picosat | 535 | 59 | 20% | 75% | 14213.77 | 240.91 |
9 | minisat_cumr | p-2009-03-18 | 58 | 20% | 73% | 9145.58 | 157.68 |
10 | kw | 2009-03-20 | 58 | 20% | 73% | 10004.54 | 172.49 |
11 | LySAT | i/2009-03-20 | 57 | 20% | 72% | 11536.87 | 202.40 |
12 | SAT07 reference solver: Rsat | 2007-02-08 | 56 | 19% | 71% | 6923.51 | 123.63 |
13 | SApperloT | base | 55 | 19% | 70% | 9889.91 | 179.82 |
14 | glucose | 1.0 | 54 | 18% | 68% | 7478.34 | 138.49 |
15 | ManySAT 1.1 | aimd 0/2009-03-20 | 54 | 18% | 68% | 7983.87 | 147.85 |
16 | ManySAT 1.1 | aimd 1/2009-03-20 | 54 | 18% | 68% | 8270.36 | 153.15 |
17 | Rsat | 2009-03-22 | 53 | 18% | 67% | 6813.66 | 128.56 |
18 | clasp | 1.2.0-SAT09-32 | 53 | 18% | 67% | 8597.36 | 162.21 |
19 | HydraSAT | 2009-03-22-Base | 53 | 18% | 67% | 8759.45 | 165.27 |
20 | ManySAT 1.1 | aimd 2/2009-03-20 | 51 | 17% | 65% | 6647.69 | 130.35 |
21 | HydraSAT-Flat | 2009-03-22-Flat | 51 | 17% | 65% | 8488.27 | 166.44 |
22 | LySAT | c/2009-03-20 | 51 | 17% | 65% | 9499.56 | 186.27 |
23 | HydraSAT-Multi | 2009-03-22-Multi | 49 | 17% | 62% | 5579.12 | 113.86 |
24 | VARSAT-industrial | 2009-03-22 | 49 | 17% | 62% | 7133.73 | 145.59 |
25 | IUT_BMB_SAT | 1.0 | 46 | 16% | 58% | 4370.12 | 95.00 |
26 | SAT4J CORE | 2.1 RC1 | 46 | 16% | 58% | 8889.09 | 193.24 |
27 | SATzilla2009_C | 2009-03-22 | 45 | 15% | 57% | 7869.43 | 174.88 |
28 | VARSAT-crafted | 2009-03-22 | 44 | 15% | 56% | 8875.81 | 201.72 |
29 | SApperloT | hrp | 42 | 14% | 53% | 5571.58 | 132.66 |
30 | satake | 2009-03-22 | 40 | 14% | 51% | 6311.56 | 157.79 |
31 | CSat | 2009-03-22 | 40 | 14% | 51% | 7022.57 | 175.56 |
32 | SATzilla2009_R | 2009-03-22 | 36 | 12% | 46% | 3643.77 | 101.22 |
33 | VARSAT-random | 2009-03-22 | 25 | 9% | 32% | 6512.66 | 260.51 |
34 | Hybrid2 | 2009-03-22 | 11 | 4% | 14% | 3851.04 | 350.09 |
35 | march_nn | nn | 10 | 3% | 13% | 3627.52 | 362.75 |
36 | slstc | 1.0 | 9 | 3% | 11% | 2092.23 | 232.47 |
37 | NCVWr | 2009-03-22 | 9 | 3% | 11% | 2972.93 | 330.33 |
38 | march_hi | hi | 9 | 3% | 11% | 3063.43 | 340.38 |
39 | iPAWS | 2009-03-22 | 8 | 3% | 10% | 1400.34 | 175.04 |
40 | adaptg2wsat2009 | 2009-03-23 | 8 | 3% | 10% | 1651.93 | 206.49 |
41 | adaptg2wsat2009++ | 2009-03-23 | 8 | 3% | 10% | 1707.51 | 213.44 |
42 | hybridGM7 | 7 | 7 | 2% | 9% | 468.76 | 66.97 |
43 | gnovelty+ | 2009-03-22 | 7 | 2% | 9% | 1586.83 | 226.69 |
44 | gNovelty+-T | 2009-03-22 | 7 | 2% | 9% | 1826.46 | 260.92 |
45 | tts | tts-5-0 | 6 | 2% | 8% | 2460.64 | 410.11 |
46 | hybridGM | 1 | 5 | 2% | 6% | 731.62 | 146.32 |
47 | hybridGM3 | 3 | 5 | 2% | 6% | 1103.11 | 220.62 |
48 | TNM | 2009-03-22 | 5 | 2% | 6% | 1157.00 | 231.40 |
49 | gnovelty+2 | 2009-03-22 | 4 | 1% | 5% | 91.85 | 22.96 |
50 | ttsth-5-0 | 2009-03-19 | 4 | 1% | 5% | 2841.45 | 710.36 |
Rank | Solver | Version | Number of solved instances | % of all instances | % of VBS | Cumulated CPU time on solved instances | Average CPU time per solved instance |
---|---|---|---|---|---|---|---|
Total number of instances in the category: 292 | |||||||
Virtual Best Solver (VBS) | 117 | 40% | 100% | 24882.22 | 212.67 | ||
1 | precosat | 236 | 99 | 34% | 85% | 25738.12 | 259.98 |
2 | glucose | 1.0 | 98 | 34% | 84% | 27306.50 | 278.64 |
3 | LySAT | i/2009-03-20 | 96 | 33% | 82% | 23734.24 | 247.23 |
4 | ManySAT 1.1 | aimd 1/2009-03-20 | 95 | 33% | 81% | 26563.83 | 279.62 |
5 | ManySAT 1.1 | aimd 0/2009-03-20 | 95 | 33% | 81% | 30655.72 | 322.69 |
6 | ManySAT 1.1 | aimd 2/2009-03-20 | 94 | 32% | 80% | 27594.81 | 293.56 |
7 | MiniSAT 09z | 2009-03-22 | 93 | 32% | 79% | 28148.36 | 302.67 |
8 | kw | 2009-03-20 | 92 | 32% | 79% | 25075.69 | 272.56 |
9 | MiniSat | 2.1 (Sat-race'08 Edition) | 90 | 31% | 77% | 18907.45 | 210.08 |
10 | Rsat | 2009-03-22 | 90 | 31% | 77% | 24187.23 | 268.75 |
11 | MXC | 2009-03-10 | 85 | 29% | 73% | 19474.13 | 229.11 |
12 | CircUs | 2009-03-23 | 85 | 29% | 73% | 23615.84 | 277.83 |
13 | clasp | 1.2.0-SAT09-32 | 85 | 29% | 73% | 24720.01 | 290.82 |
14 | minisat_cumr | p-2009-03-18 | 83 | 28% | 71% | 20158.50 | 242.87 |
15 | SATzilla2009_I | 2009-03-22 | 82 | 28% | 70% | 23015.04 | 280.67 |
16 | SAT07 reference solver: Rsat | 2007-02-08 | 77 | 26% | 66% | 22051.72 | 286.39 |
17 | picosat | 913 | 76 | 26% | 65% | 21233.80 | 279.39 |
18 | SApperloT | base | 74 | 25% | 63% | 21872.87 | 295.58 |
19 | LySAT | c/2009-03-20 | 72 | 25% | 62% | 17365.93 | 241.19 |
20 | IUT_BMB_SAT | 1.0 | 70 | 24% | 60% | 16604.28 | 237.20 |
21 | SAT07 reference solver: picosat | 535 | 67 | 23% | 57% | 19657.36 | 293.39 |
22 | SApperloT | hrp | 65 | 22% | 56% | 15382.61 | 236.66 |
23 | HydraSAT-Flat | 2009-03-22-Flat | 64 | 22% | 55% | 17528.07 | 273.88 |
24 | HydraSAT | 2009-03-22-Base | 63 | 22% | 54% | 18096.88 | 287.25 |
25 | VARSAT-industrial | 2009-03-22 | 61 | 21% | 52% | 15620.04 | 256.07 |
26 | SATzilla2009_C | 2009-03-22 | 61 | 21% | 52% | 18105.29 | 296.81 |
27 | HydraSAT-Multi | 2009-03-22-Multi | 57 | 20% | 49% | 10729.36 | 188.23 |
28 | VARSAT-crafted | 2009-03-22 | 55 | 19% | 47% | 14677.20 | 266.86 |
29 | satake | 2009-03-22 | 52 | 18% | 44% | 11998.06 | 230.73 |
30 | CSat | 2009-03-22 | 51 | 17% | 44% | 13438.57 | 263.50 |
31 | SAT4J CORE | 2.1 RC1 | 49 | 17% | 42% | 16491.75 | 336.57 |
32 | VARSAT-random | 2009-03-22 | 34 | 12% | 29% | 10323.99 | 303.65 |
33 | SATzilla2009_R | 2009-03-22 | 23 | 8% | 20% | 2616.26 | 113.75 |
34 | march_hi | hi | 12 | 4% | 10% | 2107.36 | 175.61 |
35 | march_nn | nn | 11 | 4% | 9% | 2561.99 | 232.91 |
36 | tts | tts-5-0 | 4 | 1% | 3% | 78.39 | 19.60 |
37 | ttsth-5-0 | 2009-03-19 | 4 | 1% | 3% | 95.97 | 23.99 |
38 | adaptg2wsat2009 | 2009-03-23 | 3 | 1% | 3% | 94.52 | 31.51 |
39 | adaptg2wsat2009++ | 2009-03-23 | 3 | 1% | 3% | 98.86 | 32.95 |
40 | NCVWr | 2009-03-22 | 1 | 0% | 1% | 0.80 | 0.80 |
41 | Hybrid2 | 2009-03-22 | 1 | 0% | 1% | 0.80 | 0.80 |
42 | TNM | 2009-03-22 | 1 | 0% | 1% | 0.83 | 0.83 |
43 | slstc | 1.0 | 1 | 0% | 1% | 1.06 | 1.06 |
44 | iPAWS | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
45 | hybridGM | 1 | 0 | 0% | 0% | 0.00 | - |
46 | hybridGM3 | 3 | 0 | 0% | 0% | 0.00 | - |
47 | hybridGM7 | 7 | 0 | 0% | 0% | 0.00 | - |
48 | gnovelty+ | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
49 | gnovelty+2 | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
50 | gNovelty+-T | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
Rank | Solver | Version | Number of solved instances | Detail | % of all instances | % of VBS | Cumulated CPU time on solved instances | Average CPU time per solved instance |
---|---|---|---|---|---|---|---|---|
Total number of instances in the category: 281 | ||||||||
Virtual Best Solver (VBS) | 194 | 124 SAT, 70 UNSAT | 69% | 100% | 19204.67 | 98.99 | ||
1 | clasp | 1.2.0-SAT09-32 | 131 | 78 SAT, 53 UNSAT | 47% | 68% | 22257.76 | 169.91 |
2 | SATzilla2009_I | 2009-03-22 | 128 | 86 SAT, 42 UNSAT | 46% | 66% | 21700.11 | 169.53 |
3 | SATzilla2009_C | 2009-03-22 | 125 | 73 SAT, 52 UNSAT | 44% | 64% | 16701.85 | 133.61 |
4 | MXC | 2009-03-10 | 124 | 80 SAT, 44 UNSAT | 44% | 64% | 22256.57 | 179.49 |
5 | precosat | 236 | 122 | 81 SAT, 41 UNSAT | 43% | 63% | 22844.50 | 187.25 |
6 | IUT_BMB_SAT | 1.0 | 120 | 76 SAT, 44 UNSAT | 43% | 62% | 22395.97 | 186.63 |
7 | SAT07 reference solver: minisat | SAT 2007 | 119 | 76 SAT, 43 UNSAT | 42% | 61% | 22930.58 | 192.69 |
8 | SAT07 reference solver: SATzilla | CRAFTED | 114 | 82 SAT, 32 UNSAT | 41% | 59% | 18066.80 | 158.48 |
9 | MiniSat | 2.1 (Sat-race'08 Edition) | 114 | 74 SAT, 40 UNSAT | 41% | 59% | 18107.02 | 158.83 |
10 | glucose | 1.0 | 114 | 75 SAT, 39 UNSAT | 41% | 59% | 20823.96 | 182.67 |
11 | VARSAT-industrial | 2009-03-22 | 113 | 73 SAT, 40 UNSAT | 40% | 58% | 22306.77 | 197.41 |
12 | SApperloT | base | 113 | 73 SAT, 40 UNSAT | 40% | 58% | 22826.65 | 202.01 |
13 | picosat | 913 | 112 | 80 SAT, 32 UNSAT | 40% | 58% | 17111.73 | 152.78 |
14 | LySAT | c/2009-03-20 | 112 | 70 SAT, 42 UNSAT | 40% | 58% | 21080.61 | 188.22 |
15 | CircUs | 2009-03-23 | 107 | 70 SAT, 37 UNSAT | 38% | 55% | 16148.01 | 150.92 |
16 | kw | 2009-03-20 | 106 | 72 SAT, 34 UNSAT | 38% | 55% | 16460.37 | 155.29 |
17 | Rsat | 2009-03-22 | 105 | 71 SAT, 34 UNSAT | 37% | 54% | 14010.73 | 133.44 |
18 | SATzilla2009_R | 2009-03-22 | 104 | 78 SAT, 26 UNSAT | 37% | 54% | 14460.38 | 139.04 |
19 | ManySAT 1.1 | aimd 1/2009-03-20 | 103 | 72 SAT, 31 UNSAT | 37% | 53% | 14991.64 | 145.55 |
20 | HydraSAT-Multi | 2009-03-22-Multi | 103 | 70 SAT, 33 UNSAT | 37% | 53% | 20825.53 | 202.19 |
21 | HydraSAT-Flat | 2009-03-22-Flat | 102 | 70 SAT, 32 UNSAT | 36% | 53% | 17796.15 | 174.47 |
22 | SApperloT | hrp | 102 | 69 SAT, 33 UNSAT | 36% | 53% | 20647.84 | 202.43 |
23 | minisat_cumr | p-2009-03-18 | 102 | 75 SAT, 27 UNSAT | 36% | 53% | 23176.38 | 227.22 |
24 | VARSAT-crafted | 2009-03-22 | 102 | 61 SAT, 41 UNSAT | 36% | 53% | 23304.40 | 228.47 |
25 | LySAT | i/2009-03-20 | 100 | 69 SAT, 31 UNSAT | 36% | 52% | 14874.18 | 148.74 |
26 | ManySAT 1.1 | aimd 2/2009-03-20 | 99 | 70 SAT, 29 UNSAT | 35% | 51% | 14211.48 | 143.55 |
27 | ManySAT 1.1 | aimd 0/2009-03-20 | 99 | 71 SAT, 28 UNSAT | 35% | 51% | 15251.61 | 154.06 |
28 | HydraSAT | 2009-03-22-Base | 99 | 66 SAT, 33 UNSAT | 35% | 51% | 16718.94 | 168.88 |
29 | MiniSAT 09z | 2009-03-22 | 99 | 72 SAT, 27 UNSAT | 35% | 51% | 17027.31 | 171.99 |
30 | VARSAT-random | 2009-03-22 | 84 | 47 SAT, 37 UNSAT | 30% | 43% | 14023.19 | 166.94 |
31 | satake | 2009-03-22 | 75 | 55 SAT, 20 UNSAT | 27% | 39% | 16261.12 | 216.81 |
32 | iPAWS | 2009-03-22 | 71 | 71 SAT | 25% | 37% | 7352.89 | 103.56 |
33 | SAT4J CORE | 2.1 RC1 | 71 | 50 SAT, 21 UNSAT | 25% | 37% | 15136.95 | 213.20 |
34 | adaptg2wsat2009 | 2009-03-23 | 70 | 68 SAT, 2 UNSAT | 25% | 36% | 9425.51 | 134.65 |
35 | adaptg2wsat2009++ | 2009-03-23 | 66 | 64 SAT, 2 UNSAT | 23% | 34% | 5796.69 | 87.83 |
36 | Hybrid2 | 2009-03-22 | 66 | 66 SAT | 23% | 34% | 10425.56 | 157.96 |
37 | CSat | 2009-03-22 | 65 | 50 SAT, 15 UNSAT | 23% | 34% | 10319.33 | 158.76 |
38 | march_hi | hi | 63 | 45 SAT, 18 UNSAT | 22% | 32% | 10622.02 | 168.60 |
39 | TNM | 2009-03-22 | 62 | 62 SAT | 22% | 32% | 8181.19 | 131.95 |
40 | SAT07 reference solver: March KS | 2007-02-08 | 61 | 42 SAT, 19 UNSAT | 22% | 31% | 9021.93 | 147.90 |
41 | march_nn | nn | 58 | 43 SAT, 15 UNSAT | 21% | 30% | 6232.17 | 107.45 |
42 | gnovelty+ | 2009-03-22 | 54 | 54 SAT | 19% | 28% | 5853.95 | 108.41 |
43 | gNovelty+-T | 2009-03-22 | 53 | 53 SAT | 19% | 27% | 5073.82 | 95.73 |
44 | hybridGM | 1 | 51 | 51 SAT | 18% | 26% | 5298.30 | 103.89 |
45 | hybridGM3 | 3 | 51 | 51 SAT | 18% | 26% | 6737.29 | 132.10 |
46 | NCVWr | 2009-03-22 | 48 | 48 SAT | 17% | 25% | 12116.63 | 252.43 |
47 | tts | tts-5-0 | 46 | 25 SAT, 21 UNSAT | 16% | 24% | 2507.80 | 54.52 |
48 | ttsth-5-0 | 2009-03-19 | 46 | 24 SAT, 22 UNSAT | 16% | 24% | 4020.68 | 87.41 |
49 | gnovelty+2 | 2009-03-22 | 46 | 44 SAT, 2 UNSAT | 16% | 24% | 4840.28 | 105.22 |
50 | hybridGM7 | 7 | 38 | 38 SAT | 14% | 20% | 4385.10 | 115.40 |
51 | slstc | 1.0 | 33 | 33 SAT | 12% | 17% | 4228.67 | 128.14 |
Rank | Solver | Version | Number of solved instances | % of all instances | % of VBS | Cumulated CPU time on solved instances | Average CPU time per solved instance |
---|---|---|---|---|---|---|---|
Total number of instances in the category: 281 | |||||||
Virtual Best Solver (VBS) | 124 | 44% | 100% | 6981.50 | 56.30 | ||
1 | SATzilla2009_I | 2009-03-22 | 86 | 31% | 69% | 13179.69 | 153.25 |
2 | SAT07 reference solver: SATzilla | CRAFTED | 82 | 29% | 66% | 10771.00 | 131.35 |
3 | precosat | 236 | 81 | 29% | 65% | 14640.31 | 180.74 |
4 | picosat | 913 | 80 | 28% | 65% | 10666.41 | 133.33 |
5 | MXC | 2009-03-10 | 80 | 28% | 65% | 11550.05 | 144.38 |
6 | SATzilla2009_R | 2009-03-22 | 78 | 28% | 63% | 8500.41 | 108.98 |
7 | clasp | 1.2.0-SAT09-32 | 78 | 28% | 63% | 9083.59 | 116.46 |
8 | IUT_BMB_SAT | 1.0 | 76 | 27% | 61% | 11919.48 | 156.84 |
9 | SAT07 reference solver: minisat | SAT 2007 | 76 | 27% | 61% | 13440.20 | 176.84 |
10 | glucose | 1.0 | 75 | 27% | 60% | 12274.80 | 163.66 |
11 | minisat_cumr | p-2009-03-18 | 75 | 27% | 60% | 14494.53 | 193.26 |
12 | MiniSat | 2.1 (Sat-race'08 Edition) | 74 | 26% | 60% | 6563.28 | 88.69 |
13 | SATzilla2009_C | 2009-03-22 | 73 | 26% | 59% | 7808.26 | 106.96 |
14 | SApperloT | base | 73 | 26% | 59% | 10374.21 | 142.11 |
15 | VARSAT-industrial | 2009-03-22 | 73 | 26% | 59% | 13192.38 | 180.72 |
16 | ManySAT 1.1 | aimd 1/2009-03-20 | 72 | 26% | 58% | 7924.60 | 110.06 |
17 | MiniSAT 09z | 2009-03-22 | 72 | 26% | 58% | 9768.54 | 135.67 |
18 | kw | 2009-03-20 | 72 | 26% | 58% | 10059.34 | 139.71 |
19 | Rsat | 2009-03-22 | 71 | 25% | 57% | 6069.79 | 85.49 |
20 | iPAWS | 2009-03-22 | 71 | 25% | 57% | 7352.89 | 103.56 |
21 | ManySAT 1.1 | aimd 0/2009-03-20 | 71 | 25% | 57% | 8967.06 | 126.30 |
22 | ManySAT 1.1 | aimd 2/2009-03-20 | 70 | 25% | 56% | 7071.88 | 101.03 |
23 | CircUs | 2009-03-23 | 70 | 25% | 56% | 9182.71 | 131.18 |
24 | LySAT | c/2009-03-20 | 70 | 25% | 56% | 9188.34 | 131.26 |
25 | HydraSAT-Flat | 2009-03-22-Flat | 70 | 25% | 56% | 11042.24 | 157.75 |
26 | HydraSAT-Multi | 2009-03-22-Multi | 70 | 25% | 56% | 12819.38 | 183.13 |
27 | LySAT | i/2009-03-20 | 69 | 25% | 56% | 8309.51 | 120.43 |
28 | SApperloT | hrp | 69 | 25% | 56% | 10369.28 | 150.28 |
29 | adaptg2wsat2009 | 2009-03-23 | 68 | 24% | 55% | 9425.50 | 138.61 |
30 | HydraSAT | 2009-03-22-Base | 66 | 23% | 53% | 8839.42 | 133.93 |
31 | Hybrid2 | 2009-03-22 | 66 | 23% | 53% | 10425.56 | 157.96 |
32 | adaptg2wsat2009++ | 2009-03-23 | 64 | 23% | 52% | 5796.69 | 90.57 |
33 | TNM | 2009-03-22 | 62 | 22% | 50% | 8181.19 | 131.95 |
34 | VARSAT-crafted | 2009-03-22 | 61 | 22% | 49% | 14553.59 | 238.58 |
35 | satake | 2009-03-22 | 55 | 20% | 44% | 10012.44 | 182.04 |
36 | gnovelty+ | 2009-03-22 | 54 | 19% | 44% | 5853.95 | 108.41 |
37 | gNovelty+-T | 2009-03-22 | 53 | 19% | 43% | 5073.82 | 95.73 |
38 | hybridGM | 1 | 51 | 18% | 41% | 5298.30 | 103.89 |
39 | hybridGM3 | 3 | 51 | 18% | 41% | 6737.29 | 132.10 |
40 | CSat | 2009-03-22 | 50 | 18% | 40% | 7905.65 | 158.11 |
41 | SAT4J CORE | 2.1 RC1 | 50 | 18% | 40% | 9373.75 | 187.47 |
42 | NCVWr | 2009-03-22 | 48 | 17% | 39% | 12116.63 | 252.43 |
43 | VARSAT-random | 2009-03-22 | 47 | 17% | 38% | 8110.69 | 172.57 |
44 | march_hi | hi | 45 | 16% | 36% | 5335.21 | 118.56 |
45 | gnovelty+2 | 2009-03-22 | 44 | 16% | 35% | 4840.28 | 110.01 |
46 | march_nn | nn | 43 | 15% | 35% | 3399.05 | 79.05 |
47 | SAT07 reference solver: March KS | 2007-02-08 | 42 | 15% | 34% | 3699.64 | 88.09 |
48 | hybridGM7 | 7 | 38 | 14% | 31% | 4385.10 | 115.40 |
49 | slstc | 1.0 | 33 | 12% | 27% | 4228.67 | 128.14 |
50 | tts | tts-5-0 | 25 | 9% | 20% | 1183.31 | 47.33 |
51 | ttsth-5-0 | 2009-03-19 | 24 | 9% | 19% | 1268.49 | 52.85 |
Rank | Solver | Version | Number of solved instances | % of all instances | % of VBS | Cumulated CPU time on solved instances | Average CPU time per solved instance |
---|---|---|---|---|---|---|---|
Total number of instances in the category: 281 | |||||||
Virtual Best Solver (VBS) | 70 | 25% | 100% | 12223.17 | 174.62 | ||
1 | clasp | 1.2.0-SAT09-32 | 53 | 19% | 76% | 13174.17 | 248.57 |
2 | SATzilla2009_C | 2009-03-22 | 52 | 19% | 74% | 8893.59 | 171.03 |
3 | IUT_BMB_SAT | 1.0 | 44 | 16% | 63% | 10476.49 | 238.10 |
4 | MXC | 2009-03-10 | 44 | 16% | 63% | 10706.52 | 243.33 |
5 | SAT07 reference solver: minisat | SAT 2007 | 43 | 15% | 61% | 9490.38 | 220.71 |
6 | SATzilla2009_I | 2009-03-22 | 42 | 15% | 60% | 8520.42 | 202.87 |
7 | LySAT | c/2009-03-20 | 42 | 15% | 60% | 11892.27 | 283.15 |
8 | precosat | 236 | 41 | 15% | 59% | 8204.19 | 200.10 |
9 | VARSAT-crafted | 2009-03-22 | 41 | 15% | 59% | 8750.80 | 213.43 |
10 | VARSAT-industrial | 2009-03-22 | 40 | 14% | 57% | 9114.38 | 227.86 |
11 | MiniSat | 2.1 (Sat-race'08 Edition) | 40 | 14% | 57% | 11543.74 | 288.59 |
12 | SApperloT | base | 40 | 14% | 57% | 12452.44 | 311.31 |
13 | glucose | 1.0 | 39 | 14% | 56% | 8549.16 | 219.21 |
14 | VARSAT-random | 2009-03-22 | 37 | 13% | 53% | 5912.50 | 159.80 |
15 | CircUs | 2009-03-23 | 37 | 13% | 53% | 6965.29 | 188.25 |
16 | kw | 2009-03-20 | 34 | 12% | 49% | 6401.03 | 188.27 |
17 | Rsat | 2009-03-22 | 34 | 12% | 49% | 7940.94 | 233.56 |
18 | HydraSAT | 2009-03-22-Base | 33 | 12% | 47% | 7879.52 | 238.77 |
19 | HydraSAT-Multi | 2009-03-22-Multi | 33 | 12% | 47% | 8006.15 | 242.61 |
20 | SApperloT | hrp | 33 | 12% | 47% | 10278.56 | 311.47 |
21 | picosat | 913 | 32 | 11% | 46% | 6445.33 | 201.42 |
22 | HydraSAT-Flat | 2009-03-22-Flat | 32 | 11% | 46% | 6753.91 | 211.06 |
23 | SAT07 reference solver: SATzilla | CRAFTED | 32 | 11% | 46% | 7295.80 | 227.99 |
24 | LySAT | i/2009-03-20 | 31 | 11% | 44% | 6564.66 | 211.76 |
25 | ManySAT 1.1 | aimd 1/2009-03-20 | 31 | 11% | 44% | 7067.04 | 227.97 |
26 | ManySAT 1.1 | aimd 2/2009-03-20 | 29 | 10% | 41% | 7139.60 | 246.19 |
27 | ManySAT 1.1 | aimd 0/2009-03-20 | 28 | 10% | 40% | 6284.55 | 224.45 |
28 | MiniSAT 09z | 2009-03-22 | 27 | 10% | 39% | 7258.77 | 268.84 |
29 | minisat_cumr | p-2009-03-18 | 27 | 10% | 39% | 8681.85 | 321.55 |
30 | SATzilla2009_R | 2009-03-22 | 26 | 9% | 37% | 5959.97 | 229.23 |
31 | ttsth-5-0 | 2009-03-19 | 22 | 8% | 31% | 2752.19 | 125.10 |
32 | tts | tts-5-0 | 21 | 7% | 30% | 1324.49 | 63.07 |
33 | SAT4J CORE | 2.1 RC1 | 21 | 7% | 30% | 5763.20 | 274.44 |
34 | satake | 2009-03-22 | 20 | 7% | 29% | 6248.68 | 312.43 |
35 | SAT07 reference solver: March KS | 2007-02-08 | 19 | 7% | 27% | 5322.29 | 280.12 |
36 | march_hi | hi | 18 | 6% | 26% | 5286.81 | 293.71 |
37 | CSat | 2009-03-22 | 15 | 5% | 21% | 2413.68 | 160.91 |
38 | march_nn | nn | 15 | 5% | 21% | 2833.13 | 188.88 |
39 | adaptg2wsat2009 | 2009-03-23 | 2 | 1% | 3% | 0.00 | 0.00 |
40 | adaptg2wsat2009++ | 2009-03-23 | 2 | 1% | 3% | 0.00 | 0.00 |
41 | gnovelty+2 | 2009-03-22 | 2 | 1% | 3% | 0.00 | 0.00 |
42 | iPAWS | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
43 | Hybrid2 | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
44 | NCVWr | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
45 | TNM | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
46 | hybridGM | 1 | 0 | 0% | 0% | 0.00 | - |
47 | hybridGM3 | 3 | 0 | 0% | 0% | 0.00 | - |
48 | hybridGM7 | 7 | 0 | 0% | 0% | 0.00 | - |
49 | gnovelty+ | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
50 | slstc | 1.0 | 0 | 0% | 0% | 0.00 | - |
51 | gNovelty+-T | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
Rank | Solver | Version | Number of solved instances | Detail | % of all instances | % of VBS | Cumulated CPU time on solved instances | Average CPU time per solved instance |
---|---|---|---|---|---|---|---|---|
Total number of instances in the category: 570 | ||||||||
Virtual Best Solver (VBS) | 459 | 359 SAT, 100 UNSAT | 81% | 100% | 62339.75 | 135.82 | ||
1 | SATzilla2009_R | 2009-03-22 | 365 | 299 SAT, 66 UNSAT | 64% | 80% | 51997.72 | 142.46 |
2 | TNM | 2009-03-22 | 317 | 317 SAT | 56% | 69% | 35346.17 | 111.50 |
3 | gnovelty+2 | 2009-03-22 | 305 | 305 SAT | 54% | 66% | 26616.48 | 87.27 |
4 | hybridGM3 | 3 | 299 | 299 SAT | 52% | 65% | 23272.79 | 77.84 |
5 | hybridGM7 | 7 | 298 | 298 SAT | 52% | 65% | 25567.23 | 85.80 |
6 | adaptg2wsat2009++ | 2009-03-23 | 297 | 297 SAT | 52% | 65% | 26432.65 | 89.00 |
7 | hybridGM | 1 | 294 | 294 SAT | 52% | 64% | 23732.78 | 80.72 |
8 | adaptg2wsat2009 | 2009-03-23 | 294 | 294 SAT | 52% | 64% | 26658.47 | 90.68 |
9 | Hybrid2 | 2009-03-22 | 290 | 290 SAT | 51% | 63% | 30134.40 | 103.91 |
10 | gnovelty+ | 2009-03-22 | 281 | 281 SAT | 49% | 61% | 25523.72 | 90.83 |
11 | NCVWr | 2009-03-22 | 278 | 278 SAT | 49% | 61% | 31132.10 | 111.99 |
12 | SAT07 reference solver: gnovelty+ | 2007-02-08 | 272 | 272 SAT | 48% | 59% | 21956.28 | 80.72 |
13 | SAT07 reference solver: SATzilla | RANDOM | 268 | 177 SAT, 91 UNSAT | 47% | 58% | 42919.16 | 160.15 |
14 | gNovelty+-T | 2009-03-22 | 266 | 266 SAT | 47% | 58% | 22823.37 | 85.80 |
15 | SAT07 reference solver: adaptg2wsat+ | 2007-02-08 | 265 | 265 SAT | 46% | 58% | 22333.18 | 84.28 |
16 | iPAWS | 2009-03-22 | 258 | 258 SAT | 45% | 56% | 19296.93 | 74.79 |
17 | march_hi | hi | 247 | 147 SAT, 100 UNSAT | 43% | 54% | 65568.89 | 265.46 |
18 | march_nn | nn | 243 | 145 SAT, 98 UNSAT | 43% | 53% | 66494.85 | 273.64 |
19 | SAT07 reference solver: March KS | 2007-02-08 | 239 | 149 SAT, 90 UNSAT | 42% | 52% | 57869.03 | 242.13 |
20 | SATzilla2009_I | 2009-03-22 | 145 | 90 SAT, 55 UNSAT | 25% | 32% | 37645.86 | 259.63 |
21 | slstc | 1.0 | 118 | 118 SAT | 21% | 26% | 13250.77 | 112.29 |
22 | clasp | 1.2.0-SAT09-32 | 84 | 66 SAT, 18 UNSAT | 15% | 18% | 32979.32 | 392.61 |
23 | VARSAT-random | 2009-03-22 | 83 | 72 SAT, 11 UNSAT | 15% | 18% | 30273.41 | 364.74 |
24 | picosat | 913 | 79 | 57 SAT, 22 UNSAT | 14% | 17% | 29440.52 | 372.66 |
25 | SATzilla2009_C | 2009-03-22 | 73 | 61 SAT, 12 UNSAT | 13% | 16% | 22395.73 | 306.79 |
26 | VARSAT-industrial | 2009-03-22 | 71 | 61 SAT, 10 UNSAT | 12% | 15% | 27295.84 | 384.45 |
27 | VARSAT-crafted | 2009-03-22 | 70 | 60 SAT, 10 UNSAT | 12% | 15% | 27367.38 | 390.96 |
28 | SApperloT | base | 70 | 53 SAT, 17 UNSAT | 12% | 15% | 28249.79 | 403.57 |
29 | IUT_BMB_SAT | 1.0 | 63 | 50 SAT, 13 UNSAT | 11% | 14% | 25630.38 | 406.83 |
30 | MXC | 2009-03-10 | 61 | 50 SAT, 11 UNSAT | 11% | 13% | 28069.37 | 460.15 |
31 | LySAT | c/2009-03-20 | 60 | 48 SAT, 12 UNSAT | 11% | 13% | 24329.68 | 405.49 |
32 | MiniSat | 2.1 (Sat-race'08 Edition) | 41 | 37 SAT, 4 UNSAT | 7% | 9% | 16957.09 | 413.59 |
33 | minisat_cumr | p-2009-03-18 | 29 | 29 SAT | 5% | 6% | 14078.15 | 485.45 |
34 | precosat | 236 | 27 | 25 SAT, 2 UNSAT | 5% | 6% | 9522.84 | 352.70 |
35 | satake | 2009-03-22 | 24 | 24 SAT | 4% | 5% | 11034.05 | 459.75 |
36 | SApperloT | hrp | 17 | 13 SAT, 4 UNSAT | 3% | 4% | 7724.34 | 454.37 |
37 | glucose | 1.0 | 17 | 17 SAT | 3% | 4% | 7772.56 | 457.21 |
38 | HydraSAT-Flat | 2009-03-22-Flat | 16 | 15 SAT, 1 UNSAT | 3% | 3% | 5738.82 | 358.68 |
39 | HydraSAT-Multi | 2009-03-22-Multi | 16 | 16 SAT | 3% | 3% | 7836.19 | 489.76 |
40 | HydraSAT | 2009-03-22-Base | 13 | 13 SAT | 2% | 3% | 4930.65 | 379.28 |
41 | CircUs | 2009-03-23 | 8 | 8 SAT | 1% | 2% | 2553.24 | 319.16 |
42 | ManySAT 1.1 | aimd 0/2009-03-20 | 7 | 7 SAT | 1% | 2% | 1783.47 | 254.78 |
43 | ManySAT 1.1 | aimd 2/2009-03-20 | 6 | 6 SAT | 1% | 1% | 957.09 | 159.51 |
44 | LySAT | i/2009-03-20 | 6 | 6 SAT | 1% | 1% | 2124.49 | 354.08 |
45 | CSat | 2009-03-22 | 6 | 6 SAT | 1% | 1% | 2263.92 | 377.32 |
46 | Rsat | 2009-03-22 | 5 | 5 SAT | 1% | 1% | 1801.20 | 360.24 |
47 | ManySAT 1.1 | aimd 1/2009-03-20 | 5 | 5 SAT | 1% | 1% | 4144.36 | 828.87 |
48 | kw | 2009-03-20 | 4 | 4 SAT | 1% | 1% | 635.52 | 158.88 |
49 | SAT4J CORE | 2.1 RC1 | 4 | 4 SAT | 1% | 1% | 1440.19 | 360.05 |
50 | MiniSAT 09z | 2009-03-22 | 3 | 3 SAT | 1% | 1% | 1096.04 | 365.35 |
51 | tts | tts-5-0 | 0 | 0% | 0% | 0.00 | - | |
52 | ttsth-5-0 | 2009-03-19 | 0 | 0% | 0% | 0.00 | - |
Rank | Solver | Version | Number of solved instances | % of all instances | % of VBS | Cumulated CPU time on solved instances | Average CPU time per solved instance |
---|---|---|---|---|---|---|---|
Total number of instances in the category: 570 | |||||||
Virtual Best Solver (VBS) | 359 | 63% | 100% | 23984.32 | 66.81 | ||
1 | TNM | 2009-03-22 | 317 | 56% | 88% | 35346.17 | 111.50 |
2 | gnovelty+2 | 2009-03-22 | 305 | 54% | 85% | 26616.48 | 87.27 |
3 | hybridGM3 | 3 | 299 | 52% | 83% | 23272.79 | 77.84 |
4 | SATzilla2009_R | 2009-03-22 | 299 | 52% | 83% | 26986.91 | 90.26 |
5 | hybridGM7 | 7 | 298 | 52% | 83% | 25567.23 | 85.80 |
6 | adaptg2wsat2009++ | 2009-03-23 | 297 | 52% | 83% | 26432.65 | 89.00 |
7 | hybridGM | 1 | 294 | 52% | 82% | 23732.78 | 80.72 |
8 | adaptg2wsat2009 | 2009-03-23 | 294 | 52% | 82% | 26658.47 | 90.68 |
9 | Hybrid2 | 2009-03-22 | 290 | 51% | 81% | 30134.40 | 103.91 |
10 | gnovelty+ | 2009-03-22 | 281 | 49% | 78% | 25523.72 | 90.83 |
11 | NCVWr | 2009-03-22 | 278 | 49% | 77% | 31132.10 | 111.99 |
12 | SAT07 reference solver: gnovelty+ | 2007-02-08 | 272 | 48% | 76% | 21956.28 | 80.72 |
13 | gNovelty+-T | 2009-03-22 | 266 | 47% | 74% | 22823.37 | 85.80 |
14 | SAT07 reference solver: adaptg2wsat+ | 2007-02-08 | 265 | 46% | 74% | 22333.18 | 84.28 |
15 | iPAWS | 2009-03-22 | 258 | 45% | 72% | 19296.93 | 74.79 |
16 | SAT07 reference solver: SATzilla | RANDOM | 177 | 31% | 49% | 4555.67 | 25.74 |
17 | SAT07 reference solver: March KS | 2007-02-08 | 149 | 26% | 42% | 22567.89 | 151.46 |
18 | march_hi | hi | 147 | 26% | 41% | 25375.87 | 172.62 |
19 | march_nn | nn | 145 | 25% | 40% | 24089.39 | 166.13 |
20 | slstc | 1.0 | 118 | 21% | 33% | 13250.77 | 112.29 |
21 | SATzilla2009_I | 2009-03-22 | 90 | 16% | 25% | 20081.92 | 223.13 |
22 | VARSAT-random | 2009-03-22 | 72 | 13% | 20% | 22619.27 | 314.16 |
23 | clasp | 1.2.0-SAT09-32 | 66 | 12% | 18% | 23046.03 | 349.18 |
24 | SATzilla2009_C | 2009-03-22 | 61 | 11% | 17% | 16574.52 | 271.71 |
25 | VARSAT-industrial | 2009-03-22 | 61 | 11% | 17% | 21950.46 | 359.84 |
26 | VARSAT-crafted | 2009-03-22 | 60 | 11% | 17% | 22433.48 | 373.89 |
27 | picosat | 913 | 57 | 10% | 16% | 18768.71 | 329.28 |
28 | SApperloT | base | 53 | 9% | 15% | 18847.23 | 355.61 |
29 | IUT_BMB_SAT | 1.0 | 50 | 9% | 14% | 19072.95 | 381.46 |
30 | MXC | 2009-03-10 | 50 | 9% | 14% | 21951.88 | 439.04 |
31 | LySAT | c/2009-03-20 | 48 | 8% | 13% | 18352.61 | 382.35 |
32 | MiniSat | 2.1 (Sat-race'08 Edition) | 37 | 6% | 10% | 12866.00 | 347.73 |
33 | minisat_cumr | p-2009-03-18 | 29 | 5% | 8% | 14078.15 | 485.45 |
34 | precosat | 236 | 25 | 4% | 7% | 7912.41 | 316.50 |
35 | satake | 2009-03-22 | 24 | 4% | 7% | 11034.05 | 459.75 |
36 | glucose | 1.0 | 17 | 3% | 5% | 7772.56 | 457.21 |
37 | HydraSAT-Multi | 2009-03-22-Multi | 16 | 3% | 4% | 7836.19 | 489.76 |
38 | HydraSAT-Flat | 2009-03-22-Flat | 15 | 3% | 4% | 4847.49 | 323.17 |
39 | HydraSAT | 2009-03-22-Base | 13 | 2% | 4% | 4930.65 | 379.28 |
40 | SApperloT | hrp | 13 | 2% | 4% | 5395.32 | 415.02 |
41 | CircUs | 2009-03-23 | 8 | 1% | 2% | 2553.24 | 319.16 |
42 | ManySAT 1.1 | aimd 0/2009-03-20 | 7 | 1% | 2% | 1783.47 | 254.78 |
43 | ManySAT 1.1 | aimd 2/2009-03-20 | 6 | 1% | 2% | 957.09 | 159.51 |
44 | LySAT | i/2009-03-20 | 6 | 1% | 2% | 2124.49 | 354.08 |
45 | CSat | 2009-03-22 | 6 | 1% | 2% | 2263.92 | 377.32 |
46 | Rsat | 2009-03-22 | 5 | 1% | 1% | 1801.20 | 360.24 |
47 | ManySAT 1.1 | aimd 1/2009-03-20 | 5 | 1% | 1% | 4144.36 | 828.87 |
48 | kw | 2009-03-20 | 4 | 1% | 1% | 635.52 | 158.88 |
49 | SAT4J CORE | 2.1 RC1 | 4 | 1% | 1% | 1440.19 | 360.05 |
50 | MiniSAT 09z | 2009-03-22 | 3 | 1% | 1% | 1096.04 | 365.35 |
51 | tts | tts-5-0 | 0 | 0% | 0% | 0.00 | - |
52 | ttsth-5-0 | 2009-03-19 | 0 | 0% | 0% | 0.00 | - |
Rank | Solver | Version | Number of solved instances | % of all instances | % of VBS | Cumulated CPU time on solved instances | Average CPU time per solved instance |
---|---|---|---|---|---|---|---|
Total number of instances in the category: 570 | |||||||
Virtual Best Solver (VBS) | 100 | 18% | 100% | 38355.43 | 383.55 | ||
1 | march_hi | hi | 100 | 18% | 100% | 40193.02 | 401.93 |
2 | march_nn | nn | 98 | 17% | 98% | 42405.46 | 432.71 |
3 | SAT07 reference solver: SATzilla | RANDOM | 91 | 16% | 91% | 38363.49 | 421.58 |
4 | SAT07 reference solver: March KS | 2007-02-08 | 90 | 16% | 90% | 35301.14 | 392.23 |
5 | SATzilla2009_R | 2009-03-22 | 66 | 12% | 66% | 25010.81 | 378.95 |
6 | SATzilla2009_I | 2009-03-22 | 55 | 10% | 55% | 17563.94 | 319.34 |
7 | picosat | 913 | 22 | 4% | 22% | 10671.81 | 485.08 |
8 | clasp | 1.2.0-SAT09-32 | 18 | 3% | 18% | 9933.29 | 551.85 |
9 | SApperloT | base | 17 | 3% | 17% | 9402.56 | 553.09 |
10 | IUT_BMB_SAT | 1.0 | 13 | 2% | 13% | 6557.43 | 504.42 |
11 | SATzilla2009_C | 2009-03-22 | 12 | 2% | 12% | 5821.21 | 485.10 |
12 | LySAT | c/2009-03-20 | 12 | 2% | 12% | 5977.07 | 498.09 |
13 | MXC | 2009-03-10 | 11 | 2% | 11% | 6117.49 | 556.14 |
14 | VARSAT-random | 2009-03-22 | 11 | 2% | 11% | 7654.14 | 695.83 |
15 | VARSAT-crafted | 2009-03-22 | 10 | 2% | 10% | 4933.90 | 493.39 |
16 | VARSAT-industrial | 2009-03-22 | 10 | 2% | 10% | 5345.39 | 534.54 |
17 | SApperloT | hrp | 4 | 1% | 4% | 2329.01 | 582.25 |
18 | MiniSat | 2.1 (Sat-race'08 Edition) | 4 | 1% | 4% | 4091.09 | 1022.77 |
19 | precosat | 236 | 2 | 0% | 2% | 1610.43 | 805.21 |
20 | HydraSAT-Flat | 2009-03-22-Flat | 1 | 0% | 1% | 891.33 | 891.33 |
21 | SAT4J CORE | 2.1 RC1 | 0 | 0% | 0% | 0.00 | - |
22 | kw | 2009-03-20 | 0 | 0% | 0% | 0.00 | - |
23 | glucose | 1.0 | 0 | 0% | 0% | 0.00 | - |
24 | tts | tts-5-0 | 0 | 0% | 0% | 0.00 | - |
25 | LySAT | i/2009-03-20 | 0 | 0% | 0% | 0.00 | - |
26 | ManySAT 1.1 | aimd 0/2009-03-20 | 0 | 0% | 0% | 0.00 | - |
27 | ManySAT 1.1 | aimd 1/2009-03-20 | 0 | 0% | 0% | 0.00 | - |
28 | ManySAT 1.1 | aimd 2/2009-03-20 | 0 | 0% | 0% | 0.00 | - |
29 | iPAWS | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
30 | CSat | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
31 | HydraSAT-Multi | 2009-03-22-Multi | 0 | 0% | 0% | 0.00 | - |
32 | HydraSAT | 2009-03-22-Base | 0 | 0% | 0% | 0.00 | - |
33 | Hybrid2 | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
34 | NCVWr | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
35 | TNM | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
36 | hybridGM | 1 | 0 | 0% | 0% | 0.00 | - |
37 | hybridGM3 | 3 | 0 | 0% | 0% | 0.00 | - |
38 | hybridGM7 | 7 | 0 | 0% | 0% | 0.00 | - |
39 | gnovelty+ | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
40 | slstc | 1.0 | 0 | 0% | 0% | 0.00 | - |
41 | adaptg2wsat2009 | 2009-03-23 | 0 | 0% | 0% | 0.00 | - |
42 | adaptg2wsat2009++ | 2009-03-23 | 0 | 0% | 0% | 0.00 | - |
43 | CircUs | 2009-03-23 | 0 | 0% | 0% | 0.00 | - |
44 | Rsat | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
45 | gnovelty+2 | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
46 | ttsth-5-0 | 2009-03-19 | 0 | 0% | 0% | 0.00 | - |
47 | gNovelty+-T | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
48 | satake | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |
49 | SAT07 reference solver: gnovelty+ | 2007-02-08 | 0 | 0% | 0% | 0.00 | - |
50 | SAT07 reference solver: adaptg2wsat+ | 2007-02-08 | 0 | 0% | 0% | 0.00 | - |
51 | minisat_cumr | p-2009-03-18 | 0 | 0% | 0% | 0.00 | - |
52 | MiniSAT 09z | 2009-03-22 | 0 | 0% | 0% | 0.00 | - |