Trace number 3310314

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 35672.3 5100.02

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-bottom19.cnf
MD5SUMd725e0d6a363434dce7cb39ffdb1b83f
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables40286
Number of clauses134663
Sum of the clauses size343327
Maximum clause length1301
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 269181
Number of clauses of size 364779
Number of clauses of size 4189
Number of clauses of size 517
Number of clauses of size over 5496

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3310314-1304009413/watcher-3310314-1304009413 -o /tmp/evaluation-result-3310314-1304009413/solver-3310314-1304009413 -C 40000 -W 5100 -M 31000 ppfolio HOME/instance-3310314-1304009413.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.84 7.88 7.86 1/170 12444
/proc/meminfo: memFree=12592348/32951124 swapFree=67111528/67111528
[pid=12444] ppid=12442 vsize=18324 CPUtime=0 cores=0-7
/proc/12444/stat : 12444 (ppfolio) R 12442 12444 12375 0 -1 4202496 517 0 0 0 0 0 0 0 25 0 1 0 493357605 18763776 389 33554432000 4194304 4287625 140733331352496 18446744073709551615 212039675407 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/12444/statm: 4581 389 341 23 0 91 0

[startup+0.053776 s]
/proc/loadavg: 7.84 7.88 7.86 1/170 12444
/proc/meminfo: memFree=12592348/32951124 swapFree=67111528/67111528
[pid=12444] ppid=12442 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12444/stat : 12444 (ppfolio) S 12442 12444 12375 0 -1 4202496 519 1194 0 0 0 0 0 1 25 0 1 0 493357605 18763776 390 33554432000 4194304 4287625 140733331352496 18446744073709551615 212039675407 0 0 4096 65536 9224195386380386304 0 0 17 6 0 0 0
/proc/12444/statm: 4581 390 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+0.100765 s]
/proc/loadavg: 7.84 7.88 7.86 1/170 12444
/proc/meminfo: memFree=12592348/32951124 swapFree=67111528/67111528
[pid=12444] ppid=12442 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12444/stat : 12444 (ppfolio) S 12442 12444 12375 0 -1 4202496 519 1194 0 0 0 0 0 1 25 0 1 0 493357605 18763776 390 33554432000 4194304 4287625 140733331352496 18446744073709551615 212039675407 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/12444/statm: 4581 390 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+0.3007 s]
/proc/loadavg: 7.84 7.88 7.86 1/170 12444
/proc/meminfo: memFree=12592348/32951124 swapFree=67111528/67111528
[pid=12444] ppid=12442 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12444/stat : 12444 (ppfolio) S 12442 12444 12375 0 -1 4202496 519 1194 0 0 0 0 0 1 25 0 1 0 493357605 18763776 390 33554432000 4194304 4287625 140733331352496 18446744073709551615 212039675407 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/12444/statm: 4581 390 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+0.700626 s]
/proc/loadavg: 7.84 7.88 7.86 1/170 12444
/proc/meminfo: memFree=12592348/32951124 swapFree=67111528/67111528
[pid=12444] ppid=12442 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12444/stat : 12444 (ppfolio) S 12442 12444 12375 0 -1 4202496 519 1194 0 0 0 0 0 1 25 0 1 0 493357605 18763776 390 33554432000 4194304 4287625 140733331352496 18446744073709551615 212039675407 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/12444/statm: 4581 390 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+1.50048 s]
/proc/loadavg: 7.84 7.88 7.86 8/179 12459
/proc/meminfo: memFree=12465548/32951124 swapFree=67111528/67111528
[pid=12444] ppid=12442 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12444/stat : 12444 (ppfolio) S 12442 12444 12375 0 -1 4202496 519 1194 0 0 0 0 0 1 15 0 1 0 493357605 18763776 390 33554432000 4194304 4287625 140733331352496 18446744073709551615 212039675407 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/12444/statm: 4581 390 342 23 0 91 0
[pid=12445] ppid=12444 vsize=60088 CPUtime=1.49 cores=0
/proc/12445/stat : 12445 (cryptominisat) R 12444 12444 12375 0 -1 4202496 12646 0 0 0 147 2 0 0 25 0 1 0 493357605 61530112 9953 33554432000 4194304 4587354 140737361010816 18446744073709551615 4238611 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12445/statm: 15022 9953 330 96 0 11584 0
[pid=12446] ppid=12444 vsize=35684 CPUtime=1.48 cores=1
/proc/12446/stat : 12446 (clasp) R 12444 12444 12375 0 -1 4202496 7888 0 0 0 148 0 0 0 20 0 1 0 493357605 36540416 6146 33554432000 4194304 4785740 140735549305488 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12446/statm: 8921 6146 340 145 0 5966 0
[pid=12448] ppid=12444 vsize=92464 CPUtime=1.48 cores=3
/proc/12448/stat : 12448 (march_hi) R 12444 12444 12375 0 -1 4202496 48857 0 0 0 142 6 0 0 20 0 1 0 493357605 94683136 11394 33554432000 4194304 4331988 140735583533392 18446744073709551615 4228459 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12448/statm: 23116 11394 175 34 0 21558 0
[pid=12449] ppid=12444 vsize=150516 CPUtime=5.41 cores=4-7
/proc/12449/stat : 12449 (plingeling) S 12444 12444 12375 0 -1 4202496 19497 1249 0 0 539 2 0 0 18 0 5 0 493357605 154128384 10717 33554432000 4194304 4356188 140734681157040 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12449/statm: 37629 10717 164 40 0 36173 0
[pid=12449/tid=12456] ppid=12444 vsize=150516 CPUtime=1.31 cores=4-7
/proc/12449/task/12456/stat : 12456 (plingeling) R 12444 12444 12375 0 -1 4202560 2480 1249 0 0 131 0 0 0 25 0 5 0 493357622 154128384 10717 33554432000 4194304 4356188 140734681157040 18446744073709551615 4233360 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12449/tid=12457] ppid=12444 vsize=150516 CPUtime=1.3 cores=4-7
/proc/12449/task/12457/stat : 12457 (plingeling) R 12444 12444 12375 0 -1 4202560 2591 1249 0 0 130 0 0 0 25 0 5 0 493357622 154128384 10717 33554432000 4194304 4356188 140734681157040 18446744073709551615 4218160 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12449/tid=12458] ppid=12444 vsize=150516 CPUtime=1.29 cores=4-7
/proc/12449/task/12458/stat : 12458 (plingeling) R 12444 12444 12375 0 -1 4202560 2151 1249 0 0 129 0 0 0 25 0 5 0 493357622 154128384 10717 33554432000 4194304 4356188 140734681157040 18446744073709551615 4217704 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12449/tid=12459] ppid=12444 vsize=150516 CPUtime=1.32 cores=4-7
/proc/12449/task/12459/stat : 12459 (plingeling) R 12444 12444 12375 0 -1 4202560 3701 1249 0 0 132 0 0 0 25 0 5 0 493357622 154128384 10717 33554432000 4194304 4356188 140734681157040 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 9.87
Current children cumulated vsize (KiB) 357076

[startup+3.1012 s]
/proc/loadavg: 7.77 7.86 7.86 8/179 12459
/proc/meminfo: memFree=12431944/32951124 swapFree=67111528/67111528
[pid=12444] ppid=12442 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12444/stat : 12444 (ppfolio) S 12442 12444 12375 0 -1 4202496 519 1194 0 0 0 0 0 1 15 0 1 0 493357605 18763776 390 33554432000 4194304 4287625 140733331352496 18446744073709551615 212039675407 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/12444/statm: 4581 390 342 23 0 91 0
[pid=12445] ppid=12444 vsize=60088 CPUtime=3.09 cores=0
/proc/12445/stat : 12445 (cryptominisat) R 12444 12444 12375 0 -1 4202496 13193 0 0 0 307 2 0 0 25 0 1 0 493357605 61530112 10482 33554432000 4194304 4587354 140737361010816 18446744073709551615 4237740 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12445/statm: 15022 10482 330 96 0 11584 0
[pid=12446] ppid=12444 vsize=38056 CPUtime=3.08 cores=1
/proc/12446/stat : 12446 (clasp) R 12444 12444 12375 0 -1 4202496 8490 0 0 0 308 0 0 0 25 0 1 0 493357605 38969344 6748 33554432000 4194304 4785740 140735549305488 18446744073709551615 4545856 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12446/statm: 9514 6748 340 145 0 6559 0
[pid=12448] ppid=12444 vsize=97312 CPUtime=3.08 cores=3
/proc/12448/stat : 12448 (march_hi) R 12444 12444 12375 0 -1 4202496 50142 0 0 0 302 6 0 0 25 0 1 0 493357605 99647488 12679 33554432000 4194304 4331988 140735583533392 18446744073709551615 4237103 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12448/statm: 24328 12679 175 34 0 22770 0
[pid=12449] ppid=12444 vsize=152612 CPUtime=11.8 cores=4-7
/proc/12449/stat : 12449 (plingeling) S 12444 12444 12375 0 -1 4202496 20310 1249 0 0 1178 2 0 0 18 0 5 0 493357605 156274688 11249 33554432000 4194304 4356188 140734681157040 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12449/statm: 38153 11249 164 40 0 36697 0
[pid=12449/tid=12456] ppid=12444 vsize=152612 CPUtime=2.9 cores=4-7
/proc/12449/task/12456/stat : 12456 (plingeling) R 12444 12444 12375 0 -1 4202560 2754 1249 0 0 290 0 0 0 25 0 5 0 493357622 156274688 11249 33554432000 4194304 4356188 140734681157040 18446744073709551615 4211520 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12449/tid=12457] ppid=12444 vsize=152612 CPUtime=2.9 cores=4-7
/proc/12449/task/12457/stat : 12457 (plingeling) R 12444 12444 12375 0 -1 4202560 2591 1249 0 0 290 0 0 0 25 0 5 0 493357622 156274688 11249 33554432000 4194304 4356188 140734681157040 18446744073709551615 4332003 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12449/tid=12458] ppid=12444 vsize=152612 CPUtime=2.89 cores=4-7
/proc/12449/task/12458/stat : 12458 (plingeling) R 12444 12444 12375 0 -1 4202560 2425 1249 0 0 289 0 0 0 25 0 5 0 493357622 156274688 11249 33554432000 4194304 4356188 140734681157040 18446744073709551615 4211772 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12449/tid=12459] ppid=12444 vsize=152612 CPUtime=2.92 cores=4-7
/proc/12449/task/12459/stat : 12459 (plingeling) R 12444 12444 12375 0 -1 4202560 3966 1249 0 0 292 0 0 0 25 0 5 0 493357622 156274688 11249 33554432000 4194304 4356188 140734681157040 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 21.06
Current children cumulated vsize (KiB) 366392

[startup+6.30063 s]
/proc/loadavg: 7.77 7.86 7.86 8/179 12459
/proc/meminfo: memFree=12417684/32951124 swapFree=67111528/67111528
[pid=12444] ppid=12442 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12444/stat : 12444 (ppfolio) S 12442 12444 12375 0 -1 4202496 519 1194 0 0 0 0 0 1 15 0 1 0 493357605 18763776 390 33554432000 4194304 4287625 140733331352496 18446744073709551615 212039675407 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/12444/statm: 4581 390 342 23 0 91 0
[pid=12445] ppid=12444 vsize=917088 CPUtime=6.28 cores=0
/proc/12445/stat : 12445 (cryptominisat) R 12444 12444 12375 0 -1 4202496 15653 0 0 0 626 2 0 0 25 0 1 0 493357605 939098112 10609 33554432000 4194304 4587354 140737361010816 18446744073709551615 4450897 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12445/statm: 229272 10609 336 96 0 225834 0
[pid=12446] ppid=12444 vsize=43796 CPUtime=6.28 cores=1
/proc/12446/stat : 12446 (clasp) R 12444 12444 12375 0 -1 4202496 9924 0 0 0 627 1 0 0 25 0 1 0 493357605 44847104 8182 33554432000 4194304 4785740 140735549305488 18446744073709551615 4553032 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12446/statm: 10949 8182 340 145 0 7994 0
[pid=12448] ppid=12444 vsize=99224 CPUtime=6.28 cores=3
/proc/12448/stat : 12448 (march_hi) R 12444 12444 12375 0 -1 4202496 50626 0 0 0 622 6 0 0 25 0 1 0 493357605 101605376 13163 33554432000 4194304 4331988 140735583533392 18446744073709551615 4228605 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12448/statm: 24806 13163 175 34 0 23248 0
[pid=12449] ppid=12444 vsize=156616 CPUtime=24.59 cores=4-7
/proc/12449/stat : 12449 (plingeling) S 12444 12444 12375 0 -1 4202496 22472 1249 0 0 2456 3 0 0 18 0 5 0 493357605 160374784 12737 33554432000 4194304 4356188 140734681157040 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12449/statm: 39154 12737 164 40 0 37698 0
[pid=12449/tid=12456] ppid=12444 vsize=156616 CPUtime=6.08 cores=4-7
/proc/12449/task/12456/stat : 12456 (plingeling) R 12444 12444 12375 0 -1 4202560 2754 1249 0 0 608 0 0 0 25 0 5 0 493357622 160374784 12737 33554432000 4194304 4356188 140734681157040 18446744073709551615 4210823 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12449/tid=12457] ppid=12444 vsize=156616 CPUtime=6.09 cores=4-7
/proc/12449/task/12457/stat : 12457 (plingeling) R 12444 12444 12375 0 -1 4202560 2591 1249 0 0 609 0 0 0 25 0 5 0 493357622 160374784 12737 33554432000 4194304 4356188 140734681157040 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12449/tid=12458] ppid=12444 vsize=156616 CPUtime=6.09 cores=4-7
/proc/12449/task/12458/stat : 12458 (plingeling) R 12444 12444 12375 0 -1 4202560 2425 1249 0 0 609 0 0 0 25 0 5 0 493357622 160374784 12737 33554432000 4194304 4356188 140734681157040 18446744073709551615 4330792 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12449/tid=12459] ppid=12444 vsize=156616 CPUtime=6.12 cores=4-7
/proc/12449/task/12459/stat : 12459 (plingeling) R 12444 12444 12375 0 -1 4202560 6128 1249 0 0 612 0 0 0 25 0 5 0 493357622 160374784 12737 33554432000 4194304 4356188 140734681157040 18446744073709551615 4331383 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 43.44
Current children cumulated vsize (KiB) 1235048

[startup+12.7005 s]
/proc/loadavg: 7.65 7.84 7.85 8/179 12459
/proc/meminfo: memFree=12397672/32951124 swapFree=67111528/67111528
[pid=12444] ppid=12442 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12444/stat : 12444 (ppfolio) S 12442 12444 12375 0 -1 4202496 519 1194 0 0 0 0 0 1 15 0 1 0 493357605 18763776 390 33554432000 4194304 4287625 140733331352496 18446744073709551615 212039675407 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/12444/statm: 4581 390 342 23 0 91 0
[pid=12445] ppid=12444 vsize=923472 CPUtime=12.69 cores=0
/proc/12445/stat : 12445 (cryptominisat) R 12444 12444 12375 0 -1 4202496 20503 0 0 0 1266 3 0 0 25 0 1 0 493357605 945635328 13007 33554432000 4194304 4587354 140737361010816 18446744073709551615 4237467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12445/statm: 230868 13007 336 96 0 227430 0
[pid=12446] ppid=12444 vsize=49324 CPUtime=12.69 cores=1
/proc/12446/stat : 12446 (clasp) R 12444 12444 12375 0 -1 4202496 11293 0 0 0 1267 2 0 0 25 0 1 0 493357605 50507776 9551 33554432000 4194304 4785740 140735549305488 18446744073709551615 4309545 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12446/statm: 12331 9551 340 145 0 9376 0
[pid=12448] ppid=12444 vsize=104252 CPUtime=12.68 cores=3
/proc/12448/stat : 12448 (march_hi) R 12444 12444 12375 0 -1 4202496 51900 0 0 0 1262 6 0 0 25 0 1 0 493357605 106754048 14437 33554432000 4194304 4331988 140735583533392 18446744073709551615 4226614 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12448/statm: 26063 14437 175 34 0 24505 0
[pid=12449] ppid=12444 vsize=156596 CPUtime=50.15 cores=4-7
/proc/12449/stat : 12449 (plingeling) S 12444 12444 12375 0 -1 4202496 23748 1249 0 0 5012 3 0 0 18 0 5 0 493357605 160354304 12553 33554432000 4194304 4356188 140734681157040 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12449/statm: 39149 12553 164 40 0 37693 0
[pid=12449/tid=12456] ppid=12444 vsize=156596 CPUtime=12.45 cores=4-7
/proc/12449/task/12456/stat : 12456 (plingeling) R 12444 12444 12375 0 -1 4202560 2754 1249 0 0 1245 0 0 0 25 0 5 0 493357622 160354304 12553 33554432000 4194304 4356188 140734681157040 18446744073709551615 4250504 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12449/tid=12457] ppid=12444 vsize=156596 CPUtime=12.49 cores=4-7
/proc/12449/task/12457/stat : 12457 (plingeling) R 12444 12444 12375 0 -1 4202560 2591 1249 0 0 1249 0 0 0 25 0 5 0 493357622 160354304 12553 33554432000 4194304 4356188 140734681157040 18446744073709551615 4281639 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12449/tid=12458] ppid=12444 vsize=156596 CPUtime=12.49 cores=4-7
/proc/12449/task/12458/stat : 12458 (plingeling) R 12444 12444 12375 0 -1 4202560 2425 1249 0 0 1249 0 0 0 25 0 5 0 493357622 160354304 12553 33554432000 4194304 4356188 140734681157040 18446744073709551615 4330817 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12449/tid=12459] ppid=12444 vsize=156596 CPUtime=12.52 cores=4-7
/proc/12449/task/12459/stat : 12459 (plingeling) R 12444 12444 12375 0 -1 4202560 7404 1249 0 0 1251 1 0 0 25 0 5 0 493357622 160354304 12553 33554432000 4194304 4356188 140734681157040 18446744073709551615 4236841 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 88.22
Current children cumulated vsize (KiB) 1251968

[startup+25.5012 s]
/proc/loadavg: 7.55 7.81 7.84 8/179 12460
/proc/meminfo: memFree=12369412/32951124 swapFree=67111528/67111528
[pid=12444] ppid=12442 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12444/stat : 12444 (ppfolio) S 12442 12444 12375 0 -1 4202496 519 1194 0 0 0 0 0 1 15 0 1 0 493357605 18763776 390 33554432000 4194304 4287625 140733331352496 18446744073709551615 212039675407 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/12444/statm: 4581 390 342 23 0 91 0
[pid=12445] ppid=12444 vsize=924136 CPUtime=25.48 cores=0
/proc/12445/stat : 12445 (cryptominisat) R 12444 12444 12375 0 -1 4202496 29759 0 0 0 2544 4 0 0 25 0 1 0 493357605 946315264 17570 33554432000 4194304 4587354 140737361010816 18446744073709551615 4388350 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12445/statm: 231034 17570 336 96 0 227596 0
[pid=12446] ppid=12444 vsize=55888 CPUtime=25.49 cores=1
/proc/12446/stat : 12446 (clasp) R 12444 12444 12375 0 -1 4202496 12889 0 0 0 2546 3 0 0 25 0 1 0 493357605 57229312 11147 33554432000 4194304 4785740 140735549305488 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12446/statm: 13972 11147 340 145 0 11017 0
[pid=12448] ppid=12444 vsize=106300 CPUtime=25.49 cores=3
/proc/12448/stat : 12448 (march_hi) R 12444 12444 12375 0 -1 4202496 52417 0 0 0 2542 7 0 0 25 0 1 0 493357605 108851200 14954 33554432000 4194304 4331988 140735583533392 18446744073709551615 4242904 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12448/statm: 26575 14954 175 34 0 25017 0
[pid=12449] ppid=12444 vsize=159564 CPUtime=101.28 cores=4-7
/proc/12449/stat : 12449 (plingeling) S 12444 12444 12375 0 -1 4202496 27049 1249 0 0 10124 4 0 0 18 0 5 0 493357605 163393536 13743 33554432000 4194304 4356188 140734681157040 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12449/statm: 39891 13743 164 40 0 38435 0
[pid=12449/tid=12456] ppid=12444 vsize=159564 CPUtime=25.18 cores=4-7
/proc/12449/task/12456/stat : 12456 (plingeling) R 12444 12444 12375 0 -1 4202560 2754 1249 0 0 2518 0 0 0 25 0 5 0 493357622 163393536 13743 33554432000 4194304 4356188 140734681157040 18446744073709551615 4331028 0 0 4096 17442 0 0 0 -1 7 0 0 0

################
# More data... #
################

Current children cumulated CPU time (s) 33869.8
Current children cumulated vsize (KiB) 1961504

[startup+4902.3 s]
/proc/loadavg: 7.02 7.01 7.00 8/179 12611
/proc/meminfo: memFree=11685008/32951124 swapFree=67111528/67111528
[pid=12444] ppid=12442 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12444/stat : 12444 (ppfolio) S 12442 12444 12375 0 -1 4202496 519 1194 0 0 0 0 0 1 15 0 1 0 493357605 18763776 390 33554432000 4194304 4287625 140733331352496 18446744073709551615 212039675407 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/12444/statm: 4581 390 342 23 0 91 0
[pid=12445] ppid=12444 vsize=962316 CPUtime=4902.28 cores=0
/proc/12445/stat : 12445 (cryptominisat) R 12444 12444 12375 0 -1 4202496 1043968 0 0 0 489850 378 0 0 25 0 1 0 493357605 985411584 29786 33554432000 4194304 4587354 140737361010816 18446744073709551615 4238512 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12445/statm: 240579 29786 336 96 0 237141 0
[pid=12446] ppid=12444 vsize=347408 CPUtime=4902.41 cores=1
/proc/12446/stat : 12446 (clasp) R 12444 12444 12375 0 -1 4202496 82817 0 0 0 490203 38 0 0 25 0 1 0 493357605 355745792 81075 33554432000 4194304 4785740 140735549305488 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12446/statm: 86852 81075 340 145 0 83897 0
[pid=12448] ppid=12444 vsize=273428 CPUtime=4902.42 cores=3
/proc/12448/stat : 12448 (march_hi) R 12444 12444 12375 0 -1 4202496 89901 0 0 0 490214 28 0 0 25 0 1 0 493357605 279990272 52390 33554432000 4194304 4331988 140735583533392 18446744073709551615 4226440 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12448/statm: 68357 52390 175 34 0 66799 0
[pid=12449] ppid=12444 vsize=354156 CPUtime=19582.3 cores=4-7
/proc/12449/stat : 12449 (plingeling) S 12444 12444 12375 0 -1 4202496 718213 1249 0 0 1958127 108 0 0 18 0 5 0 493357605 362655744 63908 33554432000 4194304 4356188 140734681157040 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12449/statm: 88539 63908 164 40 0 87083 0
[pid=12449/tid=12456] ppid=12444 vsize=354156 CPUtime=4875.51 cores=4-7
/proc/12449/task/12456/stat : 12456 (plingeling) R 12444 12444 12375 0 -1 4202560 222172 1249 0 0 487519 32 0 0 25 0 5 0 493357622 362655744 63908 33554432000 4194304 4356188 140734681157040 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12449/tid=12457] ppid=12444 vsize=354156 CPUtime=4902.21 cores=4-7
/proc/12449/task/12457/stat : 12457 (plingeling) R 12444 12444 12375 0 -1 4202560 188713 1249 0 0 490191 30 0 0 25 0 5 0 493357622 362655744 63908 33554432000 4194304 4356188 140734681157040 18446744073709551615 4332322 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12449/tid=12458] ppid=12444 vsize=354156 CPUtime=4902.2 cores=4-7
/proc/12449/task/12458/stat : 12458 (plingeling) R 12444 12444 12375 0 -1 4202560 160525 1249 0 0 490195 25 0 0 25 0 5 0 493357622 362655744 63908 33554432000 4194304 4356188 140734681157040 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12449/tid=12459] ppid=12444 vsize=354156 CPUtime=4902.24 cores=4-7
/proc/12449/task/12459/stat : 12459 (plingeling) R 12444 12444 12375 0 -1 4202560 138229 1249 0 0 490205 19 0 0 25 0 5 0 493357622 362655744 63908 33554432000 4194304 4356188 140734681157040 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 34289.5
Current children cumulated vsize (KiB) 1955632

[startup+4962.3 s]
/proc/loadavg: 7.00 7.00 7.00 8/179 12612
/proc/meminfo: memFree=11722536/32951124 swapFree=67111528/67111528
[pid=12444] ppid=12442 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12444/stat : 12444 (ppfolio) S 12442 12444 12375 0 -1 4202496 519 1194 0 0 0 0 0 1 15 0 1 0 493357605 18763776 390 33554432000 4194304 4287625 140733331352496 18446744073709551615 212039675407 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/12444/statm: 4581 390 342 23 0 91 0
[pid=12445] ppid=12444 vsize=982204 CPUtime=4962.28 cores=0
/proc/12445/stat : 12445 (cryptominisat) R 12444 12444 12375 0 -1 4202496 1052532 0 0 0 495847 381 0 0 25 0 1 0 493357605 1005776896 26559 33554432000 4194304 4587354 140737361010816 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12445/statm: 245551 26559 336 96 0 242113 0
[pid=12446] ppid=12444 vsize=348476 CPUtime=4962.41 cores=1
/proc/12446/stat : 12446 (clasp) R 12444 12444 12375 0 -1 4202496 83041 0 0 0 496203 38 0 0 25 0 1 0 493357605 356839424 81299 33554432000 4194304 4785740 140735549305488 18446744073709551615 4552521 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12446/statm: 87119 81299 340 145 0 84164 0
[pid=12448] ppid=12444 vsize=273560 CPUtime=4962.42 cores=3
/proc/12448/stat : 12448 (march_hi) R 12444 12444 12375 0 -1 4202496 90227 0 0 0 496214 28 0 0 25 0 1 0 493357605 280125440 52716 33554432000 4194304 4331988 140735583533392 18446744073709551615 4243200 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12448/statm: 68390 52716 175 34 0 66832 0
[pid=12449] ppid=12444 vsize=326508 CPUtime=19822 cores=4-7
/proc/12449/stat : 12449 (plingeling) S 12444 12444 12375 0 -1 4202496 724101 1249 0 0 1982094 109 0 0 18 0 5 0 493357605 334344192 56996 33554432000 4194304 4356188 140734681157040 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12449/statm: 81627 56996 164 40 0 80171 0
[pid=12449/tid=12456] ppid=12444 vsize=326508 CPUtime=4935.18 cores=4-7
/proc/12449/task/12456/stat : 12456 (plingeling) R 12444 12444 12375 0 -1 4202560 223708 1249 0 0 493486 32 0 0 25 0 5 0 493357622 334344192 56996 33554432000 4194304 4356188 140734681157040 18446744073709551615 4279755 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12449/tid=12457] ppid=12444 vsize=326508 CPUtime=4962.21 cores=4-7
/proc/12449/task/12457/stat : 12457 (plingeling) R 12444 12444 12375 0 -1 4202560 190505 1249 0 0 496191 30 0 0 25 0 5 0 493357622 334344192 56996 33554432000 4194304 4356188 140734681157040 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12449/tid=12458] ppid=12444 vsize=326508 CPUtime=4962.21 cores=4-7
/proc/12449/task/12458/stat : 12458 (plingeling) R 12444 12444 12375 0 -1 4202560 161037 1249 0 0 496196 25 0 0 25 0 5 0 493357622 334344192 56996 33554432000 4194304 4356188 140734681157040 18446744073709551615 4210587 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12449/tid=12459] ppid=12444 vsize=326508 CPUtime=4962.24 cores=4-7
/proc/12449/task/12459/stat : 12459 (plingeling) R 12444 12444 12375 0 -1 4202560 140277 1249 0 0 496205 19 0 0 25 0 5 0 493357622 334344192 56996 33554432000 4194304 4356188 140734681157040 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 34709.1
Current children cumulated vsize (KiB) 1949072

[startup+5022.3 s]
/proc/loadavg: 7.00 7.00 7.00 8/179 12614
/proc/meminfo: memFree=11682096/32951124 swapFree=67111528/67111528
[pid=12444] ppid=12442 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12444/stat : 12444 (ppfolio) S 12442 12444 12375 0 -1 4202496 519 1194 0 0 0 0 0 1 15 0 1 0 493357605 18763776 390 33554432000 4194304 4287625 140733331352496 18446744073709551615 212039675407 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/12444/statm: 4581 390 342 23 0 91 0
[pid=12445] ppid=12444 vsize=982204 CPUtime=5022.28 cores=0
/proc/12445/stat : 12445 (cryptominisat) R 12444 12444 12375 0 -1 4202496 1058415 0 0 0 501845 383 0 0 25 0 1 0 493357605 1005776896 32354 33554432000 4194304 4587354 140737361010816 18446744073709551615 4237695 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12445/statm: 245551 32354 336 96 0 242113 0
[pid=12446] ppid=12444 vsize=349188 CPUtime=5022.41 cores=1
/proc/12446/stat : 12446 (clasp) R 12444 12444 12375 0 -1 4202496 83186 0 0 0 502203 38 0 0 25 0 1 0 493357605 357568512 81444 33554432000 4194304 4785740 140735549305488 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12446/statm: 87297 81444 340 145 0 84342 0
[pid=12448] ppid=12444 vsize=273692 CPUtime=5022.42 cores=3
/proc/12448/stat : 12448 (march_hi) R 12444 12444 12375 0 -1 4202496 90526 0 0 0 502214 28 0 0 25 0 1 0 493357605 280260608 53015 33554432000 4194304 4331988 140735583533392 18446744073709551615 4237144 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12448/statm: 68423 53015 175 34 0 66865 0
[pid=12449] ppid=12444 vsize=343916 CPUtime=20061.7 cores=4-7
/proc/12449/stat : 12449 (plingeling) S 12444 12444 12375 0 -1 4202496 740485 1249 0 0 2006060 111 0 0 18 0 5 0 493357605 352169984 60836 33554432000 4194304 4356188 140734681157040 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12449/statm: 85979 60836 164 40 0 84523 0
[pid=12449/tid=12456] ppid=12444 vsize=343916 CPUtime=4994.85 cores=4-7
/proc/12449/task/12456/stat : 12456 (plingeling) R 12444 12444 12375 0 -1 4202560 230876 1249 0 0 499452 33 0 0 25 0 5 0 493357622 352169984 60836 33554432000 4194304 4356188 140734681157040 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12449/tid=12457] ppid=12444 vsize=343916 CPUtime=5022.21 cores=4-7
/proc/12449/task/12457/stat : 12457 (plingeling) R 12444 12444 12375 0 -1 4202560 196649 1249 0 0 502190 31 0 0 25 0 5 0 493357622 352169984 60836 33554432000 4194304 4356188 140734681157040 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12449/tid=12458] ppid=12444 vsize=343916 CPUtime=5022.2 cores=4-7
/proc/12449/task/12458/stat : 12458 (plingeling) R 12444 12444 12375 0 -1 4202560 162573 1249 0 0 502195 25 0 0 25 0 5 0 493357622 352169984 60836 33554432000 4194304 4356188 140734681157040 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12449/tid=12459] ppid=12444 vsize=343916 CPUtime=5022.24 cores=4-7
/proc/12449/task/12459/stat : 12459 (plingeling) R 12444 12444 12375 0 -1 4202560 141813 1249 0 0 502205 19 0 0 25 0 5 0 493357622 352169984 60836 33554432000 4194304 4356188 140734681157040 18446744073709551615 4331023 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 35128.8
Current children cumulated vsize (KiB) 1967324

[startup+5082.3 s]
/proc/loadavg: 7.00 7.00 7.00 8/179 12616
/proc/meminfo: memFree=11671136/32951124 swapFree=67111528/67111528
[pid=12444] ppid=12442 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12444/stat : 12444 (ppfolio) S 12442 12444 12375 0 -1 4202496 519 1194 0 0 0 0 0 1 15 0 1 0 493357605 18763776 390 33554432000 4194304 4287625 140733331352496 18446744073709551615 212039675407 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/12444/statm: 4581 390 342 23 0 91 0
[pid=12445] ppid=12444 vsize=968452 CPUtime=5082.28 cores=0
/proc/12445/stat : 12445 (cryptominisat) R 12444 12444 12375 0 -1 4202496 1067830 0 0 0 507840 388 0 0 25 0 1 0 493357605 991694848 29915 33554432000 4194304 4587354 140737361010816 18446744073709551615 4237609 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12445/statm: 242113 29915 336 96 0 238675 0
[pid=12446] ppid=12444 vsize=351964 CPUtime=5082.41 cores=1
/proc/12446/stat : 12446 (clasp) R 12444 12444 12375 0 -1 4202496 83818 0 0 0 508203 38 0 0 25 0 1 0 493357605 360411136 82076 33554432000 4194304 4785740 140735549305488 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12446/statm: 87991 82076 340 145 0 85036 0
[pid=12448] ppid=12444 vsize=273824 CPUtime=5082.42 cores=3
/proc/12448/stat : 12448 (march_hi) R 12444 12444 12375 0 -1 4202496 90815 0 0 0 508213 29 0 0 25 0 1 0 493357605 280395776 53304 33554432000 4194304 4331988 140735583533392 18446744073709551615 4226440 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12448/statm: 68456 53304 175 34 0 66898 0
[pid=12449] ppid=12444 vsize=361324 CPUtime=20301.4 cores=4-7
/proc/12449/stat : 12449 (plingeling) S 12444 12444 12375 0 -1 4202496 747110 1249 0 0 2030026 112 0 0 18 0 5 0 493357605 369995776 65412 33554432000 4194304 4356188 140734681157040 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12449/statm: 90331 65412 164 40 0 88875 0
[pid=12449/tid=12456] ppid=12444 vsize=361324 CPUtime=5054.53 cores=4-7
/proc/12449/task/12456/stat : 12456 (plingeling) R 12444 12444 12375 0 -1 4202560 234204 1249 0 0 505419 34 0 0 25 0 5 0 493357622 369995776 65412 33554432000 4194304 4356188 140734681157040 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12449/tid=12457] ppid=12444 vsize=361324 CPUtime=5082.21 cores=4-7
/proc/12449/task/12457/stat : 12457 (plingeling) R 12444 12444 12375 0 -1 4202560 197161 1249 0 0 508190 31 0 0 25 0 5 0 493357622 369995776 65412 33554432000 4194304 4356188 140734681157040 18446744073709551615 4210776 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12449/tid=12458] ppid=12444 vsize=361324 CPUtime=5082.2 cores=4-7
/proc/12449/task/12458/stat : 12458 (plingeling) R 12444 12444 12375 0 -1 4202560 164621 1249 0 0 508195 25 0 0 25 0 5 0 493357622 369995776 65412 33554432000 4194304 4356188 140734681157040 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12449/tid=12459] ppid=12444 vsize=361324 CPUtime=5082.24 cores=4-7
/proc/12449/task/12459/stat : 12459 (plingeling) R 12444 12444 12375 0 -1 4202560 142550 1249 0 0 508205 19 0 0 25 0 5 0 493357622 369995776 65412 33554432000 4194304 4356188 140734681157040 18446744073709551615 4331383 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 35548.5
Current children cumulated vsize (KiB) 1973888



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 7.00 7.00 7.00 8/179 12616
/proc/meminfo: memFree=11665296/32951124 swapFree=67111528/67111528
[pid=12444] ppid=12442 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12444/stat : 12444 (ppfolio) S 12442 12444 12375 0 -1 4202496 519 1194 0 0 0 0 0 1 15 0 1 0 493357605 18763776 390 33554432000 4194304 4287625 140733331352496 18446744073709551615 212039675407 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/12444/statm: 4581 390 342 23 0 91 0
[pid=12445] ppid=12444 vsize=968452 CPUtime=5099.99 cores=0
/proc/12445/stat : 12445 (cryptominisat) R 12444 12444 12375 0 -1 4202496 1069956 0 0 0 509609 390 0 0 25 0 1 0 493357605 991694848 32014 33554432000 4194304 4587354 140737361010816 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12445/statm: 242113 32014 336 96 0 238675 0
[pid=12446] ppid=12444 vsize=352508 CPUtime=5100.11 cores=1
/proc/12446/stat : 12446 (clasp) R 12444 12444 12375 0 -1 4202496 83915 0 0 0 509973 38 0 0 25 0 1 0 493357605 360968192 82173 33554432000 4194304 4785740 140735549305488 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12446/statm: 88127 82173 340 145 0 85172 0
[pid=12448] ppid=12444 vsize=273824 CPUtime=5100.13 cores=3
/proc/12448/stat : 12448 (march_hi) R 12444 12444 12375 0 -1 4202496 90937 0 0 0 509984 29 0 0 25 0 1 0 493357605 280395776 53426 33554432000 4194304 4331988 140735583533392 18446744073709551615 4226448 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12448/statm: 68456 53426 175 34 0 66898 0
[pid=12449] ppid=12444 vsize=357228 CPUtime=20372.1 cores=4-7
/proc/12449/stat : 12449 (plingeling) S 12444 12444 12375 0 -1 4202496 748134 1249 0 0 2037099 112 0 0 18 0 5 0 493357605 365801472 64388 33554432000 4194304 4356188 140734681157040 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12449/statm: 89307 64388 164 40 0 87851 0
[pid=12449/tid=12456] ppid=12444 vsize=357228 CPUtime=5072.13 cores=4-7
/proc/12449/task/12456/stat : 12456 (plingeling) R 12444 12444 12375 0 -1 4202560 234716 1249 0 0 507179 34 0 0 25 0 5 0 493357622 365801472 64388 33554432000 4194304 4356188 140734681157040 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12449/tid=12457] ppid=12444 vsize=357228 CPUtime=5099.92 cores=4-7
/proc/12449/task/12457/stat : 12457 (plingeling) R 12444 12444 12375 0 -1 4202560 197161 1249 0 0 509961 31 0 0 25 0 5 0 493357622 365801472 64388 33554432000 4194304 4356188 140734681157040 18446744073709551615 4331047 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12449/tid=12458] ppid=12444 vsize=357228 CPUtime=5099.91 cores=4-7
/proc/12449/task/12458/stat : 12458 (plingeling) R 12444 12444 12375 0 -1 4202560 164621 1249 0 0 509966 25 0 0 25 0 5 0 493357622 365801472 64388 33554432000 4194304 4356188 140734681157040 18446744073709551615 4332165 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12449/tid=12459] ppid=12444 vsize=357228 CPUtime=5099.94 cores=4-7
/proc/12449/task/12459/stat : 12459 (plingeling) R 12444 12444 12375 0 -1 4202560 143062 1249 0 0 509975 19 0 0 25 0 5 0 493357622 365801472 64388 33554432000 4194304 4356188 140734681157040 18446744073709551615 4331074 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 35672.3
Current children cumulated vsize (KiB) 1970336

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 12444 and gives
#  childrusage.ru_utime.tv_sec=5096
#  childrusage.ru_utime.tv_usec=98275
#  childrusage.ru_stime.tv_sec=3
#  childrusage.ru_stime.tv_usec=924403
# CPU time returned by wait4() is 5100.02
# while last known CPU time is 35672.3
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5100.02
CPU time (s): 35672.3
CPU user time (s): 35666.6
CPU system time (s): 5.7
CPU usage (%): 699.456
Max. virtual memory (cumulated for all children) (KiB): 7884384

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 5096.1
system time used= 3.9244
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1071669
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 464
involuntary context switches= 7487

runsolver used 8.78266 second user time and 19.0451 second system time

The end

Launcher Data 

Begin job on node146 at 2011-04-28 18:50:13
IDJOB=3310314
IDBENCH=83119
IDSOLVER=1638
FILE ID=node146/3310314-1304009413
RUNJOBID= node146-1304009324-12389
PBS_JOBID= 13168852
Free space on /tmp= 72992 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-bottom19.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3310314-1304009413/watcher-3310314-1304009413 -o /tmp/evaluation-result-3310314-1304009413/solver-3310314-1304009413 -C 40000 -W 5100 -M 31000  ppfolio HOME/instance-3310314-1304009413.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= d725e0d6a363434dce7cb39ffdb1b83f
RANDOM SEED=660473830

node146.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      12592504 kB
Buffers:       2087700 kB
Cached:       17670508 kB
SwapCached:          0 kB
Active:       10117840 kB
Inactive:      9703800 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      12592504 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           12432 kB
Writeback:           0 kB
AnonPages:       63464 kB
Mapped:          14748 kB
Slab:           470600 kB
PageTables:       4456 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   183860 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72980 MiB
End job on node146 at 2011-04-28 20:15:15