Trace number 3310320

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 35672.8 5100.02

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-bottom18.cnf
MD5SUM9ece8f569c93ceee834817ef3350ed8e
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables36410
Number of clauses121578
Sum of the clauses size309786
Maximum clause length1188
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 262692
Number of clauses of size 358247
Number of clauses of size 4170
Number of clauses of size 516
Number of clauses of size over 5452

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3310320-1304010365/watcher-3310320-1304010365 -o /tmp/evaluation-result-3310320-1304010365/solver-3310320-1304010365 -C 40000 -W 5100 -M 31000 ppfolio HOME/instance-3310320-1304010365.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.23 7.77 7.91 1/170 6412
/proc/meminfo: memFree=15644092/32951124 swapFree=67111528/67111528
[pid=6412] ppid=6410 vsize=18324 CPUtime=0 cores=0-7
/proc/6412/stat : 6412 (ppfolio) R 6410 6412 5982 0 -1 4202496 517 0 0 0 0 0 0 0 22 0 1 0 512636915 18763776 389 33554432000 4194304 4287625 140736785381792 18446744073709551615 244446965263 0 0 4096 65536 0 0 0 17 2 0 0 0
/proc/6412/statm: 4581 389 341 23 0 91 0

[startup+0.072752 s]
/proc/loadavg: 7.23 7.77 7.91 1/170 6412
/proc/meminfo: memFree=15644092/32951124 swapFree=67111528/67111528
[pid=6412] ppid=6410 vsize=18324 CPUtime=0.01 cores=0-7
/proc/6412/stat : 6412 (ppfolio) S 6410 6412 5982 0 -1 4202496 520 1194 0 0 0 0 0 1 22 0 1 0 512636915 18763776 391 33554432000 4194304 4287625 140736785381792 18446744073709551615 244446965263 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/6412/statm: 4581 391 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+0.100739 s]
/proc/loadavg: 7.23 7.77 7.91 1/170 6412
/proc/meminfo: memFree=15644092/32951124 swapFree=67111528/67111528
[pid=6412] ppid=6410 vsize=18324 CPUtime=0.01 cores=0-7
/proc/6412/stat : 6412 (ppfolio) S 6410 6412 5982 0 -1 4202496 520 1194 0 0 0 0 0 1 22 0 1 0 512636915 18763776 391 33554432000 4194304 4287625 140736785381792 18446744073709551615 244446965263 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/6412/statm: 4581 391 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+0.300676 s]
/proc/loadavg: 7.23 7.77 7.91 1/170 6412
/proc/meminfo: memFree=15644092/32951124 swapFree=67111528/67111528
[pid=6412] ppid=6410 vsize=18324 CPUtime=0.01 cores=0-7
/proc/6412/stat : 6412 (ppfolio) S 6410 6412 5982 0 -1 4202496 520 1194 0 0 0 0 0 1 22 0 1 0 512636915 18763776 391 33554432000 4194304 4287625 140736785381792 18446744073709551615 244446965263 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/6412/statm: 4581 391 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+0.700602 s]
/proc/loadavg: 7.23 7.77 7.91 1/170 6412
/proc/meminfo: memFree=15644092/32951124 swapFree=67111528/67111528
[pid=6412] ppid=6410 vsize=18324 CPUtime=0.01 cores=0-7
/proc/6412/stat : 6412 (ppfolio) S 6410 6412 5982 0 -1 4202496 520 1194 0 0 0 0 0 1 22 0 1 0 512636915 18763776 391 33554432000 4194304 4287625 140736785381792 18446744073709551615 244446965263 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/6412/statm: 4581 391 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+1.50045 s]
/proc/loadavg: 7.23 7.77 7.91 8/179 6427
/proc/meminfo: memFree=15524980/32951124 swapFree=67111528/67111528
[pid=6412] ppid=6410 vsize=18324 CPUtime=0.01 cores=0-7
/proc/6412/stat : 6412 (ppfolio) S 6410 6412 5982 0 -1 4202496 520 1194 0 0 0 0 0 1 16 0 1 0 512636915 18763776 391 33554432000 4194304 4287625 140736785381792 18446744073709551615 244446965263 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/6412/statm: 4581 391 342 23 0 91 0
[pid=6413] ppid=6412 vsize=56676 CPUtime=1.48 cores=0
/proc/6413/stat : 6413 (cryptominisat) R 6412 6412 5982 0 -1 4202496 12584 0 0 0 147 1 0 0 25 0 1 0 512636915 58036224 9026 33554432000 4194304 4587354 140734877656496 18446744073709551615 4237928 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6413/statm: 14169 9026 330 96 0 10731 0
[pid=6414] ppid=6412 vsize=33264 CPUtime=1.48 cores=1
/proc/6414/stat : 6414 (clasp) R 6412 6412 5982 0 -1 4202496 7159 0 0 0 148 0 0 0 25 0 1 0 512636915 34062336 5545 33554432000 4194304 4785740 140736961024224 18446744073709551615 4553163 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6414/statm: 8316 5545 340 145 0 5361 0
[pid=6416] ppid=6412 vsize=84292 CPUtime=1.49 cores=3
/proc/6416/stat : 6416 (march_hi) R 6412 6412 5982 0 -1 4202496 44162 0 0 0 144 5 0 0 20 0 1 0 512636915 86315008 10357 33554432000 4194304 4331988 140734087688624 18446744073709551615 4228470 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6416/statm: 21073 10357 176 34 0 19515 0
[pid=6417] ppid=6412 vsize=153664 CPUtime=5.48 cores=4-7
/proc/6417/stat : 6417 (plingeling) S 6412 6412 5982 0 -1 4202496 17085 1249 0 0 546 2 0 0 18 0 5 0 512636915 157351936 10477 33554432000 4194304 4356188 140734808298720 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6417/statm: 38416 10477 164 40 0 36960 0
[pid=6417/tid=6424] ppid=6412 vsize=153664 CPUtime=1.33 cores=4-7
/proc/6417/task/6424/stat : 6424 (plingeling) R 6412 6412 5982 0 -1 4202560 2769 1249 0 0 133 0 0 0 19 0 5 0 512636931 157351936 10477 33554432000 4194304 4356188 140734808298720 18446744073709551615 4231618 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6417/tid=6425] ppid=6412 vsize=153664 CPUtime=1.33 cores=4-7
/proc/6417/task/6425/stat : 6425 (plingeling) R 6412 6412 5982 0 -1 4202560 2369 1249 0 0 133 0 0 0 25 0 5 0 512636931 157351936 10477 33554432000 4194304 4356188 140734808298720 18446744073709551615 4217730 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6417/tid=6426] ppid=6412 vsize=153664 CPUtime=1.33 cores=4-7
/proc/6417/task/6426/stat : 6426 (plingeling) R 6412 6412 5982 0 -1 4202560 1495 1249 0 0 133 0 0 0 23 0 5 0 512636931 157351936 10477 33554432000 4194304 4356188 140734808298720 18446744073709551615 4301882 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6417/tid=6427] ppid=6412 vsize=153664 CPUtime=1.32 cores=4-7
/proc/6417/task/6427/stat : 6427 (plingeling) R 6412 6412 5982 0 -1 4202560 1886 1249 0 0 132 0 0 0 23 0 5 0 512636931 157351936 10477 33554432000 4194304 4356188 140734808298720 18446744073709551615 4236939 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 9.94
Current children cumulated vsize (KiB) 346220

[startup+3.10115 s]
/proc/loadavg: 7.21 7.76 7.91 8/179 6427
/proc/meminfo: memFree=15498084/32951124 swapFree=67111528/67111528
[pid=6412] ppid=6410 vsize=18324 CPUtime=0.01 cores=0-7
/proc/6412/stat : 6412 (ppfolio) S 6410 6412 5982 0 -1 4202496 520 1194 0 0 0 0 0 1 15 0 1 0 512636915 18763776 391 33554432000 4194304 4287625 140736785381792 18446744073709551615 244446965263 0 0 4096 65536 9224195742862639362 0 0 17 6 0 0 0
/proc/6412/statm: 4581 391 342 23 0 91 0
[pid=6413] ppid=6412 vsize=56676 CPUtime=3.09 cores=0
/proc/6413/stat : 6413 (cryptominisat) R 6412 6412 5982 0 -1 4202496 13139 0 0 0 307 2 0 0 25 0 1 0 512636915 58036224 9569 33554432000 4194304 4587354 140734877656496 18446744073709551615 4238491 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6413/statm: 14169 9569 330 96 0 10731 0
[pid=6414] ppid=6412 vsize=36328 CPUtime=3.09 cores=1
/proc/6414/stat : 6414 (clasp) R 6412 6412 5982 0 -1 4202496 7929 0 0 0 308 1 0 0 25 0 1 0 512636915 37199872 6315 33554432000 4194304 4785740 140736961024224 18446744073709551615 4550575 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6414/statm: 9082 6315 340 145 0 6127 0
[pid=6416] ppid=6412 vsize=87464 CPUtime=3.09 cores=3
/proc/6416/stat : 6416 (march_hi) R 6412 6412 5982 0 -1 4202496 44883 0 0 0 304 5 0 0 25 0 1 0 512636915 89563136 11077 33554432000 4194304 4331988 140734087688624 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6416/statm: 21866 11077 175 34 0 20308 0
[pid=6417] ppid=6412 vsize=153492 CPUtime=11.88 cores=4-7
/proc/6417/stat : 6417 (plingeling) S 6412 6412 5982 0 -1 4202496 17089 1249 0 0 1186 2 0 0 18 0 5 0 512636915 157175808 10438 33554432000 4194304 4356188 140734808298720 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6417/statm: 38373 10438 164 40 0 36917 0
[pid=6417/tid=6424] ppid=6412 vsize=153492 CPUtime=2.93 cores=4-7
/proc/6417/task/6424/stat : 6424 (plingeling) R 6412 6412 5982 0 -1 4202560 2769 1249 0 0 293 0 0 0 25 0 5 0 512636931 157175808 10438 33554432000 4194304 4356188 140734808298720 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6417/tid=6425] ppid=6412 vsize=153492 CPUtime=2.93 cores=4-7
/proc/6417/task/6425/stat : 6425 (plingeling) R 6412 6412 5982 0 -1 4202560 2369 1249 0 0 293 0 0 0 25 0 5 0 512636931 157175808 10438 33554432000 4194304 4356188 140734808298720 18446744073709551615 4330823 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6417/tid=6426] ppid=6412 vsize=153492 CPUtime=2.93 cores=4-7
/proc/6417/task/6426/stat : 6426 (plingeling) R 6412 6412 5982 0 -1 4202560 1499 1249 0 0 293 0 0 0 25 0 5 0 512636931 157175808 10438 33554432000 4194304 4356188 140734808298720 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6417/tid=6427] ppid=6412 vsize=153492 CPUtime=2.91 cores=4-7
/proc/6417/task/6427/stat : 6427 (plingeling) R 6412 6412 5982 0 -1 4202560 1886 1249 0 0 291 0 0 0 25 0 5 0 512636931 157175808 10438 33554432000 4194304 4356188 140734808298720 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 21.16
Current children cumulated vsize (KiB) 352284

[startup+6.30055 s]
/proc/loadavg: 7.21 7.76 7.91 8/179 6427
/proc/meminfo: memFree=15481592/32951124 swapFree=67111528/67111528
[pid=6412] ppid=6410 vsize=18324 CPUtime=0.01 cores=0-7
/proc/6412/stat : 6412 (ppfolio) S 6410 6412 5982 0 -1 4202496 520 1194 0 0 0 0 0 1 15 0 1 0 512636915 18763776 391 33554432000 4194304 4287625 140736785381792 18446744073709551615 244446965263 0 0 4096 65536 9224195742862639362 0 0 17 6 0 0 0
/proc/6412/statm: 4581 391 342 23 0 91 0
[pid=6413] ppid=6412 vsize=1494956 CPUtime=6.28 cores=0
/proc/6413/stat : 6413 (cryptominisat) R 6412 6412 5982 0 -1 4202496 15266 0 0 0 626 2 0 0 25 0 1 0 512636915 1530834944 9456 33554432000 4194304 4587354 140734877656496 18446744073709551615 4248785 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6413/statm: 373739 9456 336 96 0 370301 0
[pid=6414] ppid=6412 vsize=42640 CPUtime=6.29 cores=1
/proc/6414/stat : 6414 (clasp) R 6412 6412 5982 0 -1 4202496 9471 0 0 0 627 2 0 0 25 0 1 0 512636915 43663360 7857 33554432000 4194304 4785740 140736961024224 18446744073709551615 244446642295 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6414/statm: 10660 7858 340 145 0 7705 0
[pid=6416] ppid=6412 vsize=89120 CPUtime=6.29 cores=3
/proc/6416/stat : 6416 (march_hi) R 6412 6412 5982 0 -1 4202496 45330 0 0 0 624 5 0 0 25 0 1 0 512636915 91258880 11524 33554432000 4194304 4331988 140734087688624 18446744073709551615 4226474 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6416/statm: 22280 11524 175 34 0 20722 0
[pid=6417] ppid=6412 vsize=160008 CPUtime=24.65 cores=4-7
/proc/6417/stat : 6417 (plingeling) S 6412 6412 5982 0 -1 4202496 18747 1249 0 0 2463 2 0 0 18 0 5 0 512636915 163848192 12095 33554432000 4194304 4356188 140734808298720 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6417/statm: 40002 12095 164 40 0 38546 0
[pid=6417/tid=6424] ppid=6412 vsize=160008 CPUtime=6.13 cores=4-7
/proc/6417/task/6424/stat : 6424 (plingeling) R 6412 6412 5982 0 -1 4202560 2769 1249 0 0 613 0 0 0 25 0 5 0 512636931 163848192 12095 33554432000 4194304 4356188 140734808298720 18446744073709551615 4215940 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6417/tid=6425] ppid=6412 vsize=160008 CPUtime=6.13 cores=4-7
/proc/6417/task/6425/stat : 6425 (plingeling) R 6412 6412 5982 0 -1 4202560 2369 1249 0 0 613 0 0 0 25 0 5 0 512636931 163848192 12095 33554432000 4194304 4356188 140734808298720 18446744073709551615 4322024 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6417/tid=6426] ppid=6412 vsize=160008 CPUtime=6.13 cores=4-7
/proc/6417/task/6426/stat : 6426 (plingeling) R 6412 6412 5982 0 -1 4202560 1527 1249 0 0 613 0 0 0 25 0 5 0 512636931 163848192 12095 33554432000 4194304 4356188 140734808298720 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6417/tid=6427] ppid=6412 vsize=160008 CPUtime=6.09 cores=4-7
/proc/6417/task/6427/stat : 6427 (plingeling) R 6412 6412 5982 0 -1 4202560 3516 1249 0 0 609 0 0 0 25 0 5 0 512636931 163848192 12095 33554432000 4194304 4356188 140734808298720 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 43.52
Current children cumulated vsize (KiB) 1805048

[startup+12.7014 s]
/proc/loadavg: 7.18 7.73 7.90 8/179 6427
/proc/meminfo: memFree=15459796/32951124 swapFree=67111528/67111528
[pid=6412] ppid=6410 vsize=18324 CPUtime=0.01 cores=0-7
/proc/6412/stat : 6412 (ppfolio) S 6410 6412 5982 0 -1 4202496 520 1194 0 0 0 0 0 1 15 0 1 0 512636915 18763776 391 33554432000 4194304 4287625 140736785381792 18446744073709551615 244446965263 0 0 4096 65536 9224195742862639362 0 0 17 6 0 0 0
/proc/6412/statm: 4581 391 342 23 0 91 0
[pid=6413] ppid=6412 vsize=1513996 CPUtime=12.69 cores=0
/proc/6413/stat : 6413 (cryptominisat) R 6412 6412 5982 0 -1 4202496 21079 0 0 0 1265 4 0 0 25 0 1 0 512636915 1550331904 12925 33554432000 4194304 4587354 140734877656496 18446744073709551615 4238591 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6413/statm: 378499 12925 336 96 0 375061 0
[pid=6414] ppid=6412 vsize=50632 CPUtime=12.69 cores=1
/proc/6414/stat : 6414 (clasp) R 6412 6412 5982 0 -1 4202496 11482 0 0 0 1266 3 0 0 25 0 1 0 512636915 51847168 9868 33554432000 4194304 4785740 140736961024224 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6414/statm: 12658 9868 340 145 0 9703 0
[pid=6416] ppid=6412 vsize=89920 CPUtime=12.69 cores=3
/proc/6416/stat : 6416 (march_hi) R 6412 6412 5982 0 -1 4202496 45510 0 0 0 1264 5 0 0 25 0 1 0 512636915 92078080 11704 33554432000 4194304 4331988 140734087688624 18446744073709551615 4228470 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6416/statm: 22480 11704 175 34 0 20922 0
[pid=6417] ppid=6412 vsize=159988 CPUtime=50.22 cores=4-7
/proc/6417/stat : 6417 (plingeling) S 6412 6412 5982 0 -1 4202496 18747 1249 0 0 5020 2 0 0 18 0 5 0 512636915 163827712 12090 33554432000 4194304 4356188 140734808298720 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6417/statm: 39997 12090 164 40 0 38541 0
[pid=6417/tid=6424] ppid=6412 vsize=159988 CPUtime=12.53 cores=4-7
/proc/6417/task/6424/stat : 6424 (plingeling) R 6412 6412 5982 0 -1 4202560 2769 1249 0 0 1253 0 0 0 25 0 5 0 512636931 163827712 12090 33554432000 4194304 4356188 140734808298720 18446744073709551615 4317150 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6417/tid=6425] ppid=6412 vsize=159988 CPUtime=12.53 cores=4-7
/proc/6417/task/6425/stat : 6425 (plingeling) R 6412 6412 5982 0 -1 4202560 2369 1249 0 0 1253 0 0 0 25 0 5 0 512636931 163827712 12090 33554432000 4194304 4356188 140734808298720 18446744073709551615 4331419 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6417/tid=6426] ppid=6412 vsize=159988 CPUtime=12.53 cores=4-7
/proc/6417/task/6426/stat : 6426 (plingeling) R 6412 6412 5982 0 -1 4202560 1527 1249 0 0 1253 0 0 0 25 0 5 0 512636931 163827712 12090 33554432000 4194304 4356188 140734808298720 18446744073709551615 4238570 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6417/tid=6427] ppid=6412 vsize=159988 CPUtime=12.46 cores=4-7
/proc/6417/task/6427/stat : 6427 (plingeling) R 6412 6412 5982 0 -1 4202560 3516 1249 0 0 1246 0 0 0 25 0 5 0 512636931 163827712 12090 33554432000 4194304 4356188 140734808298720 18446744073709551615 4236939 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 88.3
Current children cumulated vsize (KiB) 1832860

[startup+25.501 s]
/proc/loadavg: 7.15 7.71 7.89 8/179 6428
/proc/meminfo: memFree=15420432/32951124 swapFree=67111528/67111528
[pid=6412] ppid=6410 vsize=18324 CPUtime=0.01 cores=0-7
/proc/6412/stat : 6412 (ppfolio) S 6410 6412 5982 0 -1 4202496 520 1194 0 0 0 0 0 1 15 0 1 0 512636915 18763776 391 33554432000 4194304 4287625 140736785381792 18446744073709551615 244446965263 0 0 4096 65536 9224195742862639362 0 0 17 6 0 0 0
/proc/6412/statm: 4581 391 342 23 0 91 0
[pid=6413] ppid=6412 vsize=1526304 CPUtime=25.48 cores=0
/proc/6413/stat : 6413 (cryptominisat) R 6412 6412 5982 0 -1 4202496 32127 0 0 0 2542 6 0 0 25 0 1 0 512636915 1562935296 18965 33554432000 4194304 4587354 140734877656496 18446744073709551615 244446641809 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6413/statm: 381576 18965 336 96 0 378138 0
[pid=6414] ppid=6412 vsize=58652 CPUtime=25.48 cores=1
/proc/6414/stat : 6414 (clasp) R 6412 6412 5982 0 -1 4202496 13486 0 0 0 2544 4 0 0 25 0 1 0 512636915 60059648 11872 33554432000 4194304 4785740 140736961024224 18446744073709551615 4278121 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6414/statm: 14663 11872 340 145 0 11708 0
[pid=6416] ppid=6412 vsize=91936 CPUtime=25.49 cores=3
/proc/6416/stat : 6416 (march_hi) R 6412 6412 5982 0 -1 4202496 46072 0 0 0 2544 5 0 0 25 0 1 0 512636915 94142464 12266 33554432000 4194304 4331988 140734087688624 18446744073709551615 4299158 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6416/statm: 22984 12266 175 34 0 21426 0
[pid=6417] ppid=6412 vsize=164436 CPUtime=101.36 cores=4-7
/proc/6417/stat : 6417 (plingeling) S 6412 6412 5982 0 -1 4202496 19860 1249 0 0 10133 3 0 0 18 0 5 0 512636915 168382464 13202 33554432000 4194304 4356188 140734808298720 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6417/statm: 41109 13202 164 40 0 39653 0
[pid=6417/tid=6424] ppid=6412 vsize=164436 CPUtime=25.33 cores=4-7
/proc/6417/task/6424/stat : 6424 (plingeling) R 6412 6412 5982 0 -1 4202560 2769 1249 0 0 2533 0 0 0 25 0 5 0 512636931 168382464 13202 33554432000 4194304 4356188 140734808298720 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 4 0 0 0

################
# More data... #
################

Current children cumulated CPU time (s) 33870.2
Current children cumulated vsize (KiB) 2672896

[startup+4902.3 s]
/proc/loadavg: 7.13 7.07 7.01 8/179 6570
/proc/meminfo: memFree=14615840/32951124 swapFree=67111528/67111528
[pid=6412] ppid=6410 vsize=18324 CPUtime=0.01 cores=0-7
/proc/6412/stat : 6412 (ppfolio) S 6410 6412 5982 0 -1 4202496 520 1194 0 0 0 0 0 1 15 0 1 0 512636915 18763776 391 33554432000 4194304 4287625 140736785381792 18446744073709551615 244446965263 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/6412/statm: 4581 391 342 23 0 91 0
[pid=6413] ppid=6412 vsize=1729924 CPUtime=4902.31 cores=0
/proc/6413/stat : 6413 (cryptominisat) R 6412 6412 5982 0 -1 4202496 484103 0 0 0 490014 217 0 0 25 0 1 0 512636915 1771442176 67626 33554432000 4194304 4587354 140734877656496 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6413/statm: 432481 67626 336 96 0 429043 0
[pid=6414] ppid=6412 vsize=333580 CPUtime=4902.46 cores=1
/proc/6414/stat : 6414 (clasp) R 6412 6412 5982 0 -1 4202496 79007 0 0 0 490208 38 0 0 25 0 1 0 512636915 341585920 77393 33554432000 4194304 4785740 140736961024224 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6414/statm: 83395 77393 340 145 0 80440 0
[pid=6416] ppid=6412 vsize=265940 CPUtime=4902.45 cores=3
/proc/6416/stat : 6416 (march_hi) R 6412 6412 5982 0 -1 4202496 83816 0 0 0 490217 28 0 0 25 0 1 0 512636915 272322560 49965 33554432000 4194304 4331988 140734087688624 18446744073709551615 4284558 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6416/statm: 66485 49965 175 34 0 64927 0
[pid=6417] ppid=6412 vsize=359096 CPUtime=19582.6 cores=4-7
/proc/6417/stat : 6417 (plingeling) S 6412 6412 5982 0 -1 4202496 668722 1249 0 0 1958166 97 0 0 18 0 5 0 512636915 367714304 61931 33554432000 4194304 4356188 140734808298720 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6417/statm: 89774 61931 164 40 0 88318 0
[pid=6417/tid=6424] ppid=6412 vsize=359096 CPUtime=4902.3 cores=4-7
/proc/6417/task/6424/stat : 6424 (plingeling) R 6412 6412 5982 0 -1 4202560 209789 1249 0 0 490199 31 0 0 25 0 5 0 512636931 367714304 61931 33554432000 4194304 4356188 140734808298720 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6417/tid=6425] ppid=6412 vsize=359096 CPUtime=4902.29 cores=4-7
/proc/6417/task/6425/stat : 6425 (plingeling) R 6412 6412 5982 0 -1 4202560 187757 1249 0 0 490202 27 0 0 25 0 5 0 512636931 367714304 61931 33554432000 4194304 4356188 140734808298720 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6417/tid=6426] ppid=6412 vsize=359096 CPUtime=4902.3 cores=4-7
/proc/6417/task/6426/stat : 6426 (plingeling) R 6412 6412 5982 0 -1 4202560 195226 1249 0 0 490202 28 0 0 25 0 5 0 512636931 367714304 61931 33554432000 4194304 4356188 140734808298720 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6417/tid=6427] ppid=6412 vsize=359096 CPUtime=4875.56 cores=4-7
/proc/6417/task/6427/stat : 6427 (plingeling) R 6412 6412 5982 0 -1 4202560 67384 1249 0 0 487547 9 0 0 25 0 5 0 512636931 367714304 61931 33554432000 4194304 4356188 140734808298720 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 34289.9
Current children cumulated vsize (KiB) 2706864

[startup+4962.3 s]
/proc/loadavg: 7.12 7.07 7.00 8/179 6571
/proc/meminfo: memFree=14624496/32951124 swapFree=67111528/67111528
[pid=6412] ppid=6410 vsize=18324 CPUtime=0.01 cores=0-7
/proc/6412/stat : 6412 (ppfolio) S 6410 6412 5982 0 -1 4202496 520 1194 0 0 0 0 0 1 15 0 1 0 512636915 18763776 391 33554432000 4194304 4287625 140736785381792 18446744073709551615 244446965263 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/6412/statm: 4581 391 342 23 0 91 0
[pid=6413] ppid=6412 vsize=1705572 CPUtime=4962.32 cores=0
/proc/6413/stat : 6413 (cryptominisat) R 6412 6412 5982 0 -1 4202496 487732 0 0 0 496013 219 0 0 25 0 1 0 512636915 1746505728 64896 33554432000 4194304 4587354 140734877656496 18446744073709551615 4238224 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6413/statm: 426393 64896 336 96 0 422955 0
[pid=6414] ppid=6412 vsize=335180 CPUtime=4962.46 cores=1
/proc/6414/stat : 6414 (clasp) R 6412 6412 5982 0 -1 4202496 79373 0 0 0 496208 38 0 0 25 0 1 0 512636915 343224320 77759 33554432000 4194304 4785740 140736961024224 18446744073709551615 4553094 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6414/statm: 83795 77759 340 145 0 80840 0
[pid=6416] ppid=6412 vsize=266208 CPUtime=4962.46 cores=3
/proc/6416/stat : 6416 (march_hi) R 6412 6412 5982 0 -1 4202496 84388 0 0 0 496217 29 0 0 25 0 1 0 512636915 272596992 50537 33554432000 4194304 4331988 140734087688624 18446744073709551615 4243048 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6416/statm: 66552 50537 175 34 0 64994 0
[pid=6417] ppid=6412 vsize=357048 CPUtime=19822.3 cores=4-7
/proc/6417/stat : 6417 (plingeling) S 6412 6412 5982 0 -1 4202496 674387 1249 0 0 1982133 98 0 0 18 0 5 0 512636915 365617152 61451 33554432000 4194304 4356188 140734808298720 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6417/statm: 89262 61451 164 40 0 87806 0
[pid=6417/tid=6424] ppid=6412 vsize=357048 CPUtime=4962.3 cores=4-7
/proc/6417/task/6424/stat : 6424 (plingeling) R 6412 6412 5982 0 -1 4202560 211325 1249 0 0 496199 31 0 0 25 0 5 0 512636931 365617152 61451 33554432000 4194304 4356188 140734808298720 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6417/tid=6425] ppid=6412 vsize=357048 CPUtime=4962.29 cores=4-7
/proc/6417/task/6425/stat : 6425 (plingeling) R 6412 6412 5982 0 -1 4202560 189805 1249 0 0 496202 27 0 0 25 0 5 0 512636931 365617152 61451 33554432000 4194304 4356188 140734808298720 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6417/tid=6426] ppid=6412 vsize=357048 CPUtime=4962.3 cores=4-7
/proc/6417/task/6426/stat : 6426 (plingeling) R 6412 6412 5982 0 -1 4202560 197307 1249 0 0 496202 28 0 0 25 0 5 0 512636931 365617152 61451 33554432000 4194304 4356188 140734808298720 18446744073709551615 4332109 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6417/tid=6427] ppid=6412 vsize=357048 CPUtime=4935.23 cores=4-7
/proc/6417/task/6427/stat : 6427 (plingeling) R 6412 6412 5982 0 -1 4202560 67384 1249 0 0 493514 9 0 0 25 0 5 0 512636931 365617152 61451 33554432000 4194304 4356188 140734808298720 18446744073709551615 4211032 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 34709.6
Current children cumulated vsize (KiB) 2682332

[startup+5022.3 s]
/proc/loadavg: 7.11 7.07 7.00 8/179 6573
/proc/meminfo: memFree=14626832/32951124 swapFree=67111528/67111528
[pid=6412] ppid=6410 vsize=18324 CPUtime=0.01 cores=0-7
/proc/6412/stat : 6412 (ppfolio) S 6410 6412 5982 0 -1 4202496 520 1194 0 0 0 0 0 1 15 0 1 0 512636915 18763776 391 33554432000 4194304 4287625 140736785381792 18446744073709551615 244446965263 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/6412/statm: 4581 391 342 23 0 91 0
[pid=6413] ppid=6412 vsize=1705572 CPUtime=5022.31 cores=0
/proc/6413/stat : 6413 (cryptominisat) R 6412 6412 5982 0 -1 4202496 492546 0 0 0 502009 222 0 0 25 0 1 0 512636915 1746505728 69615 33554432000 4194304 4587354 140734877656496 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6413/statm: 426393 69615 336 96 0 422955 0
[pid=6414] ppid=6412 vsize=336036 CPUtime=5022.46 cores=1
/proc/6414/stat : 6414 (clasp) R 6412 6412 5982 0 -1 4202496 79594 0 0 0 502208 38 0 0 25 0 1 0 512636915 344100864 77980 33554432000 4194304 4785740 140736961024224 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6414/statm: 84009 77980 340 145 0 81054 0
[pid=6416] ppid=6412 vsize=266604 CPUtime=5022.46 cores=3
/proc/6416/stat : 6416 (march_hi) R 6412 6412 5982 0 -1 4202496 85026 0 0 0 502217 29 0 0 25 0 1 0 512636915 273002496 51175 33554432000 4194304 4331988 140734087688624 18446744073709551615 4228459 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6416/statm: 66651 51175 175 34 0 65093 0
[pid=6417] ppid=6412 vsize=332472 CPUtime=20062 cores=4-7
/proc/6417/stat : 6417 (plingeling) S 6412 6412 5982 0 -1 4202496 681043 1249 0 0 2006100 99 0 0 18 0 5 0 512636915 340451328 55275 33554432000 4194304 4356188 140734808298720 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6417/statm: 83118 55275 164 40 0 81662 0
[pid=6417/tid=6424] ppid=6412 vsize=332472 CPUtime=5022.3 cores=4-7
/proc/6417/task/6424/stat : 6424 (plingeling) R 6412 6412 5982 0 -1 4202560 212861 1249 0 0 502198 32 0 0 25 0 5 0 512636931 340451328 55275 33554432000 4194304 4356188 140734808298720 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6417/tid=6425] ppid=6412 vsize=332472 CPUtime=5022.29 cores=4-7
/proc/6417/task/6425/stat : 6425 (plingeling) R 6412 6412 5982 0 -1 4202560 189805 1249 0 0 502202 27 0 0 25 0 5 0 512636931 340451328 55275 33554432000 4194304 4356188 140734808298720 18446744073709551615 4332272 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6417/tid=6426] ppid=6412 vsize=332472 CPUtime=5022.3 cores=4-7
/proc/6417/task/6426/stat : 6426 (plingeling) R 6412 6412 5982 0 -1 4202560 201403 1249 0 0 502201 29 0 0 25 0 5 0 512636931 340451328 55275 33554432000 4194304 4356188 140734808298720 18446744073709551615 4330856 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6417/tid=6427] ppid=6412 vsize=332472 CPUtime=4994.91 cores=4-7
/proc/6417/task/6427/stat : 6427 (plingeling) R 6412 6412 5982 0 -1 4202560 68408 1249 0 0 499482 9 0 0 25 0 5 0 512636931 340451328 55275 33554432000 4194304 4356188 140734808298720 18446744073709551615 4331028 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 35129.2
Current children cumulated vsize (KiB) 2659008

[startup+5082.3 s]
/proc/loadavg: 7.10 7.07 7.00 8/179 6574
/proc/meminfo: memFree=14606348/32951124 swapFree=67111528/67111528
[pid=6412] ppid=6410 vsize=18324 CPUtime=0.01 cores=0-7
/proc/6412/stat : 6412 (ppfolio) S 6410 6412 5982 0 -1 4202496 520 1194 0 0 0 0 0 1 15 0 1 0 512636915 18763776 391 33554432000 4194304 4287625 140736785381792 18446744073709551615 244446965263 0 0 4096 65536 9224195742862639363 0 0 17 6 0 0 0
/proc/6412/statm: 4581 391 342 23 0 91 0
[pid=6413] ppid=6412 vsize=1725520 CPUtime=5082.31 cores=0
/proc/6413/stat : 6413 (cryptominisat) R 6412 6412 5982 0 -1 4202496 496232 0 0 0 508007 224 0 0 25 0 1 0 512636915 1766932480 67679 33554432000 4194304 4587354 140734877656496 18446744073709551615 4238525 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6413/statm: 431380 67679 336 96 0 427942 0
[pid=6414] ppid=6412 vsize=339080 CPUtime=5082.46 cores=1
/proc/6414/stat : 6414 (clasp) R 6412 6412 5982 0 -1 4202496 80289 0 0 0 508208 38 0 0 25 0 1 0 512636915 347217920 78675 33554432000 4194304 4785740 140736961024224 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6414/statm: 84770 78675 340 145 0 81815 0
[pid=6416] ppid=6412 vsize=266872 CPUtime=5082.46 cores=3
/proc/6416/stat : 6416 (march_hi) R 6412 6412 5982 0 -1 4202496 85625 0 0 0 508217 29 0 0 25 0 1 0 512636915 273276928 51774 33554432000 4194304 4331988 140734087688624 18446744073709551615 4299275 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6416/statm: 66718 51774 175 34 0 65160 0
[pid=6417] ppid=6412 vsize=357048 CPUtime=20301.7 cores=4-7
/proc/6417/stat : 6417 (plingeling) S 6412 6412 5982 0 -1 4202496 687187 1249 0 0 2030067 100 0 0 18 0 5 0 512636915 365617152 61419 33554432000 4194304 4356188 140734808298720 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6417/statm: 89262 61419 164 40 0 87806 0
[pid=6417/tid=6424] ppid=6412 vsize=357048 CPUtime=5082.31 cores=4-7
/proc/6417/task/6424/stat : 6424 (plingeling) R 6412 6412 5982 0 -1 4202560 213373 1249 0 0 508199 32 0 0 25 0 5 0 512636931 365617152 61419 33554432000 4194304 4356188 140734808298720 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6417/tid=6425] ppid=6412 vsize=357048 CPUtime=5082.29 cores=4-7
/proc/6417/task/6425/stat : 6425 (plingeling) R 6412 6412 5982 0 -1 4202560 191853 1249 0 0 508202 27 0 0 25 0 5 0 512636931 365617152 61419 33554432000 4194304 4356188 140734808298720 18446744073709551615 4330964 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6417/tid=6426] ppid=6412 vsize=357048 CPUtime=5082.3 cores=4-7
/proc/6417/task/6426/stat : 6426 (plingeling) R 6412 6412 5982 0 -1 4202560 203963 1249 0 0 508201 29 0 0 25 0 5 0 512636931 365617152 61419 33554432000 4194304 4356188 140734808298720 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6417/tid=6427] ppid=6412 vsize=357048 CPUtime=5054.58 cores=4-7
/proc/6417/task/6427/stat : 6427 (plingeling) R 6412 6412 5982 0 -1 4202560 69432 1249 0 0 505449 9 0 0 25 0 5 0 512636931 365617152 61419 33554432000 4194304 4356188 140734808298720 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 35548.9
Current children cumulated vsize (KiB) 2706844



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 7.08 7.06 7.00 8/179 6575
/proc/meminfo: memFree=14597292/32951124 swapFree=67111528/67111528
[pid=6412] ppid=6410 vsize=18324 CPUtime=0.01 cores=0-7
/proc/6412/stat : 6412 (ppfolio) S 6410 6412 5982 0 -1 4202496 520 1194 0 0 0 0 0 1 15 0 1 0 512636915 18763776 391 33554432000 4194304 4287625 140736785381792 18446744073709551615 244446965263 0 0 4096 65536 9224195742862639363 0 0 17 6 0 0 0
/proc/6412/statm: 4581 391 342 23 0 91 0
[pid=6413] ppid=6412 vsize=1725520 CPUtime=5100.02 cores=0
/proc/6413/stat : 6413 (cryptominisat) R 6412 6412 5982 0 -1 4202496 497675 0 0 0 509777 225 0 0 25 0 1 0 512636915 1766932480 69102 33554432000 4194304 4587354 140734877656496 18446744073709551615 4238418 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6413/statm: 431380 69102 336 96 0 427942 0
[pid=6414] ppid=6412 vsize=339080 CPUtime=5100.17 cores=1
/proc/6414/stat : 6414 (clasp) R 6412 6412 5982 0 -1 4202496 80291 0 0 0 509979 38 0 0 25 0 1 0 512636915 347217920 78677 33554432000 4194304 4785740 140736961024224 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6414/statm: 84770 78677 340 145 0 81815 0
[pid=6416] ppid=6412 vsize=266872 CPUtime=5100.17 cores=3
/proc/6416/stat : 6416 (march_hi) R 6412 6412 5982 0 -1 4202496 85728 0 0 0 509988 29 0 0 25 0 1 0 512636915 273276928 51877 33554432000 4194304 4331988 140734087688624 18446744073709551615 4237136 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6416/statm: 66718 51877 175 34 0 65160 0
[pid=6417] ppid=6412 vsize=359096 CPUtime=20372.4 cores=4-7
/proc/6417/stat : 6417 (plingeling) S 6412 6412 5982 0 -1 4202496 690292 1249 0 0 2037139 100 0 0 18 0 5 0 512636915 367714304 61931 33554432000 4194304 4356188 140734808298720 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6417/statm: 89774 61931 164 40 0 88318 0
[pid=6417/tid=6424] ppid=6412 vsize=359096 CPUtime=5100.01 cores=4-7
/proc/6417/task/6424/stat : 6424 (plingeling) R 6412 6412 5982 0 -1 4202560 214397 1249 0 0 509969 32 0 0 25 0 5 0 512636931 367714304 61931 33554432000 4194304 4356188 140734808298720 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6417/tid=6425] ppid=6412 vsize=359096 CPUtime=5100 cores=4-7
/proc/6417/task/6425/stat : 6425 (plingeling) R 6412 6412 5982 0 -1 4202560 191853 1249 0 0 509973 27 0 0 25 0 5 0 512636931 367714304 61931 33554432000 4194304 4356188 140734808298720 18446744073709551615 4331106 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6417/tid=6426] ppid=6412 vsize=359096 CPUtime=5100.01 cores=4-7
/proc/6417/task/6426/stat : 6426 (plingeling) R 6412 6412 5982 0 -1 4202560 204508 1249 0 0 509972 29 0 0 25 0 5 0 512636931 367714304 61931 33554432000 4194304 4356188 140734808298720 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6417/tid=6427] ppid=6412 vsize=359096 CPUtime=5072.18 cores=4-7
/proc/6417/task/6427/stat : 6427 (plingeling) R 6412 6412 5982 0 -1 4202560 70968 1249 0 0 507209 9 0 0 25 0 5 0 512636931 367714304 61931 33554432000 4194304 4356188 140734808298720 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 35672.8
Current children cumulated vsize (KiB) 2708892

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 6412 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=5999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=20996
# CPU time returned by wait4() is 0.026995
# while last known CPU time is 35672.8
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5100.02
CPU time (s): 35672.8
CPU user time (s): 35668.8
CPU system time (s): 3.93
CPU usage (%): 699.463
Max. virtual memory (cumulated for all children) (KiB): 16575336

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.005999
system time used= 0.020996
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1714
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 531
involuntary context switches= 10

runsolver used 8.76967 second user time and 19.0571 second system time

The end

Launcher Data 

Begin job on node105 at 2011-04-28 19:06:05
IDJOB=3310320
IDBENCH=83127
IDSOLVER=1638
FILE ID=node105/3310320-1304010365
RUNJOBID= node105-1304003715-5996
PBS_JOBID= 13168806
Free space on /tmp= 71936 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-bottom18.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3310320-1304010365/watcher-3310320-1304010365 -o /tmp/evaluation-result-3310320-1304010365/solver-3310320-1304010365 -C 40000 -W 5100 -M 31000  ppfolio HOME/instance-3310320-1304010365.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 9ece8f569c93ceee834817ef3350ed8e
RANDOM SEED=130301729

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      15644372 kB
Buffers:       2295852 kB
Cached:       14374008 kB
SwapCached:          0 kB
Active:        8858244 kB
Inactive:      7877644 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      15644372 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           12108 kB
Writeback:           8 kB
AnonPages:       65448 kB
Mapped:          14588 kB
Slab:           506424 kB
PageTables:       4620 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   186620 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71924 MiB
End job on node105 at 2011-04-28 20:31:07