Trace number 3310316

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 35629.1 5100.02

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-top25.cnf
MD5SUM1d36146d5d79c91e0605a19242395228
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark1033.04
Satisfiable
(Un)Satisfiability was proved
Number of variables71356
Number of clauses227126
Sum of the clauses size555264
Maximum clause length2747
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 2137909
Number of clauses of size 387742
Number of clauses of size 4324
Number of clauses of size 523
Number of clauses of size over 51127

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3310316-1304009735/watcher-3310316-1304009735 -o /tmp/evaluation-result-3310316-1304009735/solver-3310316-1304009735 -C 40000 -W 5100 -M 31000 ppfolio HOME/instance-3310316-1304009735.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.01 8.04 7.97 1/170 25925
/proc/meminfo: memFree=13446172/32951124 swapFree=67111528/67111528
[pid=25925] ppid=25923 vsize=18324 CPUtime=0 cores=0-7
/proc/25925/stat : 25925 (ppfolio) S 25923 25925 25435 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 512574266 18763776 390 33554432000 4194304 4287625 140737173188656 18446744073709551615 222921797135 0 0 4096 65536 4575693234709200895 0 0 17 6 0 0 0
/proc/25925/statm: 4581 390 341 23 0 91 0

[startup+0.0143839 s]
/proc/loadavg: 8.01 8.04 7.97 1/170 25925
/proc/meminfo: memFree=13446172/32951124 swapFree=67111528/67111528
[pid=25925] ppid=25923 vsize=18324 CPUtime=0 cores=0-7
/proc/25925/stat : 25925 (ppfolio) S 25923 25925 25435 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 512574266 18763776 390 33554432000 4194304 4287625 140737173188656 18446744073709551615 222921797135 0 0 4096 65536 4575693234709200895 0 0 17 6 0 0 0
/proc/25925/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100366 s]
/proc/loadavg: 8.01 8.04 7.97 1/170 25925
/proc/meminfo: memFree=13446172/32951124 swapFree=67111528/67111528
[pid=25925] ppid=25923 vsize=18324 CPUtime=0.01 cores=0-7
/proc/25925/stat : 25925 (ppfolio) S 25923 25925 25435 0 -1 4202496 523 1193 0 0 0 0 0 1 25 0 1 0 512574266 18763776 391 33554432000 4194304 4287625 140737173188656 18446744073709551615 222921797135 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/25925/statm: 4581 391 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+0.300331 s]
/proc/loadavg: 8.01 8.04 7.97 1/170 25925
/proc/meminfo: memFree=13446172/32951124 swapFree=67111528/67111528
[pid=25925] ppid=25923 vsize=18324 CPUtime=0.01 cores=0-7
/proc/25925/stat : 25925 (ppfolio) S 25923 25925 25435 0 -1 4202496 523 1193 0 0 0 0 0 1 25 0 1 0 512574266 18763776 391 33554432000 4194304 4287625 140737173188656 18446744073709551615 222921797135 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/25925/statm: 4581 391 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+0.700225 s]
/proc/loadavg: 8.01 8.04 7.97 1/170 25925
/proc/meminfo: memFree=13446172/32951124 swapFree=67111528/67111528
[pid=25925] ppid=25923 vsize=18324 CPUtime=0.01 cores=0-7
/proc/25925/stat : 25925 (ppfolio) S 25923 25925 25435 0 -1 4202496 523 1193 0 0 0 0 0 1 25 0 1 0 512574266 18763776 391 33554432000 4194304 4287625 140737173188656 18446744073709551615 222921797135 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/25925/statm: 4581 391 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+1.50111 s]
/proc/loadavg: 8.01 8.04 7.97 8/179 25940
/proc/meminfo: memFree=13286636/32951124 swapFree=67111528/67111528
[pid=25925] ppid=25923 vsize=18324 CPUtime=0.01 cores=0-7
/proc/25925/stat : 25925 (ppfolio) S 25923 25925 25435 0 -1 4202496 523 1193 0 0 0 0 0 1 17 0 1 0 512574266 18763776 391 33554432000 4194304 4287625 140737173188656 18446744073709551615 222921797135 0 0 4096 65536 9224195742862639362 0 0 17 0 0 0 0
/proc/25925/statm: 4581 391 342 23 0 91 0
[pid=25926] ppid=25925 vsize=63004 CPUtime=1.48 cores=0
/proc/25926/stat : 25926 (cryptominisat) R 25925 25925 25435 0 -1 4202496 10196 0 0 0 147 1 0 0 25 0 1 0 512574267 64516096 9915 33554432000 4194304 4587354 140737180006336 18446744073709551615 4367348 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/25926/statm: 15751 9915 327 96 0 12313 0
[pid=25927] ppid=25925 vsize=50900 CPUtime=1.49 cores=1
/proc/25927/stat : 25927 (clasp) R 25925 25925 25435 0 -1 4202496 12853 0 0 0 147 2 0 0 20 0 1 0 512574267 52121600 9732 33554432000 4194304 4785740 140736755224832 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/25927/statm: 12725 9732 340 145 0 9770 0
[pid=25929] ppid=25925 vsize=52768 CPUtime=1.48 cores=3
/proc/25929/stat : 25929 (march_hi) R 25925 25925 25435 0 -1 4202496 66609 0 0 0 139 9 0 0 20 0 1 0 512574267 54034432 9568 33554432000 4194304 4331988 140734158658672 18446744073709551615 4253303 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/25929/statm: 13192 9568 152 34 0 11634 0
[pid=25930] ppid=25925 vsize=111488 CPUtime=3.87 cores=4-7
/proc/25930/stat : 25930 (plingeling) S 25925 25925 25435 0 -1 4202496 28028 1251 0 0 384 3 0 0 25 0 5 0 512574267 114163712 16097 33554432000 4194304 4356188 140734685529840 18446744073709551615 222933579749 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25930/statm: 27872 16097 155 40 0 26416 0
[pid=25930/tid=25937] ppid=25925 vsize=111488 CPUtime=1.2 cores=4-7
/proc/25930/task/25937/stat : 25937 (plingeling) R 25925 25925 25435 0 -1 4202560 4505 1251 0 0 120 0 0 0 25 0 5 0 512574294 114163712 16097 33554432000 4194304 4356188 140734685529840 18446744073709551615 4322472 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25930/tid=25938] ppid=25925 vsize=111488 CPUtime=1.17 cores=4-7
/proc/25930/task/25938/stat : 25938 (plingeling) R 25925 25925 25435 0 -1 4202560 4399 1251 0 0 117 0 0 0 25 0 5 0 512574294 114163712 16097 33554432000 4194304 4356188 140734685529840 18446744073709551615 4322000 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25930/tid=25939] ppid=25925 vsize=111488 CPUtime=0.61 cores=4-7
/proc/25930/task/25939/stat : 25939 (plingeling) R 25925 25925 25435 0 -1 4202560 3807 1251 0 0 61 0 0 0 25 0 5 0 512574294 114163712 16097 33554432000 4194304 4356188 140734685529840 18446744073709551615 4219653 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25930/tid=25940] ppid=25925 vsize=111488 CPUtime=0.59 cores=4-7
/proc/25930/task/25940/stat : 25940 (plingeling) R 25925 25925 25435 0 -1 4202560 2680 1251 0 0 59 0 0 0 25 0 5 0 512574294 114163712 16097 33554432000 4194304 4356188 140734685529840 18446744073709551615 222921471069 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 8.33
Current children cumulated vsize (KiB) 296484

[startup+3.10082 s]
/proc/loadavg: 8.01 8.04 7.97 8/179 25940
/proc/meminfo: memFree=13230092/32951124 swapFree=67111528/67111528
[pid=25925] ppid=25923 vsize=18324 CPUtime=0.01 cores=0-7
/proc/25925/stat : 25925 (ppfolio) S 25923 25925 25435 0 -1 4202496 523 1193 0 0 0 0 0 1 15 0 1 0 512574266 18763776 391 33554432000 4194304 4287625 140737173188656 18446744073709551615 222921797135 0 0 4096 65536 9224195742862639362 0 0 17 0 0 0 0
/proc/25925/statm: 4581 391 342 23 0 91 0
[pid=25926] ppid=25925 vsize=85272 CPUtime=3.09 cores=0
/proc/25926/stat : 25926 (cryptominisat) R 25925 25925 25435 0 -1 4202496 20385 0 0 0 306 3 0 0 25 0 1 0 512574267 87318528 16207 33554432000 4194304 4587354 140737180006336 18446744073709551615 4231968 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/25926/statm: 21318 16207 324 96 0 17880 0
[pid=25927] ppid=25925 vsize=50900 CPUtime=3.09 cores=1
/proc/25927/stat : 25927 (clasp) R 25925 25925 25435 0 -1 4202496 12858 0 0 0 307 2 0 0 25 0 1 0 512574267 52121600 9737 33554432000 4194304 4785740 140736755224832 18446744073709551615 4287017 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/25927/statm: 12725 9737 340 145 0 9770 0
[pid=25929] ppid=25925 vsize=160076 CPUtime=3.09 cores=3
/proc/25929/stat : 25929 (march_hi) R 25925 25925 25435 0 -1 4202496 87050 0 0 0 297 12 0 0 25 0 1 0 512574267 163917824 20584 33554432000 4194304 4331988 140734158658672 18446744073709551615 4228605 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/25929/statm: 40019 20584 177 34 0 38461 0
[pid=25930] ppid=25925 vsize=119772 CPUtime=8.67 cores=4-7
/proc/25930/stat : 25930 (plingeling) S 25925 25925 25435 0 -1 4202496 32390 1251 0 0 863 4 0 0 25 0 5 0 512574267 122646528 18386 33554432000 4194304 4356188 140734685529840 18446744073709551615 222933579749 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25930/statm: 29943 18386 163 40 0 28487 0
[pid=25930/tid=25937] ppid=25925 vsize=119772 CPUtime=2.8 cores=4-7
/proc/25930/task/25937/stat : 25937 (plingeling) R 25925 25925 25435 0 -1 4202560 6027 1251 0 0 280 0 0 0 25 0 5 0 512574294 122646528 18386 33554432000 4194304 4356188 140734685529840 18446744073709551615 4231564 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25930/tid=25938] ppid=25925 vsize=119772 CPUtime=2.77 cores=4-7
/proc/25930/task/25938/stat : 25938 (plingeling) R 25925 25925 25435 0 -1 4202560 5942 1251 0 0 277 0 0 0 25 0 5 0 512574294 122646528 18386 33554432000 4194304 4356188 140734685529840 18446744073709551615 4231344 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25930/tid=25939] ppid=25925 vsize=119772 CPUtime=1.41 cores=4-7
/proc/25930/task/25939/stat : 25939 (plingeling) R 25925 25925 25435 0 -1 4202560 4211 1251 0 0 141 0 0 0 25 0 5 0 512574294 122646528 18386 33554432000 4194304 4356188 140734685529840 18446744073709551615 4292929 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25930/tid=25940] ppid=25925 vsize=119772 CPUtime=1.39 cores=4-7
/proc/25930/task/25940/stat : 25940 (plingeling) R 25925 25925 25435 0 -1 4202560 3573 1251 0 0 139 0 0 0 25 0 5 0 512574294 122646528 18386 33554432000 4194304 4356188 140734685529840 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 17.95
Current children cumulated vsize (KiB) 434344

[startup+6.30123 s]
/proc/loadavg: 7.93 8.02 7.97 8/179 25940
/proc/meminfo: memFree=13164128/32951124 swapFree=67111528/67111528
[pid=25925] ppid=25923 vsize=18324 CPUtime=0.01 cores=0-7
/proc/25925/stat : 25925 (ppfolio) S 25923 25925 25435 0 -1 4202496 523 1193 0 0 0 0 0 1 15 0 1 0 512574266 18763776 391 33554432000 4194304 4287625 140737173188656 18446744073709551615 222921797135 0 0 4096 65536 9224195742862639362 0 0 17 0 0 0 0
/proc/25925/statm: 4581 391 342 23 0 91 0
[pid=25926] ppid=25925 vsize=126812 CPUtime=6.29 cores=0
/proc/25926/stat : 25926 (cryptominisat) R 25925 25925 25435 0 -1 4202496 21940 0 0 0 626 3 0 0 25 0 1 0 512574267 129855488 17727 33554432000 4194304 4587354 140737180006336 18446744073709551615 4237818 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/25926/statm: 31703 17727 330 96 0 28265 0
[pid=25927] ppid=25925 vsize=53984 CPUtime=6.28 cores=1
/proc/25927/stat : 25927 (clasp) R 25925 25925 25435 0 -1 4202496 13737 0 0 0 626 2 0 0 25 0 1 0 512574267 55279616 10616 33554432000 4194304 4785740 140736755224832 18446744073709551615 4550862 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/25927/statm: 13496 10616 340 145 0 10541 0
[pid=25929] ppid=25925 vsize=168576 CPUtime=6.28 cores=3
/proc/25929/stat : 25929 (march_hi) R 25925 25925 25435 0 -1 4202496 88913 0 0 0 616 12 0 0 25 0 1 0 512574267 172621824 22445 33554432000 4194304 4331988 140734158658672 18446744073709551615 4263331 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/25929/statm: 42144 22445 175 34 0 40586 0
[pid=25930] ppid=25925 vsize=127060 CPUtime=18.27 cores=4-7
/proc/25930/stat : 25930 (plingeling) S 25925 25925 25435 0 -1 4202496 34212 1251 0 0 1823 4 0 0 25 0 5 0 512574267 130109440 20208 33554432000 4194304 4356188 140734685529840 18446744073709551615 222933579749 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25930/statm: 31765 20208 163 40 0 30309 0
[pid=25930/tid=25937] ppid=25925 vsize=127060 CPUtime=6 cores=4-7
/proc/25930/task/25937/stat : 25937 (plingeling) R 25925 25925 25435 0 -1 4202560 6027 1251 0 0 600 0 0 0 25 0 5 0 512574294 130109440 20208 33554432000 4194304 4356188 140734685529840 18446744073709551615 4210587 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25930/tid=25938] ppid=25925 vsize=127060 CPUtime=5.97 cores=4-7
/proc/25930/task/25938/stat : 25938 (plingeling) R 25925 25925 25435 0 -1 4202560 6739 1251 0 0 597 0 0 0 25 0 5 0 512574294 130109440 20208 33554432000 4194304 4356188 140734685529840 18446744073709551615 4233426 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25930/tid=25939] ppid=25925 vsize=127060 CPUtime=3.01 cores=4-7
/proc/25930/task/25939/stat : 25939 (plingeling) R 25925 25925 25435 0 -1 4202560 5236 1251 0 0 301 0 0 0 25 0 5 0 512574294 130109440 20208 33554432000 4194304 4356188 140734685529840 18446744073709551615 4217730 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25930/tid=25940] ppid=25925 vsize=127060 CPUtime=2.99 cores=4-7
/proc/25930/task/25940/stat : 25940 (plingeling) R 25925 25925 25435 0 -1 4202560 3573 1251 0 0 299 0 0 0 25 0 5 0 512574294 130109440 20208 33554432000 4194304 4356188 140734685529840 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37.13
Current children cumulated vsize (KiB) 494756

[startup+12.7011 s]
/proc/loadavg: 7.85 8.00 7.96 8/179 25940
/proc/meminfo: memFree=13151620/32951124 swapFree=67111528/67111528
[pid=25925] ppid=25923 vsize=18324 CPUtime=0.01 cores=0-7
/proc/25925/stat : 25925 (ppfolio) S 25923 25925 25435 0 -1 4202496 523 1193 0 0 0 0 0 1 15 0 1 0 512574266 18763776 391 33554432000 4194304 4287625 140737173188656 18446744073709551615 222921797135 0 0 4096 65536 9224195742862639362 0 0 17 0 0 0 0
/proc/25925/statm: 4581 391 342 23 0 91 0
[pid=25926] ppid=25925 vsize=126812 CPUtime=12.68 cores=0
/proc/25926/stat : 25926 (cryptominisat) R 25925 25925 25435 0 -1 4202496 22567 0 0 0 1265 3 0 0 25 0 1 0 512574267 129855488 18348 33554432000 4194304 4587354 140737180006336 18446744073709551615 4422548 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/25926/statm: 31703 18348 336 96 0 28265 0
[pid=25927] ppid=25925 vsize=62716 CPUtime=12.69 cores=1
/proc/25927/stat : 25927 (clasp) R 25925 25925 25435 0 -1 4202496 15854 0 0 0 1266 3 0 0 25 0 1 0 512574267 64221184 12733 33554432000 4194304 4785740 140736755224832 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/25927/statm: 15679 12733 340 145 0 12724 0
[pid=25929] ppid=25925 vsize=169704 CPUtime=12.69 cores=3
/proc/25929/stat : 25929 (march_hi) R 25925 25925 25435 0 -1 4202496 89287 0 0 0 1256 13 0 0 25 0 1 0 512574267 173776896 22819 33554432000 4194304 4331988 140734158658672 18446744073709551615 222921172015 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/25929/statm: 42426 22819 175 34 0 40868 0
[pid=25930] ppid=25925 vsize=121752 CPUtime=37.47 cores=4-7
/proc/25930/stat : 25930 (plingeling) S 25925 25925 25435 0 -1 4202496 35020 1251 0 0 3743 4 0 0 25 0 5 0 512574267 124674048 18881 33554432000 4194304 4356188 140734685529840 18446744073709551615 222933579749 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25930/statm: 30438 18881 163 40 0 28982 0
[pid=25930/tid=25937] ppid=25925 vsize=121752 CPUtime=12.4 cores=4-7
/proc/25930/task/25937/stat : 25937 (plingeling) R 25925 25925 25435 0 -1 4202560 6027 1251 0 0 1240 0 0 0 25 0 5 0 512574294 124674048 18881 33554432000 4194304 4356188 140734685529840 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25930/tid=25938] ppid=25925 vsize=121752 CPUtime=12.36 cores=4-7
/proc/25930/task/25938/stat : 25938 (plingeling) R 25925 25925 25435 0 -1 4202560 6739 1251 0 0 1236 0 0 0 25 0 5 0 512574294 124674048 18881 33554432000 4194304 4356188 140734685529840 18446744073709551615 4237483 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25930/tid=25939] ppid=25925 vsize=121752 CPUtime=6.21 cores=4-7
/proc/25930/task/25939/stat : 25939 (plingeling) R 25925 25925 25435 0 -1 4202560 5236 1251 0 0 621 0 0 0 25 0 5 0 512574294 124674048 18881 33554432000 4194304 4356188 140734685529840 18446744073709551615 4210557 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25930/tid=25940] ppid=25925 vsize=121752 CPUtime=6.19 cores=4-7
/proc/25930/task/25940/stat : 25940 (plingeling) R 25925 25925 25435 0 -1 4202560 4381 1251 0 0 619 0 0 0 25 0 5 0 512574294 124674048 18881 33554432000 4194304 4356188 140734685529840 18446744073709551615 4236992 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 75.54
Current children cumulated vsize (KiB) 499308

[startup+25.5007 s]
/proc/loadavg: 7.66 7.95 7.95 8/179 25941
/proc/meminfo: memFree=13146352/32951124 swapFree=67111528/67111528
[pid=25925] ppid=25923 vsize=18324 CPUtime=0.01 cores=0-7
/proc/25925/stat : 25925 (ppfolio) S 25923 25925 25435 0 -1 4202496 523 1193 0 0 0 0 0 1 15 0 1 0 512574266 18763776 391 33554432000 4194304 4287625 140737173188656 18446744073709551615 222921797135 0 0 4096 65536 9224195742862639362 0 0 17 0 0 0 0
/proc/25925/statm: 4581 391 342 23 0 91 0
[pid=25926] ppid=25925 vsize=2064204 CPUtime=25.48 cores=0
/proc/25926/stat : 25926 (cryptominisat) R 25925 25925 25435 0 -1 4202496 32951 0 0 0 2543 5 0 0 25 0 1 0 512574267 2113744896 19387 33554432000 4194304 4587354 140737180006336 18446744073709551615 4237997 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/25926/statm: 516051 19387 338 96 0 512613 0
[pid=25927] ppid=25925 vsize=70840 CPUtime=25.49 cores=1
/proc/25927/stat : 25927 (clasp) R 25925 25925 25435 0 -1 4202496 17844 0 0 0 2545 4 0 0 25 0 1 0 512574267 72540160 14723 33554432000 4194304 4785740 140736755224832 18446744073709551615 4552646 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/25927/statm: 17710 14723 340 145 0 14755 0
[pid=25929] ppid=25925 vsize=170392 CPUtime=25.49 cores=3
/proc/25929/stat : 25929 (march_hi) R 25925 25925 25435 0 -1 4202496 89480 0 0 0 2536 13 0 0 25 0 1 0 512574267 174481408 23012 33554432000 4194304 4331988 140734158658672 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/25929/statm: 42598 23012 175 34 0 41040 0
[pid=25930] ppid=25925 vsize=117392 CPUtime=75.87 cores=4-7
/proc/25930/stat : 25930 (plingeling) S 25925 25925 25435 0 -1 4202496 35020 1251 0 0 7583 4 0 0 25 0 5 0 512574267 120209408 17791 33554432000 4194304 4356188 140734685529840 18446744073709551615 222933579749 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25930/statm: 29348 17791 163 40 0 27892 0
[pid=25930/tid=25937] ppid=25925 vsize=117392 CPUtime=25.2 cores=4-7
/proc/25930/task/25937/stat : 25937 (plingeling) R 25925 25925 25435 0 -1 4202560 6027 1251 0 0 2520 0 0 0 25 0 5 0 512574294 120209408 17791 33554432000 4194304 4356188 140734685529840 18446744073709551615 4322024 0 0 4096 17442 0 0 0 -1 4 0 0 0

################
# More data... #
################

Current children cumulated CPU time (s) 33826.5
Current children cumulated vsize (KiB) 3120716

[startup+4902.3 s]
/proc/loadavg: 7.04 7.03 7.00 8/179 26095
/proc/meminfo: memFree=12440216/32951124 swapFree=67111528/67111528
[pid=25925] ppid=25923 vsize=18324 CPUtime=0.01 cores=0-7
/proc/25925/stat : 25925 (ppfolio) S 25923 25925 25435 0 -1 4202496 523 1193 0 0 0 0 0 1 15 0 1 0 512574266 18763776 391 33554432000 4194304 4287625 140737173188656 18446744073709551615 222921797135 0 0 4096 65536 9224195742862639362 0 0 17 0 0 0 0
/proc/25925/statm: 4581 391 342 23 0 91 0
[pid=25926] ppid=25925 vsize=2225736 CPUtime=4902.44 cores=0
/proc/25926/stat : 25926 (cryptominisat) R 25925 25925 25435 0 -1 4202496 435017 0 0 0 490079 165 0 0 25 0 1 0 512574267 2279153664 48898 33554432000 4194304 4587354 140737180006336 18446744073709551615 4237661 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/25926/statm: 556434 48898 336 96 0 552996 0
[pid=25927] ppid=25925 vsize=413172 CPUtime=4902.41 cores=1
/proc/25927/stat : 25927 (clasp) R 25925 25925 25435 0 -1 4202496 100003 0 0 0 490197 44 0 0 25 0 1 0 512574267 423088128 96882 33554432000 4194304 4785740 140736755224832 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/25927/statm: 103293 96882 340 145 0 100338 0
[pid=25929] ppid=25925 vsize=193520 CPUtime=4902.44 cores=3
/proc/25929/stat : 25929 (march_hi) R 25925 25925 25435 0 -1 4202496 93968 0 0 0 490228 16 0 0 25 0 1 0 512574267 198164480 27500 33554432000 4194304 4331988 140734158658672 18446744073709551615 4265452 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/25929/statm: 48380 27500 175 34 0 46822 0
[pid=25930] ppid=25925 vsize=360668 CPUtime=19538.9 cores=4-7
/proc/25930/stat : 25930 (plingeling) S 25925 25925 25435 0 -1 4202496 780553 1251 0 0 1953779 108 0 0 25 0 5 0 512574267 369324032 78610 33554432000 4194304 4356188 140734685529840 18446744073709551615 222933579749 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25930/statm: 90167 78610 163 40 0 88711 0
[pid=25930/tid=25937] ppid=25925 vsize=360668 CPUtime=4902.15 cores=4-7
/proc/25930/task/25937/stat : 25937 (plingeling) R 25925 25925 25435 0 -1 4202560 253135 1251 0 0 490180 35 0 0 25 0 5 0 512574294 369324032 78610 33554432000 4194304 4356188 140734685529840 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25930/tid=25938] ppid=25925 vsize=360668 CPUtime=4901.97 cores=4-7
/proc/25930/task/25938/stat : 25938 (plingeling) R 25925 25925 25435 0 -1 4202560 209303 1251 0 0 490168 29 0 0 25 0 5 0 512574294 369324032 78610 33554432000 4194304 4356188 140734685529840 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25930/tid=25939] ppid=25925 vsize=360668 CPUtime=4853.98 cores=4-7
/proc/25930/task/25939/stat : 25939 (plingeling) R 25925 25925 25435 0 -1 4202560 204544 1251 0 0 485371 27 0 0 25 0 5 0 512574294 369324032 78610 33554432000 4194304 4356188 140734685529840 18446744073709551615 4280666 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25930/tid=25940] ppid=25925 vsize=360668 CPUtime=4880.46 cores=4-7
/proc/25930/task/25940/stat : 25940 (plingeling) R 25925 25925 25435 0 -1 4202560 100934 1251 0 0 488033 13 0 0 25 0 5 0 512574294 369324032 78610 33554432000 4194304 4356188 140734685529840 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 34246.2
Current children cumulated vsize (KiB) 3211420

[startup+4962.3 s]
/proc/loadavg: 7.05 7.03 7.00 8/179 26096
/proc/meminfo: memFree=12463876/32951124 swapFree=67111528/67111528
[pid=25925] ppid=25923 vsize=18324 CPUtime=0.01 cores=0-7
/proc/25925/stat : 25925 (ppfolio) S 25923 25925 25435 0 -1 4202496 523 1193 0 0 0 0 0 1 15 0 1 0 512574266 18763776 391 33554432000 4194304 4287625 140737173188656 18446744073709551615 222921797135 0 0 4096 65536 9224195742862639362 0 0 17 0 0 0 0
/proc/25925/statm: 4581 391 342 23 0 91 0
[pid=25926] ppid=25925 vsize=2166300 CPUtime=4962.44 cores=0
/proc/25926/stat : 25926 (cryptominisat) R 25925 25925 25435 0 -1 4202496 438683 0 0 0 496076 168 0 0 25 0 1 0 512574267 2218291200 48898 33554432000 4194304 4587354 140737180006336 18446744073709551615 4238421 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/25926/statm: 541575 48898 336 96 0 538137 0
[pid=25927] ppid=25925 vsize=415196 CPUtime=4962.41 cores=1
/proc/25927/stat : 25927 (clasp) R 25925 25925 25435 0 -1 4202496 100445 0 0 0 496197 44 0 0 25 0 1 0 512574267 425160704 97324 33554432000 4194304 4785740 140736755224832 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/25927/statm: 103799 97324 340 145 0 100844 0
[pid=25929] ppid=25925 vsize=193520 CPUtime=4962.44 cores=3
/proc/25929/stat : 25929 (march_hi) R 25925 25925 25435 0 -1 4202496 94016 0 0 0 496228 16 0 0 25 0 1 0 512574267 198164480 27548 33554432000 4194304 4331988 140734158658672 18446744073709551615 4299872 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/25929/statm: 48380 27548 175 34 0 46822 0
[pid=25930] ppid=25925 vsize=333020 CPUtime=19778.5 cores=4-7
/proc/25930/stat : 25930 (plingeling) S 25925 25925 25435 0 -1 4202496 783881 1251 0 0 1977746 108 0 0 25 0 5 0 512574267 341012480 71698 33554432000 4194304 4356188 140734685529840 18446744073709551615 222933579749 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25930/statm: 83255 71698 163 40 0 81799 0
[pid=25930/tid=25937] ppid=25925 vsize=333020 CPUtime=4962.16 cores=4-7
/proc/25930/task/25937/stat : 25937 (plingeling) R 25925 25925 25435 0 -1 4202560 253903 1251 0 0 496180 36 0 0 25 0 5 0 512574294 341012480 71698 33554432000 4194304 4356188 140734685529840 18446744073709551615 4331305 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25930/tid=25938] ppid=25925 vsize=333020 CPUtime=4961.97 cores=4-7
/proc/25930/task/25938/stat : 25938 (plingeling) R 25925 25925 25435 0 -1 4202560 210327 1251 0 0 496168 29 0 0 25 0 5 0 512574294 341012480 71698 33554432000 4194304 4356188 140734685529840 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25930/tid=25939] ppid=25925 vsize=333020 CPUtime=4913.66 cores=4-7
/proc/25930/task/25939/stat : 25939 (plingeling) R 25925 25925 25435 0 -1 4202560 205568 1251 0 0 491339 27 0 0 25 0 5 0 512574294 341012480 71698 33554432000 4194304 4356188 140734685529840 18446744073709551615 4331326 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25930/tid=25940] ppid=25925 vsize=333020 CPUtime=4940.46 cores=4-7
/proc/25930/task/25940/stat : 25940 (plingeling) R 25925 25925 25435 0 -1 4202560 101446 1251 0 0 494033 13 0 0 25 0 5 0 512574294 341012480 71698 33554432000 4194304 4356188 140734685529840 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 34665.8
Current children cumulated vsize (KiB) 3126360

[startup+5022.3 s]
/proc/loadavg: 7.13 7.06 7.01 8/179 26098
/proc/meminfo: memFree=12469692/32951124 swapFree=67111528/67111528
[pid=25925] ppid=25923 vsize=18324 CPUtime=0.01 cores=0-7
/proc/25925/stat : 25925 (ppfolio) S 25923 25925 25435 0 -1 4202496 523 1193 0 0 0 0 0 1 15 0 1 0 512574266 18763776 391 33554432000 4194304 4287625 140737173188656 18446744073709551615 222921797135 0 0 4096 65536 9224195742862639362 0 0 17 0 0 0 0
/proc/25925/statm: 4581 391 342 23 0 91 0
[pid=25926] ppid=25925 vsize=2166300 CPUtime=5022.44 cores=0
/proc/25926/stat : 25926 (cryptominisat) R 25925 25925 25435 0 -1 4202496 440637 0 0 0 502075 169 0 0 25 0 1 0 512574267 2218291200 50820 33554432000 4194304 4587354 140737180006336 18446744073709551615 4238421 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/25926/statm: 541575 50820 336 96 0 538137 0
[pid=25927] ppid=25925 vsize=419420 CPUtime=5022.42 cores=1
/proc/25927/stat : 25927 (clasp) R 25925 25925 25435 0 -1 4202496 101517 0 0 0 502197 45 0 0 25 0 1 0 512574267 429486080 98396 33554432000 4194304 4785740 140736755224832 18446744073709551615 4553064 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/25927/statm: 104855 98396 340 145 0 101900 0
[pid=25929] ppid=25925 vsize=193520 CPUtime=5022.44 cores=3
/proc/25929/stat : 25929 (march_hi) R 25925 25925 25435 0 -1 4202496 94059 0 0 0 502228 16 0 0 25 0 1 0 512574267 198164480 27591 33554432000 4194304 4331988 140734158658672 18446744073709551615 4242907 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/25929/statm: 48380 27591 175 34 0 46822 0
[pid=25930] ppid=25925 vsize=315612 CPUtime=20018.2 cores=4-7
/proc/25930/stat : 25930 (plingeling) S 25925 25925 25435 0 -1 4202496 794377 1251 0 0 2001712 111 0 0 25 0 5 0 512574267 323186688 67346 33554432000 4194304 4356188 140734685529840 18446744073709551615 222933579749 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25930/statm: 78903 67346 163 40 0 77447 0
[pid=25930/tid=25937] ppid=25925 vsize=315612 CPUtime=5022.15 cores=4-7
/proc/25930/task/25937/stat : 25937 (plingeling) R 25925 25925 25435 0 -1 4202560 260559 1251 0 0 502179 36 0 0 25 0 5 0 512574294 323186688 67346 33554432000 4194304 4356188 140734685529840 18446744073709551615 4332225 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25930/tid=25938] ppid=25925 vsize=315612 CPUtime=5021.97 cores=4-7
/proc/25930/task/25938/stat : 25938 (plingeling) R 25925 25925 25435 0 -1 4202560 212375 1251 0 0 502167 30 0 0 25 0 5 0 512574294 323186688 67346 33554432000 4194304 4356188 140734685529840 18446744073709551615 4330995 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25930/tid=25939] ppid=25925 vsize=315612 CPUtime=4973.34 cores=4-7
/proc/25930/task/25939/stat : 25939 (plingeling) R 25925 25925 25435 0 -1 4202560 206080 1251 0 0 497306 28 0 0 25 0 5 0 512574294 323186688 67346 33554432000 4194304 4356188 140734685529840 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25930/tid=25940] ppid=25925 vsize=315612 CPUtime=5000.46 cores=4-7
/proc/25930/task/25940/stat : 25940 (plingeling) R 25925 25925 25435 0 -1 4202560 102726 1251 0 0 500033 13 0 0 25 0 5 0 512574294 323186688 67346 33554432000 4194304 4356188 140734685529840 18446744073709551615 4331002 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 35085.5
Current children cumulated vsize (KiB) 3113176

[startup+5082.3 s]
/proc/loadavg: 7.12 7.06 7.00 8/179 26099
/proc/meminfo: memFree=12478220/32951124 swapFree=67111528/67111528
[pid=25925] ppid=25923 vsize=18324 CPUtime=0.01 cores=0-7
/proc/25925/stat : 25925 (ppfolio) S 25923 25925 25435 0 -1 4202496 523 1193 0 0 0 0 0 1 15 0 1 0 512574266 18763776 391 33554432000 4194304 4287625 140737173188656 18446744073709551615 222921797135 0 0 4096 65536 9224195742862639362 0 0 17 0 0 0 0
/proc/25925/statm: 4581 391 342 23 0 91 0
[pid=25926] ppid=25925 vsize=2153800 CPUtime=5082.44 cores=0
/proc/25926/stat : 25926 (cryptominisat) R 25925 25925 25435 0 -1 4202496 442414 0 0 0 508074 170 0 0 25 0 1 0 512574267 2205491200 48898 33554432000 4194304 4587354 140737180006336 18446744073709551615 4236698 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/25926/statm: 538450 48898 336 96 0 535012 0
[pid=25927] ppid=25925 vsize=420512 CPUtime=5082.42 cores=1
/proc/25927/stat : 25927 (clasp) R 25925 25925 25435 0 -1 4202496 101713 0 0 0 508197 45 0 0 25 0 1 0 512574267 430604288 98592 33554432000 4194304 4785740 140736755224832 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/25927/statm: 105128 98592 340 145 0 102173 0
[pid=25929] ppid=25925 vsize=193652 CPUtime=5082.44 cores=3
/proc/25929/stat : 25929 (march_hi) R 25925 25925 25435 0 -1 4202496 94100 0 0 0 508228 16 0 0 25 0 1 0 512574267 198299648 27632 33554432000 4194304 4331988 140734158658672 18446744073709551615 4292147 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/25929/statm: 48413 27632 175 34 0 46855 0
[pid=25930] ppid=25925 vsize=311516 CPUtime=20257.9 cores=4-7
/proc/25930/stat : 25930 (plingeling) S 25925 25925 25435 0 -1 4202496 800777 1251 0 0 2025679 111 0 0 25 0 5 0 512574267 318992384 66322 33554432000 4194304 4356188 140734685529840 18446744073709551615 222933579749 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25930/statm: 77879 66322 163 40 0 76423 0
[pid=25930/tid=25937] ppid=25925 vsize=311516 CPUtime=5082.16 cores=4-7
/proc/25930/task/25937/stat : 25937 (plingeling) R 25925 25925 25435 0 -1 4202560 260559 1251 0 0 508180 36 0 0 25 0 5 0 512574294 318992384 66322 33554432000 4194304 4356188 140734685529840 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25930/tid=25938] ppid=25925 vsize=311516 CPUtime=5081.97 cores=4-7
/proc/25930/task/25938/stat : 25938 (plingeling) R 25925 25925 25435 0 -1 4202560 212375 1251 0 0 508167 30 0 0 25 0 5 0 512574294 318992384 66322 33554432000 4194304 4356188 140734685529840 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25930/tid=25939] ppid=25925 vsize=311516 CPUtime=5033 cores=4-7
/proc/25930/task/25939/stat : 25939 (plingeling) R 25925 25925 25435 0 -1 4202560 211712 1251 0 0 503272 28 0 0 25 0 5 0 512574294 318992384 66322 33554432000 4194304 4356188 140734685529840 18446744073709551615 4331249 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25930/tid=25940] ppid=25925 vsize=311516 CPUtime=5060.46 cores=4-7
/proc/25930/task/25940/stat : 25940 (plingeling) R 25925 25925 25435 0 -1 4202560 103494 1251 0 0 506033 13 0 0 25 0 5 0 512574294 318992384 66322 33554432000 4194304 4356188 140734685529840 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 35505.2
Current children cumulated vsize (KiB) 3097804



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 7.08 7.06 7.00 8/179 26100
/proc/meminfo: memFree=12480080/32951124 swapFree=67111528/67111528
[pid=25925] ppid=25923 vsize=18324 CPUtime=0.01 cores=0-7
/proc/25925/stat : 25925 (ppfolio) S 25923 25925 25435 0 -1 4202496 523 1193 0 0 0 0 0 1 15 0 1 0 512574266 18763776 391 33554432000 4194304 4287625 140737173188656 18446744073709551615 222921797135 0 0 4096 65536 9224195742862639362 0 0 17 0 0 0 0
/proc/25925/statm: 4581 391 342 23 0 91 0
[pid=25926] ppid=25925 vsize=2153800 CPUtime=5100.14 cores=0
/proc/25926/stat : 25926 (cryptominisat) R 25925 25925 25435 0 -1 4202496 443607 0 0 0 509844 170 0 0 25 0 1 0 512574267 2205491200 50063 33554432000 4194304 4587354 140737180006336 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/25926/statm: 538450 50063 336 96 0 535012 0
[pid=25927] ppid=25925 vsize=420512 CPUtime=5100.12 cores=1
/proc/25927/stat : 25927 (clasp) R 25925 25925 25435 0 -1 4202496 101715 0 0 0 509967 45 0 0 25 0 1 0 512574267 430604288 98594 33554432000 4194304 4785740 140736755224832 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/25927/statm: 105128 98594 340 145 0 102173 0
[pid=25929] ppid=25925 vsize=193652 CPUtime=5100.15 cores=3
/proc/25929/stat : 25929 (march_hi) R 25925 25925 25435 0 -1 4202496 94113 0 0 0 509999 16 0 0 25 0 1 0 512574267 198299648 27645 33554432000 4194304 4331988 140734158658672 18446744073709551615 4265684 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/25929/statm: 48413 27645 175 34 0 46855 0
[pid=25930] ppid=25925 vsize=307420 CPUtime=20328.6 cores=4-7
/proc/25930/stat : 25930 (plingeling) S 25925 25925 25435 0 -1 4202496 803849 1251 0 0 2032751 112 0 0 25 0 5 0 512574267 314798080 65298 33554432000 4194304 4356188 140734685529840 18446744073709551615 222933579749 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/25930/statm: 76855 65298 163 40 0 75399 0
[pid=25930/tid=25937] ppid=25925 vsize=307420 CPUtime=5099.86 cores=4-7
/proc/25930/task/25937/stat : 25937 (plingeling) R 25925 25925 25435 0 -1 4202560 262095 1251 0 0 509950 36 0 0 25 0 5 0 512574294 314798080 65298 33554432000 4194304 4356188 140734685529840 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25930/tid=25938] ppid=25925 vsize=307420 CPUtime=5099.67 cores=4-7
/proc/25930/task/25938/stat : 25938 (plingeling) R 25925 25925 25435 0 -1 4202560 213399 1251 0 0 509937 30 0 0 25 0 5 0 512574294 314798080 65298 33554432000 4194304 4356188 140734685529840 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25930/tid=25939] ppid=25925 vsize=307420 CPUtime=5050.61 cores=4-7
/proc/25930/task/25939/stat : 25939 (plingeling) R 25925 25925 25435 0 -1 4202560 211712 1251 0 0 505033 28 0 0 25 0 5 0 512574294 314798080 65298 33554432000 4194304 4356188 140734685529840 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25930/tid=25940] ppid=25925 vsize=307420 CPUtime=5078.17 cores=4-7
/proc/25930/task/25940/stat : 25940 (plingeling) R 25925 25925 25435 0 -1 4202560 104006 1251 0 0 507804 13 0 0 25 0 5 0 512574294 314798080 65298 33554432000 4194304 4356188 140734685529840 18446744073709551615 4330931 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 35629.1
Current children cumulated vsize (KiB) 3093708

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 25925 and gives
#  childrusage.ru_utime.tv_sec=5099
#  childrusage.ru_utime.tv_usec=994682
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=191970
# CPU time returned by wait4() is 5100.19
# while last known CPU time is 35629.1
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5100.02
CPU time (s): 35629.1
CPU user time (s): 35625.6
CPU system time (s): 3.44
CPU usage (%): 698.606
Max. virtual memory (cumulated for all children) (KiB): 5137604

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 5099.99
system time used= 0.19197
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 95829
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 472
involuntary context switches= 5806

runsolver used 8.80966 second user time and 19.0161 second system time

The end

Launcher Data 

Begin job on node109 at 2011-04-28 18:55:35
IDJOB=3310316
IDBENCH=83121
IDSOLVER=1638
FILE ID=node109/3310316-1304009735
RUNJOBID= node109-1304007435-25449
PBS_JOBID= 13168834
Free space on /tmp= 72380 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-top25.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3310316-1304009735/watcher-3310316-1304009735 -o /tmp/evaluation-result-3310316-1304009735/solver-3310316-1304009735 -C 40000 -W 5100 -M 31000  ppfolio HOME/instance-3310316-1304009735.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 1d36146d5d79c91e0605a19242395228
RANDOM SEED=1334093919

node109.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      13446576 kB
Buffers:       2132460 kB
Cached:       16723924 kB
SwapCached:          0 kB
Active:        9838460 kB
Inactive:      9086992 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      13446576 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           13968 kB
Writeback:           0 kB
AnonPages:       68924 kB
Mapped:          14524 kB
Slab:           513144 kB
PageTables:       4696 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   189584 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72368 MiB
End job on node109 at 2011-04-28 20:20:37