Trace number 3219507

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 39972.2 5000.02

General information on the benchmark

Namerandom/large/
unif-k7-r85-v400-c34000-S927774663-037-UNKNOWN.cnf
MD5SUM950830e22ef92fb7f22c5b3c987a21f5
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables400
Number of clauses34000
Sum of the clauses size238000
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 534000

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3219507-1304150691/watcher-3219507-1304150691 -o /tmp/evaluation-result-3219507-1304150691/solver-3219507-1304150691 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3219507-1304150691.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.50 8.16 8.06 2/174 17123
/proc/meminfo: memFree=29982836/32951124 swapFree=67102748/67111528
[pid=17123] ppid=17121 vsize=18324 CPUtime=0 cores=0-7
/proc/17123/stat : 17123 (ppfolio) S 17121 17123 16810 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 526667338 18763776 390 33554432000 4194304 4287625 140733475364864 18446744073709551615 246516854287 0 0 4096 65536 4594964512448184319 0 0 17 6 0 0 0
/proc/17123/statm: 4581 390 341 23 0 91 0

[startup+0.0763829 s]
/proc/loadavg: 8.50 8.16 8.06 2/174 17123
/proc/meminfo: memFree=29982836/32951124 swapFree=67102748/67111528
[pid=17123] ppid=17121 vsize=18324 CPUtime=0 cores=0-7
/proc/17123/stat : 17123 (ppfolio) S 17121 17123 16810 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 526667338 18763776 390 33554432000 4194304 4287625 140733475364864 18446744073709551615 246516854287 0 0 4096 65536 4594964512448184319 0 0 17 4 0 0 0
/proc/17123/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100351 s]
/proc/loadavg: 8.50 8.16 8.06 2/174 17123
/proc/meminfo: memFree=29982836/32951124 swapFree=67102748/67111528
[pid=17123] ppid=17121 vsize=18324 CPUtime=0 cores=0-7
/proc/17123/stat : 17123 (ppfolio) S 17121 17123 16810 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 526667338 18763776 390 33554432000 4194304 4287625 140733475364864 18446744073709551615 246516854287 0 0 4096 65536 4594964512448184319 0 0 17 4 0 0 0
/proc/17123/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300315 s]
/proc/loadavg: 8.50 8.16 8.06 2/174 17123
/proc/meminfo: memFree=29982836/32951124 swapFree=67102748/67111528
[pid=17123] ppid=17121 vsize=18324 CPUtime=0 cores=0-7
/proc/17123/stat : 17123 (ppfolio) S 17121 17123 16810 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 526667338 18763776 390 33554432000 4194304 4287625 140733475364864 18446744073709551615 246516854287 0 0 4096 65536 4594964512448184319 0 0 17 4 0 0 0
/proc/17123/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700235 s]
/proc/loadavg: 8.50 8.16 8.06 2/174 17123
/proc/meminfo: memFree=29982836/32951124 swapFree=67102748/67111528
[pid=17123] ppid=17121 vsize=18324 CPUtime=0 cores=0-7
/proc/17123/stat : 17123 (ppfolio) S 17121 17123 16810 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 526667338 18763776 390 33554432000 4194304 4287625 140733475364864 18446744073709551615 246516854287 0 0 4096 65536 4594964512448184319 0 0 17 4 0 0 0
/proc/17123/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50109 s]
/proc/loadavg: 8.50 8.16 8.06 9/184 17138
/proc/meminfo: memFree=29930676/32951124 swapFree=67102748/67111528
[pid=17123] ppid=17121 vsize=18324 CPUtime=0 cores=0-7
/proc/17123/stat : 17123 (ppfolio) S 17121 17123 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 526667338 18763776 390 33554432000 4194304 4287625 140733475364864 18446744073709551615 246516854287 0 0 4096 65536 4594964512448184319 0 0 17 4 0 0 0
/proc/17123/statm: 4581 390 341 23 0 91 0
[pid=17124] ppid=17123 vsize=1287468 CPUtime=1.48 cores=0
/proc/17124/stat : 17124 (cryptominisat) R 17123 17123 16810 0 -1 4202496 6301 0 0 0 148 0 0 0 20 0 1 0 526667338 1318367232 2090 33554432000 4194304 4587354 140733997697152 18446744073709551615 4238763 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17124/statm: 321867 2090 327 96 0 318429 0
[pid=17125] ppid=17123 vsize=24336 CPUtime=1.49 cores=1
/proc/17125/stat : 17125 (clasp) R 17123 17123 16810 0 -1 4202496 3673 0 0 0 148 1 0 0 20 0 1 0 526667338 24920064 3405 33554432000 4194304 4785740 140734569087728 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17125/statm: 6084 3405 340 145 0 3129 0
[pid=17126] ppid=17123 vsize=456220 CPUtime=1.49 cores=2
/proc/17126/stat : 17126 (TNM) R 17123 17123 16810 0 -1 4202496 1457 0 0 0 149 0 0 0 20 0 1 0 526667338 467169280 1354 33554432000 134512640 134534564 4292647616 18446744073709551615 134524554 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17126/statm: 114055 1354 91 6 0 112721 0
[pid=17127] ppid=17123 vsize=12556 CPUtime=1.48 cores=3
/proc/17127/stat : 17127 (march_hi) R 17123 17123 16810 0 -1 4202496 6380 0 0 0 148 0 0 0 20 0 1 0 526667338 12857344 1360 33554432000 4194304 4331988 140735051714688 18446744073709551615 4226645 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17127/statm: 3139 1360 150 34 0 1581 0
[pid=17128] ppid=17123 vsize=128044 CPUtime=5.75 cores=4-7
/proc/17128/stat : 17128 (plingeling) S 17123 17123 16810 0 -1 4202496 7494 1248 0 0 574 1 0 0 18 0 5 0 526667338 131117056 4051 33554432000 4194304 4356188 140734483126608 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17128/statm: 32011 4051 161 40 0 30555 0
[pid=17128/tid=17135] ppid=17123 vsize=128044 CPUtime=1.42 cores=4-7
/proc/17128/task/17135/stat : 17135 (plingeling) R 17123 17123 16810 0 -1 4202560 588 1248 0 0 142 0 0 0 24 0 5 0 526667345 131117056 4051 33554432000 4194304 4356188 140734483126608 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17128/tid=17136] ppid=17123 vsize=128044 CPUtime=1.42 cores=4-7
/proc/17128/task/17136/stat : 17136 (plingeling) R 17123 17123 16810 0 -1 4202560 581 1248 0 0 142 0 0 0 20 0 5 0 526667345 131117056 4051 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17128/tid=17137] ppid=17123 vsize=128044 CPUtime=1.41 cores=4-7
/proc/17128/task/17137/stat : 17137 (plingeling) R 17123 17123 16810 0 -1 4202560 591 1248 0 0 141 0 0 0 25 0 5 0 526667345 131117056 4051 33554432000 4194304 4356188 140734483126608 18446744073709551615 4279478 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17128/tid=17138] ppid=17123 vsize=128044 CPUtime=1.42 cores=4-7
/proc/17128/task/17138/stat : 17138 (plingeling) R 17123 17123 16810 0 -1 4202560 1117 1248 0 0 142 0 0 0 23 0 5 0 526667345 131117056 4051 33554432000 4194304 4356188 140734483126608 18446744073709551615 246516877402 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.69
Current children cumulated vsize (KiB) 1926948

[startup+3.1008 s]
/proc/loadavg: 8.50 8.16 8.06 9/184 17138
/proc/meminfo: memFree=29934272/32951124 swapFree=67102748/67111528
[pid=17123] ppid=17121 vsize=18324 CPUtime=0 cores=0-7
/proc/17123/stat : 17123 (ppfolio) S 17121 17123 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 526667338 18763776 390 33554432000 4194304 4287625 140733475364864 18446744073709551615 246516854287 0 0 4096 65536 4594964512448184319 0 0 17 4 0 0 0
/proc/17123/statm: 4581 390 341 23 0 91 0
[pid=17124] ppid=17123 vsize=1288716 CPUtime=3.09 cores=0
/proc/17124/stat : 17124 (cryptominisat) R 17123 17123 16810 0 -1 4202496 9495 0 0 0 308 1 0 0 25 0 1 0 526667338 1319645184 2801 33554432000 4194304 4587354 140733997697152 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17124/statm: 322179 2801 327 96 0 318741 0
[pid=17125] ppid=17123 vsize=25404 CPUtime=3.09 cores=1
/proc/17125/stat : 17125 (clasp) R 17123 17123 16810 0 -1 4202496 3932 0 0 0 308 1 0 0 25 0 1 0 526667338 26013696 3664 33554432000 4194304 4785740 140734569087728 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17125/statm: 6351 3664 340 145 0 3396 0
[pid=17126] ppid=17123 vsize=456220 CPUtime=3.09 cores=2
/proc/17126/stat : 17126 (TNM) R 17123 17123 16810 0 -1 4202496 1457 0 0 0 309 0 0 0 25 0 1 0 526667338 467169280 1354 33554432000 134512640 134534564 4292647616 18446744073709551615 134524469 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17126/statm: 114055 1354 91 6 0 112721 0
[pid=17127] ppid=17123 vsize=17024 CPUtime=3.09 cores=3
/proc/17127/stat : 17127 (march_hi) R 17123 17123 16810 0 -1 4202496 7762 0 0 0 308 1 0 0 25 0 1 0 526667338 17432576 2242 33554432000 4194304 4331988 140735051714688 18446744073709551615 4287467 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17127/statm: 4256 2242 174 34 0 2698 0
[pid=17128] ppid=17123 vsize=129340 CPUtime=12.14 cores=4-7
/proc/17128/stat : 17128 (plingeling) S 17123 17123 16810 0 -1 4202496 9485 1248 0 0 1213 1 0 0 18 0 5 0 526667338 132444160 4407 33554432000 4194304 4356188 140734483126608 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17128/statm: 32335 4407 161 40 0 30879 0
[pid=17128/tid=17135] ppid=17123 vsize=129340 CPUtime=3.02 cores=4-7
/proc/17128/task/17135/stat : 17135 (plingeling) R 17123 17123 16810 0 -1 4202560 1230 1248 0 0 302 0 0 0 25 0 5 0 526667345 132444160 4407 33554432000 4194304 4356188 140734483126608 18446744073709551615 4279586 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17128/tid=17136] ppid=17123 vsize=129340 CPUtime=3.02 cores=4-7
/proc/17128/task/17136/stat : 17136 (plingeling) R 17123 17123 16810 0 -1 4202560 1191 1248 0 0 302 0 0 0 25 0 5 0 526667345 132444160 4407 33554432000 4194304 4356188 140734483126608 18446744073709551615 4279461 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17128/tid=17137] ppid=17123 vsize=129340 CPUtime=3 cores=4-7
/proc/17128/task/17137/stat : 17137 (plingeling) R 17123 17123 16810 0 -1 4202560 1201 1248 0 0 300 0 0 0 25 0 5 0 526667345 132444160 4407 33554432000 4194304 4356188 140734483126608 18446744073709551615 4267687 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17128/tid=17138] ppid=17123 vsize=129340 CPUtime=3.02 cores=4-7
/proc/17128/task/17138/stat : 17138 (plingeling) R 17123 17123 16810 0 -1 4202560 1246 1248 0 0 302 0 0 0 25 0 5 0 526667345 132444160 4407 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331045 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 24.5
Current children cumulated vsize (KiB) 1935028

[startup+6.30122 s]
/proc/loadavg: 8.46 8.16 8.06 9/184 17138
/proc/meminfo: memFree=29908232/32951124 swapFree=67102748/67111528
[pid=17123] ppid=17121 vsize=18324 CPUtime=0 cores=0-7
/proc/17123/stat : 17123 (ppfolio) S 17121 17123 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 526667338 18763776 390 33554432000 4194304 4287625 140733475364864 18446744073709551615 246516854287 0 0 4096 65536 4594964512448184319 0 0 17 4 0 0 0
/proc/17123/statm: 4581 390 341 23 0 91 0
[pid=17124] ppid=17123 vsize=1313320 CPUtime=6.28 cores=0
/proc/17124/stat : 17124 (cryptominisat) R 17123 17123 16810 0 -1 4202496 15364 0 0 0 627 1 0 0 25 0 1 0 526667338 1344839680 7221 33554432000 4194304 4587354 140733997697152 18446744073709551615 4238668 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17124/statm: 328330 7221 337 96 0 324892 0
[pid=17125] ppid=17123 vsize=28276 CPUtime=6.29 cores=1
/proc/17125/stat : 17125 (clasp) R 17123 17123 16810 0 -1 4202496 4654 0 0 0 628 1 0 0 25 0 1 0 526667338 28954624 4386 33554432000 4194304 4785740 140734569087728 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17125/statm: 7069 4386 340 145 0 4114 0
[pid=17126] ppid=17123 vsize=460708 CPUtime=6.28 cores=2
/proc/17126/stat : 17126 (TNM) R 17123 17123 16810 0 -1 4202496 1816 0 0 0 628 0 0 0 25 0 1 0 526667338 471764992 1713 33554432000 134512640 134534564 4292647616 18446744073709551615 134530632 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17126/statm: 115177 1713 95 6 0 113843 0
[pid=17127] ppid=17123 vsize=17024 CPUtime=6.29 cores=3
/proc/17127/stat : 17127 (march_hi) R 17123 17123 16810 0 -1 4202496 7763 0 0 0 628 1 0 0 25 0 1 0 526667338 17432576 2243 33554432000 4194304 4331988 140735051714688 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17127/statm: 4256 2243 174 34 0 2698 0
[pid=17128] ppid=17123 vsize=132772 CPUtime=24.93 cores=4-7
/proc/17128/stat : 17128 (plingeling) S 17123 17123 16810 0 -1 4202496 11847 1248 0 0 2491 2 0 0 18 0 5 0 526667338 135958528 5233 33554432000 4194304 4356188 140734483126608 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17128/statm: 33193 5233 161 40 0 31737 0
[pid=17128/tid=17135] ppid=17123 vsize=132772 CPUtime=6.22 cores=4-7
/proc/17128/task/17135/stat : 17135 (plingeling) R 17123 17123 16810 0 -1 4202560 1989 1248 0 0 622 0 0 0 25 0 5 0 526667345 135958528 5233 33554432000 4194304 4356188 140734483126608 18446744073709551615 4280864 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17128/tid=17136] ppid=17123 vsize=132772 CPUtime=6.22 cores=4-7
/proc/17128/task/17136/stat : 17136 (plingeling) R 17123 17123 16810 0 -1 4202560 1416 1248 0 0 622 0 0 0 25 0 5 0 526667345 135958528 5233 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331141 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17128/tid=17137] ppid=17123 vsize=132772 CPUtime=6.19 cores=4-7
/proc/17128/task/17137/stat : 17137 (plingeling) R 17123 17123 16810 0 -1 4202560 1329 1248 0 0 619 0 0 0 25 0 5 0 526667345 135958528 5233 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331045 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17128/tid=17138] ppid=17123 vsize=132772 CPUtime=6.21 cores=4-7
/proc/17128/task/17138/stat : 17138 (plingeling) R 17123 17123 16810 0 -1 4202560 2496 1248 0 0 621 0 0 0 25 0 5 0 526667345 135958528 5233 33554432000 4194304 4356188 140734483126608 18446744073709551615 4242494 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 50.07
Current children cumulated vsize (KiB) 1970424

[startup+12.701 s]
/proc/loadavg: 8.42 8.16 8.06 9/184 17138
/proc/meminfo: memFree=29874076/32951124 swapFree=67102748/67111528
[pid=17123] ppid=17121 vsize=18324 CPUtime=0 cores=0-7
/proc/17123/stat : 17123 (ppfolio) S 17121 17123 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 526667338 18763776 390 33554432000 4194304 4287625 140733475364864 18446744073709551615 246516854287 0 0 4096 65536 4594964512448184319 0 0 17 4 0 0 0
/proc/17123/statm: 4581 390 341 23 0 91 0
[pid=17124] ppid=17123 vsize=1318852 CPUtime=12.69 cores=0
/proc/17124/stat : 17124 (cryptominisat) R 17123 17123 16810 0 -1 4202496 22035 0 0 0 1266 3 0 0 25 0 1 0 526667338 1350504448 12307 33554432000 4194304 4587354 140733997697152 18446744073709551615 4365356 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17124/statm: 329713 12307 337 96 0 326275 0
[pid=17125] ppid=17123 vsize=32548 CPUtime=12.69 cores=1
/proc/17125/stat : 17125 (clasp) R 17123 17123 16810 0 -1 4202496 5671 0 0 0 1268 1 0 0 25 0 1 0 526667338 33329152 5403 33554432000 4194304 4785740 140734569087728 18446744073709551615 4286919 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17125/statm: 8137 5403 340 145 0 5182 0
[pid=17126] ppid=17123 vsize=460708 CPUtime=12.68 cores=2
/proc/17126/stat : 17126 (TNM) R 17123 17123 16810 0 -1 4202496 1816 0 0 0 1268 0 0 0 25 0 1 0 526667338 471764992 1713 33554432000 134512640 134534564 4292647616 18446744073709551615 134528504 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17126/statm: 115177 1713 95 6 0 113843 0
[pid=17127] ppid=17123 vsize=17024 CPUtime=12.69 cores=3
/proc/17127/stat : 17127 (march_hi) R 17123 17123 16810 0 -1 4202496 7763 0 0 0 1268 1 0 0 25 0 1 0 526667338 17432576 2243 33554432000 4194304 4331988 140735051714688 18446744073709551615 4228205 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17127/statm: 4256 2243 174 34 0 2698 0
[pid=17128] ppid=17123 vsize=136124 CPUtime=50.49 cores=4-7
/proc/17128/stat : 17128 (plingeling) S 17123 17123 16810 0 -1 4202496 15981 1248 0 0 5047 2 0 0 18 0 5 0 526667338 139390976 6071 33554432000 4194304 4356188 140734483126608 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17128/statm: 34031 6071 161 40 0 32575 0
[pid=17128/tid=17135] ppid=17123 vsize=136124 CPUtime=12.62 cores=4-7
/proc/17128/task/17135/stat : 17135 (plingeling) R 17123 17123 16810 0 -1 4202560 3175 1248 0 0 1262 0 0 0 25 0 5 0 526667345 139390976 6071 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17128/tid=17136] ppid=17123 vsize=136124 CPUtime=12.61 cores=4-7
/proc/17128/task/17136/stat : 17136 (plingeling) R 17123 17123 16810 0 -1 4202560 2761 1248 0 0 1261 0 0 0 25 0 5 0 526667345 139390976 6071 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17128/tid=17137] ppid=17123 vsize=136124 CPUtime=12.55 cores=4-7
/proc/17128/task/17137/stat : 17137 (plingeling) R 17123 17123 16810 0 -1 4202560 2001 1248 0 0 1255 0 0 0 25 0 5 0 526667345 139390976 6071 33554432000 4194304 4356188 140734483126608 18446744073709551615 4332272 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17128/tid=17138] ppid=17123 vsize=136124 CPUtime=12.61 cores=4-7
/proc/17128/task/17138/stat : 17138 (plingeling) R 17123 17123 16810 0 -1 4202560 3427 1248 0 0 1261 0 0 0 25 0 5 0 526667345 139390976 6071 33554432000 4194304 4356188 140734483126608 18446744073709551615 4281627 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 101.24
Current children cumulated vsize (KiB) 1983580

[startup+25.5007 s]
/proc/loadavg: 8.33 8.15 8.05 9/184 17139
/proc/meminfo: memFree=29867552/32951124 swapFree=67102748/67111528
[pid=17123] ppid=17121 vsize=18324 CPUtime=0 cores=0-7
/proc/17123/stat : 17123 (ppfolio) S 17121 17123 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 526667338 18763776 390 33554432000 4194304 4287625 140733475364864 18446744073709551615 246516854287 0 0 4096 65536 4594964512448184319 0 0 17 4 0 0 0
/proc/17123/statm: 4581 390 341 23 0 91 0
[pid=17124] ppid=17123 vsize=1328064 CPUtime=25.49 cores=0
/proc/17124/stat : 17124 (cryptominisat) R 17123 17123 16810 0 -1 4202496 31253 0 0 0 2545 4 0 0 25 0 1 0 526667338 1359937536 13178 33554432000 4194304 4587354 140733997697152 18446744073709551615 4365336 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=17126] ppid=17123 vsize=460708 CPUtime=4781.52 cores=2
/proc/17126/stat : 17126 (TNM) R 17123 17123 16810 0 -1 4202496 1816 0 0 0 478150 2 0 0 25 0 1 0 526667338 471764992 1713 33554432000 134512640 134534564 4292647616 18446744073709551615 134530329 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17126/statm: 115177 1713 95 6 0 113843 0
[pid=17127] ppid=17123 vsize=472372 CPUtime=4782.43 cores=3
/proc/17127/stat : 17127 (march_hi) R 17123 17123 16810 0 -1 4202496 118960 0 0 0 478168 75 0 0 25 0 1 0 526667338 483708928 113440 33554432000 4194304 4331988 140735051714688 18446744073709551615 4316460 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17127/statm: 118093 113440 175 34 0 116535 0
[pid=17128] ppid=17123 vsize=231068 CPUtime=19102.9 cores=4-7
/proc/17128/stat : 17128 (plingeling) S 17123 17123 16810 0 -1 4202496 373937 1248 0 0 1910236 59 0 0 18 0 5 0 526667338 236613632 29807 33554432000 4194304 4356188 140734483126608 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17128/statm: 57767 29807 161 40 0 56311 0
[pid=17128/tid=17135] ppid=17123 vsize=231068 CPUtime=4782.36 cores=4-7
/proc/17128/task/17135/stat : 17135 (plingeling) R 17123 17123 16810 0 -1 4202560 102701 1248 0 0 478221 15 0 0 25 0 5 0 526667345 236613632 29807 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17128/tid=17136] ppid=17123 vsize=231068 CPUtime=4782.35 cores=4-7
/proc/17128/task/17136/stat : 17136 (plingeling) R 17123 17123 16810 0 -1 4202560 79219 1248 0 0 478220 15 0 0 25 0 5 0 526667345 236613632 29807 33554432000 4194304 4356188 140734483126608 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17128/tid=17137] ppid=17123 vsize=231068 CPUtime=4756.26 cores=4-7
/proc/17128/task/17137/stat : 17137 (plingeling) R 17123 17123 16810 0 -1 4202560 85373 1248 0 0 475615 11 0 0 25 0 5 0 526667345 236613632 29807 33554432000 4194304 4356188 140734483126608 18446744073709551615 4332260 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17128/tid=17138] ppid=17123 vsize=231068 CPUtime=4781.88 cores=4-7
/proc/17128/task/17138/stat : 17138 (plingeling) R 17123 17123 16810 0 -1 4202560 102027 1248 0 0 478172 16 0 0 25 0 5 0 526667345 236613632 29807 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38231.8
Current children cumulated vsize (KiB) 2848280

[startup+4842.3 s]
/proc/loadavg: 8.27 8.16 8.10 9/184 17279
/proc/meminfo: memFree=29172356/32951124 swapFree=67102748/67111528
[pid=17123] ppid=17121 vsize=18324 CPUtime=0 cores=0-7
/proc/17123/stat : 17123 (ppfolio) S 17121 17123 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 526667338 18763776 390 33554432000 4194304 4287625 140733475364864 18446744073709551615 246516854287 0 0 4096 65536 4594964512448184319 0 0 17 4 0 0 0
/proc/17123/statm: 4581 390 341 23 0 91 0
[pid=17124] ppid=17123 vsize=1327244 CPUtime=4842.43 cores=0
/proc/17124/stat : 17124 (cryptominisat) R 17123 17123 16810 0 -1 4202496 1224832 0 0 0 483744 499 0 0 25 0 1 0 526667338 1359097856 19033 33554432000 4194304 4587354 140733997697152 18446744073709551615 4238568 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17124/statm: 331811 19033 337 96 0 328373 0
[pid=17125] ppid=17123 vsize=196456 CPUtime=4842.42 cores=1
/proc/17125/stat : 17125 (clasp) R 17123 17123 16810 0 -1 4202496 42666 0 0 0 484231 11 0 0 25 0 1 0 526667338 201170944 42000 33554432000 4194304 4785740 140734569087728 18446744073709551615 4309522 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17125/statm: 49114 42000 341 145 0 46159 0
[pid=17126] ppid=17123 vsize=460708 CPUtime=4841.52 cores=2
/proc/17126/stat : 17126 (TNM) R 17123 17123 16810 0 -1 4202496 1816 0 0 0 484150 2 0 0 25 0 1 0 526667338 471764992 1713 33554432000 134512640 134534564 4292647616 18446744073709551615 134528483 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17126/statm: 115177 1713 95 6 0 113843 0
[pid=17127] ppid=17123 vsize=472372 CPUtime=4842.44 cores=3
/proc/17127/stat : 17127 (march_hi) R 17123 17123 16810 0 -1 4202496 120456 0 0 0 484167 77 0 0 25 0 1 0 526667338 483708928 114936 33554432000 4194304 4331988 140735051714688 18446744073709551615 4239564 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17127/statm: 118093 114936 175 34 0 116535 0
[pid=17128] ppid=17123 vsize=213148 CPUtime=19342.6 cores=4-7
/proc/17128/stat : 17128 (plingeling) S 17123 17123 16810 0 -1 4202496 373937 1248 0 0 1934202 60 0 0 18 0 5 0 526667338 218263552 25327 33554432000 4194304 4356188 140734483126608 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17128/statm: 53287 25327 161 40 0 51831 0
[pid=17128/tid=17135] ppid=17123 vsize=213148 CPUtime=4842.36 cores=4-7
/proc/17128/task/17135/stat : 17135 (plingeling) R 17123 17123 16810 0 -1 4202560 102701 1248 0 0 484221 15 0 0 25 0 5 0 526667345 218263552 25327 33554432000 4194304 4356188 140734483126608 18446744073709551615 4279578 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17128/tid=17136] ppid=17123 vsize=213148 CPUtime=4842.35 cores=4-7
/proc/17128/task/17136/stat : 17136 (plingeling) R 17123 17123 16810 0 -1 4202560 79219 1248 0 0 484220 15 0 0 25 0 5 0 526667345 218263552 25327 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331141 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17128/tid=17137] ppid=17123 vsize=213148 CPUtime=4815.93 cores=4-7
/proc/17128/task/17137/stat : 17137 (plingeling) R 17123 17123 16810 0 -1 4202560 85373 1248 0 0 481582 11 0 0 25 0 5 0 526667345 218263552 25327 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17128/tid=17138] ppid=17123 vsize=213148 CPUtime=4841.87 cores=4-7
/proc/17128/task/17138/stat : 17138 (plingeling) R 17123 17123 16810 0 -1 4202560 102027 1248 0 0 484171 16 0 0 25 0 5 0 526667345 218263552 25327 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331309 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38711.4
Current children cumulated vsize (KiB) 2688252

[startup+4902.3 s]
/proc/loadavg: 8.15 8.14 8.10 9/184 17281
/proc/meminfo: memFree=29178536/32951124 swapFree=67102748/67111528
[pid=17123] ppid=17121 vsize=18324 CPUtime=0 cores=0-7
/proc/17123/stat : 17123 (ppfolio) S 17121 17123 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 526667338 18763776 390 33554432000 4194304 4287625 140733475364864 18446744073709551615 246516854287 0 0 4096 65536 4594964512448184319 0 0 17 4 0 0 0
/proc/17123/statm: 4581 390 341 23 0 91 0
[pid=17124] ppid=17123 vsize=1372300 CPUtime=4902.43 cores=0
/proc/17124/stat : 17124 (cryptominisat) R 17123 17123 16810 0 -1 4202496 1236725 0 0 0 489738 505 0 0 25 0 1 0 526667338 1405235200 15551 33554432000 4194304 4587354 140733997697152 18446744073709551615 4238770 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17124/statm: 343075 15551 337 96 0 339637 0
[pid=17125] ppid=17123 vsize=197736 CPUtime=4902.43 cores=1
/proc/17125/stat : 17125 (clasp) R 17123 17123 16810 0 -1 4202496 42953 0 0 0 490231 12 0 0 25 0 1 0 526667338 202481664 42287 33554432000 4194304 4785740 140734569087728 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17125/statm: 49434 42287 341 145 0 46479 0
[pid=17126] ppid=17123 vsize=460708 CPUtime=4901.52 cores=2
/proc/17126/stat : 17126 (TNM) R 17123 17123 16810 0 -1 4202496 1816 0 0 0 490150 2 0 0 25 0 1 0 526667338 471764992 1713 33554432000 134512640 134534564 4292647616 18446744073709551615 134530586 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17126/statm: 115177 1713 95 6 0 113843 0
[pid=17127] ppid=17123 vsize=696136 CPUtime=4902.43 cores=3
/proc/17127/stat : 17127 (march_hi) R 17123 17123 16810 0 -1 4202496 121849 0 0 0 490166 77 0 0 25 0 1 0 526667338 712843264 116329 33554432000 4194304 4331988 140735051714688 18446744073709551615 4237419 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17127/statm: 174034 116329 175 34 0 172476 0
[pid=17128] ppid=17123 vsize=216220 CPUtime=19582.3 cores=4-7
/proc/17128/stat : 17128 (plingeling) S 17123 17123 16810 0 -1 4202496 374705 1248 0 0 1958170 60 0 0 18 0 5 0 526667338 221409280 26095 33554432000 4194304 4356188 140734483126608 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17128/statm: 54055 26095 161 40 0 52599 0
[pid=17128/tid=17135] ppid=17123 vsize=216220 CPUtime=4902.37 cores=4-7
/proc/17128/task/17135/stat : 17135 (plingeling) R 17123 17123 16810 0 -1 4202560 102893 1248 0 0 490222 15 0 0 25 0 5 0 526667345 221409280 26095 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17128/tid=17136] ppid=17123 vsize=216220 CPUtime=4902.35 cores=4-7
/proc/17128/task/17136/stat : 17136 (plingeling) R 17123 17123 16810 0 -1 4202560 79603 1248 0 0 490220 15 0 0 25 0 5 0 526667345 221409280 26095 33554432000 4194304 4356188 140734483126608 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17128/tid=17137] ppid=17123 vsize=216220 CPUtime=4875.61 cores=4-7
/proc/17128/task/17137/stat : 17137 (plingeling) R 17123 17123 16810 0 -1 4202560 85373 1248 0 0 487550 11 0 0 25 0 5 0 526667345 221409280 26095 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331082 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17128/tid=17138] ppid=17123 vsize=216220 CPUtime=4901.87 cores=4-7
/proc/17128/task/17138/stat : 17138 (plingeling) R 17123 17123 16810 0 -1 4202560 102219 1248 0 0 490171 16 0 0 25 0 5 0 526667345 221409280 26095 33554432000 4194304 4356188 140734483126608 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39191.1
Current children cumulated vsize (KiB) 2961424

[startup+4962.3 s]
/proc/loadavg: 8.22 8.16 8.10 9/184 17282
/proc/meminfo: memFree=29136592/32951124 swapFree=67102748/67111528
[pid=17123] ppid=17121 vsize=18324 CPUtime=0 cores=0-7
/proc/17123/stat : 17123 (ppfolio) S 17121 17123 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 526667338 18763776 390 33554432000 4194304 4287625 140733475364864 18446744073709551615 246516854287 0 0 4096 65536 4594964512448184319 0 0 17 4 0 0 0
/proc/17123/statm: 4581 390 341 23 0 91 0
[pid=17124] ppid=17123 vsize=1372300 CPUtime=4962.43 cores=0
/proc/17124/stat : 17124 (cryptominisat) R 17123 17123 16810 0 -1 4202496 1245862 0 0 0 495733 510 0 0 25 0 1 0 526667338 1405235200 24591 33554432000 4194304 4587354 140733997697152 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17124/statm: 343075 24591 337 96 0 339637 0
[pid=17125] ppid=17123 vsize=197736 CPUtime=4962.43 cores=1
/proc/17125/stat : 17125 (clasp) R 17123 17123 16810 0 -1 4202496 42985 0 0 0 496231 12 0 0 25 0 1 0 526667338 202481664 42319 33554432000 4194304 4785740 140734569087728 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17125/statm: 49434 42319 341 145 0 46479 0
[pid=17126] ppid=17123 vsize=460708 CPUtime=4961.52 cores=2
/proc/17126/stat : 17126 (TNM) R 17123 17123 16810 0 -1 4202496 1816 0 0 0 496150 2 0 0 25 0 1 0 526667338 471764992 1713 33554432000 134512640 134534564 4292647616 18446744073709551615 134530158 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17126/statm: 115177 1713 95 6 0 113843 0
[pid=17127] ppid=17123 vsize=696136 CPUtime=4962.44 cores=3
/proc/17127/stat : 17127 (march_hi) R 17123 17123 16810 0 -1 4202496 123295 0 0 0 496165 79 0 0 25 0 1 0 526667338 712843264 117775 33554432000 4194304 4331988 140735051714688 18446744073709551615 4308792 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17127/statm: 174034 117775 175 34 0 172476 0
[pid=17128] ppid=17123 vsize=214428 CPUtime=19822 cores=4-7
/proc/17128/stat : 17128 (plingeling) S 17123 17123 16810 0 -1 4202496 383601 1248 0 0 1982136 61 0 0 18 0 5 0 526667338 219574272 25647 33554432000 4194304 4356188 140734483126608 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17128/statm: 53607 25647 161 40 0 52151 0
[pid=17128/tid=17135] ppid=17123 vsize=214428 CPUtime=4962.36 cores=4-7
/proc/17128/task/17135/stat : 17135 (plingeling) R 17123 17123 16810 0 -1 4202560 105133 1248 0 0 496221 15 0 0 25 0 5 0 526667345 219574272 25647 33554432000 4194304 4356188 140734483126608 18446744073709551615 4281677 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17128/tid=17136] ppid=17123 vsize=214428 CPUtime=4962.35 cores=4-7
/proc/17128/task/17136/stat : 17136 (plingeling) R 17123 17123 16810 0 -1 4202560 81651 1248 0 0 496220 15 0 0 25 0 5 0 526667345 219574272 25647 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331264 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17128/tid=17137] ppid=17123 vsize=214428 CPUtime=4935.28 cores=4-7
/proc/17128/task/17137/stat : 17137 (plingeling) R 17123 17123 16810 0 -1 4202560 85693 1248 0 0 493517 11 0 0 25 0 5 0 526667345 219574272 25647 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17128/tid=17138] ppid=17123 vsize=214428 CPUtime=4961.87 cores=4-7
/proc/17128/task/17138/stat : 17138 (plingeling) R 17123 17123 16810 0 -1 4202560 106507 1248 0 0 496170 17 0 0 25 0 5 0 526667345 219574272 25647 33554432000 4194304 4356188 140734483126608 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39670.8
Current children cumulated vsize (KiB) 2959632



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5000 s]
/proc/loadavg: 8.19 8.16 8.10 9/184 17284
/proc/meminfo: memFree=29156424/32951124 swapFree=67102748/67111528
[pid=17123] ppid=17121 vsize=18324 CPUtime=0 cores=0-7
/proc/17123/stat : 17123 (ppfolio) S 17121 17123 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 526667338 18763776 390 33554432000 4194304 4287625 140733475364864 18446744073709551615 246516854287 0 0 4096 65536 4594964512448184319 0 0 17 4 0 0 0
/proc/17123/statm: 4581 390 341 23 0 91 0
[pid=17124] ppid=17123 vsize=1363112 CPUtime=5000.15 cores=0
/proc/17124/stat : 17124 (cryptominisat) R 17123 17123 16810 0 -1 4202496 1255259 0 0 0 499500 515 0 0 25 0 1 0 526667338 1395826688 18630 33554432000 4194304 4587354 140733997697152 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17124/statm: 340778 18630 337 96 0 337340 0
[pid=17125] ppid=17123 vsize=198664 CPUtime=5000.14 cores=1
/proc/17125/stat : 17125 (clasp) R 17123 17123 16810 0 -1 4202496 43185 0 0 0 500002 12 0 0 25 0 1 0 526667338 203431936 42519 33554432000 4194304 4785740 140734569087728 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17125/statm: 49666 42519 341 145 0 46711 0
[pid=17126] ppid=17123 vsize=460708 CPUtime=4999.23 cores=2
/proc/17126/stat : 17126 (TNM) R 17123 17123 16810 0 -1 4202496 1816 0 0 0 499921 2 0 0 25 0 1 0 526667338 471764992 1713 33554432000 134512640 134534564 4292647616 18446744073709551615 134530438 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17126/statm: 115177 1713 95 6 0 113843 0
[pid=17127] ppid=17123 vsize=696136 CPUtime=5000.15 cores=3
/proc/17127/stat : 17127 (march_hi) R 17123 17123 16810 0 -1 4202496 124247 0 0 0 499935 80 0 0 25 0 1 0 526667338 712843264 118727 33554432000 4194304 4331988 140735051714688 18446744073709551615 4246530 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17127/statm: 174034 118727 175 34 0 172476 0
[pid=17128] ppid=17123 vsize=213148 CPUtime=19972.6 cores=4-7
/proc/17128/stat : 17128 (plingeling) S 17123 17123 16810 0 -1 4202496 385713 1248 0 0 1997197 61 0 0 18 0 5 0 526667338 218263552 25327 33554432000 4194304 4356188 140734483126608 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17128/statm: 53287 25327 161 40 0 51831 0
[pid=17128/tid=17135] ppid=17123 vsize=213148 CPUtime=5000.07 cores=4-7
/proc/17128/task/17135/stat : 17135 (plingeling) R 17123 17123 16810 0 -1 4202560 105133 1248 0 0 499992 15 0 0 25 0 5 0 526667345 218263552 25327 33554432000 4194304 4356188 140734483126608 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17128/tid=17136] ppid=17123 vsize=213148 CPUtime=5000.06 cores=4-7
/proc/17128/task/17136/stat : 17136 (plingeling) R 17123 17123 16810 0 -1 4202560 81651 1248 0 0 499991 15 0 0 25 0 5 0 526667345 218263552 25327 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331165 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17128/tid=17137] ppid=17123 vsize=213148 CPUtime=4972.77 cores=4-7
/proc/17128/task/17137/stat : 17137 (plingeling) R 17123 17123 16810 0 -1 4202560 87805 1248 0 0 497266 11 0 0 25 0 5 0 526667345 218263552 25327 33554432000 4194304 4356188 140734483126608 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17128/tid=17138] ppid=17123 vsize=213148 CPUtime=4999.57 cores=4-7
/proc/17128/task/17138/stat : 17138 (plingeling) R 17123 17123 16810 0 -1 4202560 106507 1248 0 0 499940 17 0 0 25 0 5 0 526667345 218263552 25327 33554432000 4194304 4356188 140734483126608 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39972.2
Current children cumulated vsize (KiB) 2950092

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 17123 and gives
#  childrusage.ru_utime.tv_sec=14994
#  childrusage.ru_utime.tv_usec=236529
#  childrusage.ru_stime.tv_sec=5
#  childrusage.ru_stime.tv_usec=321191
# CPU time returned by wait4() is 14999.6
# while last known CPU time is 39972.2
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.02
CPU time (s): 39972.2
CPU user time (s): 39965.6
CPU system time (s): 6.7
CPU usage (%): 799.442
Max. virtual memory (cumulated for all children) (KiB): 8952376

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 14994.2
system time used= 5.32119
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1300798
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 635
involuntary context switches= 33296

runsolver used 7.66083 second user time and 19.62 second system time

The end

Launcher Data 

Begin job on node134 at 2011-04-30 10:04:51
IDJOB=3219507
IDBENCH=82601
IDSOLVER=1638
FILE ID=node134/3219507-1304150691
RUNJOBID= node134-1304145462-16824
PBS_JOBID= 13170886
Free space on /tmp= 72512 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k7-r85-v400-c34000-S927774663-037-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3219507-1304150691/watcher-3219507-1304150691 -o /tmp/evaluation-result-3219507-1304150691/solver-3219507-1304150691 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3219507-1304150691.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 950830e22ef92fb7f22c5b3c987a21f5
RANDOM SEED=1061983380

node134.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 7186.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      29983240 kB
Buffers:       1026760 kB
Cached:        1480724 kB
SwapCached:       4568 kB
Active:         211520 kB
Inactive:      2355560 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      29983240 kB
SwapTotal:    67111528 kB
SwapFree:     67102748 kB
Dirty:           12292 kB
Writeback:           0 kB
AnonPages:       57296 kB
Mapped:          14412 kB
Slab:           335364 kB
PageTables:       5008 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   185592 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72500 MiB
End job on node134 at 2011-04-30 11:28:13