Trace number 3219711

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 39972.2 5000.02

General information on the benchmark

Namerandom/large/
unif-k5-r20-v2000-c40000-S61914820-087-UNKNOWN.cnf
MD5SUM1b9b43753c02a2c226adda648a956e92
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2120.06
Satisfiable
(Un)Satisfiability was proved
Number of variables2000
Number of clauses40000
Sum of the clauses size200000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 540000
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3219711-1307664547/watcher-3219711-1307664547 -o /tmp/evaluation-result-3219711-1307664547/solver-3219711-1307664547 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3219711-1307664547.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.10 5.69 3.56 1/174 32195
/proc/meminfo: memFree=32076900/32951124 swapFree=67054536/67111528
[pid=32195] ppid=32193 vsize=18324 CPUtime=0 cores=0-7
/proc/32195/stat : 32195 (ppfolio) R 32193 32195 32106 0 -1 4202496 520 0 0 0 0 0 0 0 21 0 1 0 878052647 18763776 390 33554432000 4194304 4287625 140734820794064 18446744073709551615 271550557711 0 0 4096 65536 0 0 0 17 2 0 0 0
/proc/32195/statm: 4581 390 341 23 0 91 0

[startup+0.101886 s]
/proc/loadavg: 8.10 5.69 3.56 1/174 32195
/proc/meminfo: memFree=32076900/32951124 swapFree=67054536/67111528
[pid=32195] ppid=32193 vsize=18324 CPUtime=0 cores=0-7
/proc/32195/stat : 32195 (ppfolio) S 32193 32195 32106 0 -1 4202496 521 0 0 0 0 0 0 0 21 0 1 0 878052647 18763776 390 33554432000 4194304 4287625 140734820794064 18446744073709551615 271550557711 0 0 4096 65536 17891253761221853183 0 0 17 6 0 0 0
/proc/32195/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.200868 s]
/proc/loadavg: 8.10 5.69 3.56 1/174 32195
/proc/meminfo: memFree=32076900/32951124 swapFree=67054536/67111528
[pid=32195] ppid=32193 vsize=18324 CPUtime=0 cores=0-7
/proc/32195/stat : 32195 (ppfolio) S 32193 32195 32106 0 -1 4202496 521 0 0 0 0 0 0 0 21 0 1 0 878052647 18763776 390 33554432000 4194304 4287625 140734820794064 18446744073709551615 271550557711 0 0 4096 65536 17891253761221853183 0 0 17 6 0 0 0
/proc/32195/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.30085 s]
/proc/loadavg: 8.10 5.69 3.56 1/174 32195
/proc/meminfo: memFree=32076900/32951124 swapFree=67054536/67111528
[pid=32195] ppid=32193 vsize=18324 CPUtime=0 cores=0-7
/proc/32195/stat : 32195 (ppfolio) S 32193 32195 32106 0 -1 4202496 521 0 0 0 0 0 0 0 19 0 1 0 878052647 18763776 390 33554432000 4194304 4287625 140734820794064 18446744073709551615 271550557711 0 0 4096 65536 17891253761221853183 0 0 17 6 0 0 0
/proc/32195/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700776 s]
/proc/loadavg: 8.10 5.69 3.56 1/174 32195
/proc/meminfo: memFree=32076900/32951124 swapFree=67054536/67111528
[pid=32195] ppid=32193 vsize=18324 CPUtime=0 cores=0-7
/proc/32195/stat : 32195 (ppfolio) S 32193 32195 32106 0 -1 4202496 521 0 0 0 0 0 0 0 17 0 1 0 878052647 18763776 390 33554432000 4194304 4287625 140734820794064 18446744073709551615 271550557711 0 0 4096 65536 17891253761221853183 0 0 17 6 0 0 0
/proc/32195/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50063 s]
/proc/loadavg: 8.09 5.72 3.58 9/184 32210
/proc/meminfo: memFree=32018168/32951124 swapFree=67054536/67111528
[pid=32195] ppid=32193 vsize=18324 CPUtime=0 cores=0-7
/proc/32195/stat : 32195 (ppfolio) S 32193 32195 32106 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 878052647 18763776 390 33554432000 4194304 4287625 140734820794064 18446744073709551615 271550557711 0 0 4096 65536 17891253761221853183 0 0 17 6 0 0 0
/proc/32195/statm: 4581 390 341 23 0 91 0
[pid=32196] ppid=32195 vsize=1757292 CPUtime=1.49 cores=0
/proc/32196/stat : 32196 (cryptominisat) R 32195 32195 32106 0 -1 4202496 6412 0 0 0 149 0 0 0 23 0 1 0 878052648 1799467008 4212 33554432000 4194304 4587354 140736981287696 18446744073709551615 4238499 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/32196/statm: 439323 4212 327 96 0 435885 0
[pid=32197] ppid=32195 vsize=27520 CPUtime=1.49 cores=1
/proc/32197/stat : 32197 (clasp) R 32195 32195 32106 0 -1 4202496 4532 0 0 0 149 0 0 0 20 0 1 0 878052648 28180480 4237 33554432000 4194304 4785740 140735319330864 18446744073709551615 4286953 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/32197/statm: 6880 4237 340 145 0 3925 0
[pid=32198] ppid=32195 vsize=462448 CPUtime=1.49 cores=2
/proc/32198/stat : 32198 (TNM) R 32195 32195 32106 0 -1 4202496 2353 0 0 0 149 0 0 0 20 0 1 0 878052648 473546752 2250 33554432000 134512640 134534564 4294942352 18446744073709551615 134528302 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/32198/statm: 115612 2250 95 6 0 114278 0
[pid=32199] ppid=32195 vsize=19088 CPUtime=1.49 cores=3
/proc/32199/stat : 32199 (march_hi) R 32195 32195 32106 0 -1 4202496 7349 0 0 0 148 1 0 0 25 0 1 0 878052648 19546112 2355 33554432000 4194304 4331988 140734801286160 18446744073709551615 4265440 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/32199/statm: 4772 2355 174 34 0 3214 0
[pid=32200] ppid=32195 vsize=126452 CPUtime=5.72 cores=4-7
/proc/32200/stat : 32200 (plingeling) S 32195 32195 32106 0 -1 4202496 6219 1250 0 0 571 1 0 0 18 0 5 0 878052648 129486848 3686 33554432000 4194304 4356188 140735126655328 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/32200/statm: 31613 3686 162 40 0 30157 0
[pid=32200/tid=32207] ppid=32195 vsize=126452 CPUtime=1.41 cores=4-7
/proc/32200/task/32207/stat : 32207 (plingeling) R 32195 32195 32106 0 -1 4202560 612 1250 0 0 141 0 0 0 25 0 5 0 878052655 129486848 3686 33554432000 4194304 4356188 140735126655328 18446744073709551615 4234339 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32200/tid=32208] ppid=32195 vsize=126452 CPUtime=1.4 cores=4-7
/proc/32200/task/32208/stat : 32208 (plingeling) R 32195 32195 32106 0 -1 4202560 606 1250 0 0 140 0 0 0 25 0 5 0 878052655 129486848 3686 33554432000 4194304 4356188 140735126655328 18446744073709551615 4211045 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32200/tid=32209] ppid=32195 vsize=126452 CPUtime=1.41 cores=4-7
/proc/32200/task/32209/stat : 32209 (plingeling) R 32195 32195 32106 0 -1 4202560 628 1250 0 0 141 0 0 0 23 0 5 0 878052655 129486848 3686 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331086 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=32200/tid=32210] ppid=32195 vsize=126452 CPUtime=1.4 cores=4-7
/proc/32200/task/32210/stat : 32210 (plingeling) R 32195 32195 32106 0 -1 4202560 691 1250 0 0 140 0 0 0 23 0 5 0 878052655 129486848 3686 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.68
Current children cumulated vsize (KiB) 2411124

[startup+3.10134 s]
/proc/loadavg: 8.09 5.72 3.58 9/184 32210
/proc/meminfo: memFree=32009984/32951124 swapFree=67054536/67111528
[pid=32195] ppid=32193 vsize=18324 CPUtime=0 cores=0-7
/proc/32195/stat : 32195 (ppfolio) S 32193 32195 32106 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 878052647 18763776 390 33554432000 4194304 4287625 140734820794064 18446744073709551615 271550557711 0 0 4096 65536 17891253761221853183 0 0 17 6 0 0 0
/proc/32195/statm: 4581 390 341 23 0 91 0
[pid=32196] ppid=32195 vsize=1757532 CPUtime=3.09 cores=0
/proc/32196/stat : 32196 (cryptominisat) R 32195 32195 32106 0 -1 4202496 10189 0 0 0 308 1 0 0 25 0 1 0 878052648 1799712768 3775 33554432000 4194304 4587354 140736981287696 18446744073709551615 4244000 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/32196/statm: 439383 3775 327 96 0 435945 0
[pid=32197] ppid=32195 vsize=32104 CPUtime=3.09 cores=1
/proc/32197/stat : 32197 (clasp) R 32195 32195 32106 0 -1 4202496 5656 0 0 0 308 1 0 0 25 0 1 0 878052648 32874496 5361 33554432000 4194304 4785740 140735319330864 18446744073709551615 4549795 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/32197/statm: 8026 5361 340 145 0 5071 0
[pid=32198] ppid=32195 vsize=462448 CPUtime=3.09 cores=2
/proc/32198/stat : 32198 (TNM) R 32195 32195 32106 0 -1 4202496 2353 0 0 0 309 0 0 0 25 0 1 0 878052648 473546752 2250 33554432000 134512640 134534564 4294942352 18446744073709551615 134528708 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/32198/statm: 115612 2250 95 6 0 114278 0
[pid=32199] ppid=32195 vsize=19088 CPUtime=3.09 cores=3
/proc/32199/stat : 32199 (march_hi) R 32195 32195 32106 0 -1 4202496 7349 0 0 0 308 1 0 0 25 0 1 0 878052648 19546112 2355 33554432000 4194304 4331988 140734801286160 18446744073709551615 4265452 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/32199/statm: 4772 2355 174 34 0 3214 0
[pid=32200] ppid=32195 vsize=128368 CPUtime=12.12 cores=4-7
/proc/32200/stat : 32200 (plingeling) S 32195 32195 32106 0 -1 4202496 6744 1250 0 0 1211 1 0 0 18 0 5 0 878052648 131448832 4165 33554432000 4194304 4356188 140735126655328 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/32200/statm: 32092 4165 162 40 0 30636 0
[pid=32200/tid=32207] ppid=32195 vsize=128368 CPUtime=3.02 cores=4-7
/proc/32200/task/32207/stat : 32207 (plingeling) R 32195 32195 32106 0 -1 4202560 780 1250 0 0 302 0 0 0 25 0 5 0 878052655 131448832 4165 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32200/tid=32208] ppid=32195 vsize=128368 CPUtime=2.99 cores=4-7
/proc/32200/task/32208/stat : 32208 (plingeling) R 32195 32195 32106 0 -1 4202560 735 1250 0 0 299 0 0 0 25 0 5 0 878052655 131448832 4165 33554432000 4194304 4356188 140735126655328 18446744073709551615 4332225 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32200/tid=32209] ppid=32195 vsize=128368 CPUtime=3.02 cores=4-7
/proc/32200/task/32209/stat : 32209 (plingeling) R 32195 32195 32106 0 -1 4202560 773 1250 0 0 302 0 0 0 25 0 5 0 878052655 131448832 4165 33554432000 4194304 4356188 140735126655328 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=32200/tid=32210] ppid=32195 vsize=128368 CPUtime=3 cores=4-7
/proc/32200/task/32210/stat : 32210 (plingeling) R 32195 32195 32106 0 -1 4202560 774 1250 0 0 300 0 0 0 25 0 5 0 878052655 131448832 4165 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 24.48
Current children cumulated vsize (KiB) 2417864

[startup+6.30076 s]
/proc/loadavg: 8.09 5.72 3.58 9/184 32210
/proc/meminfo: memFree=31954456/32951124 swapFree=67054536/67111528
[pid=32195] ppid=32193 vsize=18324 CPUtime=0 cores=0-7
/proc/32195/stat : 32195 (ppfolio) S 32193 32195 32106 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 878052647 18763776 390 33554432000 4194304 4287625 140734820794064 18446744073709551615 271550557711 0 0 4096 65536 17891253761221853183 0 0 17 6 0 0 0
/proc/32195/statm: 4581 390 341 23 0 91 0
[pid=32196] ppid=32195 vsize=1801712 CPUtime=6.28 cores=0
/proc/32196/stat : 32196 (cryptominisat) R 32195 32195 32106 0 -1 4202496 22113 0 0 0 626 2 0 0 25 0 1 0 878052648 1844953088 15690 33554432000 4194304 4587354 140736981287696 18446744073709551615 4364908 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/32196/statm: 450428 15690 337 96 0 446990 0
[pid=32197] ppid=32195 vsize=36936 CPUtime=6.29 cores=1
/proc/32197/stat : 32197 (clasp) R 32195 32195 32106 0 -1 4202496 6866 0 0 0 628 1 0 0 25 0 1 0 878052648 37822464 6571 33554432000 4194304 4785740 140735319330864 18446744073709551615 4287058 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/32197/statm: 9234 6571 340 145 0 6279 0
[pid=32198] ppid=32195 vsize=462448 CPUtime=6.29 cores=2
/proc/32198/stat : 32198 (TNM) R 32195 32195 32106 0 -1 4202496 2353 0 0 0 629 0 0 0 25 0 1 0 878052648 473546752 2250 33554432000 134512640 134534564 4294942352 18446744073709551615 134528955 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/32198/statm: 115612 2250 95 6 0 114278 0
[pid=32199] ppid=32195 vsize=19088 CPUtime=6.29 cores=3
/proc/32199/stat : 32199 (march_hi) R 32195 32195 32106 0 -1 4202496 7349 0 0 0 628 1 0 0 25 0 1 0 878052648 19546112 2355 33554432000 4194304 4331988 140734801286160 18446744073709551615 4299854 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/32199/statm: 4772 2355 174 34 0 3214 0
[pid=32200] ppid=32195 vsize=131272 CPUtime=24.9 cores=4-7
/proc/32200/stat : 32200 (plingeling) S 32195 32195 32106 0 -1 4202496 7438 1250 0 0 2489 1 0 0 18 0 5 0 878052648 134422528 4859 33554432000 4194304 4356188 140735126655328 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/32200/statm: 32818 4859 162 40 0 31362 0
[pid=32200/tid=32207] ppid=32195 vsize=131272 CPUtime=6.21 cores=4-7
/proc/32200/task/32207/stat : 32207 (plingeling) R 32195 32195 32106 0 -1 4202560 940 1250 0 0 621 0 0 0 25 0 5 0 878052655 134422528 4859 33554432000 4194304 4356188 140735126655328 18446744073709551615 4281627 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32200/tid=32208] ppid=32195 vsize=131272 CPUtime=6.18 cores=4-7
/proc/32200/task/32208/stat : 32208 (plingeling) R 32195 32195 32106 0 -1 4202560 935 1250 0 0 618 0 0 0 25 0 5 0 878052655 134422528 4859 33554432000 4194304 4356188 140735126655328 18446744073709551615 4281643 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32200/tid=32209] ppid=32195 vsize=131272 CPUtime=6.22 cores=4-7
/proc/32200/task/32209/stat : 32209 (plingeling) R 32195 32195 32106 0 -1 4202560 901 1250 0 0 622 0 0 0 25 0 5 0 878052655 134422528 4859 33554432000 4194304 4356188 140735126655328 18446744073709551615 4279478 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=32200/tid=32210] ppid=32195 vsize=131272 CPUtime=6.2 cores=4-7
/proc/32200/task/32210/stat : 32210 (plingeling) R 32195 32195 32106 0 -1 4202560 980 1250 0 0 620 0 0 0 25 0 5 0 878052655 134422528 4859 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 50.05
Current children cumulated vsize (KiB) 2469780

[startup+12.7006 s]
/proc/loadavg: 8.08 5.80 3.63 9/184 32210
/proc/meminfo: memFree=31958308/32951124 swapFree=67054536/67111528
[pid=32195] ppid=32193 vsize=18324 CPUtime=0 cores=0-7
/proc/32195/stat : 32195 (ppfolio) S 32193 32195 32106 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 878052647 18763776 390 33554432000 4194304 4287625 140734820794064 18446744073709551615 271550557711 0 0 4096 65536 17891253761221853183 0 0 17 6 0 0 0
/proc/32195/statm: 4581 390 341 23 0 91 0
[pid=32196] ppid=32195 vsize=1784808 CPUtime=12.68 cores=0
/proc/32196/stat : 32196 (cryptominisat) R 32195 32195 32106 0 -1 4202496 26843 0 0 0 1265 3 0 0 25 0 1 0 878052648 1827643392 10776 33554432000 4194304 4587354 140736981287696 18446744073709551615 4239308 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/32196/statm: 446202 10776 337 96 0 442764 0
[pid=32197] ppid=32195 vsize=45028 CPUtime=12.69 cores=1
/proc/32197/stat : 32197 (clasp) R 32195 32195 32106 0 -1 4202496 8838 0 0 0 1267 2 0 0 25 0 1 0 878052648 46108672 8543 33554432000 4194304 4785740 140735319330864 18446744073709551615 4553440 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/32197/statm: 11257 8543 340 145 0 8302 0
[pid=32198] ppid=32195 vsize=462448 CPUtime=12.69 cores=2
/proc/32198/stat : 32198 (TNM) R 32195 32195 32106 0 -1 4202496 2353 0 0 0 1269 0 0 0 25 0 1 0 878052648 473546752 2250 33554432000 134512640 134534564 4294942352 18446744073709551615 134528906 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/32198/statm: 115612 2250 95 6 0 114278 0
[pid=32199] ppid=32195 vsize=20112 CPUtime=12.69 cores=3
/proc/32199/stat : 32199 (march_hi) R 32195 32195 32106 0 -1 4202496 7560 0 0 0 1268 1 0 0 25 0 1 0 878052648 20594688 2566 33554432000 4194304 4331988 140734801286160 18446744073709551615 271549933259 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/32199/statm: 5028 2566 174 34 0 3470 0
[pid=32200] ppid=32195 vsize=141016 CPUtime=50.46 cores=4-7
/proc/32200/stat : 32200 (plingeling) S 32195 32195 32106 0 -1 4202496 11442 1250 0 0 5044 2 0 0 18 0 5 0 878052648 144400384 7327 33554432000 4194304 4356188 140735126655328 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/32200/statm: 35254 7327 162 40 0 33798 0
[pid=32200/tid=32207] ppid=32195 vsize=141016 CPUtime=12.61 cores=4-7
/proc/32200/task/32207/stat : 32207 (plingeling) R 32195 32195 32106 0 -1 4202560 1805 1250 0 0 1261 0 0 0 25 0 5 0 878052655 144400384 7327 33554432000 4194304 4356188 140735126655328 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32200/tid=32208] ppid=32195 vsize=141016 CPUtime=12.54 cores=4-7
/proc/32200/task/32208/stat : 32208 (plingeling) R 32195 32195 32106 0 -1 4202560 2319 1250 0 0 1254 0 0 0 25 0 5 0 878052655 144400384 7327 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32200/tid=32209] ppid=32195 vsize=141016 CPUtime=12.61 cores=4-7
/proc/32200/task/32209/stat : 32209 (plingeling) R 32195 32195 32106 0 -1 4202560 1766 1250 0 0 1261 0 0 0 25 0 5 0 878052655 144400384 7327 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=32200/tid=32210] ppid=32195 vsize=141016 CPUtime=12.6 cores=4-7
/proc/32200/task/32210/stat : 32210 (plingeling) R 32195 32195 32106 0 -1 4202560 1870 1250 0 0 1260 0 0 0 25 0 5 0 878052655 144400384 7327 33554432000 4194304 4356188 140735126655328 18446744073709551615 4297458 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 101.21
Current children cumulated vsize (KiB) 2471736

[startup+25.5013 s]
/proc/loadavg: 8.21 5.90 3.69 9/184 32211
/proc/meminfo: memFree=31918412/32951124 swapFree=67054536/67111528
[pid=32195] ppid=32193 vsize=18324 CPUtime=0 cores=0-7
/proc/32195/stat : 32195 (ppfolio) S 32193 32195 32106 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 878052647 18763776 390 33554432000 4194304 4287625 140734820794064 18446744073709551615 271550557711 0 0 4096 65536 17891253761221853183 0 0 17 6 0 0 0
/proc/32195/statm: 4581 390 341 23 0 91 0
[pid=32196] ppid=32195 vsize=1833676 CPUtime=25.49 cores=0
/proc/32196/stat : 32196 (cryptominisat) R 32195 32195 32106 0 -1 4202496 40168 0 0 0 2542 7 0 0 25 0 1 0 878052648 1877684224 17847 33554432000 4194304 4587354 140736981287696 18446744073709551615 4388345 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=32198] ppid=32195 vsize=462448 CPUtime=4842.28 cores=2
/proc/32198/stat : 32198 (TNM) R 32195 32195 32106 0 -1 4202496 2353 0 0 0 484225 3 0 0 25 0 1 0 878052648 473546752 2250 33554432000 134512640 134534564 4294942352 18446744073709551615 134530659 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/32198/statm: 115612 2250 95 6 0 114278 0
[pid=32199] ppid=32195 vsize=472408 CPUtime=4842.44 cores=3
/proc/32199/stat : 32199 (march_hi) R 32195 32195 32106 0 -1 4202496 116911 0 0 0 484170 74 0 0 25 0 1 0 878052648 483745792 111917 33554432000 4194304 4331988 140734801286160 18446744073709551615 4298630 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/32199/statm: 118102 111917 176 34 0 116544 0
[pid=32200] ppid=32195 vsize=222276 CPUtime=19342 cores=4-7
/proc/32200/stat : 32200 (plingeling) S 32195 32195 32106 0 -1 4202496 637232 1250 0 0 1934107 90 0 0 18 0 5 0 878052648 227610624 27635 33554432000 4194304 4356188 140735126655328 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/32200/statm: 55569 27635 162 40 0 54113 0
[pid=32200/tid=32207] ppid=32195 vsize=222276 CPUtime=4842.36 cores=4-7
/proc/32200/task/32207/stat : 32207 (plingeling) R 32195 32195 32106 0 -1 4202560 162545 1250 0 0 484215 21 0 0 25 0 5 0 878052655 227610624 27635 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32200/tid=32208] ppid=32195 vsize=222276 CPUtime=4815.92 cores=4-7
/proc/32200/task/32208/stat : 32208 (plingeling) R 32195 32195 32106 0 -1 4202560 156519 1250 0 0 481573 19 0 0 25 0 5 0 878052655 227610624 27635 33554432000 4194304 4356188 140735126655328 18446744073709551615 4211061 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32200/tid=32209] ppid=32195 vsize=222276 CPUtime=4841.27 cores=4-7
/proc/32200/task/32209/stat : 32209 (plingeling) R 32195 32195 32106 0 -1 4202560 150185 1250 0 0 484105 22 0 0 25 0 5 0 878052655 227610624 27635 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=32200/tid=32210] ppid=32195 vsize=222276 CPUtime=4842.32 cores=4-7
/proc/32200/task/32210/stat : 32210 (plingeling) R 32195 32195 32106 0 -1 4202560 164301 1250 0 0 484206 26 0 0 25 0 5 0 878052655 227610624 27635 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331119 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38711.4
Current children cumulated vsize (KiB) 3508008

[startup+4902.3 s]
/proc/loadavg: 8.08 8.07 8.06 9/184 32353
/proc/meminfo: memFree=30880388/32951124 swapFree=67054536/67111528
[pid=32195] ppid=32193 vsize=18324 CPUtime=0 cores=0-7
/proc/32195/stat : 32195 (ppfolio) S 32193 32195 32106 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 878052647 18763776 390 33554432000 4194304 4287625 140734820794064 18446744073709551615 271550557711 0 0 4096 65536 17891253761221853183 0 0 17 6 0 0 0
/proc/32195/statm: 4581 390 341 23 0 91 0
[pid=32196] ppid=32195 vsize=2035836 CPUtime=4902.43 cores=0
/proc/32196/stat : 32196 (cryptominisat) R 32195 32195 32106 0 -1 4202496 1651056 0 0 0 489618 625 0 0 25 0 1 0 878052648 2084696064 89063 33554432000 4194304 4587354 140736981287696 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/32196/statm: 508959 89063 337 96 0 505521 0
[pid=32197] ppid=32195 vsize=296716 CPUtime=4902.28 cores=1
/proc/32197/stat : 32197 (clasp) R 32195 32195 32106 0 -1 4202496 69501 0 0 0 490209 19 0 0 25 0 1 0 878052648 303837184 68978 33554432000 4194304 4785740 140735319330864 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/32197/statm: 74179 68978 340 145 0 71224 0
[pid=32198] ppid=32195 vsize=462448 CPUtime=4902.27 cores=2
/proc/32198/stat : 32198 (TNM) R 32195 32195 32106 0 -1 4202496 2353 0 0 0 490224 3 0 0 25 0 1 0 878052648 473546752 2250 33554432000 134512640 134534564 4294942352 18446744073709551615 134530360 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/32198/statm: 115612 2250 95 6 0 114278 0
[pid=32199] ppid=32195 vsize=472408 CPUtime=4902.43 cores=3
/proc/32199/stat : 32199 (march_hi) R 32195 32195 32106 0 -1 4202496 118343 0 0 0 490168 75 0 0 25 0 1 0 878052648 483745792 113349 33554432000 4194304 4331988 140734801286160 18446744073709551615 4265440 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/32199/statm: 118102 113349 176 34 0 116544 0
[pid=32200] ppid=32195 vsize=214724 CPUtime=19581.6 cores=4-7
/proc/32200/stat : 32200 (plingeling) S 32195 32195 32106 0 -1 4202496 637776 1250 0 0 1958072 90 0 0 18 0 5 0 878052648 219877376 25747 33554432000 4194304 4356188 140735126655328 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/32200/statm: 53681 25747 162 40 0 52225 0
[pid=32200/tid=32207] ppid=32195 vsize=214724 CPUtime=4902.36 cores=4-7
/proc/32200/task/32207/stat : 32207 (plingeling) R 32195 32195 32106 0 -1 4202560 162545 1250 0 0 490215 21 0 0 25 0 5 0 878052655 219877376 25747 33554432000 4194304 4356188 140735126655328 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32200/tid=32208] ppid=32195 vsize=214724 CPUtime=4875.59 cores=4-7
/proc/32200/task/32208/stat : 32208 (plingeling) R 32195 32195 32106 0 -1 4202560 156519 1250 0 0 487540 19 0 0 25 0 5 0 878052655 219877376 25747 33554432000 4194304 4356188 140735126655328 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32200/tid=32209] ppid=32195 vsize=214724 CPUtime=4901.25 cores=4-7
/proc/32200/task/32209/stat : 32209 (plingeling) R 32195 32195 32106 0 -1 4202560 150313 1250 0 0 490103 22 0 0 25 0 5 0 878052655 219877376 25747 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=32200/tid=32210] ppid=32195 vsize=214724 CPUtime=4902.31 cores=4-7
/proc/32200/task/32210/stat : 32210 (plingeling) R 32195 32195 32106 0 -1 4202560 164717 1250 0 0 490205 26 0 0 25 0 5 0 878052655 219877376 25747 33554432000 4194304 4356188 140735126655328 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39191
Current children cumulated vsize (KiB) 3500456

[startup+4962.3 s]
/proc/loadavg: 8.27 8.12 8.07 9/184 32354
/proc/meminfo: memFree=30879720/32951124 swapFree=67054536/67111528
[pid=32195] ppid=32193 vsize=18324 CPUtime=0 cores=0-7
/proc/32195/stat : 32195 (ppfolio) S 32193 32195 32106 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 878052647 18763776 390 33554432000 4194304 4287625 140734820794064 18446744073709551615 271550557711 0 0 4096 65536 17891253761221853183 0 0 17 6 0 0 0
/proc/32195/statm: 4581 390 341 23 0 91 0
[pid=32196] ppid=32195 vsize=2030396 CPUtime=4962.43 cores=0
/proc/32196/stat : 32196 (cryptominisat) R 32195 32195 32106 0 -1 4202496 1668887 0 0 0 495609 634 0 0 25 0 1 0 878052648 2079125504 84824 33554432000 4194304 4587354 140736981287696 18446744073709551615 4238616 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/32196/statm: 507599 84824 337 96 0 504161 0
[pid=32197] ppid=32195 vsize=299272 CPUtime=4962.29 cores=1
/proc/32197/stat : 32197 (clasp) R 32195 32195 32106 0 -1 4202496 70110 0 0 0 496209 20 0 0 25 0 1 0 878052648 306454528 69587 33554432000 4194304 4785740 140735319330864 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/32197/statm: 74818 69587 340 145 0 71863 0
[pid=32198] ppid=32195 vsize=462448 CPUtime=4962.28 cores=2
/proc/32198/stat : 32198 (TNM) R 32195 32195 32106 0 -1 4202496 2353 0 0 0 496225 3 0 0 25 0 1 0 878052648 473546752 2250 33554432000 134512640 134534564 4294942352 18446744073709551615 134528473 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/32198/statm: 115612 2250 95 6 0 114278 0
[pid=32199] ppid=32195 vsize=472408 CPUtime=4962.43 cores=3
/proc/32199/stat : 32199 (march_hi) R 32195 32195 32106 0 -1 4202496 119787 0 0 0 496167 76 0 0 25 0 1 0 878052648 483745792 114793 33554432000 4194304 4331988 140734801286160 18446744073709551615 4228343 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/32199/statm: 118102 114793 176 34 0 116544 0
[pid=32200] ppid=32195 vsize=224452 CPUtime=19821.3 cores=4-7
/proc/32200/stat : 32200 (plingeling) S 32195 32195 32106 0 -1 4202496 642672 1250 0 0 1982038 91 0 0 18 0 5 0 878052648 229838848 28179 33554432000 4194304 4356188 140735126655328 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/32200/statm: 56113 28179 162 40 0 54657 0
[pid=32200/tid=32207] ppid=32195 vsize=224452 CPUtime=4962.36 cores=4-7
/proc/32200/task/32207/stat : 32207 (plingeling) R 32195 32195 32106 0 -1 4202560 164817 1250 0 0 496215 21 0 0 25 0 5 0 878052655 229838848 28179 33554432000 4194304 4356188 140735126655328 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32200/tid=32208] ppid=32195 vsize=224452 CPUtime=4935.26 cores=4-7
/proc/32200/task/32208/stat : 32208 (plingeling) R 32195 32195 32106 0 -1 4202560 156711 1250 0 0 493507 19 0 0 25 0 5 0 878052655 229838848 28179 33554432000 4194304 4356188 140735126655328 18446744073709551615 4233936 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32200/tid=32209] ppid=32195 vsize=224452 CPUtime=4961.25 cores=4-7
/proc/32200/task/32209/stat : 32209 (plingeling) R 32195 32195 32106 0 -1 4202560 152553 1250 0 0 496103 22 0 0 25 0 5 0 878052655 229838848 28179 33554432000 4194304 4356188 140735126655328 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=32200/tid=32210] ppid=32195 vsize=224452 CPUtime=4962.31 cores=4-7
/proc/32200/task/32210/stat : 32210 (plingeling) R 32195 32195 32106 0 -1 4202560 164909 1250 0 0 496205 26 0 0 25 0 5 0 878052655 229838848 28179 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331992 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39670.7
Current children cumulated vsize (KiB) 3507300



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5000 s]
/proc/loadavg: 8.13 8.10 8.07 9/184 32355
/proc/meminfo: memFree=30916156/32951124 swapFree=67054536/67111528
[pid=32195] ppid=32193 vsize=18324 CPUtime=0 cores=0-7
/proc/32195/stat : 32195 (ppfolio) S 32193 32195 32106 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 878052647 18763776 390 33554432000 4194304 4287625 140734820794064 18446744073709551615 271550557711 0 0 4096 65536 17891253761221853183 0 0 17 6 0 0 0
/proc/32195/statm: 4581 390 341 23 0 91 0
[pid=32196] ppid=32195 vsize=2028940 CPUtime=5000.14 cores=0
/proc/32196/stat : 32196 (cryptominisat) R 32195 32195 32106 0 -1 4202496 1678199 0 0 0 499377 637 0 0 25 0 1 0 878052648 2077634560 71942 33554432000 4194304 4587354 140736981287696 18446744073709551615 4238692 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/32196/statm: 507235 71942 337 96 0 503797 0
[pid=32197] ppid=32195 vsize=301172 CPUtime=5000 cores=1
/proc/32197/stat : 32197 (clasp) R 32195 32195 32106 0 -1 4202496 70560 0 0 0 499980 20 0 0 25 0 1 0 878052648 308400128 70037 33554432000 4194304 4785740 140735319330864 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/32197/statm: 75293 70037 340 145 0 72338 0
[pid=32198] ppid=32195 vsize=462448 CPUtime=4999.98 cores=2
/proc/32198/stat : 32198 (TNM) R 32195 32195 32106 0 -1 4202496 2353 0 0 0 499995 3 0 0 25 0 1 0 878052648 473546752 2250 33554432000 134512640 134534564 4294942352 18446744073709551615 134528710 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/32198/statm: 115612 2250 95 6 0 114278 0
[pid=32199] ppid=32195 vsize=696172 CPUtime=5000.14 cores=3
/proc/32199/stat : 32199 (march_hi) R 32195 32195 32106 0 -1 4202496 120680 0 0 0 499937 77 0 0 25 0 1 0 878052648 712880128 115686 33554432000 4194304 4331988 140734801286160 18446744073709551615 4265456 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/32199/statm: 174043 115686 176 34 0 172485 0
[pid=32200] ppid=32195 vsize=233796 CPUtime=19971.9 cores=4-7
/proc/32200/stat : 32200 (plingeling) S 32195 32195 32106 0 -1 4202496 645008 1250 0 0 1997099 92 0 0 18 0 5 0 878052648 239407104 30515 33554432000 4194304 4356188 140735126655328 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/32200/statm: 58449 30515 162 40 0 56993 0
[pid=32200/tid=32207] ppid=32195 vsize=233796 CPUtime=5000.06 cores=4-7
/proc/32200/task/32207/stat : 32207 (plingeling) R 32195 32195 32106 0 -1 4202560 164849 1250 0 0 499985 21 0 0 25 0 5 0 878052655 239407104 30515 33554432000 4194304 4356188 140735126655328 18446744073709551615 4330976 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32200/tid=32208] ppid=32195 vsize=233796 CPUtime=4972.77 cores=4-7
/proc/32200/task/32208/stat : 32208 (plingeling) R 32195 32195 32106 0 -1 4202560 158887 1250 0 0 497257 20 0 0 25 0 5 0 878052655 239407104 30515 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32200/tid=32209] ppid=32195 vsize=233796 CPUtime=4998.95 cores=4-7
/proc/32200/task/32209/stat : 32209 (plingeling) R 32195 32195 32106 0 -1 4202560 152681 1250 0 0 499873 22 0 0 25 0 5 0 878052655 239407104 30515 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=32200/tid=32210] ppid=32195 vsize=233796 CPUtime=5000.02 cores=4-7
/proc/32200/task/32210/stat : 32210 (plingeling) R 32195 32195 32106 0 -1 4202560 164909 1250 0 0 499976 26 0 0 25 0 5 0 878052655 239407104 30515 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331624 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39972.2
Current children cumulated vsize (KiB) 3740852

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5000 s]
/proc/loadavg: 8.13 8.10 8.07 9/184 32355
/proc/meminfo: memFree=30916156/32951124 swapFree=67054536/67111528
[pid=32195] ppid=32193 vsize=18324 CPUtime=0 cores=0-7
/proc/32195/stat : 32195 (ppfolio) S 32193 32195 32106 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 878052647 18763776 390 33554432000 4194304 4287625 140734820794064 18446744073709551615 271550557711 0 0 4096 65536 17891253761221853183 0 0 17 6 0 0 0
/proc/32195/statm: 4581 390 341 23 0 91 0
[pid=32196] ppid=32195 vsize=2028940 CPUtime=5000.14 cores=0
/proc/32196/stat : 32196 (cryptominisat) R 32195 32195 32106 0 -1 4202496 1678199 0 0 0 499377 637 0 0 25 0 1 0 878052648 2077634560 71942 33554432000 4194304 4587354 140736981287696 18446744073709551615 4238692 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/32196/statm: 507235 71942 337 96 0 503797 0
[pid=32197] ppid=32195 vsize=301172 CPUtime=5000 cores=1
/proc/32197/stat : 32197 (clasp) R 32195 32195 32106 0 -1 4202496 70560 0 0 0 499980 20 0 0 25 0 1 0 878052648 308400128 70037 33554432000 4194304 4785740 140735319330864 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/32197/statm: 75293 70037 340 145 0 72338 0
[pid=32198] ppid=32195 vsize=462448 CPUtime=4999.98 cores=2
/proc/32198/stat : 32198 (TNM) R 32195 32195 32106 0 -1 4202496 2353 0 0 0 499995 3 0 0 25 0 1 0 878052648 473546752 2250 33554432000 134512640 134534564 4294942352 18446744073709551615 134528710 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/32198/statm: 115612 2250 95 6 0 114278 0
[pid=32199] ppid=32195 vsize=696172 CPUtime=5000.14 cores=3
/proc/32199/stat : 32199 (march_hi) R 32195 32195 32106 0 -1 4202496 120680 0 0 0 499937 77 0 0 25 0 1 0 878052648 712880128 115686 33554432000 4194304 4331988 140734801286160 18446744073709551615 4265456 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/32199/statm: 174043 115686 176 34 0 172485 0
[pid=32200] ppid=32195 vsize=233796 CPUtime=19971.9 cores=4-7
/proc/32200/stat : 32200 (plingeling) S 32195 32195 32106 0 -1 4202496 645008 1250 0 0 1997099 92 0 0 18 0 5 0 878052648 239407104 30515 33554432000 4194304 4356188 140735126655328 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/32200/statm: 58449 30515 162 40 0 56993 0
[pid=32200/tid=32207] ppid=32195 vsize=233796 CPUtime=5000.06 cores=4-7
/proc/32200/task/32207/stat : 32207 (plingeling) R 32195 32195 32106 0 -1 4202560 164849 1250 0 0 499985 21 0 0 25 0 5 0 878052655 239407104 30515 33554432000 4194304 4356188 140735126655328 18446744073709551615 4330976 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32200/tid=32208] ppid=32195 vsize=233796 CPUtime=4972.77 cores=4-7
/proc/32200/task/32208/stat : 32208 (plingeling) R 32195 32195 32106 0 -1 4202560 158887 1250 0 0 497257 20 0 0 25 0 5 0 878052655 239407104 30515 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32200/tid=32209] ppid=32195 vsize=233796 CPUtime=4998.95 cores=4-7
/proc/32200/task/32209/stat : 32209 (plingeling) R 32195 32195 32106 0 -1 4202560 152681 1250 0 0 499873 22 0 0 25 0 5 0 878052655 239407104 30515 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=32200/tid=32210] ppid=32195 vsize=233796 CPUtime=5000.02 cores=4-7
/proc/32200/task/32210/stat : 32210 (plingeling) R 32195 32195 32106 0 -1 4202560 164909 1250 0 0 499976 26 0 0 25 0 5 0 878052655 239407104 30515 33554432000 4194304 4356188 140735126655328 18446744073709551615 4331624 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39972.2
Current children cumulated vsize (KiB) 3740852

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 32195 and gives
#  childrusage.ru_utime.tv_sec=4999
#  childrusage.ru_utime.tv_usec=958890
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=34994
# CPU time returned by wait4() is 4999.99
# while last known CPU time is 39972.2
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.02
CPU time (s): 39972.2
CPU user time (s): 39963.9
CPU system time (s): 8.29
CPU usage (%): 799.44
Max. virtual memory (cumulated for all children) (KiB): 5293352

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4999.96
system time used= 0.034994
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2875
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 376
involuntary context switches= 212940

runsolver used 7.65584 second user time and 19.626 second system time

The end

Launcher Data 

Begin job on node133 at 2011-06-10 02:09:08
IDJOB=3219711
IDBENCH=82635
IDSOLVER=1638
FILE ID=node133/3219711-1307664547
RUNJOBID= node133-1307664283-32120
PBS_JOBID= 13549429
Free space on /tmp= 74172 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k5-r20-v2000-c40000-S61914820-087-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3219711-1307664547/watcher-3219711-1307664547 -o /tmp/evaluation-result-3219711-1307664547/solver-3219711-1307664547 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3219711-1307664547.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 1b9b43753c02a2c226adda648a956e92
RANDOM SEED=1365853189

node133.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6741.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      32077336 kB
Buffers:        206528 kB
Cached:         448460 kB
SwapCached:      10420 kB
Active:         366972 kB
Inactive:       307720 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      32077336 kB
SwapTotal:    67111528 kB
SwapFree:     67054536 kB
Dirty:           10896 kB
Writeback:           0 kB
AnonPages:       17404 kB
Mapped:          12564 kB
Slab:           132824 kB
PageTables:       4536 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   188004 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74160 MiB
End job on node133 at 2011-06-10 03:32:30