Trace number 3275186

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 40000.2 5003.62

General information on the benchmark

NameCRAFTED/rbsat/crafted/
forced/rbsat-v1560c133795gyes1.cnf
MD5SUM118632220c6f3388c45a2582e9a2ac0e
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark3144.24
Satisfiable
(Un)Satisfiability was proved
Number of variables1560
Number of clauses133795
Sum of the clauses size269030
Maximum clause length26
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2133735
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 560

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3275186-1304145421/watcher-3275186-1304145421 -o /tmp/evaluation-result-3275186-1304145421/solver-3275186-1304145421 -C 40000 -W 5100 -M 31000 ppfolio HOME/instance-3275186-1304145421.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.31 2.82 2.86 2/170 11523
/proc/meminfo: memFree=30095748/32950928 swapFree=67054472/67111528
[pid=11523] ppid=11521 vsize=18324 CPUtime=0 cores=0-7
/proc/11523/stat : 11523 (ppfolio) S 11521 11523 11372 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 604772759 18763776 390 33554432000 4194304 4287625 140733872346208 18446744073709551615 260905414159 0 0 4096 65536 3908006964721876991 0 0 17 6 0 0 0
/proc/11523/statm: 4581 390 341 23 0 91 0

[startup+0.046189 s]
/proc/loadavg: 5.31 2.82 2.86 2/170 11523
/proc/meminfo: memFree=30095748/32950928 swapFree=67054472/67111528
[pid=11523] ppid=11521 vsize=18324 CPUtime=0 cores=0-7
/proc/11523/stat : 11523 (ppfolio) S 11521 11523 11372 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 604772759 18763776 390 33554432000 4194304 4287625 140733872346208 18446744073709551615 260905414159 0 0 4096 65536 3908006964721876991 0 0 17 6 0 0 0
/proc/11523/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100174 s]
/proc/loadavg: 5.31 2.82 2.86 2/170 11523
/proc/meminfo: memFree=30095748/32950928 swapFree=67054472/67111528
[pid=11523] ppid=11521 vsize=18324 CPUtime=0 cores=0-7
/proc/11523/stat : 11523 (ppfolio) S 11521 11523 11372 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 604772759 18763776 390 33554432000 4194304 4287625 140733872346208 18446744073709551615 260905414159 0 0 4096 65536 3908006964721876991 0 0 17 6 0 0 0
/proc/11523/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300141 s]
/proc/loadavg: 5.31 2.82 2.86 2/170 11523
/proc/meminfo: memFree=30095748/32950928 swapFree=67054472/67111528
[pid=11523] ppid=11521 vsize=18324 CPUtime=0 cores=0-7
/proc/11523/stat : 11523 (ppfolio) R 11521 11523 11372 0 -1 4202496 522 0 0 0 0 0 0 0 23 0 1 0 604772759 18763776 390 33554432000 4194304 4287625 140733872346208 18446744073709551615 260905414159 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/11523/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.701064 s]
/proc/loadavg: 5.31 2.82 2.86 2/170 11523
/proc/meminfo: memFree=30095748/32950928 swapFree=67054472/67111528
[pid=11523] ppid=11521 vsize=18324 CPUtime=0 cores=0-7
/proc/11523/stat : 11523 (ppfolio) R 11521 11523 11372 0 -1 4202496 522 0 0 0 0 0 0 0 23 0 1 0 604772759 18763776 390 33554432000 4194304 4287625 140733872346208 18446744073709551615 260905414159 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/11523/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50092 s]
/proc/loadavg: 5.53 2.91 2.89 9/180 11538
/proc/meminfo: memFree=30008644/32950928 swapFree=67054472/67111528
[pid=11523] ppid=11521 vsize=18324 CPUtime=0 cores=0-7
/proc/11523/stat : 11523 (ppfolio) S 11521 11523 11372 0 -1 4202496 522 0 0 0 0 0 0 0 17 0 1 0 604772759 18763776 390 33554432000 4194304 4287625 140733872346208 18446744073709551615 260905414159 0 0 4096 65536 3908006964721876991 0 0 17 6 0 0 0
/proc/11523/statm: 4581 390 341 23 0 91 0
[pid=11524] ppid=11523 vsize=45576 CPUtime=1.49 cores=0
/proc/11524/stat : 11524 (cryptominisat) R 11523 11523 11372 0 -1 4202496 9299 0 0 0 148 1 0 0 25 0 1 0 604772759 46669824 6623 33554432000 4194304 4587354 140733927697376 18446744073709551615 260905090712 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11524/statm: 11394 6623 326 96 0 7956 0
[pid=11525] ppid=11523 vsize=21480 CPUtime=1.49 cores=1
/proc/11525/stat : 11525 (clasp) R 11523 11523 11372 0 -1 4202496 3565 0 0 0 149 0 0 0 20 0 1 0 604772759 21995520 2707 33554432000 4194304 4785740 140734842320368 18446744073709551615 4287042 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11525/statm: 5370 2707 340 145 0 2415 0
[pid=11526] ppid=11523 vsize=463116 CPUtime=1.48 cores=2
/proc/11526/stat : 11526 (TNM) R 11523 11523 11372 0 -1 4202496 3345 0 0 0 148 0 0 0 20 0 1 0 604772759 474230784 3241 33554432000 134512640 134534564 4287966448 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11526/statm: 115779 3241 95 6 0 114445 0
[pid=11527] ppid=11523 vsize=26180 CPUtime=1.49 cores=3
/proc/11527/stat : 11527 (march_hi) R 11523 11523 11372 0 -1 4202496 11065 0 0 0 148 1 0 0 20 0 1 0 604772759 26808320 2730 33554432000 4194304 4331988 140735002353808 18446744073709551615 4237154 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11527/statm: 6545 2730 174 34 0 4987 0
[pid=11528] ppid=11523 vsize=144164 CPUtime=5.7 cores=4-7
/proc/11528/stat : 11528 (plingeling) S 11523 11523 11372 0 -1 4202496 10278 1249 0 0 569 1 0 0 20 0 5 0 604772759 147623936 8101 33554432000 4194304 4356188 140735876926288 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11528/statm: 36041 8101 163 40 0 34585 0
[pid=11528/tid=11535] ppid=11523 vsize=144164 CPUtime=1.4 cores=4-7
/proc/11528/task/11535/stat : 11535 (plingeling) R 11523 11523 11372 0 -1 4202560 1342 1249 0 0 140 0 0 0 23 0 5 0 604772768 147623936 8101 33554432000 4194304 4356188 140735876926288 18446744073709551615 4271040 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11528/tid=11536] ppid=11523 vsize=144164 CPUtime=1.4 cores=4-7
/proc/11528/task/11536/stat : 11536 (plingeling) R 11523 11523 11372 0 -1 4202560 710 1249 0 0 140 0 0 0 23 0 5 0 604772768 147623936 8101 33554432000 4194304 4356188 140735876926288 18446744073709551615 4270848 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11528/tid=11537] ppid=11523 vsize=144164 CPUtime=1.39 cores=4-7
/proc/11528/task/11537/stat : 11537 (plingeling) R 11523 11523 11372 0 -1 4202560 1434 1249 0 0 139 0 0 0 25 0 5 0 604772768 147623936 8101 33554432000 4194304 4356188 140735876926288 18446744073709551615 4211540 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11528/tid=11538] ppid=11523 vsize=144164 CPUtime=1.4 cores=4-7
/proc/11528/task/11538/stat : 11538 (plingeling) R 11523 11523 11372 0 -1 4202560 2243 1249 0 0 140 0 0 0 20 0 5 0 604772768 147623936 8101 33554432000 4194304 4356188 140735876926288 18446744073709551615 4330827 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.65
Current children cumulated vsize (KiB) 718840

[startup+3.10063 s]
/proc/loadavg: 5.53 2.91 2.89 9/180 11538
/proc/meminfo: memFree=29999220/32950928 swapFree=67054472/67111528
[pid=11523] ppid=11521 vsize=18324 CPUtime=0 cores=0-7
/proc/11523/stat : 11523 (ppfolio) S 11521 11523 11372 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 604772759 18763776 390 33554432000 4194304 4287625 140733872346208 18446744073709551615 260905414159 0 0 4096 65536 3908006964721876991 0 0 17 6 0 0 0
/proc/11523/statm: 4581 390 341 23 0 91 0
[pid=11524] ppid=11523 vsize=1004720 CPUtime=3.09 cores=0
/proc/11524/stat : 11524 (cryptominisat) R 11523 11523 11372 0 -1 4202496 10611 0 0 0 308 1 0 0 25 0 1 0 604772759 1028833280 6780 33554432000 4194304 4587354 140733927697376 18446744073709551615 4365232 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11524/statm: 251180 6780 337 96 0 247742 0
[pid=11525] ppid=11523 vsize=28036 CPUtime=3.08 cores=1
/proc/11525/stat : 11525 (clasp) R 11523 11523 11372 0 -1 4202496 5223 0 0 0 308 0 0 0 25 0 1 0 604772759 28708864 4365 33554432000 4194304 4785740 140734842320368 18446744073709551615 4287053 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11525/statm: 7009 4365 340 145 0 4054 0
[pid=11526] ppid=11523 vsize=463116 CPUtime=3.08 cores=2
/proc/11526/stat : 11526 (TNM) R 11523 11523 11372 0 -1 4202496 3345 0 0 0 308 0 0 0 25 0 1 0 604772759 474230784 3241 33554432000 134512640 134534564 4287966448 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11526/statm: 115779 3241 95 6 0 114445 0
[pid=11527] ppid=11523 vsize=26500 CPUtime=3.09 cores=3
/proc/11527/stat : 11527 (march_hi) R 11523 11523 11372 0 -1 4202496 11132 0 0 0 308 1 0 0 25 0 1 0 604772759 27136000 2796 33554432000 4194304 4331988 140735002353808 18446744073709551615 4239560 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11527/statm: 6625 2796 174 34 0 5067 0
[pid=11528] ppid=11523 vsize=160804 CPUtime=12.09 cores=4-7
/proc/11528/stat : 11528 (plingeling) S 11523 11523 11372 0 -1 4202496 14183 1249 0 0 1207 2 0 0 20 0 5 0 604772759 164663296 12006 33554432000 4194304 4356188 140735876926288 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11528/statm: 40201 12006 163 40 0 38745 0
[pid=11528/tid=11535] ppid=11523 vsize=160804 CPUtime=3 cores=4-7
/proc/11528/task/11535/stat : 11535 (plingeling) R 11523 11523 11372 0 -1 4202560 1982 1249 0 0 300 0 0 0 25 0 5 0 604772768 164663296 12006 33554432000 4194304 4356188 140735876926288 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11528/tid=11536] ppid=11523 vsize=160804 CPUtime=3 cores=4-7
/proc/11528/task/11536/stat : 11536 (plingeling) R 11523 11523 11372 0 -1 4202560 2247 1249 0 0 300 0 0 0 25 0 5 0 604772768 164663296 12006 33554432000 4194304 4356188 140735876926288 18446744073709551615 4331992 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11528/tid=11537] ppid=11523 vsize=160804 CPUtime=2.99 cores=4-7
/proc/11528/task/11537/stat : 11537 (plingeling) R 11523 11523 11372 0 -1 4202560 2522 1249 0 0 299 0 0 0 25 0 5 0 604772768 164663296 12006 33554432000 4194304 4356188 140735876926288 18446744073709551615 4331510 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11528/tid=11538] ppid=11523 vsize=160804 CPUtime=3 cores=4-7
/proc/11528/task/11538/stat : 11538 (plingeling) R 11523 11523 11372 0 -1 4202560 2883 1249 0 0 300 0 0 0 25 0 5 0 604772768 164663296 12006 33554432000 4194304 4356188 140735876926288 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.43
Current children cumulated vsize (KiB) 1701500

[startup+6.30104 s]
/proc/loadavg: 5.53 2.91 2.89 9/180 11538
/proc/meminfo: memFree=29946280/32950928 swapFree=67054472/67111528
[pid=11523] ppid=11521 vsize=18324 CPUtime=0 cores=0-7
/proc/11523/stat : 11523 (ppfolio) S 11521 11523 11372 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 604772759 18763776 390 33554432000 4194304 4287625 140733872346208 18446744073709551615 260905414159 0 0 4096 65536 3908006964721876991 0 0 17 6 0 0 0
/proc/11523/statm: 4581 390 341 23 0 91 0
[pid=11524] ppid=11523 vsize=1030800 CPUtime=6.29 cores=0
/proc/11524/stat : 11524 (cryptominisat) R 11523 11523 11372 0 -1 4202496 16830 0 0 0 628 1 0 0 25 0 1 0 604772759 1055539200 12998 33554432000 4194304 4587354 140733927697376 18446744073709551615 4238597 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11524/statm: 257700 12998 337 96 0 254262 0
[pid=11525] ppid=11523 vsize=39048 CPUtime=6.29 cores=1
/proc/11525/stat : 11525 (clasp) R 11523 11523 11372 0 -1 4202496 7921 0 0 0 628 1 0 0 25 0 1 0 604772759 39985152 7063 33554432000 4194304 4785740 140734842320368 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11525/statm: 9762 7063 340 145 0 6807 0
[pid=11526] ppid=11523 vsize=463116 CPUtime=6.28 cores=2
/proc/11526/stat : 11526 (TNM) R 11523 11523 11372 0 -1 4202496 3345 0 0 0 628 0 0 0 25 0 1 0 604772759 474230784 3241 33554432000 134512640 134534564 4287966448 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11526/statm: 115779 3241 95 6 0 114445 0
[pid=11527] ppid=11523 vsize=26848 CPUtime=6.29 cores=3
/proc/11527/stat : 11527 (march_hi) R 11523 11523 11372 0 -1 4202496 11220 0 0 0 628 1 0 0 25 0 1 0 604772759 27492352 2884 33554432000 4194304 4331988 140735002353808 18446744073709551615 4226217 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11527/statm: 6712 2884 174 34 0 5154 0
[pid=11528] ppid=11523 vsize=163876 CPUtime=24.87 cores=4-7
/proc/11528/stat : 11528 (plingeling) S 11523 11523 11372 0 -1 4202496 15206 1249 0 0 2485 2 0 0 20 0 5 0 604772759 167809024 13029 33554432000 4194304 4356188 140735876926288 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11528/statm: 40969 13029 163 40 0 39513 0
[pid=11528/tid=11535] ppid=11523 vsize=163876 CPUtime=6.2 cores=4-7
/proc/11528/task/11535/stat : 11535 (plingeling) R 11523 11523 11372 0 -1 4202560 2365 1249 0 0 620 0 0 0 25 0 5 0 604772768 167809024 13029 33554432000 4194304 4356188 140735876926288 18446744073709551615 4330992 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11528/tid=11536] ppid=11523 vsize=163876 CPUtime=6.2 cores=4-7
/proc/11528/task/11536/stat : 11536 (plingeling) R 11523 11523 11372 0 -1 4202560 2503 1249 0 0 620 0 0 0 25 0 5 0 604772768 167809024 13029 33554432000 4194304 4356188 140735876926288 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11528/tid=11537] ppid=11523 vsize=163876 CPUtime=6.17 cores=4-7
/proc/11528/task/11537/stat : 11537 (plingeling) R 11523 11523 11372 0 -1 4202560 2778 1249 0 0 617 0 0 0 25 0 5 0 604772768 167809024 13029 33554432000 4194304 4356188 140735876926288 18446744073709551615 4234297 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11528/tid=11538] ppid=11523 vsize=163876 CPUtime=6.2 cores=4-7
/proc/11528/task/11538/stat : 11538 (plingeling) R 11523 11523 11372 0 -1 4202560 3011 1249 0 0 620 0 0 0 25 0 5 0 604772768 167809024 13029 33554432000 4194304 4356188 140735876926288 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.02
Current children cumulated vsize (KiB) 1742012

[startup+12.7009 s]
/proc/loadavg: 5.91 3.08 2.95 9/180 11538
/proc/meminfo: memFree=29924048/32950928 swapFree=67054472/67111528
[pid=11523] ppid=11521 vsize=18324 CPUtime=0 cores=0-7
/proc/11523/stat : 11523 (ppfolio) S 11521 11523 11372 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 604772759 18763776 390 33554432000 4194304 4287625 140733872346208 18446744073709551615 260905414159 0 0 4096 65536 3908006964721876991 0 0 17 6 0 0 0
/proc/11523/statm: 4581 390 341 23 0 91 0
[pid=11524] ppid=11523 vsize=1049104 CPUtime=12.69 cores=0
/proc/11524/stat : 11524 (cryptominisat) R 11523 11523 11372 0 -1 4202496 21280 0 0 0 1267 2 0 0 25 0 1 0 604772759 1074282496 9830 33554432000 4194304 4587354 140733927697376 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11524/statm: 262276 9830 337 96 0 258838 0
[pid=11525] ppid=11523 vsize=43780 CPUtime=12.69 cores=1
/proc/11525/stat : 11525 (clasp) R 11523 11523 11372 0 -1 4202496 9115 0 0 0 1268 1 0 0 25 0 1 0 604772759 44830720 8257 33554432000 4194304 4785740 140734842320368 18446744073709551615 4287017 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11525/statm: 10945 8257 340 145 0 7990 0
[pid=11526] ppid=11523 vsize=463116 CPUtime=12.68 cores=2
/proc/11526/stat : 11526 (TNM) R 11523 11523 11372 0 -1 4202496 3345 0 0 0 1268 0 0 0 25 0 1 0 604772759 474230784 3241 33554432000 134512640 134534564 4287966448 18446744073709551615 134528295 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11526/statm: 115779 3241 95 6 0 114445 0
[pid=11527] ppid=11523 vsize=27652 CPUtime=12.69 cores=3
/proc/11527/stat : 11527 (march_hi) R 11523 11523 11372 0 -1 4202496 11355 0 0 0 1268 1 0 0 25 0 1 0 604772759 28315648 3019 33554432000 4194304 4331988 140735002353808 18446744073709551615 4226496 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11527/statm: 6913 3019 174 34 0 5355 0
[pid=11528] ppid=11523 vsize=175140 CPUtime=50.44 cores=4-7
/proc/11528/stat : 11528 (plingeling) S 11523 11523 11372 0 -1 4202496 17767 1249 0 0 5041 3 0 0 20 0 5 0 604772759 179343360 15590 33554432000 4194304 4356188 140735876926288 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11528/statm: 43785 15590 163 40 0 42329 0
[pid=11528/tid=11535] ppid=11523 vsize=175140 CPUtime=12.6 cores=4-7
/proc/11528/task/11535/stat : 11535 (plingeling) R 11523 11523 11372 0 -1 4202560 2750 1249 0 0 1260 0 0 0 25 0 5 0 604772768 179343360 15590 33554432000 4194304 4356188 140735876926288 18446744073709551615 4332023 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11528/tid=11536] ppid=11523 vsize=175140 CPUtime=12.59 cores=4-7
/proc/11528/task/11536/stat : 11536 (plingeling) R 11523 11523 11372 0 -1 4202560 3655 1249 0 0 1259 0 0 0 25 0 5 0 604772768 179343360 15590 33554432000 4194304 4356188 140735876926288 18446744073709551615 4244419 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11528/tid=11537] ppid=11523 vsize=175140 CPUtime=12.53 cores=4-7
/proc/11528/task/11537/stat : 11537 (plingeling) R 11523 11523 11372 0 -1 4202560 3161 1249 0 0 1253 0 0 0 25 0 5 0 604772768 179343360 15590 33554432000 4194304 4356188 140735876926288 18446744073709551615 4330856 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11528/tid=11538] ppid=11523 vsize=175140 CPUtime=12.6 cores=4-7
/proc/11528/task/11538/stat : 11538 (plingeling) R 11523 11523 11372 0 -1 4202560 3652 1249 0 0 1260 0 0 0 25 0 5 0 604772768 179343360 15590 33554432000 4194304 4356188 140735876926288 18446744073709551615 4331964 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.19
Current children cumulated vsize (KiB) 1777116

[startup+25.5005 s]
/proc/loadavg: 6.23 3.24 3.00 9/180 11539
/proc/meminfo: memFree=29874880/32950928 swapFree=67054472/67111528
[pid=11523] ppid=11521 vsize=18324 CPUtime=0 cores=0-7
/proc/11523/stat : 11523 (ppfolio) S 11521 11523 11372 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 604772759 18763776 390 33554432000 4194304 4287625 140733872346208 18446744073709551615 260905414159 0 0 4096 65536 3908006964721876991 0 0 17 6 0 0 0
/proc/11523/statm: 4581 390 341 23 0 91 0
[pid=11524] ppid=11523 vsize=1080216 CPUtime=25.49 cores=0
/proc/11524/stat : 11524 (cryptominisat) R 11523 11523 11372 0 -1 4202496 34915 0 0 0 2545 4 0 0 25 0 1 0 604772759 1106141184 21249 33554432000 4194304 4587354 140733927697376 18446744073709551615 4230209 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=11526] ppid=11523 vsize=463116 CPUtime=4781.67 cores=2
/proc/11526/stat : 11526 (TNM) R 11523 11523 11372 0 -1 4202496 3345 0 0 0 478165 2 0 0 25 0 1 0 604772759 474230784 3241 33554432000 134512640 134534564 4287966448 18446744073709551615 134530360 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11526/statm: 115779 3241 95 6 0 114445 0
[pid=11527] ppid=11523 vsize=483144 CPUtime=4782.44 cores=3
/proc/11527/stat : 11527 (march_hi) R 11523 11523 11372 0 -1 4202496 103447 0 0 0 478175 69 0 0 25 0 1 0 604772759 494739456 95111 33554432000 4194304 4331988 140735002353808 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11527/statm: 120786 95111 175 34 0 119228 0
[pid=11528] ppid=11523 vsize=527772 CPUtime=19102 cores=4-7
/proc/11528/stat : 11528 (plingeling) S 11523 11523 11372 0 -1 4202496 3864452 1249 0 0 1909676 525 0 0 20 0 5 0 604772759 540438528 104035 33554432000 4194304 4356188 140735876926288 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11528/statm: 131943 104035 163 40 0 130487 0
[pid=11528/tid=11535] ppid=11523 vsize=527772 CPUtime=4781.6 cores=4-7
/proc/11528/task/11535/stat : 11535 (plingeling) R 11523 11523 11372 0 -1 4202560 1059910 1249 0 0 478012 148 0 0 25 0 5 0 604772768 540438528 104035 33554432000 4194304 4356188 140735876926288 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11528/tid=11536] ppid=11523 vsize=527772 CPUtime=4782.35 cores=4-7
/proc/11528/task/11536/stat : 11536 (plingeling) R 11523 11523 11372 0 -1 4202560 993614 1249 0 0 478100 135 0 0 25 0 5 0 604772768 540438528 104035 33554432000 4194304 4356188 140735876926288 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11528/tid=11537] ppid=11523 vsize=527772 CPUtime=4756.27 cores=4-7
/proc/11528/task/11537/stat : 11537 (plingeling) R 11523 11523 11372 0 -1 4202560 1009120 1249 0 0 475498 129 0 0 25 0 5 0 604772768 540438528 104035 33554432000 4194304 4356188 140735876926288 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11528/tid=11538] ppid=11523 vsize=527772 CPUtime=4781.67 cores=4-7
/proc/11528/task/11538/stat : 11538 (plingeling) R 11523 11523 11372 0 -1 4202560 797259 1249 0 0 478057 110 0 0 25 0 5 0 604772768 540438528 104035 33554432000 4194304 4356188 140735876926288 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38231
Current children cumulated vsize (KiB) 3055008

[startup+4842.3 s]
/proc/loadavg: 8.09 8.07 8.00 9/180 11682
/proc/meminfo: memFree=28743412/32950928 swapFree=67054472/67111528
[pid=11523] ppid=11521 vsize=18324 CPUtime=0 cores=0-7
/proc/11523/stat : 11523 (ppfolio) S 11521 11523 11372 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 604772759 18763776 390 33554432000 4194304 4287625 140733872346208 18446744073709551615 260905414159 0 0 4096 65536 3908006964721876991 0 0 17 6 0 0 0
/proc/11523/statm: 4581 390 341 23 0 91 0
[pid=11524] ppid=11523 vsize=1342380 CPUtime=4842.42 cores=0
/proc/11524/stat : 11524 (cryptominisat) R 11523 11523 11372 0 -1 4202496 1077206 0 0 0 483767 475 0 0 25 0 1 0 604772759 1374597120 80523 33554432000 4194304 4587354 140733927697376 18446744073709551615 4238392 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11524/statm: 335595 80523 337 96 0 332157 0
[pid=11525] ppid=11523 vsize=266668 CPUtime=4842.42 cores=1
/proc/11525/stat : 11525 (clasp) R 11523 11523 11372 0 -1 4202496 60719 0 0 0 484228 14 0 0 25 0 1 0 604772759 273068032 59861 33554432000 4194304 4785740 140734842320368 18446744073709551615 4287017 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11525/statm: 66667 59861 340 145 0 63712 0
[pid=11526] ppid=11523 vsize=463116 CPUtime=4841.66 cores=2
/proc/11526/stat : 11526 (TNM) R 11523 11523 11372 0 -1 4202496 3345 0 0 0 484164 2 0 0 25 0 1 0 604772759 474230784 3241 33554432000 134512640 134534564 4287966448 18446744073709551615 134528687 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11526/statm: 115779 3241 95 6 0 114445 0
[pid=11527] ppid=11523 vsize=483144 CPUtime=4842.44 cores=3
/proc/11527/stat : 11527 (march_hi) R 11523 11523 11372 0 -1 4202496 104754 0 0 0 484174 70 0 0 25 0 1 0 604772759 494739456 96418 33554432000 4194304 4331988 140735002353808 18446744073709551615 4237171 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11527/statm: 120786 96418 175 34 0 119228 0
[pid=11528] ppid=11523 vsize=509212 CPUtime=19341.7 cores=4-7
/proc/11528/stat : 11528 (plingeling) S 11523 11523 11372 0 -1 4202496 3905956 1249 0 0 1933638 529 0 0 20 0 5 0 604772759 521433088 99395 33554432000 4194304 4356188 140735876926288 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11528/statm: 127303 99395 163 40 0 125847 0
[pid=11528/tid=11535] ppid=11523 vsize=509212 CPUtime=4841.59 cores=4-7
/proc/11528/task/11535/stat : 11535 (plingeling) R 11523 11523 11372 0 -1 4202560 1070694 1249 0 0 484010 149 0 0 25 0 5 0 604772768 521433088 99395 33554432000 4194304 4356188 140735876926288 18446744073709551615 4280579 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11528/tid=11536] ppid=11523 vsize=509212 CPUtime=4842.35 cores=4-7
/proc/11528/task/11536/stat : 11536 (plingeling) R 11523 11523 11372 0 -1 4202560 1001806 1249 0 0 484099 136 0 0 25 0 5 0 604772768 521433088 99395 33554432000 4194304 4356188 140735876926288 18446744073709551615 4330964 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11528/tid=11537] ppid=11523 vsize=509212 CPUtime=4815.94 cores=4-7
/proc/11528/task/11537/stat : 11537 (plingeling) R 11523 11523 11372 0 -1 4202560 1023456 1249 0 0 481463 131 0 0 25 0 5 0 604772768 521433088 99395 33554432000 4194304 4356188 140735876926288 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11528/tid=11538] ppid=11523 vsize=509212 CPUtime=4841.68 cores=4-7
/proc/11528/task/11538/stat : 11538 (plingeling) R 11523 11523 11372 0 -1 4202560 805451 1249 0 0 484057 111 0 0 25 0 5 0 604772768 521433088 99395 33554432000 4194304 4356188 140735876926288 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38710.6
Current children cumulated vsize (KiB) 3082844

[startup+4902.3 s]
/proc/loadavg: 8.03 8.06 8.00 9/180 11684
/proc/meminfo: memFree=28692896/32950928 swapFree=67054472/67111528
[pid=11523] ppid=11521 vsize=18324 CPUtime=0 cores=0-7
/proc/11523/stat : 11523 (ppfolio) S 11521 11523 11372 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 604772759 18763776 390 33554432000 4194304 4287625 140733872346208 18446744073709551615 260905414159 0 0 4096 65536 3908006964721876991 0 0 17 6 0 0 0
/proc/11523/statm: 4581 390 341 23 0 91 0
[pid=11524] ppid=11523 vsize=1297472 CPUtime=4902.43 cores=0
/proc/11524/stat : 11524 (cryptominisat) R 11523 11523 11372 0 -1 4202496 1088661 0 0 0 489762 481 0 0 25 0 1 0 604772759 1328611328 83614 33554432000 4194304 4587354 140733927697376 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11524/statm: 324368 83614 337 96 0 320930 0
[pid=11525] ppid=11523 vsize=267096 CPUtime=4902.42 cores=1
/proc/11525/stat : 11525 (clasp) R 11523 11523 11372 0 -1 4202496 60814 0 0 0 490228 14 0 0 25 0 1 0 604772759 273506304 59956 33554432000 4194304 4785740 140734842320368 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11525/statm: 66774 59956 340 145 0 63819 0
[pid=11526] ppid=11523 vsize=463116 CPUtime=4901.66 cores=2
/proc/11526/stat : 11526 (TNM) R 11523 11523 11372 0 -1 4202496 3345 0 0 0 490164 2 0 0 25 0 1 0 604772759 474230784 3241 33554432000 134512640 134534564 4287966448 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11526/statm: 115779 3241 95 6 0 114445 0
[pid=11527] ppid=11523 vsize=483144 CPUtime=4902.43 cores=3
/proc/11527/stat : 11527 (march_hi) R 11523 11523 11372 0 -1 4202496 106497 0 0 0 490172 71 0 0 25 0 1 0 604772759 494739456 98161 33554432000 4194304 4331988 140735002353808 18446744073709551615 4210459 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11527/statm: 120786 98161 175 34 0 119228 0
[pid=11528] ppid=11523 vsize=541980 CPUtime=19581.4 cores=4-7
/proc/11528/stat : 11528 (plingeling) S 11523 11523 11372 0 -1 4202496 3956164 1249 0 0 1957599 537 0 0 20 0 5 0 604772759 554987520 107587 33554432000 4194304 4356188 140735876926288 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11528/statm: 135495 107587 163 40 0 134039 0
[pid=11528/tid=11535] ppid=11523 vsize=541980 CPUtime=4901.59 cores=4-7
/proc/11528/task/11535/stat : 11535 (plingeling) R 11523 11523 11372 0 -1 4202560 1079942 1249 0 0 490008 151 0 0 25 0 5 0 604772768 554987520 107587 33554432000 4194304 4356188 140735876926288 18446744073709551615 4330999 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11528/tid=11536] ppid=11523 vsize=541980 CPUtime=4902.35 cores=4-7
/proc/11528/task/11536/stat : 11536 (plingeling) R 11523 11523 11372 0 -1 4202560 1015630 1249 0 0 490096 139 0 0 25 0 5 0 604772768 554987520 107587 33554432000 4194304 4356188 140735876926288 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11528/tid=11537] ppid=11523 vsize=541980 CPUtime=4875.62 cores=4-7
/proc/11528/task/11537/stat : 11537 (plingeling) R 11523 11523 11372 0 -1 4202560 1044960 1249 0 0 487429 133 0 0 25 0 5 0 604772768 554987520 107587 33554432000 4194304 4356188 140735876926288 18446744073709551615 4331268 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11528/tid=11538] ppid=11523 vsize=541980 CPUtime=4901.68 cores=4-7
/proc/11528/task/11538/stat : 11538 (plingeling) R 11523 11523 11372 0 -1 4202560 811083 1249 0 0 490056 112 0 0 25 0 5 0 604772768 554987520 107587 33554432000 4194304 4356188 140735876926288 18446744073709551615 4279985 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39190.3
Current children cumulated vsize (KiB) 3071132

[startup+4962.3 s]
/proc/loadavg: 8.01 8.04 8.00 9/180 11685
/proc/meminfo: memFree=28673872/32950928 swapFree=67054472/67111528
[pid=11523] ppid=11521 vsize=18324 CPUtime=0 cores=0-7
/proc/11523/stat : 11523 (ppfolio) S 11521 11523 11372 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 604772759 18763776 390 33554432000 4194304 4287625 140733872346208 18446744073709551615 260905414159 0 0 4096 65536 3908006964721876991 0 0 17 6 0 0 0
/proc/11523/statm: 4581 390 341 23 0 91 0
[pid=11524] ppid=11523 vsize=1296788 CPUtime=4962.44 cores=0
/proc/11524/stat : 11524 (cryptominisat) R 11523 11523 11372 0 -1 4202496 1101064 0 0 0 495757 487 0 0 25 0 1 0 604772759 1327910912 87630 33554432000 4194304 4587354 140733927697376 18446744073709551615 260905090705 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11524/statm: 324197 87630 337 96 0 320759 0
[pid=11525] ppid=11523 vsize=267520 CPUtime=4962.42 cores=1
/proc/11525/stat : 11525 (clasp) R 11523 11523 11372 0 -1 4202496 60921 0 0 0 496228 14 0 0 25 0 1 0 604772759 273940480 60063 33554432000 4194304 4785740 140734842320368 18446744073709551615 4553380 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11525/statm: 66880 60063 340 145 0 63925 0
[pid=11526] ppid=11523 vsize=463116 CPUtime=4961.65 cores=2
/proc/11526/stat : 11526 (TNM) R 11523 11523 11372 0 -1 4202496 3345 0 0 0 496163 2 0 0 25 0 1 0 604772759 474230784 3241 33554432000 134512640 134534564 4287966448 18446744073709551615 134530598 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11526/statm: 115779 3241 95 6 0 114445 0
[pid=11527] ppid=11523 vsize=483144 CPUtime=4962.44 cores=3
/proc/11527/stat : 11527 (march_hi) R 11523 11523 11372 0 -1 4202496 107421 0 0 0 496172 72 0 0 25 0 1 0 604772759 494739456 99085 33554432000 4194304 4331988 140735002353808 18446744073709551615 4228224 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11527/statm: 120786 99085 175 34 0 119228 0
[pid=11528] ppid=11523 vsize=544156 CPUtime=19821 cores=4-7
/proc/11528/stat : 11528 (plingeling) S 11523 11523 11372 0 -1 4202496 4012036 1249 0 0 1981560 543 0 0 20 0 5 0 604772759 557215744 108131 33554432000 4194304 4356188 140735876926288 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11528/statm: 136039 108131 163 40 0 134583 0
[pid=11528/tid=11535] ppid=11523 vsize=544156 CPUtime=4961.59 cores=4-7
/proc/11528/task/11535/stat : 11535 (plingeling) R 11523 11523 11372 0 -1 4202560 1096390 1249 0 0 496006 153 0 0 25 0 5 0 604772768 557215744 108131 33554432000 4194304 4356188 140735876926288 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11528/tid=11536] ppid=11523 vsize=544156 CPUtime=4962.35 cores=4-7
/proc/11528/task/11536/stat : 11536 (plingeling) R 11523 11523 11372 0 -1 4202560 1025358 1249 0 0 496096 139 0 0 25 0 5 0 604772768 557215744 108131 33554432000 4194304 4356188 140735876926288 18446744073709551615 4210992 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11528/tid=11537] ppid=11523 vsize=544156 CPUtime=4935.29 cores=4-7
/proc/11528/task/11537/stat : 11537 (plingeling) R 11523 11523 11372 0 -1 4202560 1058272 1249 0 0 493395 134 0 0 25 0 5 0 604772768 557215744 108131 33554432000 4194304 4356188 140735876926288 18446744073709551615 4331106 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11528/tid=11538] ppid=11523 vsize=544156 CPUtime=4961.68 cores=4-7
/proc/11528/task/11538/stat : 11538 (plingeling) R 11523 11523 11372 0 -1 4202560 827467 1249 0 0 496054 114 0 0 25 0 5 0 604772768 557215744 108131 33554432000 4194304 4356188 140735876926288 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39670
Current children cumulated vsize (KiB) 3073048



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5003.6 s]
/proc/loadavg: 8.00 8.04 8.00 9/180 11686
/proc/meminfo: memFree=28630448/32950928 swapFree=67054472/67111528
[pid=11523] ppid=11521 vsize=18324 CPUtime=0 cores=0-7
/proc/11523/stat : 11523 (ppfolio) S 11521 11523 11372 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 604772759 18763776 390 33554432000 4194304 4287625 140733872346208 18446744073709551615 260905414159 0 0 4096 65536 3908006964721876991 0 0 17 6 0 0 0
/proc/11523/statm: 4581 390 341 23 0 91 0
[pid=11524] ppid=11523 vsize=1298044 CPUtime=5003.74 cores=0
/proc/11524/stat : 11524 (cryptominisat) R 11523 11523 11372 0 -1 4202496 1108532 0 0 0 499883 491 0 0 25 0 1 0 604772759 1329197056 86723 33554432000 4194304 4587354 140733927697376 18446744073709551615 4238525 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11524/statm: 324511 86723 337 96 0 321073 0
[pid=11525] ppid=11523 vsize=269228 CPUtime=5003.72 cores=1
/proc/11525/stat : 11525 (clasp) R 11523 11523 11372 0 -1 4202496 61257 0 0 0 500358 14 0 0 25 0 1 0 604772759 275689472 60399 33554432000 4194304 4785740 140734842320368 18446744073709551615 4553344 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11525/statm: 67307 60399 340 145 0 64352 0
[pid=11526] ppid=11523 vsize=463116 CPUtime=5002.96 cores=2
/proc/11526/stat : 11526 (TNM) R 11523 11523 11372 0 -1 4202496 3345 0 0 0 500294 2 0 0 25 0 1 0 604772759 474230784 3241 33554432000 134512640 134534564 4287966448 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11526/statm: 115779 3241 95 6 0 114445 0
[pid=11527] ppid=11523 vsize=483144 CPUtime=5003.75 cores=3
/proc/11527/stat : 11527 (march_hi) R 11523 11523 11372 0 -1 4202496 108029 0 0 0 500302 73 0 0 25 0 1 0 604772759 494739456 99693 33554432000 4194304 4331988 140735002353808 18446744073709551615 4246947 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11527/statm: 120786 99693 175 34 0 119228 0
[pid=11528] ppid=11523 vsize=578972 CPUtime=19986 cores=4-7
/proc/11528/stat : 11528 (plingeling) S 11523 11523 11372 0 -1 4202496 4039204 1249 0 0 1998056 547 0 0 20 0 5 0 604772759 592867328 116835 33554432000 4194304 4356188 140735876926288 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11528/statm: 144743 116835 163 40 0 143287 0
[pid=11528/tid=11535] ppid=11523 vsize=578972 CPUtime=5002.9 cores=4-7
/proc/11528/task/11535/stat : 11535 (plingeling) R 11523 11523 11372 0 -1 4202560 1104614 1249 0 0 500135 155 0 0 25 0 5 0 604772768 592867328 116835 33554432000 4194304 4356188 140735876926288 18446744073709551615 4282035 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11528/tid=11536] ppid=11523 vsize=578972 CPUtime=5003.65 cores=4-7
/proc/11528/task/11536/stat : 11536 (plingeling) R 11523 11523 11372 0 -1 4202560 1025358 1249 0 0 500226 139 0 0 25 0 5 0 604772768 592867328 116835 33554432000 4194304 4356188 140735876926288 18446744073709551615 4331045 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11528/tid=11537] ppid=11523 vsize=578972 CPUtime=4976.36 cores=4-7
/proc/11528/task/11537/stat : 11537 (plingeling) R 11523 11523 11372 0 -1 4202560 1067488 1249 0 0 497501 135 0 0 25 0 5 0 604772768 592867328 116835 33554432000 4194304 4356188 140735876926288 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11528/tid=11538] ppid=11523 vsize=578972 CPUtime=5002.99 cores=4-7
/proc/11528/task/11538/stat : 11538 (plingeling) R 11523 11523 11372 0 -1 4202560 837195 1249 0 0 500184 115 0 0 25 0 5 0 604772768 592867328 116835 33554432000 4194304 4356188 140735876926288 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40000.2
Current children cumulated vsize (KiB) 3110828

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 11523 and gives
#  childrusage.ru_utime.tv_sec=5002
#  childrusage.ru_utime.tv_usec=946436
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=30995
# CPU time returned by wait4() is 5002.98
# while last known CPU time is 40000.2
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5003.62
CPU time (s): 40000.2
CPU user time (s): 39988.9
CPU system time (s): 11.27
CPU usage (%): 799.426
Max. virtual memory (cumulated for all children) (KiB): 4083508

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 5002.95
system time used= 0.030995
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3868
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 599
involuntary context switches= 16644

runsolver used 8.12077 second user time and 19.1761 second system time

The end

Launcher Data 

Begin job on node116 at 2011-04-30 08:37:01
IDJOB=3275186
IDBENCH=71593
IDSOLVER=1638
FILE ID=node116/3275186-1304145421
RUNJOBID= node116-1304145366-11386
PBS_JOBID= 13170881
Free space on /tmp= 73284 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/forced/rbsat-v1560c133795gyes1.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3275186-1304145421/watcher-3275186-1304145421 -o /tmp/evaluation-result-3275186-1304145421/solver-3275186-1304145421 -C 40000 -W 5100 -M 31000  ppfolio HOME/instance-3275186-1304145421.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 118632220c6f3388c45a2582e9a2ac0e
RANDOM SEED=1630431152

node116.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30095656 kB
Buffers:       1008404 kB
Cached:        1486624 kB
SwapCached:       9656 kB
Active:         168760 kB
Inactive:      2344084 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30095656 kB
SwapTotal:    67111528 kB
SwapFree:     67054472 kB
Dirty:           11832 kB
Writeback:           0 kB
AnonPages:       15444 kB
Mapped:          12860 kB
Slab:           276196 kB
PageTables:       4268 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   188996 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73276 MiB
End job on node116 at 2011-04-30 10:00:26