Trace number 3371137

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.45 1202.93

General information on the benchmark

NameSAT04/handmade/hirsch03/
hgen8-n320-01-S1432474240.shuffled-as.sat03-890.used-as.sat04-841.cnf
MD5SUMf97e40b22ad62886042ca9677daf337d
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables260
Number of clauses486
Sum of the clauses size1102
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2421
Number of clauses of size 30
Number of clauses of size 465
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371137-1305431037/watcher-3371137-1305431037 -o /tmp/evaluation-result-3371137-1305431037/solver-3371137-1305431037 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371137-1305431037.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.94 1.20 1.59 6/305 27533
/proc/meminfo: memFree=11998396/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=115620 CPUtime=0.01 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 396 1253 0 0 1 0 0 0 23 0 5 0 654701602 118394880 321 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 28905 321 178 55 0 26791 0
[pid=27533/tid=27539] ppid=27531 vsize=115620 CPUtime=0 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 39 1253 0 0 0 0 0 0 24 0 5 0 654701602 118394880 321 33554432000 4194304 4417452 140736710424096 18446744073709551615 4252098 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=115620 CPUtime=0 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 16 1253 0 0 0 0 0 0 25 0 5 0 654701602 118394880 321 33554432000 4194304 4417452 140736710424096 18446744073709551615 4256984 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=115620 CPUtime=0 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 1 1253 0 0 0 0 0 0 25 0 5 0 654701602 118394880 321 33554432000 4194304 4417452 140736710424096 18446744073709551615 255377406446 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=115620 CPUtime=0 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 24 1253 0 0 0 0 0 0 25 0 5 0 654701602 118394880 321 33554432000 4194304 4417452 140736710424096 18446744073709551615 4250502 0 0 4096 17442 0 0 0 -1 4 0 0 0

[startup+0.0931 s]
/proc/loadavg: 0.94 1.20 1.59 6/305 27533
/proc/meminfo: memFree=11998396/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=183980 CPUtime=0.34 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 1219 1253 0 0 34 0 0 0 23 0 5 0 654701602 188395520 1035 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 45995 1035 180 55 0 43881 0
[pid=27533/tid=27539] ppid=27531 vsize=183980 CPUtime=0.08 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 125 1253 0 0 8 0 0 0 25 0 5 0 654701602 188395520 1035 33554432000 4194304 4417452 140736710424096 18446744073709551615 4307843 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=183980 CPUtime=0.08 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 200 1253 0 0 8 0 0 0 25 0 5 0 654701602 188395520 1035 33554432000 4194304 4417452 140736710424096 18446744073709551615 4305532 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=183980 CPUtime=0.08 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 253 1253 0 0 8 0 0 0 25 0 5 0 654701602 188395520 1035 33554432000 4194304 4417452 140736710424096 18446744073709551615 4312132 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=183980 CPUtime=0.08 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 325 1253 0 0 8 0 0 0 25 0 5 0 654701602 188395520 1035 33554432000 4194304 4417452 140736710424096 18446744073709551615 4351616 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 0.34
Current children cumulated vsize (KiB) 183980

[startup+0.101098 s]
/proc/loadavg: 0.94 1.20 1.59 6/305 27533
/proc/meminfo: memFree=11998396/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=184164 CPUtime=0.37 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 1297 1253 0 0 37 0 0 0 23 0 5 0 654701602 188583936 1081 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 46041 1081 180 55 0 43927 0
[pid=27533/tid=27539] ppid=27531 vsize=184164 CPUtime=0.09 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 190 1253 0 0 9 0 0 0 25 0 5 0 654701602 188583936 1081 33554432000 4194304 4417452 140736710424096 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=184164 CPUtime=0.09 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 200 1253 0 0 9 0 0 0 25 0 5 0 654701602 188583936 1081 33554432000 4194304 4417452 140736710424096 18446744073709551615 4316274 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=184164 CPUtime=0.08 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 253 1253 0 0 8 0 0 0 25 0 5 0 654701602 188583936 1081 33554432000 4194304 4417452 140736710424096 18446744073709551615 4308818 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=184164 CPUtime=0.09 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 338 1253 0 0 9 0 0 0 25 0 5 0 654701602 188583936 1081 33554432000 4194304 4417452 140736710424096 18446744073709551615 4265415 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 0.37
Current children cumulated vsize (KiB) 184164

[startup+0.301063 s]
/proc/loadavg: 0.94 1.20 1.59 6/305 27533
/proc/meminfo: memFree=11998396/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=186500 CPUtime=1.17 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 3560 1253 0 0 117 0 0 0 23 0 5 0 654701602 190976000 1703 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 46625 1703 180 55 0 44511 0
[pid=27533/tid=27539] ppid=27531 vsize=186500 CPUtime=0.29 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 868 1253 0 0 29 0 0 0 25 0 5 0 654701602 190976000 1703 33554432000 4194304 4417452 140736710424096 18446744073709551615 4250690 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=186500 CPUtime=0.29 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 653 1253 0 0 29 0 0 0 25 0 5 0 654701602 190976000 1703 33554432000 4194304 4417452 140736710424096 18446744073709551615 4358082 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=186500 CPUtime=0.28 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 785 1253 0 0 28 0 0 0 25 0 5 0 654701602 190976000 1703 33554432000 4194304 4417452 140736710424096 18446744073709551615 18446744073699065955 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=186500 CPUtime=0.29 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 938 1253 0 0 29 0 0 0 25 0 5 0 654701602 190976000 1703 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1.17
Current children cumulated vsize (KiB) 186500

[startup+0.700989 s]
/proc/loadavg: 0.94 1.20 1.59 6/305 27533
/proc/meminfo: memFree=11998396/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=190708 CPUtime=2.77 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 7970 1253 0 0 276 1 0 0 23 0 5 0 654701602 195284992 2755 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 47677 2755 180 55 0 45563 0
[pid=27533/tid=27539] ppid=27531 vsize=190708 CPUtime=0.69 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 1645 1253 0 0 69 0 0 0 25 0 5 0 654701602 195284992 2755 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=190708 CPUtime=0.69 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 1778 1253 0 0 69 0 0 0 25 0 5 0 654701602 195284992 2755 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=190708 CPUtime=0.68 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 2058 1253 0 0 68 0 0 0 25 0 5 0 654701602 195284992 2755 33554432000 4194304 4417452 140736710424096 18446744073709551615 4355852 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=190708 CPUtime=0.68 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 2173 1253 0 0 68 0 0 0 25 0 5 0 654701602 195284992 2755 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2.77
Current children cumulated vsize (KiB) 190708

[startup+1.50084 s]
/proc/loadavg: 0.94 1.20 1.59 10/310 27543
/proc/meminfo: memFree=11980168/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=195764 CPUtime=5.96 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 14586 1253 0 0 594 2 0 0 23 0 5 0 654701602 200462336 3987 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 48941 3987 180 55 0 46827 0
[pid=27533/tid=27539] ppid=27531 vsize=195764 CPUtime=1.48 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 3461 1253 0 0 148 0 0 0 25 0 5 0 654701602 200462336 3987 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=195764 CPUtime=1.49 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 3335 1253 0 0 149 0 0 0 25 0 5 0 654701602 200462336 3987 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=195764 CPUtime=1.47 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 3865 1253 0 0 147 0 0 0 25 0 5 0 654701602 200462336 3987 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364735 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=195764 CPUtime=1.48 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 3609 1253 0 0 148 0 0 0 25 0 5 0 654701602 200462336 3987 33554432000 4194304 4417452 140736710424096 18446744073709551615 4352347 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 5.96
Current children cumulated vsize (KiB) 195764

[startup+3.10055 s]
/proc/loadavg: 1.50 1.31 1.63 10/310 27543
/proc/meminfo: memFree=11971736/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=198604 CPUtime=12.35 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 27440 1253 0 0 1231 4 0 0 23 0 5 0 654701602 203370496 4712 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 49651 4712 180 55 0 47537 0
[pid=27533/tid=27539] ppid=27531 vsize=198604 CPUtime=3.09 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 7119 1253 0 0 308 1 0 0 25 0 5 0 654701602 203370496 4712 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=198604 CPUtime=3.08 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 6001 1253 0 0 308 0 0 0 25 0 5 0 654701602 203370496 4712 33554432000 4194304 4417452 140736710424096 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=198604 CPUtime=3.06 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 7470 1253 0 0 305 1 0 0 25 0 5 0 654701602 203370496 4712 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=198604 CPUtime=3.09 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 6534 1253 0 0 308 1 0 0 25 0 5 0 654701602 203370496 4712 33554432000 4194304 4417452 140736710424096 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 12.35
Current children cumulated vsize (KiB) 198604

[startup+6.30095 s]
/proc/loadavg: 1.50 1.31 1.63 10/310 27543
/proc/meminfo: memFree=11955624/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=210732 CPUtime=25.12 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 49460 1253 0 0 2504 8 0 0 23 0 5 0 654701602 215789568 7760 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 52683 7760 180 55 0 50569 0
[pid=27533/tid=27539] ppid=27531 vsize=210732 CPUtime=6.28 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 12069 1253 0 0 627 1 0 0 25 0 5 0 654701602 215789568 7760 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=210732 CPUtime=6.28 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 12696 1253 0 0 627 1 0 0 25 0 5 0 654701602 215789568 7760 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=210732 CPUtime=6.23 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 13738 1253 0 0 621 2 0 0 25 0 5 0 654701602 215789568 7760 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=210732 CPUtime=6.28 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 10641 1253 0 0 627 1 0 0 25 0 5 0 654701602 215789568 7760 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364896 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 25.12
Current children cumulated vsize (KiB) 210732

[startup+12.7008 s]
/proc/loadavg: 2.02 1.42 1.66 10/310 27543
/proc/meminfo: memFree=11942880/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=215956 CPUtime=50.65 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 87553 1253 0 0 5051 14 0 0 23 0 5 0 654701602 221138944 9035 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 53989 9035 180 55 0 51875 0
[pid=27533/tid=27539] ppid=27531 vsize=215956 CPUtime=12.69 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 22026 1253 0 0 1266 3 0 0 25 0 5 0 654701602 221138944 9035 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=215956 CPUtime=12.68 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 22940 1253 0 0 1265 3 0 0 25 0 5 0 654701602 221138944 9035 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=215956 CPUtime=12.57 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 24271 1253 0 0 1253 4 0 0 25 0 5 0 654701602 221138944 9035 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=215956 CPUtime=12.67 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 18000 1253 0 0 1265 2 0 0 25 0 5 0 654701602 221138944 9035 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 50.65
Current children cumulated vsize (KiB) 215956

[startup+25.5004 s]
/proc/loadavg: 3.35 1.74 1.76 10/310 27543
/proc/meminfo: memFree=11908896/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=228816 CPUtime=101.73 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 161282 1253 0 0 10146 27 0 0 23 0 5 0 654701602 234307584 12250 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 57204 12250 180 55 0 55090 0
[pid=27533/tid=27539] ppid=27531 vsize=228816 CPUtime=25.48 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 40622 1253 0 0 2542 6 0 0 25 0 5 0 654701602 234307584 12250 33554432000 4194304 4417452 140736710424096 18446744073709551615 4365092 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=228816 CPUtime=25.49 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 46177 1253 0 0 2541 8 0 0 25 0 5 0 654701602 234307584 12250 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=228816 CPUtime=25.25 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 41470 1253 0 0 2518 7 0 0 25 0 5 0 654701602 234307584 12250 33554432000 4194304 4417452 140736710424096 18446744073709551615 4308025 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=228816 CPUtime=25.48 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 32697 1253 0 0 2543 5 0 0 25 0 5 0 654701602 234307584 12250 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 101.73
Current children cumulated vsize (KiB) 228816

[startup+51.1007 s]
/proc/loadavg: 4.93 2.24 1.93 11/310 27544
/proc/meminfo: memFree=11892236/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=243348 CPUtime=203.9 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 266866 1253 0 0 20348 42 0 0 23 0 5 0 654701602 249188352 15883 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 60837 15883 180 55 0 58723 0
[pid=27533/tid=27539] ppid=27531 vsize=243348 CPUtime=51.09 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 70472 1253 0 0 5099 10 0 0 25 0 5 0 654701602 249188352 15883 33554432000 4194304 4417452 140736710424096 18446744073709551615 4310225 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=243348 CPUtime=51.08 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 73660 1253 0 0 5096 12 0 0 25 0 5 0 654701602 249188352 15883 33554432000 4194304 4417452 140736710424096 18446744073709551615 4366407 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=243348 CPUtime=50.62 cores=0,2,4,6

################
# More data... #
################

[pid=27533/tid=27540] ppid=27531 vsize=390616 CPUtime=762.32 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 473183 1253 0 0 76159 73 0 0 25 0 5 0 654701602 399990784 52732 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=390616 CPUtime=755.29 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 468397 1253 0 0 75453 76 0 0 25 0 5 0 654701602 399990784 52732 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=390616 CPUtime=762.18 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 384069 1253 0 0 76159 59 0 0 25 0 5 0 654701602 399990784 52732 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3042.15
Current children cumulated vsize (KiB) 390616

[startup+822.301 s]
/proc/loadavg: 8.30 7.79 5.47 10/310 27565
/proc/meminfo: memFree=11623280/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=369880 CPUtime=3281.55 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 1908804 1253 0 0 327860 295 0 0 23 0 5 0 654701602 378757120 47548 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 92470 47548 180 55 0 90356 0
[pid=27533/tid=27539] ppid=27531 vsize=369880 CPUtime=822.31 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 512233 1253 0 0 82157 74 0 0 25 0 5 0 654701602 378757120 47548 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=369880 CPUtime=822.31 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 500286 1253 0 0 82154 77 0 0 25 0 5 0 654701602 378757120 47548 33554432000 4194304 4417452 140736710424096 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=369880 CPUtime=814.74 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 490042 1253 0 0 81394 80 0 0 25 0 5 0 654701602 378757120 47548 33554432000 4194304 4417452 140736710424096 18446744073709551615 4359068 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=369880 CPUtime=822.16 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 405927 1253 0 0 82154 62 0 0 25 0 5 0 654701602 378757120 47548 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3281.55
Current children cumulated vsize (KiB) 369880

[startup+882.301 s]
/proc/loadavg: 8.30 7.88 5.65 11/310 27566
/proc/meminfo: memFree=11592764/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=384988 CPUtime=3521 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 2004827 1253 0 0 351791 309 0 0 23 0 5 0 654701602 394227712 51325 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 96247 51325 180 55 0 94133 0
[pid=27533/tid=27539] ppid=27531 vsize=384988 CPUtime=882.31 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 542464 1253 0 0 88153 78 0 0 25 0 5 0 654701602 394227712 51325 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=384988 CPUtime=882.32 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 519164 1253 0 0 88152 80 0 0 25 0 5 0 654701602 394227712 51325 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=384988 CPUtime=874.18 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 517872 1253 0 0 87334 84 0 0 25 0 5 0 654701602 394227712 51325 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=384988 CPUtime=882.15 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 425011 1253 0 0 88150 65 0 0 25 0 5 0 654701602 394227712 51325 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3521
Current children cumulated vsize (KiB) 384988

[startup+942.301 s]
/proc/loadavg: 8.16 7.91 5.79 10/310 27568
/proc/meminfo: memFree=11582464/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=382804 CPUtime=3760.45 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 2097031 1253 0 0 375723 322 0 0 23 0 5 0 654701602 391991296 50779 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 95701 50779 180 55 0 93587 0
[pid=27533/tid=27539] ppid=27531 vsize=382804 CPUtime=942.32 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 561585 1253 0 0 94150 82 0 0 25 0 5 0 654701602 391991296 50779 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=382804 CPUtime=942.31 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 547563 1253 0 0 94148 83 0 0 25 0 5 0 654701602 391991296 50779 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=382804 CPUtime=933.64 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 544654 1253 0 0 93276 88 0 0 25 0 5 0 654701602 391991296 50779 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=382804 CPUtime=942.15 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 442913 1253 0 0 94147 68 0 0 25 0 5 0 654701602 391991296 50779 33554432000 4194304 4417452 140736710424096 18446744073709551615 4365042 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3760.45
Current children cumulated vsize (KiB) 382804

[startup+1002.3 s]
/proc/loadavg: 8.26 7.97 5.95 11/310 27569
/proc/meminfo: memFree=11550576/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=407476 CPUtime=3999.88 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 2184241 1253 0 0 399653 335 0 0 23 0 5 0 654701602 417255424 56915 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 101869 56915 180 55 0 99755 0
[pid=27533/tid=27539] ppid=27531 vsize=407476 CPUtime=1002.32 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 577037 1253 0 0 100148 84 0 0 25 0 5 0 654701602 417255424 56915 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=407476 CPUtime=1002.32 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 571924 1253 0 0 100146 86 0 0 25 0 5 0 654701602 417255424 56915 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=407476 CPUtime=993.07 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 571558 1253 0 0 99215 92 0 0 25 0 5 0 654701602 417255424 56915 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=407476 CPUtime=1002.14 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 463406 1253 0 0 100143 71 0 0 25 0 5 0 654701602 417255424 56915 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3999.88
Current children cumulated vsize (KiB) 407476

[startup+1062.3 s]
/proc/loadavg: 7.53 7.86 6.03 5/300 27597
/proc/meminfo: memFree=11834288/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=351364 CPUtime=4239.31 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 2284067 1253 0 0 423579 352 0 0 23 0 5 0 654701602 359796736 42703 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 87841 42703 180 55 0 85727 0
[pid=27533/tid=27539] ppid=27531 vsize=351364 CPUtime=1062.32 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 603979 1253 0 0 106144 88 0 0 25 0 5 0 654701602 359796736 42703 33554432000 4194304 4417452 140736710424096 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=351364 CPUtime=1062.31 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 592381 1253 0 0 106142 89 0 0 25 0 5 0 654701602 359796736 42703 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=351364 CPUtime=1052.51 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 598496 1253 0 0 105154 97 0 0 25 0 5 0 654701602 359796736 42703 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=351364 CPUtime=1062.14 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 488895 1253 0 0 106138 76 0 0 25 0 5 0 654701602 359796736 42703 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4239.31
Current children cumulated vsize (KiB) 351364

[startup+1122.3 s]
/proc/loadavg: 5.30 7.13 5.90 5/300 27599
/proc/meminfo: memFree=11785668/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=399096 CPUtime=4478.74 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 2368448 1253 0 0 447510 364 0 0 23 0 5 0 654701602 408674304 54820 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 99774 54820 180 55 0 97660 0
[pid=27533/tid=27539] ppid=27531 vsize=399096 CPUtime=1122.32 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 620803 1253 0 0 112142 90 0 0 25 0 5 0 654701602 408674304 54820 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=399096 CPUtime=1122.32 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 613357 1253 0 0 112139 93 0 0 25 0 5 0 654701602 408674304 54820 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=399096 CPUtime=1111.94 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 624985 1253 0 0 111093 101 0 0 25 0 5 0 654701602 408674304 54820 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=399096 CPUtime=1122.13 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 508987 1253 0 0 112134 79 0 0 25 0 5 0 654701602 408674304 54820 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364885 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4478.74
Current children cumulated vsize (KiB) 399096

[startup+1182.3 s]
/proc/loadavg: 4.55 6.62 5.79 5/300 27600
/proc/meminfo: memFree=11803132/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=383708 CPUtime=4718.18 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 2479713 1253 0 0 471437 381 0 0 23 0 5 0 654701602 392916992 50973 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 95927 50973 180 55 0 93813 0
[pid=27533/tid=27539] ppid=27531 vsize=383708 CPUtime=1182.32 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 654724 1253 0 0 118137 95 0 0 25 0 5 0 654701602 392916992 50973 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=383708 CPUtime=1182.32 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 650760 1253 0 0 118134 98 0 0 25 0 5 0 654701602 392916992 50973 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=383708 CPUtime=1171.38 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 646684 1253 0 0 117034 104 0 0 25 0 5 0 654701602 392916992 50973 33554432000 4194304 4417452 140736710424096 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=383708 CPUtime=1182.12 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 527229 1253 0 0 118131 81 0 0 25 0 5 0 654701602 392916992 50973 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4718.18
Current children cumulated vsize (KiB) 383708



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.9 s]
/proc/loadavg: 4.43 6.42 5.75 5/300 27601
/proc/meminfo: memFree=11776216/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=407792 CPUtime=4800.4 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 2502422 1253 0 0 479656 384 0 0 23 0 5 0 654701602 417579008 56994 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 101948 56994 180 55 0 99834 0
[pid=27533/tid=27539] ppid=27531 vsize=407792 CPUtime=1202.93 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 660059 1253 0 0 120197 96 0 0 25 0 5 0 654701602 417579008 56994 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=407792 CPUtime=1202.93 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 655179 1253 0 0 120195 98 0 0 25 0 5 0 654701602 417579008 56994 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=407792 CPUtime=1191.78 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 650828 1253 0 0 119073 105 0 0 25 0 5 0 654701602 417579008 56994 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=407792 CPUtime=1202.74 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 536040 1253 0 0 120191 83 0 0 25 0 5 0 654701602 417579008 56994 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.4
Current children cumulated vsize (KiB) 407792

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202.9 s]
/proc/loadavg: 4.43 6.42 5.75 5/300 27601
/proc/meminfo: memFree=11776216/32951124 swapFree=66050672/67111528
[pid=27533] ppid=27531 vsize=407792 CPUtime=4800.4 cores=0,2,4,6
/proc/27533/stat : 27533 (plingeling) S 27531 27533 18486 0 -1 4202496 2502422 1253 0 0 479656 384 0 0 23 0 5 0 654701602 417579008 56994 33554432000 4194304 4417452 140736710424096 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27533/statm: 101948 56994 180 55 0 99834 0
[pid=27533/tid=27539] ppid=27531 vsize=407792 CPUtime=1202.93 cores=0,2,4,6
/proc/27533/task/27539/stat : 27539 (plingeling) R 27531 27533 18486 0 -1 4202560 660059 1253 0 0 120197 96 0 0 25 0 5 0 654701602 417579008 56994 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27533/tid=27540] ppid=27531 vsize=407792 CPUtime=1202.93 cores=0,2,4,6
/proc/27533/task/27540/stat : 27540 (plingeling) R 27531 27533 18486 0 -1 4202560 655179 1253 0 0 120195 98 0 0 25 0 5 0 654701602 417579008 56994 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27533/tid=27541] ppid=27531 vsize=407792 CPUtime=1191.78 cores=0,2,4,6
/proc/27533/task/27541/stat : 27541 (plingeling) R 27531 27533 18486 0 -1 4202560 650828 1253 0 0 119073 105 0 0 25 0 5 0 654701602 417579008 56994 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27533/tid=27542] ppid=27531 vsize=407792 CPUtime=1202.74 cores=0,2,4,6
/proc/27533/task/27542/stat : 27542 (plingeling) R 27531 27533 18486 0 -1 4202560 536040 1253 0 0 120191 83 0 0 25 0 5 0 654701602 417579008 56994 33554432000 4194304 4417452 140736710424096 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.4
Current children cumulated vsize (KiB) 407792

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.93
CPU time (s): 4800.45
CPU user time (s): 4796.59
CPU system time (s): 3.85641
CPU usage (%): 399.064
Max. virtual memory (cumulated for all children) (KiB): 443512

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.59
system time used= 3.85641
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2503675
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 194
involuntary context switches= 71371

runsolver used 3.2675 second user time and 7.8938 second system time

The end

Launcher Data 

Begin job on node135 at 2011-05-15 05:43:57
IDJOB=3371137
IDBENCH=84715
IDSOLVER=1848
FILE ID=node135/3371137-1305431037
RUNJOBID= node135-1305431037-27504
PBS_JOBID= 13324578
Free space on /tmp= 73372 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT04/handmade/hirsch03/hgen8-n320-01-S1432474240.shuffled-as.sat03-890.used-as.sat04-841.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371137-1305431037/watcher-3371137-1305431037 -o /tmp/evaluation-result-3371137-1305431037/solver-3371137-1305431037 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371137-1305431037.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f97e40b22ad62886042ca9677daf337d
RANDOM SEED=1045359314

node135.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6640.14
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.78
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      11998536 kB
Buffers:        286768 kB
Cached:        6003580 kB
SwapCached:      50684 kB
Active:        9922080 kB
Inactive:     10784924 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      11998536 kB
SwapTotal:    67111528 kB
SwapFree:     66050672 kB
Dirty:             820 kB
Writeback:           0 kB
AnonPages:    14371116 kB
Mapped:          19156 kB
Slab:           137084 kB
PageTables:      46788 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 57102032 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73372 MiB
End job on node135 at 2011-05-15 06:04:02