Trace number 3394720

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.8 5009.72

General information on the benchmark

NameSAT03/handmade/anton/UNSAT/
SGI_30_70_20_70_1-dir.shuffled-as.sat03-146.cnf
MD5SUM7ef2b563e57ba91e22d23e55abe69485
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark2999.01
Satisfiable
(Un)Satisfiability was proved
Number of variables600
Number of clauses46988
Sum of the clauses size94536
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 246968
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 520

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394720-1306103948/watcher-3394720-1306103948 -o /tmp/evaluation-result-3394720-1306103948/solver-3394720-1306103948 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394720-1306103948.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.43 7.97 8.04 1/446 14420
/proc/meminfo: memFree=26587552/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=10600 CPUtime=0 cores=0-7
/proc/14420/stat : 14420 (plingeling) R 14418 14420 14087 0 -1 4202496 741 1255 0 0 0 0 0 0 24 0 1 0 721992741 10854400 672 33554432000 4194304 4417452 140734481997296 18446744073709551615 271550537472 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/14420/statm: 2650 672 147 55 0 536 0

[startup+0.045914 s]
/proc/loadavg: 7.43 7.97 8.04 1/446 14420
/proc/meminfo: memFree=26587552/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=17728 CPUtime=0.03 cores=0-7
/proc/14420/stat : 14420 (plingeling) R 14418 14420 14087 0 -1 4202496 2633 1255 0 0 3 0 0 0 25 0 1 0 721992741 18153472 2455 33554432000 4194304 4417452 140734481997296 18446744073709551615 4262065 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/14420/statm: 4432 2455 148 55 0 2318 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 17728

[startup+0.110877 s]
/proc/loadavg: 7.43 7.97 8.04 1/446 14420
/proc/meminfo: memFree=26587552/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=231940 CPUtime=0.38 cores=0-7
/proc/14420/stat : 14420 (plingeling) S 14418 14420 14087 0 -1 4202496 3017 1255 0 0 38 0 0 0 18 0 9 0 721992741 237506560 2815 33554432000 4194304 4417452 140734481997296 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/14420/statm: 57985 2815 174 55 0 55871 0
Current children cumulated CPU time (s) 0.38
Current children cumulated vsize (KiB) 231940

[startup+0.300818 s]
/proc/loadavg: 7.43 7.97 8.04 1/446 14420
/proc/meminfo: memFree=26587552/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=237592 CPUtime=1.9 cores=0-7
/proc/14420/stat : 14420 (plingeling) S 14418 14420 14087 0 -1 4202496 7400 1255 0 0 189 1 0 0 18 0 9 0 721992741 243294208 4316 33554432000 4194304 4417452 140734481997296 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/14420/statm: 59398 4316 180 55 0 57284 0
Current children cumulated CPU time (s) 1.9
Current children cumulated vsize (KiB) 237592

[startup+0.700745 s]
/proc/loadavg: 7.43 7.97 8.04 1/446 14420
/proc/meminfo: memFree=26587552/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=242008 CPUtime=5.1 cores=0-7
/proc/14420/stat : 14420 (plingeling) S 14418 14420 14087 0 -1 4202496 9227 1255 0 0 509 1 0 0 18 0 9 0 721992741 247816192 5420 33554432000 4194304 4417452 140734481997296 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/14420/statm: 60502 5420 180 55 0 58388 0
Current children cumulated CPU time (s) 5.1
Current children cumulated vsize (KiB) 242008

[startup+1.5006 s]
/proc/loadavg: 7.43 7.97 8.04 9/455 14434
/proc/meminfo: memFree=26561928/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=259672 CPUtime=11.49 cores=0-7
/proc/14420/stat : 14420 (plingeling) S 14418 14420 14087 0 -1 4202496 14919 1255 0 0 1147 2 0 0 18 0 9 0 721992741 265904128 9836 33554432000 4194304 4417452 140734481997296 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/14420/statm: 64918 9836 180 55 0 62804 0
[pid=14420/tid=14427] ppid=14418 vsize=259672 CPUtime=1.43 cores=0-7
/proc/14420/task/14427/stat : 14427 (plingeling) R 14418 14420 14087 0 -1 4202560 695 1255 0 0 143 0 0 0 19 0 9 0 721992747 265904128 9836 33554432000 4194304 4417452 140734481997296 18446744073709551615 4250017 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14420/tid=14428] ppid=14418 vsize=259672 CPUtime=1.43 cores=0-7
/proc/14420/task/14428/stat : 14428 (plingeling) R 14418 14420 14087 0 -1 4202560 538 1255 0 0 143 0 0 0 20 0 9 0 721992747 265904128 9836 33554432000 4194304 4417452 140734481997296 18446744073709551615 4258183 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14420/tid=14429] ppid=14418 vsize=259672 CPUtime=1.42 cores=0-7
/proc/14420/task/14429/stat : 14429 (plingeling) R 14418 14420 14087 0 -1 4202560 1982 1255 0 0 142 0 0 0 20 0 9 0 721992747 265904128 9836 33554432000 4194304 4417452 140734481997296 18446744073709551615 4277827 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14420/tid=14430] ppid=14418 vsize=259672 CPUtime=1.4 cores=0-7
/proc/14420/task/14430/stat : 14430 (plingeling) R 14418 14420 14087 0 -1 4202560 6016 1255 0 0 139 1 0 0 25 0 9 0 721992747 265904128 9836 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=14420/tid=14431] ppid=14418 vsize=259672 CPUtime=1.43 cores=0-7
/proc/14420/task/14431/stat : 14431 (plingeling) R 14418 14420 14087 0 -1 4202560 449 1255 0 0 143 0 0 0 23 0 9 0 721992747 265904128 9836 33554432000 4194304 4417452 140734481997296 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14420/tid=14432] ppid=14418 vsize=259672 CPUtime=1.43 cores=0-7
/proc/14420/task/14432/stat : 14432 (plingeling) R 14418 14420 14087 0 -1 4202560 573 1255 0 0 143 0 0 0 20 0 9 0 721992747 265904128 9836 33554432000 4194304 4417452 140734481997296 18446744073709551615 4250058 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14420/tid=14433] ppid=14418 vsize=259672 CPUtime=1.42 cores=0-7
/proc/14420/task/14433/stat : 14433 (plingeling) R 14418 14420 14087 0 -1 4202560 1341 1255 0 0 142 0 0 0 20 0 9 0 721992747 265904128 9836 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14420/tid=14434] ppid=14418 vsize=259672 CPUtime=1.42 cores=0-7
/proc/14420/task/14434/stat : 14434 (plingeling) R 14418 14420 14087 0 -1 4202560 676 1255 0 0 142 0 0 0 23 0 9 0 721992748 265904128 9836 33554432000 4194304 4417452 140734481997296 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.49
Current children cumulated vsize (KiB) 259672

[startup+3.10031 s]
/proc/loadavg: 7.43 7.97 8.04 9/455 14434
/proc/meminfo: memFree=26546552/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=262964 CPUtime=24.26 cores=0-7
/proc/14420/stat : 14420 (plingeling) S 14418 14420 14087 0 -1 4202496 19136 1255 0 0 2424 2 0 0 18 0 9 0 721992741 269275136 10659 33554432000 4194304 4417452 140734481997296 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/14420/statm: 65741 10659 180 55 0 63627 0
[pid=14420/tid=14427] ppid=14418 vsize=262964 CPUtime=3.03 cores=0-7
/proc/14420/task/14427/stat : 14427 (plingeling) R 14418 14420 14087 0 -1 4202560 1006 1255 0 0 303 0 0 0 25 0 9 0 721992747 269275136 10659 33554432000 4194304 4417452 140734481997296 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14420/tid=14428] ppid=14418 vsize=262964 CPUtime=3.03 cores=0-7
/proc/14420/task/14428/stat : 14428 (plingeling) R 14418 14420 14087 0 -1 4202560 849 1255 0 0 303 0 0 0 25 0 9 0 721992747 269275136 10659 33554432000 4194304 4417452 140734481997296 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14420/tid=14429] ppid=14418 vsize=262964 CPUtime=3.02 cores=0-7
/proc/14420/task/14429/stat : 14429 (plingeling) R 14418 14420 14087 0 -1 4202560 2177 1255 0 0 302 0 0 0 25 0 9 0 721992747 269275136 10659 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364805 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14420/tid=14430] ppid=14418 vsize=262964 CPUtime=2.98 cores=0-7
/proc/14420/task/14430/stat : 14430 (plingeling) R 14418 14420 14087 0 -1 4202560 8794 1255 0 0 297 1 0 0 25 0 9 0 721992747 269275136 10659 33554432000 4194304 4417452 140734481997296 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=14420/tid=14431] ppid=14418 vsize=262964 CPUtime=3.03 cores=0-7
/proc/14420/task/14431/stat : 14431 (plingeling) R 14418 14420 14087 0 -1 4202560 449 1255 0 0 303 0 0 0 25 0 9 0 721992747 269275136 10659 33554432000 4194304 4417452 140734481997296 18446744073709551615 4249143 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14420/tid=14432] ppid=14418 vsize=262964 CPUtime=3.03 cores=0-7
/proc/14420/task/14432/stat : 14432 (plingeling) R 14418 14420 14087 0 -1 4202560 884 1255 0 0 303 0 0 0 25 0 9 0 721992747 269275136 10659 33554432000 4194304 4417452 140734481997296 18446744073709551615 4249601 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14420/tid=14433] ppid=14418 vsize=262964 CPUtime=3.02 cores=0-7
/proc/14420/task/14433/stat : 14433 (plingeling) R 14418 14420 14087 0 -1 4202560 1341 1255 0 0 302 0 0 0 25 0 9 0 721992747 269275136 10659 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14420/tid=14434] ppid=14418 vsize=262964 CPUtime=3.02 cores=0-7
/proc/14420/task/14434/stat : 14434 (plingeling) R 14418 14420 14087 0 -1 4202560 987 1255 0 0 302 0 0 0 25 0 9 0 721992748 269275136 10659 33554432000 4194304 4417452 140734481997296 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.26
Current children cumulated vsize (KiB) 262964

[startup+6.30074 s]
/proc/loadavg: 7.47 7.97 8.04 9/455 14434
/proc/meminfo: memFree=26518604/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=311052 CPUtime=49.82 cores=0-7
/proc/14420/stat : 14420 (plingeling) S 14418 14420 14087 0 -1 4202496 34385 1255 0 0 4977 5 0 0 18 0 9 0 721992741 318517248 22681 33554432000 4194304 4417452 140734481997296 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/14420/statm: 77763 22681 180 55 0 75649 0
[pid=14420/tid=14427] ppid=14418 vsize=311052 CPUtime=6.22 cores=0-7
/proc/14420/task/14427/stat : 14427 (plingeling) R 14418 14420 14087 0 -1 4202560 1670 1255 0 0 622 0 0 0 25 0 9 0 721992747 318517248 22681 33554432000 4194304 4417452 140734481997296 18446744073709551615 4365078 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14420/tid=14428] ppid=14418 vsize=311052 CPUtime=6.23 cores=0-7
/proc/14420/task/14428/stat : 14428 (plingeling) R 14418 14420 14087 0 -1 4202560 1267 1255 0 0 623 0 0 0 25 0 9 0 721992747 318517248 22681 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14420/tid=14429] ppid=14418 vsize=311052 CPUtime=6.22 cores=0-7
/proc/14420/task/14429/stat : 14429 (plingeling) R 14418 14420 14087 0 -1 4202560 4260 1255 0 0 622 0 0 0 25 0 9 0 721992747 318517248 22681 33554432000 4194304 4417452 140734481997296 18446744073709551615 4249601 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14420/tid=14430] ppid=14418 vsize=311052 CPUtime=6.13 cores=0-7
/proc/14420/task/14430/stat : 14430 (plingeling) R 14418 14420 14087 0 -1 4202560 17347 1255 0 0 611 2 0 0 25 0 9 0 721992747 318517248 22681 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=14420/tid=14431] ppid=14418 vsize=311052 CPUtime=6.23 cores=0-7
/proc/14420/task/14431/stat : 14431 (plingeling) R 14418 14420 14087 0 -1 4202560 859 1255 0 0 623 0 0 0 25 0 9 0 721992747 318517248 22681 33554432000 4194304 4417452 140734481997296 18446744073709551615 4354510 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14420/tid=14432] ppid=14418 vsize=311052 CPUtime=6.23 cores=0-7
/proc/14420/task/14432/stat : 14432 (plingeling) R 14418 14420 14087 0 -1 4202560 1528 1255 0 0 623 0 0 0 25 0 9 0 721992747 318517248 22681 33554432000 4194304 4417452 140734481997296 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14420/tid=14433] ppid=14418 vsize=311052 CPUtime=6.22 cores=0-7
/proc/14420/task/14433/stat : 14433 (plingeling) R 14418 14420 14087 0 -1 4202560 2639 1255 0 0 622 0 0 0 25 0 9 0 721992747 318517248 22681 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14420/tid=14434] ppid=14418 vsize=311052 CPUtime=6.23 cores=0-7
/proc/14420/task/14434/stat : 14434 (plingeling) R 14418 14420 14087 0 -1 4202560 2166 1255 0 0 623 0 0 0 25 0 9 0 721992748 318517248 22681 33554432000 4194304 4417452 140734481997296 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 49.82
Current children cumulated vsize (KiB) 311052

[startup+12.7006 s]
/proc/loadavg: 7.52 7.97 8.04 9/455 14434
/proc/meminfo: memFree=26419432/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=400856 CPUtime=100.92 cores=0-7
/proc/14420/stat : 14420 (plingeling) S 14418 14420 14087 0 -1 4202496 73880 1255 0 0 10080 12 0 0 18 0 9 0 721992741 410476544 45132 33554432000 4194304 4417452 140734481997296 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/14420/statm: 100214 45132 180 55 0 98100 0
[pid=14420/tid=14427] ppid=14418 vsize=400856 CPUtime=12.62 cores=0-7
/proc/14420/task/14427/stat : 14427 (plingeling) R 14418 14420 14087 0 -1 4202560 3371 1255 0 0 1262 0 0 0 25 0 9 0 721992747 410476544 45132 33554432000 4194304 4417452 140734481997296 18446744073709551615 4236590 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14420/tid=14428] ppid=14418 vsize=400856 CPUtime=12.62 cores=0-7
/proc/14420/task/14428/stat : 14428 (plingeling) R 14418 14420 14087 0 -1 4202560 5089 1255 0 0 1262 0 0 0 25 0 9 0 721992747 410476544 45132 33554432000 4194304 4417452 140734481997296 18446744073709551615 4255698 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14420/tid=14429] ppid=14418 vsize=400856 CPUtime=12.62 cores=0-7
/proc/14420/task/14429/stat : 14429 (plingeling) R 14418 14420 14087 0 -1 4202560 8005 1255 0 0 1261 1 0 0 25 0 9 0 721992747 410476544 45132 33554432000 4194304 4417452 140734481997296 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14420/tid=14430] ppid=14418 vsize=400856 CPUtime=12.43 cores=0-7
/proc/14420/task/14430/stat : 14430 (plingeling) R 14418 14420 14087 0 -1 4202560 30369 1255 0 0 1238 5 0 0 25 0 9 0 721992747 410476544 45132 33554432000 4194304 4417452 140734481997296 18446744073709551615 4250678 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=14420/tid=14431] ppid=14418 vsize=400856 CPUtime=12.63 cores=0-7
/proc/14420/task/14431/stat : 14431 (plingeling) R 14418 14420 14087 0 -1 4202560 8447 1255 0 0 1262 1 0 0 25 0 9 0 721992747 410476544 45132 33554432000 4194304 4417452 140734481997296 18446744073709551615 4251160 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14420/tid=14432] ppid=14418 vsize=400856 CPUtime=12.62 cores=0-7
/proc/14420/task/14432/stat : 14432 (plingeling) R 14418 14420 14087 0 -1 4202560 3164 1255 0 0 1262 0 0 0 25 0 9 0 721992747 410476544 45132 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14420/tid=14433] ppid=14418 vsize=400856 CPUtime=12.62 cores=0-7
/proc/14420/task/14433/stat : 14433 (plingeling) R 14418 14420 14087 0 -1 4202560 7882 1255 0 0 1261 1 0 0 25 0 9 0 721992747 410476544 45132 33554432000 4194304 4417452 140734481997296 18446744073709551615 4256139 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14420/tid=14434] ppid=14418 vsize=400856 CPUtime=12.62 cores=0-7
/proc/14420/task/14434/stat : 14434 (plingeling) R 14418 14420 14087 0 -1 4202560 4904 1255 0 0 1262 0 0 0 25 0 9 0 721992748 410476544 45132 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 100.92
Current children cumulated vsize (KiB) 400856

[startup+25.5013 s]
/proc/loadavg: 7.62 7.97 8.04 9/455 14435
/proc/meminfo: memFree=26264608/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=545808 CPUtime=203.11 cores=0-7
/proc/14420/stat : 14420 (plingeling) S 14418 14420 14087 0 -1 4202496 150677 1255 0 0 20287 24 0 0 18 0 9 0 721992741 558907392 81460 33554432000 4194304 4417452 140734481997296 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/14420/statm: 136452 81460 180 55 0 134338 0
[pid=14420/tid=14427] ppid=14418 vsize=545808 CPUtime=25.42 cores=0-7
/proc/14420/task/14427/stat : 14427 (plingeling) R 14418 14420 14087 0 -1 4202560 11132 1255 0 0 2541 1 0 0 25 0 9 0 721992747 558907392 81460 33554432000 4194304 4417452 140734481997296 18446744073709551615 4250807 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14420/tid=14428] ppid=14418 vsize=545808 CPUtime=25.42 cores=0-7
/proc/14420/task/14428/stat : 14428 (plingeling) R 14418 14420 14087 0 -1 4202560 13249 1255 0 0 2541 1 0 0 25 0 9 0 721992747 558907392 81460 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14420/tid=14429] ppid=14418 vsize=545808 CPUtime=25.42 cores=0-7
/proc/14420/task/14429/stat : 14429 (plingeling) R 14418 14420 14087 0 -1 4202560 22860 1255 0 0 2539 3 0 0 25 0 9 0 721992747 558907392 81460 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14420/tid=14430] ppid=14418 vsize=545808 CPUtime=25.1 cores=0-7
/proc/14420/task/14430/stat : 14430 (plingeling) R 14418 14420 14087 0 -1 4202560 46528 1255 0 0 2501 9 0 0 25 0 9 0 721992747 558907392 81460 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364542 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=14420/tid=14431] ppid=14418 vsize=545808 CPUtime=25.42 cores=0-7
/proc/14420/task/14431/stat : 14431 (plingeling) R 14418 14420 14087 0 -1 4202560 18691 1255 0 0 2540 2 0 0 25 0 9 0 721992747 558907392 81460 33554432000 4194304 4417452 140734481997296 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14420/tid=14432] ppid=14418 vsize=545808 CPUtime=25.42 cores=0-7
/proc/14420/task/14432/stat : 14432 (plingeling) R 14418 14420 14087 0 -1 4202560 6585 1255 0 0 2542 0 0 0 25 0 9 0 721992747 558907392 81460 33554432000 4194304 4417452 140734481997296 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14420/tid=14433] ppid=14418 vsize=545808 CPUtime=25.43 cores=0-7
/proc/14420/task/14433/stat : 14433 (plingeling) R 14418 14420 14087 0 -1 4202560 16831 1255 0 0 2540 3 0 0 25 0 9 0 721992747 558907392 81460 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14420/tid=14434] ppid=14418 vsize=545808 CPUtime=25.36 cores=0-7
/proc/14420/task/14434/stat : 14434 (plingeling) R 14418 14420 14087 0 -1 4202560 12152 1255 0 0 2535 1 0 0 25 0 9 0 721992748 558907392 81460 33554432000 4194304 4417452 140734481997296 18446744073709551615 4277559 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.11
Current children cumulated vsize (KiB) 545808

[startup+51.1006 s]
/proc/loadavg: 7.83 7.99 8.04 9/455 14435
/proc/meminfo: memFree=26132532/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=677480 CPUtime=407.53 cores=0-7
/proc/14420/stat : 14420 (plingeling) S 14418 14420 14087 0 -1 4202496 270272 1255 0 0 40708 45 0 0 18 0 9 0 721992741 693739520 114418 33554432000 4194304 4417452 140734481997296 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/14420/statm: 169370 114418 180 55 0 167256 0
[pid=14420/tid=14427] ppid=14418 vsize=677480 CPUtime=51.02 cores=0-7
/proc/14420/task/14427/stat : 14427 (plingeling) R 14418 14420 14087 0 -1 4202560 28667 1255 0 0 5098 4 0 0 25 0 9 0 721992747 693739520 114418 33554432000 4194304 4417452 140734481997296 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14420/tid=14428] ppid=14418 vsize=677480 CPUtime=51.01 cores=0-7
/proc/14420/task/14428/stat : 14428 (plingeling) R 14418 14420 14087 0 -1 4202560 25965 1255 0 0 5098 3 0 0 25 0 9 0 721992747 693739520 114418 33554432000 4194304 4417452 140734481997296 18446744073709551615 4366407 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14420/tid=14429] ppid=14418 vsize=677480 CPUtime=51.02 cores=0-7

################
# More data... #
################

[pid=14420/tid=14429] ppid=14418 vsize=6857376 CPUtime=4722.34 cores=0-7
/proc/14420/task/14429/stat : 14429 (plingeling) R 14418 14420 14087 0 -1 4202560 1697660 1255 0 0 471895 339 0 0 25 0 9 0 721992747 7021953024 1664316 33554432000 4194304 4417452 140734481997296 18446744073709551615 4365092 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14420/tid=14430] ppid=14418 vsize=6857376 CPUtime=4720.88 cores=0-7
/proc/14420/task/14430/stat : 14430 (plingeling) R 14418 14420 14087 0 -1 4202560 859269 1255 0 0 471900 188 0 0 25 0 9 0 721992747 7021953024 1664316 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=14420/tid=14431] ppid=14418 vsize=6857376 CPUtime=4721.7 cores=0-7
/proc/14420/task/14431/stat : 14431 (plingeling) R 14418 14420 14087 0 -1 4202560 1453298 1255 0 0 471869 301 0 0 25 0 9 0 721992747 7021953024 1664316 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364735 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14420/tid=14432] ppid=14418 vsize=6857376 CPUtime=4722.34 cores=0-7
/proc/14420/task/14432/stat : 14432 (plingeling) R 14418 14420 14087 0 -1 4202560 1624445 1255 0 0 471915 319 0 0 25 0 9 0 721992747 7021953024 1664316 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14420/tid=14433] ppid=14418 vsize=6857376 CPUtime=4722.24 cores=0-7
/proc/14420/task/14433/stat : 14433 (plingeling) R 14418 14420 14087 0 -1 4202560 1761217 1255 0 0 471862 362 0 0 25 0 9 0 721992747 7021953024 1664316 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14420/tid=14434] ppid=14418 vsize=6857376 CPUtime=4653.64 cores=0-7
/proc/14420/task/14434/stat : 14434 (plingeling) R 14418 14420 14087 0 -1 4202560 1804345 1255 0 0 465024 340 0 0 25 0 9 0 721992748 7021953024 1664316 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 37707.8
Current children cumulated vsize (KiB) 6857376

[startup+4782.3 s]
/proc/loadavg: 8.19 8.16 8.10 9/455 14573
/proc/meminfo: memFree=19871348/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=6899268 CPUtime=38186.8 cores=0-7
/proc/14420/stat : 14420 (plingeling) S 14418 14420 14087 0 -1 4202496 12889667 1255 0 0 3816112 2566 0 0 18 0 9 0 721992741 7064850432 1674789 33554432000 4194304 4417452 140734481997296 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/14420/statm: 1724817 1674789 180 55 0 1722703 0
[pid=14420/tid=14427] ppid=14418 vsize=6899268 CPUtime=4782.34 cores=0-7
/proc/14420/task/14427/stat : 14427 (plingeling) R 14418 14420 14087 0 -1 4202560 2079123 1255 0 0 477838 396 0 0 25 0 9 0 721992747 7064850432 1674789 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14420/tid=14428] ppid=14418 vsize=6899268 CPUtime=4782.13 cores=0-7
/proc/14420/task/14428/stat : 14428 (plingeling) R 14418 14420 14087 0 -1 4202560 1580804 1255 0 0 477901 312 0 0 25 0 9 0 721992747 7064850432 1674789 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14420/tid=14429] ppid=14418 vsize=6899268 CPUtime=4782.32 cores=0-7
/proc/14420/task/14429/stat : 14429 (plingeling) R 14418 14420 14087 0 -1 4202560 1697660 1255 0 0 477893 339 0 0 25 0 9 0 721992747 7064850432 1674789 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14420/tid=14430] ppid=14418 vsize=6899268 CPUtime=4780.85 cores=0-7
/proc/14420/task/14430/stat : 14430 (plingeling) R 14418 14420 14087 0 -1 4202560 875653 1255 0 0 477896 189 0 0 25 0 9 0 721992747 7064850432 1674789 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=14420/tid=14431] ppid=14418 vsize=6899268 CPUtime=4781.69 cores=0-7
/proc/14420/task/14431/stat : 14431 (plingeling) R 14418 14420 14087 0 -1 4202560 1453298 1255 0 0 477868 301 0 0 25 0 9 0 721992747 7064850432 1674789 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14420/tid=14432] ppid=14418 vsize=6899268 CPUtime=4782.33 cores=0-7
/proc/14420/task/14432/stat : 14432 (plingeling) R 14418 14420 14087 0 -1 4202560 1624445 1255 0 0 477914 319 0 0 25 0 9 0 721992747 7064850432 1674789 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14420/tid=14433] ppid=14418 vsize=6899268 CPUtime=4782.23 cores=0-7
/proc/14420/task/14433/stat : 14433 (plingeling) R 14418 14420 14087 0 -1 4202560 1771690 1255 0 0 477857 366 0 0 25 0 9 0 721992747 7064850432 1674789 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14420/tid=14434] ppid=14418 vsize=6899268 CPUtime=4712.75 cores=0-7
/proc/14420/task/14434/stat : 14434 (plingeling) R 14418 14420 14087 0 -1 4202560 1804345 1255 0 0 470935 340 0 0 25 0 9 0 721992748 7064850432 1674789 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38186.8
Current children cumulated vsize (KiB) 6899268

[startup+4842.3 s]
/proc/loadavg: 8.12 8.14 8.09 9/455 14575
/proc/meminfo: memFree=19789476/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=6981424 CPUtime=38665.9 cores=0-7
/proc/14420/stat : 14420 (plingeling) S 14418 14420 14087 0 -1 4202496 13027969 1255 0 0 3863997 2591 0 0 18 0 9 0 721992741 7148978176 1695351 33554432000 4194304 4417452 140734481997296 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/14420/statm: 1745356 1695351 180 55 0 1743242 0
[pid=14420/tid=14427] ppid=14418 vsize=6981424 CPUtime=4842.35 cores=0-7
/proc/14420/task/14427/stat : 14427 (plingeling) R 14418 14420 14087 0 -1 4202560 2201041 1255 0 0 483816 419 0 0 25 0 9 0 721992747 7148978176 1695351 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14420/tid=14428] ppid=14418 vsize=6981424 CPUtime=4842.13 cores=0-7
/proc/14420/task/14428/stat : 14428 (plingeling) R 14418 14420 14087 0 -1 4202560 1580804 1255 0 0 483901 312 0 0 25 0 9 0 721992747 7148978176 1695351 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14420/tid=14429] ppid=14418 vsize=6981424 CPUtime=4842.32 cores=0-7
/proc/14420/task/14429/stat : 14429 (plingeling) R 14418 14420 14087 0 -1 4202560 1697660 1255 0 0 483893 339 0 0 25 0 9 0 721992747 7148978176 1695351 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14420/tid=14430] ppid=14418 vsize=6981424 CPUtime=4840.84 cores=0-7
/proc/14420/task/14430/stat : 14430 (plingeling) R 14418 14420 14087 0 -1 4202560 875653 1255 0 0 483894 190 0 0 25 0 9 0 721992747 7148978176 1695351 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=14420/tid=14431] ppid=14418 vsize=6981424 CPUtime=4841.69 cores=0-7
/proc/14420/task/14431/stat : 14431 (plingeling) R 14418 14420 14087 0 -1 4202560 1453298 1255 0 0 483868 301 0 0 25 0 9 0 721992747 7148978176 1695351 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14420/tid=14432] ppid=14418 vsize=6981424 CPUtime=4842.33 cores=0-7
/proc/14420/task/14432/stat : 14432 (plingeling) R 14418 14420 14087 0 -1 4202560 1624445 1255 0 0 483914 319 0 0 25 0 9 0 721992747 7148978176 1695351 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14420/tid=14433] ppid=14418 vsize=6981424 CPUtime=4842.23 cores=0-7
/proc/14420/task/14433/stat : 14433 (plingeling) R 14418 14420 14087 0 -1 4202560 1788074 1255 0 0 483855 368 0 0 25 0 9 0 721992747 7148978176 1695351 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14420/tid=14434] ppid=14418 vsize=6981424 CPUtime=4771.87 cores=0-7
/proc/14420/task/14434/stat : 14434 (plingeling) R 14418 14420 14087 0 -1 4202560 1804345 1255 0 0 476847 340 0 0 25 0 9 0 721992748 7148978176 1695351 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38665.9
Current children cumulated vsize (KiB) 6981424

[startup+4902.3 s]
/proc/loadavg: 8.18 8.16 8.10 9/455 14577
/proc/meminfo: memFree=20177544/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=6539164 CPUtime=39145 cores=0-7
/proc/14420/stat : 14420 (plingeling) S 14418 14420 14087 0 -1 4202496 13177768 1255 0 0 3911883 2619 0 0 18 0 9 0 721992741 6696103936 1582274 33554432000 4194304 4417452 140734481997296 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/14420/statm: 1634791 1582274 180 55 0 1632677 0
[pid=14420/tid=14427] ppid=14418 vsize=6539164 CPUtime=4902.34 cores=0-7
/proc/14420/task/14427/stat : 14427 (plingeling) R 14418 14420 14087 0 -1 4202560 2230399 1255 0 0 489809 425 0 0 25 0 9 0 721992747 6696103936 1582274 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14420/tid=14428] ppid=14418 vsize=6539164 CPUtime=4902.13 cores=0-7
/proc/14420/task/14428/stat : 14428 (plingeling) R 14418 14420 14087 0 -1 4202560 1674466 1255 0 0 489886 327 0 0 25 0 9 0 721992747 6696103936 1582274 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14420/tid=14429] ppid=14418 vsize=6539164 CPUtime=4902.33 cores=0-7
/proc/14420/task/14429/stat : 14429 (plingeling) R 14418 14420 14087 0 -1 4202560 1697660 1255 0 0 489894 339 0 0 25 0 9 0 721992747 6696103936 1582274 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364861 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14420/tid=14430] ppid=14418 vsize=6539164 CPUtime=4900.82 cores=0-7
/proc/14420/task/14430/stat : 14430 (plingeling) R 14418 14420 14087 0 -1 4202560 892037 1255 0 0 489887 195 0 0 25 0 9 0 721992747 6696103936 1582274 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=14420/tid=14431] ppid=14418 vsize=6539164 CPUtime=4901.69 cores=0-7
/proc/14420/task/14431/stat : 14431 (plingeling) R 14418 14420 14087 0 -1 4202560 1453298 1255 0 0 489868 301 0 0 25 0 9 0 721992747 6696103936 1582274 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14420/tid=14432] ppid=14418 vsize=6539164 CPUtime=4902.33 cores=0-7
/proc/14420/task/14432/stat : 14432 (plingeling) R 14418 14420 14087 0 -1 4202560 1624445 1255 0 0 489914 319 0 0 25 0 9 0 721992747 6696103936 1582274 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14420/tid=14433] ppid=14418 vsize=6539164 CPUtime=4902.23 cores=0-7
/proc/14420/task/14433/stat : 14433 (plingeling) R 14418 14420 14087 0 -1 4202560 1798469 1255 0 0 489854 369 0 0 25 0 9 0 721992747 6696103936 1582274 33554432000 4194304 4417452 140734481997296 18446744073709551615 4263224 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14420/tid=14434] ppid=14418 vsize=6539164 CPUtime=4831 cores=0-7
/proc/14420/task/14434/stat : 14434 (plingeling) R 14418 14420 14087 0 -1 4202560 1804345 1255 0 0 482760 340 0 0 25 0 9 0 721992748 6696103936 1582274 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39145
Current children cumulated vsize (KiB) 6539164

[startup+4962.3 s]
/proc/loadavg: 8.24 8.19 8.11 9/455 14587
/proc/meminfo: memFree=20231480/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=6541760 CPUtime=39624.1 cores=0-7
/proc/14420/stat : 14420 (plingeling) S 14418 14420 14087 0 -1 4202496 13290623 1255 0 0 3959776 2633 0 0 18 0 9 0 721992741 6698762240 1584971 33554432000 4194304 4417452 140734481997296 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/14420/statm: 1635440 1584971 180 55 0 1633326 0
[pid=14420/tid=14427] ppid=14418 vsize=6541760 CPUtime=4962.34 cores=0-7
/proc/14420/task/14427/stat : 14427 (plingeling) R 14418 14420 14087 0 -1 4202560 2233231 1255 0 0 495809 425 0 0 25 0 9 0 721992747 6698762240 1584971 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14420/tid=14428] ppid=14418 vsize=6541760 CPUtime=4962.13 cores=0-7
/proc/14420/task/14428/stat : 14428 (plingeling) R 14418 14420 14087 0 -1 4202560 1674466 1255 0 0 495886 327 0 0 25 0 9 0 721992747 6698762240 1584971 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14420/tid=14429] ppid=14418 vsize=6541760 CPUtime=4962.33 cores=0-7
/proc/14420/task/14429/stat : 14429 (plingeling) R 14418 14420 14087 0 -1 4202560 1697660 1255 0 0 495894 339 0 0 25 0 9 0 721992747 6698762240 1584971 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14420/tid=14430] ppid=14418 vsize=6541760 CPUtime=4960.81 cores=0-7
/proc/14420/task/14430/stat : 14430 (plingeling) R 14418 14420 14087 0 -1 4202560 908421 1255 0 0 495884 197 0 0 25 0 9 0 721992747 6698762240 1584971 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=14420/tid=14431] ppid=14418 vsize=6541760 CPUtime=4961.69 cores=0-7
/proc/14420/task/14431/stat : 14431 (plingeling) R 14418 14420 14087 0 -1 4202560 1453298 1255 0 0 495868 301 0 0 25 0 9 0 721992747 6698762240 1584971 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14420/tid=14432] ppid=14418 vsize=6541760 CPUtime=4962.31 cores=0-7
/proc/14420/task/14432/stat : 14432 (plingeling) R 14418 14420 14087 0 -1 4202560 1624445 1255 0 0 495912 319 0 0 25 0 9 0 721992747 6698762240 1584971 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14420/tid=14433] ppid=14418 vsize=6541760 CPUtime=4962.23 cores=0-7
/proc/14420/task/14433/stat : 14433 (plingeling) R 14418 14420 14087 0 -1 4202560 1798469 1255 0 0 495854 369 0 0 25 0 9 0 721992747 6698762240 1584971 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364690 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14420/tid=14434] ppid=14418 vsize=6541760 CPUtime=4890.12 cores=0-7
/proc/14420/task/14434/stat : 14434 (plingeling) R 14418 14420 14087 0 -1 4202560 1897984 1255 0 0 488660 352 0 0 25 0 9 0 721992748 6698762240 1584971 33554432000 4194304 4417452 140734481997296 18446744073709551615 4270186 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39624.1
Current children cumulated vsize (KiB) 6541760



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5009.42 s]
/proc/loadavg: 8.21 8.19 8.11 9/455 14589
/proc/meminfo: memFree=19977492/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=6770916 CPUtime=40000.4 cores=0-7
/proc/14420/stat : 14420 (plingeling) S 14418 14420 14087 0 -1 4202496 13370440 1255 0 0 3997398 2642 0 0 18 0 9 0 721992741 6933417984 1648404 33554432000 4194304 4417452 140734481997296 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/14420/statm: 1692729 1648404 180 55 0 1690615 0
[pid=14420/tid=14427] ppid=14418 vsize=6770916 CPUtime=5009.47 cores=0-7
/proc/14420/task/14427/stat : 14427 (plingeling) R 14418 14420 14087 0 -1 4202560 2240943 1255 0 0 500521 426 0 0 25 0 9 0 721992747 6933417984 1648404 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14420/tid=14428] ppid=14418 vsize=6770916 CPUtime=5009.26 cores=0-7
/proc/14420/task/14428/stat : 14428 (plingeling) R 14418 14420 14087 0 -1 4202560 1674466 1255 0 0 500599 327 0 0 25 0 9 0 721992747 6933417984 1648404 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14420/tid=14429] ppid=14418 vsize=6770916 CPUtime=5009.46 cores=0-7
/proc/14420/task/14429/stat : 14429 (plingeling) R 14418 14420 14087 0 -1 4202560 1697660 1255 0 0 500607 339 0 0 25 0 9 0 721992747 6933417984 1648404 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14420/tid=14430] ppid=14418 vsize=6770916 CPUtime=5007.93 cores=0-7
/proc/14420/task/14430/stat : 14430 (plingeling) R 14418 14420 14087 0 -1 4202560 908421 1255 0 0 500596 197 0 0 25 0 9 0 721992747 6933417984 1648404 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364812 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=14420/tid=14431] ppid=14418 vsize=6770916 CPUtime=5008.83 cores=0-7
/proc/14420/task/14431/stat : 14431 (plingeling) R 14418 14420 14087 0 -1 4202560 1453298 1255 0 0 500582 301 0 0 25 0 9 0 721992747 6933417984 1648404 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14420/tid=14432] ppid=14418 vsize=6770916 CPUtime=5009.44 cores=0-7
/proc/14420/task/14432/stat : 14432 (plingeling) R 14418 14420 14087 0 -1 4202560 1624445 1255 0 0 500625 319 0 0 25 0 9 0 721992747 6933417984 1648404 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14420/tid=14433] ppid=14418 vsize=6770916 CPUtime=5009.36 cores=0-7
/proc/14420/task/14433/stat : 14433 (plingeling) R 14418 14420 14087 0 -1 4202560 1798469 1255 0 0 500567 369 0 0 25 0 9 0 721992747 6933417984 1648404 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14420/tid=14434] ppid=14418 vsize=6770916 CPUtime=4936.55 cores=0-7
/proc/14420/task/14434/stat : 14434 (plingeling) R 14418 14420 14087 0 -1 4202560 1970089 1255 0 0 493295 360 0 0 25 0 9 0 721992748 6933417984 1648404 33554432000 4194304 4417452 140734481997296 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 6770916

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5009.6 s]
/proc/loadavg: 8.21 8.19 8.11 9/455 14589
/proc/meminfo: memFree=19977492/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=0 CPUtime=40000.7 cores=0-7
/proc/14420/stat : 14420 (plingeling) Z 14418 14420 14087 0 -1 4203548 13370440 1255 0 0 3997407 2659 0 0 18 0 2 0 721992741 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/14420/statm: 0 0 0 0 0 0 0
[pid=14420/tid=14434] ppid=14418 vsize=0 CPUtime=4936.71 cores=0-7
/proc/14420/task/14434/stat : 14434 (plingeling) R 14418 14420 14087 0 -1 4203588 1970089 1255 0 0 493295 376 0 0 25 0 2 0 721992748 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 0

[startup+5009.7 s]
/proc/loadavg: 8.21 8.19 8.11 9/455 14589
/proc/meminfo: memFree=19977492/32951124 swapFree=61585512/67111528
[pid=14420] ppid=14418 vsize=0 CPUtime=40000.8 cores=0-7
/proc/14420/stat : 14420 (plingeling) Z 14418 14420 14087 0 -1 4203548 13370440 1255 0 0 3997407 2669 0 0 18 0 2 0 721992741 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/14420/statm: 0 0 0 0 0 0 0
[pid=14420/tid=14434] ppid=14418 vsize=0 CPUtime=4936.81 cores=0-7
/proc/14420/task/14434/stat : 14434 (plingeling) R 14418 14420 14087 0 -1 4203588 1970089 1255 0 0 493295 386 0 0 25 0 2 0 721992748 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5009.72
CPU time (s): 40000.8
CPU user time (s): 39974.1
CPU system time (s): 26.7139
CPU usage (%): 798.463
Max. virtual memory (cumulated for all children) (KiB): 7607276

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39974.1
system time used= 26.7139
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 13371695
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 426
involuntary context switches= 399363

runsolver used 20.1769 second user time and 53.0419 second system time

The end

Launcher Data 

Begin job on node133 at 2011-05-23 00:39:09
IDJOB=3394720
IDBENCH=83952
IDSOLVER=1851
FILE ID=node133/3394720-1306103948
RUNJOBID= node133-1306098924-14101
PBS_JOBID= 13387977
Free space on /tmp= 73028 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT03/handmade/anton/UNSAT/SGI_30_70_20_70_1-dir.shuffled-as.sat03-146.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394720-1306103948/watcher-3394720-1306103948 -o /tmp/evaluation-result-3394720-1306103948/solver-3394720-1306103948 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394720-1306103948.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 7ef2b563e57ba91e22d23e55abe69485
RANDOM SEED=2041758066

node133.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6741.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      26587848 kB
Buffers:         12472 kB
Cached:         141836 kB
SwapCached:      56188 kB
Active:          70664 kB
Inactive:      6097520 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      26587848 kB
SwapTotal:    67111528 kB
SwapFree:     61585512 kB
Dirty:             824 kB
Writeback:           0 kB
AnonPages:     5964024 kB
Mapped:          30216 kB
Slab:            72108 kB
PageTables:      53404 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 114886592 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73028 MiB
End job on node133 at 2011-05-23 02:02:40