Trace number 3372161

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.8 5007.99

General information on the benchmark

Nameindustrial/IBM_FV_2002_13_rule_1/
SAT_dat.k45.cnf
MD5SUM8a76c8add64173ae39149aae572a02d9
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark168.225
Satisfiable
(Un)Satisfiability was proved
Number of variables37432
Number of clauses147345
Sum of the clauses size401667
Maximum clause length37
Minimum clause length1
Number of clauses of size 1603
Number of clauses of size 269621
Number of clauses of size 371450
Number of clauses of size 4266
Number of clauses of size 5505
Number of clauses of size over 54900

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372161-1305436000/watcher-3372161-1305436000 -o /tmp/evaluation-result-3372161-1305436000/solver-3372161-1305436000 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3372161-1305436000.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.82 7.96 7.85 1/351 4974
/proc/meminfo: memFree=23980716/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=8728 CPUtime=0 cores=0-7
/proc/4974/stat : 4974 (plingeling) R 4972 4974 4839 0 -1 4202496 256 1252 0 0 0 0 0 0 25 0 1 0 655200436 8937472 185 33554432000 4194304 4417452 140736022623792 18446744073709551615 253640859392 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/4974/statm: 2182 185 138 55 0 68 0

[startup+0.0634099 s]
/proc/loadavg: 7.82 7.96 7.85 1/351 4974
/proc/meminfo: memFree=23980716/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=19108 CPUtime=0.05 cores=0-7
/proc/4974/stat : 4974 (plingeling) R 4972 4974 4839 0 -1 4202496 2872 1252 0 0 5 0 0 0 25 0 1 0 655200436 19566592 2801 33554432000 4194304 4417452 140736022623792 18446744073709551615 4259612 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/4974/statm: 4777 2801 149 55 0 2663 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 19108

[startup+0.100405 s]
/proc/loadavg: 7.82 7.96 7.85 1/351 4974
/proc/meminfo: memFree=23980716/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=28220 CPUtime=0.09 cores=0-7
/proc/4974/stat : 4974 (plingeling) R 4972 4974 4839 0 -1 4202496 5152 1252 0 0 9 0 0 0 25 0 1 0 655200436 28897280 5079 33554432000 4194304 4417452 140736022623792 18446744073709551615 253640478285 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/4974/statm: 7055 5079 149 55 0 4941 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 28220

[startup+0.300367 s]
/proc/loadavg: 7.82 7.96 7.85 1/351 4974
/proc/meminfo: memFree=23980716/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=63596 CPUtime=0.28 cores=0-7
/proc/4974/stat : 4974 (plingeling) R 4972 4974 4839 0 -1 4202496 13996 1252 0 0 27 1 0 0 25 0 1 0 655200436 65122304 13923 33554432000 4194304 4417452 140736022623792 18446744073709551615 253640859392 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/4974/statm: 15899 13923 149 55 0 13785 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 63596

[startup+1.18017 s]
/proc/loadavg: 7.82 7.96 7.85 1/351 4974
/proc/meminfo: memFree=23980716/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=231664 CPUtime=6.86 cores=0-7
/proc/4974/stat : 4974 (plingeling) S 4972 4974 4839 0 -1 4202496 31331 1252 0 0 682 4 0 0 18 0 9 0 655200436 237223936 21222 33554432000 4194304 4417452 140736022623792 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/4974/statm: 57916 21222 176 55 0 55802 0
Current children cumulated CPU time (s) 6.86
Current children cumulated vsize (KiB) 231664

[startup+1.50112 s]
/proc/loadavg: 7.82 7.96 7.85 1/351 4974
/proc/meminfo: memFree=23980716/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=230792 CPUtime=9.43 cores=0-7
/proc/4974/stat : 4974 (plingeling) S 4972 4974 4839 0 -1 4202496 33818 1252 0 0 939 4 0 0 18 0 9 0 655200436 236331008 20040 33554432000 4194304 4417452 140736022623792 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/4974/statm: 57698 20040 177 55 0 55584 0
Current children cumulated CPU time (s) 9.43
Current children cumulated vsize (KiB) 230792

[startup+3.10082 s]
/proc/loadavg: 7.91 7.98 7.85 9/360 4988
/proc/meminfo: memFree=23868052/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=262036 CPUtime=22.2 cores=0-7
/proc/4974/stat : 4974 (plingeling) S 4972 4974 4839 0 -1 4202496 47890 1252 0 0 2213 7 0 0 18 0 9 0 655200436 268324864 28347 33554432000 4194304 4417452 140736022623792 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/4974/statm: 65509 28347 180 55 0 63395 0
[pid=4974/tid=4981] ppid=4972 vsize=262036 CPUtime=2.72 cores=0-7
/proc/4974/task/4981/stat : 4981 (plingeling) R 4972 4974 4839 0 -1 4202560 4653 1252 0 0 272 0 0 0 25 0 9 0 655200472 268324864 28347 33554432000 4194304 4417452 140736022623792 18446744073709551615 4257081 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4974/tid=4982] ppid=4972 vsize=262036 CPUtime=2.7 cores=0-7
/proc/4974/task/4982/stat : 4982 (plingeling) R 4972 4974 4839 0 -1 4202560 3485 1252 0 0 270 0 0 0 23 0 9 0 655200472 268324864 28347 33554432000 4194304 4417452 140736022623792 18446744073709551615 4253405 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4974/tid=4983] ppid=4972 vsize=262036 CPUtime=2.73 cores=0-7
/proc/4974/task/4983/stat : 4983 (plingeling) R 4972 4974 4839 0 -1 4202560 1325 1252 0 0 273 0 0 0 25 0 9 0 655200472 268324864 28347 33554432000 4194304 4417452 140736022623792 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4974/tid=4984] ppid=4972 vsize=262036 CPUtime=2.73 cores=0-7
/proc/4974/task/4984/stat : 4984 (plingeling) R 4972 4974 4839 0 -1 4202560 1865 1252 0 0 273 0 0 0 23 0 9 0 655200472 268324864 28347 33554432000 4194304 4417452 140736022623792 18446744073709551615 4354497 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4974/tid=4985] ppid=4972 vsize=262036 CPUtime=2.73 cores=0-7
/proc/4974/task/4985/stat : 4985 (plingeling) R 4972 4974 4839 0 -1 4202560 5471 1252 0 0 272 1 0 0 23 0 9 0 655200472 268324864 28347 33554432000 4194304 4417452 140736022623792 18446744073709551615 4252943 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4974/tid=4986] ppid=4972 vsize=262036 CPUtime=2.69 cores=0-7
/proc/4974/task/4986/stat : 4986 (plingeling) R 4972 4974 4839 0 -1 4202560 2930 1252 0 0 269 0 0 0 25 0 9 0 655200472 268324864 28347 33554432000 4194304 4417452 140736022623792 18446744073709551615 4257111 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4974/tid=4987] ppid=4972 vsize=262036 CPUtime=2.72 cores=0-7
/proc/4974/task/4987/stat : 4987 (plingeling) R 4972 4974 4839 0 -1 4202560 5420 1252 0 0 272 0 0 0 23 0 9 0 655200472 268324864 28347 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4974/tid=4988] ppid=4972 vsize=262036 CPUtime=2.73 cores=0-7
/proc/4974/task/4988/stat : 4988 (plingeling) R 4972 4974 4839 0 -1 4202560 4019 1252 0 0 273 0 0 0 25 0 9 0 655200472 268324864 28347 33554432000 4194304 4417452 140736022623792 18446744073709551615 4253320 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 22.2
Current children cumulated vsize (KiB) 262036

[startup+6.30124 s]
/proc/loadavg: 7.92 7.98 7.86 9/360 4988
/proc/meminfo: memFree=23855156/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=274300 CPUtime=47.76 cores=0-7
/proc/4974/stat : 4974 (plingeling) S 4972 4974 4839 0 -1 4202496 51422 1252 0 0 4769 7 0 0 18 0 9 0 655200436 280883200 31413 33554432000 4194304 4417452 140736022623792 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/4974/statm: 68575 31413 180 55 0 66461 0
[pid=4974/tid=4981] ppid=4972 vsize=274300 CPUtime=5.93 cores=0-7
/proc/4974/task/4981/stat : 4981 (plingeling) R 4972 4974 4839 0 -1 4202560 5209 1252 0 0 592 1 0 0 25 0 9 0 655200472 280883200 31413 33554432000 4194304 4417452 140736022623792 18446744073709551615 4321595 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4974/tid=4982] ppid=4972 vsize=274300 CPUtime=5.86 cores=0-7
/proc/4974/task/4982/stat : 4982 (plingeling) R 4972 4974 4839 0 -1 4202560 3997 1252 0 0 586 0 0 0 25 0 9 0 655200472 280883200 31413 33554432000 4194304 4417452 140736022623792 18446744073709551615 4366657 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4974/tid=4983] ppid=4972 vsize=274300 CPUtime=5.94 cores=0-7
/proc/4974/task/4983/stat : 4983 (plingeling) R 4972 4974 4839 0 -1 4202560 2093 1252 0 0 594 0 0 0 25 0 9 0 655200472 280883200 31413 33554432000 4194304 4417452 140736022623792 18446744073709551615 4366105 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4974/tid=4984] ppid=4972 vsize=274300 CPUtime=5.93 cores=0-7
/proc/4974/task/4984/stat : 4984 (plingeling) R 4972 4974 4839 0 -1 4202560 2467 1252 0 0 593 0 0 0 25 0 9 0 655200472 280883200 31413 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4974/tid=4985] ppid=4972 vsize=274300 CPUtime=5.93 cores=0-7
/proc/4974/task/4985/stat : 4985 (plingeling) R 4972 4974 4839 0 -1 4202560 5747 1252 0 0 592 1 0 0 25 0 9 0 655200472 280883200 31413 33554432000 4194304 4417452 140736022623792 18446744073709551615 4236790 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4974/tid=4986] ppid=4972 vsize=274300 CPUtime=5.89 cores=0-7
/proc/4974/task/4986/stat : 4986 (plingeling) R 4972 4974 4839 0 -1 4202560 3328 1252 0 0 589 0 0 0 25 0 9 0 655200472 280883200 31413 33554432000 4194304 4417452 140736022623792 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4974/tid=4987] ppid=4972 vsize=274300 CPUtime=5.92 cores=0-7
/proc/4974/task/4987/stat : 4987 (plingeling) R 4972 4974 4839 0 -1 4202560 5840 1252 0 0 592 0 0 0 25 0 9 0 655200472 280883200 31413 33554432000 4194304 4417452 140736022623792 18446744073709551615 4365965 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4974/tid=4988] ppid=4972 vsize=274300 CPUtime=5.93 cores=0-7
/proc/4974/task/4988/stat : 4988 (plingeling) R 4972 4974 4839 0 -1 4202560 4019 1252 0 0 593 0 0 0 25 0 9 0 655200472 280883200 31413 33554432000 4194304 4417452 140736022623792 18446744073709551615 4237002 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 47.76
Current children cumulated vsize (KiB) 274300

[startup+12.7011 s]
/proc/loadavg: 7.93 7.98 7.86 9/360 4988
/proc/meminfo: memFree=23822072/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=307932 CPUtime=98.93 cores=0-7
/proc/4974/stat : 4974 (plingeling) S 4972 4974 4839 0 -1 4202496 62936 1252 0 0 9884 9 0 0 18 0 9 0 655200436 315322368 39821 33554432000 4194304 4417452 140736022623792 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/4974/statm: 76983 39821 180 55 0 74869 0
[pid=4974/tid=4981] ppid=4972 vsize=307932 CPUtime=12.34 cores=0-7
/proc/4974/task/4981/stat : 4981 (plingeling) R 4972 4974 4839 0 -1 4202560 7600 1252 0 0 1233 1 0 0 25 0 9 0 655200472 315322368 39821 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4974/tid=4982] ppid=4972 vsize=307932 CPUtime=12.2 cores=0-7
/proc/4974/task/4982/stat : 4982 (plingeling) R 4972 4974 4839 0 -1 4202560 4024 1252 0 0 1220 0 0 0 25 0 9 0 655200472 315322368 39821 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4974/tid=4983] ppid=4972 vsize=307932 CPUtime=12.34 cores=0-7
/proc/4974/task/4983/stat : 4983 (plingeling) R 4972 4974 4839 0 -1 4202560 4486 1252 0 0 1234 0 0 0 25 0 9 0 655200472 315322368 39821 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4974/tid=4984] ppid=4972 vsize=307932 CPUtime=12.34 cores=0-7
/proc/4974/task/4984/stat : 4984 (plingeling) R 4972 4974 4839 0 -1 4202560 4778 1252 0 0 1234 0 0 0 25 0 9 0 655200472 315322368 39821 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4974/tid=4985] ppid=4972 vsize=307932 CPUtime=12.34 cores=0-7
/proc/4974/task/4985/stat : 4985 (plingeling) R 4972 4974 4839 0 -1 4202560 7103 1252 0 0 1233 1 0 0 25 0 9 0 655200472 315322368 39821 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4974/tid=4986] ppid=4972 vsize=307932 CPUtime=12.31 cores=0-7
/proc/4974/task/4986/stat : 4986 (plingeling) R 4972 4974 4839 0 -1 4202560 3971 1252 0 0 1231 0 0 0 25 0 9 0 655200472 315322368 39821 33554432000 4194304 4417452 140736022623792 18446744073709551615 4357805 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4974/tid=4987] ppid=4972 vsize=307932 CPUtime=12.34 cores=0-7
/proc/4974/task/4987/stat : 4987 (plingeling) R 4972 4974 4839 0 -1 4202560 6887 1252 0 0 1234 0 0 0 25 0 9 0 655200472 315322368 39821 33554432000 4194304 4417452 140736022623792 18446744073709551615 4321702 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4974/tid=4988] ppid=4972 vsize=307932 CPUtime=12.34 cores=0-7
/proc/4974/task/4988/stat : 4988 (plingeling) R 4972 4974 4839 0 -1 4202560 5365 1252 0 0 1234 0 0 0 25 0 9 0 655200472 315322368 39821 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 98.93
Current children cumulated vsize (KiB) 307932

[startup+25.5007 s]
/proc/loadavg: 7.94 7.98 7.86 9/360 4988
/proc/meminfo: memFree=23806748/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=333276 CPUtime=201.14 cores=0-7
/proc/4974/stat : 4974 (plingeling) S 4972 4974 4839 0 -1 4202496 100085 1252 0 0 20100 14 0 0 18 0 9 0 655200436 341274624 46189 33554432000 4194304 4417452 140736022623792 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/4974/statm: 83319 46189 180 55 0 81205 0
[pid=4974/tid=4981] ppid=4972 vsize=333276 CPUtime=25.13 cores=0-7
/proc/4974/task/4981/stat : 4981 (plingeling) R 4972 4974 4839 0 -1 4202560 12001 1252 0 0 2512 1 0 0 25 0 9 0 655200472 341274624 46189 33554432000 4194304 4417452 140736022623792 18446744073709551615 4366099 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4974/tid=4982] ppid=4972 vsize=333276 CPUtime=24.84 cores=0-7
/proc/4974/task/4982/stat : 4982 (plingeling) R 4972 4974 4839 0 -1 4202560 10686 1252 0 0 2483 1 0 0 25 0 9 0 655200472 341274624 46189 33554432000 4194304 4417452 140736022623792 18446744073709551615 4272314 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4974/tid=4983] ppid=4972 vsize=333276 CPUtime=25.13 cores=0-7
/proc/4974/task/4983/stat : 4983 (plingeling) R 4972 4974 4839 0 -1 4202560 9701 1252 0 0 2512 1 0 0 25 0 9 0 655200472 341274624 46189 33554432000 4194304 4417452 140736022623792 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4974/tid=4984] ppid=4972 vsize=333276 CPUtime=25.13 cores=0-7
/proc/4974/task/4984/stat : 4984 (plingeling) R 4972 4974 4839 0 -1 4202560 7565 1252 0 0 2513 0 0 0 25 0 9 0 655200472 341274624 46189 33554432000 4194304 4417452 140736022623792 18446744073709551615 4356867 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4974/tid=4985] ppid=4972 vsize=333276 CPUtime=25.13 cores=0-7
/proc/4974/task/4985/stat : 4985 (plingeling) R 4972 4974 4839 0 -1 4202560 13606 1252 0 0 2511 2 0 0 25 0 9 0 655200472 341274624 46189 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4974/tid=4986] ppid=4972 vsize=333276 CPUtime=25.09 cores=0-7
/proc/4974/task/4986/stat : 4986 (plingeling) R 4972 4974 4839 0 -1 4202560 4794 1252 0 0 2509 0 0 0 25 0 9 0 655200472 341274624 46189 33554432000 4194304 4417452 140736022623792 18446744073709551615 4253214 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4974/tid=4987] ppid=4972 vsize=333276 CPUtime=25.13 cores=0-7
/proc/4974/task/4987/stat : 4987 (plingeling) R 4972 4974 4839 0 -1 4202560 7675 1252 0 0 2512 1 0 0 25 0 9 0 655200472 341274624 46189 33554432000 4194304 4417452 140736022623792 18446744073709551615 4365949 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4974/tid=4988] ppid=4972 vsize=333276 CPUtime=25.12 cores=0-7
/proc/4974/task/4988/stat : 4988 (plingeling) R 4972 4974 4839 0 -1 4202560 15335 1252 0 0 2511 1 0 0 25 0 9 0 655200472 341274624 46189 33554432000 4194304 4417452 140736022623792 18446744073709551615 4236930 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 201.14
Current children cumulated vsize (KiB) 333276

[startup+51.1011 s]
/proc/loadavg: 7.96 7.98 7.86 9/360 4989
/proc/meminfo: memFree=23729964/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=417184 CPUtime=405.6 cores=0-7
/proc/4974/stat : 4974 (plingeling) S 4972 4974 4839 0 -1 4202496 209779 1252 0 0 40525 35 0 0 18 0 9 0 655200436 427196416 67360 33554432000 4194304 4417452 140736022623792 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/4974/statm: 104296 67360 180 55 0 102182 0
[pid=4974/tid=4981] ppid=4972 vsize=417184 CPUtime=50.73 cores=0-7
/proc/4974/task/4981/stat : 4981 (plingeling) R 4972 4974 4839 0 -1 4202560 21236 1252 0 0 5070 3 0 0 25 0 9 0 655200472 427196416 67360 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4974/tid=4982] ppid=4972 vsize=417184 CPUtime=50.24 cores=0-7
/proc/4974/task/4982/stat : 4982 (plingeling) R 4972 4974 4839 0 -1 4202560 29099 1252 0 0 5020 4 0 0 25 0 9 0 655200472 427196416 67360 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4974/tid=4983] ppid=4972 vsize=417184 CPUtime=50.73 cores=0-7
/proc/4974/task/4983/stat : 4983 (plingeling) R 4972 4974 4839 0 -1 4202560 14532 1252 0 0 5072 1 0 0 25 0 9 0 655200472 427196416 67360 33554432000 4194304 4417452 140736022623792 18446744073709551615 4353543 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4974/tid=4984] ppid=4972 vsize=417184 CPUtime=50.73 cores=0-7
/proc/4974/task/4984/stat : 4984 (plingeling) R 4972 4974 4839 0 -1 4202560 26171 1252 0 0 5069 4 0 0 25 0 9 0 655200472 427196416 67360 33554432000 4194304 4417452 140736022623792 18446744073709551615 4353504 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4974/tid=4985] ppid=4972 vsize=417184 CPUtime=50.73 cores=0-7
/proc/4974/task/4985/stat : 4985 (plingeling) R 4972 4974 4839 0 -1 4202560 31342 1252 0 0 5067 6 0 0 25 0 9 0 655200472 427196416 67360 33554432000 4194304 4417452 140736022623792 18446744073709551615 4366316 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4974/tid=4986] ppid=4972 vsize=417184 CPUtime=50.69 cores=0-7
/proc/4974/task/4986/stat : 4986 (plingeling) R 4972 4974 4839 0 -1 4202560 18557 1252 0 0 5066 3 0 0 25 0 9 0 655200472 427196416 67360 33554432000 4194304 4417452 140736022623792 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4974/tid=4987] ppid=4972 vsize=417184 CPUtime=50.59 cores=0-7
/proc/4974/task/4987/stat : 4987 (plingeling) R 4972 4974 4839 0 -1 4202560 15598 1252 0 0 5057 2 0 0 25 0 9 0 655200472 427196416 67360 33554432000 4194304 4417452 140736022623792 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4974/tid=4988] ppid=4972 vsize=417184 CPUtime=50.72 cores=0-7
/proc/4974/task/4988/stat : 4988 (plingeling) R 4972 4974 4839 0 -1 4202560 34522 1252 0 0 5067 5 0 0 25 0 9 0 655200472 427196416 67360 33554432000 4194304 4417452 140736022623792 18446744073709551615 4251261 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 405.6
Current children cumulated vsize (KiB) 417184

[startup+102.301 s]
/proc/loadavg: 7.90 7.96 7.86 9/360 4990

################
# More data... #
################

/proc/4974/task/4984/stat : 4984 (plingeling) R 4972 4974 4839 0 -1 4202560 2055413 1252 0 0 465875 330 0 0 25 0 9 0 655200472 1943396352 437344 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4974/tid=4985] ppid=4972 vsize=1897848 CPUtime=4661.9 cores=0-7
/proc/4974/task/4985/stat : 4985 (plingeling) R 4972 4974 4839 0 -1 4202560 1017344 1252 0 0 466024 166 0 0 25 0 9 0 655200472 1943396352 437344 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4974/tid=4986] ppid=4972 vsize=1897848 CPUtime=4622.63 cores=0-7
/proc/4974/task/4986/stat : 4986 (plingeling) R 4972 4974 4839 0 -1 4202560 1066643 1252 0 0 462093 170 0 0 25 0 9 0 655200472 1943396352 437344 33554432000 4194304 4417452 140736022623792 18446744073709551615 4357291 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4974/tid=4987] ppid=4972 vsize=1897848 CPUtime=4650.78 cores=0-7
/proc/4974/task/4987/stat : 4987 (plingeling) R 4972 4974 4839 0 -1 4202560 1128702 1252 0 0 464884 194 0 0 25 0 9 0 655200472 1943396352 437344 33554432000 4194304 4417452 140736022623792 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4974/tid=4988] ppid=4972 vsize=1897848 CPUtime=4661.85 cores=0-7
/proc/4974/task/4988/stat : 4988 (plingeling) R 4972 4974 4839 0 -1 4202560 1771824 1252 0 0 465892 293 0 0 25 0 9 0 655200472 1943396352 437344 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37239.8
Current children cumulated vsize (KiB) 1897848

[startup+4722.3 s]
/proc/loadavg: 8.09 8.05 8.00 9/360 5125
/proc/meminfo: memFree=22552844/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=1531944 CPUtime=37719 cores=0-7
/proc/4974/stat : 4974 (plingeling) S 4972 4974 4839 0 -1 4202496 11079141 1252 0 0 3770121 1782 0 0 18 0 9 0 655200436 1568710656 357132 33554432000 4194304 4417452 140736022623792 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/4974/statm: 382986 357132 180 55 0 380872 0
[pid=4974/tid=4981] ppid=4972 vsize=1531944 CPUtime=4722.02 cores=0-7
/proc/4974/task/4981/stat : 4981 (plingeling) R 4972 4974 4839 0 -1 4202560 1086155 1252 0 0 472032 170 0 0 25 0 9 0 655200472 1568710656 357132 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4974/tid=4982] ppid=4972 vsize=1531944 CPUtime=4720.07 cores=0-7
/proc/4974/task/4982/stat : 4982 (plingeling) R 4972 4974 4839 0 -1 4202560 1710347 1252 0 0 471754 253 0 0 25 0 9 0 655200472 1568710656 357132 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4974/tid=4983] ppid=4972 vsize=1531944 CPUtime=4718.02 cores=0-7
/proc/4974/task/4983/stat : 4983 (plingeling) R 4972 4974 4839 0 -1 4202560 1082933 1252 0 0 471629 173 0 0 25 0 9 0 655200472 1568710656 357132 33554432000 4194304 4417452 140736022623792 18446744073709551615 4302932 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4974/tid=4984] ppid=4972 vsize=1531944 CPUtime=4722.04 cores=0-7
/proc/4974/task/4984/stat : 4984 (plingeling) R 4972 4974 4839 0 -1 4202560 2077170 1252 0 0 471871 333 0 0 25 0 9 0 655200472 1568710656 357132 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4974/tid=4985] ppid=4972 vsize=1531944 CPUtime=4721.89 cores=0-7
/proc/4974/task/4985/stat : 4985 (plingeling) R 4972 4974 4839 0 -1 4202560 1025204 1252 0 0 472022 167 0 0 25 0 9 0 655200472 1568710656 357132 33554432000 4194304 4417452 140736022623792 18446744073709551615 4236549 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4974/tid=4986] ppid=4972 vsize=1531944 CPUtime=4682.63 cores=0-7
/proc/4974/task/4986/stat : 4986 (plingeling) R 4972 4974 4839 0 -1 4202560 1098865 1252 0 0 468084 179 0 0 25 0 9 0 655200472 1568710656 357132 33554432000 4194304 4417452 140736022623792 18446744073709551615 4365946 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4974/tid=4987] ppid=4972 vsize=1531944 CPUtime=4710.06 cores=0-7
/proc/4974/task/4987/stat : 4987 (plingeling) R 4972 4974 4839 0 -1 4202560 1148910 1252 0 0 470806 200 0 0 25 0 9 0 655200472 1568710656 357132 33554432000 4194304 4417452 140736022623792 18446744073709551615 4366105 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4974/tid=4988] ppid=4972 vsize=1531944 CPUtime=4721.86 cores=0-7
/proc/4974/task/4988/stat : 4988 (plingeling) R 4972 4974 4839 0 -1 4202560 1830835 1252 0 0 471885 301 0 0 25 0 9 0 655200472 1568710656 357132 33554432000 4194304 4417452 140736022623792 18446744073709551615 4356867 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37719
Current children cumulated vsize (KiB) 1531944

[startup+4782.3 s]
/proc/loadavg: 8.08 8.05 8.00 9/360 5127
/proc/meminfo: memFree=22520600/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=1579368 CPUtime=38198.4 cores=0-7
/proc/4974/stat : 4974 (plingeling) S 4972 4974 4839 0 -1 4202496 11314126 1252 0 0 3818015 1821 0 0 18 0 9 0 655200436 1617272832 365092 33554432000 4194304 4417452 140736022623792 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/4974/statm: 394842 365092 180 55 0 392728 0
[pid=4974/tid=4981] ppid=4972 vsize=1579368 CPUtime=4782.04 cores=0-7
/proc/4974/task/4981/stat : 4981 (plingeling) R 4972 4974 4839 0 -1 4202560 1105074 1252 0 0 478032 172 0 0 25 0 9 0 655200472 1617272832 365092 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4974/tid=4982] ppid=4972 vsize=1579368 CPUtime=4780.07 cores=0-7
/proc/4974/task/4982/stat : 4982 (plingeling) R 4972 4974 4839 0 -1 4202560 1725973 1252 0 0 477753 254 0 0 25 0 9 0 655200472 1617272832 365092 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4974/tid=4983] ppid=4972 vsize=1579368 CPUtime=4778.03 cores=0-7
/proc/4974/task/4983/stat : 4983 (plingeling) R 4972 4974 4839 0 -1 4202560 1105886 1252 0 0 477627 176 0 0 25 0 9 0 655200472 1617272832 365092 33554432000 4194304 4417452 140736022623792 18446744073709551615 4357869 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4974/tid=4984] ppid=4972 vsize=1579368 CPUtime=4782.06 cores=0-7
/proc/4974/task/4984/stat : 4984 (plingeling) R 4972 4974 4839 0 -1 4202560 2129312 1252 0 0 477864 342 0 0 25 0 9 0 655200472 1617272832 365092 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4974/tid=4985] ppid=4972 vsize=1579368 CPUtime=4781.91 cores=0-7
/proc/4974/task/4985/stat : 4985 (plingeling) R 4972 4974 4839 0 -1 4202560 1033028 1252 0 0 478021 170 0 0 25 0 9 0 655200472 1617272832 365092 33554432000 4194304 4417452 140736022623792 18446744073709551615 4357436 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4974/tid=4986] ppid=4972 vsize=1579368 CPUtime=4742.64 cores=0-7
/proc/4974/task/4986/stat : 4986 (plingeling) R 4972 4974 4839 0 -1 4202560 1098877 1252 0 0 474085 179 0 0 25 0 9 0 655200472 1617272832 365092 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4974/tid=4987] ppid=4972 vsize=1579368 CPUtime=4769.36 cores=0-7
/proc/4974/task/4987/stat : 4987 (plingeling) R 4972 4974 4839 0 -1 4202560 1169860 1252 0 0 476733 203 0 0 25 0 9 0 655200472 1617272832 365092 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4974/tid=4988] ppid=4972 vsize=1579368 CPUtime=4781.88 cores=0-7
/proc/4974/task/4988/stat : 4988 (plingeling) R 4972 4974 4839 0 -1 4202560 1927394 1252 0 0 477869 319 0 0 25 0 9 0 655200472 1617272832 365092 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38198.4
Current children cumulated vsize (KiB) 1579368

[startup+4842.3 s]
/proc/loadavg: 8.08 8.06 8.00 9/360 5129
/proc/meminfo: memFree=22720872/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=1379608 CPUtime=38677.6 cores=0-7
/proc/4974/stat : 4974 (plingeling) S 4972 4974 4839 0 -1 4202496 11481967 1252 0 0 3865902 1858 0 0 18 0 9 0 655200436 1412718592 315120 33554432000 4194304 4417452 140736022623792 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/4974/statm: 344902 315120 180 55 0 342788 0
[pid=4974/tid=4981] ppid=4972 vsize=1379608 CPUtime=4842.02 cores=0-7
/proc/4974/task/4981/stat : 4981 (plingeling) R 4972 4974 4839 0 -1 4202560 1122128 1252 0 0 484027 175 0 0 25 0 9 0 655200472 1412718592 315120 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4974/tid=4982] ppid=4972 vsize=1379608 CPUtime=4840.06 cores=0-7
/proc/4974/task/4982/stat : 4982 (plingeling) R 4972 4974 4839 0 -1 4202560 1749314 1252 0 0 483746 260 0 0 25 0 9 0 655200472 1412718592 315120 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4974/tid=4983] ppid=4972 vsize=1379608 CPUtime=4838.02 cores=0-7
/proc/4974/task/4983/stat : 4983 (plingeling) R 4972 4974 4839 0 -1 4202560 1118952 1252 0 0 483622 180 0 0 25 0 9 0 655200472 1412718592 315120 33554432000 4194304 4417452 140736022623792 18446744073709551615 4236935 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4974/tid=4984] ppid=4972 vsize=1379608 CPUtime=4842.05 cores=0-7
/proc/4974/task/4984/stat : 4984 (plingeling) R 4972 4974 4839 0 -1 4202560 2155016 1252 0 0 483856 349 0 0 25 0 9 0 655200472 1412718592 315120 33554432000 4194304 4417452 140736022623792 18446744073709551615 4237250 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4974/tid=4985] ppid=4972 vsize=1379608 CPUtime=4841.89 cores=0-7
/proc/4974/task/4985/stat : 4985 (plingeling) R 4972 4974 4839 0 -1 4202560 1058581 1252 0 0 484015 174 0 0 25 0 9 0 655200472 1412718592 315120 33554432000 4194304 4417452 140736022623792 18446744073709551615 4365042 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4974/tid=4986] ppid=4972 vsize=1379608 CPUtime=4802.63 cores=0-7
/proc/4974/task/4986/stat : 4986 (plingeling) R 4972 4974 4839 0 -1 4202560 1098877 1252 0 0 480084 179 0 0 25 0 9 0 655200472 1412718592 315120 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4974/tid=4987] ppid=4972 vsize=1379608 CPUtime=4829.07 cores=0-7
/proc/4974/task/4987/stat : 4987 (plingeling) R 4972 4974 4839 0 -1 4202560 1189543 1252 0 0 482698 209 0 0 25 0 9 0 655200472 1412718592 315120 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4974/tid=4988] ppid=4972 vsize=1379608 CPUtime=4841.44 cores=0-7
/proc/4974/task/4988/stat : 4988 (plingeling) R 4972 4974 4839 0 -1 4202560 1970834 1252 0 0 483817 327 0 0 25 0 9 0 655200472 1412718592 315120 33554432000 4194304 4417452 140736022623792 18446744073709551615 4354510 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38677.6
Current children cumulated vsize (KiB) 1379608

[startup+4902.31 s]
/proc/loadavg: 8.07 8.06 8.00 9/360 5130
/proc/meminfo: memFree=22404824/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=1723444 CPUtime=39157 cores=0-7
/proc/4974/stat : 4974 (plingeling) S 4972 4974 4839 0 -1 4202496 11683361 1252 0 0 3913805 1895 0 0 18 0 9 0 655200436 1764806656 394223 33554432000 4194304 4417452 140736022623792 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/4974/statm: 430861 394223 180 55 0 428747 0
[pid=4974/tid=4981] ppid=4972 vsize=1723444 CPUtime=4902.04 cores=0-7
/proc/4974/task/4981/stat : 4981 (plingeling) R 4972 4974 4839 0 -1 4202560 1140971 1252 0 0 490025 179 0 0 25 0 9 0 655200472 1764806656 394223 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4974/tid=4982] ppid=4972 vsize=1723444 CPUtime=4900.06 cores=0-7
/proc/4974/task/4982/stat : 4982 (plingeling) R 4972 4974 4839 0 -1 4202560 1759958 1252 0 0 489744 262 0 0 25 0 9 0 655200472 1764806656 394223 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4974/tid=4983] ppid=4972 vsize=1723444 CPUtime=4898.04 cores=0-7
/proc/4974/task/4983/stat : 4983 (plingeling) R 4972 4974 4839 0 -1 4202560 1127702 1252 0 0 489622 182 0 0 25 0 9 0 655200472 1764806656 394223 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4974/tid=4984] ppid=4972 vsize=1723444 CPUtime=4902.06 cores=0-7
/proc/4974/task/4984/stat : 4984 (plingeling) R 4972 4974 4839 0 -1 4202560 2215757 1252 0 0 489844 362 0 0 25 0 9 0 655200472 1764806656 394223 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4974/tid=4985] ppid=4972 vsize=1723444 CPUtime=4901.91 cores=0-7
/proc/4974/task/4985/stat : 4985 (plingeling) R 4972 4974 4839 0 -1 4202560 1059692 1252 0 0 490017 174 0 0 25 0 9 0 655200472 1764806656 394223 33554432000 4194304 4417452 140736022623792 18446744073709551615 4365181 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4974/tid=4986] ppid=4972 vsize=1723444 CPUtime=4862.64 cores=0-7
/proc/4974/task/4986/stat : 4986 (plingeling) R 4972 4974 4839 0 -1 4202560 1127303 1252 0 0 486082 182 0 0 25 0 9 0 655200472 1764806656 394223 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4974/tid=4987] ppid=4972 vsize=1723444 CPUtime=4889.08 cores=0-7
/proc/4974/task/4987/stat : 4987 (plingeling) R 4972 4974 4839 0 -1 4202560 1203108 1252 0 0 488698 210 0 0 25 0 9 0 655200472 1764806656 394223 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4974/tid=4988] ppid=4972 vsize=1723444 CPUtime=4900.73 cores=0-7
/proc/4974/task/4988/stat : 4988 (plingeling) R 4972 4974 4839 0 -1 4202560 2030148 1252 0 0 489736 337 0 0 25 0 9 0 655200472 1764806656 394223 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39157
Current children cumulated vsize (KiB) 1723444

[startup+4962.3 s]
/proc/loadavg: 8.02 8.05 8.00 9/360 5132
/proc/meminfo: memFree=22617896/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=1508184 CPUtime=39636.2 cores=0-7
/proc/4974/stat : 4974 (plingeling) S 4972 4974 4839 0 -1 4202496 11832552 1252 0 0 3961692 1927 0 0 18 0 9 0 655200436 1544380416 340920 33554432000 4194304 4417452 140736022623792 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/4974/statm: 377046 340920 180 55 0 374932 0
[pid=4974/tid=4981] ppid=4972 vsize=1508184 CPUtime=4962.02 cores=0-7
/proc/4974/task/4981/stat : 4981 (plingeling) R 4972 4974 4839 0 -1 4202560 1155010 1252 0 0 496022 180 0 0 25 0 9 0 655200472 1544380416 340920 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4974/tid=4982] ppid=4972 vsize=1508184 CPUtime=4960.04 cores=0-7
/proc/4974/task/4982/stat : 4982 (plingeling) R 4972 4974 4839 0 -1 4202560 1774711 1252 0 0 495739 265 0 0 25 0 9 0 655200472 1544380416 340920 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4974/tid=4983] ppid=4972 vsize=1508184 CPUtime=4958.03 cores=0-7
/proc/4974/task/4983/stat : 4983 (plingeling) R 4972 4974 4839 0 -1 4202560 1137702 1252 0 0 495618 185 0 0 25 0 9 0 655200472 1544380416 340920 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4974/tid=4984] ppid=4972 vsize=1508184 CPUtime=4962.05 cores=0-7
/proc/4974/task/4984/stat : 4984 (plingeling) R 4972 4974 4839 0 -1 4202560 2247152 1252 0 0 495836 369 0 0 25 0 9 0 655200472 1544380416 340920 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364542 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4974/tid=4985] ppid=4972 vsize=1508184 CPUtime=4961.89 cores=0-7
/proc/4974/task/4985/stat : 4985 (plingeling) R 4972 4974 4839 0 -1 4202560 1068596 1252 0 0 496014 175 0 0 25 0 9 0 655200472 1544380416 340920 33554432000 4194304 4417452 140736022623792 18446744073709551615 4366102 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4974/tid=4986] ppid=4972 vsize=1508184 CPUtime=4922.64 cores=0-7
/proc/4974/task/4986/stat : 4986 (plingeling) R 4972 4974 4839 0 -1 4202560 1157066 1252 0 0 492074 190 0 0 25 0 9 0 655200472 1544380416 340920 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4974/tid=4987] ppid=4972 vsize=1508184 CPUtime=4949.07 cores=0-7
/proc/4974/task/4987/stat : 4987 (plingeling) R 4972 4974 4839 0 -1 4202560 1211628 1252 0 0 494696 211 0 0 25 0 9 0 655200472 1544380416 340920 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4974/tid=4988] ppid=4972 vsize=1508184 CPUtime=4960.02 cores=0-7
/proc/4974/task/4988/stat : 4988 (plingeling) R 4972 4974 4839 0 -1 4202560 2061965 1252 0 0 495656 346 0 0 25 0 9 0 655200472 1544380416 340920 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39636.2
Current children cumulated vsize (KiB) 1508184



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5007.91 s]
/proc/loadavg: 8.01 8.04 8.00 9/360 5133
/proc/meminfo: memFree=22507924/32951124 swapFree=56449224/67111528
[pid=4974] ppid=4972 vsize=1613740 CPUtime=40000.6 cores=0-7
/proc/4974/stat : 4974 (plingeling) S 4972 4974 4839 0 -1 4202496 11949291 1252 0 0 3998118 1942 0 0 18 0 9 0 655200436 1652469760 368333 33554432000 4194304 4417452 140736022623792 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/4974/statm: 403435 368333 180 55 0 401321 0
[pid=4974/tid=4981] ppid=4972 vsize=1613740 CPUtime=5007.65 cores=0-7
/proc/4974/task/4981/stat : 4981 (plingeling) R 4972 4974 4839 0 -1 4202560 1173577 1252 0 0 500583 182 0 0 25 0 9 0 655200472 1652469760 368333 33554432000 4194304 4417452 140736022623792 18446744073709551615 4365088 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4974/tid=4982] ppid=4972 vsize=1613740 CPUtime=5005.65 cores=0-7
/proc/4974/task/4982/stat : 4982 (plingeling) R 4972 4974 4839 0 -1 4202560 1780588 1252 0 0 500299 266 0 0 25 0 9 0 655200472 1652469760 368333 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4974/tid=4983] ppid=4972 vsize=1613740 CPUtime=5003.66 cores=0-7
/proc/4974/task/4983/stat : 4983 (plingeling) R 4972 4974 4839 0 -1 4202560 1156889 1252 0 0 500178 188 0 0 25 0 9 0 655200472 1652469760 368333 33554432000 4194304 4417452 140736022623792 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4974/tid=4984] ppid=4972 vsize=1613740 CPUtime=5007.68 cores=0-7
/proc/4974/task/4984/stat : 4984 (plingeling) R 4972 4974 4839 0 -1 4202560 2275807 1252 0 0 500396 372 0 0 25 0 9 0 655200472 1652469760 368333 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4974/tid=4985] ppid=4972 vsize=1613740 CPUtime=5007.52 cores=0-7
/proc/4974/task/4985/stat : 4985 (plingeling) R 4972 4974 4839 0 -1 4202560 1078493 1252 0 0 500575 177 0 0 25 0 9 0 655200472 1652469760 368333 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4974/tid=4986] ppid=4972 vsize=1613740 CPUtime=4968.26 cores=0-7
/proc/4974/task/4986/stat : 4986 (plingeling) R 4972 4974 4839 0 -1 4202560 1172031 1252 0 0 496635 191 0 0 25 0 9 0 655200472 1652469760 368333 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4974/tid=4987] ppid=4972 vsize=1613740 CPUtime=4994.7 cores=0-7
/proc/4974/task/4987/stat : 4987 (plingeling) R 4972 4974 4839 0 -1 4202560 1216354 1252 0 0 499258 212 0 0 25 0 9 0 655200472 1652469760 368333 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4974/tid=4988] ppid=4972 vsize=1613740 CPUtime=5005.09 cores=0-7
/proc/4974/task/4988/stat : 4988 (plingeling) R 4972 4974 4839 0 -1 4202560 2076830 1252 0 0 500161 348 0 0 25 0 9 0 655200472 1652469760 368333 33554432000 4194304 4417452 140736022623792 18446744073709551615 4364850 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 1613740

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5007.99
CPU time (s): 40000.8
CPU user time (s): 39981.3
CPU system time (s): 19.485
CPU usage (%): 798.738
Max. virtual memory (cumulated for all children) (KiB): 2033564

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39981.3
system time used= 19.485
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 11950543
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 437
involuntary context switches= 141927

runsolver used 17.4283 second user time and 41.7636 second system time

The end

Launcher Data 

Begin job on node106 at 2011-05-15 07:06:40
IDJOB=3372161
IDBENCH=20523
IDSOLVER=1851
FILE ID=node106/3372161-1305436000
RUNJOBID= node106-1305435364-4853
PBS_JOBID= 13324735
Free space on /tmp= 73464 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT07/industrial/IBM_FV_2002_13_rule_1/SAT_dat.k45.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372161-1305436000/watcher-3372161-1305436000 -o /tmp/evaluation-result-3372161-1305436000/solver-3372161-1305436000 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3372161-1305436000.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 8a76c8add64173ae39149aae572a02d9
RANDOM SEED=242326358

node106.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.20
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23981244 kB
Buffers:        279740 kB
Cached:        5708304 kB
SwapCached:      27544 kB
Active:         761396 kB
Inactive:      7977696 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23981244 kB
SwapTotal:    67111528 kB
SwapFree:     56449224 kB
Dirty:            3668 kB
Writeback:           0 kB
AnonPages:     2732080 kB
Mapped:           9636 kB
Slab:           114092 kB
PageTables:      47008 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 74988884 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73464 MiB
End job on node106 at 2011-05-15 08:30:10