Trace number 3370999

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.47 1203.74

General information on the benchmark

NameSAT05/crafted/jarvisalo05/mod2c-rand3bip-sat/
mod2c-rand3bip-sat-240-2.shuffled-as.sat05-2519.cnf
MD5SUMb6a88c8ae73b20cfc873207d1a0caa09
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark139.097
Satisfiable
(Un)Satisfiability was proved
Number of variables338
Number of clauses2376
Sum of the clauses size10480
Maximum clause length6
Minimum clause length4
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 41520
Number of clauses of size 5736
Number of clauses of size over 5120

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3370999-1305431508/watcher-3370999-1305431508 -o /tmp/evaluation-result-3370999-1305431508/solver-3370999-1305431508 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3370999-1305431508.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.85 0.97 1.03 2/365 16491
/proc/meminfo: memFree=18263724/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=116544 CPUtime=0.01 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 643 1259 0 0 1 0 0 0 18 0 5 0 733381457 119341056 568 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 29136 568 176 55 0 27022 0
[pid=16489/tid=16502] ppid=16486 vsize=116544 CPUtime=0 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 112 1259 0 0 0 0 0 0 19 0 5 0 733381457 119341056 568 33554432000 4194304 4417452 140736052497776 18446744073709551615 4278143 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=116544 CPUtime=0 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 37 1259 0 0 0 0 0 0 19 0 5 0 733381457 119341056 568 33554432000 4194304 4417452 140736052497776 18446744073709551615 4278386 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=116544 CPUtime=0 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 39 1259 0 0 0 0 0 0 20 0 5 0 733381457 119341056 568 33554432000 4194304 4417452 140736052497776 18446744073709551615 4329568 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=116544 CPUtime=0 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 4 1259 0 0 0 0 0 0 21 0 5 0 733381457 119341056 568 33554432000 4194304 4417452 140736052497776 18446744073709551615 224285030894 0 0 4096 17442 0 0 0 -1 5 0 0 0

[startup+0.114272 s]
/proc/loadavg: 0.85 0.97 1.03 2/365 16491
/proc/meminfo: memFree=18263724/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=117680 CPUtime=0.31 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 945 1259 0 0 31 0 0 0 18 0 5 0 733381457 120504320 868 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 29420 868 179 55 0 27306 0
[pid=16489/tid=16502] ppid=16486 vsize=117680 CPUtime=0.08 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 119 1259 0 0 8 0 0 0 19 0 5 0 733381457 120504320 868 33554432000 4194304 4417452 140736052497776 18446744073709551615 4252447 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=117680 CPUtime=0.1 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 82 1259 0 0 10 0 0 0 20 0 5 0 733381457 120504320 868 33554432000 4194304 4417452 140736052497776 18446744073709551615 4273022 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=117680 CPUtime=0.1 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 224 1259 0 0 10 0 0 0 20 0 5 0 733381457 120504320 868 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=117680 CPUtime=0.02 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 69 1259 0 0 2 0 0 0 20 0 5 0 733381457 120504320 868 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 0.31
Current children cumulated vsize (KiB) 117680

[startup+0.214255 s]
/proc/loadavg: 0.85 0.97 1.03 2/365 16491
/proc/meminfo: memFree=18263724/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=118028 CPUtime=0.61 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 1069 1259 0 0 61 0 0 0 18 0 5 0 733381457 120860672 992 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 29507 992 179 55 0 27393 0
[pid=16489/tid=16502] ppid=16486 vsize=118028 CPUtime=0.18 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 119 1259 0 0 18 0 0 0 19 0 5 0 733381457 120860672 992 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=118028 CPUtime=0.2 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 119 1259 0 0 20 0 0 0 20 0 5 0 733381457 120860672 992 33554432000 4194304 4417452 140736052497776 18446744073709551615 4356960 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=118028 CPUtime=0.2 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 311 1259 0 0 20 0 0 0 21 0 5 0 733381457 120860672 992 33554432000 4194304 4417452 140736052497776 18446744073709551615 4237237 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=118028 CPUtime=0.02 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 69 1259 0 0 2 0 0 0 20 0 5 0 733381457 120860672 992 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 0.61
Current children cumulated vsize (KiB) 118028

[startup+0.300242 s]
/proc/loadavg: 0.85 0.97 1.03 2/365 16491
/proc/meminfo: memFree=18263724/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=119088 CPUtime=0.92 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 1487 1259 0 0 92 0 0 0 18 0 5 0 733381457 121946112 1263 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 29772 1263 179 55 0 27658 0
[pid=16489/tid=16502] ppid=16486 vsize=119088 CPUtime=0.25 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 288 1259 0 0 25 0 0 0 20 0 5 0 733381457 121946112 1263 33554432000 4194304 4417452 140736052497776 18446744073709551615 4354082 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=119088 CPUtime=0.28 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 197 1259 0 0 28 0 0 0 20 0 5 0 733381457 121946112 1263 33554432000 4194304 4417452 140736052497776 18446744073709551615 4277785 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=119088 CPUtime=0.28 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 364 1259 0 0 28 0 0 0 21 0 5 0 733381457 121946112 1263 33554432000 4194304 4417452 140736052497776 18446744073709551615 4366450 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=119088 CPUtime=0.08 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 187 1259 0 0 8 0 0 0 20 0 5 0 733381457 121946112 1263 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 0.92
Current children cumulated vsize (KiB) 119088

[startup+0.700175 s]
/proc/loadavg: 0.85 0.97 1.03 2/365 16491
/proc/meminfo: memFree=18263724/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=121768 CPUtime=2.52 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 3421 1259 0 0 252 0 0 0 18 0 5 0 733381457 124690432 1961 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 30442 1961 180 55 0 28328 0
[pid=16489/tid=16502] ppid=16486 vsize=121768 CPUtime=0.65 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 715 1259 0 0 65 0 0 0 20 0 5 0 733381457 124690432 1961 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=121768 CPUtime=0.68 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 695 1259 0 0 68 0 0 0 21 0 5 0 733381457 124690432 1961 33554432000 4194304 4417452 140736052497776 18446744073709551615 4353661 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=121768 CPUtime=0.68 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 812 1259 0 0 68 0 0 0 22 0 5 0 733381457 124690432 1961 33554432000 4194304 4417452 140736052497776 18446744073709551615 4250936 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=121768 CPUtime=0.48 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 748 1259 0 0 48 0 0 0 21 0 5 0 733381457 124690432 1961 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2.52
Current children cumulated vsize (KiB) 121768

[startup+1.50104 s]
/proc/loadavg: 0.85 0.97 1.03 9/375 16511
/proc/meminfo: memFree=18242208/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=126932 CPUtime=5.71 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 6341 1259 0 0 570 1 0 0 18 0 5 0 733381457 129978368 3225 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 31733 3225 180 55 0 29619 0
[pid=16489/tid=16502] ppid=16486 vsize=126932 CPUtime=1.45 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 1351 1259 0 0 145 0 0 0 23 0 5 0 733381457 129978368 3225 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=126932 CPUtime=1.47 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 1321 1259 0 0 147 0 0 0 23 0 5 0 733381457 129978368 3225 33554432000 4194304 4417452 140736052497776 18446744073709551615 4252487 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=126932 CPUtime=1.48 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 1690 1259 0 0 148 0 0 0 25 0 5 0 733381457 129978368 3225 33554432000 4194304 4417452 140736052497776 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=126932 CPUtime=1.28 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 1528 1259 0 0 128 0 0 0 25 0 5 0 733381457 129978368 3225 33554432000 4194304 4417452 140736052497776 18446744073709551615 4366813 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 5.71
Current children cumulated vsize (KiB) 126932

[startup+3.10078 s]
/proc/loadavg: 0.85 0.97 1.03 9/375 16511
/proc/meminfo: memFree=18231548/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=134636 CPUtime=12.1 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 12005 1259 0 0 1208 2 0 0 18 0 5 0 733381457 137867264 5117 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 33659 5117 180 55 0 31545 0
[pid=16489/tid=16502] ppid=16486 vsize=134636 CPUtime=3.05 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 2881 1259 0 0 305 0 0 0 25 0 5 0 733381457 137867264 5117 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=134636 CPUtime=3.05 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 2636 1259 0 0 305 0 0 0 25 0 5 0 733381457 137867264 5117 33554432000 4194304 4417452 140736052497776 18446744073709551615 4354418 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=134636 CPUtime=3.08 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 2753 1259 0 0 308 0 0 0 25 0 5 0 733381457 137867264 5117 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=134636 CPUtime=2.88 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 3284 1259 0 0 288 0 0 0 25 0 5 0 733381457 137867264 5117 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364780 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 12.1
Current children cumulated vsize (KiB) 134636

[startup+6.30025 s]
/proc/loadavg: 1.42 1.08 1.06 9/375 16513
/proc/meminfo: memFree=18204644/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=143464 CPUtime=24.86 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 19154 1259 0 0 2483 3 0 0 18 0 5 0 733381457 146907136 7324 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 35866 7324 180 55 0 33752 0
[pid=16489/tid=16502] ppid=16486 vsize=143464 CPUtime=6.24 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 4781 1259 0 0 624 0 0 0 25 0 5 0 733381457 146907136 7324 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=143464 CPUtime=6.21 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 4389 1259 0 0 621 0 0 0 25 0 5 0 733381457 146907136 7324 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364873 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=143464 CPUtime=6.28 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 4748 1259 0 0 628 0 0 0 25 0 5 0 733381457 146907136 7324 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364616 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=143464 CPUtime=6.08 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 4785 1259 0 0 607 1 0 0 25 0 5 0 733381457 146907136 7324 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 24.86
Current children cumulated vsize (KiB) 143464

[startup+12.7012 s]
/proc/loadavg: 2.03 1.21 1.10 9/375 16513
/proc/meminfo: memFree=18161724/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=169420 CPUtime=50.38 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 30351 1259 0 0 5033 5 0 0 18 0 5 0 733381457 173486080 13843 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 42355 13843 180 55 0 40241 0
[pid=16489/tid=16502] ppid=16486 vsize=169420 CPUtime=12.65 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 7778 1259 0 0 1264 1 0 0 25 0 5 0 733381457 173486080 13843 33554432000 4194304 4417452 140736052497776 18446744073709551615 4365434 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=169420 CPUtime=12.54 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 7501 1259 0 0 1253 1 0 0 25 0 5 0 733381457 173486080 13843 33554432000 4194304 4417452 140736052497776 18446744073709551615 4358341 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=169420 CPUtime=12.68 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 7248 1259 0 0 1268 0 0 0 25 0 5 0 733381457 173486080 13843 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=169420 CPUtime=12.48 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 7373 1259 0 0 1247 1 0 0 25 0 5 0 733381457 173486080 13843 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 50.38
Current children cumulated vsize (KiB) 169420

[startup+25.501 s]
/proc/loadavg: 3.35 1.55 1.21 10/375 16513
/proc/meminfo: memFree=18104448/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=188208 CPUtime=101.42 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 52930 1259 0 0 10133 9 0 0 18 0 5 0 733381457 192724992 18510 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 47052 18510 180 55 0 44938 0
[pid=16489/tid=16502] ppid=16486 vsize=188208 CPUtime=25.45 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 13427 1259 0 0 2543 2 0 0 25 0 5 0 733381457 192724992 18510 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=188208 CPUtime=25.18 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 13333 1259 0 0 2516 2 0 0 25 0 5 0 733381457 192724992 18510 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=188208 CPUtime=25.48 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 13191 1259 0 0 2547 1 0 0 25 0 5 0 733381457 192724992 18510 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=188208 CPUtime=25.28 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 12528 1259 0 0 2526 2 0 0 25 0 5 0 733381457 192724992 18510 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 101.42
Current children cumulated vsize (KiB) 188208

[startup+51.1008 s]
/proc/loadavg: 5.01 2.08 1.40 10/375 16513
/proc/meminfo: memFree=18071408/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=211908 CPUtime=203.52 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 73137 1259 0 0 20340 12 0 0 18 0 5 0 733381457 216993792 24476 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 52977 24476 180 55 0 50863 0
[pid=16489/tid=16502] ppid=16486 vsize=211908 CPUtime=51.05 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 19046 1259 0 0 5102 3 0 0 25 0 5 0 733381457 216993792 24476 33554432000 4194304 4417452 140736052497776 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=211908 CPUtime=50.48 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 20370 1259 0 0 5045 3 0 0 25 0 5 0 733381457 216993792 24476 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=211908 CPUtime=51.08 cores=1,3,5,7

################
# More data... #
################

[pid=16489/tid=16503] ppid=16486 vsize=506052 CPUtime=693.99 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 74557 1259 0 0 69387 12 0 0 25 0 5 0 733381457 518197248 97984 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=506052 CPUtime=702.29 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 71069 1259 0 0 70217 12 0 0 25 0 5 0 733381457 518197248 97984 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=506052 CPUtime=702.09 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 70080 1259 0 0 70197 12 0 0 25 0 5 0 733381457 518197248 97984 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2800.67
Current children cumulated vsize (KiB) 506052

[startup+762.301 s]
/proc/loadavg: 4.07 5.20 4.00 5/365 16559
/proc/meminfo: memFree=17884308/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=491284 CPUtime=3040.01 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 310980 1259 0 0 303946 55 0 0 18 0 5 0 733381457 503074816 94292 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 122821 94292 180 55 0 120707 0
[pid=16489/tid=16502] ppid=16486 vsize=491284 CPUtime=762.27 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 90323 1259 0 0 76212 15 0 0 25 0 5 0 733381457 503074816 94292 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=491284 CPUtime=753.29 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 74557 1259 0 0 75317 12 0 0 25 0 5 0 733381457 503074816 94292 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=491284 CPUtime=762.31 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 71069 1259 0 0 76219 12 0 0 25 0 5 0 733381457 503074816 94292 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=491284 CPUtime=762.11 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 74580 1259 0 0 76197 14 0 0 25 0 5 0 733381457 503074816 94292 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3040.01
Current children cumulated vsize (KiB) 491284

[startup+822.301 s]
/proc/loadavg: 4.07 5.00 4.00 5/365 16559
/proc/meminfo: memFree=17884148/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=491892 CPUtime=3279.26 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 311132 1259 0 0 327871 55 0 0 18 0 5 0 733381457 503697408 94444 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 122973 94444 180 55 0 120859 0
[pid=16489/tid=16502] ppid=16486 vsize=491892 CPUtime=822.26 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 90323 1259 0 0 82211 15 0 0 25 0 5 0 733381457 503697408 94444 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=491892 CPUtime=812.57 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 74709 1259 0 0 81245 12 0 0 25 0 5 0 733381457 503697408 94444 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=491892 CPUtime=822.3 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 71069 1259 0 0 82218 12 0 0 25 0 5 0 733381457 503697408 94444 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=491892 CPUtime=822.1 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 74580 1259 0 0 82196 14 0 0 25 0 5 0 733381457 503697408 94444 33554432000 4194304 4417452 140736052497776 18446744073709551615 4365004 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3279.26
Current children cumulated vsize (KiB) 491892

[startup+882.314 s]
/proc/loadavg: 4.02 4.81 4.00 5/365 16561
/proc/meminfo: memFree=17846168/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=529844 CPUtime=3518.59 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 337004 1259 0 0 351798 61 0 0 18 0 5 0 733381457 542560256 103932 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 132461 103932 180 55 0 130347 0
[pid=16489/tid=16502] ppid=16486 vsize=529844 CPUtime=882.28 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 104040 1259 0 0 88209 19 0 0 25 0 5 0 733381457 542560256 103932 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364716 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=529844 CPUtime=871.88 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 82901 1259 0 0 87173 15 0 0 25 0 5 0 733381457 542560256 103932 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=529844 CPUtime=882.3 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 75032 1259 0 0 88218 12 0 0 25 0 5 0 733381457 542560256 103932 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364637 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=529844 CPUtime=882.11 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 74580 1259 0 0 88197 14 0 0 25 0 5 0 733381457 542560256 103932 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3518.59
Current children cumulated vsize (KiB) 529844

[startup+942.301 s]
/proc/loadavg: 4.05 4.67 4.00 5/365 16563
/proc/meminfo: memFree=17812896/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=562612 CPUtime=3757.85 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 353388 1259 0 0 375719 66 0 0 18 0 5 0 733381457 576114688 112124 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 140653 112124 180 55 0 138539 0
[pid=16489/tid=16502] ppid=16486 vsize=562612 CPUtime=942.26 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 104040 1259 0 0 94207 19 0 0 25 0 5 0 733381457 576114688 112124 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=562612 CPUtime=931.16 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 82901 1259 0 0 93101 15 0 0 25 0 5 0 733381457 576114688 112124 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=562612 CPUtime=942.29 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 83224 1259 0 0 94215 14 0 0 25 0 5 0 733381457 576114688 112124 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=562612 CPUtime=942.1 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 82772 1259 0 0 94194 16 0 0 25 0 5 0 733381457 576114688 112124 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3757.85
Current children cumulated vsize (KiB) 562612

[startup+1002.3 s]
/proc/loadavg: 4.02 4.55 4.00 5/365 16565
/proc/meminfo: memFree=17845224/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=529844 CPUtime=3997.14 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 353388 1259 0 0 399648 66 0 0 18 0 5 0 733381457 542560256 103932 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 132461 103932 180 55 0 130347 0
[pid=16489/tid=16502] ppid=16486 vsize=529844 CPUtime=1002.27 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 104040 1259 0 0 100208 19 0 0 25 0 5 0 733381457 542560256 103932 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=529844 CPUtime=990.45 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 82901 1259 0 0 99030 15 0 0 25 0 5 0 733381457 542560256 103932 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=529844 CPUtime=1002.29 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 83224 1259 0 0 100215 14 0 0 25 0 5 0 733381457 542560256 103932 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=529844 CPUtime=1002.1 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 82772 1259 0 0 100194 16 0 0 25 0 5 0 733381457 542560256 103932 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3997.14
Current children cumulated vsize (KiB) 529844

[startup+1062.3 s]
/proc/loadavg: 4.04 4.46 4.00 5/365 16567
/proc/meminfo: memFree=17768564/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=607112 CPUtime=4236.44 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 396444 1259 0 0 423571 73 0 0 18 0 5 0 733381457 621682688 123250 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 151778 123250 180 55 0 149664 0
[pid=16489/tid=16502] ppid=16486 vsize=607112 CPUtime=1062.27 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 117835 1259 0 0 106205 22 0 0 25 0 5 0 733381457 621682688 123250 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=607112 CPUtime=1049.74 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 101984 1259 0 0 104957 17 0 0 25 0 5 0 733381457 621682688 123250 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364657 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=607112 CPUtime=1062.29 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 83224 1259 0 0 106215 14 0 0 25 0 5 0 733381457 621682688 123250 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=607112 CPUtime=1062.1 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 92950 1259 0 0 106192 18 0 0 25 0 5 0 733381457 621682688 123250 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4236.44
Current children cumulated vsize (KiB) 607112

[startup+1122.3 s]
/proc/loadavg: 4.12 4.41 4.01 5/365 16567
/proc/meminfo: memFree=17671068/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=705416 CPUtime=4475.73 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 421020 1259 0 0 447495 78 0 0 18 0 5 0 733381457 722345984 147826 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 176354 147826 180 55 0 174240 0
[pid=16489/tid=16502] ppid=16486 vsize=705416 CPUtime=1122.26 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 126027 1259 0 0 112203 23 0 0 25 0 5 0 733381457 722345984 147826 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=705416 CPUtime=1109.03 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 101984 1259 0 0 110886 17 0 0 25 0 5 0 733381457 722345984 147826 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364885 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=705416 CPUtime=1122.3 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 91416 1259 0 0 112214 16 0 0 25 0 5 0 733381457 722345984 147826 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=705416 CPUtime=1122.1 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 101142 1259 0 0 112190 20 0 0 25 0 5 0 733381457 722345984 147826 33554432000 4194304 4417452 140736052497776 18446744073709551615 4366385 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4475.73
Current children cumulated vsize (KiB) 705416

[startup+1182.3 s]
/proc/loadavg: 4.12 4.35 4.00 5/365 16569
/proc/meminfo: memFree=17686412/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=690188 CPUtime=4715.02 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 438343 1259 0 0 471421 81 0 0 18 0 5 0 733381457 706752512 144019 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 172547 144019 180 55 0 170433 0
[pid=16489/tid=16502] ppid=16486 vsize=690188 CPUtime=1182.26 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 126027 1259 0 0 118203 23 0 0 25 0 5 0 733381457 706752512 144019 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=690188 CPUtime=1168.33 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 101984 1259 0 0 116816 17 0 0 25 0 5 0 733381457 706752512 144019 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=690188 CPUtime=1182.3 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 108739 1259 0 0 118211 19 0 0 25 0 5 0 733381457 706752512 144019 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=690188 CPUtime=1182.1 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 101142 1259 0 0 118190 20 0 0 25 0 5 0 733381457 706752512 144019 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4715.02
Current children cumulated vsize (KiB) 690188



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.7 s]
/proc/loadavg: 4.08 4.32 4.00 5/365 16569
/proc/meminfo: memFree=17686264/32951124 swapFree=45193364/67111528
[pid=16489] ppid=16486 vsize=690188 CPUtime=4800.4 cores=1,3,5,7
/proc/16489/stat : 16489 (plingeling) S 16486 16489 6932 0 -1 4202496 438343 1259 0 0 479959 81 0 0 18 0 5 0 733381457 706752512 144019 33554432000 4194304 4417452 140736052497776 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16489/statm: 172547 144019 180 55 0 170433 0
[pid=16489/tid=16502] ppid=16486 vsize=690188 CPUtime=1203.68 cores=1,3,5,7
/proc/16489/task/16502/stat : 16502 (plingeling) R 16486 16489 6932 0 -1 4202560 126027 1259 0 0 120345 23 0 0 25 0 5 0 733381457 706752512 144019 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16489/tid=16503] ppid=16486 vsize=690188 CPUtime=1189.47 cores=1,3,5,7
/proc/16489/task/16503/stat : 16503 (plingeling) R 16486 16489 6932 0 -1 4202560 101984 1259 0 0 118930 17 0 0 25 0 5 0 733381457 706752512 144019 33554432000 4194304 4417452 140736052497776 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16489/tid=16504] ppid=16486 vsize=690188 CPUtime=1203.71 cores=1,3,5,7
/proc/16489/task/16504/stat : 16504 (plingeling) R 16486 16489 6932 0 -1 4202560 108739 1259 0 0 120352 19 0 0 25 0 5 0 733381457 706752512 144019 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16489/tid=16505] ppid=16486 vsize=690188 CPUtime=1203.51 cores=1,3,5,7
/proc/16489/task/16505/stat : 16505 (plingeling) R 16486 16489 6932 0 -1 4202560 101142 1259 0 0 120331 20 0 0 25 0 5 0 733381457 706752512 144019 33554432000 4194304 4417452 140736052497776 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.4
Current children cumulated vsize (KiB) 690188

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.74
CPU time (s): 4800.47
CPU user time (s): 4799.62
CPU system time (s): 0.846871
CPU usage (%): 398.795
Max. virtual memory (cumulated for all children) (KiB): 739108

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4799.62
system time used= 0.846871
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 439602
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 356
involuntary context switches= 37599

runsolver used 3.80042 second user time and 10.4484 second system time

The end

Launcher Data 

Begin job on node118 at 2011-05-15 05:51:48
IDJOB=3370999
IDBENCH=85362
IDSOLVER=1848
FILE ID=node118/3370999-1305431508
RUNJOBID= node118-1305431508-16473
PBS_JOBID= 13324586
Free space on /tmp= 73328 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT05/crafted/jarvisalo05/mod2c-rand3bip-sat/mod2c-rand3bip-sat-240-2.shuffled-as.sat05-2519.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3370999-1305431508/watcher-3370999-1305431508 -o /tmp/evaluation-result-3370999-1305431508/solver-3370999-1305431508 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3370999-1305431508.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b6a88c8ae73b20cfc873207d1a0caa09
RANDOM SEED=1679046710

node118.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5374.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      18264284 kB
Buffers:        285804 kB
Cached:        4405028 kB
SwapCached:      22084 kB
Active:       11777292 kB
Inactive:      2607816 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      18264284 kB
SwapTotal:    67111528 kB
SwapFree:     45193364 kB
Dirty:             496 kB
Writeback:           8 kB
AnonPages:     9685228 kB
Mapped:          14524 kB
Slab:           147080 kB
PageTables:      89844 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 88629096 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73328 MiB
End job on node118 at 2011-05-15 06:11:54