Trace number 3370997

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.27 1203.98

General information on the benchmark

NameSAT04/handmade/kexu/frb-0.8/
frb45-21-1.used-as.sat04-883.cnf
MD5SUMd30b88c897dd41570f0430a76eb70b55
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark1.36879
Satisfiable
(Un)Satisfiability was proved
Number of variables945
Number of clauses61855
Sum of the clauses size124565
Maximum clause length21
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 261810
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3370997-1305430922/watcher-3370997-1305430922 -o /tmp/evaluation-result-3370997-1305430922/solver-3370997-1305430922 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3370997-1305430922.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.16 8.07 6.36 5/387 21156
/proc/meminfo: memFree=14685460/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=10188 CPUtime=0.01 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) R 21154 21156 14218 0 -1 4202496 644 1259 0 0 1 0 0 0 25 0 1 0 733322839 10432512 570 33554432000 4194304 4417452 140737470276656 18446744073709551615 216735236522 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/21156/statm: 2611 584 148 55 0 497 0

[startup+0.147835 s]
/proc/loadavg: 8.16 8.07 6.36 5/387 21156
/proc/meminfo: memFree=14685460/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=62696 CPUtime=0.45 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 4440 1259 0 0 45 0 0 0 25 0 5 0 733322839 64200704 3394 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 15674 3394 176 55 0 13560 0
Current children cumulated CPU time (s) 0.45
Current children cumulated vsize (KiB) 62696

[startup+0.200824 s]
/proc/loadavg: 8.16 8.07 6.36 5/387 21156
/proc/meminfo: memFree=14685460/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=63016 CPUtime=0.66 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 4520 1259 0 0 66 0 0 0 25 0 5 0 733322839 64528384 3474 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 15754 3474 176 55 0 13640 0
Current children cumulated CPU time (s) 0.66
Current children cumulated vsize (KiB) 63016

[startup+0.300808 s]
/proc/loadavg: 8.16 8.07 6.36 5/387 21156
/proc/meminfo: memFree=14685460/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=64188 CPUtime=1.06 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 5079 1259 0 0 106 0 0 0 25 0 5 0 733322839 65728512 3833 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 16047 3833 179 55 0 13933 0
Current children cumulated CPU time (s) 1.06
Current children cumulated vsize (KiB) 64188

[startup+0.700737 s]
/proc/loadavg: 8.16 8.07 6.36 5/387 21156
/proc/meminfo: memFree=14685460/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=68476 CPUtime=2.65 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 6916 1259 0 0 265 0 0 0 25 0 5 0 733322839 70119424 4905 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 17119 4905 179 55 0 15005 0
Current children cumulated CPU time (s) 2.65
Current children cumulated vsize (KiB) 68476

[startup+1.5006 s]
/proc/loadavg: 8.15 8.06 6.37 9/392 21166
/proc/meminfo: memFree=14665092/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=71512 CPUtime=5.85 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 10738 1259 0 0 584 1 0 0 25 0 5 0 733322839 73228288 5664 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 17878 5664 179 55 0 15764 0
[pid=21156/tid=21163] ppid=21154 vsize=71512 CPUtime=1.45 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 1243 1259 0 0 145 0 0 0 25 0 5 0 733322843 73228288 5664 33554432000 4194304 4417452 140737470276656 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=71512 CPUtime=1.45 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 430 1259 0 0 145 0 0 0 25 0 5 0 733322843 73228288 5664 33554432000 4194304 4417452 140737470276656 18446744073709551615 4249143 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=71512 CPUtime=1.45 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 801 1259 0 0 145 0 0 0 25 0 5 0 733322843 73228288 5664 33554432000 4194304 4417452 140737470276656 18446744073709551615 4365023 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=71512 CPUtime=1.43 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 5751 1259 0 0 143 0 0 0 25 0 5 0 733322843 73228288 5664 33554432000 4194304 4417452 140737470276656 18446744073709551615 4250532 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 5.85
Current children cumulated vsize (KiB) 71512

[startup+3.10032 s]
/proc/loadavg: 8.15 8.06 6.37 9/392 21166
/proc/meminfo: memFree=14668072/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=85648 CPUtime=12.23 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 16129 1259 0 0 1221 2 0 0 25 0 5 0 733322839 87703552 9198 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 21412 9198 179 55 0 19298 0
[pid=21156/tid=21163] ppid=21154 vsize=85648 CPUtime=3.05 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 1243 1259 0 0 305 0 0 0 25 0 5 0 733322843 87703552 9198 33554432000 4194304 4417452 140737470276656 18446744073709551615 4336683 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=85648 CPUtime=3.05 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 430 1259 0 0 305 0 0 0 25 0 5 0 733322843 87703552 9198 33554432000 4194304 4417452 140737470276656 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=85648 CPUtime=3.05 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 1322 1259 0 0 305 0 0 0 25 0 5 0 733322843 87703552 9198 33554432000 4194304 4417452 140737470276656 18446744073709551615 4232808 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=85648 CPUtime=3.02 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 10621 1259 0 0 301 1 0 0 25 0 5 0 733322843 87703552 9198 33554432000 4194304 4417452 140737470276656 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 12.23
Current children cumulated vsize (KiB) 85648

[startup+6.30076 s]
/proc/loadavg: 8.14 8.06 6.38 10/392 21166
/proc/meminfo: memFree=14652820/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=89156 CPUtime=24.99 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 24671 1259 0 0 2496 3 0 0 25 0 5 0 733322839 91295744 10075 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 22289 10075 179 55 0 20175 0
[pid=21156/tid=21163] ppid=21154 vsize=89156 CPUtime=6.25 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 1738 1259 0 0 625 0 0 0 25 0 5 0 733322843 91295744 10075 33554432000 4194304 4417452 140737470276656 18446744073709551615 4353572 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=89156 CPUtime=6.25 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 1421 1259 0 0 625 0 0 0 25 0 5 0 733322843 91295744 10075 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=89156 CPUtime=6.25 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 2518 1259 0 0 625 0 0 0 25 0 5 0 733322843 91295744 10075 33554432000 4194304 4417452 140737470276656 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=89156 CPUtime=6.18 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 16481 1259 0 0 616 2 0 0 25 0 5 0 733322843 91295744 10075 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.99
Current children cumulated vsize (KiB) 89156

[startup+12.7006 s]
/proc/loadavg: 8.13 8.06 6.38 10/392 21166
/proc/meminfo: memFree=14606704/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=115008 CPUtime=50.51 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 38111 1259 0 0 5046 5 0 0 25 0 5 0 733322839 117768192 16538 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 28752 16538 179 55 0 26638 0
[pid=21156/tid=21163] ppid=21154 vsize=115008 CPUtime=12.65 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 3096 1259 0 0 1265 0 0 0 25 0 5 0 733322843 117768192 16538 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364805 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=115008 CPUtime=12.65 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 2007 1259 0 0 1265 0 0 0 25 0 5 0 733322843 117768192 16538 33554432000 4194304 4417452 140737470276656 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=115008 CPUtime=12.65 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 3524 1259 0 0 1265 0 0 0 25 0 5 0 733322843 117768192 16538 33554432000 4194304 4417452 140737470276656 18446744073709551615 4249186 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=115008 CPUtime=12.5 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 26971 1259 0 0 1246 4 0 0 25 0 5 0 733322843 117768192 16538 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.51
Current children cumulated vsize (KiB) 115008

[startup+25.5004 s]
/proc/loadavg: 8.10 8.06 6.41 9/392 21167
/proc/meminfo: memFree=14555960/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=158700 CPUtime=101.55 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 70013 1259 0 0 10145 10 0 0 25 0 5 0 733322839 162508800 27461 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 39675 27461 179 55 0 37561 0
[pid=21156/tid=21163] ppid=21154 vsize=158700 CPUtime=25.45 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 6439 1259 0 0 2545 0 0 0 25 0 5 0 733322843 162508800 27461 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=158700 CPUtime=25.44 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 5112 1259 0 0 2544 0 0 0 25 0 5 0 733322843 162508800 27461 33554432000 4194304 4417452 140737470276656 18446744073709551615 4366646 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=158700 CPUtime=25.45 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 10625 1259 0 0 2544 1 0 0 25 0 5 0 733322843 162508800 27461 33554432000 4194304 4417452 140737470276656 18446744073709551615 4366491 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=158700 CPUtime=25.14 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 45324 1259 0 0 2507 7 0 0 25 0 5 0 733322843 162508800 27461 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.55
Current children cumulated vsize (KiB) 158700

[startup+51.1009 s]
/proc/loadavg: 8.13 8.07 6.46 10/392 21167
/proc/meminfo: memFree=14539312/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=185836 CPUtime=203.63 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 126199 1259 0 0 20345 18 0 0 25 0 5 0 733322839 190296064 34246 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 46459 34246 180 55 0 44345 0
[pid=21156/tid=21163] ppid=21154 vsize=185836 CPUtime=51.03 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 16072 1259 0 0 5102 1 0 0 25 0 5 0 733322843 190296064 34246 33554432000 4194304 4417452 140737470276656 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=185836 CPUtime=51.04 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 18484 1259 0 0 5102 2 0 0 25 0 5 0 733322843 190296064 34246 33554432000 4194304 4417452 140737470276656 18446744073709551615 4250936 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=185836 CPUtime=51.05 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 20869 1259 0 0 5103 2 0 0 25 0 5 0 733322843 190296064 34246 33554432000 4194304 4417452 140737470276656 18446744073709551615 4366197 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=185836 CPUtime=50.44 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 68261 1259 0 0 5033 11 0 0 25 0 5 0 733322843 190296064 34246 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 203.63
Current children cumulated vsize (KiB) 185836

[startup+102.301 s]
/proc/loadavg: 8.12 8.07 6.54 9/392 21169
/proc/meminfo: memFree=14502224/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=223156 CPUtime=407.83 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 218638 1259 0 0 40751 32 0 0 25 0 5 0 733322839 228511744 43576 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 55789 43576 180 55 0 53675 0
[pid=21156/tid=21163] ppid=21154 vsize=223156 CPUtime=102.24 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 38912 1259 0 0 10219 5 0 0 25 0 5 0 733322843 228511744 43576 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=223156 CPUtime=102.26 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 41423 1259 0 0 10220 6 0 0 25 0 5 0 733322843 228511744 43576 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=223156 CPUtime=102.26 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 38505 1259 0 0 10221 5 0 0 25 0 5 0 733322843 228511744 43576 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=223156 CPUtime=101.01 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 97285 1259 0 0 10086 15 0 0 25 0 5 0 733322843 228511744 43576 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 407.83
Current children cumulated vsize (KiB) 223156

[startup+162.3 s]
/proc/loadavg: 8.04 8.05 6.63 9/392 21170
/proc/meminfo: memFree=14495748/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=209208 CPUtime=647.04 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 304688 1259 0 0 64659 45 0 0 25 0 5 0 733322839 214228992 40116 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 52302 40116 180 55 0 50188 0
[pid=21156/tid=21163] ppid=21154 vsize=209208 CPUtime=162.21 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 60948 1259 0 0 16213 8 0 0 25 0 5 0 733322843 214228992 40116 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=209208 CPUtime=162.24 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 68282 1259 0 0 16215 9 0 0 25 0 5 0 733322843 214228992 40116 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=209208 CPUtime=162.25 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 55977 1259 0 0 16217 8 0 0 25 0 5 0 733322843 214228992 40116 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=209208 CPUtime=160.27 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 116968 1259 0 0 16009 18 0 0 25 0 5 0 733322843 214228992 40116 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 647.04
Current children cumulated vsize (KiB) 209208


################
# More data... #
################

[pid=21156/tid=21164] ppid=21154 vsize=761244 CPUtime=702.26 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 217556 1259 0 0 70193 33 0 0 25 0 5 0 733322843 779513856 178098 33554432000 4194304 4417452 140737470276656 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=761244 CPUtime=702.27 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 176578 1259 0 0 70198 29 0 0 25 0 5 0 733322843 779513856 178098 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=761244 CPUtime=693.5 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 270993 1259 0 0 69306 44 0 0 25 0 5 0 733322843 779513856 178098 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2800.19
Current children cumulated vsize (KiB) 761244

[startup+762.3 s]
/proc/loadavg: 8.15 8.12 7.32 10/392 21186
/proc/meminfo: memFree=13808028/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=821272 CPUtime=3039.47 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 871504 1259 0 0 303813 134 0 0 25 0 5 0 733322839 840982528 193105 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 205318 193105 180 55 0 203204 0
[pid=21156/tid=21163] ppid=21154 vsize=821272 CPUtime=762.09 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 195248 1259 0 0 76182 27 0 0 25 0 5 0 733322843 840982528 193105 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=821272 CPUtime=762.27 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 226172 1259 0 0 76194 33 0 0 25 0 5 0 733322843 840982528 193105 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=821272 CPUtime=762.28 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 176578 1259 0 0 76199 29 0 0 25 0 5 0 733322843 840982528 193105 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=821272 CPUtime=752.76 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 270993 1259 0 0 75232 44 0 0 25 0 5 0 733322843 840982528 193105 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3039.47
Current children cumulated vsize (KiB) 821272

[startup+822.301 s]
/proc/loadavg: 8.13 8.11 7.37 9/392 21188
/proc/meminfo: memFree=13927048/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=692284 CPUtime=3278.66 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 968385 1259 0 0 327715 151 0 0 25 0 5 0 733322839 708898816 160890 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 173071 160890 180 55 0 170957 0
[pid=21156/tid=21163] ppid=21154 vsize=692284 CPUtime=822.08 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 225265 1259 0 0 82174 34 0 0 25 0 5 0 733322843 708898816 160890 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=692284 CPUtime=822.26 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 242398 1259 0 0 82191 35 0 0 25 0 5 0 733322843 708898816 160890 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=692284 CPUtime=822.26 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 191375 1259 0 0 82195 31 0 0 25 0 5 0 733322843 708898816 160890 33554432000 4194304 4417452 140737470276656 18446744073709551615 4251133 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=692284 CPUtime=812 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 306834 1259 0 0 81151 49 0 0 25 0 5 0 733322843 708898816 160890 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3278.66
Current children cumulated vsize (KiB) 692284

[startup+882.314 s]
/proc/loadavg: 8.00 8.07 7.40 9/392 21226
/proc/meminfo: memFree=13890844/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=704364 CPUtime=3517.94 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 1029416 1259 0 0 351632 162 0 0 25 0 5 0 733322839 721268736 163878 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 176091 163878 180 55 0 173977 0
[pid=21156/tid=21163] ppid=21154 vsize=704364 CPUtime=882.07 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 236488 1259 0 0 88172 35 0 0 25 0 5 0 733322843 721268736 163878 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364616 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=704364 CPUtime=882.27 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 272052 1259 0 0 88186 41 0 0 25 0 5 0 733322843 721268736 163878 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=704364 CPUtime=882.28 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 203337 1259 0 0 88195 33 0 0 25 0 5 0 733322843 721268736 163878 33554432000 4194304 4417452 140737470276656 18446744073709551615 4278321 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=704364 CPUtime=871.25 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 315026 1259 0 0 87074 51 0 0 25 0 5 0 733322843 721268736 163878 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3517.94
Current children cumulated vsize (KiB) 704364

[startup+942.301 s]
/proc/loadavg: 8.07 8.07 7.44 9/392 21228
/proc/meminfo: memFree=13738424/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=839620 CPUtime=3757.12 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 1085532 1259 0 0 375540 172 0 0 25 0 5 0 733322839 859770880 197692 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 209905 197692 180 55 0 207791 0
[pid=21156/tid=21163] ppid=21154 vsize=839620 CPUtime=942.04 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 240584 1259 0 0 94168 36 0 0 25 0 5 0 733322843 859770880 197692 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=839620 CPUtime=942.26 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 286448 1259 0 0 94183 43 0 0 25 0 5 0 733322843 859770880 197692 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=839620 CPUtime=942.27 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 220152 1259 0 0 94191 36 0 0 25 0 5 0 733322843 859770880 197692 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=839620 CPUtime=930.48 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 335835 1259 0 0 92993 55 0 0 25 0 5 0 733322843 859770880 197692 33554432000 4194304 4417452 140737470276656 18446744073709551615 4365088 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3757.12
Current children cumulated vsize (KiB) 839620

[startup+1002.3 s]
/proc/loadavg: 8.09 8.07 7.48 9/392 21230
/proc/meminfo: memFree=13671496/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=876364 CPUtime=3996.35 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 1094718 1259 0 0 399462 173 0 0 25 0 5 0 733322839 897396736 206878 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 219091 206878 180 55 0 216977 0
[pid=21156/tid=21163] ppid=21154 vsize=876364 CPUtime=1002.03 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 245674 1259 0 0 100166 37 0 0 25 0 5 0 733322843 897396736 206878 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=876364 CPUtime=1002.27 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 290544 1259 0 0 100183 44 0 0 25 0 5 0 733322843 897396736 206878 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364893 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=876364 CPUtime=1002.27 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 220152 1259 0 0 100191 36 0 0 25 0 5 0 733322843 897396736 206878 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=876364 CPUtime=989.72 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 335835 1259 0 0 98917 55 0 0 25 0 5 0 733322843 897396736 206878 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3996.35
Current children cumulated vsize (KiB) 876364

[startup+1062.3 s]
/proc/loadavg: 8.13 8.09 7.52 9/392 21231
/proc/meminfo: memFree=13461404/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=975168 CPUtime=4235.6 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 1119419 1259 0 0 423382 178 0 0 25 0 5 0 733322839 998572032 231579 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 243792 231579 180 55 0 241678 0
[pid=21156/tid=21163] ppid=21154 vsize=975168 CPUtime=1062.02 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 255878 1259 0 0 106164 38 0 0 25 0 5 0 733322843 998572032 231579 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=975168 CPUtime=1062.27 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 305041 1259 0 0 106180 47 0 0 25 0 5 0 733322843 998572032 231579 33554432000 4194304 4417452 140737470276656 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=975168 CPUtime=1062.27 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 220152 1259 0 0 106191 36 0 0 25 0 5 0 733322843 998572032 231579 33554432000 4194304 4417452 140737470276656 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=975168 CPUtime=1048.97 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 335835 1259 0 0 104842 55 0 0 25 0 5 0 733322843 998572032 231579 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4235.6
Current children cumulated vsize (KiB) 975168

[startup+1122.3 s]
/proc/loadavg: 8.14 8.10 7.56 9/392 21233
/proc/meminfo: memFree=13290004/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=1079416 CPUtime=4474.81 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 1158453 1259 0 0 447296 185 0 0 25 0 5 0 733322839 1105321984 257641 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 269854 257641 180 55 0 267740 0
[pid=21156/tid=21163] ppid=21154 vsize=1079416 CPUtime=1122 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 280647 1259 0 0 112158 42 0 0 25 0 5 0 733322843 1105321984 257641 33554432000 4194304 4417452 140737470276656 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=1079416 CPUtime=1122.27 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 305041 1259 0 0 112180 47 0 0 25 0 5 0 733322843 1105321984 257641 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364735 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=1079416 CPUtime=1122.27 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 222319 1259 0 0 112191 36 0 0 25 0 5 0 733322843 1105321984 257641 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=1079416 CPUtime=1108.2 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 347933 1259 0 0 110762 58 0 0 25 0 5 0 733322843 1105321984 257641 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4474.81
Current children cumulated vsize (KiB) 1079416

[startup+1182.3 s]
/proc/loadavg: 8.21 8.12 7.60 9/392 21243
/proc/meminfo: memFree=13136436/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=1256200 CPUtime=4714.02 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 1243194 1259 0 0 471201 201 0 0 25 0 5 0 733322839 1286348800 301837 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 314050 301837 180 55 0 311936 0
[pid=21156/tid=21163] ppid=21154 vsize=1256200 CPUtime=1182 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 307218 1259 0 0 118153 47 0 0 25 0 5 0 733322843 1286348800 301837 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=1256200 CPUtime=1182.27 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 334858 1259 0 0 118174 53 0 0 25 0 5 0 733322843 1286348800 301837 33554432000 4194304 4417452 140737470276656 18446744073709551615 4310182 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=1256200 CPUtime=1182.25 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 234754 1259 0 0 118186 39 0 0 25 0 5 0 733322843 1286348800 301837 33554432000 4194304 4417452 140737470276656 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=1256200 CPUtime=1167.44 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 363851 1259 0 0 116683 61 0 0 25 0 5 0 733322843 1286348800 301837 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364711 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4714.02
Current children cumulated vsize (KiB) 1256200



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.9 s]
/proc/loadavg: 8.15 8.11 7.60 9/392 21244
/proc/meminfo: memFree=12987620/32951124 swapFree=65586204/67111528
[pid=21156] ppid=21154 vsize=1371964 CPUtime=4800.18 cores=0,2,4,6
/proc/21156/stat : 21156 (plingeling) S 21154 21156 14218 0 -1 4202496 1306119 1259 0 0 479804 214 0 0 25 0 5 0 733322839 1404891136 330778 33554432000 4194304 4417452 140737470276656 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21156/statm: 342991 330778 180 55 0 340877 0
[pid=21156/tid=21163] ppid=21154 vsize=1371964 CPUtime=1203.6 cores=0,2,4,6
/proc/21156/task/21163/stat : 21163 (plingeling) R 21154 21156 14218 0 -1 4202560 307218 1259 0 0 120313 47 0 0 25 0 5 0 733322843 1404891136 330778 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21156/tid=21164] ppid=21154 vsize=1371964 CPUtime=1203.88 cores=0,2,4,6
/proc/21156/task/21164/stat : 21164 (plingeling) R 21154 21156 14218 0 -1 4202560 363561 1259 0 0 120330 58 0 0 25 0 5 0 733322843 1404891136 330778 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21156/tid=21165] ppid=21154 vsize=1371964 CPUtime=1203.87 cores=0,2,4,6
/proc/21156/task/21165/stat : 21165 (plingeling) R 21154 21156 14218 0 -1 4202560 268976 1259 0 0 120341 46 0 0 25 0 5 0 733322843 1404891136 330778 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21156/tid=21166] ppid=21154 vsize=1371964 CPUtime=1188.76 cores=0,2,4,6
/proc/21156/task/21166/stat : 21166 (plingeling) R 21154 21156 14218 0 -1 4202560 363851 1259 0 0 118815 61 0 0 25 0 5 0 733322843 1404891136 330778 33554432000 4194304 4417452 140737470276656 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.18
Current children cumulated vsize (KiB) 1371964

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.98
CPU time (s): 4800.27
CPU user time (s): 4798.07
CPU system time (s): 2.19967
CPU usage (%): 398.701
Max. virtual memory (cumulated for all children) (KiB): 1371964

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.07
system time used= 2.19966
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1307378
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 51
involuntary context switches= 25833

runsolver used 4.48432 second user time and 10.6524 second system time

The end

Launcher Data 

Begin job on node119 at 2011-05-15 05:42:02
IDJOB=3370997
IDBENCH=84559
IDSOLVER=1848
FILE ID=node119/3370997-1305430922
RUNJOBID= node119-1305429715-21019
PBS_JOBID= 13324599
Free space on /tmp= 73332 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT04/handmade/kexu/frb-0.8/frb45-21-1.used-as.sat04-883.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3370997-1305430922/watcher-3370997-1305430922 -o /tmp/evaluation-result-3370997-1305430922/solver-3370997-1305430922 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3370997-1305430922.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= d30b88c897dd41570f0430a76eb70b55
RANDOM SEED=1807934317

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      14685500 kB
Buffers:        290256 kB
Cached:        4627404 kB
SwapCached:       4756 kB
Active:       14830724 kB
Inactive:      3105808 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      14685500 kB
SwapTotal:    67111528 kB
SwapFree:     65586204 kB
Dirty:             820 kB
Writeback:           0 kB
AnonPages:    13017164 kB
Mapped:          30400 kB
Slab:           208288 kB
PageTables:      56996 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 102658344 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73308 MiB
End job on node119 at 2011-05-15 06:02:08