Trace number 3370898

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.38 1203.62

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m42-q663-n40-p30-s67876261.cnf
MD5SUMed0227339111fb337104188c725538d0
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark1.24481
Satisfiable
(Un)Satisfiability was proved
Number of variables1634
Number of clauses212885
Sum of the clauses size427324
Maximum clause length42
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2212845
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 540

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3370898-1305433137/watcher-3370898-1305433137 -o /tmp/evaluation-result-3370898-1305433137/solver-3370898-1305433137 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3370898-1305433137.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.67 7.77 7.94 2/351 22019
/proc/meminfo: memFree=21254684/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=9668 CPUtime=0 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) R 22017 22019 21414 0 -1 4202496 509 1253 0 0 0 0 0 0 22 0 1 0 654911442 9900032 439 33554432000 4194304 4417452 140735914603552 18446744073709551615 218905730816 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/22019/statm: 2417 439 147 55 0 303 0

[startup+0.0241041 s]
/proc/loadavg: 6.67 7.77 7.94 2/351 22019
/proc/meminfo: memFree=21254684/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=11540 CPUtime=0.02 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) R 22017 22019 21414 0 -1 4202496 1026 1253 0 0 2 0 0 0 22 0 1 0 654911442 11816960 908 33554432000 4194304 4417452 140735914603552 18446744073709551615 218905730816 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/22019/statm: 2885 908 148 55 0 771 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 11540

[startup+0.101121 s]
/proc/loadavg: 6.67 7.77 7.94 2/351 22019
/proc/meminfo: memFree=21254684/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=25876 CPUtime=0.09 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) R 22017 22019 21414 0 -1 4202496 4610 1253 0 0 9 0 0 0 23 0 1 0 654911442 26497024 4492 33554432000 4194304 4417452 140735914603552 18446744073709551615 4262065 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/22019/statm: 6469 4492 148 55 0 4355 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 25876

[startup+0.300991 s]
/proc/loadavg: 6.67 7.77 7.94 2/351 22019
/proc/meminfo: memFree=21254684/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=70288 CPUtime=0.74 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 5890 1253 0 0 73 1 0 0 23 0 5 0 654911442 71974912 5386 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 17572 5386 175 55 0 15458 0
Current children cumulated CPU time (s) 0.74
Current children cumulated vsize (KiB) 70288

[startup+0.700919 s]
/proc/loadavg: 6.67 7.77 7.94 2/351 22019
/proc/meminfo: memFree=21254684/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=150984 CPUtime=2.33 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 17959 1253 0 0 230 3 0 0 23 0 5 0 654911442 154607616 9206 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 37746 9206 180 55 0 35632 0
Current children cumulated CPU time (s) 2.33
Current children cumulated vsize (KiB) 150984

[startup+1.50077 s]
/proc/loadavg: 6.67 7.77 7.94 9/363 22044
/proc/meminfo: memFree=21155448/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=175052 CPUtime=5.52 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 25760 1253 0 0 548 4 0 0 23 0 5 0 654911442 179253248 15271 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 43763 15271 180 55 0 41649 0
[pid=22019/tid=22037] ppid=22017 vsize=175052 CPUtime=1.34 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 3819 1253 0 0 134 0 0 0 25 0 5 0 654911456 179253248 15271 33554432000 4194304 4417452 140735914603552 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=175052 CPUtime=1.34 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 3691 1253 0 0 134 0 0 0 25 0 5 0 654911456 179253248 15271 33554432000 4194304 4417452 140735914603552 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=175052 CPUtime=1.34 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 4599 1253 0 0 133 1 0 0 25 0 5 0 654911456 179253248 15271 33554432000 4194304 4417452 140735914603552 18446744073709551615 4336723 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=175052 CPUtime=1.33 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 9029 1253 0 0 132 1 0 0 25 0 5 0 654911456 179253248 15271 33554432000 4194304 4417452 140735914603552 18446744073709551615 4251258 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 5.52
Current children cumulated vsize (KiB) 175052

[startup+3.10048 s]
/proc/loadavg: 6.67 7.77 7.94 9/363 22044
/proc/meminfo: memFree=21132508/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=178444 CPUtime=11.9 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 33137 1253 0 0 1185 5 0 0 23 0 5 0 654911442 182726656 16050 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 44611 16050 180 55 0 42497 0
[pid=22019/tid=22037] ppid=22017 vsize=178444 CPUtime=2.94 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 3819 1253 0 0 294 0 0 0 25 0 5 0 654911456 182726656 16050 33554432000 4194304 4417452 140735914603552 18446744073709551615 4258513 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=178444 CPUtime=2.94 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 3691 1253 0 0 294 0 0 0 25 0 5 0 654911456 182726656 16050 33554432000 4194304 4417452 140735914603552 18446744073709551615 4236008 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=178444 CPUtime=2.94 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 4599 1253 0 0 293 1 0 0 25 0 5 0 654911456 182726656 16050 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=178444 CPUtime=2.91 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 16406 1253 0 0 289 2 0 0 25 0 5 0 654911456 182726656 16050 33554432000 4194304 4417452 140735914603552 18446744073709551615 4366063 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.9
Current children cumulated vsize (KiB) 178444

[startup+6.3009 s]
/proc/loadavg: 6.77 7.78 7.94 9/363 22044
/proc/meminfo: memFree=21098156/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=204408 CPUtime=24.67 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 47573 1253 0 0 2460 7 0 0 23 0 5 0 654911442 209313792 22188 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 51102 22188 180 55 0 48988 0
[pid=22019/tid=22037] ppid=22017 vsize=204408 CPUtime=6.14 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 4285 1253 0 0 614 0 0 0 25 0 5 0 654911456 209313792 22188 33554432000 4194304 4417452 140735914603552 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=204408 CPUtime=6.14 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 3691 1253 0 0 614 0 0 0 25 0 5 0 654911456 209313792 22188 33554432000 4194304 4417452 140735914603552 18446744073709551615 4258565 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=204408 CPUtime=6.14 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 5419 1253 0 0 613 1 0 0 25 0 5 0 654911456 209313792 22188 33554432000 4194304 4417452 140735914603552 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=204408 CPUtime=6.08 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 29556 1253 0 0 604 4 0 0 25 0 5 0 654911456 209313792 22188 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.67
Current children cumulated vsize (KiB) 204408

[startup+12.7007 s]
/proc/loadavg: 6.95 7.80 7.95 9/363 22045
/proc/meminfo: memFree=20963960/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=271888 CPUtime=50.2 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 70881 1253 0 0 5010 10 0 0 23 0 5 0 654911442 278413312 39411 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 67972 39411 180 55 0 65858 0
[pid=22019/tid=22037] ppid=22017 vsize=271888 CPUtime=12.54 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 4285 1253 0 0 1254 0 0 0 25 0 5 0 654911456 278413312 39411 33554432000 4194304 4417452 140735914603552 18446744073709551615 4249151 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=271888 CPUtime=12.54 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 3691 1253 0 0 1254 0 0 0 25 0 5 0 654911456 278413312 39411 33554432000 4194304 4417452 140735914603552 18446744073709551615 4236239 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=271888 CPUtime=12.54 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 6499 1253 0 0 1253 1 0 0 25 0 5 0 654911456 278413312 39411 33554432000 4194304 4417452 140735914603552 18446744073709551615 4309583 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=271888 CPUtime=12.41 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 51784 1253 0 0 1234 7 0 0 25 0 5 0 654911456 278413312 39411 33554432000 4194304 4417452 140735914603552 18446744073709551615 4250447 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.2
Current children cumulated vsize (KiB) 271888

[startup+25.5004 s]
/proc/loadavg: 7.18 7.81 7.95 9/363 22045
/proc/meminfo: memFree=20905200/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=274144 CPUtime=101.25 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 104439 1253 0 0 10109 16 0 0 23 0 5 0 654911442 280723456 39975 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 68536 39975 180 55 0 66422 0
[pid=22019/tid=22037] ppid=22017 vsize=274144 CPUtime=25.34 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 5310 1253 0 0 2534 0 0 0 25 0 5 0 654911456 280723456 39975 33554432000 4194304 4417452 140735914603552 18446744073709551615 4237761 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=274144 CPUtime=25.32 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 4765 1253 0 0 2532 0 0 0 25 0 5 0 654911456 280723456 39975 33554432000 4194304 4417452 140735914603552 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=274144 CPUtime=25.34 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 6499 1253 0 0 2533 1 0 0 25 0 5 0 654911456 280723456 39975 33554432000 4194304 4417452 140735914603552 18446744073709551615 4306366 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=274144 CPUtime=25.06 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 83243 1253 0 0 2494 12 0 0 25 0 5 0 654911456 280723456 39975 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.25
Current children cumulated vsize (KiB) 274144

[startup+51.1008 s]
/proc/loadavg: 7.46 7.82 7.95 9/363 22046
/proc/meminfo: memFree=20707104/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=364796 CPUtime=203.37 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 147052 1253 0 0 20315 22 0 0 23 0 5 0 654911442 373551104 62638 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 91199 62638 180 55 0 89085 0
[pid=22019/tid=22037] ppid=22017 vsize=364796 CPUtime=50.94 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 5310 1253 0 0 5094 0 0 0 25 0 5 0 654911456 373551104 62638 33554432000 4194304 4417452 140735914603552 18446744073709551615 4258161 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=364796 CPUtime=50.93 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 4765 1253 0 0 5093 0 0 0 25 0 5 0 654911456 373551104 62638 33554432000 4194304 4417452 140735914603552 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=364796 CPUtime=50.94 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 6499 1253 0 0 5093 1 0 0 25 0 5 0 654911456 373551104 62638 33554432000 4194304 4417452 140735914603552 18446744073709551615 4365004 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=364796 CPUtime=50.39 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 125856 1253 0 0 5021 18 0 0 25 0 5 0 654911456 373551104 62638 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 203.37
Current children cumulated vsize (KiB) 364796

[startup+102.314 s]
/proc/loadavg: 7.81 7.86 7.95 9/363 22047
/proc/meminfo: memFree=20392492/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=579252 CPUtime=407.64 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 245332 1253 0 0 40730 34 0 0 23 0 5 0 654911442 593154048 116252 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 144813 116252 180 55 0 142699 0
[pid=22019/tid=22037] ppid=22017 vsize=579252 CPUtime=102.14 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 9281 1253 0 0 10214 0 0 0 25 0 5 0 654911456 593154048 116252 33554432000 4194304 4417452 140735914603552 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=579252 CPUtime=101.7 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 8922 1253 0 0 10170 0 0 0 25 0 5 0 654911456 593154048 116252 33554432000 4194304 4417452 140735914603552 18446744073709551615 4357737 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=579252 CPUtime=102.15 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 17055 1253 0 0 10213 2 0 0 25 0 5 0 654911456 593154048 116252 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=579252 CPUtime=101.46 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 205452 1253 0 0 10118 28 0 0 25 0 5 0 654911456 593154048 116252 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 407.64
Current children cumulated vsize (KiB) 579252

[startup+162.301 s]
/proc/loadavg: 8.01 7.91 7.96 9/363 22049
/proc/meminfo: memFree=19991448/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=687040 CPUtime=646.93 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 367836 1253 0 0 64642 51 0 0 23 0 5 0 654911442 703528960 141123 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 171760 141123 180 55 0 169646 0
[pid=22019/tid=22037] ppid=22017 vsize=687040 CPUtime=162.13 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 38102 1253 0 0 16210 3 0 0 25 0 5 0 654911456 703528960 141123 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=687040 CPUtime=161.05 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 21395 1253 0 0 16103 2 0 0 25 0 5 0 654911456 703528960 141123 33554432000 4194304 4417452 140735914603552 18446744073709551615 4237191 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=687040 CPUtime=162.14 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 51681 1253 0 0 16208 6 0 0 25 0 5 0 654911456 703528960 141123 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=687040 CPUtime=161.43 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 252036 1253 0 0 16106 37 0 0 25 0 5 0 654911456 703528960 141123 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 646.93
Current children cumulated vsize (KiB) 687040


################
# More data... #
################

/proc/meminfo: memFree=18106920/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=1561644 CPUtime=3040.35 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 1391981 1253 0 0 303826 209 0 0 23 0 5 0 654911442 1599123456 361850 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 390411 361850 180 55 0 388297 0
[pid=22019/tid=22037] ppid=22017 vsize=1561644 CPUtime=762.15 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 227637 1253 0 0 76184 31 0 0 25 0 5 0 654911456 1599123456 361850 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=1561644 CPUtime=754.51 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 290742 1253 0 0 75413 38 0 0 25 0 5 0 654911456 1599123456 361850 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=1561644 CPUtime=762.17 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 237394 1253 0 0 76185 32 0 0 25 0 5 0 654911456 1599123456 361850 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=1561644 CPUtime=761.35 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 631586 1253 0 0 76029 106 0 0 25 0 5 0 654911456 1599123456 361850 33554432000 4194304 4417452 140735914603552 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3040.35
Current children cumulated vsize (KiB) 1561644

[startup+822.3 s]
/proc/loadavg: 8.17 8.11 8.03 9/363 22066
/proc/meminfo: memFree=17813368/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=1762532 CPUtime=3279.64 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 1474234 1253 0 0 327744 220 0 0 23 0 5 0 654911442 1804832768 412072 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 440633 412072 180 55 0 438519 0
[pid=22019/tid=22037] ppid=22017 vsize=1762532 CPUtime=822.14 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 252197 1253 0 0 82179 35 0 0 25 0 5 0 654911456 1804832768 412072 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=1762532 CPUtime=813.85 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 290742 1253 0 0 81347 38 0 0 25 0 5 0 654911456 1804832768 412072 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=1762532 CPUtime=822.16 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 253778 1253 0 0 82182 34 0 0 25 0 5 0 654911456 1804832768 412072 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=1762532 CPUtime=821.33 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 672895 1253 0 0 82021 112 0 0 25 0 5 0 654911456 1804832768 412072 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3279.64
Current children cumulated vsize (KiB) 1762532

[startup+882.3 s]
/proc/loadavg: 8.15 8.12 8.03 9/363 22068
/proc/meminfo: memFree=17561096/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=1854780 CPUtime=3518.96 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 1545179 1253 0 0 351667 229 0 0 23 0 5 0 654911442 1899294720 435134 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 463695 435134 180 55 0 461581 0
[pid=22019/tid=22037] ppid=22017 vsize=1854780 CPUtime=882.13 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 284965 1253 0 0 88174 39 0 0 25 0 5 0 654911456 1899294720 435134 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=1854780 CPUtime=873.18 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 320727 1253 0 0 87277 41 0 0 25 0 5 0 654911456 1899294720 435134 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=1854780 CPUtime=882.16 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 261970 1253 0 0 88181 35 0 0 25 0 5 0 654911456 1899294720 435134 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=1854780 CPUtime=881.32 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 672895 1253 0 0 88020 112 0 0 25 0 5 0 654911456 1899294720 435134 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3518.96
Current children cumulated vsize (KiB) 1854780

[startup+942.3 s]
/proc/loadavg: 8.14 8.13 8.04 9/363 22069
/proc/meminfo: memFree=17076604/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=2079020 CPUtime=3758.31 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 1687845 1253 0 0 375584 247 0 0 23 0 5 0 654911442 2128916480 491194 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 519755 491194 180 55 0 517641 0
[pid=22019/tid=22037] ppid=22017 vsize=2079020 CPUtime=942.14 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 349063 1253 0 0 94165 49 0 0 25 0 5 0 654911456 2128916480 491194 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=2079020 CPUtime=932.54 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 337111 1253 0 0 93210 44 0 0 25 0 5 0 654911456 2128916480 491194 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=2079020 CPUtime=942.16 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 270162 1253 0 0 94181 35 0 0 25 0 5 0 654911456 2128916480 491194 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=2079020 CPUtime=941.3 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 726887 1253 0 0 94013 117 0 0 25 0 5 0 654911456 2128916480 491194 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3758.31
Current children cumulated vsize (KiB) 2079020

[startup+1002.3 s]
/proc/loadavg: 8.10 8.11 8.04 9/363 22071
/proc/meminfo: memFree=16584916/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=2376328 CPUtime=3997.63 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 1794940 1253 0 0 399501 262 0 0 23 0 5 0 654911442 2433359872 565521 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 594082 565521 180 55 0 591968 0
[pid=22019/tid=22037] ppid=22017 vsize=2376328 CPUtime=1002.13 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 373639 1253 0 0 100162 51 0 0 25 0 5 0 654911456 2433359872 565521 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=2376328 CPUtime=991.86 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 369657 1253 0 0 99139 47 0 0 25 0 5 0 654911456 2433359872 565521 33554432000 4194304 4417452 140735914603552 18446744073709551615 4366646 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=2376328 CPUtime=1002.16 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 286546 1253 0 0 100177 39 0 0 25 0 5 0 654911456 2433359872 565521 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=2376328 CPUtime=1001.29 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 760476 1253 0 0 100007 122 0 0 25 0 5 0 654911456 2433359872 565521 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3997.63
Current children cumulated vsize (KiB) 2376328

[startup+1062.3 s]
/proc/loadavg: 8.03 8.09 8.03 9/363 22073
/proc/meminfo: memFree=16522752/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=2412212 CPUtime=4236.97 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 1876926 1253 0 0 423424 273 0 0 23 0 5 0 654911442 2470105088 574492 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 603053 574492 180 55 0 600939 0
[pid=22019/tid=22037] ppid=22017 vsize=2412212 CPUtime=1062.13 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 373639 1253 0 0 106162 51 0 0 25 0 5 0 654911456 2470105088 574492 33554432000 4194304 4417452 140735914603552 18446744073709551615 4263247 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=2412212 CPUtime=1051.22 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 447253 1253 0 0 105065 57 0 0 25 0 5 0 654911456 2470105088 574492 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=2412212 CPUtime=1062.17 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 290936 1253 0 0 106177 40 0 0 25 0 5 0 654911456 2470105088 574492 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=2412212 CPUtime=1061.28 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 760476 1253 0 0 106006 122 0 0 25 0 5 0 654911456 2470105088 574492 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4236.97
Current children cumulated vsize (KiB) 2412212

[startup+1122.3 s]
/proc/loadavg: 8.05 8.08 8.03 9/363 22074
/proc/meminfo: memFree=16252888/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=2529380 CPUtime=4476.31 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 1922836 1253 0 0 447352 279 0 0 23 0 5 0 654911442 2590085120 603784 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 632345 603784 180 55 0 630231 0
[pid=22019/tid=22037] ppid=22017 vsize=2529380 CPUtime=1122.13 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 393341 1253 0 0 112159 54 0 0 25 0 5 0 654911456 2590085120 603784 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=2529380 CPUtime=1110.56 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 447253 1253 0 0 110999 57 0 0 25 0 5 0 654911456 2590085120 603784 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=2529380 CPUtime=1122.17 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 317144 1253 0 0 112174 43 0 0 25 0 5 0 654911456 2590085120 603784 33554432000 4194304 4417452 140735914603552 18446744073709551615 4310337 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=2529380 CPUtime=1121.27 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 760476 1253 0 0 112005 122 0 0 25 0 5 0 654911456 2590085120 603784 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4476.31
Current children cumulated vsize (KiB) 2529380

[startup+1182.3 s]
/proc/loadavg: 8.05 8.08 8.03 9/363 22076
/proc/meminfo: memFree=17091584/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=2682940 CPUtime=4715.67 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 2010261 1253 0 0 471272 295 0 0 23 0 5 0 654911442 2747330560 642174 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 670735 642174 180 55 0 668621 0
[pid=22019/tid=22037] ppid=22017 vsize=2682940 CPUtime=1182.13 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 393341 1253 0 0 118159 54 0 0 25 0 5 0 654911456 2747330560 642174 33554432000 4194304 4417452 140735914603552 18446744073709551615 4263067 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=2682940 CPUtime=1169.91 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 492127 1253 0 0 116926 65 0 0 25 0 5 0 654911456 2747330560 642174 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=2682940 CPUtime=1182.17 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 336015 1253 0 0 118171 46 0 0 25 0 5 0 654911456 2747330560 642174 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=2682940 CPUtime=1181.27 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 784156 1253 0 0 118000 127 0 0 25 0 5 0 654911456 2747330560 642174 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4715.67
Current children cumulated vsize (KiB) 2682940



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.5 s]
/proc/loadavg: 8.10 8.09 8.04 9/363 22076
/proc/meminfo: memFree=17051524/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=2682940 CPUtime=4800.24 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) S 22017 22019 21414 0 -1 4202496 2010261 1253 0 0 479729 295 0 0 23 0 5 0 654911442 2747330560 642174 33554432000 4194304 4417452 140735914603552 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 670735 642174 180 55 0 668621 0
[pid=22019/tid=22037] ppid=22017 vsize=2682940 CPUtime=1203.35 cores=0,2,4,6
/proc/22019/task/22037/stat : 22037 (plingeling) R 22017 22019 21414 0 -1 4202560 393341 1253 0 0 120281 54 0 0 25 0 5 0 654911456 2747330560 642174 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=2682940 CPUtime=1190.87 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) R 22017 22019 21414 0 -1 4202560 492127 1253 0 0 119022 65 0 0 25 0 5 0 654911456 2747330560 642174 33554432000 4194304 4417452 140735914603552 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22019/tid=22039] ppid=22017 vsize=2682940 CPUtime=1203.38 cores=0,2,4,6
/proc/22019/task/22039/stat : 22039 (plingeling) R 22017 22019 21414 0 -1 4202560 336015 1253 0 0 120292 46 0 0 25 0 5 0 654911456 2747330560 642174 33554432000 4194304 4417452 140735914603552 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22019/tid=22040] ppid=22017 vsize=2682940 CPUtime=1202.47 cores=0,2,4,6
/proc/22019/task/22040/stat : 22040 (plingeling) R 22017 22019 21414 0 -1 4202560 784156 1253 0 0 120120 127 0 0 25 0 5 0 654911456 2747330560 642174 33554432000 4194304 4417452 140735914603552 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 2682940

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.62 s]
/proc/loadavg: 8.10 8.09 8.04 9/363 22076
/proc/meminfo: memFree=17051524/32951124 swapFree=65581052/67111528
[pid=22019] ppid=22017 vsize=0 CPUtime=4800.37 cores=0,2,4,6
/proc/22019/stat : 22019 (plingeling) Z 22017 22019 21414 0 -1 4203548 2010261 1253 0 0 479733 304 0 0 18 0 2 0 654911442 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/22019/statm: 0 0 0 0 0 0 0
[pid=22019/tid=22038] ppid=22017 vsize=0 CPUtime=1190.98 cores=0,2,4,6
/proc/22019/task/22038/stat : 22038 (plingeling) S 22017 22019 21414 0 -1 4203588 492127 1253 0 0 119023 75 0 0 25 0 2 0 654911456 0 0 33554432000 0 0 0 0 0 0 2147483391 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.37
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.62
CPU time (s): 4800.38
CPU user time (s): 4797.33
CPU system time (s): 3.05054
CPU usage (%): 398.829
Max. virtual memory (cumulated for all children) (KiB): 2682940

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.33
system time used= 3.05054
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2011514
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 78
involuntary context switches= 21986

runsolver used 3.80542 second user time and 9.38657 second system time

The end

Launcher Data 

Begin job on node137 at 2011-05-15 06:18:57
IDJOB=3370898
IDBENCH=88222
IDSOLVER=1848
FILE ID=node137/3370898-1305433137
RUNJOBID= node137-1305433137-22005
PBS_JOBID= 13324606
Free space on /tmp= 73796 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m42-q663-n40-p30-s67876261.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3370898-1305433137/watcher-3370898-1305433137 -o /tmp/evaluation-result-3370898-1305433137/solver-3370898-1305433137 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3370898-1305433137.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= ed0227339111fb337104188c725538d0
RANDOM SEED=1250668192

node137.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21255088 kB
Buffers:        292540 kB
Cached:        3735100 kB
SwapCached:      36328 kB
Active:        1663952 kB
Inactive:      9807136 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21255088 kB
SwapTotal:    67111528 kB
SwapFree:     65581052 kB
Dirty:            5160 kB
Writeback:           0 kB
AnonPages:     7415856 kB
Mapped:          18724 kB
Slab:           121208 kB
PageTables:      38776 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 75899440 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73792 MiB
End job on node137 at 2011-05-15 06:39:03