Trace number 3371047

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.25 1203.65

General information on the benchmark

NameSAT05/crafted/sorge05/QG7a/
gensys-ukn004.shuffled-as.sat05-3844.cnf
MD5SUM7d96ba103689bec761c9e7df7b158323
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark3199.02
Satisfiable
(Un)Satisfiability was proved
Number of variables2731
Number of clauses17983
Sum of the clauses size126997
Maximum clause length13
Minimum clause length1
Number of clauses of size 142
Number of clauses of size 2442
Number of clauses of size 32370
Number of clauses of size 4477
Number of clauses of size 5642
Number of clauses of size over 514010

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371047-1305431928/watcher-3371047-1305431928 -o /tmp/evaluation-result-3371047-1305431928/solver-3371047-1305431928 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371047-1305431928.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.79 8.03 7.99 5/358 21912
/proc/meminfo: memFree=21207632/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=9840 CPUtime=0 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) R 21910 21912 21414 0 -1 4202496 529 1252 0 0 0 0 0 0 25 0 1 0 654790521 10076160 459 33554432000 4194304 4417452 140735708141440 18446744073709551615 218905424872 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/21912/statm: 2460 459 148 55 0 346 0

[startup+0.0197801 s]
/proc/loadavg: 7.79 8.03 7.99 5/358 21912
/proc/meminfo: memFree=21207632/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=12544 CPUtime=0.01 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) R 21910 21912 21414 0 -1 4202496 1246 1252 0 0 1 0 0 0 25 0 1 0 654790521 12845056 1160 33554432000 4194304 4417452 140735708141440 18446744073709551615 4231256 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/21912/statm: 3136 1160 149 55 0 1022 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 12544

[startup+0.160757 s]
/proc/loadavg: 7.79 8.03 7.99 5/358 21912
/proc/meminfo: memFree=21207632/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=124068 CPUtime=0.5 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 3309 1252 0 0 50 0 0 0 25 0 5 0 654790521 127045632 2453 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 31017 2453 180 55 0 28903 0
Current children cumulated CPU time (s) 0.5
Current children cumulated vsize (KiB) 124068

[startup+0.300731 s]
/proc/loadavg: 7.79 8.03 7.99 5/358 21912
/proc/meminfo: memFree=21207632/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=125140 CPUtime=1.06 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 3734 1252 0 0 106 0 0 0 25 0 5 0 654790521 128143360 2753 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 31285 2753 180 55 0 29171 0
Current children cumulated CPU time (s) 1.06
Current children cumulated vsize (KiB) 125140

[startup+0.70066 s]
/proc/loadavg: 7.79 8.03 7.99 5/358 21912
/proc/meminfo: memFree=21207632/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=130416 CPUtime=2.65 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 6338 1252 0 0 265 0 0 0 25 0 5 0 654790521 133545984 4079 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 32604 4079 180 55 0 30490 0
Current children cumulated CPU time (s) 2.65
Current children cumulated vsize (KiB) 130416

[startup+1.50052 s]
/proc/loadavg: 7.79 8.03 7.99 9/363 21922
/proc/meminfo: memFree=21191728/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=135132 CPUtime=5.84 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 11942 1252 0 0 583 1 0 0 25 0 5 0 654790521 138375168 5231 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 33783 5231 180 55 0 31669 0
[pid=21912/tid=21919] ppid=21910 vsize=135132 CPUtime=1.45 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 2390 1252 0 0 145 0 0 0 25 0 5 0 654790526 138375168 5231 33554432000 4194304 4417452 140735708141440 18446744073709551615 4262960 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=135132 CPUtime=1.43 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 2166 1252 0 0 143 0 0 0 25 0 5 0 654790526 138375168 5231 33554432000 4194304 4417452 140735708141440 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=135132 CPUtime=1.45 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 2160 1252 0 0 145 0 0 0 25 0 5 0 654790526 138375168 5231 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=135132 CPUtime=1.45 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 2944 1252 0 0 145 0 0 0 25 0 5 0 654790526 138375168 5231 33554432000 4194304 4417452 140735708141440 18446744073709551615 4250848 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 5.84
Current children cumulated vsize (KiB) 135132

[startup+3.10023 s]
/proc/loadavg: 7.79 8.03 7.99 9/363 21922
/proc/meminfo: memFree=21162276/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=143868 CPUtime=12.23 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 22884 1252 0 0 1220 3 0 0 25 0 5 0 654790521 147320832 7444 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 35967 7444 180 55 0 33853 0
[pid=21912/tid=21919] ppid=21910 vsize=143868 CPUtime=3.04 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 4708 1252 0 0 304 0 0 0 25 0 5 0 654790526 147320832 7444 33554432000 4194304 4417452 140735708141440 18446744073709551615 4236549 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=143868 CPUtime=3.01 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 5129 1252 0 0 301 0 0 0 25 0 5 0 654790526 147320832 7444 33554432000 4194304 4417452 140735708141440 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=143868 CPUtime=3.04 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 5071 1252 0 0 304 0 0 0 25 0 5 0 654790526 147320832 7444 33554432000 4194304 4417452 140735708141440 18446744073709551615 4278624 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=143868 CPUtime=3.04 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 5694 1252 0 0 304 0 0 0 25 0 5 0 654790526 147320832 7444 33554432000 4194304 4417452 140735708141440 18446744073709551615 4251164 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 12.23
Current children cumulated vsize (KiB) 143868

[startup+6.30065 s]
/proc/loadavg: 7.80 8.03 7.99 9/363 21922
/proc/meminfo: memFree=21153584/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=149208 CPUtime=24.99 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 36768 1252 0 0 2493 6 0 0 25 0 5 0 654790521 152788992 8767 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 37302 8767 180 55 0 35188 0
[pid=21912/tid=21919] ppid=21910 vsize=149208 CPUtime=6.25 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 8351 1252 0 0 624 1 0 0 25 0 5 0 654790526 152788992 8767 33554432000 4194304 4417452 140735708141440 18446744073709551615 4263090 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=149208 CPUtime=6.18 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 8565 1252 0 0 617 1 0 0 25 0 5 0 654790526 152788992 8767 33554432000 4194304 4417452 140735708141440 18446744073709551615 4232664 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=149208 CPUtime=6.24 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 8436 1252 0 0 623 1 0 0 25 0 5 0 654790526 152788992 8767 33554432000 4194304 4417452 140735708141440 18446744073709551615 4237292 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=149208 CPUtime=6.25 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 9134 1252 0 0 624 1 0 0 25 0 5 0 654790526 152788992 8767 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 24.99
Current children cumulated vsize (KiB) 149208

[startup+12.7005 s]
/proc/loadavg: 7.90 8.05 7.99 9/363 21922
/proc/meminfo: memFree=21129112/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=169092 CPUtime=50.51 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 58375 1252 0 0 5041 10 0 0 25 0 5 0 654790521 173150208 13727 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 42273 13727 180 55 0 40159 0
[pid=21912/tid=21919] ppid=21910 vsize=169092 CPUtime=12.65 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 14189 1252 0 0 1263 2 0 0 25 0 5 0 654790526 173150208 13727 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=169092 CPUtime=12.5 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 13034 1252 0 0 1248 2 0 0 25 0 5 0 654790526 173150208 13727 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=169092 CPUtime=12.65 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 12940 1252 0 0 1263 2 0 0 25 0 5 0 654790526 173150208 13727 33554432000 4194304 4417452 140735708141440 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=169092 CPUtime=12.65 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 15930 1252 0 0 1262 3 0 0 25 0 5 0 654790526 173150208 13727 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364780 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 50.51
Current children cumulated vsize (KiB) 169092

[startup+25.5012 s]
/proc/loadavg: 7.99 8.06 8.00 9/363 21923
/proc/meminfo: memFree=21047288/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=218160 CPUtime=101.57 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 89274 1252 0 0 10140 17 0 0 25 0 5 0 654790521 223395840 25994 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 54540 25994 180 55 0 52426 0
[pid=21912/tid=21919] ppid=21910 vsize=218160 CPUtime=25.45 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 22434 1252 0 0 2541 4 0 0 25 0 5 0 654790526 223395840 25994 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=218160 CPUtime=25.15 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 21717 1252 0 0 2511 4 0 0 25 0 5 0 654790526 223395840 25994 33554432000 4194304 4417452 140735708141440 18446744073709551615 4357222 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=218160 CPUtime=25.45 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 20072 1252 0 0 2541 4 0 0 25 0 5 0 654790526 223395840 25994 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=218160 CPUtime=25.45 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 22769 1252 0 0 2541 4 0 0 25 0 5 0 654790526 223395840 25994 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 101.57
Current children cumulated vsize (KiB) 218160

[startup+51.1005 s]
/proc/loadavg: 7.99 8.05 8.00 9/363 21923
/proc/meminfo: memFree=20976564/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=261296 CPUtime=203.66 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 135754 1252 0 0 20343 23 0 0 25 0 5 0 654790521 267567104 36778 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 65324 36778 180 55 0 63210 0
[pid=21912/tid=21919] ppid=21910 vsize=261296 CPUtime=51.05 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 35715 1252 0 0 5099 6 0 0 25 0 5 0 654790526 267567104 36778 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=261296 CPUtime=50.45 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 35239 1252 0 0 5040 5 0 0 25 0 5 0 654790526 267567104 36778 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=261296 CPUtime=51.05 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 32049 1252 0 0 5100 5 0 0 25 0 5 0 654790526 267567104 36778 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=261296 CPUtime=51.05 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 30469 1252 0 0 5100 5 0 0 25 0 5 0 654790526 267567104 36778 33554432000 4194304 4417452 140735708141440 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 203.66
Current children cumulated vsize (KiB) 261296

[startup+102.314 s]
/proc/loadavg: 8.04 8.06 8.00 9/363 21925
/proc/meminfo: memFree=20870124/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=315092 CPUtime=407.91 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 195245 1252 0 0 40759 32 0 0 25 0 5 0 654790521 322654208 50227 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 78773 50227 180 55 0 76659 0
[pid=21912/tid=21919] ppid=21910 vsize=315092 CPUtime=102.27 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 49472 1252 0 0 10218 9 0 0 25 0 5 0 654790526 322654208 50227 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=315092 CPUtime=101.05 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 53897 1252 0 0 10097 8 0 0 25 0 5 0 654790526 322654208 50227 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=315092 CPUtime=102.26 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 49119 1252 0 0 10219 7 0 0 25 0 5 0 654790526 322654208 50227 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=315092 CPUtime=102.26 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 40475 1252 0 0 10219 7 0 0 25 0 5 0 654790526 322654208 50227 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 407.91
Current children cumulated vsize (KiB) 315092

[startup+162.3 s]
/proc/loadavg: 8.06 8.06 8.00 9/363 21937
/proc/meminfo: memFree=20729832/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=382976 CPUtime=647.13 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 241317 1252 0 0 64672 41 0 0 25 0 5 0 654790521 392167424 67198 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 95744 67198 180 55 0 93630 0
[pid=21912/tid=21919] ppid=21910 vsize=382976 CPUtime=162.26 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 61441 1252 0 0 16215 11 0 0 25 0 5 0 654790526 392167424 67198 33554432000 4194304 4417452 140735708141440 18446744073709551615 4366458 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=382976 CPUtime=160.33 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 61713 1252 0 0 16024 9 0 0 25 0 5 0 654790526 392167424 67198 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=382976 CPUtime=162.22 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 60382 1252 0 0 16213 9 0 0 25 0 5 0 654790526 392167424 67198 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=382976 CPUtime=162.25 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 55499 1252 0 0 16215 10 0 0 25 0 5 0 654790526 392167424 67198 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 647.13
Current children cumulated vsize (KiB) 382976


################
# More data... #
################

[pid=21912/tid=21920] ppid=21910 vsize=679004 CPUtime=693.96 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 128092 1252 0 0 69377 19 0 0 25 0 5 0 654790526 695300096 141205 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=679004 CPUtime=702.25 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 125231 1252 0 0 70205 20 0 0 25 0 5 0 654790526 695300096 141205 33554432000 4194304 4417452 140735708141440 18446744073709551615 4310188 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=679004 CPUtime=702.26 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 104963 1252 0 0 70208 18 0 0 25 0 5 0 654790526 695300096 141205 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2800.81
Current children cumulated vsize (KiB) 679004

[startup+762.314 s]
/proc/loadavg: 8.15 8.12 8.04 9/363 21956
/proc/meminfo: memFree=20107044/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=708972 CPUtime=3040.15 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 549113 1252 0 0 303925 90 0 0 25 0 5 0 654790521 725987328 148697 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 177243 148697 180 55 0 175129 0
[pid=21912/tid=21919] ppid=21910 vsize=708972 CPUtime=762.29 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 157129 1252 0 0 76202 27 0 0 25 0 5 0 654790526 725987328 148697 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=708972 CPUtime=753.26 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 130229 1252 0 0 75307 19 0 0 25 0 5 0 654790526 725987328 148697 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=708972 CPUtime=762.26 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 145406 1252 0 0 76203 23 0 0 25 0 5 0 654790526 725987328 148697 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=708972 CPUtime=762.28 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 114067 1252 0 0 76208 20 0 0 25 0 5 0 654790526 725987328 148697 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3040.15
Current children cumulated vsize (KiB) 708972

[startup+822.301 s]
/proc/loadavg: 8.11 8.11 8.04 9/363 21958
/proc/meminfo: memFree=19956680/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=740484 CPUtime=3279.41 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 570334 1252 0 0 327847 94 0 0 25 0 5 0 654790521 758255616 156575 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 185121 156575 180 55 0 183007 0
[pid=21912/tid=21919] ppid=21910 vsize=740484 CPUtime=822.27 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 157129 1252 0 0 82200 27 0 0 25 0 5 0 654790526 758255616 156575 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=740484 CPUtime=812.55 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 149415 1252 0 0 81233 22 0 0 25 0 5 0 654790526 758255616 156575 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=740484 CPUtime=822.25 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 145406 1252 0 0 82202 23 0 0 25 0 5 0 654790526 758255616 156575 33554432000 4194304 4417452 140735708141440 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=740484 CPUtime=822.27 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 116102 1252 0 0 82207 20 0 0 25 0 5 0 654790526 758255616 156575 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3279.41
Current children cumulated vsize (KiB) 740484

[startup+882.301 s]
/proc/loadavg: 8.18 8.13 8.05 9/363 21959
/proc/meminfo: memFree=19957976/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=677444 CPUtime=3518.7 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 595561 1252 0 0 351772 98 0 0 25 0 5 0 654790521 693702656 140815 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 169361 140815 180 55 0 167247 0
[pid=21912/tid=21919] ppid=21910 vsize=677444 CPUtime=882.28 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 157129 1252 0 0 88201 27 0 0 25 0 5 0 654790526 693702656 140815 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=677444 CPUtime=871.84 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 149415 1252 0 0 87162 22 0 0 25 0 5 0 654790526 693702656 140815 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=677444 CPUtime=882.25 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 165003 1252 0 0 88199 26 0 0 25 0 5 0 654790526 693702656 140815 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=677444 CPUtime=882.27 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 121732 1252 0 0 88206 21 0 0 25 0 5 0 654790526 693702656 140815 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3518.7
Current children cumulated vsize (KiB) 677444

[startup+942.3 s]
/proc/loadavg: 8.11 8.12 8.05 9/363 21961
/proc/meminfo: memFree=20007012/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=688132 CPUtime=3758 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 598381 1252 0 0 375702 98 0 0 25 0 5 0 654790521 704647168 143487 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 172033 143487 180 55 0 169919 0
[pid=21912/tid=21919] ppid=21910 vsize=688132 CPUtime=942.28 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 157129 1252 0 0 94201 27 0 0 25 0 5 0 654790526 704647168 143487 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=688132 CPUtime=931.14 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 149415 1252 0 0 93092 22 0 0 25 0 5 0 654790526 704647168 143487 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=688132 CPUtime=942.25 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 165003 1252 0 0 94199 26 0 0 25 0 5 0 654790526 704647168 143487 33554432000 4194304 4417452 140735708141440 18446744073709551615 4357422 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=688132 CPUtime=942.27 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 124552 1252 0 0 94206 21 0 0 25 0 5 0 654790526 704647168 143487 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3758
Current children cumulated vsize (KiB) 688132

[startup+1002.3 s]
/proc/loadavg: 8.08 8.11 8.05 9/363 21963
/proc/meminfo: memFree=20058676/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=708172 CPUtime=3997.3 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 603468 1252 0 0 399631 99 0 0 25 0 5 0 654790521 725168128 148497 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 177043 148497 180 55 0 174929 0
[pid=21912/tid=21919] ppid=21910 vsize=708172 CPUtime=1002.28 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 157129 1252 0 0 100201 27 0 0 25 0 5 0 654790526 725168128 148497 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=708172 CPUtime=990.42 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 149415 1252 0 0 99020 22 0 0 25 0 5 0 654790526 725168128 148497 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=708172 CPUtime=1002.25 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 170090 1252 0 0 100198 27 0 0 25 0 5 0 654790526 725168128 148497 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=708172 CPUtime=1002.27 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 124552 1252 0 0 100206 21 0 0 25 0 5 0 654790526 725168128 148497 33554432000 4194304 4417452 140735708141440 18446744073709551615 4366079 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3997.3
Current children cumulated vsize (KiB) 708172

[startup+1062.3 s]
/proc/loadavg: 8.07 8.10 8.05 9/363 21964
/proc/meminfo: memFree=19991304/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=757384 CPUtime=4236.6 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 615771 1252 0 0 423559 101 0 0 25 0 5 0 654790521 775561216 160800 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 189346 160800 180 55 0 187232 0
[pid=21912/tid=21919] ppid=21910 vsize=757384 CPUtime=1062.27 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 159992 1252 0 0 106200 27 0 0 25 0 5 0 654790526 775561216 160800 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=757384 CPUtime=1049.72 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 149415 1252 0 0 104950 22 0 0 25 0 5 0 654790526 775561216 160800 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=757384 CPUtime=1062.26 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 179530 1252 0 0 106197 29 0 0 25 0 5 0 654790526 775561216 160800 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=757384 CPUtime=1062.27 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 124552 1252 0 0 106206 21 0 0 25 0 5 0 654790526 775561216 160800 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4236.6
Current children cumulated vsize (KiB) 757384

[startup+1122.3 s]
/proc/loadavg: 8.06 8.10 8.05 9/363 21966
/proc/meminfo: memFree=19960984/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=791076 CPUtime=4475.9 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 686193 1252 0 0 447477 113 0 0 25 0 5 0 654790521 810061824 169223 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 197769 169223 180 55 0 195655 0
[pid=21912/tid=21919] ppid=21910 vsize=791076 CPUtime=1122.28 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 192930 1252 0 0 112194 34 0 0 25 0 5 0 654790526 810061824 169223 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=791076 CPUtime=1109.02 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 164460 1252 0 0 110877 25 0 0 25 0 5 0 654790526 810061824 169223 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=791076 CPUtime=1122.26 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 201969 1252 0 0 112194 32 0 0 25 0 5 0 654790526 810061824 169223 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=791076 CPUtime=1122.27 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 124552 1252 0 0 112206 21 0 0 25 0 5 0 654790526 810061824 169223 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4475.9
Current children cumulated vsize (KiB) 791076

[startup+1182.3 s]
/proc/loadavg: 8.02 8.08 8.04 9/363 21967
/proc/meminfo: memFree=19869276/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=889168 CPUtime=4715.21 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 710716 1252 0 0 471404 117 0 0 25 0 5 0 654790521 910508032 193746 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 222292 193746 180 55 0 220178 0
[pid=21912/tid=21919] ppid=21910 vsize=889168 CPUtime=1182.29 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 192930 1252 0 0 118195 34 0 0 25 0 5 0 654790526 910508032 193746 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=889168 CPUtime=1168.32 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 172652 1252 0 0 116806 26 0 0 25 0 5 0 654790526 910508032 193746 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364597 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=889168 CPUtime=1182.26 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 210161 1252 0 0 118193 33 0 0 25 0 5 0 654790526 910508032 193746 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=889168 CPUtime=1182.28 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 132691 1252 0 0 118205 23 0 0 25 0 5 0 654790526 910508032 193746 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4715.21
Current children cumulated vsize (KiB) 889168



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.6 s]
/proc/loadavg: 7.25 7.90 7.99 5/353 21986
/proc/meminfo: memFree=20564744/32951124 swapFree=65581052/67111528
[pid=21912] ppid=21910 vsize=810856 CPUtime=4800.17 cores=1,3,5,7
/proc/21912/stat : 21912 (plingeling) S 21910 21912 21414 0 -1 4202496 719177 1252 0 0 479899 118 0 0 25 0 5 0 654790521 830316544 174168 33554432000 4194304 4417452 140735708141440 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21912/statm: 202714 174168 180 55 0 200600 0
[pid=21912/tid=21919] ppid=21910 vsize=810856 CPUtime=1203.59 cores=1,3,5,7
/proc/21912/task/21919/stat : 21919 (plingeling) R 21910 21912 21414 0 -1 4202560 198271 1252 0 0 120325 34 0 0 25 0 5 0 654790526 830316544 174168 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21912/tid=21920] ppid=21910 vsize=810856 CPUtime=1189.36 cores=1,3,5,7
/proc/21912/task/21920/stat : 21920 (plingeling) R 21910 21912 21414 0 -1 4202560 172652 1252 0 0 118910 26 0 0 25 0 5 0 654790526 830316544 174168 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21912/tid=21921] ppid=21910 vsize=810856 CPUtime=1203.58 cores=1,3,5,7
/proc/21912/task/21921/stat : 21921 (plingeling) R 21910 21912 21414 0 -1 4202560 210161 1252 0 0 120325 33 0 0 25 0 5 0 654790526 830316544 174168 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21912/tid=21922] ppid=21910 vsize=810856 CPUtime=1203.59 cores=1,3,5,7
/proc/21912/task/21922/stat : 21922 (plingeling) R 21910 21912 21414 0 -1 4202560 135811 1252 0 0 120336 23 0 0 25 0 5 0 654790526 830316544 174168 33554432000 4194304 4417452 140735708141440 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.17
Current children cumulated vsize (KiB) 810856

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.65
CPU time (s): 4800.25
CPU user time (s): 4799.04
CPU system time (s): 1.20982
CPU usage (%): 398.809
Max. virtual memory (cumulated for all children) (KiB): 901648

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4799.04
system time used= 1.20982
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 720430
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 84
involuntary context switches= 36754

runsolver used 3.87841 second user time and 10.3674 second system time

The end

Launcher Data 

Begin job on node137 at 2011-05-15 05:58:48
IDJOB=3371047
IDBENCH=85805
IDSOLVER=1848
FILE ID=node137/3371047-1305431928
RUNJOBID= node137-1305430710-21772
PBS_JOBID= 13324606
Free space on /tmp= 73796 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT05/crafted/sorge05/QG7a/gensys-ukn004.shuffled-as.sat05-3844.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371047-1305431928/watcher-3371047-1305431928 -o /tmp/evaluation-result-3371047-1305431928/solver-3371047-1305431928 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371047-1305431928.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 7d96ba103689bec761c9e7df7b158323
RANDOM SEED=1832968612

node137.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21207928 kB
Buffers:        292172 kB
Cached:        3730684 kB
SwapCached:      36328 kB
Active:        1715344 kB
Inactive:      9803780 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21207928 kB
SwapTotal:    67111528 kB
SwapFree:     65581052 kB
Dirty:            1856 kB
Writeback:           0 kB
AnonPages:     7469776 kB
Mapped:          19536 kB
Slab:           121412 kB
PageTables:      38940 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 76011036 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73796 MiB
End job on node137 at 2011-05-15 06:18:54