Trace number 3394562

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40001 5012.27

General information on the benchmark

Namecrafted/kullmann/GreenTao/
GreenTaok1k2/GreenTao_2-4-4_512.cnf
MD5SUMc1f047ec179094963b21d7b90509d2b9
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables510
Number of clauses4492
Sum of the clauses size17968
Maximum clause length4
Minimum clause length4
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 44492
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394562-1306099741/watcher-3394562-1306099741 -o /tmp/evaluation-result-3394562-1306099741/solver-3394562-1306099741 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394562-1306099741.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.42 7.92 7.96 2/551 32457
/proc/meminfo: memFree=25551428/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=8728 CPUtime=0 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 214 0 0 0 0 0 0 0 25 0 1 0 800204806 8937472 146 33554432000 4194304 4417452 140735192165024 18446744073709551615 224284925696 0 0 4096 0 18446744071562362840 0 0 17 6 0 0 0
/proc/32457/statm: 2182 146 122 55 0 68 0

[startup+0.0634941 s]
/proc/loadavg: 7.42 7.92 7.96 2/551 32457
/proc/meminfo: memFree=25551428/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=8728 CPUtime=0 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 214 0 0 0 0 0 0 0 25 0 1 0 800204806 8937472 146 33554432000 4194304 4417452 140735192165024 18446744073709551615 224284925696 0 0 4096 0 18446744071562362840 0 0 17 6 0 0 0
/proc/32457/statm: 2182 146 122 55 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8728

[startup+0.125631 s]
/proc/loadavg: 7.42 7.92 7.96 2/551 32457
/proc/meminfo: memFree=25551428/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=290852 CPUtime=0.24 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 1321 1246 0 5 24 0 0 0 18 0 9 0 800204806 297832448 1181 33554432000 4194304 4417452 140735192165024 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/32457/statm: 72713 1181 179 55 0 70599 0
Current children cumulated CPU time (s) 0.24
Current children cumulated vsize (KiB) 290852

[startup+0.300401 s]
/proc/loadavg: 7.42 7.92 7.96 2/551 32457
/proc/meminfo: memFree=25551428/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=231260 CPUtime=1.64 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 3727 1246 0 5 164 0 0 0 18 0 9 0 800204806 236810240 2783 33554432000 4194304 4417452 140735192165024 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/32457/statm: 57815 2783 179 55 0 55701 0
Current children cumulated CPU time (s) 1.64
Current children cumulated vsize (KiB) 231260

[startup+0.700337 s]
/proc/loadavg: 7.42 7.92 7.96 2/551 32457
/proc/meminfo: memFree=25551428/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=237088 CPUtime=4.84 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 9467 1246 0 5 483 1 0 0 18 0 9 0 800204806 242778112 4248 33554432000 4194304 4417452 140735192165024 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/32457/statm: 59272 4248 179 55 0 57158 0
Current children cumulated CPU time (s) 4.84
Current children cumulated vsize (KiB) 237088

[startup+1.50121 s]
/proc/loadavg: 7.42 7.92 7.96 9/560 32471
/proc/meminfo: memFree=25530648/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=244496 CPUtime=11.22 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 20901 1246 0 5 1119 3 0 0 18 0 9 0 800204806 250363904 6155 33554432000 4194304 4417452 140735192165024 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/32457/statm: 61124 6155 179 55 0 59010 0
[pid=32457/tid=32464] ppid=32455 vsize=244496 CPUtime=1.4 cores=0-7
/proc/32457/task/32464/stat : 32464 (plingeling) R 32455 32457 32431 0 -1 4202560 2665 1246 0 5 140 0 0 0 19 0 9 0 800204815 250363904 6155 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32457/tid=32465] ppid=32455 vsize=244496 CPUtime=1.39 cores=0-7
/proc/32457/task/32465/stat : 32465 (plingeling) R 32455 32457 32431 0 -1 4202560 2324 1246 0 5 139 0 0 0 25 0 9 0 800204815 250363904 6155 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=32457/tid=32466] ppid=32455 vsize=244496 CPUtime=1.4 cores=0-7
/proc/32457/task/32466/stat : 32466 (plingeling) R 32455 32457 32431 0 -1 4202560 2318 1246 0 5 140 0 0 0 19 0 9 0 800204815 250363904 6155 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=32457/tid=32467] ppid=32455 vsize=244496 CPUtime=1.39 cores=0-7
/proc/32457/task/32467/stat : 32467 (plingeling) R 32455 32457 32431 0 -1 4202560 2484 1246 0 5 139 0 0 0 25 0 9 0 800204815 250363904 6155 33554432000 4194304 4417452 140735192165024 18446744073709551615 4199406 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=32457/tid=32468] ppid=32455 vsize=244496 CPUtime=1.4 cores=0-7
/proc/32457/task/32468/stat : 32468 (plingeling) R 32455 32457 32431 0 -1 4202560 2549 1246 0 5 140 0 0 0 20 0 9 0 800204815 250363904 6155 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=32457/tid=32469] ppid=32455 vsize=244496 CPUtime=1.4 cores=0-7
/proc/32457/task/32469/stat : 32469 (plingeling) R 32455 32457 32431 0 -1 4202560 2452 1246 0 5 140 0 0 0 25 0 9 0 800204815 250363904 6155 33554432000 4194304 4417452 140735192165024 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=32457/tid=32470] ppid=32455 vsize=244496 CPUtime=1.38 cores=0-7
/proc/32457/task/32470/stat : 32470 (plingeling) R 32455 32457 32431 0 -1 4202560 2703 1246 0 5 138 0 0 0 25 0 9 0 800204815 250363904 6155 33554432000 4194304 4417452 140735192165024 18446744073709551615 4353607 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32457/tid=32471] ppid=32455 vsize=244496 CPUtime=1.4 cores=0-7
/proc/32457/task/32471/stat : 32471 (plingeling) R 32455 32457 32431 0 -1 4202560 2469 1246 0 5 140 0 0 0 19 0 9 0 800204816 250363904 6155 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.22
Current children cumulated vsize (KiB) 244496

[startup+3.10094 s]
/proc/loadavg: 7.42 7.92 7.96 9/560 32471
/proc/meminfo: memFree=25522588/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=255736 CPUtime=24 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 41061 1246 0 5 2392 8 0 0 18 0 9 0 800204806 261873664 8985 33554432000 4194304 4417452 140735192165024 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/32457/statm: 63934 8985 179 55 0 61820 0
[pid=32457/tid=32464] ppid=32455 vsize=255736 CPUtime=3 cores=0-7
/proc/32457/task/32464/stat : 32464 (plingeling) R 32455 32457 32431 0 -1 4202560 5661 1246 0 5 299 1 0 0 25 0 9 0 800204815 261873664 8985 33554432000 4194304 4417452 140735192165024 18446744073709551615 4365075 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32457/tid=32465] ppid=32455 vsize=255736 CPUtime=2.99 cores=0-7
/proc/32457/task/32465/stat : 32465 (plingeling) R 32455 32457 32431 0 -1 4202560 4585 1246 0 5 298 1 0 0 25 0 9 0 800204815 261873664 8985 33554432000 4194304 4417452 140735192165024 18446744073709551615 4356075 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=32457/tid=32466] ppid=32455 vsize=255772 CPUtime=2.99 cores=0-7
/proc/32457/task/32466/stat : 32466 (plingeling) R 32455 32457 32431 0 -1 4202560 5212 1246 0 5 299 0 0 0 25 0 9 0 800204815 261910528 8994 33554432000 4194304 4417452 140735192165024 18446744073709551615 224284619752 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=32457/tid=32467] ppid=32455 vsize=255772 CPUtime=2.99 cores=0-7
/proc/32457/task/32467/stat : 32467 (plingeling) R 32455 32457 32431 0 -1 4202560 4945 1246 0 5 298 1 0 0 25 0 9 0 800204815 261910528 8994 33554432000 4194304 4417452 140735192165024 18446744073709551615 4199406 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=32457/tid=32468] ppid=32455 vsize=255772 CPUtime=3 cores=0-7
/proc/32457/task/32468/stat : 32468 (plingeling) R 32455 32457 32431 0 -1 4202560 4305 1246 0 5 300 0 0 0 25 0 9 0 800204815 261910528 8994 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=32457/tid=32469] ppid=32455 vsize=255772 CPUtime=2.99 cores=0-7
/proc/32457/task/32469/stat : 32469 (plingeling) R 32455 32457 32431 0 -1 4202560 5028 1246 0 5 299 0 0 0 25 0 9 0 800204815 261910528 8994 33554432000 4194304 4417452 140735192165024 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=32457/tid=32470] ppid=32455 vsize=255772 CPUtime=2.95 cores=0-7
/proc/32457/task/32470/stat : 32470 (plingeling) R 32455 32457 32431 0 -1 4202560 5702 1246 0 5 295 0 0 0 25 0 9 0 800204815 261910528 8994 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32457/tid=32471] ppid=32455 vsize=255772 CPUtime=2.99 cores=0-7
/proc/32457/task/32471/stat : 32471 (plingeling) R 32455 32457 32431 0 -1 4202560 4695 1246 0 5 299 0 0 0 25 0 9 0 800204816 261910528 8994 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24
Current children cumulated vsize (KiB) 255736

[startup+6.30041 s]
/proc/loadavg: 7.47 7.92 7.96 9/560 32471
/proc/meminfo: memFree=25510820/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=264836 CPUtime=49.54 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 70442 1246 0 5 4939 15 0 0 18 0 9 0 800204806 271192064 11294 33554432000 4194304 4417452 140735192165024 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/32457/statm: 66209 11294 179 55 0 64095 0
[pid=32457/tid=32464] ppid=32455 vsize=264836 CPUtime=6.2 cores=0-7
/proc/32457/task/32464/stat : 32464 (plingeling) R 32455 32457 32431 0 -1 4202560 9771 1246 0 5 618 2 0 0 25 0 9 0 800204815 271192064 11294 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32457/tid=32465] ppid=32455 vsize=264836 CPUtime=6.2 cores=0-7
/proc/32457/task/32465/stat : 32465 (plingeling) R 32455 32457 32431 0 -1 4202560 8025 1246 0 5 618 2 0 0 25 0 9 0 800204815 271192064 11294 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=32457/tid=32466] ppid=32455 vsize=264836 CPUtime=6.19 cores=0-7
/proc/32457/task/32466/stat : 32466 (plingeling) R 32455 32457 32431 0 -1 4202560 8246 1246 0 5 618 1 0 0 25 0 9 0 800204815 271192064 11294 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364984 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=32457/tid=32467] ppid=32455 vsize=264836 CPUtime=6.19 cores=0-7
/proc/32457/task/32467/stat : 32467 (plingeling) R 32455 32457 32431 0 -1 4202560 8572 1246 0 5 617 2 0 0 25 0 9 0 800204815 271192064 11294 33554432000 4194304 4417452 140735192165024 18446744073709551615 4199406 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=32457/tid=32468] ppid=32455 vsize=264836 CPUtime=6.2 cores=0-7
/proc/32457/task/32468/stat : 32468 (plingeling) R 32455 32457 32431 0 -1 4202560 7966 1246 0 5 619 1 0 0 25 0 9 0 800204815 271192064 11294 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=32457/tid=32469] ppid=32455 vsize=264836 CPUtime=6.19 cores=0-7
/proc/32457/task/32469/stat : 32469 (plingeling) R 32455 32457 32431 0 -1 4202560 8218 1246 0 5 618 1 0 0 25 0 9 0 800204815 271192064 11294 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=32457/tid=32470] ppid=32455 vsize=264836 CPUtime=6.09 cores=0-7
/proc/32457/task/32470/stat : 32470 (plingeling) R 32455 32457 32431 0 -1 4202560 10539 1246 0 5 608 1 0 0 25 0 9 0 800204815 271192064 11294 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32457/tid=32471] ppid=32455 vsize=264836 CPUtime=6.19 cores=0-7
/proc/32457/task/32471/stat : 32471 (plingeling) R 32455 32457 32431 0 -1 4202560 8168 1246 0 5 618 1 0 0 25 0 9 0 800204816 271192064 11294 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 49.54
Current children cumulated vsize (KiB) 264836

[startup+12.7004 s]
/proc/loadavg: 7.51 7.92 7.96 9/560 32471
/proc/meminfo: memFree=25482712/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=291300 CPUtime=100.6 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 121613 1246 0 5 10034 26 0 0 18 0 9 0 800204806 298291200 17943 33554432000 4194304 4417452 140735192165024 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/32457/statm: 72825 17943 179 55 0 70711 0
[pid=32457/tid=32464] ppid=32455 vsize=291300 CPUtime=12.6 cores=0-7
/proc/32457/task/32464/stat : 32464 (plingeling) R 32455 32457 32431 0 -1 4202560 16310 1246 0 5 1257 3 0 0 25 0 9 0 800204815 298291200 17943 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32457/tid=32465] ppid=32455 vsize=291300 CPUtime=12.59 cores=0-7
/proc/32457/task/32465/stat : 32465 (plingeling) R 32455 32457 32431 0 -1 4202560 14708 1246 0 5 1256 3 0 0 25 0 9 0 800204815 298291200 17943 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=32457/tid=32466] ppid=32455 vsize=291300 CPUtime=12.59 cores=0-7
/proc/32457/task/32466/stat : 32466 (plingeling) R 32455 32457 32431 0 -1 4202560 14750 1246 0 5 1257 2 0 0 25 0 9 0 800204815 298291200 17943 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=32457/tid=32467] ppid=32455 vsize=291300 CPUtime=12.59 cores=0-7
/proc/32457/task/32467/stat : 32467 (plingeling) R 32455 32457 32431 0 -1 4202560 14384 1246 0 5 1256 3 0 0 25 0 9 0 800204815 298291200 17943 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=32457/tid=32468] ppid=32455 vsize=291300 CPUtime=12.6 cores=0-7
/proc/32457/task/32468/stat : 32468 (plingeling) R 32455 32457 32431 0 -1 4202560 14529 1246 0 5 1258 2 0 0 25 0 9 0 800204815 298291200 17943 33554432000 4194304 4417452 140735192165024 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=32457/tid=32469] ppid=32455 vsize=291300 CPUtime=12.6 cores=0-7
/proc/32457/task/32469/stat : 32469 (plingeling) R 32455 32457 32431 0 -1 4202560 13902 1246 0 5 1257 3 0 0 25 0 9 0 800204815 298291200 17943 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=32457/tid=32470] ppid=32455 vsize=291300 CPUtime=12.38 cores=0-7
/proc/32457/task/32470/stat : 32470 (plingeling) R 32455 32457 32431 0 -1 4202560 17922 1246 0 5 1234 4 0 0 25 0 9 0 800204815 298291200 17943 33554432000 4194304 4417452 140735192165024 18446744073709551615 18446744073699065932 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32457/tid=32471] ppid=32455 vsize=291300 CPUtime=12.58 cores=0-7
/proc/32457/task/32471/stat : 32471 (plingeling) R 32455 32457 32431 0 -1 4202560 14171 1246 0 5 1256 2 0 0 25 0 9 0 800204816 298291200 17943 33554432000 4194304 4417452 140735192165024 18446744073709551615 4366469 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 100.6
Current children cumulated vsize (KiB) 291300

[startup+25.5012 s]
/proc/loadavg: 7.62 7.93 7.96 9/560 32473
/proc/meminfo: memFree=25475016/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=304796 CPUtime=202.77 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 214349 1246 0 5 20234 43 0 0 18 0 9 0 800204806 312111104 21465 33554432000 4194304 4417452 140735192165024 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/32457/statm: 76199 21465 179 55 0 74085 0
[pid=32457/tid=32464] ppid=32455 vsize=304796 CPUtime=25.4 cores=0-7
/proc/32457/task/32464/stat : 32464 (plingeling) R 32455 32457 32431 0 -1 4202560 28450 1246 0 5 2535 5 0 0 25 0 9 0 800204815 312111104 21465 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32457/tid=32465] ppid=32455 vsize=304796 CPUtime=25.4 cores=0-7
/proc/32457/task/32465/stat : 32465 (plingeling) R 32455 32457 32431 0 -1 4202560 29622 1246 0 5 2534 6 0 0 25 0 9 0 800204815 312111104 21465 33554432000 4194304 4417452 140735192165024 18446744073709551615 4236906 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=32457/tid=32466] ppid=32455 vsize=304796 CPUtime=25.39 cores=0-7
/proc/32457/task/32466/stat : 32466 (plingeling) R 32455 32457 32431 0 -1 4202560 28546 1246 0 5 2534 5 0 0 25 0 9 0 800204815 312111104 21465 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=32457/tid=32467] ppid=32455 vsize=304796 CPUtime=25.39 cores=0-7
/proc/32457/task/32467/stat : 32467 (plingeling) R 32455 32457 32431 0 -1 4202560 24238 1246 0 5 2534 5 0 0 25 0 9 0 800204815 312111104 21465 33554432000 4194304 4417452 140735192165024 18446744073709551615 4366402 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=32457/tid=32468] ppid=32455 vsize=304796 CPUtime=25.4 cores=0-7
/proc/32457/task/32468/stat : 32468 (plingeling) R 32455 32457 32431 0 -1 4202560 23765 1246 0 5 2535 5 0 0 25 0 9 0 800204815 312111104 21465 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=32457/tid=32469] ppid=32455 vsize=304796 CPUtime=25.4 cores=0-7
/proc/32457/task/32469/stat : 32469 (plingeling) R 32455 32457 32431 0 -1 4202560 24943 1246 0 5 2535 5 0 0 25 0 9 0 800204815 312111104 21465 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=32457/tid=32470] ppid=32455 vsize=304796 CPUtime=24.93 cores=0-7
/proc/32457/task/32470/stat : 32470 (plingeling) R 32455 32457 32431 0 -1 4202560 29902 1246 0 5 2487 6 0 0 25 0 9 0 800204815 312111104 21465 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32457/tid=32471] ppid=32455 vsize=304796 CPUtime=25.38 cores=0-7
/proc/32457/task/32471/stat : 32471 (plingeling) R 32455 32457 32431 0 -1 4202560 23946 1246 0 5 2534 4 0 0 25 0 9 0 800204816 312111104 21465 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 202.77
Current children cumulated vsize (KiB) 304796

[startup+51.101 s]
/proc/loadavg: 7.75 7.93 7.96 9/560 32473
/proc/meminfo: memFree=25445956/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=337012 CPUtime=407.11 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 372442 1246 0 5 40635 76 0 0 18 0 9 0 800204806 345100288 29653 33554432000 4194304 4417452 140735192165024 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/32457/statm: 84253 29653 179 55 0 82139 0
[pid=32457/tid=32464] ppid=32455 vsize=337012 CPUtime=50.99 cores=0-7
/proc/32457/task/32464/stat : 32464 (plingeling) R 32455 32457 32431 0 -1 4202560 47740 1246 0 5 5090 9 0 0 25 0 9 0 800204815 345100288 29653 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32457/tid=32465] ppid=32455 vsize=337012 CPUtime=51 cores=0-7
/proc/32457/task/32465/stat : 32465 (plingeling) R 32455 32457 32431 0 -1 4202560 49934 1246 0 5 5090 10 0 0 25 0 9 0 800204815 345100288 29653 33554432000 4194304 4417452 140735192165024 18446744073709551615 4357879 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=32457/tid=32466] ppid=32455 vsize=337012 CPUtime=51 cores=0-7

################
# More data... #
################

/proc/32457/task/32467/stat : 32467 (plingeling) R 32455 32457 32431 0 -1 4202560 842088 1246 0 5 466076 147 0 0 25 0 9 0 800204815 841609216 153338 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=32457/tid=32468] ppid=32455 vsize=821884 CPUtime=4662.09 cores=0-7
/proc/32457/task/32468/stat : 32468 (plingeling) R 32455 32457 32431 0 -1 4202560 877366 1246 0 5 466052 157 0 0 25 0 9 0 800204815 841609216 153338 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=32457/tid=32469] ppid=32455 vsize=821884 CPUtime=4662.26 cores=0-7
/proc/32457/task/32469/stat : 32469 (plingeling) R 32455 32457 32431 0 -1 4202560 739887 1246 0 5 466091 135 0 0 25 0 9 0 800204815 841609216 153338 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=32457/tid=32470] ppid=32455 vsize=821884 CPUtime=4575.45 cores=0-7
/proc/32457/task/32470/stat : 32470 (plingeling) R 32455 32457 32431 0 -1 4202560 972905 1246 0 5 457373 172 0 0 25 0 9 0 800204815 841609216 153338 33554432000 4194304 4417452 140735192165024 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32457/tid=32471] ppid=32455 vsize=821884 CPUtime=4661.34 cores=0-7
/proc/32457/task/32471/stat : 32471 (plingeling) R 32455 32457 32431 0 -1 4202560 835030 1246 0 5 465990 144 0 0 25 0 9 0 800204816 841609216 153338 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37208.1
Current children cumulated vsize (KiB) 821884

[startup+4722.32 s]
/proc/loadavg: 8.12 8.10 8.04 9/560 32612
/proc/meminfo: memFree=24876064/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=880612 CPUtime=37687.1 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 6935996 1246 0 5 3767506 1208 0 0 18 0 9 0 800204806 901746688 168053 33554432000 4194304 4417452 140735192165024 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/32457/statm: 220153 168053 179 55 0 218039 0
[pid=32457/tid=32464] ppid=32455 vsize=880612 CPUtime=4721.78 cores=0-7
/proc/32457/task/32464/stat : 32464 (plingeling) R 32455 32457 32431 0 -1 4202560 964802 1246 0 5 472018 160 0 0 25 0 9 0 800204815 901746688 168053 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32457/tid=32465] ppid=32455 vsize=880612 CPUtime=4722.28 cores=0-7
/proc/32457/task/32465/stat : 32465 (plingeling) R 32455 32457 32431 0 -1 4202560 868413 1246 0 5 472076 152 0 0 25 0 9 0 800204815 901746688 168053 33554432000 4194304 4417452 140735192165024 18446744073709551615 4365004 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=32457/tid=32466] ppid=32455 vsize=880612 CPUtime=4720.66 cores=0-7
/proc/32457/task/32466/stat : 32466 (plingeling) R 32455 32457 32431 0 -1 4202560 803480 1246 0 5 471932 134 0 0 25 0 9 0 800204815 901746688 168053 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=32457/tid=32467] ppid=32455 vsize=880612 CPUtime=4722.25 cores=0-7
/proc/32457/task/32467/stat : 32467 (plingeling) R 32455 32457 32431 0 -1 4202560 845789 1246 0 5 472077 148 0 0 25 0 9 0 800204815 901746688 168053 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=32457/tid=32468] ppid=32455 vsize=880612 CPUtime=4722.1 cores=0-7
/proc/32457/task/32468/stat : 32468 (plingeling) R 32455 32457 32431 0 -1 4202560 890783 1246 0 5 472052 158 0 0 25 0 9 0 800204815 901746688 168053 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=32457/tid=32469] ppid=32455 vsize=880612 CPUtime=4722.28 cores=0-7
/proc/32457/task/32469/stat : 32469 (plingeling) R 32455 32457 32431 0 -1 4202560 744173 1246 0 5 472093 135 0 0 25 0 9 0 800204815 901746688 168053 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=32457/tid=32470] ppid=32455 vsize=880612 CPUtime=4634.35 cores=0-7
/proc/32457/task/32470/stat : 32470 (plingeling) R 32455 32457 32431 0 -1 4202560 975362 1246 0 5 463263 172 0 0 25 0 9 0 800204815 901746688 168053 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32457/tid=32471] ppid=32455 vsize=880612 CPUtime=4721.36 cores=0-7
/proc/32457/task/32471/stat : 32471 (plingeling) R 32455 32457 32431 0 -1 4202560 842257 1246 0 5 471991 145 0 0 25 0 9 0 800204816 901746688 168053 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37687.1
Current children cumulated vsize (KiB) 880612

[startup+4782.3 s]
/proc/loadavg: 8.04 8.08 8.03 9/560 32614
/proc/meminfo: memFree=24880992/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=880156 CPUtime=38165.9 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 6993610 1246 0 5 3815368 1219 0 0 18 0 9 0 800204806 901279744 166745 33554432000 4194304 4417452 140735192165024 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/32457/statm: 220039 166745 179 55 0 217925 0
[pid=32457/tid=32464] ppid=32455 vsize=880156 CPUtime=4781.74 cores=0-7
/proc/32457/task/32464/stat : 32464 (plingeling) R 32455 32457 32431 0 -1 4202560 971803 1246 0 5 478012 162 0 0 25 0 9 0 800204815 901279744 166745 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32457/tid=32465] ppid=32455 vsize=880156 CPUtime=4782.26 cores=0-7
/proc/32457/task/32465/stat : 32465 (plingeling) R 32455 32457 32431 0 -1 4202560 874973 1246 0 5 478074 152 0 0 25 0 9 0 800204815 901279744 166745 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=32457/tid=32466] ppid=32455 vsize=880156 CPUtime=4780.64 cores=0-7
/proc/32457/task/32466/stat : 32466 (plingeling) R 32455 32457 32431 0 -1 4202560 813036 1246 0 5 477928 136 0 0 25 0 9 0 800204815 901279744 166745 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=32457/tid=32467] ppid=32455 vsize=880156 CPUtime=4782.23 cores=0-7
/proc/32457/task/32467/stat : 32467 (plingeling) R 32455 32457 32431 0 -1 4202560 853981 1246 0 5 478072 151 0 0 25 0 9 0 800204815 901279744 166745 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=32457/tid=32468] ppid=32455 vsize=880156 CPUtime=4782.08 cores=0-7
/proc/32457/task/32468/stat : 32468 (plingeling) R 32455 32457 32431 0 -1 4202560 890783 1246 0 5 478050 158 0 0 25 0 9 0 800204815 901279744 166745 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=32457/tid=32469] ppid=32455 vsize=880156 CPUtime=4782.26 cores=0-7
/proc/32457/task/32469/stat : 32469 (plingeling) R 32455 32457 32431 0 -1 4202560 750672 1246 0 5 478089 137 0 0 25 0 9 0 800204815 901279744 166745 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=32457/tid=32470] ppid=32455 vsize=880156 CPUtime=4693.24 cores=0-7
/proc/32457/task/32470/stat : 32470 (plingeling) R 32455 32457 32431 0 -1 4202560 988966 1246 0 5 469150 174 0 0 25 0 9 0 800204815 901279744 166745 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32457/tid=32471] ppid=32455 vsize=880156 CPUtime=4781.34 cores=0-7
/proc/32457/task/32471/stat : 32471 (plingeling) R 32455 32457 32431 0 -1 4202560 848459 1246 0 5 477988 146 0 0 25 0 9 0 800204816 901279744 166745 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38165.9
Current children cumulated vsize (KiB) 880156

[startup+4842.3 s]
/proc/loadavg: 8.01 8.06 8.02 9/560 32616
/proc/meminfo: memFree=24925088/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=834088 CPUtime=38644.7 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 7048193 1246 0 5 3863247 1227 0 0 18 0 9 0 800204806 854106112 155786 33554432000 4194304 4417452 140735192165024 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/32457/statm: 208522 155786 179 55 0 206408 0
[pid=32457/tid=32464] ppid=32455 vsize=834088 CPUtime=4841.72 cores=0-7
/proc/32457/task/32464/stat : 32464 (plingeling) R 32455 32457 32431 0 -1 4202560 976395 1246 0 5 484010 162 0 0 25 0 9 0 800204815 854106112 155786 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32457/tid=32465] ppid=32455 vsize=834088 CPUtime=4842.26 cores=0-7
/proc/32457/task/32465/stat : 32465 (plingeling) R 32455 32457 32431 0 -1 4202560 884233 1246 0 5 484072 154 0 0 25 0 9 0 800204815 854106112 155786 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=32457/tid=32466] ppid=32455 vsize=834088 CPUtime=4840.64 cores=0-7
/proc/32457/task/32466/stat : 32466 (plingeling) R 32455 32457 32431 0 -1 4202560 816694 1246 0 5 483928 136 0 0 25 0 9 0 800204815 854106112 155786 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=32457/tid=32467] ppid=32455 vsize=834088 CPUtime=4842.23 cores=0-7
/proc/32457/task/32467/stat : 32467 (plingeling) R 32455 32457 32431 0 -1 4202560 863066 1246 0 5 484070 153 0 0 25 0 9 0 800204815 854106112 155786 33554432000 4194304 4417452 140735192165024 18446744073709551615 4365062 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=32457/tid=32468] ppid=32455 vsize=834088 CPUtime=4842.08 cores=0-7
/proc/32457/task/32468/stat : 32468 (plingeling) R 32455 32457 32431 0 -1 4202560 903580 1246 0 5 484048 160 0 0 25 0 9 0 800204815 854106112 155786 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=32457/tid=32469] ppid=32455 vsize=834088 CPUtime=4842.26 cores=0-7
/proc/32457/task/32469/stat : 32469 (plingeling) R 32455 32457 32431 0 -1 4202560 754627 1246 0 5 484089 137 0 0 25 0 9 0 800204815 854106112 155786 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=32457/tid=32470] ppid=32455 vsize=834088 CPUtime=4752.13 cores=0-7
/proc/32457/task/32470/stat : 32470 (plingeling) R 32455 32457 32431 0 -1 4202560 994895 1246 0 5 475039 174 0 0 25 0 9 0 800204815 854106112 155786 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32457/tid=32471] ppid=32455 vsize=834088 CPUtime=4841.34 cores=0-7
/proc/32457/task/32471/stat : 32471 (plingeling) R 32455 32457 32431 0 -1 4202560 853766 1246 0 5 483987 147 0 0 25 0 9 0 800204816 854106112 155786 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38644.7
Current children cumulated vsize (KiB) 834088

[startup+4902.3 s]
/proc/loadavg: 8.07 8.06 8.02 9/560 32618
/proc/meminfo: memFree=24809984/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=946744 CPUtime=39123.6 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 7104442 1246 0 5 3911127 1237 0 0 18 0 9 0 800204806 969465856 184595 33554432000 4194304 4417452 140735192165024 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/32457/statm: 236686 184595 179 55 0 234572 0
[pid=32457/tid=32464] ppid=32455 vsize=946744 CPUtime=4901.71 cores=0-7
/proc/32457/task/32464/stat : 32464 (plingeling) R 32455 32457 32431 0 -1 4202560 984664 1246 0 5 490008 163 0 0 25 0 9 0 800204815 969465856 184595 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32457/tid=32465] ppid=32455 vsize=946744 CPUtime=4902.25 cores=0-7
/proc/32457/task/32465/stat : 32465 (plingeling) R 32455 32457 32431 0 -1 4202560 889700 1246 0 5 490071 154 0 0 25 0 9 0 800204815 969465856 184595 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364558 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=32457/tid=32466] ppid=32455 vsize=946744 CPUtime=4900.64 cores=0-7
/proc/32457/task/32466/stat : 32466 (plingeling) R 32455 32457 32431 0 -1 4202560 830703 1246 0 5 489925 139 0 0 25 0 9 0 800204815 969465856 184595 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=32457/tid=32467] ppid=32455 vsize=946744 CPUtime=4902.23 cores=0-7
/proc/32457/task/32467/stat : 32467 (plingeling) R 32455 32457 32431 0 -1 4202560 868342 1246 0 5 490070 153 0 0 25 0 9 0 800204815 969465856 184595 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=32457/tid=32468] ppid=32455 vsize=946744 CPUtime=4902.09 cores=0-7
/proc/32457/task/32468/stat : 32468 (plingeling) R 32455 32457 32431 0 -1 4202560 910035 1246 0 5 490047 162 0 0 25 0 9 0 800204815 969465856 184595 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=32457/tid=32469] ppid=32455 vsize=946744 CPUtime=4902.26 cores=0-7
/proc/32457/task/32469/stat : 32469 (plingeling) R 32455 32457 32431 0 -1 4202560 756950 1246 0 5 490088 138 0 0 25 0 9 0 800204815 969465856 184595 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=32457/tid=32470] ppid=32455 vsize=946744 CPUtime=4811.04 cores=0-7
/proc/32457/task/32470/stat : 32470 (plingeling) R 32455 32457 32431 0 -1 4202560 1004678 1246 0 5 480928 176 0 0 25 0 9 0 800204815 969465856 184595 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32457/tid=32471] ppid=32455 vsize=946744 CPUtime=4901.34 cores=0-7
/proc/32457/task/32471/stat : 32471 (plingeling) R 32455 32457 32431 0 -1 4202560 858433 1246 0 5 489986 148 0 0 25 0 9 0 800204816 969465856 184595 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39123.6
Current children cumulated vsize (KiB) 946744

[startup+4962.3 s]
/proc/loadavg: 8.08 8.06 8.02 9/560 32618
/proc/meminfo: memFree=24898972/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=873048 CPUtime=39602.5 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 7149696 1246 0 5 3959006 1244 0 0 18 0 9 0 800204806 894001152 166203 33554432000 4194304 4417452 140735192165024 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/32457/statm: 218262 166203 179 55 0 216148 0
[pid=32457/tid=32464] ppid=32455 vsize=873048 CPUtime=4961.68 cores=0-7
/proc/32457/task/32464/stat : 32464 (plingeling) R 32455 32457 32431 0 -1 4202560 989353 1246 0 5 496004 164 0 0 25 0 9 0 800204815 894001152 166203 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32457/tid=32465] ppid=32455 vsize=873048 CPUtime=4962.26 cores=0-7
/proc/32457/task/32465/stat : 32465 (plingeling) R 32455 32457 32431 0 -1 4202560 897629 1246 0 5 496070 156 0 0 25 0 9 0 800204815 894001152 166203 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=32457/tid=32466] ppid=32455 vsize=873048 CPUtime=4960.64 cores=0-7
/proc/32457/task/32466/stat : 32466 (plingeling) R 32455 32457 32431 0 -1 4202560 836190 1246 0 5 495924 140 0 0 25 0 9 0 800204815 894001152 166203 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=32457/tid=32467] ppid=32455 vsize=873048 CPUtime=4962.23 cores=0-7
/proc/32457/task/32467/stat : 32467 (plingeling) R 32455 32457 32431 0 -1 4202560 869566 1246 0 5 496070 153 0 0 25 0 9 0 800204815 894001152 166203 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=32457/tid=32468] ppid=32455 vsize=873048 CPUtime=4962.08 cores=0-7
/proc/32457/task/32468/stat : 32468 (plingeling) R 32455 32457 32431 0 -1 4202560 917443 1246 0 5 496045 163 0 0 25 0 9 0 800204815 894001152 166203 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364812 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=32457/tid=32469] ppid=32455 vsize=873048 CPUtime=4962.26 cores=0-7
/proc/32457/task/32469/stat : 32469 (plingeling) R 32455 32457 32431 0 -1 4202560 763729 1246 0 5 496087 139 0 0 25 0 9 0 800204815 894001152 166203 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=32457/tid=32470] ppid=32455 vsize=873048 CPUtime=4869.92 cores=0-7
/proc/32457/task/32470/stat : 32470 (plingeling) R 32455 32457 32431 0 -1 4202560 1010942 1246 0 5 486816 176 0 0 25 0 9 0 800204815 894001152 166203 33554432000 4194304 4417452 140735192165024 18446744073709551615 4251091 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32457/tid=32471] ppid=32455 vsize=873048 CPUtime=4961.34 cores=0-7
/proc/32457/task/32471/stat : 32471 (plingeling) R 32455 32457 32431 0 -1 4202560 863907 1246 0 5 495985 149 0 0 25 0 9 0 800204816 894001152 166203 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39602.5
Current children cumulated vsize (KiB) 873048



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5012.2 s]
/proc/loadavg: 8.10 8.07 8.02 9/560 32620
/proc/meminfo: memFree=24905748/32951124 swapFree=39849776/67111528
[pid=32457] ppid=32455 vsize=853232 CPUtime=40000.9 cores=0-7
/proc/32457/stat : 32457 (plingeling) S 32455 32457 32431 0 -1 4202496 7209617 1246 0 5 3998838 1254 0 0 18 0 9 0 800204806 873709568 160616 33554432000 4194304 4417452 140735192165024 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/32457/statm: 213308 160616 179 55 0 211194 0
[pid=32457/tid=32464] ppid=32455 vsize=853232 CPUtime=5011.59 cores=0-7
/proc/32457/task/32464/stat : 32464 (plingeling) R 32455 32457 32431 0 -1 4202560 996401 1246 0 5 500994 165 0 0 25 0 9 0 800204815 873709568 160616 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=32457/tid=32465] ppid=32455 vsize=853232 CPUtime=5012.18 cores=0-7
/proc/32457/task/32465/stat : 32465 (plingeling) R 32455 32457 32431 0 -1 4202560 916914 1246 0 5 501060 158 0 0 25 0 9 0 800204815 873709568 160616 33554432000 4194304 4417452 140735192165024 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=32457/tid=32466] ppid=32455 vsize=853232 CPUtime=5010.57 cores=0-7
/proc/32457/task/32466/stat : 32466 (plingeling) R 32455 32457 32431 0 -1 4202560 841028 1246 0 5 500916 141 0 0 25 0 9 0 800204815 873709568 160616 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364780 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=32457/tid=32467] ppid=32455 vsize=853232 CPUtime=5012.15 cores=0-7
/proc/32457/task/32467/stat : 32467 (plingeling) R 32455 32457 32431 0 -1 4202560 875115 1246 0 5 501060 155 0 0 25 0 9 0 800204815 873709568 160616 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=32457/tid=32468] ppid=32455 vsize=853232 CPUtime=5012 cores=0-7
/proc/32457/task/32468/stat : 32468 (plingeling) R 32455 32457 32431 0 -1 4202560 927520 1246 0 5 501035 165 0 0 25 0 9 0 800204815 873709568 160616 33554432000 4194304 4417452 140735192165024 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=32457/tid=32469] ppid=32455 vsize=853232 CPUtime=5012.19 cores=0-7
/proc/32457/task/32469/stat : 32469 (plingeling) R 32455 32457 32431 0 -1 4202560 764096 1246 0 5 501080 139 0 0 25 0 9 0 800204815 873709568 160616 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=32457/tid=32470] ppid=32455 vsize=853232 CPUtime=4918.9 cores=0-7
/proc/32457/task/32470/stat : 32470 (plingeling) R 32455 32457 32431 0 -1 4202560 1018870 1246 0 5 491712 178 0 0 25 0 9 0 800204815 873709568 160616 33554432000 4194304 4417452 140735192165024 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=32457/tid=32471] ppid=32455 vsize=853232 CPUtime=5011.26 cores=0-7
/proc/32457/task/32471/stat : 32471 (plingeling) R 32455 32457 32431 0 -1 4202560 868736 1246 0 5 500977 149 0 0 25 0 9 0 800204816 873709568 160616 33554432000 4194304 4417452 140735192165024 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 853232

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5012.27
CPU time (s): 40001
CPU user time (s): 39988.4
CPU system time (s): 12.5821
CPU usage (%): 798.062
Max. virtual memory (cumulated for all children) (KiB): 947360

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39988.4
system time used= 12.5821
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 7210863
page faults= 5
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 474
involuntary context switches= 190430

runsolver used 26.208 second user time and 67.0988 second system time

The end

Launcher Data 

Begin job on node118 at 2011-05-22 23:29:01
IDJOB=3394562
IDBENCH=83247
IDSOLVER=1851
FILE ID=node118/3394562-1306099741
RUNJOBID= node118-1306099741-32445
PBS_JOBID= 13387979
Free space on /tmp= 72944 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/kullmann/GreenTao/GreenTaok1k2/GreenTao_2-4-4_512.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394562-1306099741/watcher-3394562-1306099741 -o /tmp/evaluation-result-3394562-1306099741/solver-3394562-1306099741 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394562-1306099741.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= c1f047ec179094963b21d7b90509d2b9
RANDOM SEED=932010774

node118.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5374.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25551576 kB
Buffers:          9976 kB
Cached:         145788 kB
SwapCached:      51948 kB
Active:         481356 kB
Inactive:      6655156 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25551576 kB
SwapTotal:    67111528 kB
SwapFree:     39849776 kB
Dirty:             412 kB
Writeback:           0 kB
AnonPages:     6941864 kB
Mapped:          24776 kB
Slab:            74928 kB
PageTables:     115384 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 183774252 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72944 MiB
End job on node118 at 2011-05-23 00:52:36