Trace number 3394468

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.8 5010.84

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-6x40.cnf
MD5SUM3a7a68cedc6ab84c0df9c27d166b0b6b
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables960
Number of clauses48480
Sum of the clauses size191040
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21440
Number of clauses of size 30
Number of clauses of size 447040
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394468-1306097224/watcher-3394468-1306097224 -o /tmp/evaluation-result-3394468-1306097224/solver-3394468-1306097224 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394468-1306097224.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.38 7.92 7.96 2/474 8091
/proc/meminfo: memFree=9850396/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=10812 CPUtime=0 cores=0-7
/proc/8091/stat : 8091 (plingeling) R 8089 8091 8065 0 -1 4202496 749 1251 0 0 0 0 0 0 25 0 1 0 721322069 11071488 679 33554432000 4194304 4417452 140737128022352 18446744073709551615 208079894886 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/8091/statm: 2703 691 147 55 0 589 0

[startup+0.0697909 s]
/proc/loadavg: 7.38 7.92 7.96 2/474 8091
/proc/meminfo: memFree=9850396/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=25940 CPUtime=0.05 cores=0-7
/proc/8091/stat : 8091 (plingeling) R 8089 8091 8065 0 -1 4202496 4844 1251 0 0 5 0 0 0 25 0 1 0 721322069 26562560 4509 33554432000 4194304 4417452 140737128022352 18446744073709551615 4227841 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/8091/statm: 6485 4509 148 55 0 4371 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 25940

[startup+0.100781 s]
/proc/loadavg: 7.38 7.92 7.96 2/474 8091
/proc/meminfo: memFree=9850396/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=34132 CPUtime=0.09 cores=0-7
/proc/8091/stat : 8091 (plingeling) R 8089 8091 8065 0 -1 4202496 6892 1251 0 0 8 1 0 0 25 0 1 0 721322069 34951168 6557 33554432000 4194304 4417452 140737128022352 18446744073709551615 4227769 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/8091/statm: 8533 6557 148 55 0 6419 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 34132

[startup+0.300748 s]
/proc/loadavg: 7.38 7.92 7.96 2/474 8091
/proc/meminfo: memFree=9850396/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=267300 CPUtime=1.59 cores=0-7
/proc/8091/stat : 8091 (plingeling) S 8089 8091 8065 0 -1 4202496 12370 1251 0 0 157 2 0 0 25 0 9 0 721322069 273715200 11682 33554432000 4194304 4417452 140737128022352 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/8091/statm: 66825 11682 180 55 0 64711 0
Current children cumulated CPU time (s) 1.59
Current children cumulated vsize (KiB) 267300

[startup+0.700669 s]
/proc/loadavg: 7.38 7.92 7.96 2/474 8091
/proc/meminfo: memFree=9850396/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=284588 CPUtime=4.79 cores=0-7
/proc/8091/stat : 8091 (plingeling) S 8089 8091 8065 0 -1 4202496 17700 1251 0 0 476 3 0 0 25 0 9 0 721322069 291418112 15999 33554432000 4194304 4417452 140737128022352 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/8091/statm: 71147 15999 180 55 0 69033 0
Current children cumulated CPU time (s) 4.79
Current children cumulated vsize (KiB) 284588

[startup+1.50052 s]
/proc/loadavg: 7.38 7.92 7.96 9/483 8105
/proc/meminfo: memFree=9779660/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=295924 CPUtime=11.17 cores=0-7
/proc/8091/stat : 8091 (plingeling) S 8089 8091 8065 0 -1 4202496 30474 1251 0 0 1111 6 0 0 25 0 9 0 721322069 303026176 18972 33554432000 4194304 4417452 140737128022352 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/8091/statm: 73981 18972 180 55 0 71867 0
[pid=8091/tid=8098] ppid=8089 vsize=295924 CPUtime=1.37 cores=0-7
/proc/8091/task/8098/stat : 8098 (plingeling) R 8089 8091 8065 0 -1 4202560 2213 1251 0 0 137 0 0 0 25 0 9 0 721322080 303026176 18972 33554432000 4194304 4417452 140737128022352 18446744073709551615 4357204 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8091/tid=8099] ppid=8089 vsize=295924 CPUtime=1.36 cores=0-7
/proc/8091/task/8099/stat : 8099 (plingeling) R 8089 8091 8065 0 -1 4202560 1772 1251 0 0 136 0 0 0 25 0 9 0 721322080 303026176 18972 33554432000 4194304 4417452 140737128022352 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8091/tid=8100] ppid=8089 vsize=295924 CPUtime=1.37 cores=0-7
/proc/8091/task/8100/stat : 8100 (plingeling) R 8089 8091 8065 0 -1 4202560 3666 1251 0 0 137 0 0 0 25 0 9 0 721322080 303026176 18972 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8091/tid=8101] ppid=8089 vsize=295924 CPUtime=1.38 cores=0-7
/proc/8091/task/8101/stat : 8101 (plingeling) R 8089 8091 8065 0 -1 4202560 5510 1251 0 0 137 1 0 0 20 0 9 0 721322080 303026176 18972 33554432000 4194304 4417452 140737128022352 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8091/tid=8102] ppid=8089 vsize=295924 CPUtime=1.38 cores=0-7
/proc/8091/task/8102/stat : 8102 (plingeling) R 8089 8091 8065 0 -1 4202560 1754 1251 0 0 138 0 0 0 25 0 9 0 721322080 303026176 18972 33554432000 4194304 4417452 140737128022352 18446744073709551615 4232648 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8091/tid=8103] ppid=8089 vsize=295924 CPUtime=1.38 cores=0-7
/proc/8091/task/8103/stat : 8103 (plingeling) R 8089 8091 8065 0 -1 4202560 2382 1251 0 0 138 0 0 0 25 0 9 0 721322080 303026176 18972 33554432000 4194304 4417452 140737128022352 18446744073709551615 4357263 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8091/tid=8104] ppid=8089 vsize=295924 CPUtime=1.37 cores=0-7
/proc/8091/task/8104/stat : 8104 (plingeling) R 8089 8091 8065 0 -1 4202560 4526 1251 0 0 137 0 0 0 25 0 9 0 721322080 303026176 18972 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8091/tid=8105] ppid=8089 vsize=295924 CPUtime=1.37 cores=0-7
/proc/8091/task/8105/stat : 8105 (plingeling) R 8089 8091 8065 0 -1 4202560 1742 1251 0 0 137 0 0 0 25 0 9 0 721322080 303026176 18972 33554432000 4194304 4417452 140737128022352 18446744073709551615 4251076 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.17
Current children cumulated vsize (KiB) 295924

[startup+3.10121 s]
/proc/loadavg: 7.38 7.92 7.96 9/483 8105
/proc/meminfo: memFree=9754736/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=326116 CPUtime=23.95 cores=0-7
/proc/8091/stat : 8091 (plingeling) S 8089 8091 8065 0 -1 4202496 67092 1251 0 0 2382 13 0 0 25 0 9 0 721322069 333942784 26373 33554432000 4194304 4417452 140737128022352 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/8091/statm: 81529 26373 180 55 0 79415 0
[pid=8091/tid=8098] ppid=8089 vsize=326116 CPUtime=2.98 cores=0-7
/proc/8091/task/8098/stat : 8098 (plingeling) R 8089 8091 8065 0 -1 4202560 6545 1251 0 0 297 1 0 0 25 0 9 0 721322080 333942784 26373 33554432000 4194304 4417452 140737128022352 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8091/tid=8099] ppid=8089 vsize=326116 CPUtime=2.94 cores=0-7
/proc/8091/task/8099/stat : 8099 (plingeling) R 8089 8091 8065 0 -1 4202560 4024 1251 0 0 294 0 0 0 25 0 9 0 721322080 333942784 26373 33554432000 4194304 4417452 140737128022352 18446744073709551615 4258267 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8091/tid=8100] ppid=8089 vsize=326116 CPUtime=2.97 cores=0-7
/proc/8091/task/8100/stat : 8100 (plingeling) R 8089 8091 8065 0 -1 4202560 8376 1251 0 0 296 1 0 0 25 0 9 0 721322080 333942784 26373 33554432000 4194304 4417452 140737128022352 18446744073709551615 4248242 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8091/tid=8101] ppid=8089 vsize=326116 CPUtime=2.98 cores=0-7
/proc/8091/task/8101/stat : 8101 (plingeling) R 8089 8091 8065 0 -1 4202560 11109 1251 0 0 296 2 0 0 25 0 9 0 721322080 333942784 26373 33554432000 4194304 4417452 140737128022352 18446744073709551615 4199406 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8091/tid=8102] ppid=8089 vsize=326116 CPUtime=2.98 cores=0-7
/proc/8091/task/8102/stat : 8102 (plingeling) R 8089 8091 8065 0 -1 4202560 6939 1251 0 0 297 1 0 0 25 0 9 0 721322080 333942784 26373 33554432000 4194304 4417452 140737128022352 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8091/tid=8103] ppid=8089 vsize=326116 CPUtime=2.98 cores=0-7
/proc/8091/task/8103/stat : 8103 (plingeling) R 8089 8091 8065 0 -1 4202560 8725 1251 0 0 297 1 0 0 25 0 9 0 721322080 333942784 26373 33554432000 4194304 4417452 140737128022352 18446744073709551615 4366639 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8091/tid=8104] ppid=8089 vsize=326116 CPUtime=2.97 cores=0-7
/proc/8091/task/8104/stat : 8104 (plingeling) R 8089 8091 8065 0 -1 4202560 8278 1251 0 0 296 1 0 0 25 0 9 0 721322080 333942784 26373 33554432000 4194304 4417452 140737128022352 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8091/tid=8105] ppid=8089 vsize=326116 CPUtime=2.97 cores=0-7
/proc/8091/task/8105/stat : 8105 (plingeling) R 8089 8091 8065 0 -1 4202560 6187 1251 0 0 296 1 0 0 25 0 9 0 721322080 333942784 26373 33554432000 4194304 4417452 140737128022352 18446744073709551615 4258491 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 23.95
Current children cumulated vsize (KiB) 326116

[startup+6.3006 s]
/proc/loadavg: 7.43 7.92 7.96 9/483 8105
/proc/meminfo: memFree=9680088/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=425236 CPUtime=49.49 cores=0-7
/proc/8091/stat : 8091 (plingeling) S 8089 8091 8065 0 -1 4202496 144800 1251 0 0 4921 28 0 0 25 0 9 0 721322069 435441664 51153 33554432000 4194304 4417452 140737128022352 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/8091/statm: 106309 51153 180 55 0 104195 0
[pid=8091/tid=8098] ppid=8089 vsize=425236 CPUtime=6.17 cores=0-7
/proc/8091/task/8098/stat : 8098 (plingeling) R 8089 8091 8065 0 -1 4202560 17957 1251 0 0 614 3 0 0 25 0 9 0 721322080 435441664 51153 33554432000 4194304 4417452 140737128022352 18446744073709551615 4250476 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8091/tid=8099] ppid=8089 vsize=425236 CPUtime=6.09 cores=0-7
/proc/8091/task/8099/stat : 8099 (plingeling) R 8089 8091 8065 0 -1 4202560 9111 1251 0 0 608 1 0 0 25 0 9 0 721322080 435441664 51153 33554432000 4194304 4417452 140737128022352 18446744073709551615 4354497 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8091/tid=8100] ppid=8089 vsize=425236 CPUtime=6.17 cores=0-7
/proc/8091/task/8100/stat : 8100 (plingeling) R 8089 8091 8065 0 -1 4202560 18862 1251 0 0 613 4 0 0 25 0 9 0 721322080 435441664 51153 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8091/tid=8101] ppid=8089 vsize=425236 CPUtime=6.17 cores=0-7
/proc/8091/task/8101/stat : 8101 (plingeling) R 8089 8091 8065 0 -1 4202560 16410 1251 0 0 614 3 0 0 25 0 9 0 721322080 435441664 51153 33554432000 4194304 4417452 140737128022352 18446744073709551615 4199406 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8091/tid=8102] ppid=8089 vsize=425236 CPUtime=6.18 cores=0-7
/proc/8091/task/8102/stat : 8102 (plingeling) R 8089 8091 8065 0 -1 4202560 19687 1251 0 0 615 3 0 0 25 0 9 0 721322080 435441664 51153 33554432000 4194304 4417452 140737128022352 18446744073709551615 4250488 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8091/tid=8103] ppid=8089 vsize=425236 CPUtime=6.17 cores=0-7
/proc/8091/task/8103/stat : 8103 (plingeling) R 8089 8091 8065 0 -1 4202560 19014 1251 0 0 615 2 0 0 25 0 9 0 721322080 435441664 51153 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8091/tid=8104] ppid=8089 vsize=425236 CPUtime=6.17 cores=0-7
/proc/8091/task/8104/stat : 8104 (plingeling) R 8089 8091 8065 0 -1 4202560 18851 1251 0 0 614 3 0 0 25 0 9 0 721322080 435441664 51153 33554432000 4194304 4417452 140737128022352 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8091/tid=8105] ppid=8089 vsize=425236 CPUtime=6.17 cores=0-7
/proc/8091/task/8105/stat : 8105 (plingeling) R 8089 8091 8065 0 -1 4202560 17999 1251 0 0 613 4 0 0 25 0 9 0 721322080 435441664 51153 33554432000 4194304 4417452 140737128022352 18446744073709551615 4251160 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 49.49
Current children cumulated vsize (KiB) 425236

[startup+12.7004 s]
/proc/loadavg: 7.47 7.92 7.96 9/483 8106
/proc/meminfo: memFree=9507256/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=606788 CPUtime=100.56 cores=0-7
/proc/8091/stat : 8091 (plingeling) S 8089 8091 8065 0 -1 4202496 255279 1251 0 0 10004 52 0 0 25 0 9 0 721322069 621350912 96541 33554432000 4194304 4417452 140737128022352 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/8091/statm: 151697 96541 180 55 0 149583 0
[pid=8091/tid=8098] ppid=8089 vsize=606788 CPUtime=12.56 cores=0-7
/proc/8091/task/8098/stat : 8098 (plingeling) R 8089 8091 8065 0 -1 4202560 38021 1251 0 0 1249 7 0 0 25 0 9 0 721322080 621350912 96541 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8091/tid=8099] ppid=8089 vsize=606788 CPUtime=12.38 cores=0-7
/proc/8091/task/8099/stat : 8099 (plingeling) R 8089 8091 8065 0 -1 4202560 12712 1251 0 0 1237 1 0 0 25 0 9 0 721322080 621350912 96541 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8091/tid=8100] ppid=8089 vsize=606788 CPUtime=12.56 cores=0-7
/proc/8091/task/8100/stat : 8100 (plingeling) R 8089 8091 8065 0 -1 4202560 34086 1251 0 0 1249 7 0 0 25 0 9 0 721322080 621350912 96541 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8091/tid=8101] ppid=8089 vsize=606788 CPUtime=12.57 cores=0-7
/proc/8091/task/8101/stat : 8101 (plingeling) R 8089 8091 8065 0 -1 4202560 23197 1251 0 0 1252 5 0 0 25 0 9 0 721322080 621350912 96541 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364780 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8091/tid=8102] ppid=8089 vsize=606788 CPUtime=12.57 cores=0-7
/proc/8091/task/8102/stat : 8102 (plingeling) R 8089 8091 8065 0 -1 4202560 34673 1251 0 0 1251 6 0 0 25 0 9 0 721322080 621350912 96541 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8091/tid=8103] ppid=8089 vsize=606788 CPUtime=12.58 cores=0-7
/proc/8091/task/8103/stat : 8103 (plingeling) R 8089 8091 8065 0 -1 4202560 37521 1251 0 0 1251 7 0 0 25 0 9 0 721322080 621350912 96541 33554432000 4194304 4417452 140737128022352 18446744073709551615 4354497 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8091/tid=8104] ppid=8089 vsize=606788 CPUtime=12.57 cores=0-7
/proc/8091/task/8104/stat : 8104 (plingeling) R 8089 8091 8065 0 -1 4202560 34428 1251 0 0 1251 6 0 0 25 0 9 0 721322080 621350912 96541 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8091/tid=8105] ppid=8089 vsize=606788 CPUtime=12.57 cores=0-7
/proc/8091/task/8105/stat : 8105 (plingeling) R 8089 8091 8065 0 -1 4202560 33732 1251 0 0 1250 7 0 0 25 0 9 0 721322080 621350912 96541 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 100.56
Current children cumulated vsize (KiB) 606788

[startup+25.5009 s]
/proc/loadavg: 7.59 7.92 7.96 9/483 8106
/proc/meminfo: memFree=9206660/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=867520 CPUtime=202.76 cores=0-7
/proc/8091/stat : 8091 (plingeling) S 8089 8091 8065 0 -1 4202496 450380 1251 0 0 20177 99 0 0 25 0 9 0 721322069 888340480 161724 33554432000 4194304 4417452 140737128022352 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/8091/statm: 216880 161724 180 55 0 214766 0
[pid=8091/tid=8098] ppid=8089 vsize=867520 CPUtime=25.36 cores=0-7
/proc/8091/task/8098/stat : 8098 (plingeling) R 8089 8091 8065 0 -1 4202560 65810 1251 0 0 2522 14 0 0 25 0 9 0 721322080 888340480 161724 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8091/tid=8099] ppid=8089 vsize=867520 CPUtime=24.98 cores=0-7
/proc/8091/task/8099/stat : 8099 (plingeling) R 8089 8091 8065 0 -1 4202560 21354 1251 0 0 2495 3 0 0 25 0 9 0 721322080 888340480 161724 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8091/tid=8100] ppid=8089 vsize=867520 CPUtime=25.37 cores=0-7
/proc/8091/task/8100/stat : 8100 (plingeling) R 8089 8091 8065 0 -1 4202560 63582 1251 0 0 2523 14 0 0 25 0 9 0 721322080 888340480 161724 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8091/tid=8101] ppid=8089 vsize=867520 CPUtime=25.38 cores=0-7
/proc/8091/task/8101/stat : 8101 (plingeling) R 8089 8091 8065 0 -1 4202560 38443 1251 0 0 2529 9 0 0 25 0 9 0 721322080 888340480 161724 33554432000 4194304 4417452 140737128022352 18446744073709551615 4365042 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8091/tid=8102] ppid=8089 vsize=867520 CPUtime=25.37 cores=0-7
/proc/8091/task/8102/stat : 8102 (plingeling) R 8089 8091 8065 0 -1 4202560 57390 1251 0 0 2525 12 0 0 25 0 9 0 721322080 888340480 161724 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8091/tid=8103] ppid=8089 vsize=867520 CPUtime=25.37 cores=0-7
/proc/8091/task/8103/stat : 8103 (plingeling) R 8089 8091 8065 0 -1 4202560 65787 1251 0 0 2524 13 0 0 25 0 9 0 721322080 888340480 161724 33554432000 4194304 4417452 140737128022352 18446744073709551615 4354524 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8091/tid=8104] ppid=8089 vsize=867520 CPUtime=25.37 cores=0-7
/proc/8091/task/8104/stat : 8104 (plingeling) R 8089 8091 8065 0 -1 4202560 66747 1251 0 0 2523 14 0 0 25 0 9 0 721322080 888340480 161724 33554432000 4194304 4417452 140737128022352 18446744073709551615 4263523 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8091/tid=8105] ppid=8089 vsize=867520 CPUtime=25.37 cores=0-7
/proc/8091/task/8105/stat : 8105 (plingeling) R 8089 8091 8065 0 -1 4202560 64358 1251 0 0 2522 15 0 0 25 0 9 0 721322080 888340480 161724 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 202.76
Current children cumulated vsize (KiB) 867520

[startup+51.1011 s]
/proc/loadavg: 7.85 7.96 7.97 9/483 8107
/proc/meminfo: memFree=8928464/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=1183920 CPUtime=407.14 cores=0-7
/proc/8091/stat : 8091 (plingeling) S 8089 8091 8065 0 -1 4202496 717547 1251 0 0 40558 156 0 0 25 0 9 0 721322069 1212334080 240824 33554432000 4194304 4417452 140737128022352 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/8091/statm: 295980 240824 180 55 0 293866 0
[pid=8091/tid=8098] ppid=8089 vsize=1183920 CPUtime=50.96 cores=0-7
/proc/8091/task/8098/stat : 8098 (plingeling) R 8089 8091 8065 0 -1 4202560 90128 1251 0 0 5077 19 0 0 25 0 9 0 721322080 1212334080 240824 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8091/tid=8099] ppid=8089 vsize=1183920 CPUtime=50.18 cores=0-7
/proc/8091/task/8099/stat : 8099 (plingeling) R 8089 8091 8065 0 -1 4202560 25450 1251 0 0 5015 3 0 0 25 0 9 0 721322080 1212334080 240824 33554432000 4194304 4417452 140737128022352 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8091/tid=8100] ppid=8089 vsize=1183920 CPUtime=50.97 cores=0-7

################
# More data... #
################

[pid=8091/tid=8100] ppid=8089 vsize=6675132 CPUtime=4722.2 cores=0-7
/proc/8091/task/8100/stat : 8100 (plingeling) R 8089 8091 8065 0 -1 4202560 2860106 1251 0 0 471663 557 0 0 25 0 9 0 721322080 6835335168 1613626 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8091/tid=8101] ppid=8089 vsize=6675132 CPUtime=4722.32 cores=0-7
/proc/8091/task/8101/stat : 8101 (plingeling) R 8089 8091 8065 0 -1 4202560 934627 1251 0 0 472008 224 0 0 25 0 9 0 721322080 6835335168 1613626 33554432000 4194304 4417452 140737128022352 18446744073709551615 4342461 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8091/tid=8102] ppid=8089 vsize=6675132 CPUtime=4721.7 cores=0-7
/proc/8091/task/8102/stat : 8102 (plingeling) R 8089 8091 8065 0 -1 4202560 3721072 1251 0 0 471362 808 0 0 25 0 9 0 721322080 6835335168 1613626 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8091/tid=8103] ppid=8089 vsize=6675132 CPUtime=4722.34 cores=0-7
/proc/8091/task/8103/stat : 8103 (plingeling) R 8089 8091 8065 0 -1 4202560 3312816 1251 0 0 471581 653 0 0 25 0 9 0 721322080 6835335168 1613626 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8091/tid=8104] ppid=8089 vsize=6675132 CPUtime=4722.22 cores=0-7
/proc/8091/task/8104/stat : 8104 (plingeling) R 8089 8091 8065 0 -1 4202560 2198928 1251 0 0 471864 358 0 0 25 0 9 0 721322080 6835335168 1613626 33554432000 4194304 4417452 140737128022352 18446744073709551615 4365075 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8091/tid=8105] ppid=8089 vsize=6675132 CPUtime=4722.26 cores=0-7
/proc/8091/task/8105/stat : 8105 (plingeling) R 8089 8091 8065 0 -1 4202560 3233486 1251 0 0 471568 658 0 0 25 0 9 0 721322080 6835335168 1613626 33554432000 4194304 4417452 140737128022352 18446744073709551615 4344176 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37699.6
Current children cumulated vsize (KiB) 6675132

[startup+4782.3 s]
/proc/loadavg: 8.12 8.13 8.04 9/483 8257
/proc/meminfo: memFree=3196364/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=6865300 CPUtime=38178.4 cores=0-7
/proc/8091/stat : 8091 (plingeling) S 8089 8091 8065 0 -1 4202496 21268021 1251 0 0 3813586 4252 0 0 25 0 9 0 721322069 7030067200 1661168 33554432000 4194304 4417452 140737128022352 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/8091/statm: 1716325 1661168 180 55 0 1714211 0
[pid=8091/tid=8098] ppid=8089 vsize=6865300 CPUtime=4780.09 cores=0-7
/proc/8091/task/8098/stat : 8098 (plingeling) R 8089 8091 8065 0 -1 4202560 2480606 1251 0 0 477510 499 0 0 25 0 9 0 721322080 7030067200 1661168 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8091/tid=8099] ppid=8089 vsize=6865300 CPUtime=4705.23 cores=0-7
/proc/8091/task/8099/stat : 8099 (plingeling) R 8089 8091 8065 0 -1 4202560 2208761 1251 0 0 470107 416 0 0 25 0 9 0 721322080 7030067200 1661168 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8091/tid=8100] ppid=8089 vsize=6865300 CPUtime=4782.18 cores=0-7
/proc/8091/task/8100/stat : 8100 (plingeling) R 8089 8091 8065 0 -1 4202560 2876490 1251 0 0 477659 559 0 0 25 0 9 0 721322080 7030067200 1661168 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8091/tid=8101] ppid=8089 vsize=6865300 CPUtime=4782.31 cores=0-7
/proc/8091/task/8101/stat : 8101 (plingeling) R 8089 8091 8065 0 -1 4202560 934627 1251 0 0 478007 224 0 0 25 0 9 0 721322080 7030067200 1661168 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8091/tid=8102] ppid=8089 vsize=6865300 CPUtime=4781.65 cores=0-7
/proc/8091/task/8102/stat : 8102 (plingeling) R 8089 8091 8065 0 -1 4202560 3813545 1251 0 0 477330 835 0 0 25 0 9 0 721322080 7030067200 1661168 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8091/tid=8103] ppid=8089 vsize=6865300 CPUtime=4782.3 cores=0-7
/proc/8091/task/8103/stat : 8103 (plingeling) R 8089 8091 8065 0 -1 4202560 3454522 1251 0 0 477541 689 0 0 25 0 9 0 721322080 7030067200 1661168 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8091/tid=8104] ppid=8089 vsize=6865300 CPUtime=4782.19 cores=0-7
/proc/8091/task/8104/stat : 8104 (plingeling) R 8089 8091 8065 0 -1 4202560 2215970 1251 0 0 477857 362 0 0 25 0 9 0 721322080 7030067200 1661168 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8091/tid=8105] ppid=8089 vsize=6865300 CPUtime=4782.25 cores=0-7
/proc/8091/task/8105/stat : 8105 (plingeling) R 8089 8091 8065 0 -1 4202560 3276591 1251 0 0 477561 664 0 0 25 0 9 0 721322080 7030067200 1661168 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38178.4
Current children cumulated vsize (KiB) 6865300

[startup+4842.3 s]
/proc/loadavg: 8.04 8.10 8.03 9/483 8258
/proc/meminfo: memFree=3159128/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=6902324 CPUtime=38657.4 cores=0-7
/proc/8091/stat : 8091 (plingeling) S 8089 8091 8065 0 -1 4202496 21277277 1251 0 0 3861486 4254 0 0 25 0 9 0 721322069 7067979776 1670424 33554432000 4194304 4417452 140737128022352 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/8091/statm: 1725581 1670424 180 55 0 1723467 0
[pid=8091/tid=8098] ppid=8089 vsize=6902324 CPUtime=4840.08 cores=0-7
/proc/8091/task/8098/stat : 8098 (plingeling) R 8089 8091 8065 0 -1 4202560 2480606 1251 0 0 483509 499 0 0 25 0 9 0 721322080 7067979776 1670424 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8091/tid=8099] ppid=8089 vsize=6902324 CPUtime=4764.26 cores=0-7
/proc/8091/task/8099/stat : 8099 (plingeling) R 8089 8091 8065 0 -1 4202560 2214097 1251 0 0 476009 417 0 0 25 0 9 0 721322080 7067979776 1670424 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8091/tid=8100] ppid=8089 vsize=6902324 CPUtime=4842.18 cores=0-7
/proc/8091/task/8100/stat : 8100 (plingeling) R 8089 8091 8065 0 -1 4202560 2876490 1251 0 0 483659 559 0 0 25 0 9 0 721322080 7067979776 1670424 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8091/tid=8101] ppid=8089 vsize=6902324 CPUtime=4842.31 cores=0-7
/proc/8091/task/8101/stat : 8101 (plingeling) R 8089 8091 8065 0 -1 4202560 934627 1251 0 0 484007 224 0 0 25 0 9 0 721322080 7067979776 1670424 33554432000 4194304 4417452 140737128022352 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8091/tid=8102] ppid=8089 vsize=6902324 CPUtime=4841.65 cores=0-7
/proc/8091/task/8102/stat : 8102 (plingeling) R 8089 8091 8065 0 -1 4202560 3817465 1251 0 0 483329 836 0 0 25 0 9 0 721322080 7067979776 1670424 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8091/tid=8103] ppid=8089 vsize=6902324 CPUtime=4842.3 cores=0-7
/proc/8091/task/8103/stat : 8103 (plingeling) R 8089 8091 8065 0 -1 4202560 3454522 1251 0 0 483541 689 0 0 25 0 9 0 721322080 7067979776 1670424 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8091/tid=8104] ppid=8089 vsize=6902324 CPUtime=4842.19 cores=0-7
/proc/8091/task/8104/stat : 8104 (plingeling) R 8089 8091 8065 0 -1 4202560 2215970 1251 0 0 483857 362 0 0 25 0 9 0 721322080 7067979776 1670424 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8091/tid=8105] ppid=8089 vsize=6902324 CPUtime=4842.25 cores=0-7
/proc/8091/task/8105/stat : 8105 (plingeling) R 8089 8091 8065 0 -1 4202560 3276591 1251 0 0 483561 664 0 0 25 0 9 0 721322080 7067979776 1670424 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38657.4
Current children cumulated vsize (KiB) 6902324

[startup+4902.3 s]
/proc/loadavg: 8.01 8.08 8.03 9/483 8260
/proc/meminfo: memFree=2927844/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=7133264 CPUtime=39136.5 cores=0-7
/proc/8091/stat : 8091 (plingeling) S 8089 8091 8065 0 -1 4202496 21335011 1251 0 0 3909381 4265 0 0 25 0 9 0 721322069 7304462336 1728158 33554432000 4194304 4417452 140737128022352 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/8091/statm: 1783316 1728158 180 55 0 1781202 0
[pid=8091/tid=8098] ppid=8089 vsize=7133264 CPUtime=4900.07 cores=0-7
/proc/8091/task/8098/stat : 8098 (plingeling) R 8089 8091 8065 0 -1 4202560 2480606 1251 0 0 489508 499 0 0 25 0 9 0 721322080 7304462336 1728158 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8091/tid=8099] ppid=8089 vsize=7133264 CPUtime=4823.31 cores=0-7
/proc/8091/task/8099/stat : 8099 (plingeling) R 8089 8091 8065 0 -1 4202560 2239117 1251 0 0 481912 419 0 0 25 0 9 0 721322080 7304462336 1728158 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8091/tid=8100] ppid=8089 vsize=7133264 CPUtime=4902.19 cores=0-7
/proc/8091/task/8100/stat : 8100 (plingeling) R 8089 8091 8065 0 -1 4202560 2886015 1251 0 0 489659 560 0 0 25 0 9 0 721322080 7304462336 1728158 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8091/tid=8101] ppid=8089 vsize=7133264 CPUtime=4902.31 cores=0-7
/proc/8091/task/8101/stat : 8101 (plingeling) R 8089 8091 8065 0 -1 4202560 934627 1251 0 0 490007 224 0 0 25 0 9 0 721322080 7304462336 1728158 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8091/tid=8102] ppid=8089 vsize=7133264 CPUtime=4901.65 cores=0-7
/proc/8091/task/8102/stat : 8102 (plingeling) R 8089 8091 8065 0 -1 4202560 3817465 1251 0 0 489329 836 0 0 25 0 9 0 721322080 7304462336 1728158 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364850 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8091/tid=8103] ppid=8089 vsize=7133264 CPUtime=4902.3 cores=0-7
/proc/8091/task/8103/stat : 8103 (plingeling) R 8089 8091 8065 0 -1 4202560 3456423 1251 0 0 489541 689 0 0 25 0 9 0 721322080 7304462336 1728158 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8091/tid=8104] ppid=8089 vsize=7133264 CPUtime=4902.2 cores=0-7
/proc/8091/task/8104/stat : 8104 (plingeling) R 8089 8091 8065 0 -1 4202560 2237258 1251 0 0 489851 369 0 0 25 0 9 0 721322080 7304462336 1728158 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8091/tid=8105] ppid=8089 vsize=7133264 CPUtime=4902.25 cores=0-7
/proc/8091/task/8105/stat : 8105 (plingeling) R 8089 8091 8065 0 -1 4202560 3276591 1251 0 0 489561 664 0 0 25 0 9 0 721322080 7304462336 1728158 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39136.5
Current children cumulated vsize (KiB) 7133264

[startup+4962.3 s]
/proc/loadavg: 8.00 8.06 8.02 9/483 8262
/proc/meminfo: memFree=1653952/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=8404728 CPUtime=39615.4 cores=0-7
/proc/8091/stat : 8091 (plingeling) S 8089 8091 8065 0 -1 4202496 21942773 1251 0 0 3957122 4418 0 0 25 0 9 0 721322069 8606441472 2046026 33554432000 4194304 4417452 140737128022352 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/8091/statm: 2101182 2046026 180 55 0 2099068 0
[pid=8091/tid=8098] ppid=8089 vsize=8404728 CPUtime=4960.05 cores=0-7
/proc/8091/task/8098/stat : 8098 (plingeling) R 8089 8091 8065 0 -1 4202560 2650067 1251 0 0 495455 550 0 0 25 0 9 0 721322080 8606441472 2046026 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8091/tid=8099] ppid=8089 vsize=8404728 CPUtime=4882.34 cores=0-7
/proc/8091/task/8099/stat : 8099 (plingeling) R 8089 8091 8065 0 -1 4202560 2262788 1251 0 0 487809 425 0 0 25 0 9 0 721322080 8606441472 2046026 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8091/tid=8100] ppid=8089 vsize=8404728 CPUtime=4962.11 cores=0-7
/proc/8091/task/8100/stat : 8100 (plingeling) R 8089 8091 8065 0 -1 4202560 3040538 1251 0 0 495625 586 0 0 25 0 9 0 721322080 8606441472 2046026 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8091/tid=8101] ppid=8089 vsize=8404728 CPUtime=4962.31 cores=0-7
/proc/8091/task/8101/stat : 8101 (plingeling) R 8089 8091 8065 0 -1 4202560 980735 1251 0 0 495994 237 0 0 25 0 9 0 721322080 8606441472 2046026 33554432000 4194304 4417452 140737128022352 18446744073709551615 4365004 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8091/tid=8102] ppid=8089 vsize=8404728 CPUtime=4961.65 cores=0-7
/proc/8091/task/8102/stat : 8102 (plingeling) R 8089 8091 8065 0 -1 4202560 3817465 1251 0 0 495329 836 0 0 25 0 9 0 721322080 8606441472 2046026 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8091/tid=8103] ppid=8089 vsize=8404728 CPUtime=4962.3 cores=0-7
/proc/8091/task/8103/stat : 8103 (plingeling) R 8089 8091 8065 0 -1 4202560 3642339 1251 0 0 495491 739 0 0 25 0 9 0 721322080 8606441472 2046026 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8091/tid=8104] ppid=8089 vsize=8404728 CPUtime=4962.2 cores=0-7
/proc/8091/task/8104/stat : 8104 (plingeling) R 8089 8091 8065 0 -1 4202560 2237258 1251 0 0 495851 369 0 0 25 0 9 0 721322080 8606441472 2046026 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8091/tid=8105] ppid=8089 vsize=8404728 CPUtime=4962.26 cores=0-7
/proc/8091/task/8105/stat : 8105 (plingeling) R 8089 8091 8065 0 -1 4202560 3304674 1251 0 0 495555 671 0 0 25 0 9 0 721322080 8606441472 2046026 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39615.4
Current children cumulated vsize (KiB) 8404728



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.52 s]
/proc/loadavg: 8.04 8.06 8.02 9/483 8263
/proc/meminfo: memFree=2011136/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=8048620 CPUtime=40000.4 cores=0-7
/proc/8091/stat : 8091 (plingeling) S 8089 8091 8065 0 -1 4202496 21994215 1251 0 0 3995616 4426 0 0 25 0 9 0 721322069 8241786880 1956999 33554432000 4194304 4417452 140737128022352 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/8091/statm: 2012155 1956999 180 55 0 2010041 0
[pid=8091/tid=8098] ppid=8089 vsize=8048620 CPUtime=5008.27 cores=0-7
/proc/8091/task/8098/stat : 8098 (plingeling) R 8089 8091 8065 0 -1 4202560 2650067 1251 0 0 500277 550 0 0 25 0 9 0 721322080 8241786880 1956999 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8091/tid=8099] ppid=8089 vsize=8048620 CPUtime=4929.76 cores=0-7
/proc/8091/task/8099/stat : 8099 (plingeling) R 8089 8091 8065 0 -1 4202560 2272843 1251 0 0 492549 427 0 0 25 0 9 0 721322080 8241786880 1956999 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8091/tid=8100] ppid=8089 vsize=8048620 CPUtime=5010.35 cores=0-7
/proc/8091/task/8100/stat : 8100 (plingeling) R 8089 8091 8065 0 -1 4202560 3040538 1251 0 0 500449 586 0 0 25 0 9 0 721322080 8241786880 1956999 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8091/tid=8101] ppid=8089 vsize=8048620 CPUtime=5010.55 cores=0-7
/proc/8091/task/8101/stat : 8101 (plingeling) R 8089 8091 8065 0 -1 4202560 980735 1251 0 0 500818 237 0 0 25 0 9 0 721322080 8241786880 1956999 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8091/tid=8102] ppid=8089 vsize=8048620 CPUtime=5009.89 cores=0-7
/proc/8091/task/8102/stat : 8102 (plingeling) R 8089 8091 8065 0 -1 4202560 3817465 1251 0 0 500153 836 0 0 25 0 9 0 721322080 8241786880 1956999 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8091/tid=8103] ppid=8089 vsize=8048620 CPUtime=5010.54 cores=0-7
/proc/8091/task/8103/stat : 8103 (plingeling) R 8089 8091 8065 0 -1 4202560 3642339 1251 0 0 500315 739 0 0 25 0 9 0 721322080 8241786880 1956999 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8091/tid=8104] ppid=8089 vsize=8048620 CPUtime=5010.44 cores=0-7
/proc/8091/task/8104/stat : 8104 (plingeling) R 8089 8091 8065 0 -1 4202560 2264595 1251 0 0 500671 373 0 0 25 0 9 0 721322080 8241786880 1956999 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8091/tid=8105] ppid=8089 vsize=8048620 CPUtime=5010.5 cores=0-7
/proc/8091/task/8105/stat : 8105 (plingeling) R 8089 8091 8065 0 -1 4202560 3318724 1251 0 0 500376 674 0 0 25 0 9 0 721322080 8241786880 1956999 33554432000 4194304 4417452 140737128022352 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 8048620

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.62 s]
/proc/loadavg: 8.04 8.06 8.02 9/483 8263
/proc/meminfo: memFree=2011136/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=0 CPUtime=40000.6 cores=0-7
/proc/8091/stat : 8091 (plingeling) Z 8089 8091 8065 0 -1 4203548 21994215 1251 0 0 3995625 4435 0 0 18 0 2 0 721322069 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 7 0 0 0
/proc/8091/statm: 0 0 0 0 0 0 0
[pid=8091/tid=8099] ppid=8089 vsize=0 CPUtime=4929.85 cores=0-7
/proc/8091/task/8099/stat : 8099 (plingeling) R 8089 8091 8065 0 -1 4203588 2272843 1251 0 0 492550 435 0 0 25 0 2 0 721322080 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 0

[startup+5010.8 s]
/proc/loadavg: 8.04 8.06 8.02 9/483 8263
/proc/meminfo: memFree=2011136/32950928 swapFree=54423884/67111528
[pid=8091] ppid=8089 vsize=0 CPUtime=40000.8 cores=0-7
/proc/8091/stat : 8091 (plingeling) Z 8089 8091 8065 0 -1 4203548 21994215 1251 0 0 3995625 4452 0 0 18 0 2 0 721322069 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 7 0 0 0
/proc/8091/statm: 0 0 0 0 0 0 0
[pid=8091/tid=8099] ppid=8089 vsize=0 CPUtime=4930.03 cores=0-7
/proc/8091/task/8099/stat : 8099 (plingeling) R 8089 8091 8065 0 -1 4203588 2272843 1251 0 0 492550 453 0 0 25 0 2 0 721322080 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5010.84
CPU time (s): 40000.8
CPU user time (s): 39956.3
CPU system time (s): 44.5622
CPU usage (%): 798.286
Max. virtual memory (cumulated for all children) (KiB): 8404728

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39956.3
system time used= 44.5622
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 21995466
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 440
involuntary context switches= 193127

runsolver used 22.0097 second user time and 58.7941 second system time

The end

Launcher Data 

Begin job on node127 at 2011-05-22 22:47:04
IDJOB=3394468
IDBENCH=83178
IDSOLVER=1851
FILE ID=node127/3394468-1306097224
RUNJOBID= node127-1306097224-8079
PBS_JOBID= 13387975
Free space on /tmp= 73344 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-6x40.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394468-1306097224/watcher-3394468-1306097224 -o /tmp/evaluation-result-3394468-1306097224/solver-3394468-1306097224 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394468-1306097224.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 3a7a68cedc6ab84c0df9c27d166b0b6b
RANDOM SEED=1901376321

node127.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.60
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:       9850676 kB
Buffers:        217976 kB
Cached:        3835808 kB
SwapCached:      22608 kB
Active:        6425012 kB
Inactive:     16405056 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:       9850676 kB
SwapTotal:    67111528 kB
SwapFree:     54423884 kB
Dirty:            1200 kB
Writeback:           0 kB
AnonPages:    18760808 kB
Mapped:          28464 kB
Slab:           100072 kB
PageTables:      94740 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 119721972 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73340 MiB
End job on node127 at 2011-05-23 00:10:37