Trace number 3394086

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40001.1 5010.51

General information on the benchmark

NameCRAFTED/rbsat/random/
forced/rbsat-v2640c305320gyes3.cnf
MD5SUMd6877629cc818d7e31dc245fb95f0588
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2640
Number of clauses305320
Sum of the clauses size613120
Maximum clause length33
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2305240
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 580

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394086-1306087259/watcher-3394086-1306087259 -o /tmp/evaluation-result-3394086-1306087259/solver-3394086-1306087259 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394086-1306087259.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.39 7.93 7.96 2/462 4788
/proc/meminfo: memFree=12613856/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=9268 CPUtime=0 cores=0-7
/proc/4788/stat : 4788 (plingeling) R 4786 4788 4762 0 -1 4202496 385 1253 0 0 0 0 0 0 21 0 1 0 798956387 9490432 315 33554432000 4194304 4417452 140734906472512 18446744073709551615 252458065664 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/4788/statm: 2317 315 147 55 0 203 0

[startup+0.071623 s]
/proc/loadavg: 7.39 7.93 7.96 2/462 4788
/proc/meminfo: memFree=12613856/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=19612 CPUtime=0.06 cores=0-7
/proc/4788/stat : 4788 (plingeling) R 4786 4788 4762 0 -1 4202496 2996 1253 0 0 6 0 0 0 21 0 1 0 798956387 20082688 2926 33554432000 4194304 4417452 140734906472512 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/4788/statm: 4903 2926 148 55 0 2789 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 19612

[startup+0.100595 s]
/proc/loadavg: 7.39 7.93 7.96 2/462 4788
/proc/meminfo: memFree=12613856/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=27804 CPUtime=0.08 cores=0-7
/proc/4788/stat : 4788 (plingeling) R 4786 4788 4762 0 -1 4202496 5044 1253 0 0 8 0 0 0 21 0 1 0 798956387 28471296 4974 33554432000 4194304 4417452 140734906472512 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/4788/statm: 6951 4974 148 55 0 4837 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 27804

[startup+0.300559 s]
/proc/loadavg: 7.39 7.93 7.96 2/462 4788
/proc/meminfo: memFree=12613856/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=76956 CPUtime=0.29 cores=0-7
/proc/4788/stat : 4788 (plingeling) R 4786 4788 4762 0 -1 4202496 17332 1253 0 0 28 1 0 0 22 0 1 0 798956387 78802944 17262 33554432000 4194304 4417452 140734906472512 18446744073709551615 4227854 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/4788/statm: 19239 17262 148 55 0 17125 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 76956

[startup+0.700448 s]
/proc/loadavg: 7.39 7.93 7.96 2/462 4788
/proc/meminfo: memFree=12613856/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=254028 CPUtime=2.78 cores=0-7
/proc/4788/stat : 4788 (plingeling) S 4786 4788 4762 0 -1 4202496 32653 1253 0 0 275 3 0 0 18 0 9 0 798956387 260124672 25733 33554432000 4194304 4417452 140734906472512 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4788/statm: 63507 25733 177 55 0 61393 0
Current children cumulated CPU time (s) 2.78
Current children cumulated vsize (KiB) 254028

[startup+1.5013 s]
/proc/loadavg: 7.39 7.93 7.96 9/471 4802
/proc/meminfo: memFree=12507880/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=265584 CPUtime=9.17 cores=0-7
/proc/4788/stat : 4788 (plingeling) S 4786 4788 4762 0 -1 4202496 41983 1253 0 0 912 5 0 0 18 0 9 0 798956387 271958016 27633 33554432000 4194304 4417452 140734906472512 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4788/statm: 66396 27633 180 55 0 64282 0
[pid=4788/tid=4795] ppid=4786 vsize=265584 CPUtime=1.1 cores=0-7
/proc/4788/task/4795/stat : 4795 (plingeling) R 4786 4788 4762 0 -1 4202560 3093 1253 0 0 110 0 0 0 19 0 9 0 798956426 271958016 27633 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249897 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4788/tid=4796] ppid=4786 vsize=265584 CPUtime=1.05 cores=0-7
/proc/4788/task/4796/stat : 4796 (plingeling) R 4786 4788 4762 0 -1 4202560 2834 1253 0 0 105 0 0 0 18 0 9 0 798956426 271958016 27633 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4788/tid=4797] ppid=4786 vsize=265584 CPUtime=1.08 cores=0-7
/proc/4788/task/4797/stat : 4797 (plingeling) R 4786 4788 4762 0 -1 4202560 1738 1253 0 0 108 0 0 0 25 0 9 0 798956426 271958016 27633 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4788/tid=4798] ppid=4786 vsize=265584 CPUtime=1.1 cores=0-7
/proc/4788/task/4798/stat : 4798 (plingeling) R 4786 4788 4762 0 -1 4202560 4471 1253 0 0 110 0 0 0 19 0 9 0 798956426 271958016 27633 33554432000 4194304 4417452 140734906472512 18446744073709551615 18446744073699065863 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4788/tid=4799] ppid=4786 vsize=265584 CPUtime=1.09 cores=0-7
/proc/4788/task/4799/stat : 4799 (plingeling) R 4786 4788 4762 0 -1 4202560 3114 1253 0 0 109 0 0 0 19 0 9 0 798956426 271958016 27633 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4788/tid=4800] ppid=4786 vsize=265584 CPUtime=1.1 cores=0-7
/proc/4788/task/4800/stat : 4800 (plingeling) R 4786 4788 4762 0 -1 4202560 3420 1253 0 0 110 0 0 0 22 0 9 0 798956426 271958016 27633 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4788/tid=4801] ppid=4786 vsize=265584 CPUtime=1.1 cores=0-7
/proc/4788/task/4801/stat : 4801 (plingeling) R 4786 4788 4762 0 -1 4202560 3082 1253 0 0 110 0 0 0 22 0 9 0 798956426 271958016 27633 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4788/tid=4802] ppid=4786 vsize=265584 CPUtime=1.09 cores=0-7
/proc/4788/task/4802/stat : 4802 (plingeling) R 4786 4788 4762 0 -1 4202560 2882 1253 0 0 109 0 0 0 21 0 9 0 798956427 271958016 27633 33554432000 4194304 4417452 140734906472512 18446744073709551615 4239395 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 9.17
Current children cumulated vsize (KiB) 265584

[startup+3.101 s]
/proc/loadavg: 7.39 7.93 7.96 9/472 4803
/proc/meminfo: memFree=12494968/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=289868 CPUtime=21.92 cores=0-7
/proc/4788/stat : 4788 (plingeling) S 4786 4788 4762 0 -1 4202496 52721 1253 0 0 2186 6 0 0 18 0 9 0 798956387 296824832 33704 33554432000 4194304 4417452 140734906472512 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4788/statm: 72467 33704 180 55 0 70353 0
[pid=4788/tid=4795] ppid=4786 vsize=289868 CPUtime=2.7 cores=0-7
/proc/4788/task/4795/stat : 4795 (plingeling) R 4786 4788 4762 0 -1 4202560 4454 1253 0 0 270 0 0 0 23 0 9 0 798956426 296824832 33704 33554432000 4194304 4417452 140734906472512 18446744073709551615 4235378 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4788/tid=4796] ppid=4786 vsize=289868 CPUtime=2.64 cores=0-7
/proc/4788/task/4796/stat : 4796 (plingeling) R 4786 4788 4762 0 -1 4202560 2834 1253 0 0 264 0 0 0 22 0 9 0 798956426 296824832 33704 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4788/tid=4797] ppid=4786 vsize=289868 CPUtime=2.68 cores=0-7
/proc/4788/task/4797/stat : 4797 (plingeling) R 4786 4788 4762 0 -1 4202560 1738 1253 0 0 268 0 0 0 25 0 9 0 798956426 296824832 33704 33554432000 4194304 4417452 140734906472512 18446744073709551615 4258436 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4788/tid=4798] ppid=4786 vsize=289868 CPUtime=2.68 cores=0-7
/proc/4788/task/4798/stat : 4798 (plingeling) R 4786 4788 4762 0 -1 4202560 9765 1253 0 0 267 1 0 0 22 0 9 0 798956426 296824832 33704 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4788/tid=4799] ppid=4786 vsize=289868 CPUtime=2.69 cores=0-7
/proc/4788/task/4799/stat : 4799 (plingeling) R 4786 4788 4762 0 -1 4202560 4475 1253 0 0 269 0 0 0 22 0 9 0 798956426 296824832 33704 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249186 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4788/tid=4800] ppid=4786 vsize=289868 CPUtime=2.7 cores=0-7
/proc/4788/task/4800/stat : 4800 (plingeling) R 4786 4788 4762 0 -1 4202560 4781 1253 0 0 270 0 0 0 25 0 9 0 798956426 296824832 33704 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249143 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4788/tid=4801] ppid=4786 vsize=289868 CPUtime=2.7 cores=0-7
/proc/4788/task/4801/stat : 4801 (plingeling) R 4786 4788 4762 0 -1 4202560 3082 1253 0 0 270 0 0 0 25 0 9 0 798956426 296824832 33704 33554432000 4194304 4417452 140734906472512 18446744073709551615 4378066 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4788/tid=4802] ppid=4786 vsize=289868 CPUtime=2.69 cores=0-7
/proc/4788/task/4802/stat : 4802 (plingeling) R 4786 4788 4762 0 -1 4202560 4243 1253 0 0 269 0 0 0 25 0 9 0 798956427 296824832 33704 33554432000 4194304 4417452 140734906472512 18446744073709551615 4258374 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 21.92
Current children cumulated vsize (KiB) 289868

[startup+6.30041 s]
/proc/loadavg: 7.44 7.93 7.96 9/471 4811
/proc/meminfo: memFree=12470920/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=290996 CPUtime=47.46 cores=0-7
/proc/4788/stat : 4788 (plingeling) S 4786 4788 4762 0 -1 4202496 60190 1253 0 0 4739 7 0 0 18 0 9 0 798956387 297979904 33986 33554432000 4194304 4417452 140734906472512 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4788/statm: 72749 33986 180 55 0 70635 0
[pid=4788/tid=4795] ppid=4786 vsize=290996 CPUtime=5.9 cores=0-7
/proc/4788/task/4795/stat : 4795 (plingeling) R 4786 4788 4762 0 -1 4202560 4454 1253 0 0 590 0 0 0 25 0 9 0 798956426 297979904 33986 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4788/tid=4796] ppid=4786 vsize=290996 CPUtime=5.79 cores=0-7
/proc/4788/task/4796/stat : 4796 (plingeling) R 4786 4788 4762 0 -1 4202560 2834 1253 0 0 579 0 0 0 25 0 9 0 798956426 297979904 33986 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4788/tid=4797] ppid=4786 vsize=290996 CPUtime=5.88 cores=0-7
/proc/4788/task/4797/stat : 4797 (plingeling) R 4786 4788 4762 0 -1 4202560 1738 1253 0 0 588 0 0 0 25 0 9 0 798956426 297979904 33986 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4788/tid=4798] ppid=4786 vsize=290996 CPUtime=5.88 cores=0-7
/proc/4788/task/4798/stat : 4798 (plingeling) R 4786 4788 4762 0 -1 4202560 17234 1253 0 0 586 2 0 0 25 0 9 0 798956426 297979904 33986 33554432000 4194304 4417452 140734906472512 18446744073709551615 4356884 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4788/tid=4799] ppid=4786 vsize=290996 CPUtime=5.89 cores=0-7
/proc/4788/task/4799/stat : 4799 (plingeling) R 4786 4788 4762 0 -1 4202560 4475 1253 0 0 589 0 0 0 25 0 9 0 798956426 297979904 33986 33554432000 4194304 4417452 140734906472512 18446744073709551615 4258410 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4788/tid=4800] ppid=4786 vsize=290996 CPUtime=5.9 cores=0-7
/proc/4788/task/4800/stat : 4800 (plingeling) R 4786 4788 4762 0 -1 4202560 4781 1253 0 0 590 0 0 0 25 0 9 0 798956426 297979904 33986 33554432000 4194304 4417452 140734906472512 18446744073709551615 4236355 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4788/tid=4801] ppid=4786 vsize=290996 CPUtime=5.9 cores=0-7
/proc/4788/task/4801/stat : 4801 (plingeling) R 4786 4788 4762 0 -1 4202560 3082 1253 0 0 590 0 0 0 25 0 9 0 798956426 297979904 33986 33554432000 4194304 4417452 140734906472512 18446744073709551615 4261313 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4788/tid=4802] ppid=4786 vsize=290996 CPUtime=5.89 cores=0-7
/proc/4788/task/4802/stat : 4802 (plingeling) R 4786 4788 4762 0 -1 4202560 4243 1253 0 0 589 0 0 0 25 0 9 0 798956427 297979904 33986 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 47.46
Current children cumulated vsize (KiB) 290996

[startup+12.7012 s]
/proc/loadavg: 7.48 7.93 7.96 9/471 4811
/proc/meminfo: memFree=12409968/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=326696 CPUtime=98.57 cores=0-7
/proc/4788/stat : 4788 (plingeling) S 4786 4788 4762 0 -1 4202496 91914 1253 0 0 9844 13 0 0 18 0 9 0 798956387 334536704 42911 33554432000 4194304 4417452 140734906472512 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4788/statm: 81674 42911 180 55 0 79560 0
[pid=4788/tid=4795] ppid=4786 vsize=326696 CPUtime=12.3 cores=0-7
/proc/4788/task/4795/stat : 4795 (plingeling) R 4786 4788 4762 0 -1 4202560 5814 1253 0 0 1230 0 0 0 25 0 9 0 798956426 334536704 42911 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4788/tid=4796] ppid=4786 vsize=326696 CPUtime=12.08 cores=0-7
/proc/4788/task/4796/stat : 4796 (plingeling) R 4786 4788 4762 0 -1 4202560 5555 1253 0 0 1208 0 0 0 25 0 9 0 798956426 334536704 42911 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249158 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4788/tid=4797] ppid=4786 vsize=326696 CPUtime=12.28 cores=0-7
/proc/4788/task/4797/stat : 4797 (plingeling) R 4786 4788 4762 0 -1 4202560 1738 1253 0 0 1228 0 0 0 25 0 9 0 798956426 334536704 42911 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4788/tid=4798] ppid=4786 vsize=326696 CPUtime=12.28 cores=0-7
/proc/4788/task/4798/stat : 4798 (plingeling) R 4786 4788 4762 0 -1 4202560 40797 1253 0 0 1222 6 0 0 25 0 9 0 798956426 334536704 42911 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4788/tid=4799] ppid=4786 vsize=326696 CPUtime=12.3 cores=0-7
/proc/4788/task/4799/stat : 4799 (plingeling) R 4786 4788 4762 0 -1 4202560 5835 1253 0 0 1229 1 0 0 25 0 9 0 798956426 334536704 42911 33554432000 4194304 4417452 140734906472512 18446744073709551615 4250017 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4788/tid=4800] ppid=4786 vsize=326696 CPUtime=12.29 cores=0-7
/proc/4788/task/4800/stat : 4800 (plingeling) R 4786 4788 4762 0 -1 4202560 6141 1253 0 0 1229 0 0 0 25 0 9 0 798956426 334536704 42911 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4788/tid=4801] ppid=4786 vsize=326696 CPUtime=12.29 cores=0-7
/proc/4788/task/4801/stat : 4801 (plingeling) R 4786 4788 4762 0 -1 4202560 3082 1253 0 0 1229 0 0 0 25 0 9 0 798956426 334536704 42911 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4788/tid=4802] ppid=4786 vsize=326696 CPUtime=12.3 cores=0-7
/proc/4788/task/4802/stat : 4802 (plingeling) R 4786 4788 4762 0 -1 4202560 5603 1253 0 0 1229 1 0 0 25 0 9 0 798956427 334536704 42911 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249143 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 98.57
Current children cumulated vsize (KiB) 326696

[startup+25.5008 s]
/proc/loadavg: 7.76 7.96 7.97 9/471 4811
/proc/meminfo: memFree=12380944/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=372688 CPUtime=200.75 cores=0-7
/proc/4788/stat : 4788 (plingeling) S 4786 4788 4762 0 -1 4202496 138050 1253 0 0 20054 21 0 0 18 0 9 0 798956387 381632512 54413 33554432000 4194304 4417452 140734906472512 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4788/statm: 93172 54413 180 55 0 91058 0
[pid=4788/tid=4795] ppid=4786 vsize=372688 CPUtime=25.1 cores=0-7
/proc/4788/task/4795/stat : 4795 (plingeling) R 4786 4788 4762 0 -1 4202560 6886 1253 0 0 2509 1 0 0 25 0 9 0 798956426 381632512 54413 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4788/tid=4796] ppid=4786 vsize=372688 CPUtime=24.68 cores=0-7
/proc/4788/task/4796/stat : 4796 (plingeling) R 4786 4788 4762 0 -1 4202560 5748 1253 0 0 2468 0 0 0 25 0 9 0 798956426 381632512 54413 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4788/tid=4797] ppid=4786 vsize=372688 CPUtime=25.08 cores=0-7
/proc/4788/task/4797/stat : 4797 (plingeling) R 4786 4788 4762 0 -1 4202560 3114 1253 0 0 2508 0 0 0 25 0 9 0 798956426 381632512 54413 33554432000 4194304 4417452 140734906472512 18446744073709551615 4258191 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4788/tid=4798] ppid=4786 vsize=372688 CPUtime=25.08 cores=0-7
/proc/4788/task/4798/stat : 4798 (plingeling) R 4786 4788 4762 0 -1 4202560 82404 1253 0 0 2495 13 0 0 25 0 9 0 798956426 381632512 54413 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364542 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4788/tid=4799] ppid=4786 vsize=372688 CPUtime=25.09 cores=0-7
/proc/4788/task/4799/stat : 4799 (plingeling) R 4786 4788 4762 0 -1 4202560 6859 1253 0 0 2508 1 0 0 25 0 9 0 798956426 381632512 54413 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4788/tid=4800] ppid=4786 vsize=372688 CPUtime=25.09 cores=0-7
/proc/4788/task/4800/stat : 4800 (plingeling) R 4786 4788 4762 0 -1 4202560 6141 1253 0 0 2509 0 0 0 25 0 9 0 798956426 381632512 54413 33554432000 4194304 4417452 140734906472512 18446744073709551615 4237326 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4788/tid=4801] ppid=4786 vsize=372688 CPUtime=25.08 cores=0-7
/proc/4788/task/4801/stat : 4801 (plingeling) R 4786 4788 4762 0 -1 4202560 3946 1253 0 0 2508 0 0 0 25 0 9 0 798956426 381632512 54413 33554432000 4194304 4417452 140734906472512 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4788/tid=4802] ppid=4786 vsize=372688 CPUtime=25.09 cores=0-7
/proc/4788/task/4802/stat : 4802 (plingeling) R 4786 4788 4762 0 -1 4202560 5603 1253 0 0 2508 1 0 0 25 0 9 0 798956427 381632512 54413 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 200.75
Current children cumulated vsize (KiB) 372688

[startup+51.1011 s]
/proc/loadavg: 7.84 7.97 7.97 9/471 4812
/proc/meminfo: memFree=12246660/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=525608 CPUtime=405.15 cores=0-7
/proc/4788/stat : 4788 (plingeling) S 4786 4788 4762 0 -1 4202496 214050 1253 0 0 40482 33 0 0 18 0 9 0 798956387 538222592 92644 33554432000 4194304 4417452 140734906472512 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4788/statm: 131402 92644 180 55 0 129288 0
[pid=4788/tid=4795] ppid=4786 vsize=525608 CPUtime=50.7 cores=0-7
/proc/4788/task/4795/stat : 4795 (plingeling) R 4786 4788 4762 0 -1 4202560 8842 1253 0 0 5069 1 0 0 25 0 9 0 798956426 538222592 92644 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4788/tid=4796] ppid=4786 vsize=525608 CPUtime=49.9 cores=0-7
/proc/4788/task/4796/stat : 4796 (plingeling) R 4786 4788 4762 0 -1 4202560 7927 1253 0 0 4989 1 0 0 25 0 9 0 798956426 538222592 92644 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4788/tid=4797] ppid=4786 vsize=525608 CPUtime=50.67 cores=0-7

################
# More data... #
################


[startup+4722.32 s]
/proc/loadavg: 8.06 8.05 8.00 9/471 4948
/proc/meminfo: memFree=7811188/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=4950744 CPUtime=37701.4 cores=0-7
/proc/4788/stat : 4788 (plingeling) S 4786 4788 4762 0 -1 4202496 6891758 1253 0 0 3768998 1140 0 0 18 0 9 0 798956387 5069561856 1198958 33554432000 4194304 4417452 140734906472512 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4788/statm: 1237686 1198958 180 55 0 1235572 0
[pid=4788/tid=4795] ppid=4786 vsize=4950744 CPUtime=4721.71 cores=0-7
/proc/4788/task/4795/stat : 4795 (plingeling) R 4786 4788 4762 0 -1 4202560 520347 1253 0 0 472089 82 0 0 25 0 9 0 798956426 5069561856 1198958 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4788/tid=4796] ppid=4786 vsize=4950744 CPUtime=4674.91 cores=0-7
/proc/4788/task/4796/stat : 4796 (plingeling) R 4786 4788 4762 0 -1 4202560 718381 1253 0 0 467380 111 0 0 25 0 9 0 798956426 5069561856 1198958 33554432000 4194304 4417452 140734906472512 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4788/tid=4797] ppid=4786 vsize=4950744 CPUtime=4695.95 cores=0-7
/proc/4788/task/4797/stat : 4797 (plingeling) R 4786 4788 4762 0 -1 4202560 650443 1253 0 0 469498 97 0 0 25 0 9 0 798956426 5069561856 1198958 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4788/tid=4798] ppid=4786 vsize=4950744 CPUtime=4722.03 cores=0-7
/proc/4788/task/4798/stat : 4798 (plingeling) R 4786 4788 4762 0 -1 4202560 2501722 1253 0 0 471694 509 0 0 25 0 9 0 798956426 5069561856 1198958 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4788/tid=4799] ppid=4786 vsize=4950744 CPUtime=4722.03 cores=0-7
/proc/4788/task/4799/stat : 4799 (plingeling) R 4786 4788 4762 0 -1 4202560 1010866 1253 0 0 472067 136 0 0 25 0 9 0 798956426 5069561856 1198958 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4788/tid=4800] ppid=4786 vsize=4950744 CPUtime=4721.87 cores=0-7
/proc/4788/task/4800/stat : 4800 (plingeling) R 4786 4788 4762 0 -1 4202560 598958 1253 0 0 472113 74 0 0 25 0 9 0 798956426 5069561856 1198958 33554432000 4194304 4417452 140734906472512 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4788/tid=4801] ppid=4786 vsize=4950744 CPUtime=4720.41 cores=0-7
/proc/4788/task/4801/stat : 4801 (plingeling) R 4786 4788 4762 0 -1 4202560 483638 1253 0 0 471970 71 0 0 25 0 9 0 798956426 5069561856 1198958 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364877 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4788/tid=4802] ppid=4786 vsize=4950744 CPUtime=4722.02 cores=0-7
/proc/4788/task/4802/stat : 4802 (plingeling) R 4786 4788 4762 0 -1 4202560 390054 1253 0 0 472147 55 0 0 25 0 9 0 798956427 5069561856 1198958 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37701.4
Current children cumulated vsize (KiB) 4950744

[startup+4782.3 s]
/proc/loadavg: 8.06 8.05 8.00 9/471 4950
/proc/meminfo: memFree=7765420/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=4996880 CPUtime=38180.3 cores=0-7
/proc/4788/stat : 4788 (plingeling) S 4786 4788 4762 0 -1 4202496 7004865 1253 0 0 3816870 1163 0 0 18 0 9 0 798956387 5116805120 1210492 33554432000 4194304 4417452 140734906472512 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4788/statm: 1249220 1210492 180 55 0 1247106 0
[pid=4788/tid=4795] ppid=4786 vsize=4996880 CPUtime=4781.69 cores=0-7
/proc/4788/task/4795/stat : 4795 (plingeling) R 4786 4788 4762 0 -1 4202560 550356 1253 0 0 478080 89 0 0 25 0 9 0 798956426 5116805120 1210492 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4788/tid=4796] ppid=4786 vsize=4996880 CPUtime=4733.97 cores=0-7
/proc/4788/task/4796/stat : 4796 (plingeling) R 4786 4788 4762 0 -1 4202560 724092 1253 0 0 473284 113 0 0 25 0 9 0 798956426 5116805120 1210492 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4788/tid=4797] ppid=4786 vsize=4996880 CPUtime=4755.94 cores=0-7
/proc/4788/task/4797/stat : 4797 (plingeling) R 4786 4788 4762 0 -1 4202560 663244 1253 0 0 475495 99 0 0 25 0 9 0 798956426 5116805120 1210492 33554432000 4194304 4417452 140734906472512 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4788/tid=4798] ppid=4786 vsize=4996880 CPUtime=4782.02 cores=0-7
/proc/4788/task/4798/stat : 4798 (plingeling) R 4786 4788 4762 0 -1 4202560 2501722 1253 0 0 477693 509 0 0 25 0 9 0 798956426 5116805120 1210492 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4788/tid=4799] ppid=4786 vsize=4996880 CPUtime=4782.02 cores=0-7
/proc/4788/task/4799/stat : 4799 (plingeling) R 4786 4788 4762 0 -1 4202560 1041546 1253 0 0 478058 144 0 0 25 0 9 0 798956426 5116805120 1210492 33554432000 4194304 4417452 140734906472512 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4788/tid=4800] ppid=4786 vsize=4996880 CPUtime=4781.86 cores=0-7
/proc/4788/task/4800/stat : 4800 (plingeling) R 4786 4788 4762 0 -1 4202560 623975 1253 0 0 478108 78 0 0 25 0 9 0 798956426 5116805120 1210492 33554432000 4194304 4417452 140734906472512 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4788/tid=4801] ppid=4786 vsize=4996880 CPUtime=4780.39 cores=0-7
/proc/4788/task/4801/stat : 4801 (plingeling) R 4786 4788 4762 0 -1 4202560 492527 1253 0 0 477967 72 0 0 25 0 9 0 798956426 5116805120 1210492 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4788/tid=4802] ppid=4786 vsize=4996880 CPUtime=4782 cores=0-7
/proc/4788/task/4802/stat : 4802 (plingeling) R 4786 4788 4762 0 -1 4202560 390054 1253 0 0 478145 55 0 0 25 0 9 0 798956427 5116805120 1210492 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38180.3
Current children cumulated vsize (KiB) 4996880

[startup+4842.3 s]
/proc/loadavg: 8.05 8.06 8.00 9/471 4952
/proc/meminfo: memFree=7704404/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=5083008 CPUtime=38659.4 cores=0-7
/proc/4788/stat : 4788 (plingeling) S 4786 4788 4762 0 -1 4202496 7087040 1253 0 0 3864762 1176 0 0 18 0 9 0 798956387 5205000192 1232024 33554432000 4194304 4417452 140734906472512 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4788/statm: 1270752 1232024 180 55 0 1268638 0
[pid=4788/tid=4795] ppid=4786 vsize=5083008 CPUtime=4841.68 cores=0-7
/proc/4788/task/4795/stat : 4795 (plingeling) R 4786 4788 4762 0 -1 4202560 565188 1253 0 0 484077 91 0 0 25 0 9 0 798956426 5205000192 1232024 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4788/tid=4796] ppid=4786 vsize=5083008 CPUtime=4793.04 cores=0-7
/proc/4788/task/4796/stat : 4796 (plingeling) R 4786 4788 4762 0 -1 4202560 730413 1253 0 0 479191 113 0 0 25 0 9 0 798956426 5205000192 1232024 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4788/tid=4797] ppid=4786 vsize=5083008 CPUtime=4815.93 cores=0-7
/proc/4788/task/4797/stat : 4797 (plingeling) R 4786 4788 4762 0 -1 4202560 693966 1253 0 0 481488 105 0 0 25 0 9 0 798956426 5205000192 1232024 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4788/tid=4798] ppid=4786 vsize=5083008 CPUtime=4842.02 cores=0-7
/proc/4788/task/4798/stat : 4798 (plingeling) R 4786 4788 4762 0 -1 4202560 2501722 1253 0 0 483693 509 0 0 25 0 9 0 798956426 5205000192 1232024 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4788/tid=4799] ppid=4786 vsize=5083008 CPUtime=4842.02 cores=0-7
/proc/4788/task/4799/stat : 4799 (plingeling) R 4786 4788 4762 0 -1 4202560 1043611 1253 0 0 484058 144 0 0 25 0 9 0 798956426 5205000192 1232024 33554432000 4194304 4417452 140734906472512 18446744073709551615 4366074 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4788/tid=4800] ppid=4786 vsize=5083008 CPUtime=4841.87 cores=0-7
/proc/4788/task/4800/stat : 4800 (plingeling) R 4786 4788 4762 0 -1 4202560 629927 1253 0 0 484108 79 0 0 25 0 9 0 798956426 5205000192 1232024 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4788/tid=4801] ppid=4786 vsize=5083008 CPUtime=4840.37 cores=0-7
/proc/4788/task/4801/stat : 4801 (plingeling) R 4786 4788 4762 0 -1 4202560 498425 1253 0 0 483964 73 0 0 25 0 9 0 798956426 5205000192 1232024 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4788/tid=4802] ppid=4786 vsize=5083008 CPUtime=4842 cores=0-7
/proc/4788/task/4802/stat : 4802 (plingeling) R 4786 4788 4762 0 -1 4202560 406439 1253 0 0 484143 57 0 0 25 0 9 0 798956427 5205000192 1232024 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38659.4
Current children cumulated vsize (KiB) 5083008

[startup+4902.3 s]
/proc/loadavg: 8.16 8.09 8.01 9/471 4953
/proc/meminfo: memFree=7479932/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=5291940 CPUtime=39138.4 cores=0-7
/proc/4788/stat : 4788 (plingeling) S 4786 4788 4762 0 -1 4202496 7203312 1253 0 0 3912654 1191 0 0 18 0 9 0 798956387 5418946560 1284257 33554432000 4194304 4417452 140734906472512 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4788/statm: 1322985 1284257 180 55 0 1320871 0
[pid=4788/tid=4795] ppid=4786 vsize=5291940 CPUtime=4901.69 cores=0-7
/proc/4788/task/4795/stat : 4795 (plingeling) R 4786 4788 4762 0 -1 4202560 575956 1253 0 0 490076 93 0 0 25 0 9 0 798956426 5418946560 1284257 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4788/tid=4796] ppid=4786 vsize=5291940 CPUtime=4852.12 cores=0-7
/proc/4788/task/4796/stat : 4796 (plingeling) R 4786 4788 4762 0 -1 4202560 745830 1253 0 0 485097 115 0 0 25 0 9 0 798956426 5418946560 1284257 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4788/tid=4797] ppid=4786 vsize=5291940 CPUtime=4875.93 cores=0-7
/proc/4788/task/4797/stat : 4797 (plingeling) R 4786 4788 4762 0 -1 4202560 708082 1253 0 0 487486 107 0 0 25 0 9 0 798956426 5418946560 1284257 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4788/tid=4798] ppid=4786 vsize=5291940 CPUtime=4902.02 cores=0-7
/proc/4788/task/4798/stat : 4798 (plingeling) R 4786 4788 4762 0 -1 4202560 2505571 1253 0 0 489693 509 0 0 25 0 9 0 798956426 5418946560 1284257 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4788/tid=4799] ppid=4786 vsize=5291940 CPUtime=4902.02 cores=0-7
/proc/4788/task/4799/stat : 4799 (plingeling) R 4786 4788 4762 0 -1 4202560 1053539 1253 0 0 490055 147 0 0 25 0 9 0 798956426 5418946560 1284257 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364657 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4788/tid=4800] ppid=4786 vsize=5291940 CPUtime=4901.85 cores=0-7
/proc/4788/task/4800/stat : 4800 (plingeling) R 4786 4788 4762 0 -1 4202560 661287 1253 0 0 490103 82 0 0 25 0 9 0 798956426 5418946560 1284257 33554432000 4194304 4417452 140734906472512 18446744073709551615 4344176 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4788/tid=4801] ppid=4786 vsize=5291940 CPUtime=4900.36 cores=0-7
/proc/4788/task/4801/stat : 4801 (plingeling) R 4786 4788 4762 0 -1 4202560 504681 1253 0 0 489963 73 0 0 25 0 9 0 798956426 5418946560 1284257 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4788/tid=4802] ppid=4786 vsize=5291940 CPUtime=4902 cores=0-7
/proc/4788/task/4802/stat : 4802 (plingeling) R 4786 4788 4762 0 -1 4202560 431017 1253 0 0 490141 59 0 0 25 0 9 0 798956427 5418946560 1284257 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39138.4
Current children cumulated vsize (KiB) 5291940

[startup+4962.3 s]
/proc/loadavg: 8.06 8.07 8.00 9/471 4955
/proc/meminfo: memFree=7226956/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=5534108 CPUtime=39617.5 cores=0-7
/proc/4788/stat : 4788 (plingeling) S 4786 4788 4762 0 -1 4202496 7353274 1253 0 0 3960536 1216 0 0 18 0 9 0 798956387 5666926592 1344799 33554432000 4194304 4417452 140734906472512 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4788/statm: 1383527 1344799 180 55 0 1381413 0
[pid=4788/tid=4795] ppid=4786 vsize=5534108 CPUtime=4961.69 cores=0-7
/proc/4788/task/4795/stat : 4795 (plingeling) R 4786 4788 4762 0 -1 4202560 580313 1253 0 0 496075 94 0 0 25 0 9 0 798956426 5666926592 1344799 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4788/tid=4796] ppid=4786 vsize=5534108 CPUtime=4911.19 cores=0-7
/proc/4788/task/4796/stat : 4796 (plingeling) R 4786 4788 4762 0 -1 4202560 760278 1253 0 0 491001 118 0 0 25 0 9 0 798956426 5666926592 1344799 33554432000 4194304 4417452 140734906472512 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4788/tid=4797] ppid=4786 vsize=5534108 CPUtime=4935.94 cores=0-7
/proc/4788/task/4797/stat : 4797 (plingeling) R 4786 4788 4762 0 -1 4202560 717730 1253 0 0 493484 110 0 0 25 0 9 0 798956426 5666926592 1344799 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4788/tid=4798] ppid=4786 vsize=5534108 CPUtime=4962.02 cores=0-7
/proc/4788/task/4798/stat : 4798 (plingeling) R 4786 4788 4762 0 -1 4202560 2505571 1253 0 0 495693 509 0 0 25 0 9 0 798956426 5666926592 1344799 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4788/tid=4799] ppid=4786 vsize=5534108 CPUtime=4962.03 cores=0-7
/proc/4788/task/4799/stat : 4799 (plingeling) R 4786 4788 4762 0 -1 4202560 1081859 1253 0 0 496047 156 0 0 25 0 9 0 798956426 5666926592 1344799 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4788/tid=4800] ppid=4786 vsize=5534108 CPUtime=4961.86 cores=0-7
/proc/4788/task/4800/stat : 4800 (plingeling) R 4786 4788 4762 0 -1 4202560 676934 1253 0 0 496102 84 0 0 25 0 9 0 798956426 5666926592 1344799 33554432000 4194304 4417452 140734906472512 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4788/tid=4801] ppid=4786 vsize=5534108 CPUtime=4960.35 cores=0-7
/proc/4788/task/4801/stat : 4801 (plingeling) R 4786 4788 4762 0 -1 4202560 509185 1253 0 0 495961 74 0 0 25 0 9 0 798956426 5666926592 1344799 33554432000 4194304 4417452 140734906472512 18446744073709551615 4251160 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4788/tid=4802] ppid=4786 vsize=5534108 CPUtime=4962.01 cores=0-7
/proc/4788/task/4802/stat : 4802 (plingeling) R 4786 4788 4762 0 -1 4202560 504055 1253 0 0 496133 68 0 0 25 0 9 0 798956427 5666926592 1344799 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39617.5
Current children cumulated vsize (KiB) 5534108



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.3 s]
/proc/loadavg: 8.02 8.06 8.00 9/471 4956
/proc/meminfo: memFree=6971988/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=5788740 CPUtime=40000.8 cores=0-7
/proc/4788/stat : 4788 (plingeling) S 4786 4788 4762 0 -1 4202496 7439663 1253 0 0 3998845 1238 0 0 18 0 9 0 798956387 5927669760 1408457 33554432000 4194304 4417452 140734906472512 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4788/statm: 1447185 1408457 180 55 0 1445071 0
[pid=4788/tid=4795] ppid=4786 vsize=5788740 CPUtime=5009.71 cores=0-7
/proc/4788/task/4795/stat : 4795 (plingeling) R 4786 4788 4762 0 -1 4202560 583337 1253 0 0 500877 94 0 0 25 0 9 0 798956426 5927669760 1408457 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4788/tid=4796] ppid=4786 vsize=5788740 CPUtime=4958.45 cores=0-7
/proc/4788/task/4796/stat : 4796 (plingeling) R 4786 4788 4762 0 -1 4202560 786277 1253 0 0 495720 125 0 0 25 0 9 0 798956426 5927669760 1408457 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4788/tid=4797] ppid=4786 vsize=5788740 CPUtime=4983.95 cores=0-7
/proc/4788/task/4797/stat : 4797 (plingeling) R 4786 4788 4762 0 -1 4202560 743455 1253 0 0 498279 116 0 0 25 0 9 0 798956426 5927669760 1408457 33554432000 4194304 4417452 140734906472512 18446744073709551615 4236706 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4788/tid=4798] ppid=4786 vsize=5788740 CPUtime=5010.04 cores=0-7
/proc/4788/task/4798/stat : 4798 (plingeling) R 4786 4788 4762 0 -1 4202560 2513269 1253 0 0 500493 511 0 0 25 0 9 0 798956426 5927669760 1408457 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364805 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4788/tid=4799] ppid=4786 vsize=5788740 CPUtime=5010.04 cores=0-7
/proc/4788/task/4799/stat : 4799 (plingeling) R 4786 4788 4762 0 -1 4202560 1090936 1253 0 0 500846 158 0 0 25 0 9 0 798956426 5927669760 1408457 33554432000 4194304 4417452 140734906472512 18446744073709551615 4343760 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4788/tid=4800] ppid=4786 vsize=5788740 CPUtime=5009.87 cores=0-7
/proc/4788/task/4800/stat : 4800 (plingeling) R 4786 4788 4762 0 -1 4202560 685232 1253 0 0 500902 85 0 0 25 0 9 0 798956426 5927669760 1408457 33554432000 4194304 4417452 140734906472512 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4788/tid=4801] ppid=4786 vsize=5788740 CPUtime=5008.36 cores=0-7
/proc/4788/task/4801/stat : 4801 (plingeling) R 4786 4788 4762 0 -1 4202560 515753 1253 0 0 500761 75 0 0 25 0 9 0 798956426 5927669760 1408457 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4788/tid=4802] ppid=4786 vsize=5788740 CPUtime=5010.03 cores=0-7
/proc/4788/task/4802/stat : 4802 (plingeling) R 4786 4788 4762 0 -1 4202560 504055 1253 0 0 500935 68 0 0 25 0 9 0 798956427 5927669760 1408457 33554432000 4194304 4417452 140734906472512 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 5788740

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.42 s]
/proc/loadavg: 8.02 8.06 8.00 2/465 4957
/proc/meminfo: memFree=10054428/32951124 swapFree=57414376/67111528
[pid=4788] ppid=4786 vsize=0 CPUtime=40001.1 cores=0-7
/proc/4788/stat : 4788 (plingeling) Z 4786 4788 4762 0 -1 4203548 7439664 1253 0 0 3998857 1248 0 0 18 0 2 0 798956387 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/4788/statm: 0 0 0 0 0 0 0
[pid=4788/tid=4796] ppid=4786 vsize=0 CPUtime=4958.57 cores=0-7
/proc/4788/task/4796/stat : 4796 (plingeling) R 4786 4788 4762 0 -1 4203588 786277 1253 0 0 495721 136 0 0 25 0 2 0 798956426 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40001.1
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5010.51
CPU time (s): 40001.1
CPU user time (s): 39988.6
CPU system time (s): 12.5661
CPU usage (%): 798.345
Max. virtual memory (cumulated for all children) (KiB): 5879428

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39988.6
system time used= 12.5661
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 7440917
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 916
involuntary context switches= 143710

runsolver used 22.1576 second user time and 55.2736 second system time

The end

Launcher Data 

Begin job on node111 at 2011-05-22 20:00:59
IDJOB=3394086
IDBENCH=71423
IDSOLVER=1851
FILE ID=node111/3394086-1306087259
RUNJOBID= node111-1306087259-4776
PBS_JOBID= 13387923
Free space on /tmp= 73340 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/random/forced/rbsat-v2640c305320gyes3.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394086-1306087259/watcher-3394086-1306087259 -o /tmp/evaluation-result-3394086-1306087259/solver-3394086-1306087259 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394086-1306087259.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= d6877629cc818d7e31dc245fb95f0588
RANDOM SEED=1363977941

node111.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5428.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      12614012 kB
Buffers:        335076 kB
Cached:       11191484 kB
SwapCached:      19180 kB
Active:       10283476 kB
Inactive:      9760684 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      12614012 kB
SwapTotal:    67111528 kB
SwapFree:     57414376 kB
Dirty:            4300 kB
Writeback:           0 kB
AnonPages:     8507348 kB
Mapped:          26784 kB
Slab:           155636 kB
PageTables:      65808 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 149059236 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73340 MiB
End job on node111 at 2011-05-22 21:24:31