Trace number 3394844

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40001.4 5020.71

General information on the benchmark

NameSAT05/crafted/sabharwal05/counting/php/unsat/
easier/php-045-035.shuffled-as.sat05-1181.cnf
MD5SUM057b78ff24cc0773c9c67e770a7b9de2
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark44.4792
Satisfiable
(Un)Satisfiability was proved
Number of variables1575
Number of clauses34695
Sum of the clauses size70875
Maximum clause length35
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 234650
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394844-1306108151/watcher-3394844-1306108151 -o /tmp/evaluation-result-3394844-1306108151/solver-3394844-1306108151 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394844-1306108151.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.15 8.08 8.01 2/510 2837
/proc/meminfo: memFree=9312276/32950928 swapFree=44933860/67111528
[pid=2837] ppid=2835 vsize=10188 CPUtime=0 cores=0-7
/proc/2837/stat : 2837 (plingeling) R 2835 2837 2575 0 -1 4202496 607 1249 0 0 0 0 0 0 22 0 1 0 722411779 10432512 537 33554432000 4194304 4417452 140736128838384 18446744073709551615 4228083 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/2837/statm: 2547 537 147 55 0 433 0

[startup+0.081685 s]
/proc/loadavg: 8.15 8.08 8.01 2/510 2837
/proc/meminfo: memFree=9312276/32950928 swapFree=44933860/67111528
[pid=2837] ppid=2835 vsize=166752 CPUtime=0.22 cores=0-7
/proc/2837/stat : 2837 (plingeling) S 2835 2837 2575 0 -1 4202496 3113 1249 0 0 22 0 0 0 25 0 9 0 722411779 170754048 2936 33554432000 4194304 4417452 140736128838384 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 41688 2936 174 55 0 39574 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 166752

[startup+0.100629 s]
/proc/loadavg: 8.15 8.08 8.01 2/510 2837
/proc/meminfo: memFree=9312276/32950928 swapFree=44933860/67111528
[pid=2837] ppid=2835 vsize=170208 CPUtime=0.36 cores=0-7
/proc/2837/stat : 2837 (plingeling) S 2835 2837 2575 0 -1 4202496 4399 1249 0 0 36 0 0 0 25 0 9 0 722411779 174292992 3706 33554432000 4194304 4417452 140736128838384 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 42552 3706 175 55 0 40438 0
Current children cumulated CPU time (s) 0.36
Current children cumulated vsize (KiB) 170208

[startup+0.300587 s]
/proc/loadavg: 8.15 8.08 8.01 2/510 2837
/proc/meminfo: memFree=9312276/32950928 swapFree=44933860/67111528
[pid=2837] ppid=2835 vsize=173608 CPUtime=1.95 cores=0-7
/proc/2837/stat : 2837 (plingeling) S 2835 2837 2575 0 -1 4202496 6855 1249 0 0 195 0 0 0 25 0 9 0 722411779 177774592 4624 33554432000 4194304 4417452 140736128838384 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 43402 4624 180 55 0 41288 0
Current children cumulated CPU time (s) 1.95
Current children cumulated vsize (KiB) 173608

[startup+0.700511 s]
/proc/loadavg: 8.15 8.08 8.01 2/510 2837
/proc/meminfo: memFree=9312276/32950928 swapFree=44933860/67111528
[pid=2837] ppid=2835 vsize=190268 CPUtime=5.16 cores=0-7
/proc/2837/stat : 2837 (plingeling) S 2835 2837 2575 0 -1 4202496 12972 1249 0 0 514 2 0 0 25 0 9 0 722411779 194834432 8844 33554432000 4194304 4417452 140736128838384 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 47567 8844 180 55 0 45453 0
Current children cumulated CPU time (s) 5.16
Current children cumulated vsize (KiB) 190268

[startup+1.50035 s]
/proc/loadavg: 8.15 8.08 8.01 9/519 2859
/proc/meminfo: memFree=9271408/32950928 swapFree=44933860/67111528
[pid=2837] ppid=2835 vsize=215484 CPUtime=11.53 cores=0-7
/proc/2837/stat : 2837 (plingeling) S 2835 2837 2575 0 -1 4202496 39347 1249 0 0 1145 8 0 0 25 0 9 0 722411779 220655616 15131 33554432000 4194304 4417452 140736128838384 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 53871 15131 180 55 0 51757 0
[pid=2837/tid=2844] ppid=2835 vsize=215484 CPUtime=1.43 cores=0-7
/proc/2837/task/2844/stat : 2844 (plingeling) R 2835 2837 2575 0 -1 4202560 3809 1249 0 0 143 0 0 0 20 0 9 0 722411785 220655616 15131 33554432000 4194304 4417452 140736128838384 18446744073709551615 4356867 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2837/tid=2845] ppid=2835 vsize=215484 CPUtime=1.44 cores=0-7
/proc/2837/task/2845/stat : 2845 (plingeling) R 2835 2837 2575 0 -1 4202560 4542 1249 0 0 143 1 0 0 25 0 9 0 722411785 220655616 15131 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2837/tid=2846] ppid=2835 vsize=215484 CPUtime=1.43 cores=0-7
/proc/2837/task/2846/stat : 2846 (plingeling) R 2835 2837 2575 0 -1 4202560 3849 1249 0 0 143 0 0 0 18 0 9 0 722411785 220655616 15131 33554432000 4194304 4417452 140736128838384 18446744073709551615 4232473 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=2837/tid=2851] ppid=2835 vsize=215484 CPUtime=1.43 cores=0-7
/proc/2837/task/2851/stat : 2851 (plingeling) R 2835 2837 2575 0 -1 4202560 9248 1249 0 0 141 2 0 0 25 0 9 0 722411785 220655616 15131 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364877 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2837/tid=2856] ppid=2835 vsize=215484 CPUtime=1.43 cores=0-7
/proc/2837/task/2856/stat : 2856 (plingeling) R 2835 2837 2575 0 -1 4202560 3940 1249 0 0 143 0 0 0 19 0 9 0 722411785 220655616 15131 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2837/tid=2857] ppid=2835 vsize=215484 CPUtime=1.38 cores=0-7
/proc/2837/task/2857/stat : 2857 (plingeling) R 2835 2837 2575 0 -1 4202560 3572 1249 0 0 138 0 0 0 25 0 9 0 722411785 220655616 15131 33554432000 4194304 4417452 140736128838384 18446744073709551615 4250762 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2837/tid=2858] ppid=2835 vsize=215484 CPUtime=1.43 cores=0-7
/proc/2837/task/2858/stat : 2858 (plingeling) R 2835 2837 2575 0 -1 4202560 3521 1249 0 0 143 0 0 0 19 0 9 0 722411785 220655616 15131 33554432000 4194304 4417452 140736128838384 18446744073709551615 4232664 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=2837/tid=2859] ppid=2835 vsize=215484 CPUtime=1.43 cores=0-7
/proc/2837/task/2859/stat : 2859 (plingeling) R 2835 2837 2575 0 -1 4202560 4066 1249 0 0 143 0 0 0 25 0 9 0 722411785 220655616 15131 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.53
Current children cumulated vsize (KiB) 215484

[startup+3.10104 s]
/proc/loadavg: 8.15 8.08 8.01 9/519 2859
/proc/meminfo: memFree=9236316/32950928 swapFree=44933860/67111528
[pid=2837] ppid=2835 vsize=238424 CPUtime=24.3 cores=0-7
/proc/2837/stat : 2837 (plingeling) S 2835 2837 2575 0 -1 4202496 89777 1249 0 0 2411 19 0 0 25 0 9 0 722411779 244146176 20882 33554432000 4194304 4417452 140736128838384 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 59606 20882 180 55 0 57492 0
[pid=2837/tid=2844] ppid=2835 vsize=238424 CPUtime=3.04 cores=0-7
/proc/2837/task/2844/stat : 2844 (plingeling) R 2835 2837 2575 0 -1 4202560 9270 1249 0 0 302 2 0 0 25 0 9 0 722411785 244146176 20882 33554432000 4194304 4417452 140736128838384 18446744073709551615 4254843 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2837/tid=2845] ppid=2835 vsize=238424 CPUtime=3.03 cores=0-7
/proc/2837/task/2845/stat : 2845 (plingeling) R 2835 2837 2575 0 -1 4202560 10024 1249 0 0 301 2 0 0 25 0 9 0 722411785 244146176 20882 33554432000 4194304 4417452 140736128838384 18446744073709551615 4232695 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2837/tid=2846] ppid=2835 vsize=238424 CPUtime=3.03 cores=0-7
/proc/2837/task/2846/stat : 2846 (plingeling) R 2835 2837 2575 0 -1 4202560 10131 1249 0 0 302 1 0 0 20 0 9 0 722411785 244146176 20882 33554432000 4194304 4417452 140736128838384 18446744073709551615 4253491 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=2837/tid=2851] ppid=2835 vsize=238424 CPUtime=3.03 cores=0-7
/proc/2837/task/2851/stat : 2851 (plingeling) R 2835 2837 2575 0 -1 4202560 20208 1249 0 0 298 5 0 0 20 0 9 0 722411785 244146176 20882 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2837/tid=2856] ppid=2835 vsize=238424 CPUtime=3.03 cores=0-7
/proc/2837/task/2856/stat : 2856 (plingeling) R 2835 2837 2575 0 -1 4202560 9347 1249 0 0 302 1 0 0 25 0 9 0 722411785 244146176 20882 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2837/tid=2857] ppid=2835 vsize=238424 CPUtime=2.97 cores=0-7
/proc/2837/task/2857/stat : 2857 (plingeling) R 2835 2837 2575 0 -1 4202560 9284 1249 0 0 295 2 0 0 25 0 9 0 722411785 244146176 20882 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2837/tid=2858] ppid=2835 vsize=238424 CPUtime=3.03 cores=0-7
/proc/2837/task/2858/stat : 2858 (plingeling) R 2835 2837 2575 0 -1 4202560 9988 1249 0 0 301 2 0 0 25 0 9 0 722411785 244146176 20882 33554432000 4194304 4417452 140736128838384 18446744073709551615 4358214 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=2837/tid=2859] ppid=2835 vsize=238424 CPUtime=3.03 cores=0-7
/proc/2837/task/2859/stat : 2859 (plingeling) R 2835 2837 2575 0 -1 4202560 8725 1249 0 0 302 1 0 0 25 0 9 0 722411785 244146176 20882 33554432000 4194304 4417452 140736128838384 18446744073709551615 4258359 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 24.3
Current children cumulated vsize (KiB) 238424

[startup+6.3004 s]
/proc/loadavg: 8.14 8.08 8.01 9/519 2859
/proc/meminfo: memFree=9179964/32950928 swapFree=44933860/67111528
[pid=2837] ppid=2835 vsize=318044 CPUtime=49.82 cores=0-7
/proc/2837/stat : 2837 (plingeling) S 2835 2837 2575 0 -1 4202496 195986 1249 0 0 4943 39 0 0 25 0 9 0 722411779 325677056 40896 33554432000 4194304 4417452 140736128838384 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 79511 40896 180 55 0 77397 0
[pid=2837/tid=2844] ppid=2835 vsize=318044 CPUtime=6.23 cores=0-7
/proc/2837/task/2844/stat : 2844 (plingeling) R 2835 2837 2575 0 -1 4202560 23932 1249 0 0 619 4 0 0 25 0 9 0 722411785 325677056 40896 33554432000 4194304 4417452 140736128838384 18446744073709551615 4366117 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2837/tid=2845] ppid=2835 vsize=318044 CPUtime=6.24 cores=0-7
/proc/2837/task/2845/stat : 2845 (plingeling) R 2835 2837 2575 0 -1 4202560 22624 1249 0 0 619 5 0 0 25 0 9 0 722411785 325677056 40896 33554432000 4194304 4417452 140736128838384 18446744073709551615 4251241 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2837/tid=2846] ppid=2835 vsize=318044 CPUtime=6.22 cores=0-7
/proc/2837/task/2846/stat : 2846 (plingeling) R 2835 2837 2575 0 -1 4202560 23886 1249 0 0 619 3 0 0 25 0 9 0 722411785 325677056 40896 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=2837/tid=2851] ppid=2835 vsize=318044 CPUtime=6.23 cores=0-7
/proc/2837/task/2851/stat : 2851 (plingeling) R 2835 2837 2575 0 -1 4202560 30765 1249 0 0 616 7 0 0 25 0 9 0 722411785 325677056 40896 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2837/tid=2856] ppid=2835 vsize=318044 CPUtime=6.23 cores=0-7
/proc/2837/task/2856/stat : 2856 (plingeling) R 2835 2837 2575 0 -1 4202560 23233 1249 0 0 620 3 0 0 25 0 9 0 722411785 325677056 40896 33554432000 4194304 4417452 140736128838384 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2837/tid=2857] ppid=2835 vsize=318044 CPUtime=6.11 cores=0-7
/proc/2837/task/2857/stat : 2857 (plingeling) R 2835 2837 2575 0 -1 4202560 24333 1249 0 0 606 5 0 0 25 0 9 0 722411785 325677056 40896 33554432000 4194304 4417452 140736128838384 18446744073709551615 4321709 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2837/tid=2858] ppid=2835 vsize=318044 CPUtime=6.23 cores=0-7
/proc/2837/task/2858/stat : 2858 (plingeling) R 2835 2837 2575 0 -1 4202560 20738 1249 0 0 619 4 0 0 25 0 9 0 722411785 325677056 40896 33554432000 4194304 4417452 140736128838384 18446744073709551615 4357805 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=2837/tid=2859] ppid=2835 vsize=318044 CPUtime=6.23 cores=0-7
/proc/2837/task/2859/stat : 2859 (plingeling) R 2835 2837 2575 0 -1 4202560 23675 1249 0 0 618 5 0 0 25 0 9 0 722411785 325677056 40896 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 49.82
Current children cumulated vsize (KiB) 318044

[startup+12.7011 s]
/proc/loadavg: 8.12 8.07 8.01 9/519 2859
/proc/meminfo: memFree=9100768/32950928 swapFree=44933860/67111528
[pid=2837] ppid=2835 vsize=382648 CPUtime=100.92 cores=0-7
/proc/2837/stat : 2837 (plingeling) S 2835 2837 2575 0 -1 4202496 382302 1249 0 0 10017 75 0 0 25 0 9 0 722411779 391831552 57206 33554432000 4194304 4417452 140736128838384 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 95662 57206 180 55 0 93548 0
[pid=2837/tid=2844] ppid=2835 vsize=382648 CPUtime=12.63 cores=0-7
/proc/2837/task/2844/stat : 2844 (plingeling) R 2835 2837 2575 0 -1 4202560 45427 1249 0 0 1255 8 0 0 25 0 9 0 722411785 391831552 57206 33554432000 4194304 4417452 140736128838384 18446744073709551615 4259283 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2837/tid=2845] ppid=2835 vsize=382648 CPUtime=12.63 cores=0-7
/proc/2837/task/2845/stat : 2845 (plingeling) R 2835 2837 2575 0 -1 4202560 44601 1249 0 0 1254 9 0 0 25 0 9 0 722411785 391831552 57206 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2837/tid=2846] ppid=2835 vsize=382648 CPUtime=12.63 cores=0-7
/proc/2837/task/2846/stat : 2846 (plingeling) R 2835 2837 2575 0 -1 4202560 48658 1249 0 0 1256 7 0 0 25 0 9 0 722411785 391831552 57206 33554432000 4194304 4417452 140736128838384 18446744073709551615 4251133 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=2837/tid=2851] ppid=2835 vsize=382648 CPUtime=12.63 cores=0-7
/proc/2837/task/2851/stat : 2851 (plingeling) R 2835 2837 2575 0 -1 4202560 48626 1249 0 0 1252 11 0 0 25 0 9 0 722411785 391831552 57206 33554432000 4194304 4417452 140736128838384 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2837/tid=2856] ppid=2835 vsize=382648 CPUtime=12.62 cores=0-7
/proc/2837/task/2856/stat : 2856 (plingeling) R 2835 2837 2575 0 -1 4202560 47981 1249 0 0 1254 8 0 0 25 0 9 0 722411785 391831552 57206 33554432000 4194304 4417452 140736128838384 18446744073709551615 4270119 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2837/tid=2857] ppid=2835 vsize=382648 CPUtime=12.39 cores=0-7
/proc/2837/task/2857/stat : 2857 (plingeling) R 2835 2837 2575 0 -1 4202560 48587 1249 0 0 1230 9 0 0 25 0 9 0 722411785 391831552 57206 33554432000 4194304 4417452 140736128838384 18446744073709551615 4250571 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2837/tid=2858] ppid=2835 vsize=382648 CPUtime=12.63 cores=0-7
/proc/2837/task/2858/stat : 2858 (plingeling) R 2835 2837 2575 0 -1 4202560 48731 1249 0 0 1253 10 0 0 25 0 9 0 722411785 391831552 57206 33554432000 4194304 4417452 140736128838384 18446744073709551615 4232508 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=2837/tid=2859] ppid=2835 vsize=382648 CPUtime=12.63 cores=0-7
/proc/2837/task/2859/stat : 2859 (plingeling) R 2835 2837 2575 0 -1 4202560 46891 1249 0 0 1255 8 0 0 25 0 9 0 722411785 391831552 57206 33554432000 4194304 4417452 140736128838384 18446744073709551615 4358176 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 100.92
Current children cumulated vsize (KiB) 382648

[startup+25.5006 s]
/proc/loadavg: 8.10 8.07 8.01 9/519 2860
/proc/meminfo: memFree=8863708/32950928 swapFree=44933860/67111528
[pid=2837] ppid=2835 vsize=586004 CPUtime=203.05 cores=0-7
/proc/2837/stat : 2837 (plingeling) S 2835 2837 2575 0 -1 4202496 702785 1249 0 0 20171 134 0 0 25 0 9 0 722411779 600068096 108328 33554432000 4194304 4417452 140736128838384 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 146501 108328 180 55 0 144387 0
[pid=2837/tid=2844] ppid=2835 vsize=586004 CPUtime=25.43 cores=0-7
/proc/2837/task/2844/stat : 2844 (plingeling) R 2835 2837 2575 0 -1 4202560 86412 1249 0 0 2528 15 0 0 25 0 9 0 722411785 600068096 108328 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2837/tid=2845] ppid=2835 vsize=586004 CPUtime=25.43 cores=0-7
/proc/2837/task/2845/stat : 2845 (plingeling) R 2835 2837 2575 0 -1 4202560 88287 1249 0 0 2527 16 0 0 25 0 9 0 722411785 600068096 108328 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364972 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2837/tid=2846] ppid=2835 vsize=586004 CPUtime=25.42 cores=0-7
/proc/2837/task/2846/stat : 2846 (plingeling) R 2835 2837 2575 0 -1 4202560 84261 1249 0 0 2528 14 0 0 25 0 9 0 722411785 600068096 108328 33554432000 4194304 4417452 140736128838384 18446744073709551615 4265025 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=2837/tid=2851] ppid=2835 vsize=586004 CPUtime=25.43 cores=0-7
/proc/2837/task/2851/stat : 2851 (plingeling) R 2835 2837 2575 0 -1 4202560 87190 1249 0 0 2523 20 0 0 25 0 9 0 722411785 600068096 108328 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2837/tid=2856] ppid=2835 vsize=586004 CPUtime=25.43 cores=0-7
/proc/2837/task/2856/stat : 2856 (plingeling) R 2835 2837 2575 0 -1 4202560 84369 1249 0 0 2528 15 0 0 25 0 9 0 722411785 600068096 108328 33554432000 4194304 4417452 140736128838384 18446744073709551615 4250651 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2837/tid=2857] ppid=2835 vsize=586004 CPUtime=24.96 cores=0-7
/proc/2837/task/2857/stat : 2857 (plingeling) R 2835 2837 2575 0 -1 4202560 88860 1249 0 0 2479 17 0 0 25 0 9 0 722411785 600068096 108328 33554432000 4194304 4417452 140736128838384 18446744073709551615 4244976 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2837/tid=2858] ppid=2835 vsize=586004 CPUtime=25.42 cores=0-7
/proc/2837/task/2858/stat : 2858 (plingeling) R 2835 2837 2575 0 -1 4202560 89158 1249 0 0 2524 18 0 0 25 0 9 0 722411785 600068096 108328 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=2837/tid=2859] ppid=2835 vsize=586004 CPUtime=25.43 cores=0-7
/proc/2837/task/2859/stat : 2859 (plingeling) R 2835 2837 2575 0 -1 4202560 91448 1249 0 0 2527 16 0 0 25 0 9 0 722411785 600068096 108328 33554432000 4194304 4417452 140736128838384 18446744073709551615 4366094 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 203.05
Current children cumulated vsize (KiB) 586004

[startup+51.1005 s]
/proc/loadavg: 8.06 8.06 8.00 9/519 2860
/proc/meminfo: memFree=8707624/32950928 swapFree=44933860/67111528
[pid=2837] ppid=2835 vsize=746152 CPUtime=407.39 cores=0-7
/proc/2837/stat : 2837 (plingeling) S 2835 2837 2575 0 -1 4202496 1164457 1249 0 0 40524 215 0 0 25 0 9 0 722411779 764059648 147751 33554432000 4194304 4417452 140736128838384 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 186538 147751 180 55 0 184424 0
[pid=2837/tid=2844] ppid=2835 vsize=746152 CPUtime=51.02 cores=0-7
/proc/2837/task/2844/stat : 2844 (plingeling) R 2835 2837 2575 0 -1 4202560 147494 1249 0 0 5076 26 0 0 25 0 9 0 722411785 764059648 147751 33554432000 4194304 4417452 140736128838384 18446744073709551615 4311298 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2837/tid=2845] ppid=2835 vsize=746152 CPUtime=51.02 cores=0-7
/proc/2837/task/2845/stat : 2845 (plingeling) R 2835 2837 2575 0 -1 4202560 145476 1249 0 0 5077 25 0 0 25 0 9 0 722411785 764059648 147751 33554432000 4194304 4417452 140736128838384 18446744073709551615 4310373 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2837/tid=2846] ppid=2835 vsize=746152 CPUtime=51 cores=0-7

################
# More data... #
################

[pid=2837/tid=2846] ppid=2835 vsize=16508252 CPUtime=4714.48 cores=0-7
/proc/2837/task/2846/stat : 2846 (plingeling) R 2835 2837 2575 0 -1 4202560 3352763 1249 0 0 470640 808 0 0 25 0 9 0 722411785 16904450048 4088234 33554432000 4194304 4417452 140736128838384 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 5 0 0 54
[pid=2837/tid=2851] ppid=2835 vsize=16508252 CPUtime=4707.97 cores=0-7
/proc/2837/task/2851/stat : 2851 (plingeling) R 2835 2837 2575 0 -1 4202560 1759599 1249 0 0 470118 679 0 0 25 0 9 0 722411785 16904450048 4088234 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364716 0 0 4096 17442 0 0 0 -1 0 0 0 228
[pid=2837/tid=2856] ppid=2835 vsize=16508252 CPUtime=4696.48 cores=0-7
/proc/2837/task/2856/stat : 2856 (plingeling) R 2835 2837 2575 0 -1 4202560 3676842 1249 0 0 468903 745 0 0 25 0 9 0 722411785 16904450048 4088234 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 112
[pid=2837/tid=2857] ppid=2835 vsize=16508252 CPUtime=4692.57 cores=0-7
/proc/2837/task/2857/stat : 2857 (plingeling) R 2835 2837 2575 0 -1 4202560 3420916 1249 0 0 468355 902 0 0 25 0 9 0 722411785 16904450048 4088234 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 176
[pid=2837/tid=2858] ppid=2835 vsize=16508252 CPUtime=4706.88 cores=0-7
/proc/2837/task/2858/stat : 2858 (plingeling) R 2835 2837 2575 0 -1 4202560 2989082 1249 0 0 469989 699 0 0 25 0 9 0 722411785 16904450048 4088234 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 121
[pid=2837/tid=2859] ppid=2835 vsize=16508252 CPUtime=4703.43 cores=0-7
/proc/2837/task/2859/stat : 2859 (plingeling) R 2835 2837 2575 0 -1 4202560 3091138 1249 0 0 469575 768 0 0 25 0 9 0 722411785 16904450048 4088234 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 199
Current children cumulated CPU time (s) 37630.9
Current children cumulated vsize (KiB) 16508252

[startup+4782.3 s]
/proc/loadavg: 8.13 8.26 8.17 9/519 3020
/proc/meminfo: memFree=103252/32950928 swapFree=38704864/67111528
[pid=2837] ppid=2835 vsize=16508252 CPUtime=38109.9 cores=0-7
/proc/2837/stat : 2837 (plingeling) S 2835 2837 2575 0 -1 4202496 25840499 1249 0 0 3804397 6588 0 0 25 0 9 0 722411779 16904450048 4088234 33554432000 4194304 4417452 140736128838384 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 4127063 4088234 180 55 0 4124949 0
[pid=2837/tid=2844] ppid=2835 vsize=16508252 CPUtime=4756.93 cores=0-7
/proc/2837/task/2844/stat : 2844 (plingeling) R 2835 2837 2575 0 -1 4202560 3541065 1249 0 0 474840 853 0 0 25 0 9 0 722411785 16904450048 4088234 33554432000 4194304 4417452 140736128838384 18446744073709551615 4259603 0 0 4096 17442 0 0 0 -1 4 0 0 56
[pid=2837/tid=2845] ppid=2835 vsize=16508252 CPUtime=4772.02 cores=0-7
/proc/2837/task/2845/stat : 2845 (plingeling) R 2835 2837 2575 0 -1 4202560 4006294 1249 0 0 476073 1129 0 0 25 0 9 0 722411785 16904450048 4088234 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 253
[pid=2837/tid=2846] ppid=2835 vsize=16508252 CPUtime=4774.48 cores=0-7
/proc/2837/task/2846/stat : 2846 (plingeling) R 2835 2837 2575 0 -1 4202560 3352763 1249 0 0 476640 808 0 0 25 0 9 0 722411785 16904450048 4088234 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 54
[pid=2837/tid=2851] ppid=2835 vsize=16508252 CPUtime=4767.98 cores=0-7
/proc/2837/task/2851/stat : 2851 (plingeling) R 2835 2837 2575 0 -1 4202560 1759599 1249 0 0 476119 679 0 0 25 0 9 0 722411785 16904450048 4088234 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 228
[pid=2837/tid=2856] ppid=2835 vsize=16508252 CPUtime=4756.49 cores=0-7
/proc/2837/task/2856/stat : 2856 (plingeling) R 2835 2837 2575 0 -1 4202560 3676842 1249 0 0 474904 745 0 0 25 0 9 0 722411785 16904450048 4088234 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 112
[pid=2837/tid=2857] ppid=2835 vsize=16508252 CPUtime=4752.57 cores=0-7
/proc/2837/task/2857/stat : 2857 (plingeling) R 2835 2837 2575 0 -1 4202560 3420916 1249 0 0 474355 902 0 0 25 0 9 0 722411785 16904450048 4088234 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 176
[pid=2837/tid=2858] ppid=2835 vsize=16508252 CPUtime=4766.87 cores=0-7
/proc/2837/task/2858/stat : 2858 (plingeling) R 2835 2837 2575 0 -1 4202560 2989082 1249 0 0 475988 699 0 0 25 0 9 0 722411785 16904450048 4088234 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 121
[pid=2837/tid=2859] ppid=2835 vsize=16508252 CPUtime=4762.39 cores=0-7
/proc/2837/task/2859/stat : 2859 (plingeling) R 2835 2837 2575 0 -1 4202560 3091138 1249 0 0 475471 768 0 0 25 0 9 0 722411785 16904450048 4088234 33554432000 4194304 4417452 140736128838384 18446744073709551615 4309183 0 0 4096 17442 0 0 0 -1 7 0 0 199
Current children cumulated CPU time (s) 38109.8
Current children cumulated vsize (KiB) 16508252

[startup+4842.3 s]
/proc/loadavg: 8.23 8.26 8.18 9/519 3021
/proc/meminfo: memFree=103396/32950928 swapFree=38329860/67111528
[pid=2837] ppid=2835 vsize=17115480 CPUtime=38584.5 cores=0-7
/proc/2837/stat : 2837 (plingeling) S 2835 2837 2575 0 -1 4202496 26116268 1249 0 0 3851666 6784 0 0 25 0 9 0 722411779 17526251520 4240041 33554432000 4194304 4417452 140736128838384 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 4278870 4240041 180 55 0 4276756 0
[pid=2837/tid=2844] ppid=2835 vsize=17115480 CPUtime=4815.92 cores=0-7
/proc/2837/task/2844/stat : 2844 (plingeling) R 2835 2837 2575 0 -1 4202560 3541065 1249 0 0 480739 853 0 0 25 0 9 0 722411785 17526251520 4240041 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 56
[pid=2837/tid=2845] ppid=2835 vsize=17115480 CPUtime=4831.63 cores=0-7
/proc/2837/task/2845/stat : 2845 (plingeling) R 2835 2837 2575 0 -1 4202560 4006294 1249 0 0 482034 1129 0 0 25 0 9 0 722411785 17526251520 4240041 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 1 0 0 253
[pid=2837/tid=2846] ppid=2835 vsize=17115480 CPUtime=4833.83 cores=0-7
/proc/2837/task/2846/stat : 2846 (plingeling) R 2835 2837 2575 0 -1 4202560 3352763 1249 0 0 482575 808 0 0 25 0 9 0 722411785 17526251520 4240041 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 54
[pid=2837/tid=2851] ppid=2835 vsize=17115480 CPUtime=4827.52 cores=0-7
/proc/2837/task/2851/stat : 2851 (plingeling) R 2835 2837 2575 0 -1 4202560 1759599 1249 0 0 482072 680 0 0 25 0 9 0 722411785 17526251520 4240041 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 2 0 0 228
[pid=2837/tid=2856] ppid=2835 vsize=17115480 CPUtime=4815.48 cores=0-7
/proc/2837/task/2856/stat : 2856 (plingeling) R 2835 2837 2575 0 -1 4202560 3676842 1249 0 0 480802 746 0 0 25 0 9 0 722411785 17526251520 4240041 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364812 0 0 4096 17442 0 0 0 -1 7 0 0 112
[pid=2837/tid=2857] ppid=2835 vsize=17115480 CPUtime=4812.19 cores=0-7
/proc/2837/task/2857/stat : 2857 (plingeling) R 2835 2837 2575 0 -1 4202560 3420916 1249 0 0 480317 902 0 0 25 0 9 0 722411785 17526251520 4240041 33554432000 4194304 4417452 140736128838384 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 3 0 0 176
[pid=2837/tid=2858] ppid=2835 vsize=17115480 CPUtime=4826.72 cores=0-7
/proc/2837/task/2858/stat : 2858 (plingeling) R 2835 2837 2575 0 -1 4202560 2989082 1249 0 0 481967 705 0 0 25 0 9 0 722411785 17526251520 4240041 33554432000 4194304 4417452 140736128838384 18446744073709551615 4263056 0 0 4096 17442 0 0 0 -1 6 0 0 121
[pid=2837/tid=2859] ppid=2835 vsize=17115480 CPUtime=4821.08 cores=0-7
/proc/2837/task/2859/stat : 2859 (plingeling) R 2835 2837 2575 0 -1 4202560 3366907 1249 0 0 481152 956 0 0 25 0 9 0 722411785 17526251520 4240041 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 211
Current children cumulated CPU time (s) 38584.5
Current children cumulated vsize (KiB) 17115480

[startup+4902.3 s]
/proc/loadavg: 8.15 8.23 8.17 9/519 3023
/proc/meminfo: memFree=447572/32950928 swapFree=38263552/67111528
[pid=2837] ppid=2835 vsize=17126792 CPUtime=39062.5 cores=0-7
/proc/2837/stat : 2837 (plingeling) S 2835 2837 2575 0 -1 4202496 26207018 1249 0 0 3899406 6843 0 0 25 0 9 0 722411779 17537835008 4242869 33554432000 4194304 4417452 140736128838384 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 4281698 4242869 180 55 0 4279584 0
[pid=2837/tid=2844] ppid=2835 vsize=17126792 CPUtime=4875.62 cores=0-7
/proc/2837/task/2844/stat : 2844 (plingeling) R 2835 2837 2575 0 -1 4202560 3541065 1249 0 0 486708 854 0 0 25 0 9 0 722411785 17537835008 4242869 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 56
[pid=2837/tid=2845] ppid=2835 vsize=17126792 CPUtime=4891.24 cores=0-7
/proc/2837/task/2845/stat : 2845 (plingeling) R 2835 2837 2575 0 -1 4202560 4006294 1249 0 0 487995 1129 0 0 25 0 9 0 722411785 17537835008 4242869 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 253
[pid=2837/tid=2846] ppid=2835 vsize=17126792 CPUtime=4893.81 cores=0-7
/proc/2837/task/2846/stat : 2846 (plingeling) R 2835 2837 2575 0 -1 4202560 3352763 1249 0 0 488573 808 0 0 25 0 9 0 722411785 17537835008 4242869 33554432000 4194304 4417452 140736128838384 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 4 0 0 54
[pid=2837/tid=2851] ppid=2835 vsize=17126792 CPUtime=4887.53 cores=0-7
/proc/2837/task/2851/stat : 2851 (plingeling) R 2835 2837 2575 0 -1 4202560 1825135 1249 0 0 488027 726 0 0 25 0 9 0 722411785 17537835008 4242869 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 228
[pid=2837/tid=2856] ppid=2835 vsize=17126792 CPUtime=4874.44 cores=0-7
/proc/2837/task/2856/stat : 2856 (plingeling) R 2835 2837 2575 0 -1 4202560 3702056 1249 0 0 486687 757 0 0 25 0 9 0 722411785 17537835008 4242869 33554432000 4194304 4417452 140736128838384 18446744073709551615 4274781 0 0 4096 17442 0 0 0 -1 7 0 0 112
[pid=2837/tid=2857] ppid=2835 vsize=17126792 CPUtime=4872.12 cores=0-7
/proc/2837/task/2857/stat : 2857 (plingeling) R 2835 2837 2575 0 -1 4202560 3420916 1249 0 0 486310 902 0 0 25 0 9 0 722411785 17537835008 4242869 33554432000 4194304 4417452 140736128838384 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 5 0 0 176
[pid=2837/tid=2858] ppid=2835 vsize=17126792 CPUtime=4886.73 cores=0-7
/proc/2837/task/2858/stat : 2858 (plingeling) R 2835 2837 2575 0 -1 4202560 2989082 1249 0 0 487966 707 0 0 25 0 9 0 722411785 17537835008 4242869 33554432000 4194304 4417452 140736128838384 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 6 0 0 121
[pid=2837/tid=2859] ppid=2835 vsize=17126792 CPUtime=4880.88 cores=0-7
/proc/2837/task/2859/stat : 2859 (plingeling) R 2835 2837 2575 0 -1 4202560 3366907 1249 0 0 487132 956 0 0 25 0 9 0 722411785 17537835008 4242869 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 211
Current children cumulated CPU time (s) 39062.5
Current children cumulated vsize (KiB) 17126792

[startup+4962.3 s]
/proc/loadavg: 8.22 8.23 8.17 9/519 3024
/proc/meminfo: memFree=1117912/32950928 swapFree=38156884/67111528
[pid=2837] ppid=2835 vsize=16909460 CPUtime=39540 cores=0-7
/proc/2837/stat : 2837 (plingeling) S 2835 2837 2575 0 -1 4202496 26507624 1249 0 0 3947053 6943 0 0 25 0 9 0 722411779 17315287040 4188536 33554432000 4194304 4417452 140736128838384 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 4227365 4188536 180 55 0 4225251 0
[pid=2837/tid=2844] ppid=2835 vsize=16909460 CPUtime=4935.5 cores=0-7
/proc/2837/task/2844/stat : 2844 (plingeling) R 2835 2837 2575 0 -1 4202560 3541065 1249 0 0 492695 855 0 0 25 0 9 0 722411785 17315287040 4188536 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 56
[pid=2837/tid=2845] ppid=2835 vsize=16909460 CPUtime=4951.18 cores=0-7
/proc/2837/task/2845/stat : 2845 (plingeling) R 2835 2837 2575 0 -1 4202560 4006294 1249 0 0 493989 1129 0 0 25 0 9 0 722411785 17315287040 4188536 33554432000 4194304 4417452 140736128838384 18446744073709551615 4259896 0 0 4096 17442 0 0 0 -1 3 0 0 253
[pid=2837/tid=2846] ppid=2835 vsize=16909460 CPUtime=4953.8 cores=0-7
/proc/2837/task/2846/stat : 2846 (plingeling) R 2835 2837 2575 0 -1 4202560 3352763 1249 0 0 494572 808 0 0 25 0 9 0 722411785 17315287040 4188536 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 4 0 0 54
[pid=2837/tid=2851] ppid=2835 vsize=16909460 CPUtime=4947.55 cores=0-7
/proc/2837/task/2851/stat : 2851 (plingeling) R 2835 2837 2575 0 -1 4202560 1885143 1249 0 0 493979 776 0 0 25 0 9 0 722411785 17315287040 4188536 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 228
[pid=2837/tid=2856] ppid=2835 vsize=16909460 CPUtime=4933.84 cores=0-7
/proc/2837/task/2856/stat : 2856 (plingeling) R 2835 2837 2575 0 -1 4202560 3886599 1249 0 0 492589 795 0 0 25 0 9 0 722411785 17315287040 4188536 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 112
[pid=2837/tid=2857] ppid=2835 vsize=16909460 CPUtime=4930.92 cores=0-7
/proc/2837/task/2857/stat : 2857 (plingeling) R 2835 2837 2575 0 -1 4202560 3420916 1249 0 0 492190 902 0 0 25 0 9 0 722411785 17315287040 4188536 33554432000 4194304 4417452 140736128838384 18446744073709551615 4342442 0 0 4096 17442 0 0 0 -1 7 0 0 176
[pid=2837/tid=2858] ppid=2835 vsize=16909460 CPUtime=4946.43 cores=0-7
/proc/2837/task/2858/stat : 2858 (plingeling) R 2835 2837 2575 0 -1 4202560 3045137 1249 0 0 493925 718 0 0 25 0 9 0 722411785 17315287040 4188536 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 121
[pid=2837/tid=2859] ppid=2835 vsize=16909460 CPUtime=4940.71 cores=0-7
/proc/2837/task/2859/stat : 2859 (plingeling) R 2835 2837 2575 0 -1 4202560 3366907 1249 0 0 493115 956 0 0 25 0 9 0 722411785 17315287040 4188536 33554432000 4194304 4417452 140736128838384 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 5 0 0 211
Current children cumulated CPU time (s) 39540
Current children cumulated vsize (KiB) 16909460



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5020 s]
/proc/loadavg: 8.13 8.20 8.17 9/519 3026
/proc/meminfo: memFree=1014364/32950928 swapFree=38156884/67111528
[pid=2837] ppid=2835 vsize=17012736 CPUtime=40000.6 cores=0-7
/proc/2837/stat : 2837 (plingeling) S 2835 2837 2575 0 -1 4202496 26533443 1249 0 0 3993106 6951 0 0 25 0 9 0 722411779 17421041664 4214355 33554432000 4194304 4417452 140736128838384 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 4253184 4214355 180 55 0 4251070 0
[pid=2837/tid=2844] ppid=2835 vsize=17012736 CPUtime=4993.2 cores=0-7
/proc/2837/task/2844/stat : 2844 (plingeling) R 2835 2837 2575 0 -1 4202560 3541065 1249 0 0 498465 855 0 0 25 0 9 0 722411785 17421041664 4214355 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 56
[pid=2837/tid=2845] ppid=2835 vsize=17012736 CPUtime=5008.88 cores=0-7
/proc/2837/task/2845/stat : 2845 (plingeling) R 2835 2837 2575 0 -1 4202560 4006294 1249 0 0 499759 1129 0 0 25 0 9 0 722411785 17421041664 4214355 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 253
[pid=2837/tid=2846] ppid=2835 vsize=17012736 CPUtime=5011.49 cores=0-7
/proc/2837/task/2846/stat : 2846 (plingeling) R 2835 2837 2575 0 -1 4202560 3352763 1249 0 0 500341 808 0 0 25 0 9 0 722411785 17421041664 4214355 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 54
[pid=2837/tid=2851] ppid=2835 vsize=17012736 CPUtime=5005.25 cores=0-7
/proc/2837/task/2851/stat : 2851 (plingeling) R 2835 2837 2575 0 -1 4202560 1885143 1249 0 0 499749 776 0 0 25 0 9 0 722411785 17421041664 4214355 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 2 0 0 228
[pid=2837/tid=2856] ppid=2835 vsize=17012736 CPUtime=4991.55 cores=0-7
/proc/2837/task/2856/stat : 2856 (plingeling) R 2835 2837 2575 0 -1 4202560 3907872 1249 0 0 498353 802 0 0 25 0 9 0 722411785 17421041664 4214355 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 112
[pid=2837/tid=2857] ppid=2835 vsize=17012736 CPUtime=4987.61 cores=0-7
/proc/2837/task/2857/stat : 2857 (plingeling) R 2835 2837 2575 0 -1 4202560 3420916 1249 0 0 497859 902 0 0 25 0 9 0 722411785 17421041664 4214355 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 7 0 0 176
[pid=2837/tid=2858] ppid=2835 vsize=17012736 CPUtime=5004.13 cores=0-7
/proc/2837/task/2858/stat : 2858 (plingeling) R 2835 2837 2575 0 -1 4202560 3049683 1249 0 0 499694 719 0 0 25 0 9 0 722411785 17421041664 4214355 33554432000 4194304 4417452 140736128838384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 121
[pid=2837/tid=2859] ppid=2835 vsize=17012736 CPUtime=4998.41 cores=0-7
/proc/2837/task/2859/stat : 2859 (plingeling) R 2835 2837 2575 0 -1 4202560 3366907 1249 0 0 498885 956 0 0 25 0 9 0 722411785 17421041664 4214355 33554432000 4194304 4417452 140736128838384 18446744073709551615 4365075 0 0 4096 17442 0 0 0 -1 5 0 0 211
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 17012736

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5020.4 s]
/proc/loadavg: 8.13 8.20 8.17 9/519 3026
/proc/meminfo: memFree=1014364/32950928 swapFree=38156884/67111528
[pid=2837] ppid=2835 vsize=0 CPUtime=40001.1 cores=0-7
/proc/2837/stat : 2837 (plingeling) Z 2835 2837 2575 0 -1 4203548 26533443 1249 0 0 3993118 6988 0 0 18 0 2 0 722411779 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 0 0 0 0 0 0 0
[pid=2837/tid=2857] ppid=2835 vsize=0 CPUtime=4988 cores=0-7
/proc/2837/task/2857/stat : 2857 (plingeling) R 2835 2837 2575 0 -1 4203588 3420916 1249 0 0 497860 940 0 0 25 0 2 0 722411785 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 176
Current children cumulated CPU time (s) 40001.1
Current children cumulated vsize (KiB) 0

[startup+5020.6 s]
/proc/loadavg: 8.13 8.20 8.17 2/513 3027
/proc/meminfo: memFree=15231888/32950928 swapFree=38156884/67111528
[pid=2837] ppid=2835 vsize=0 CPUtime=40001.3 cores=0-7
/proc/2837/stat : 2837 (plingeling) Z 2835 2837 2575 0 -1 4203548 26533443 1249 0 0 3993118 7009 0 0 18 0 2 0 722411779 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/2837/statm: 0 0 0 0 0 0 0
[pid=2837/tid=2857] ppid=2835 vsize=0 CPUtime=4988.21 cores=0-7
/proc/2837/task/2857/stat : 2857 (plingeling) R 2835 2837 2575 0 -1 4203588 3420916 1249 0 0 497860 961 0 0 25 0 2 0 722411785 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 176
Current children cumulated CPU time (s) 40001.3
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5020.71
CPU time (s): 40001.4
CPU user time (s): 39931.2
CPU system time (s): 70.1963
CPU usage (%): 796.728
Max. virtual memory (cumulated for all children) (KiB): 17925224

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39931.2
system time used= 70.1963
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 26534692
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1182
involuntary context switches= 196166

runsolver used 24.3343 second user time and 63.0124 second system time

The end

Launcher Data 

Begin job on node149 at 2011-05-23 01:49:11
IDJOB=3394844
IDBENCH=85438
IDSOLVER=1851
FILE ID=node149/3394844-1306108151
RUNJOBID= node149-1306103137-2589
PBS_JOBID= 13388007
Free space on /tmp= 73412 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT05/crafted/sabharwal05/counting/php/unsat/easier/php-045-035.shuffled-as.sat05-1181.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394844-1306108151/watcher-3394844-1306108151 -o /tmp/evaluation-result-3394844-1306108151/solver-3394844-1306108151 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394844-1306108151.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 057b78ff24cc0773c9c67e770a7b9de2
RANDOM SEED=502151537

node149.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:       9312432 kB
Buffers:        217180 kB
Cached:        4997952 kB
SwapCached:      76012 kB
Active:       10366100 kB
Inactive:     12950156 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:       9312432 kB
SwapTotal:    67111528 kB
SwapFree:     44933860 kB
Dirty:             404 kB
Writeback:           0 kB
AnonPages:    18038220 kB
Mapped:          31460 kB
Slab:           129964 kB
PageTables:     116948 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 173080992 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73412 MiB
End job on node149 at 2011-05-23 03:12:53