Trace number 3394042

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40001.2 5008.35

General information on the benchmark

NameCRAFTED/rbsat/random/unforced/
rbsat-v2400c266431g10.cnf
MD5SUM885d1d8595262b8f0aa94d123211b47c
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2400
Number of clauses266431
Sum of the clauses size535112
Maximum clause length32
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2266356
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 575

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394042-1306080023/watcher-3394042-1306080023 -o /tmp/evaluation-result-3394042-1306080023/solver-3394042-1306080023 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394042-1306080023.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.36 7.89 7.96 2/366 11478
/proc/meminfo: memFree=23111512/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=9740 CPUtime=0 cores=0-7
/proc/11478/stat : 11478 (plingeling) R 11476 11478 11452 0 -1 4202496 494 1253 0 0 0 0 0 0 18 0 1 0 719602762 9973760 425 33554432000 4194304 4417452 140733323103376 18446744073709551615 4227901 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/11478/statm: 2435 425 147 55 0 321 0

[startup+0.0894649 s]
/proc/loadavg: 7.36 7.89 7.96 2/366 11478
/proc/meminfo: memFree=23111512/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=27796 CPUtime=0.07 cores=0-7
/proc/11478/stat : 11478 (plingeling) R 11476 11478 11452 0 -1 4202496 5025 1253 0 0 7 0 0 0 18 0 1 0 719602762 28463104 4956 33554432000 4194304 4417452 140733323103376 18446744073709551615 4259618 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/11478/statm: 6949 4956 148 55 0 4835 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 27796

[startup+0.100449 s]
/proc/loadavg: 7.36 7.89 7.96 2/366 11478
/proc/meminfo: memFree=23111512/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=27796 CPUtime=0.08 cores=0-7
/proc/11478/stat : 11478 (plingeling) R 11476 11478 11452 0 -1 4202496 5025 1253 0 0 8 0 0 0 18 0 1 0 719602762 28463104 4956 33554432000 4194304 4417452 140733323103376 18446744073709551615 4259767 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/11478/statm: 6949 4956 148 55 0 4835 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 27796

[startup+0.300412 s]
/proc/loadavg: 7.36 7.89 7.96 2/366 11478
/proc/meminfo: memFree=23111512/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=76948 CPUtime=0.29 cores=0-7
/proc/11478/stat : 11478 (plingeling) R 11476 11478 11452 0 -1 4202496 17313 1253 0 0 27 2 0 0 19 0 1 0 719602762 78794752 17244 33554432000 4194304 4417452 140733323103376 18446744073709551615 4229149 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/11478/statm: 19237 17244 148 55 0 17123 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 76948

[startup+0.988228 s]
/proc/loadavg: 7.36 7.89 7.96 2/366 11478
/proc/meminfo: memFree=23111512/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=319348 CPUtime=5.42 cores=0-7
/proc/11478/stat : 11478 (plingeling) S 11476 11478 11452 0 -1 4202496 38677 1253 0 0 536 6 0 0 18 0 9 0 719602762 327012352 24077 33554432000 4194304 4417452 140733323103376 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11478/statm: 79837 24107 180 55 0 77723 0
Current children cumulated CPU time (s) 5.42
Current children cumulated vsize (KiB) 319348

[startup+1.50116 s]
/proc/loadavg: 7.36 7.89 7.96 9/375 11492
/proc/meminfo: memFree=23007148/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=324364 CPUtime=9.5 cores=0-7
/proc/11478/stat : 11478 (plingeling) S 11476 11478 11452 0 -1 4202496 43213 1253 0 0 944 6 0 0 18 0 9 0 719602762 332148736 26255 33554432000 4194304 4417452 140733323103376 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11478/statm: 81091 26255 180 55 0 78977 0
[pid=11478/tid=11485] ppid=11476 vsize=324364 CPUtime=1.14 cores=0-7
/proc/11478/task/11485/stat : 11485 (plingeling) R 11476 11478 11452 0 -1 4202560 2431 1253 0 0 114 0 0 0 24 0 9 0 719602796 332148736 26255 33554432000 4194304 4417452 140733323103376 18446744073709551615 4258654 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11478/tid=11486] ppid=11476 vsize=324364 CPUtime=1.13 cores=0-7
/proc/11478/task/11486/stat : 11486 (plingeling) R 11476 11478 11452 0 -1 4202560 3452 1253 0 0 113 0 0 0 25 0 9 0 719602796 332148736 26255 33554432000 4194304 4417452 140733323103376 18446744073709551615 4250306 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11478/tid=11487] ppid=11476 vsize=324364 CPUtime=1.13 cores=0-7
/proc/11478/task/11487/stat : 11487 (plingeling) R 11476 11478 11452 0 -1 4202560 1504 1253 0 0 113 0 0 0 25 0 9 0 719602796 332148736 26255 33554432000 4194304 4417452 140733323103376 18446744073709551615 4235997 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11478/tid=11488] ppid=11476 vsize=324364 CPUtime=1.14 cores=0-7
/proc/11478/task/11488/stat : 11488 (plingeling) R 11476 11478 11452 0 -1 4202560 7003 1253 0 0 114 0 0 0 22 0 9 0 719602796 332148736 26255 33554432000 4194304 4417452 140733323103376 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11478/tid=11489] ppid=11476 vsize=324364 CPUtime=1.14 cores=0-7
/proc/11478/task/11489/stat : 11489 (plingeling) R 11476 11478 11452 0 -1 4202560 2884 1253 0 0 114 0 0 0 22 0 9 0 719602796 332148736 26258 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=11478/tid=11490] ppid=11476 vsize=324364 CPUtime=1.14 cores=0-7
/proc/11478/task/11490/stat : 11490 (plingeling) R 11476 11478 11452 0 -1 4202560 3458 1253 0 0 114 0 0 0 19 0 9 0 719602796 332148736 26271 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=11478/tid=11491] ppid=11476 vsize=324364 CPUtime=1.13 cores=0-7
/proc/11478/task/11491/stat : 11491 (plingeling) R 11476 11478 11452 0 -1 4202560 2908 1253 0 0 113 0 0 0 22 0 9 0 719602796 332148736 26285 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249943 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=11478/tid=11492] ppid=11476 vsize=324364 CPUtime=1.14 cores=0-7
/proc/11478/task/11492/stat : 11492 (plingeling) R 11476 11478 11452 0 -1 4202560 2244 1253 0 0 114 0 0 0 22 0 9 0 719602797 332148736 26302 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 9.5
Current children cumulated vsize (KiB) 324364

[startup+3.10087 s]
/proc/loadavg: 7.36 7.89 7.96 9/375 11492
/proc/meminfo: memFree=23004916/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=348412 CPUtime=22.29 cores=0-7
/proc/11478/stat : 11478 (plingeling) S 11476 11478 11452 0 -1 4202496 57384 1253 0 0 2221 8 0 0 18 0 9 0 719602762 356773888 35348 33554432000 4194304 4417452 140733323103376 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11478/statm: 87103 35348 180 55 0 84989 0
[pid=11478/tid=11485] ppid=11476 vsize=348412 CPUtime=2.74 cores=0-7
/proc/11478/task/11485/stat : 11485 (plingeling) R 11476 11478 11452 0 -1 4202560 3669 1253 0 0 274 0 0 0 25 0 9 0 719602796 356773888 35348 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11478/tid=11486] ppid=11476 vsize=348412 CPUtime=2.72 cores=0-7
/proc/11478/task/11486/stat : 11486 (plingeling) R 11476 11478 11452 0 -1 4202560 3452 1253 0 0 272 0 0 0 25 0 9 0 719602796 356773888 35348 33554432000 4194304 4417452 140733323103376 18446744073709551615 4236294 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11478/tid=11487] ppid=11476 vsize=348412 CPUtime=2.73 cores=0-7
/proc/11478/task/11487/stat : 11487 (plingeling) R 11476 11478 11452 0 -1 4202560 1504 1253 0 0 273 0 0 0 25 0 9 0 719602796 356773888 35348 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11478/tid=11488] ppid=11476 vsize=348412 CPUtime=2.75 cores=0-7
/proc/11478/task/11488/stat : 11488 (plingeling) R 11476 11478 11452 0 -1 4202560 18698 1253 0 0 273 2 0 0 25 0 9 0 719602796 356773888 35348 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11478/tid=11489] ppid=11476 vsize=348412 CPUtime=2.74 cores=0-7
/proc/11478/task/11489/stat : 11489 (plingeling) R 11476 11478 11452 0 -1 4202560 2884 1253 0 0 274 0 0 0 25 0 9 0 719602796 356773888 35348 33554432000 4194304 4417452 140733323103376 18446744073709551615 4235378 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=11478/tid=11490] ppid=11476 vsize=348412 CPUtime=2.74 cores=0-7
/proc/11478/task/11490/stat : 11490 (plingeling) R 11476 11478 11452 0 -1 4202560 4696 1253 0 0 274 0 0 0 22 0 9 0 719602796 356773888 35348 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249151 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=11478/tid=11491] ppid=11476 vsize=348412 CPUtime=2.73 cores=0-7
/proc/11478/task/11491/stat : 11491 (plingeling) R 11476 11478 11452 0 -1 4202560 2908 1253 0 0 273 0 0 0 25 0 9 0 719602796 356773888 35348 33554432000 4194304 4417452 140733323103376 18446744073709551615 4270927 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=11478/tid=11492] ppid=11476 vsize=348412 CPUtime=2.74 cores=0-7
/proc/11478/task/11492/stat : 11492 (plingeling) R 11476 11478 11452 0 -1 4202560 2244 1253 0 0 274 0 0 0 25 0 9 0 719602797 356773888 35348 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249186 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 22.29
Current children cumulated vsize (KiB) 348412

[startup+6.30128 s]
/proc/loadavg: 7.41 7.90 7.96 9/375 11492
/proc/meminfo: memFree=22971812/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=365000 CPUtime=47.85 cores=0-7
/proc/11478/stat : 11478 (plingeling) S 11476 11478 11452 0 -1 4202496 73333 1253 0 0 4775 10 0 0 18 0 9 0 719602762 373760000 36995 33554432000 4194304 4417452 140733323103376 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11478/statm: 91250 36995 180 55 0 89136 0
[pid=11478/tid=11485] ppid=11476 vsize=365000 CPUtime=5.94 cores=0-7
/proc/11478/task/11485/stat : 11485 (plingeling) R 11476 11478 11452 0 -1 4202560 3669 1253 0 0 594 0 0 0 25 0 9 0 719602796 373760000 36995 33554432000 4194304 4417452 140733323103376 18446744073709551615 4258587 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11478/tid=11486] ppid=11476 vsize=365000 CPUtime=5.88 cores=0-7
/proc/11478/task/11486/stat : 11486 (plingeling) R 11476 11478 11452 0 -1 4202560 3452 1253 0 0 588 0 0 0 25 0 9 0 719602796 373760000 36995 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11478/tid=11487] ppid=11476 vsize=365000 CPUtime=5.93 cores=0-7
/proc/11478/task/11487/stat : 11487 (plingeling) R 11476 11478 11452 0 -1 4202560 2706 1253 0 0 593 0 0 0 25 0 9 0 719602796 373760000 36995 33554432000 4194304 4417452 140733323103376 18446744073709551615 4366412 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11478/tid=11488] ppid=11476 vsize=365000 CPUtime=5.95 cores=0-7
/proc/11478/task/11488/stat : 11488 (plingeling) R 11476 11478 11452 0 -1 4202560 33204 1253 0 0 591 4 0 0 25 0 9 0 719602796 373760000 36995 33554432000 4194304 4417452 140733323103376 18446744073709551615 4320831 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11478/tid=11489] ppid=11476 vsize=365000 CPUtime=5.94 cores=0-7
/proc/11478/task/11489/stat : 11489 (plingeling) R 11476 11478 11452 0 -1 4202560 2884 1253 0 0 594 0 0 0 25 0 9 0 719602796 373760000 36995 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=11478/tid=11490] ppid=11476 vsize=365000 CPUtime=5.94 cores=0-7
/proc/11478/task/11490/stat : 11490 (plingeling) R 11476 11478 11452 0 -1 4202560 4696 1253 0 0 594 0 0 0 25 0 9 0 719602796 373760000 36995 33554432000 4194304 4417452 140733323103376 18446744073709551615 4258247 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=11478/tid=11491] ppid=11476 vsize=365000 CPUtime=5.93 cores=0-7
/proc/11478/task/11491/stat : 11491 (plingeling) R 11476 11478 11452 0 -1 4202560 3149 1253 0 0 593 0 0 0 25 0 9 0 719602796 373760000 36995 33554432000 4194304 4417452 140733323103376 18446744073709551615 4250869 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=11478/tid=11492] ppid=11476 vsize=365000 CPUtime=5.94 cores=0-7
/proc/11478/task/11492/stat : 11492 (plingeling) R 11476 11478 11452 0 -1 4202560 2244 1253 0 0 594 0 0 0 25 0 9 0 719602797 373760000 36995 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249186 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 47.85
Current children cumulated vsize (KiB) 365000

[startup+12.7011 s]
/proc/loadavg: 7.46 7.90 7.96 9/375 11493
/proc/meminfo: memFree=22932588/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=403384 CPUtime=98.98 cores=0-7
/proc/11478/stat : 11478 (plingeling) S 11476 11478 11452 0 -1 4202496 112614 1253 0 0 9881 17 0 0 18 0 9 0 719602762 413065216 47231 33554432000 4194304 4417452 140733323103376 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11478/statm: 100846 47231 180 55 0 98732 0
[pid=11478/tid=11485] ppid=11476 vsize=403384 CPUtime=12.33 cores=0-7
/proc/11478/task/11485/stat : 11485 (plingeling) R 11476 11478 11452 0 -1 4202560 4907 1253 0 0 1233 0 0 0 25 0 9 0 719602796 413065216 47231 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249186 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11478/tid=11486] ppid=11476 vsize=403384 CPUtime=12.2 cores=0-7
/proc/11478/task/11486/stat : 11486 (plingeling) R 11476 11478 11452 0 -1 4202560 3452 1253 0 0 1220 0 0 0 25 0 9 0 719602796 413065216 47231 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11478/tid=11487] ppid=11476 vsize=403384 CPUtime=12.33 cores=0-7
/proc/11478/task/11487/stat : 11487 (plingeling) R 11476 11478 11452 0 -1 4202560 2706 1253 0 0 1233 0 0 0 25 0 9 0 719602796 413065216 47231 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11478/tid=11488] ppid=11476 vsize=403384 CPUtime=12.35 cores=0-7
/proc/11478/task/11488/stat : 11488 (plingeling) R 11476 11478 11452 0 -1 4202560 65057 1253 0 0 1226 9 0 0 25 0 9 0 719602796 413065216 47231 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364850 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11478/tid=11489] ppid=11476 vsize=403384 CPUtime=12.33 cores=0-7
/proc/11478/task/11489/stat : 11489 (plingeling) R 11476 11478 11452 0 -1 4202560 5360 1253 0 0 1233 0 0 0 25 0 9 0 719602796 413065216 47231 33554432000 4194304 4417452 140733323103376 18446744073709551615 4235411 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=11478/tid=11490] ppid=11476 vsize=403384 CPUtime=12.35 cores=0-7
/proc/11478/task/11490/stat : 11490 (plingeling) R 11476 11478 11452 0 -1 4202560 5934 1253 0 0 1234 1 0 0 25 0 9 0 719602796 413065216 47231 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249489 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=11478/tid=11491] ppid=11476 vsize=403384 CPUtime=12.33 cores=0-7
/proc/11478/task/11491/stat : 11491 (plingeling) R 11476 11478 11452 0 -1 4202560 3149 1253 0 0 1233 0 0 0 25 0 9 0 719602796 413065216 47231 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=11478/tid=11492] ppid=11476 vsize=403384 CPUtime=12.34 cores=0-7
/proc/11478/task/11492/stat : 11492 (plingeling) R 11476 11478 11452 0 -1 4202560 4720 1253 0 0 1234 0 0 0 25 0 9 0 719602797 413065216 47231 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 98.98
Current children cumulated vsize (KiB) 403384

[startup+25.5138 s]
/proc/loadavg: 7.58 7.90 7.96 9/375 11493
/proc/meminfo: memFree=22848172/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=476608 CPUtime=201.32 cores=0-7
/proc/11478/stat : 11478 (plingeling) S 11476 11478 11452 0 -1 4202496 166975 1253 0 0 20108 24 0 0 18 0 9 0 719602762 488046592 66008 33554432000 4194304 4417452 140733323103376 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11478/statm: 119152 66008 180 55 0 117038 0
[pid=11478/tid=11485] ppid=11476 vsize=476608 CPUtime=25.15 cores=0-7
/proc/11478/task/11485/stat : 11485 (plingeling) R 11476 11478 11452 0 -1 4202560 5931 1253 0 0 2515 0 0 0 25 0 9 0 719602796 488046592 66008 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364885 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11478/tid=11486] ppid=11476 vsize=476608 CPUtime=24.85 cores=0-7
/proc/11478/task/11486/stat : 11486 (plingeling) R 11476 11478 11452 0 -1 4202560 4500 1253 0 0 2485 0 0 0 25 0 9 0 719602796 488046592 66008 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11478/tid=11487] ppid=11476 vsize=476608 CPUtime=25.14 cores=0-7
/proc/11478/task/11487/stat : 11487 (plingeling) R 11476 11478 11452 0 -1 4202560 7239 1253 0 0 2514 0 0 0 25 0 9 0 719602796 488046592 66008 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11478/tid=11488] ppid=11476 vsize=476608 CPUtime=25.16 cores=0-7
/proc/11478/task/11488/stat : 11488 (plingeling) R 11476 11478 11452 0 -1 4202560 108029 1253 0 0 2501 15 0 0 25 0 9 0 719602796 488046592 66008 33554432000 4194304 4417452 140733323103376 18446744073709551615 4321595 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11478/tid=11489] ppid=11476 vsize=476608 CPUtime=25.14 cores=0-7
/proc/11478/task/11489/stat : 11489 (plingeling) R 11476 11478 11452 0 -1 4202560 6385 1253 0 0 2514 0 0 0 25 0 9 0 719602796 488046592 66008 33554432000 4194304 4417452 140733323103376 18446744073709551615 4355852 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=11478/tid=11490] ppid=11476 vsize=476608 CPUtime=25.16 cores=0-7
/proc/11478/task/11490/stat : 11490 (plingeling) R 11476 11478 11452 0 -1 4202560 5934 1253 0 0 2515 1 0 0 25 0 9 0 719602796 488046592 66008 33554432000 4194304 4417452 140733323103376 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=11478/tid=11491] ppid=11476 vsize=476608 CPUtime=25.14 cores=0-7
/proc/11478/task/11491/stat : 11491 (plingeling) R 11476 11478 11452 0 -1 4202560 6593 1253 0 0 2514 0 0 0 25 0 9 0 719602796 488046592 66008 33554432000 4194304 4417452 140733323103376 18446744073709551615 4249151 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=11478/tid=11492] ppid=11476 vsize=476608 CPUtime=25.15 cores=0-7
/proc/11478/task/11492/stat : 11492 (plingeling) R 11476 11478 11452 0 -1 4202560 5035 1253 0 0 2515 0 0 0 25 0 9 0 719602797 488046592 66008 33554432000 4194304 4417452 140733323103376 18446744073709551615 4260039 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 201.32
Current children cumulated vsize (KiB) 476608

[startup+51.1011 s]
/proc/loadavg: 7.72 7.91 7.96 9/375 11503
/proc/meminfo: memFree=22677880/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=650252 CPUtime=405.67 cores=0-7
/proc/11478/stat : 11478 (plingeling) S 11476 11478 11452 0 -1 4202496 250564 1253 0 0 40528 39 0 0 18 0 9 0 719602762 665858048 108569 33554432000 4194304 4417452 140733323103376 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11478/statm: 162563 108569 180 55 0 160449 0
[pid=11478/tid=11485] ppid=11476 vsize=650252 CPUtime=50.73 cores=0-7
/proc/11478/task/11485/stat : 11485 (plingeling) R 11476 11478 11452 0 -1 4202560 6930 1253 0 0 5072 1 0 0 25 0 9 0 719602796 665858048 108569 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11478/tid=11486] ppid=11476 vsize=650252 CPUtime=50.14 cores=0-7
/proc/11478/task/11486/stat : 11486 (plingeling) R 11476 11478 11452 0 -1 4202560 5224 1253 0 0 5014 0 0 0 25 0 9 0 719602796 665858048 108569 33554432000 4194304 4417452 140733323103376 18446744073709551615 4278151 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11478/tid=11487] ppid=11476 vsize=650252 CPUtime=50.73 cores=0-7

################
# More data... #
################

[pid=11478/tid=11487] ppid=11476 vsize=6408892 CPUtime=4722.03 cores=0-7
/proc/11478/task/11487/stat : 11487 (plingeling) R 11476 11478 11452 0 -1 4202560 532782 1253 0 0 472124 79 0 0 25 0 9 0 719602796 6562705408 1547116 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11478/tid=11488] ppid=11476 vsize=6408892 CPUtime=4722.09 cores=0-7
/proc/11478/task/11488/stat : 11488 (plingeling) R 11476 11478 11452 0 -1 4202560 2555125 1253 0 0 471668 541 0 0 25 0 9 0 719602796 6562705408 1547116 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11478/tid=11489] ppid=11476 vsize=6408892 CPUtime=4720.91 cores=0-7
/proc/11478/task/11489/stat : 11489 (plingeling) R 11476 11478 11452 0 -1 4202560 589425 1253 0 0 472006 85 0 0 25 0 9 0 719602796 6562705408 1547116 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=11478/tid=11490] ppid=11476 vsize=6408892 CPUtime=4720.51 cores=0-7
/proc/11478/task/11490/stat : 11490 (plingeling) R 11476 11478 11452 0 -1 4202560 583712 1253 0 0 471969 82 0 0 25 0 9 0 719602796 6562705408 1547116 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=11478/tid=11491] ppid=11476 vsize=6408892 CPUtime=4721.89 cores=0-7
/proc/11478/task/11491/stat : 11491 (plingeling) R 11476 11478 11452 0 -1 4202560 230548 1253 0 0 472156 33 0 0 25 0 9 0 719602796 6562705408 1547116 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=11478/tid=11492] ppid=11476 vsize=6408892 CPUtime=4722.01 cores=0-7
/proc/11478/task/11492/stat : 11492 (plingeling) R 11476 11478 11452 0 -1 4202560 682138 1253 0 0 472080 121 0 0 25 0 9 0 719602797 6562705408 1547116 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37717.8
Current children cumulated vsize (KiB) 6408892

[startup+4782.3 s]
/proc/loadavg: 8.09 8.06 8.00 9/375 11644
/proc/meminfo: memFree=16775564/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=6545356 CPUtime=38197.1 cores=0-7
/proc/11478/stat : 11478 (plingeling) S 11476 11478 11452 0 -1 4202496 6351996 1253 0 0 3818557 1154 0 0 18 0 9 0 719602762 6702444544 1581232 33554432000 4194304 4417452 140733323103376 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11478/statm: 1636339 1581232 180 55 0 1634225 0
[pid=11478/tid=11485] ppid=11476 vsize=6545356 CPUtime=4782.04 cores=0-7
/proc/11478/task/11485/stat : 11485 (plingeling) R 11476 11478 11452 0 -1 4202560 482325 1253 0 0 478109 95 0 0 25 0 9 0 719602796 6702444544 1581232 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11478/tid=11486] ppid=11476 vsize=6545356 CPUtime=4725.21 cores=0-7
/proc/11478/task/11486/stat : 11486 (plingeling) R 11476 11478 11452 0 -1 4202560 648539 1253 0 0 472415 106 0 0 25 0 9 0 719602796 6702444544 1581232 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11478/tid=11487] ppid=11476 vsize=6545356 CPUtime=4782.03 cores=0-7
/proc/11478/task/11487/stat : 11487 (plingeling) R 11476 11478 11452 0 -1 4202560 549550 1253 0 0 478120 83 0 0 25 0 9 0 719602796 6702444544 1581232 33554432000 4194304 4417452 140733323103376 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11478/tid=11488] ppid=11476 vsize=6545356 CPUtime=4782.09 cores=0-7
/proc/11478/task/11488/stat : 11488 (plingeling) R 11476 11478 11452 0 -1 4202560 2555125 1253 0 0 477668 541 0 0 25 0 9 0 719602796 6702444544 1581232 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11478/tid=11489] ppid=11476 vsize=6545356 CPUtime=4780.9 cores=0-7
/proc/11478/task/11489/stat : 11489 (plingeling) R 11476 11478 11452 0 -1 4202560 597482 1253 0 0 478004 86 0 0 25 0 9 0 719602796 6702444544 1581232 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=11478/tid=11490] ppid=11476 vsize=6545356 CPUtime=4780.5 cores=0-7
/proc/11478/task/11490/stat : 11490 (plingeling) R 11476 11478 11452 0 -1 4202560 588960 1253 0 0 477968 82 0 0 25 0 9 0 719602796 6702444544 1581232 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=11478/tid=11491] ppid=11476 vsize=6545356 CPUtime=4781.9 cores=0-7
/proc/11478/task/11491/stat : 11491 (plingeling) R 11476 11478 11452 0 -1 4202560 230548 1253 0 0 478157 33 0 0 25 0 9 0 719602796 6702444544 1581232 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=11478/tid=11492] ppid=11476 vsize=6545356 CPUtime=4782.01 cores=0-7
/proc/11478/task/11492/stat : 11492 (plingeling) R 11476 11478 11452 0 -1 4202560 682138 1253 0 0 478080 121 0 0 25 0 9 0 719602797 6702444544 1581232 33554432000 4194304 4417452 140733323103376 18446744073709551615 4344164 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38197.1
Current children cumulated vsize (KiB) 6545356

[startup+4842.3 s]
/proc/loadavg: 8.09 8.06 8.00 9/375 11645
/proc/meminfo: memFree=16738224/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=6582532 CPUtime=38676.5 cores=0-7
/proc/11478/stat : 11478 (plingeling) S 11476 11478 11452 0 -1 4202496 6475095 1253 0 0 3866472 1174 0 0 18 0 9 0 719602762 6740512768 1590526 33554432000 4194304 4417452 140733323103376 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11478/statm: 1645633 1590526 180 55 0 1643519 0
[pid=11478/tid=11485] ppid=11476 vsize=6582532 CPUtime=4842.06 cores=0-7
/proc/11478/task/11485/stat : 11485 (plingeling) R 11476 11478 11452 0 -1 4202560 482325 1253 0 0 484111 95 0 0 25 0 9 0 719602796 6740512768 1590526 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364607 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11478/tid=11486] ppid=11476 vsize=6582532 CPUtime=4784.5 cores=0-7
/proc/11478/task/11486/stat : 11486 (plingeling) R 11476 11478 11452 0 -1 4202560 664892 1253 0 0 478341 109 0 0 25 0 9 0 719602796 6740512768 1590526 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364614 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11478/tid=11487] ppid=11476 vsize=6582532 CPUtime=4842.05 cores=0-7
/proc/11478/task/11487/stat : 11487 (plingeling) R 11476 11478 11452 0 -1 4202560 552910 1253 0 0 484121 84 0 0 25 0 9 0 719602796 6740512768 1590526 33554432000 4194304 4417452 140733323103376 18446744073709551615 4353546 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11478/tid=11488] ppid=11476 vsize=6582532 CPUtime=4842.11 cores=0-7
/proc/11478/task/11488/stat : 11488 (plingeling) R 11476 11478 11452 0 -1 4202560 2555125 1253 0 0 483670 541 0 0 25 0 9 0 719602796 6740512768 1590526 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11478/tid=11489] ppid=11476 vsize=6582532 CPUtime=4840.9 cores=0-7
/proc/11478/task/11489/stat : 11489 (plingeling) R 11476 11478 11452 0 -1 4202560 616083 1253 0 0 484002 88 0 0 25 0 9 0 719602796 6740512768 1590526 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=11478/tid=11490] ppid=11476 vsize=6582532 CPUtime=4840.52 cores=0-7
/proc/11478/task/11490/stat : 11490 (plingeling) R 11476 11478 11452 0 -1 4202560 602873 1253 0 0 483967 85 0 0 25 0 9 0 719602796 6740512768 1590526 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364958 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=11478/tid=11491] ppid=11476 vsize=6582532 CPUtime=4841.91 cores=0-7
/proc/11478/task/11491/stat : 11491 (plingeling) R 11476 11478 11452 0 -1 4202560 230548 1253 0 0 484158 33 0 0 25 0 9 0 719602796 6740512768 1590526 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=11478/tid=11492] ppid=11476 vsize=6582532 CPUtime=4842.03 cores=0-7
/proc/11478/task/11492/stat : 11492 (plingeling) R 11476 11478 11452 0 -1 4202560 753010 1253 0 0 484070 133 0 0 25 0 9 0 719602797 6740512768 1590526 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38676.5
Current children cumulated vsize (KiB) 6582532

[startup+4902.3 s]
/proc/loadavg: 8.08 8.06 8.00 9/375 11647
/proc/meminfo: memFree=16716636/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=6604104 CPUtime=39155.7 cores=0-7
/proc/11478/stat : 11478 (plingeling) S 11476 11478 11452 0 -1 4202496 6510424 1253 0 0 3914388 1179 0 0 18 0 9 0 719602762 6762602496 1595919 33554432000 4194304 4417452 140733323103376 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11478/statm: 1651026 1595919 180 55 0 1648912 0
[pid=11478/tid=11485] ppid=11476 vsize=6604104 CPUtime=4902.05 cores=0-7
/proc/11478/task/11485/stat : 11485 (plingeling) R 11476 11478 11452 0 -1 4202560 482325 1253 0 0 490110 95 0 0 25 0 9 0 719602796 6762602496 1595919 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11478/tid=11486] ppid=11476 vsize=6604104 CPUtime=4843.78 cores=0-7
/proc/11478/task/11486/stat : 11486 (plingeling) R 11476 11478 11452 0 -1 4202560 671676 1253 0 0 484268 110 0 0 25 0 9 0 719602796 6762602496 1595919 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11478/tid=11487] ppid=11476 vsize=6604104 CPUtime=4902.04 cores=0-7
/proc/11478/task/11487/stat : 11487 (plingeling) R 11476 11478 11452 0 -1 4202560 569615 1253 0 0 490118 86 0 0 25 0 9 0 719602796 6762602496 1595919 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11478/tid=11488] ppid=11476 vsize=6604104 CPUtime=4902.1 cores=0-7
/proc/11478/task/11488/stat : 11488 (plingeling) R 11476 11478 11452 0 -1 4202560 2555125 1253 0 0 489669 541 0 0 25 0 9 0 719602796 6762602496 1595919 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11478/tid=11489] ppid=11476 vsize=6604104 CPUtime=4900.87 cores=0-7
/proc/11478/task/11489/stat : 11489 (plingeling) R 11476 11478 11452 0 -1 4202560 616083 1253 0 0 489999 88 0 0 25 0 9 0 719602796 6762602496 1595919 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=11478/tid=11490] ppid=11476 vsize=6604104 CPUtime=4900.51 cores=0-7
/proc/11478/task/11490/stat : 11490 (plingeling) R 11476 11478 11452 0 -1 4202560 614713 1253 0 0 489965 86 0 0 25 0 9 0 719602796 6762602496 1595919 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=11478/tid=11491] ppid=11476 vsize=6604104 CPUtime=4901.9 cores=0-7
/proc/11478/task/11491/stat : 11491 (plingeling) R 11476 11478 11452 0 -1 4202560 230548 1253 0 0 490157 33 0 0 25 0 9 0 719602796 6762602496 1595919 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=11478/tid=11492] ppid=11476 vsize=6604104 CPUtime=4902.01 cores=0-7
/proc/11478/task/11492/stat : 11492 (plingeling) R 11476 11478 11452 0 -1 4202560 753010 1253 0 0 490068 133 0 0 25 0 9 0 719602797 6762602496 1595919 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39155.7
Current children cumulated vsize (KiB) 6604104

[startup+4962.31 s]
/proc/loadavg: 8.08 8.06 8.00 9/375 11649
/proc/meminfo: memFree=16511396/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=6808640 CPUtime=39635 cores=0-7
/proc/11478/stat : 11478 (plingeling) S 11476 11478 11452 0 -1 4202496 6587874 1253 0 0 3962310 1193 0 0 18 0 9 0 719602762 6972047360 1647053 33554432000 4194304 4417452 140733323103376 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11478/statm: 1702160 1647053 180 55 0 1700046 0
[pid=11478/tid=11485] ppid=11476 vsize=6808640 CPUtime=4962.06 cores=0-7
/proc/11478/task/11485/stat : 11485 (plingeling) R 11476 11478 11452 0 -1 4202560 482325 1253 0 0 496111 95 0 0 25 0 9 0 719602796 6972047360 1647053 33554432000 4194304 4417452 140733323103376 18446744073709551615 4263247 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11478/tid=11486] ppid=11476 vsize=6808640 CPUtime=4903.07 cores=0-7
/proc/11478/task/11486/stat : 11486 (plingeling) R 11476 11478 11452 0 -1 4202560 671676 1253 0 0 490197 110 0 0 25 0 9 0 719602796 6972047360 1647053 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11478/tid=11487] ppid=11476 vsize=6808640 CPUtime=4962.05 cores=0-7
/proc/11478/task/11487/stat : 11487 (plingeling) R 11476 11478 11452 0 -1 4202560 575090 1253 0 0 496119 86 0 0 25 0 9 0 719602796 6972047360 1647053 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11478/tid=11488] ppid=11476 vsize=6808640 CPUtime=4962.12 cores=0-7
/proc/11478/task/11488/stat : 11488 (plingeling) R 11476 11478 11452 0 -1 4202560 2569281 1253 0 0 495666 546 0 0 25 0 9 0 719602796 6972047360 1647053 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11478/tid=11489] ppid=11476 vsize=6808640 CPUtime=4960.88 cores=0-7
/proc/11478/task/11489/stat : 11489 (plingeling) R 11476 11478 11452 0 -1 4202560 664218 1253 0 0 495993 95 0 0 25 0 9 0 719602796 6972047360 1647053 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=11478/tid=11490] ppid=11476 vsize=6808640 CPUtime=4960.53 cores=0-7
/proc/11478/task/11490/stat : 11490 (plingeling) R 11476 11478 11452 0 -1 4202560 624397 1253 0 0 495965 88 0 0 25 0 9 0 719602796 6972047360 1647053 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=11478/tid=11491] ppid=11476 vsize=6808640 CPUtime=4961.88 cores=0-7
/proc/11478/task/11491/stat : 11491 (plingeling) R 11476 11478 11452 0 -1 4202560 230548 1253 0 0 496155 33 0 0 25 0 9 0 719602796 6972047360 1647053 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=11478/tid=11492] ppid=11476 vsize=6808640 CPUtime=4962.03 cores=0-7
/proc/11478/task/11492/stat : 11492 (plingeling) R 11476 11478 11452 0 -1 4202560 753010 1253 0 0 496070 133 0 0 25 0 9 0 719602797 6972047360 1647053 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39635
Current children cumulated vsize (KiB) 6808640



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5008.1 s]
/proc/loadavg: 8.15 8.08 8.01 9/375 11650
/proc/meminfo: memFree=16377840/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=6941984 CPUtime=40000.9 cores=0-7
/proc/11478/stat : 11478 (plingeling) S 11476 11478 11452 0 -1 4202496 6625545 1253 0 0 3998884 1201 0 0 18 0 9 0 719602762 7108591616 1680389 33554432000 4194304 4417452 140733323103376 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11478/statm: 1735496 1680389 180 55 0 1733382 0
[pid=11478/tid=11485] ppid=11476 vsize=6941984 CPUtime=5007.86 cores=0-7
/proc/11478/task/11485/stat : 11485 (plingeling) R 11476 11478 11452 0 -1 4202560 482325 1253 0 0 500691 95 0 0 25 0 9 0 719602796 7108591616 1680389 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11478/tid=11486] ppid=11476 vsize=6941984 CPUtime=4948.32 cores=0-7
/proc/11478/task/11486/stat : 11486 (plingeling) R 11476 11478 11452 0 -1 4202560 685244 1253 0 0 494719 113 0 0 25 0 9 0 719602796 7108591616 1680389 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11478/tid=11487] ppid=11476 vsize=6941984 CPUtime=5007.85 cores=0-7
/proc/11478/task/11487/stat : 11487 (plingeling) R 11476 11478 11452 0 -1 4202560 585544 1253 0 0 500697 88 0 0 25 0 9 0 719602796 7108591616 1680389 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11478/tid=11488] ppid=11476 vsize=6941984 CPUtime=5007.92 cores=0-7
/proc/11478/task/11488/stat : 11488 (plingeling) R 11476 11478 11452 0 -1 4202560 2569281 1253 0 0 500246 546 0 0 25 0 9 0 719602796 7108591616 1680389 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11478/tid=11489] ppid=11476 vsize=6941984 CPUtime=5006.67 cores=0-7
/proc/11478/task/11489/stat : 11489 (plingeling) R 11476 11478 11452 0 -1 4202560 677867 1253 0 0 500568 99 0 0 25 0 9 0 719602796 7108591616 1680389 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=11478/tid=11490] ppid=11476 vsize=6941984 CPUtime=5006.33 cores=0-7
/proc/11478/task/11490/stat : 11490 (plingeling) R 11476 11478 11452 0 -1 4202560 624397 1253 0 0 500545 88 0 0 25 0 9 0 719602796 7108591616 1680389 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=11478/tid=11491] ppid=11476 vsize=6941984 CPUtime=5007.68 cores=0-7
/proc/11478/task/11491/stat : 11491 (plingeling) R 11476 11478 11452 0 -1 4202560 230548 1253 0 0 500735 33 0 0 25 0 9 0 719602796 7108591616 1680389 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=11478/tid=11492] ppid=11476 vsize=6941984 CPUtime=5007.83 cores=0-7
/proc/11478/task/11492/stat : 11492 (plingeling) R 11476 11478 11452 0 -1 4202560 753010 1253 0 0 500650 133 0 0 25 0 9 0 719602797 7108591616 1680389 33554432000 4194304 4417452 140733323103376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 6941984

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5008.22 s]
/proc/loadavg: 8.15 8.08 8.01 9/375 11650
/proc/meminfo: memFree=16377840/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=0 CPUtime=40001 cores=0-7
/proc/11478/stat : 11478 (plingeling) Z 11476 11478 11452 0 -1 4203548 6625545 1253 0 0 3998890 1211 0 0 18 0 2 0 719602762 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 7 0 0 0
/proc/11478/statm: 0 0 0 0 0 0 0
[pid=11478/tid=11486] ppid=11476 vsize=0 CPUtime=4948.42 cores=0-7
/proc/11478/task/11486/stat : 11486 (plingeling) R 11476 11478 11452 0 -1 4203588 685244 1253 0 0 494719 123 0 0 25 0 2 0 719602796 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40001
Current children cumulated vsize (KiB) 0

[startup+5008.3 s]
/proc/loadavg: 8.15 8.08 8.01 9/375 11650
/proc/meminfo: memFree=16377840/32951124 swapFree=37160472/67111528
[pid=11478] ppid=11476 vsize=0 CPUtime=40001.1 cores=0-7
/proc/11478/stat : 11478 (plingeling) Z 11476 11478 11452 0 -1 4203548 6625545 1253 0 0 3998890 1219 0 0 18 0 2 0 719602762 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 7 0 0 0
/proc/11478/statm: 0 0 0 0 0 0 0
[pid=11478/tid=11486] ppid=11476 vsize=0 CPUtime=4948.51 cores=0-7
/proc/11478/task/11486/stat : 11486 (plingeling) R 11476 11478 11452 0 -1 4203588 685244 1253 0 0 494719 132 0 0 25 0 2 0 719602796 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40001.1
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5008.35
CPU time (s): 40001.2
CPU user time (s): 39988.9
CPU system time (s): 12.2501
CPU usage (%): 798.689
Max. virtual memory (cumulated for all children) (KiB): 6941984

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39988.9
system time used= 12.2501
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6626798
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 893
involuntary context switches= 194680

runsolver used 17.0674 second user time and 42.4995 second system time

The end

Launcher Data 

Begin job on node104 at 2011-05-22 18:00:23
IDJOB=3394042
IDBENCH=71270
IDSOLVER=1851
FILE ID=node104/3394042-1306080023
RUNJOBID= node104-1306080023-11466
PBS_JOBID= 13387914
Free space on /tmp= 73236 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/random/unforced/rbsat-v2400c266431g10.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394042-1306080023/watcher-3394042-1306080023 -o /tmp/evaluation-result-3394042-1306080023/solver-3394042-1306080023 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394042-1306080023.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 885d1d8595262b8f0aa94d123211b47c
RANDOM SEED=1931559039

node104.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5293.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23111792 kB
Buffers:        338252 kB
Cached:        7421716 kB
SwapCached:     152248 kB
Active:        3217300 kB
Inactive:      6316276 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23111792 kB
SwapTotal:    67111528 kB
SwapFree:     37160472 kB
Dirty:            3500 kB
Writeback:           4 kB
AnonPages:     1634936 kB
Mapped:          10056 kB
Slab:           149368 kB
PageTables:      89432 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 87700284 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73236 MiB
End job on node104 at 2011-05-22 19:23:53