Trace number 3394442

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40001.1 5012.38

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-21x33.cnf
MD5SUMf7c29bbc3bef00d1e7cc799553ba4813
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2772
Number of clauses448371
Sum of the clauses size1785168
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 24158
Number of clauses of size 30
Number of clauses of size 4444213
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394442-1306096173/watcher-3394442-1306096173 -o /tmp/evaluation-result-3394442-1306096173/solver-3394442-1306096173 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394442-1306096173.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.15 8.12 8.03 2/514 18811
/proc/meminfo: memFree=9081448/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=12024 CPUtime=0 cores=0-7
/proc/18811/stat : 18811 (plingeling) R 18809 18811 18555 0 -1 4202496 1083 1251 0 0 0 0 0 0 21 0 1 0 702032804 12312576 1013 33554432000 4194304 4417452 140735488037504 18446744073709551615 228095450880 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/18811/statm: 3006 1013 147 55 0 892 0

[startup+0.0936521 s]
/proc/loadavg: 8.15 8.12 8.03 2/514 18811
/proc/meminfo: memFree=9081448/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=36184 CPUtime=0.08 cores=0-7
/proc/18811/stat : 18811 (plingeling) R 18809 18811 18555 0 -1 4202496 7348 1251 0 0 8 0 0 0 21 0 1 0 702032804 37052416 7070 33554432000 4194304 4417452 140735488037504 18446744073709551615 4320811 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/18811/statm: 9046 7070 148 55 0 6932 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 36184

[startup+0.10065 s]
/proc/loadavg: 8.15 8.12 8.03 2/514 18811
/proc/meminfo: memFree=9081448/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=36184 CPUtime=0.08 cores=0-7
/proc/18811/stat : 18811 (plingeling) R 18809 18811 18555 0 -1 4202496 7348 1251 0 0 8 0 0 0 21 0 1 0 702032804 37052416 7070 33554432000 4194304 4417452 140735488037504 18446744073709551615 4231335 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/18811/statm: 9046 7070 148 55 0 6932 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 36184

[startup+0.300613 s]
/proc/loadavg: 8.15 8.12 8.03 2/514 18811
/proc/meminfo: memFree=9081448/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=77144 CPUtime=0.28 cores=0-7
/proc/18811/stat : 18811 (plingeling) R 18809 18811 18555 0 -1 4202496 17588 1251 0 0 27 1 0 0 22 0 1 0 702032804 78995456 17310 33554432000 4194304 4417452 140735488037504 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/18811/statm: 19286 17310 148 55 0 17172 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 77144

[startup+0.70056 s]
/proc/loadavg: 8.15 8.12 8.03 2/514 18811
/proc/meminfo: memFree=9081448/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=142680 CPUtime=0.69 cores=0-7
/proc/18811/stat : 18811 (plingeling) R 18809 18811 18555 0 -1 4202496 33972 1251 0 0 66 3 0 0 24 0 1 0 702032804 146104320 33694 33554432000 4194304 4417452 140735488037504 18446744073709551615 4229136 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/18811/statm: 35670 33694 148 55 0 33556 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 142680

[startup+1.50036 s]
/proc/loadavg: 7.58 8.00 7.99 2/515 18817
/proc/meminfo: memFree=8815220/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=361648 CPUtime=4.21 cores=0-7
/proc/18811/stat : 18811 (plingeling) S 18809 18811 18555 0 -1 4202496 68372 1251 0 0 413 8 0 0 18 0 9 0 702032804 370327552 51666 33554432000 4194304 4417452 140735488037504 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/18811/statm: 90412 51666 175 55 0 88298 0
Current children cumulated CPU time (s) 4.21
Current children cumulated vsize (KiB) 361648

[startup+3.10107 s]
/proc/loadavg: 7.58 8.00 7.99 9/523 18825
/proc/meminfo: memFree=8747824/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=495240 CPUtime=16.97 cores=0-7
/proc/18811/stat : 18811 (plingeling) S 18809 18811 18555 0 -1 4202496 107332 1251 0 0 1680 17 0 0 18 0 9 0 702032804 507125760 85027 33554432000 4194304 4417452 140735488037504 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/18811/statm: 123810 85027 180 55 0 121696 0
[pid=18811/tid=18818] ppid=18809 vsize=495240 CPUtime=2.05 cores=0-7
/proc/18811/task/18818/stat : 18818 (plingeling) R 18809 18811 18555 0 -1 4202560 4566 1251 0 0 204 1 0 0 21 0 9 0 702032907 507125760 85027 33554432000 4194304 4417452 140735488037504 18446744073709551615 4260519 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18811/tid=18819] ppid=18809 vsize=495240 CPUtime=1.97 cores=0-7
/proc/18811/task/18819/stat : 18819 (plingeling) R 18809 18811 18555 0 -1 4202560 4583 1251 0 0 196 1 0 0 25 0 9 0 702032907 507125760 85027 33554432000 4194304 4417452 140735488037504 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=18811/tid=18820] ppid=18809 vsize=495240 CPUtime=1.75 cores=0-7
/proc/18811/task/18820/stat : 18820 (plingeling) R 18809 18811 18555 0 -1 4202560 6278 1251 0 0 173 2 0 0 25 0 9 0 702032907 507125760 85027 33554432000 4194304 4417452 140735488037504 18446744073709551615 4239584 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=18811/tid=18821] ppid=18809 vsize=495240 CPUtime=2.06 cores=0-7
/proc/18811/task/18821/stat : 18821 (plingeling) R 18809 18811 18555 0 -1 4202560 5095 1251 0 0 206 0 0 0 21 0 9 0 702032907 507125760 85027 33554432000 4194304 4417452 140735488037504 18446744073709551615 4245648 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18811/tid=18822] ppid=18809 vsize=495240 CPUtime=2.03 cores=0-7
/proc/18811/task/18822/stat : 18822 (plingeling) R 18809 18811 18555 0 -1 4202560 4592 1251 0 0 203 0 0 0 21 0 9 0 702032907 507125760 85027 33554432000 4194304 4417452 140735488037504 18446744073709551615 4255615 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=18811/tid=18823] ppid=18809 vsize=495240 CPUtime=2.05 cores=0-7
/proc/18811/task/18823/stat : 18823 (plingeling) R 18809 18811 18555 0 -1 4202560 4578 1251 0 0 204 1 0 0 21 0 9 0 702032907 507125760 85027 33554432000 4194304 4417452 140735488037504 18446744073709551615 4255024 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18811/tid=18824] ppid=18809 vsize=495240 CPUtime=1.92 cores=0-7
/proc/18811/task/18824/stat : 18824 (plingeling) R 18809 18811 18555 0 -1 4202560 6310 1251 0 0 191 1 0 0 25 0 9 0 702032907 507125760 85027 33554432000 4194304 4417452 140735488037504 18446744073709551615 4239418 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18811/tid=18825] ppid=18809 vsize=495240 CPUtime=2.05 cores=0-7
/proc/18811/task/18825/stat : 18825 (plingeling) R 18809 18811 18555 0 -1 4202560 4574 1251 0 0 204 1 0 0 25 0 9 0 702032907 507125760 85027 33554432000 4194304 4417452 140735488037504 18446744073709551615 4261313 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 16.97
Current children cumulated vsize (KiB) 495240

[startup+6.30048 s]
/proc/loadavg: 7.58 8.00 7.99 9/523 18825
/proc/meminfo: memFree=8619468/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=670260 CPUtime=42.51 cores=0-7
/proc/18811/stat : 18811 (plingeling) S 18809 18811 18555 0 -1 4202496 154381 1251 0 0 4225 26 0 0 18 0 9 0 702032804 686346240 128766 33554432000 4194304 4417452 140735488037504 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/18811/statm: 167565 128766 180 55 0 165451 0
[pid=18811/tid=18818] ppid=18809 vsize=670260 CPUtime=5.25 cores=0-7
/proc/18811/task/18818/stat : 18818 (plingeling) R 18809 18811 18555 0 -1 4202560 11073 1251 0 0 522 3 0 0 25 0 9 0 702032907 686346240 128766 33554432000 4194304 4417452 140735488037504 18446744073709551615 4259618 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18811/tid=18819] ppid=18809 vsize=670260 CPUtime=5.17 cores=0-7
/proc/18811/task/18819/stat : 18819 (plingeling) R 18809 18811 18555 0 -1 4202560 11090 1251 0 0 516 1 0 0 25 0 9 0 702032907 686346240 128766 33554432000 4194304 4417452 140735488037504 18446744073709551615 4320383 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=18811/tid=18820] ppid=18809 vsize=670260 CPUtime=4.95 cores=0-7
/proc/18811/task/18820/stat : 18820 (plingeling) R 18809 18811 18555 0 -1 4202560 10616 1251 0 0 492 3 0 0 25 0 9 0 702032907 686346240 128766 33554432000 4194304 4417452 140735488037504 18446744073709551615 4239408 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=18811/tid=18821] ppid=18809 vsize=670260 CPUtime=5.26 cores=0-7
/proc/18811/task/18821/stat : 18821 (plingeling) R 18809 18811 18555 0 -1 4202560 10933 1251 0 0 524 2 0 0 25 0 9 0 702032907 686346240 128766 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18811/tid=18822] ppid=18809 vsize=670260 CPUtime=5.17 cores=0-7
/proc/18811/task/18822/stat : 18822 (plingeling) R 18809 18811 18555 0 -1 4202560 11099 1251 0 0 515 2 0 0 25 0 9 0 702032907 686346240 128766 33554432000 4194304 4417452 140735488037504 18446744073709551615 4239393 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=18811/tid=18823] ppid=18809 vsize=670260 CPUtime=5.25 cores=0-7
/proc/18811/task/18823/stat : 18823 (plingeling) R 18809 18811 18555 0 -1 4202560 11085 1251 0 0 523 2 0 0 25 0 9 0 702032907 686346240 128766 33554432000 4194304 4417452 140735488037504 18446744073709551615 4239601 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18811/tid=18824] ppid=18809 vsize=670260 CPUtime=5.11 cores=0-7
/proc/18811/task/18824/stat : 18824 (plingeling) R 18809 18811 18555 0 -1 4202560 10648 1251 0 0 510 1 0 0 25 0 9 0 702032907 686346240 128766 33554432000 4194304 4417452 140735488037504 18446744073709551615 4239408 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18811/tid=18825] ppid=18809 vsize=670260 CPUtime=5.25 cores=0-7
/proc/18811/task/18825/stat : 18825 (plingeling) R 18809 18811 18555 0 -1 4202560 11081 1251 0 0 523 2 0 0 25 0 9 0 702032907 686346240 128766 33554432000 4194304 4417452 140735488037504 18446744073709551615 4239595 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 42.51
Current children cumulated vsize (KiB) 670260

[startup+12.7003 s]
/proc/loadavg: 7.72 8.02 7.99 9/523 18825
/proc/meminfo: memFree=8535360/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=701168 CPUtime=93.6 cores=0-7
/proc/18811/stat : 18811 (plingeling) S 18809 18811 18555 0 -1 4202496 167781 1251 0 0 9331 29 0 0 18 0 9 0 702032804 717996032 136493 33554432000 4194304 4417452 140735488037504 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/18811/statm: 175292 136493 180 55 0 173178 0
[pid=18811/tid=18818] ppid=18809 vsize=701168 CPUtime=11.65 cores=0-7
/proc/18811/task/18818/stat : 18818 (plingeling) R 18809 18811 18555 0 -1 4202560 11156 1251 0 0 1162 3 0 0 25 0 9 0 702032907 717996032 136493 33554432000 4194304 4417452 140735488037504 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18811/tid=18819] ppid=18809 vsize=701168 CPUtime=11.57 cores=0-7
/proc/18811/task/18819/stat : 18819 (plingeling) R 18809 18811 18555 0 -1 4202560 11090 1251 0 0 1156 1 0 0 25 0 9 0 702032907 717996032 136493 33554432000 4194304 4417452 140735488037504 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=18811/tid=18820] ppid=18809 vsize=701168 CPUtime=11.35 cores=0-7
/proc/18811/task/18820/stat : 18820 (plingeling) R 18809 18811 18555 0 -1 4202560 10616 1251 0 0 1132 3 0 0 25 0 9 0 702032907 717996032 136493 33554432000 4194304 4417452 140735488037504 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=18811/tid=18821] ppid=18809 vsize=701168 CPUtime=11.66 cores=0-7
/proc/18811/task/18821/stat : 18821 (plingeling) R 18809 18811 18555 0 -1 4202560 24042 1251 0 0 1162 4 0 0 25 0 9 0 702032907 717996032 136493 33554432000 4194304 4417452 140735488037504 18446744073709551615 4358436 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18811/tid=18822] ppid=18809 vsize=701168 CPUtime=11.46 cores=0-7
/proc/18811/task/18822/stat : 18822 (plingeling) R 18809 18811 18555 0 -1 4202560 11203 1251 0 0 1144 2 0 0 25 0 9 0 702032907 717996032 136493 33554432000 4194304 4417452 140735488037504 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=18811/tid=18823] ppid=18809 vsize=701168 CPUtime=11.65 cores=0-7
/proc/18811/task/18823/stat : 18823 (plingeling) R 18809 18811 18555 0 -1 4202560 11085 1251 0 0 1163 2 0 0 25 0 9 0 702032907 717996032 136493 33554432000 4194304 4417452 140735488037504 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18811/tid=18824] ppid=18809 vsize=701168 CPUtime=11.51 cores=0-7
/proc/18811/task/18824/stat : 18824 (plingeling) R 18809 18811 18555 0 -1 4202560 10648 1251 0 0 1150 1 0 0 25 0 9 0 702032907 717996032 136493 33554432000 4194304 4417452 140735488037504 18446744073709551615 4235378 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18811/tid=18825] ppid=18809 vsize=701168 CPUtime=11.65 cores=0-7
/proc/18811/task/18825/stat : 18825 (plingeling) R 18809 18811 18555 0 -1 4202560 11185 1251 0 0 1163 2 0 0 25 0 9 0 702032907 717996032 136493 33554432000 4194304 4417452 140735488037504 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 93.6
Current children cumulated vsize (KiB) 701168

[startup+25.5009 s]
/proc/loadavg: 7.76 8.01 7.99 9/523 18826
/proc/meminfo: memFree=8587504/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=644976 CPUtime=195.77 cores=0-7
/proc/18811/stat : 18811 (plingeling) S 18809 18811 18555 0 -1 4202496 206811 1251 0 0 19541 36 0 0 18 0 9 0 702032804 660455424 122449 33554432000 4194304 4417452 140735488037504 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/18811/statm: 161244 122449 180 55 0 159130 0
[pid=18811/tid=18818] ppid=18809 vsize=644976 CPUtime=24.45 cores=0-7
/proc/18811/task/18818/stat : 18818 (plingeling) R 18809 18811 18555 0 -1 4202560 13072 1251 0 0 2442 3 0 0 25 0 9 0 702032907 660455424 122449 33554432000 4194304 4417452 140735488037504 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18811/tid=18819] ppid=18809 vsize=644976 CPUtime=24.38 cores=0-7
/proc/18811/task/18819/stat : 18819 (plingeling) R 18809 18811 18555 0 -1 4202560 12293 1251 0 0 2436 2 0 0 25 0 9 0 702032907 660455424 122449 33554432000 4194304 4417452 140735488037504 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=18811/tid=18820] ppid=18809 vsize=644976 CPUtime=24.14 cores=0-7
/proc/18811/task/18820/stat : 18820 (plingeling) R 18809 18811 18555 0 -1 4202560 16725 1251 0 0 2410 4 0 0 25 0 9 0 702032907 660455424 122449 33554432000 4194304 4417452 140735488037504 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=18811/tid=18821] ppid=18809 vsize=644976 CPUtime=24.46 cores=0-7
/proc/18811/task/18821/stat : 18821 (plingeling) R 18809 18811 18555 0 -1 4202560 35628 1251 0 0 2439 7 0 0 25 0 9 0 702032907 660455424 122449 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18811/tid=18822] ppid=18809 vsize=644976 CPUtime=24.04 cores=0-7
/proc/18811/task/18822/stat : 18822 (plingeling) R 18809 18811 18555 0 -1 4202560 11636 1251 0 0 2402 2 0 0 25 0 9 0 702032907 660455424 122449 33554432000 4194304 4417452 140735488037504 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=18811/tid=18823] ppid=18809 vsize=644976 CPUtime=24.45 cores=0-7
/proc/18811/task/18823/stat : 18823 (plingeling) R 18809 18811 18555 0 -1 4202560 26242 1251 0 0 2441 4 0 0 25 0 9 0 702032907 660455424 122449 33554432000 4194304 4417452 140735488037504 18446744073709551615 18446744073699066094 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18811/tid=18824] ppid=18809 vsize=644976 CPUtime=24.31 cores=0-7
/proc/18811/task/18824/stat : 18824 (plingeling) R 18809 18811 18555 0 -1 4202560 12990 1251 0 0 2430 1 0 0 25 0 9 0 702032907 660455424 122449 33554432000 4194304 4417452 140735488037504 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18811/tid=18825] ppid=18809 vsize=644976 CPUtime=24.45 cores=0-7
/proc/18811/task/18825/stat : 18825 (plingeling) R 18809 18811 18555 0 -1 4202560 11469 1251 0 0 2443 2 0 0 25 0 9 0 702032907 660455424 122449 33554432000 4194304 4417452 140735488037504 18446744073709551615 4249151 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 195.77
Current children cumulated vsize (KiB) 644976

[startup+51.1013 s]
/proc/loadavg: 7.84 8.01 7.99 9/523 18826
/proc/meminfo: memFree=8459580/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=767948 CPUtime=400.14 cores=0-7
/proc/18811/stat : 18811 (plingeling) S 18809 18811 18555 0 -1 4202496 341250 1251 0 0 39956 58 0 0 18 0 9 0 702032804 786378752 153192 33554432000 4194304 4417452 140735488037504 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/18811/statm: 191987 153192 180 55 0 189873 0
[pid=18811/tid=18818] ppid=18809 vsize=767948 CPUtime=50.05 cores=0-7
/proc/18811/task/18818/stat : 18818 (plingeling) R 18809 18811 18555 0 -1 4202560 19860 1251 0 0 5001 4 0 0 25 0 9 0 702032907 786378752 153192 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18811/tid=18819] ppid=18809 vsize=767948 CPUtime=49.98 cores=0-7
/proc/18811/task/18819/stat : 18819 (plingeling) R 18809 18811 18555 0 -1 4202560 37278 1251 0 0 4992 6 0 0 25 0 9 0 702032907 786378752 153192 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=18811/tid=18820] ppid=18809 vsize=767948 CPUtime=49.74 cores=0-7
/proc/18811/task/18820/stat : 18820 (plingeling) R 18809 18811 18555 0 -1 4202560 39937 1251 0 0 4966 8 0 0 25 0 9 0 702032907 786378752 153192 33554432000 4194304 4417452 140735488037504 18446744073709551615 4254099 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=18811/tid=18821] ppid=18809 vsize=767948 CPUtime=50.06 cores=0-7
/proc/18811/task/18821/stat : 18821 (plingeling) R 18809 18811 18555 0 -1 4202560 51994 1251 0 0 4996 10 0 0 25 0 9 0 702032907 786378752 153192 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18811/tid=18822] ppid=18809 vsize=767948 CPUtime=49.2 cores=0-7
/proc/18811/task/18822/stat : 18822 (plingeling) R 18809 18811 18555 0 -1 4202560 19900 1251 0 0 4917 3 0 0 25 0 9 0 702032907 786378752 153192 33554432000 4194304 4417452 140735488037504 18446744073709551615 4356934 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=18811/tid=18823] ppid=18809 vsize=767948 CPUtime=50.05 cores=0-7
/proc/18811/task/18823/stat : 18823 (plingeling) R 18809 18811 18555 0 -1 4202560 48699 1251 0 0 4997 8 0 0 25 0 9 0 702032907 786378752 153192 33554432000 4194304 4417452 140735488037504 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18811/tid=18824] ppid=18809 vsize=767948 CPUtime=49.91 cores=0-7
/proc/18811/task/18824/stat : 18824 (plingeling) R 18809 18811 18555 0 -1 4202560 30447 1251 0 0 4987 4 0 0 25 0 9 0 702032907 786378752 153192 33554432000 4194304 4417452 140735488037504 18446744073709551615 4245661 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18811/tid=18825] ppid=18809 vsize=767948 CPUtime=50.05 cores=0-7
/proc/18811/task/18825/stat : 18825 (plingeling) R 18809 18811 18555 0 -1 4202560 26379 1251 0 0 5001 4 0 0 25 0 9 0 702032907 786378752 153192 33554432000 4194304 4417452 140735488037504 18446744073709551615 4245616 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 400.14
Current children cumulated vsize (KiB) 767948

[startup+102.32 s]
/proc/loadavg: 8.02 8.04 8.00 9/523 18828

################
# More data... #
################

[pid=18811/tid=18820] ppid=18809 vsize=6595176 CPUtime=4719.75 cores=0-7
/proc/18811/task/18820/stat : 18820 (plingeling) R 18809 18811 18555 0 -1 4202560 2962132 1251 0 0 471411 564 0 0 25 0 9 0 702032907 6753460224 1609999 33554432000 4194304 4417452 140735488037504 18446744073709551615 4258457 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=18811/tid=18821] ppid=18809 vsize=6595176 CPUtime=4720.62 cores=0-7
/proc/18811/task/18821/stat : 18821 (plingeling) R 18809 18811 18555 0 -1 4202560 571722 1251 0 0 471910 152 0 0 25 0 9 0 702032907 6753460224 1609999 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18811/tid=18822] ppid=18809 vsize=6595176 CPUtime=4717.61 cores=0-7
/proc/18811/task/18822/stat : 18822 (plingeling) R 18809 18811 18555 0 -1 4202560 2617134 1251 0 0 471348 413 0 0 25 0 9 0 702032907 6753460224 1609999 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18811/tid=18823] ppid=18809 vsize=6595176 CPUtime=4721.35 cores=0-7
/proc/18811/task/18823/stat : 18823 (plingeling) R 18809 18811 18555 0 -1 4202560 4116851 1251 0 0 471338 797 0 0 25 0 9 0 702032907 6753460224 1609999 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18811/tid=18824] ppid=18809 vsize=6595176 CPUtime=4666.78 cores=0-7
/proc/18811/task/18824/stat : 18824 (plingeling) R 18809 18811 18555 0 -1 4202560 4230515 1251 0 0 465890 788 0 0 25 0 9 0 702032907 6753460224 1609999 33554432000 4194304 4417452 140735488037504 18446744073709551615 4275210 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=18811/tid=18825] ppid=18809 vsize=6595176 CPUtime=4721.16 cores=0-7
/proc/18811/task/18825/stat : 18825 (plingeling) R 18809 18811 18555 0 -1 4202560 2103211 1251 0 0 471794 322 0 0 25 0 9 0 702032907 6753460224 1609999 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37687.4
Current children cumulated vsize (KiB) 6595176

[startup+4782.3 s]
/proc/loadavg: 8.11 8.07 8.02 9/523 18964
/proc/meminfo: memFree=1795792/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=7427900 CPUtime=38166.2 cores=0-7
/proc/18811/stat : 18811 (plingeling) S 18809 18811 18555 0 -1 4202496 22509755 1251 0 0 3812552 4070 0 0 18 0 9 0 702032804 7606169600 1818180 33554432000 4194304 4417452 140735488037504 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/18811/statm: 1856975 1818180 180 55 0 1854861 0
[pid=18811/tid=18818] ppid=18809 vsize=7427900 CPUtime=4759.61 cores=0-7
/proc/18811/task/18818/stat : 18818 (plingeling) R 18809 18811 18555 0 -1 4202560 2738816 1251 0 0 475483 478 0 0 25 0 9 0 702032907 7606169600 1818180 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18811/tid=18819] ppid=18809 vsize=7427900 CPUtime=4778.35 cores=0-7
/proc/18811/task/18819/stat : 18819 (plingeling) R 18809 18811 18555 0 -1 4202560 2582923 1251 0 0 477398 437 0 0 25 0 9 0 702032907 7606169600 1818180 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=18811/tid=18820] ppid=18809 vsize=7427900 CPUtime=4779.73 cores=0-7
/proc/18811/task/18820/stat : 18820 (plingeling) R 18809 18811 18555 0 -1 4202560 3076530 1251 0 0 477376 597 0 0 25 0 9 0 702032907 7606169600 1818180 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=18811/tid=18821] ppid=18809 vsize=7427900 CPUtime=4780.6 cores=0-7
/proc/18811/task/18821/stat : 18821 (plingeling) R 18809 18811 18555 0 -1 4202560 571722 1251 0 0 477908 152 0 0 25 0 9 0 702032907 7606169600 1818180 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18811/tid=18822] ppid=18809 vsize=7427900 CPUtime=4777.59 cores=0-7
/proc/18811/task/18822/stat : 18822 (plingeling) R 18809 18811 18555 0 -1 4202560 2666286 1251 0 0 477338 421 0 0 25 0 9 0 702032907 7606169600 1818180 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18811/tid=18823] ppid=18809 vsize=7427900 CPUtime=4781.33 cores=0-7
/proc/18811/task/18823/stat : 18823 (plingeling) R 18809 18811 18555 0 -1 4202560 4116851 1251 0 0 477336 797 0 0 25 0 9 0 702032907 7606169600 1818180 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18811/tid=18824] ppid=18809 vsize=7427900 CPUtime=4726.76 cores=0-7
/proc/18811/task/18824/stat : 18824 (plingeling) R 18809 18811 18555 0 -1 4202560 4534500 1251 0 0 471836 840 0 0 25 0 9 0 702032907 7606169600 1818180 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=18811/tid=18825] ppid=18809 vsize=7427900 CPUtime=4781.14 cores=0-7
/proc/18811/task/18825/stat : 18825 (plingeling) R 18809 18811 18555 0 -1 4202560 2155371 1251 0 0 477777 337 0 0 25 0 9 0 702032907 7606169600 1818180 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38166.2
Current children cumulated vsize (KiB) 7427900

[startup+4842.3 s]
/proc/loadavg: 8.10 8.07 8.02 9/523 18966
/proc/meminfo: memFree=2077268/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=7146496 CPUtime=38645.2 cores=0-7
/proc/18811/stat : 18811 (plingeling) S 18809 18811 18555 0 -1 4202496 22682852 1251 0 0 3860424 4092 0 0 18 0 9 0 702032804 7318011904 1747829 33554432000 4194304 4417452 140735488037504 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/18811/statm: 1786624 1747829 180 55 0 1784510 0
[pid=18811/tid=18818] ppid=18809 vsize=7146496 CPUtime=4819.61 cores=0-7
/proc/18811/task/18818/stat : 18818 (plingeling) R 18809 18811 18555 0 -1 4202560 2771584 1251 0 0 481479 482 0 0 25 0 9 0 702032907 7318011904 1747829 33554432000 4194304 4417452 140735488037504 18446744073709551615 4263118 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18811/tid=18819] ppid=18809 vsize=7146496 CPUtime=4837.31 cores=0-7
/proc/18811/task/18819/stat : 18819 (plingeling) R 18809 18811 18555 0 -1 4202560 2625221 1251 0 0 483288 443 0 0 25 0 9 0 702032907 7318011904 1747829 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=18811/tid=18820] ppid=18809 vsize=7146496 CPUtime=4839.71 cores=0-7
/proc/18811/task/18820/stat : 18820 (plingeling) R 18809 18811 18555 0 -1 4202560 3079506 1251 0 0 483373 598 0 0 25 0 9 0 702032907 7318011904 1747829 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=18811/tid=18821] ppid=18809 vsize=7146496 CPUtime=4840.6 cores=0-7
/proc/18811/task/18821/stat : 18821 (plingeling) R 18809 18811 18555 0 -1 4202560 571722 1251 0 0 483908 152 0 0 25 0 9 0 702032907 7318011904 1747829 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18811/tid=18822] ppid=18809 vsize=7146496 CPUtime=4837.6 cores=0-7
/proc/18811/task/18822/stat : 18822 (plingeling) R 18809 18811 18555 0 -1 4202560 2681596 1251 0 0 483336 424 0 0 25 0 9 0 702032907 7318011904 1747829 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18811/tid=18823] ppid=18809 vsize=7146496 CPUtime=4841.33 cores=0-7
/proc/18811/task/18823/stat : 18823 (plingeling) R 18809 18811 18555 0 -1 4202560 4116851 1251 0 0 483336 797 0 0 25 0 9 0 702032907 7318011904 1747829 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18811/tid=18824] ppid=18809 vsize=7146496 CPUtime=4786.77 cores=0-7
/proc/18811/task/18824/stat : 18824 (plingeling) R 18809 18811 18555 0 -1 4202560 4600036 1251 0 0 477829 848 0 0 25 0 9 0 702032907 7318011904 1747829 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=18811/tid=18825] ppid=18809 vsize=7146496 CPUtime=4841.14 cores=0-7
/proc/18811/task/18825/stat : 18825 (plingeling) R 18809 18811 18555 0 -1 4202560 2169580 1251 0 0 483775 339 0 0 25 0 9 0 702032907 7318011904 1747829 33554432000 4194304 4417452 140735488037504 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38645.2
Current children cumulated vsize (KiB) 7146496

[startup+4902.3 s]
/proc/loadavg: 8.10 8.07 8.02 9/523 18968
/proc/meminfo: memFree=2375228/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=6848368 CPUtime=39124.1 cores=0-7
/proc/18811/stat : 18811 (plingeling) S 18809 18811 18555 0 -1 4202496 22833459 1251 0 0 3908303 4110 0 0 18 0 9 0 702032804 7012728832 1673297 33554432000 4194304 4417452 140735488037504 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/18811/statm: 1712092 1673297 180 55 0 1709978 0
[pid=18811/tid=18818] ppid=18809 vsize=6848368 CPUtime=4879.61 cores=0-7
/proc/18811/task/18818/stat : 18818 (plingeling) R 18809 18811 18555 0 -1 4202560 2837120 1251 0 0 487472 489 0 0 25 0 9 0 702032907 7012728832 1673297 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18811/tid=18819] ppid=18809 vsize=6848368 CPUtime=4896.29 cores=0-7
/proc/18811/task/18819/stat : 18819 (plingeling) R 18809 18811 18555 0 -1 4202560 2636581 1251 0 0 489185 444 0 0 25 0 9 0 702032907 7012728832 1673297 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=18811/tid=18820] ppid=18809 vsize=6848368 CPUtime=4899.7 cores=0-7
/proc/18811/task/18820/stat : 18820 (plingeling) R 18809 18811 18555 0 -1 4202560 3079506 1251 0 0 489372 598 0 0 25 0 9 0 702032907 7012728832 1673297 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=18811/tid=18821] ppid=18809 vsize=6848368 CPUtime=4900.61 cores=0-7
/proc/18811/task/18821/stat : 18821 (plingeling) R 18809 18811 18555 0 -1 4202560 571722 1251 0 0 489909 152 0 0 25 0 9 0 702032907 7012728832 1673297 33554432000 4194304 4417452 140735488037504 18446744073709551615 4251076 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18811/tid=18822] ppid=18809 vsize=6848368 CPUtime=4897.6 cores=0-7
/proc/18811/task/18822/stat : 18822 (plingeling) R 18809 18811 18555 0 -1 4202560 2702684 1251 0 0 489334 426 0 0 25 0 9 0 702032907 7012728832 1673297 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18811/tid=18823] ppid=18809 vsize=6848368 CPUtime=4901.33 cores=0-7
/proc/18811/task/18823/stat : 18823 (plingeling) R 18809 18811 18555 0 -1 4202560 4116851 1251 0 0 489336 797 0 0 25 0 9 0 702032907 7012728832 1673297 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18811/tid=18824] ppid=18809 vsize=6848368 CPUtime=4846.77 cores=0-7
/proc/18811/task/18824/stat : 18824 (plingeling) R 18809 18811 18555 0 -1 4202560 4632804 1251 0 0 483825 852 0 0 25 0 9 0 702032907 7012728832 1673297 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=18811/tid=18825] ppid=18809 vsize=6848368 CPUtime=4901.13 cores=0-7
/proc/18811/task/18825/stat : 18825 (plingeling) R 18809 18811 18555 0 -1 4202560 2189435 1251 0 0 489771 342 0 0 25 0 9 0 702032907 7012728832 1673297 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39124.1
Current children cumulated vsize (KiB) 6848368

[startup+4962.3 s]
/proc/loadavg: 8.09 8.07 8.02 9/523 18969
/proc/meminfo: memFree=2037620/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=7184448 CPUtime=39603.1 cores=0-7
/proc/18811/stat : 18811 (plingeling) S 18809 18811 18555 0 -1 4202496 23198907 1251 0 0 3956147 4161 0 0 18 0 9 0 702032804 7356874752 1757317 33554432000 4194304 4417452 140735488037504 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/18811/statm: 1796112 1757317 180 55 0 1793998 0
[pid=18811/tid=18818] ppid=18809 vsize=7184448 CPUtime=4939.61 cores=0-7
/proc/18811/task/18818/stat : 18818 (plingeling) R 18809 18811 18555 0 -1 4202560 2869888 1251 0 0 493466 495 0 0 25 0 9 0 702032907 7356874752 1757317 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18811/tid=18819] ppid=18809 vsize=7184448 CPUtime=4955.24 cores=0-7
/proc/18811/task/18819/stat : 18819 (plingeling) R 18809 18811 18555 0 -1 4202560 2741611 1251 0 0 495064 460 0 0 25 0 9 0 702032907 7356874752 1757317 33554432000 4194304 4417452 140735488037504 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=18811/tid=18820] ppid=18809 vsize=7184448 CPUtime=4959.68 cores=0-7
/proc/18811/task/18820/stat : 18820 (plingeling) R 18809 18811 18555 0 -1 4202560 3145477 1251 0 0 495358 610 0 0 25 0 9 0 702032907 7356874752 1757317 33554432000 4194304 4417452 140735488037504 18446744073709551615 4251096 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=18811/tid=18821] ppid=18809 vsize=7184448 CPUtime=4960.61 cores=0-7
/proc/18811/task/18821/stat : 18821 (plingeling) R 18809 18811 18555 0 -1 4202560 571722 1251 0 0 495909 152 0 0 25 0 9 0 702032907 7356874752 1757317 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18811/tid=18822] ppid=18809 vsize=7184448 CPUtime=4957.6 cores=0-7
/proc/18811/task/18822/stat : 18822 (plingeling) R 18809 18811 18555 0 -1 4202560 2787051 1251 0 0 495325 435 0 0 25 0 9 0 702032907 7356874752 1757317 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18811/tid=18823] ppid=18809 vsize=7184448 CPUtime=4961.33 cores=0-7
/proc/18811/task/18823/stat : 18823 (plingeling) R 18809 18811 18555 0 -1 4202560 4116851 1251 0 0 495336 797 0 0 25 0 9 0 702032907 7356874752 1757317 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18811/tid=18824] ppid=18809 vsize=7184448 CPUtime=4906.77 cores=0-7
/proc/18811/task/18824/stat : 18824 (plingeling) R 18809 18811 18555 0 -1 4202560 4698340 1251 0 0 489818 859 0 0 25 0 9 0 702032907 7356874752 1757317 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=18811/tid=18825] ppid=18809 vsize=7184448 CPUtime=4961.13 cores=0-7
/proc/18811/task/18825/stat : 18825 (plingeling) R 18809 18811 18555 0 -1 4202560 2201211 1251 0 0 495770 343 0 0 25 0 9 0 702032907 7356874752 1757317 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39603.1
Current children cumulated vsize (KiB) 7184448



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5012.1 s]
/proc/loadavg: 8.10 8.07 8.02 9/523 18971
/proc/meminfo: memFree=2232884/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=6991740 CPUtime=40000.7 cores=0-7
/proc/18811/stat : 18811 (plingeling) S 18809 18811 18555 0 -1 4202496 23541212 1251 0 0 3995841 4230 0 0 18 0 9 0 702032804 7159541760 1709140 33554432000 4194304 4417452 140735488037504 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/18811/statm: 1747935 1709140 180 55 0 1745821 0
[pid=18811/tid=18818] ppid=18809 vsize=6991740 CPUtime=4989.44 cores=0-7
/proc/18811/task/18818/stat : 18818 (plingeling) R 18809 18811 18555 0 -1 4202560 2902656 1251 0 0 498443 501 0 0 25 0 9 0 702032907 7159541760 1709140 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18811/tid=18819] ppid=18809 vsize=6991740 CPUtime=5004.18 cores=0-7
/proc/18811/task/18819/stat : 18819 (plingeling) R 18809 18811 18555 0 -1 4202560 2757995 1251 0 0 499957 461 0 0 25 0 9 0 702032907 7159541760 1709140 33554432000 4194304 4417452 140735488037504 18446744073709551615 4322101 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=18811/tid=18820] ppid=18809 vsize=6991740 CPUtime=5009.48 cores=0-7
/proc/18811/task/18820/stat : 18820 (plingeling) R 18809 18811 18555 0 -1 4202560 3145477 1251 0 0 500338 610 0 0 25 0 9 0 702032907 7159541760 1709140 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=18811/tid=18821] ppid=18809 vsize=6991740 CPUtime=5010.43 cores=0-7
/proc/18811/task/18821/stat : 18821 (plingeling) R 18809 18811 18555 0 -1 4202560 571722 1251 0 0 500891 152 0 0 25 0 9 0 702032907 7159541760 1709140 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18811/tid=18822] ppid=18809 vsize=6991740 CPUtime=5007.42 cores=0-7
/proc/18811/task/18822/stat : 18822 (plingeling) R 18809 18811 18555 0 -1 4202560 2834911 1251 0 0 500299 443 0 0 25 0 9 0 702032907 7159541760 1709140 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18811/tid=18823] ppid=18809 vsize=6991740 CPUtime=5011.16 cores=0-7
/proc/18811/task/18823/stat : 18823 (plingeling) R 18809 18811 18555 0 -1 4202560 4286629 1251 0 0 500283 833 0 0 25 0 9 0 702032907 7159541760 1709140 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18811/tid=18824] ppid=18809 vsize=6991740 CPUtime=4956.59 cores=0-7
/proc/18811/task/18824/stat : 18824 (plingeling) R 18809 18811 18555 0 -1 4202560 4733918 1251 0 0 494786 873 0 0 25 0 9 0 702032907 7159541760 1709140 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=18811/tid=18825] ppid=18809 vsize=6991740 CPUtime=5010.95 cores=0-7
/proc/18811/task/18825/stat : 18825 (plingeling) R 18809 18811 18555 0 -1 4202560 2241148 1251 0 0 500748 347 0 0 25 0 9 0 702032907 7159541760 1709140 33554432000 4194304 4417452 140735488037504 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 6991740

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5012.23 s]
/proc/loadavg: 8.10 8.07 8.02 9/523 18971
/proc/meminfo: memFree=2232884/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=0 CPUtime=40000.9 cores=0-7
/proc/18811/stat : 18811 (plingeling) Z 18809 18811 18555 0 -1 4203548 23541212 1251 0 0 3995851 4240 0 0 18 0 2 0 702032804 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18811/statm: 0 0 0 0 0 0 0
[pid=18811/tid=18819] ppid=18809 vsize=0 CPUtime=5004.29 cores=0-7
/proc/18811/task/18819/stat : 18819 (plingeling) R 18809 18811 18555 0 -1 4203588 2757995 1251 0 0 499958 471 0 0 25 0 2 0 702032907 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 0

[startup+5012.3 s]
/proc/loadavg: 8.10 8.07 8.02 9/523 18971
/proc/meminfo: memFree=2232884/32951124 swapFree=65066528/67111528
[pid=18811] ppid=18809 vsize=0 CPUtime=40001 cores=0-7
/proc/18811/stat : 18811 (plingeling) Z 18809 18811 18555 0 -1 4203548 23541212 1251 0 0 3995851 4247 0 0 18 0 2 0 702032804 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18811/statm: 0 0 0 0 0 0 0
[pid=18811/tid=18819] ppid=18809 vsize=0 CPUtime=5004.37 cores=0-7
/proc/18811/task/18819/stat : 18819 (plingeling) R 18809 18811 18555 0 -1 4203588 2757995 1251 0 0 499958 479 0 0 25 0 2 0 702032907 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40001
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5012.38
CPU time (s): 40001.1
CPU user time (s): 39958.5
CPU system time (s): 42.5575
CPU usage (%): 798.045
Max. virtual memory (cumulated for all children) (KiB): 7771352

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39958.5
system time used= 42.5575
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 23542463
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 465
involuntary context switches= 193763

runsolver used 24.4353 second user time and 62.1236 second system time

The end

Launcher Data 

Begin job on node142 at 2011-05-22 22:29:33
IDJOB=3394442
IDBENCH=83165
IDSOLVER=1851
FILE ID=node142/3394442-1306096173
RUNJOBID= node142-1306091158-18569
PBS_JOBID= 13387937
Free space on /tmp= 73396 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-21x33.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394442-1306096173/watcher-3394442-1306096173 -o /tmp/evaluation-result-3394442-1306096173/solver-3394442-1306096173 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394442-1306096173.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= f7c29bbc3bef00d1e7cc799553ba4813
RANDOM SEED=667505998

node142.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5131.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:       9081728 kB
Buffers:        243496 kB
Cached:        6607256 kB
SwapCached:      20520 kB
Active:       11870724 kB
Inactive:     11666316 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:       9081728 kB
SwapTotal:    67111528 kB
SwapFree:     65066528 kB
Dirty:           10892 kB
Writeback:           0 kB
AnonPages:    16675212 kB
Mapped:          33380 kB
Slab:           185648 kB
PageTables:      75364 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 145377576 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73396 MiB
End job on node142 at 2011-05-22 23:53:08