Trace number 3372164

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.4 5005.71

General information on the benchmark

Nameindustrial/IBM_FV_2002_13_rule_1/
SAT_dat.k20.cnf
MD5SUM2bee695ce4278b72ee9295631ad94252
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark162.983
Satisfiable
(Un)Satisfiability was proved
Number of variables31357
Number of clauses133420
Sum of the clauses size370492
Maximum clause length37
Minimum clause length1
Number of clauses of size 1278
Number of clauses of size 258046
Number of clauses of size 369900
Number of clauses of size 4166
Number of clauses of size 5305
Number of clauses of size over 54725

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372164-1305436232/watcher-3372164-1305436232 -o /tmp/evaluation-result-3372164-1305436232/solver-3372164-1305436232 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3372164-1305436232.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.73 4.01 4.79 1/267 8223
/proc/meminfo: memFree=25126652/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=8728 CPUtime=0 cores=0-7
/proc/8223/stat : 8223 (plingeling) R 8221 8223 8197 0 -1 4202496 254 1251 0 0 0 0 0 0 22 0 1 0 733853776 8937472 185 33554432000 4194304 4417452 140733232609984 18446744073709551615 260020395776 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/8223/statm: 2182 185 138 55 0 68 0

[startup+0.097059 s]
/proc/loadavg: 3.73 4.01 4.79 1/267 8223
/proc/meminfo: memFree=25126652/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=28216 CPUtime=0.08 cores=0-7
/proc/8223/stat : 8223 (plingeling) R 8221 8223 8197 0 -1 4202496 5147 1251 0 0 8 0 0 0 22 0 1 0 733853776 28893184 5078 33554432000 4194304 4417452 140733232609984 18446744073709551615 4231629 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/8223/statm: 7054 5078 149 55 0 4940 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 28216

[startup+0.101052 s]
/proc/loadavg: 3.73 4.01 4.79 1/267 8223
/proc/meminfo: memFree=25126652/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=28216 CPUtime=0.09 cores=0-7
/proc/8223/stat : 8223 (plingeling) R 8221 8223 8197 0 -1 4202496 5147 1251 0 0 9 0 0 0 22 0 1 0 733853776 28893184 5078 33554432000 4194304 4417452 140733232609984 18446744073709551615 4265019 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/8223/statm: 7054 5078 149 55 0 4940 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 28216

[startup+0.301019 s]
/proc/loadavg: 3.73 4.01 4.79 1/267 8223
/proc/meminfo: memFree=25126652/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=63716 CPUtime=0.29 cores=0-7
/proc/8223/stat : 8223 (plingeling) R 8221 8223 8197 0 -1 4202496 14022 1251 0 0 28 1 0 0 23 0 1 0 733853776 65245184 13953 33554432000 4194304 4417452 140733232609984 18446744073709551615 4209458 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/8223/statm: 15929 13953 149 55 0 13815 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 63716

[startup+0.700912 s]
/proc/loadavg: 3.73 4.01 4.79 1/267 8223
/proc/meminfo: memFree=25126652/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=156816 CPUtime=2.96 cores=0-7
/proc/8223/stat : 8223 (plingeling) S 8221 8223 8197 0 -1 4202496 20112 1251 0 0 295 1 0 0 18 0 9 0 733853776 160579584 16750 33554432000 4194304 4417452 140733232609984 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/8223/statm: 39204 16750 175 55 0 37090 0
Current children cumulated CPU time (s) 2.96
Current children cumulated vsize (KiB) 156816

[startup+1.50076 s]
/proc/loadavg: 3.73 4.01 4.79 9/276 8237
/proc/meminfo: memFree=25054792/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=222176 CPUtime=9.35 cores=0-7
/proc/8223/stat : 8223 (plingeling) S 8221 8223 8197 0 -1 4202496 24817 1251 0 0 933 2 0 0 18 0 9 0 733853776 227508224 16743 33554432000 4194304 4417452 140733232609984 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/8223/statm: 55544 16743 179 55 0 53430 0
[pid=8223/tid=8230] ppid=8221 vsize=222176 CPUtime=0.99 cores=0-7
/proc/8223/task/8230/stat : 8230 (plingeling) R 8221 8223 8197 0 -1 4202560 248 1251 0 0 99 0 0 0 25 0 9 0 733853808 227508224 16743 33554432000 4194304 4417452 140733232609984 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8223/tid=8231] ppid=8221 vsize=222176 CPUtime=1.17 cores=0-7
/proc/8223/task/8231/stat : 8231 (plingeling) R 8221 8223 8197 0 -1 4202560 2067 1251 0 0 117 0 0 0 19 0 9 0 733853808 227508224 16743 33554432000 4194304 4417452 140733232609984 18446744073709551615 4346677 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8223/tid=8232] ppid=8221 vsize=222176 CPUtime=1.17 cores=0-7
/proc/8223/task/8232/stat : 8232 (plingeling) R 8221 8223 8197 0 -1 4202560 1166 1251 0 0 117 0 0 0 19 0 9 0 733853808 227508224 16743 33554432000 4194304 4417452 140733232609984 18446744073709551615 4346643 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8223/tid=8233] ppid=8221 vsize=222176 CPUtime=1.17 cores=0-7
/proc/8223/task/8233/stat : 8233 (plingeling) R 8221 8223 8197 0 -1 4202560 651 1251 0 0 117 0 0 0 22 0 9 0 733853808 227508224 16743 33554432000 4194304 4417452 140733232609984 18446744073709551615 4357902 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8223/tid=8234] ppid=8221 vsize=222176 CPUtime=1.13 cores=0-7
/proc/8223/task/8234/stat : 8234 (plingeling) R 8221 8223 8197 0 -1 4202560 785 1251 0 0 113 0 0 0 22 0 9 0 733853808 227508224 16743 33554432000 4194304 4417452 140733232609984 18446744073709551615 4346677 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8223/tid=8235] ppid=8221 vsize=222176 CPUtime=1.17 cores=0-7
/proc/8223/task/8235/stat : 8235 (plingeling) R 8221 8223 8197 0 -1 4202560 1853 1251 0 0 117 0 0 0 22 0 9 0 733853808 227508224 16743 33554432000 4194304 4417452 140733232609984 18446744073709551615 4275748 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8223/tid=8236] ppid=8221 vsize=222176 CPUtime=1.16 cores=0-7
/proc/8223/task/8236/stat : 8236 (plingeling) R 8221 8223 8197 0 -1 4202560 1884 1251 0 0 116 0 0 0 19 0 9 0 733853808 227508224 16743 33554432000 4194304 4417452 140733232609984 18446744073709551615 4320345 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=8223/tid=8237] ppid=8221 vsize=222176 CPUtime=1.03 cores=0-7
/proc/8223/task/8237/stat : 8237 (plingeling) R 8221 8223 8197 0 -1 4202560 2125 1251 0 0 103 0 0 0 21 0 9 0 733853809 227508224 16743 33554432000 4194304 4417452 140733232609984 18446744073709551615 4275892 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 9.35
Current children cumulated vsize (KiB) 222176

[startup+3.10047 s]
/proc/loadavg: 3.73 4.01 4.79 9/276 8238
/proc/meminfo: memFree=25049360/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=242364 CPUtime=22.14 cores=0-7
/proc/8223/stat : 8223 (plingeling) S 8221 8223 8197 0 -1 4202496 34369 1251 0 0 2211 3 0 0 18 0 9 0 733853776 248180736 21827 33554432000 4194304 4417452 140733232609984 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/8223/statm: 60591 21827 180 55 0 58477 0
[pid=8223/tid=8230] ppid=8221 vsize=242364 CPUtime=2.59 cores=0-7
/proc/8223/task/8230/stat : 8230 (plingeling) R 8221 8223 8197 0 -1 4202560 1476 1251 0 0 259 0 0 0 25 0 9 0 733853808 248180736 21827 33554432000 4194304 4417452 140733232609984 18446744073709551615 4252252 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8223/tid=8231] ppid=8221 vsize=242364 CPUtime=2.77 cores=0-7
/proc/8223/task/8231/stat : 8231 (plingeling) R 8221 8223 8197 0 -1 4202560 3680 1251 0 0 277 0 0 0 23 0 9 0 733853808 248180736 21827 33554432000 4194304 4417452 140733232609984 18446744073709551615 4250080 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8223/tid=8232] ppid=8221 vsize=242364 CPUtime=2.77 cores=0-7
/proc/8223/task/8232/stat : 8232 (plingeling) R 8221 8223 8197 0 -1 4202560 1311 1251 0 0 277 0 0 0 23 0 9 0 733853808 248180736 21827 33554432000 4194304 4417452 140733232609984 18446744073709551615 4352152 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8223/tid=8233] ppid=8221 vsize=242364 CPUtime=2.77 cores=0-7
/proc/8223/task/8233/stat : 8233 (plingeling) R 8221 8223 8197 0 -1 4202560 724 1251 0 0 277 0 0 0 25 0 9 0 733853808 248180736 21827 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8223/tid=8234] ppid=8221 vsize=242364 CPUtime=2.72 cores=0-7
/proc/8223/task/8234/stat : 8234 (plingeling) R 8221 8223 8197 0 -1 4202560 1246 1251 0 0 272 0 0 0 25 0 9 0 733853808 248180736 21827 33554432000 4194304 4417452 140733232609984 18446744073709551615 4258587 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8223/tid=8235] ppid=8221 vsize=242364 CPUtime=2.76 cores=0-7
/proc/8223/task/8235/stat : 8235 (plingeling) R 8221 8223 8197 0 -1 4202560 4551 1251 0 0 276 0 0 0 25 0 9 0 733853808 248180736 21827 33554432000 4194304 4417452 140733232609984 18446744073709551615 4252879 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8223/tid=8236] ppid=8221 vsize=242364 CPUtime=2.76 cores=0-7
/proc/8223/task/8236/stat : 8236 (plingeling) R 8221 8223 8197 0 -1 4202560 2055 1251 0 0 276 0 0 0 23 0 9 0 733853808 248180736 21827 33554432000 4194304 4417452 140733232609984 18446744073709551615 4250825 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=8223/tid=8237] ppid=8221 vsize=242364 CPUtime=2.62 cores=0-7
/proc/8223/task/8237/stat : 8237 (plingeling) R 8221 8223 8197 0 -1 4202560 5288 1251 0 0 262 0 0 0 25 0 9 0 733853809 248180736 21827 33554432000 4194304 4417452 140733232609984 18446744073709551615 4253152 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 22.14
Current children cumulated vsize (KiB) 242364

[startup+6.30088 s]
/proc/loadavg: 4.07 4.07 4.81 9/276 8238
/proc/meminfo: memFree=25009844/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=276564 CPUtime=47.71 cores=0-7
/proc/8223/stat : 8223 (plingeling) S 8221 8223 8197 0 -1 4202496 42978 1251 0 0 4767 4 0 0 18 0 9 0 733853776 283201536 30397 33554432000 4194304 4417452 140733232609984 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/8223/statm: 69141 30397 180 55 0 67027 0
[pid=8223/tid=8230] ppid=8221 vsize=276564 CPUtime=5.79 cores=0-7
/proc/8223/task/8230/stat : 8230 (plingeling) R 8221 8223 8197 0 -1 4202560 3012 1251 0 0 579 0 0 0 25 0 9 0 733853808 283201536 30397 33554432000 4194304 4417452 140733232609984 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8223/tid=8231] ppid=8221 vsize=276564 CPUtime=5.97 cores=0-7
/proc/8223/task/8231/stat : 8231 (plingeling) R 8221 8223 8197 0 -1 4202560 4112 1251 0 0 597 0 0 0 25 0 9 0 733853808 283201536 30397 33554432000 4194304 4417452 140733232609984 18446744073709551615 4264542 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8223/tid=8232] ppid=8221 vsize=276564 CPUtime=5.97 cores=0-7
/proc/8223/task/8232/stat : 8232 (plingeling) R 8221 8223 8197 0 -1 4202560 1434 1251 0 0 597 0 0 0 25 0 9 0 733853808 283201536 30397 33554432000 4194304 4417452 140733232609984 18446744073709551615 4236866 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8223/tid=8233] ppid=8221 vsize=276564 CPUtime=5.97 cores=0-7
/proc/8223/task/8233/stat : 8233 (plingeling) R 8221 8223 8197 0 -1 4202560 2147 1251 0 0 597 0 0 0 25 0 9 0 733853808 283201536 30397 33554432000 4194304 4417452 140733232609984 18446744073709551615 4357094 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8223/tid=8234] ppid=8221 vsize=276564 CPUtime=5.9 cores=0-7
/proc/8223/task/8234/stat : 8234 (plingeling) R 8221 8223 8197 0 -1 4202560 2504 1251 0 0 590 0 0 0 25 0 9 0 733853808 283201536 30397 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8223/tid=8235] ppid=8221 vsize=276564 CPUtime=5.96 cores=0-7
/proc/8223/task/8235/stat : 8235 (plingeling) R 8221 8223 8197 0 -1 4202560 4679 1251 0 0 596 0 0 0 25 0 9 0 733853808 283201536 30397 33554432000 4194304 4417452 140733232609984 18446744073709551615 4276039 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8223/tid=8236] ppid=8221 vsize=276564 CPUtime=5.96 cores=0-7
/proc/8223/task/8236/stat : 8236 (plingeling) R 8221 8223 8197 0 -1 4202560 3977 1251 0 0 596 0 0 0 25 0 9 0 733853808 283201536 30397 33554432000 4194304 4417452 140733232609984 18446744073709551615 4250852 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=8223/tid=8237] ppid=8221 vsize=276564 CPUtime=5.82 cores=0-7
/proc/8223/task/8237/stat : 8237 (plingeling) R 8221 8223 8197 0 -1 4202560 7075 1251 0 0 582 0 0 0 25 0 9 0 733853809 283201536 30397 33554432000 4194304 4417452 140733232609984 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 47.71
Current children cumulated vsize (KiB) 276564

[startup+12.7007 s]
/proc/loadavg: 4.39 4.14 4.83 9/276 8238
/proc/meminfo: memFree=24963772/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=319488 CPUtime=98.85 cores=0-7
/proc/8223/stat : 8223 (plingeling) S 8221 8223 8197 0 -1 4202496 58944 1251 0 0 9879 6 0 0 18 0 9 0 733853776 327155712 41133 33554432000 4194304 4417452 140733232609984 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/8223/statm: 79872 41133 180 55 0 77758 0
[pid=8223/tid=8230] ppid=8221 vsize=319488 CPUtime=12.18 cores=0-7
/proc/8223/task/8230/stat : 8230 (plingeling) R 8221 8223 8197 0 -1 4202560 3294 1251 0 0 1218 0 0 0 25 0 9 0 733853808 327155712 41133 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8223/tid=8231] ppid=8221 vsize=319488 CPUtime=12.37 cores=0-7
/proc/8223/task/8231/stat : 8231 (plingeling) R 8221 8223 8197 0 -1 4202560 4968 1251 0 0 1237 0 0 0 25 0 9 0 733853808 327155712 41133 33554432000 4194304 4417452 140733232609984 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8223/tid=8232] ppid=8221 vsize=319488 CPUtime=12.37 cores=0-7
/proc/8223/task/8232/stat : 8232 (plingeling) R 8221 8223 8197 0 -1 4202560 4194 1251 0 0 1237 0 0 0 25 0 9 0 733853808 327155712 41133 33554432000 4194304 4417452 140733232609984 18446744073709551615 4250757 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8223/tid=8233] ppid=8221 vsize=319488 CPUtime=12.37 cores=0-7
/proc/8223/task/8233/stat : 8233 (plingeling) R 8221 8223 8197 0 -1 4202560 4051 1251 0 0 1237 0 0 0 25 0 9 0 733853808 327155712 41133 33554432000 4194304 4417452 140733232609984 18446744073709551615 4250447 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8223/tid=8234] ppid=8221 vsize=319488 CPUtime=12.24 cores=0-7
/proc/8223/task/8234/stat : 8234 (plingeling) R 8221 8223 8197 0 -1 4202560 5952 1251 0 0 1224 0 0 0 25 0 9 0 733853808 327155712 41133 33554432000 4194304 4417452 140733232609984 18446744073709551615 4308810 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8223/tid=8235] ppid=8221 vsize=319488 CPUtime=12.36 cores=0-7
/proc/8223/task/8235/stat : 8235 (plingeling) R 8221 8223 8197 0 -1 4202560 6062 1251 0 0 1236 0 0 0 25 0 9 0 733853808 327155712 41133 33554432000 4194304 4417452 140733232609984 18446744073709551615 4252336 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8223/tid=8236] ppid=8221 vsize=319488 CPUtime=12.36 cores=0-7
/proc/8223/task/8236/stat : 8236 (plingeling) R 8221 8223 8197 0 -1 4202560 6389 1251 0 0 1236 0 0 0 25 0 9 0 733853808 327155712 41133 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364567 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=8223/tid=8237] ppid=8221 vsize=319488 CPUtime=12.23 cores=0-7
/proc/8223/task/8237/stat : 8237 (plingeling) R 8221 8223 8197 0 -1 4202560 9996 1251 0 0 1222 1 0 0 25 0 9 0 733853809 327155712 41133 33554432000 4194304 4417452 140733232609984 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 98.85
Current children cumulated vsize (KiB) 319488

[startup+25.5014 s]
/proc/loadavg: 5.19 4.33 4.88 9/276 8238
/proc/meminfo: memFree=24931812/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=354828 CPUtime=201.15 cores=0-7
/proc/8223/stat : 8223 (plingeling) S 8221 8223 8197 0 -1 4202496 92365 1251 0 0 20103 12 0 0 18 0 9 0 733853776 363343872 48925 33554432000 4194304 4417452 140733232609984 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/8223/statm: 88707 48925 180 55 0 86593 0
[pid=8223/tid=8230] ppid=8221 vsize=354828 CPUtime=24.98 cores=0-7
/proc/8223/task/8230/stat : 8230 (plingeling) R 8221 8223 8197 0 -1 4202560 4608 1251 0 0 2498 0 0 0 25 0 9 0 733853808 363343872 48925 33554432000 4194304 4417452 140733232609984 18446744073709551615 4320867 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8223/tid=8231] ppid=8221 vsize=354828 CPUtime=25.17 cores=0-7
/proc/8223/task/8231/stat : 8231 (plingeling) R 8221 8223 8197 0 -1 4202560 8978 1251 0 0 2516 1 0 0 25 0 9 0 733853808 363343872 48925 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8223/tid=8232] ppid=8221 vsize=354828 CPUtime=25.17 cores=0-7
/proc/8223/task/8232/stat : 8232 (plingeling) R 8221 8223 8197 0 -1 4202560 8553 1251 0 0 2516 1 0 0 25 0 9 0 733853808 363343872 48925 33554432000 4194304 4417452 140733232609984 18446744073709551615 4257081 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8223/tid=8233] ppid=8221 vsize=354828 CPUtime=25.17 cores=0-7
/proc/8223/task/8233/stat : 8233 (plingeling) R 8221 8223 8197 0 -1 4202560 7595 1251 0 0 2517 0 0 0 25 0 9 0 733853808 363343872 48925 33554432000 4194304 4417452 140733232609984 18446744073709551615 4236874 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8223/tid=8234] ppid=8221 vsize=354828 CPUtime=24.94 cores=0-7
/proc/8223/task/8234/stat : 8234 (plingeling) R 8221 8223 8197 0 -1 4202560 12780 1251 0 0 2492 2 0 0 25 0 9 0 733853808 363343872 48925 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364873 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8223/tid=8235] ppid=8221 vsize=354828 CPUtime=25.16 cores=0-7
/proc/8223/task/8235/stat : 8235 (plingeling) R 8221 8223 8197 0 -1 4202560 13066 1251 0 0 2515 1 0 0 25 0 9 0 733853808 363343872 48925 33554432000 4194304 4417452 140733232609984 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8223/tid=8236] ppid=8221 vsize=354828 CPUtime=25.16 cores=0-7
/proc/8223/task/8236/stat : 8236 (plingeling) R 8221 8223 8197 0 -1 4202560 8445 1251 0 0 2515 1 0 0 25 0 9 0 733853808 363343872 48925 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=8223/tid=8237] ppid=8221 vsize=354828 CPUtime=25.03 cores=0-7
/proc/8223/task/8237/stat : 8237 (plingeling) R 8221 8223 8197 0 -1 4202560 14302 1251 0 0 2502 1 0 0 25 0 9 0 733853809 363343872 48925 33554432000 4194304 4417452 140733232609984 18446744073709551615 4277628 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 201.15
Current children cumulated vsize (KiB) 354828

[startup+51.1006 s]
/proc/loadavg: 6.15 4.62 4.96 9/276 8239
/proc/meminfo: memFree=24788684/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=495760 CPUtime=405.74 cores=0-7
/proc/8223/stat : 8223 (plingeling) S 8221 8223 8197 0 -1 4202496 158997 1251 0 0 40552 22 0 0 18 0 9 0 733853776 507658240 85182 33554432000 4194304 4417452 140733232609984 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/8223/statm: 123940 85182 180 55 0 121826 0
[pid=8223/tid=8230] ppid=8221 vsize=495760 CPUtime=50.58 cores=0-7
/proc/8223/task/8230/stat : 8230 (plingeling) R 8221 8223 8197 0 -1 4202560 5176 1251 0 0 5058 0 0 0 25 0 9 0 733853808 507658240 85182 33554432000 4194304 4417452 140733232609984 18446744073709551615 4366079 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8223/tid=8231] ppid=8221 vsize=495760 CPUtime=50.76 cores=0-7
/proc/8223/task/8231/stat : 8231 (plingeling) R 8221 8223 8197 0 -1 4202560 20177 1251 0 0 5074 2 0 0 25 0 9 0 733853808 507658240 85182 33554432000 4194304 4417452 140733232609984 18446744073709551615 4357842 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8223/tid=8232] ppid=8221 vsize=495760 CPUtime=50.77 cores=0-7

################
# More data... #
################

/proc/8223/task/8233/stat : 8233 (plingeling) R 8221 8223 8197 0 -1 4202560 1752808 1251 0 0 465925 285 0 0 25 0 9 0 733853808 2965626880 685272 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8223/tid=8234] ppid=8221 vsize=2896120 CPUtime=4654.45 cores=0-7
/proc/8223/task/8234/stat : 8234 (plingeling) R 8221 8223 8197 0 -1 4202560 651679 1251 0 0 465335 110 0 0 25 0 9 0 733853808 2965626880 685272 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8223/tid=8235] ppid=8221 vsize=2896120 CPUtime=4662.08 cores=0-7
/proc/8223/task/8235/stat : 8235 (plingeling) R 8221 8223 8197 0 -1 4202560 1128968 1251 0 0 466011 197 0 0 25 0 9 0 733853808 2965626880 685272 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8223/tid=8236] ppid=8221 vsize=2896120 CPUtime=4661.3 cores=0-7
/proc/8223/task/8236/stat : 8236 (plingeling) R 8221 8223 8197 0 -1 4202560 1173140 1251 0 0 465941 189 0 0 25 0 9 0 733853808 2965626880 685272 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=8223/tid=8237] ppid=8221 vsize=2896120 CPUtime=4661.93 cores=0-7
/proc/8223/task/8237/stat : 8237 (plingeling) R 8221 8223 8197 0 -1 4202560 487634 1251 0 0 466117 76 0 0 25 0 9 0 733853809 2965626880 685272 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37256.6
Current children cumulated vsize (KiB) 2896120

[startup+4722.31 s]
/proc/loadavg: 8.11 8.06 8.00 9/276 8380
/proc/meminfo: memFree=22496268/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=2782944 CPUtime=37736.1 cores=0-7
/proc/8223/stat : 8223 (plingeling) S 8221 8223 8197 0 -1 4202496 8377499 1251 0 0 3772206 1408 0 0 18 0 9 0 733853776 2849734656 656978 33554432000 4194304 4417452 140733232609984 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/8223/statm: 695736 656978 180 55 0 693622 0
[pid=8223/tid=8230] ppid=8221 vsize=2782944 CPUtime=4721.9 cores=0-7
/proc/8223/task/8230/stat : 8230 (plingeling) R 8221 8223 8197 0 -1 4202560 1144100 1251 0 0 472004 186 0 0 25 0 9 0 733853808 2849734656 656978 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8223/tid=8231] ppid=8221 vsize=2782944 CPUtime=4722.09 cores=0-7
/proc/8223/task/8231/stat : 8231 (plingeling) R 8221 8223 8197 0 -1 4202560 971395 1251 0 0 472024 185 0 0 25 0 9 0 733853808 2849734656 656978 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8223/tid=8232] ppid=8221 vsize=2782944 CPUtime=4689.84 cores=0-7
/proc/8223/task/8232/stat : 8232 (plingeling) R 8221 8223 8197 0 -1 4202560 965140 1251 0 0 468823 161 0 0 25 0 9 0 733853808 2849734656 656978 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8223/tid=8233] ppid=8221 vsize=2782944 CPUtime=4722.11 cores=0-7
/proc/8223/task/8233/stat : 8233 (plingeling) R 8221 8223 8197 0 -1 4202560 1778018 1251 0 0 471922 289 0 0 25 0 9 0 733853808 2849734656 656978 33554432000 4194304 4417452 140733232609984 18446744073709551615 4356783 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8223/tid=8234] ppid=8221 vsize=2782944 CPUtime=4714.46 cores=0-7
/proc/8223/task/8234/stat : 8234 (plingeling) R 8221 8223 8197 0 -1 4202560 670983 1251 0 0 471332 114 0 0 25 0 9 0 733853808 2849734656 656978 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8223/tid=8235] ppid=8221 vsize=2782944 CPUtime=4722.1 cores=0-7
/proc/8223/task/8235/stat : 8235 (plingeling) R 8221 8223 8197 0 -1 4202560 1128968 1251 0 0 472013 197 0 0 25 0 9 0 733853808 2849734656 656978 33554432000 4194304 4417452 140733232609984 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8223/tid=8236] ppid=8221 vsize=2782944 CPUtime=4721.3 cores=0-7
/proc/8223/task/8236/stat : 8236 (plingeling) R 8221 8223 8197 0 -1 4202560 1192198 1251 0 0 471939 191 0 0 25 0 9 0 733853808 2849734656 656978 33554432000 4194304 4417452 140733232609984 18446744073709551615 4353557 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=8223/tid=8237] ppid=8221 vsize=2782944 CPUtime=4721.94 cores=0-7
/proc/8223/task/8237/stat : 8237 (plingeling) R 8221 8223 8197 0 -1 4202560 512659 1251 0 0 472115 79 0 0 25 0 9 0 733853809 2849734656 656978 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37736.1
Current children cumulated vsize (KiB) 2782944

[startup+4782.3 s]
/proc/loadavg: 8.04 8.05 8.00 9/276 8382
/proc/meminfo: memFree=22090752/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=3188280 CPUtime=38215.6 cores=0-7
/proc/8223/stat : 8223 (plingeling) S 8221 8223 8197 0 -1 4202496 8534098 1251 0 0 3820112 1444 0 0 18 0 9 0 733853776 3264798720 758312 33554432000 4194304 4417452 140733232609984 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/8223/statm: 797070 758312 180 55 0 794956 0
[pid=8223/tid=8230] ppid=8221 vsize=3188280 CPUtime=4781.9 cores=0-7
/proc/8223/task/8230/stat : 8230 (plingeling) R 8221 8223 8197 0 -1 4202560 1156541 1251 0 0 478002 188 0 0 25 0 9 0 733853808 3264798720 758312 33554432000 4194304 4417452 140733232609984 18446744073709551615 4365103 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8223/tid=8231] ppid=8221 vsize=3188280 CPUtime=4782.08 cores=0-7
/proc/8223/task/8231/stat : 8231 (plingeling) R 8221 8223 8197 0 -1 4202560 982360 1251 0 0 478022 186 0 0 25 0 9 0 733853808 3264798720 758312 33554432000 4194304 4417452 140733232609984 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8223/tid=8232] ppid=8221 vsize=3188280 CPUtime=4749.33 cores=0-7
/proc/8223/task/8232/stat : 8232 (plingeling) R 8221 8223 8197 0 -1 4202560 969636 1251 0 0 474771 162 0 0 25 0 9 0 733853808 3264798720 758312 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8223/tid=8233] ppid=8221 vsize=3188280 CPUtime=4782.1 cores=0-7
/proc/8223/task/8233/stat : 8233 (plingeling) R 8221 8223 8197 0 -1 4202560 1803956 1251 0 0 477918 292 0 0 25 0 9 0 733853808 3264798720 758312 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364958 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8223/tid=8234] ppid=8221 vsize=3188280 CPUtime=4774.46 cores=0-7
/proc/8223/task/8234/stat : 8234 (plingeling) R 8221 8223 8197 0 -1 4202560 674041 1251 0 0 477331 115 0 0 25 0 9 0 733853808 3264798720 758312 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8223/tid=8235] ppid=8221 vsize=3188280 CPUtime=4782.09 cores=0-7
/proc/8223/task/8235/stat : 8235 (plingeling) R 8221 8223 8197 0 -1 4202560 1182302 1251 0 0 477996 213 0 0 25 0 9 0 733853808 3264798720 758312 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8223/tid=8236] ppid=8221 vsize=3188280 CPUtime=4781.29 cores=0-7
/proc/8223/task/8236/stat : 8236 (plingeling) R 8221 8223 8197 0 -1 4202560 1213978 1251 0 0 477931 198 0 0 25 0 9 0 733853808 3264798720 758312 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=8223/tid=8237] ppid=8221 vsize=3188280 CPUtime=4781.93 cores=0-7
/proc/8223/task/8237/stat : 8237 (plingeling) R 8221 8223 8197 0 -1 4202560 537246 1251 0 0 478108 85 0 0 25 0 9 0 733853809 3264798720 758312 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38215.6
Current children cumulated vsize (KiB) 3188280

[startup+4842.3 s]
/proc/loadavg: 8.05 8.05 8.00 9/276 8384
/proc/meminfo: memFree=23170632/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=2110624 CPUtime=38695.1 cores=0-7
/proc/8223/stat : 8223 (plingeling) S 8221 8223 8197 0 -1 4202496 8693578 1251 0 0 3868036 1471 0 0 18 0 9 0 733853776 2161278976 488898 33554432000 4194304 4417452 140733232609984 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/8223/statm: 527656 488898 180 55 0 525542 0
[pid=8223/tid=8230] ppid=8221 vsize=2110624 CPUtime=4841.9 cores=0-7
/proc/8223/task/8230/stat : 8230 (plingeling) R 8221 8223 8197 0 -1 4202560 1173229 1251 0 0 483998 192 0 0 25 0 9 0 733853808 2161278976 488898 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8223/tid=8231] ppid=8221 vsize=2110624 CPUtime=4842.08 cores=0-7
/proc/8223/task/8231/stat : 8231 (plingeling) R 8221 8223 8197 0 -1 4202560 984745 1251 0 0 484021 187 0 0 25 0 9 0 733853808 2161278976 488898 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8223/tid=8232] ppid=8221 vsize=2110624 CPUtime=4808.83 cores=0-7
/proc/8223/task/8232/stat : 8232 (plingeling) R 8221 8223 8197 0 -1 4202560 974136 1251 0 0 480721 162 0 0 25 0 9 0 733853808 2161278976 488898 33554432000 4194304 4417452 140733232609984 18446744073709551615 4353973 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8223/tid=8233] ppid=8221 vsize=2110624 CPUtime=4842.1 cores=0-7
/proc/8223/task/8233/stat : 8233 (plingeling) R 8221 8223 8197 0 -1 4202560 1827909 1251 0 0 483915 295 0 0 25 0 9 0 733853808 2161278976 488898 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8223/tid=8234] ppid=8221 vsize=2110624 CPUtime=4834.46 cores=0-7
/proc/8223/task/8234/stat : 8234 (plingeling) R 8221 8223 8197 0 -1 4202560 683553 1251 0 0 483330 116 0 0 25 0 9 0 733853808 2161278976 488898 33554432000 4194304 4417452 140733232609984 18446744073709551615 4366079 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8223/tid=8235] ppid=8221 vsize=2110624 CPUtime=4842.09 cores=0-7
/proc/8223/task/8235/stat : 8235 (plingeling) R 8221 8223 8197 0 -1 4202560 1230298 1251 0 0 483986 223 0 0 25 0 9 0 733853808 2161278976 488898 33554432000 4194304 4417452 140733232609984 18446744073709551615 4250969 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8223/tid=8236] ppid=8221 vsize=2110624 CPUtime=4841.29 cores=0-7
/proc/8223/task/8236/stat : 8236 (plingeling) R 8221 8223 8197 0 -1 4202560 1234023 1251 0 0 483927 202 0 0 25 0 9 0 733853808 2161278976 488898 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=8223/tid=8237] ppid=8221 vsize=2110624 CPUtime=4841.94 cores=0-7
/proc/8223/task/8237/stat : 8237 (plingeling) R 8221 8223 8197 0 -1 4202560 571647 1251 0 0 484105 89 0 0 25 0 9 0 733853809 2161278976 488898 33554432000 4194304 4417452 140733232609984 18446744073709551615 4366458 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38695.1
Current children cumulated vsize (KiB) 2110624

[startup+4902.3 s]
/proc/loadavg: 8.02 8.04 8.00 9/276 8385
/proc/meminfo: memFree=22786816/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=2369120 CPUtime=39174.6 cores=0-7
/proc/8223/stat : 8223 (plingeling) S 8221 8223 8197 0 -1 4202496 8868503 1251 0 0 3915962 1496 0 0 18 0 9 0 733853776 2425978880 553522 33554432000 4194304 4417452 140733232609984 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/8223/statm: 592280 553522 180 55 0 590166 0
[pid=8223/tid=8230] ppid=8221 vsize=2369120 CPUtime=4901.91 cores=0-7
/proc/8223/task/8230/stat : 8230 (plingeling) R 8221 8223 8197 0 -1 4202560 1173229 1251 0 0 489999 192 0 0 25 0 9 0 733853808 2425978880 553522 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8223/tid=8231] ppid=8221 vsize=2369120 CPUtime=4902.08 cores=0-7
/proc/8223/task/8231/stat : 8231 (plingeling) R 8221 8223 8197 0 -1 4202560 1000098 1251 0 0 490018 190 0 0 25 0 9 0 733853808 2425978880 553522 33554432000 4194304 4417452 140733232609984 18446744073709551615 4366024 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8223/tid=8232] ppid=8221 vsize=2369120 CPUtime=4868.34 cores=0-7
/proc/8223/task/8232/stat : 8232 (plingeling) R 8221 8223 8197 0 -1 4202560 1003293 1251 0 0 486666 168 0 0 25 0 9 0 733853808 2425978880 553522 33554432000 4194304 4417452 140733232609984 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8223/tid=8233] ppid=8221 vsize=2369120 CPUtime=4902.1 cores=0-7
/proc/8223/task/8233/stat : 8233 (plingeling) R 8221 8223 8197 0 -1 4202560 1860781 1251 0 0 489911 299 0 0 25 0 9 0 733853808 2425978880 553522 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8223/tid=8234] ppid=8221 vsize=2369120 CPUtime=4894.46 cores=0-7
/proc/8223/task/8234/stat : 8234 (plingeling) R 8221 8223 8197 0 -1 4202560 721925 1251 0 0 489326 120 0 0 25 0 9 0 733853808 2425978880 553522 33554432000 4194304 4417452 140733232609984 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8223/tid=8235] ppid=8221 vsize=2369120 CPUtime=4902.08 cores=0-7
/proc/8223/task/8235/stat : 8235 (plingeling) R 8221 8223 8197 0 -1 4202560 1235258 1251 0 0 489985 223 0 0 25 0 9 0 733853808 2425978880 553522 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8223/tid=8236] ppid=8221 vsize=2369120 CPUtime=4901.27 cores=0-7
/proc/8223/task/8236/stat : 8236 (plingeling) R 8221 8223 8197 0 -1 4202560 1256617 1251 0 0 489923 204 0 0 25 0 9 0 733853808 2425978880 553522 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=8223/tid=8237] ppid=8221 vsize=2369120 CPUtime=4901.94 cores=0-7
/proc/8223/task/8237/stat : 8237 (plingeling) R 8221 8223 8197 0 -1 4202560 603264 1251 0 0 490100 94 0 0 25 0 9 0 733853809 2425978880 553522 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39174.6
Current children cumulated vsize (KiB) 2369120

[startup+4962.3 s]
/proc/loadavg: 8.04 8.05 8.00 9/276 8387
/proc/meminfo: memFree=23055968/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=2224944 CPUtime=39654.1 cores=0-7
/proc/8223/stat : 8223 (plingeling) S 8221 8223 8197 0 -1 4202496 8985439 1251 0 0 3963891 1518 0 0 18 0 9 0 733853776 2278342656 517478 33554432000 4194304 4417452 140733232609984 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/8223/statm: 556236 517478 180 55 0 554122 0
[pid=8223/tid=8230] ppid=8221 vsize=2224944 CPUtime=4961.9 cores=0-7
/proc/8223/task/8230/stat : 8230 (plingeling) R 8221 8223 8197 0 -1 4202560 1199726 1251 0 0 495991 199 0 0 25 0 9 0 733853808 2278342656 517478 33554432000 4194304 4417452 140733232609984 18446744073709551615 4320867 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8223/tid=8231] ppid=8221 vsize=2224944 CPUtime=4962.09 cores=0-7
/proc/8223/task/8231/stat : 8231 (plingeling) R 8221 8223 8197 0 -1 4202560 1001240 1251 0 0 496018 191 0 0 25 0 9 0 733853808 2278342656 517478 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8223/tid=8232] ppid=8221 vsize=2224944 CPUtime=4927.85 cores=0-7
/proc/8223/task/8232/stat : 8232 (plingeling) R 8221 8223 8197 0 -1 4202560 1015052 1251 0 0 492616 169 0 0 25 0 9 0 733853808 2278342656 517478 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8223/tid=8233] ppid=8221 vsize=2224944 CPUtime=4962.11 cores=0-7
/proc/8223/task/8233/stat : 8233 (plingeling) R 8221 8223 8197 0 -1 4202560 1880880 1251 0 0 495910 301 0 0 25 0 9 0 733853808 2278342656 517478 33554432000 4194304 4417452 140733232609984 18446744073709551615 4353561 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8223/tid=8234] ppid=8221 vsize=2224944 CPUtime=4954.46 cores=0-7
/proc/8223/task/8234/stat : 8234 (plingeling) R 8221 8223 8197 0 -1 4202560 739303 1251 0 0 495324 122 0 0 25 0 9 0 733853808 2278342656 517478 33554432000 4194304 4417452 140733232609984 18446744073709551615 4306324 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8223/tid=8235] ppid=8221 vsize=2224944 CPUtime=4962.09 cores=0-7
/proc/8223/task/8235/stat : 8235 (plingeling) R 8221 8223 8197 0 -1 4202560 1240548 1251 0 0 495985 224 0 0 25 0 9 0 733853808 2278342656 517478 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8223/tid=8236] ppid=8221 vsize=2224944 CPUtime=4961.26 cores=0-7
/proc/8223/task/8236/stat : 8236 (plingeling) R 8221 8223 8197 0 -1 4202560 1282876 1251 0 0 495915 211 0 0 25 0 9 0 733853808 2278342656 517478 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=8223/tid=8237] ppid=8221 vsize=2224944 CPUtime=4961.94 cores=0-7
/proc/8223/task/8237/stat : 8237 (plingeling) R 8221 8223 8197 0 -1 4202560 611776 1251 0 0 496098 96 0 0 25 0 9 0 733853809 2278342656 517478 33554432000 4194304 4417452 140733232609984 18446744073709551615 4358158 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39654.1
Current children cumulated vsize (KiB) 2224944



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5005.6 s]
/proc/loadavg: 8.02 8.04 8.00 9/276 8388
/proc/meminfo: memFree=23203124/32951124 swapFree=67083432/67111528
[pid=8223] ppid=8221 vsize=2078148 CPUtime=40000.2 cores=0-7
/proc/8223/stat : 8223 (plingeling) S 8221 8223 8197 0 -1 4202496 9060724 1251 0 0 3998492 1528 0 0 18 0 9 0 733853776 2128023552 480779 33554432000 4194304 4417452 140733232609984 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/8223/statm: 519537 480779 180 55 0 517423 0
[pid=8223/tid=8230] ppid=8221 vsize=2078148 CPUtime=5005.21 cores=0-7
/proc/8223/task/8230/stat : 8230 (plingeling) R 8221 8223 8197 0 -1 4202560 1199726 1251 0 0 500322 199 0 0 25 0 9 0 733853808 2128023552 480779 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8223/tid=8231] ppid=8221 vsize=2078148 CPUtime=5005.4 cores=0-7
/proc/8223/task/8231/stat : 8231 (plingeling) R 8221 8223 8197 0 -1 4202560 1004666 1251 0 0 500349 191 0 0 25 0 9 0 733853808 2128023552 480779 33554432000 4194304 4417452 140733232609984 18446744073709551615 4365959 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8223/tid=8232] ppid=8221 vsize=2078148 CPUtime=4970.79 cores=0-7
/proc/8223/task/8232/stat : 8232 (plingeling) R 8221 8223 8197 0 -1 4202560 1021658 1251 0 0 496910 169 0 0 25 0 9 0 733853808 2128023552 480779 33554432000 4194304 4417452 140733232609984 18446744073709551615 4365023 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8223/tid=8233] ppid=8221 vsize=2078148 CPUtime=5005.41 cores=0-7
/proc/8223/task/8233/stat : 8233 (plingeling) R 8221 8223 8197 0 -1 4202560 1907467 1251 0 0 500237 304 0 0 25 0 9 0 733853808 2128023552 480779 33554432000 4194304 4417452 140733232609984 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8223/tid=8234] ppid=8221 vsize=2078148 CPUtime=4997.78 cores=0-7
/proc/8223/task/8234/stat : 8234 (plingeling) R 8221 8223 8197 0 -1 4202560 752682 1251 0 0 499654 124 0 0 25 0 9 0 733853808 2128023552 480779 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8223/tid=8235] ppid=8221 vsize=2078148 CPUtime=5005.4 cores=0-7
/proc/8223/task/8235/stat : 8235 (plingeling) R 8221 8223 8197 0 -1 4202560 1240548 1251 0 0 500316 224 0 0 25 0 9 0 733853808 2128023552 480779 33554432000 4194304 4417452 140733232609984 18446744073709551615 4309175 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8223/tid=8236] ppid=8221 vsize=2078148 CPUtime=5004.57 cores=0-7
/proc/8223/task/8236/stat : 8236 (plingeling) R 8221 8223 8197 0 -1 4202560 1299651 1251 0 0 500243 214 0 0 25 0 9 0 733853808 2128023552 480779 33554432000 4194304 4417452 140733232609984 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=8223/tid=8237] ppid=8221 vsize=2078148 CPUtime=5005.26 cores=0-7
/proc/8223/task/8237/stat : 8237 (plingeling) R 8221 8223 8197 0 -1 4202560 620288 1251 0 0 500429 97 0 0 25 0 9 0 733853809 2128023552 480779 33554432000 4194304 4417452 140733232609984 18446744073709551615 4356867 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 40000.2
Current children cumulated vsize (KiB) 2078148

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5005.71
CPU time (s): 40000.4
CPU user time (s): 39985
CPU system time (s): 15.3767
CPU usage (%): 799.095
Max. virtual memory (cumulated for all children) (KiB): 3188280

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39985
system time used= 15.3767
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 9061975
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 438
involuntary context switches= 192498

runsolver used 11.7452 second user time and 29.6175 second system time

The end

Launcher Data 

Begin job on node110 at 2011-05-15 07:10:32
IDJOB=3372164
IDBENCH=20529
IDSOLVER=1851
FILE ID=node110/3372164-1305436232
RUNJOBID= node110-1305436232-8211
PBS_JOBID= 13324745
Free space on /tmp= 73636 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT07/industrial/IBM_FV_2002_13_rule_1/SAT_dat.k20.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372164-1305436232/watcher-3372164-1305436232 -o /tmp/evaluation-result-3372164-1305436232/solver-3372164-1305436232 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3372164-1305436232.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 2bee695ce4278b72ee9295631ad94252
RANDOM SEED=1650259061

node110.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25126932 kB
Buffers:        283752 kB
Cached:        4064404 kB
SwapCached:       1440 kB
Active:        4313976 kB
Inactive:      3180656 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25126932 kB
SwapTotal:    67111528 kB
SwapFree:     67083432 kB
Dirty:            3512 kB
Writeback:          24 kB
AnonPages:     3145180 kB
Mapped:          22348 kB
Slab:           245196 kB
PageTables:      20392 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 46103592 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73636 MiB
End job on node110 at 2011-05-15 08:34:00