Trace number 3371049

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.22 1203.65

General information on the benchmark

NameSAT05/crafted/sorge05/QG7a/
gensys-ukn003.shuffled-as.sat05-3843.cnf
MD5SUM59fa4424111aa94c942a4bbd786cbd51
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark1234.33
Satisfiable
(Un)Satisfiability was proved
Number of variables2969
Number of clauses18767
Sum of the clauses size134137
Maximum clause length13
Minimum clause length1
Number of clauses of size 142
Number of clauses of size 2442
Number of clauses of size 32384
Number of clauses of size 4505
Number of clauses of size 5684
Number of clauses of size over 514710

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371049-1305431944/watcher-3371049-1305431944 -o /tmp/evaluation-result-3371049-1305431944/solver-3371049-1305431944 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371049-1305431944.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.38 4.52 6.18 3/352 12273
/proc/meminfo: memFree=26168232/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=8728 CPUtime=0 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 215 0 0 0 0 0 0 0 22 0 1 0 654796987 8937472 146 33554432000 4194304 4417452 140736488152160 18446744073709551615 216634515200 0 0 4096 0 18446744071562362840 0 0 17 4 0 0 0
/proc/12273/statm: 2182 146 122 55 0 68 0

[startup+0.032696 s]
/proc/loadavg: 3.38 4.52 6.18 3/352 12273
/proc/meminfo: memFree=26168232/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=12564 CPUtime=0.01 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) R 12271 12273 11874 0 -1 4202496 1251 1249 0 1 1 0 0 0 22 0 1 0 654796987 12865536 1165 33554432000 4194304 4417452 140736488152160 18446744073709551615 4209589 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/12273/statm: 3141 1165 149 55 0 1027 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 12564

[startup+0.100655 s]
/proc/loadavg: 3.38 4.52 6.18 3/352 12273
/proc/meminfo: memFree=26168232/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=56120 CPUtime=0.2 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 2642 1249 0 1 20 0 0 0 22 0 5 0 654796987 57466880 1831 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 14030 1831 177 55 0 11916 0
Current children cumulated CPU time (s) 0.2
Current children cumulated vsize (KiB) 56120

[startup+0.300617 s]
/proc/loadavg: 3.38 4.52 6.18 3/352 12273
/proc/meminfo: memFree=26168232/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=59388 CPUtime=0.99 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 3640 1249 0 1 99 0 0 0 22 0 5 0 654796987 60813312 2651 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 14847 2651 180 55 0 12733 0
Current children cumulated CPU time (s) 0.99
Current children cumulated vsize (KiB) 59388

[startup+0.700516 s]
/proc/loadavg: 3.38 4.52 6.18 3/352 12273
/proc/meminfo: memFree=26168232/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=64316 CPUtime=2.59 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 5578 1249 0 1 259 0 0 0 22 0 5 0 654796987 65859584 3910 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 16079 3910 180 55 0 13965 0
Current children cumulated CPU time (s) 2.59
Current children cumulated vsize (KiB) 64316

[startup+1.50038 s]
/proc/loadavg: 3.38 4.52 6.18 10/358 12324
/proc/meminfo: memFree=26139652/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=133964 CPUtime=5.78 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 10942 1249 0 1 577 1 0 0 22 0 5 0 654796987 137179136 4916 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 33491 4916 180 55 0 31377 0
[pid=12273/tid=12285] ppid=12271 vsize=133964 CPUtime=1.44 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 1620 1249 0 1 144 0 0 0 25 0 5 0 654796993 137179136 4916 33554432000 4194304 4417452 140736488152160 18446744073709551615 4270927 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=133964 CPUtime=1.43 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 2083 1249 0 1 143 0 0 0 25 0 5 0 654796993 137179136 4916 33554432000 4194304 4417452 140736488152160 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=133964 CPUtime=1.42 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 2262 1249 0 1 142 0 0 0 25 0 5 0 654796993 137179136 4916 33554432000 4194304 4417452 140736488152160 18446744073709551615 4250848 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=133964 CPUtime=1.43 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 2690 1249 0 1 143 0 0 0 25 0 5 0 654796993 137179136 4916 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364480 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.78
Current children cumulated vsize (KiB) 133964

[startup+3.10109 s]
/proc/loadavg: 3.38 4.52 6.18 9/358 12324
/proc/meminfo: memFree=26124152/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=141616 CPUtime=12.17 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 21979 1249 0 1 1214 3 0 0 22 0 5 0 654796987 145014784 6856 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 35404 6856 180 55 0 33290 0
[pid=12273/tid=12285] ppid=12271 vsize=141616 CPUtime=3.03 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 4828 1249 0 1 303 0 0 0 25 0 5 0 654796993 145014784 6856 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=141616 CPUtime=3.02 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 5089 1249 0 1 302 0 0 0 25 0 5 0 654796993 145014784 6856 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=141616 CPUtime=3 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 4614 1249 0 1 300 0 0 0 25 0 5 0 654796993 145014784 6856 33554432000 4194304 4417452 140736488152160 18446744073709551615 4352320 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=141616 CPUtime=3.03 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 5161 1249 0 1 302 1 0 0 25 0 5 0 654796993 145014784 6856 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 12.17
Current children cumulated vsize (KiB) 141616

[startup+6.30053 s]
/proc/loadavg: 3.75 4.58 6.19 9/358 12324
/proc/meminfo: memFree=26106928/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=158800 CPUtime=24.93 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 36462 1249 0 1 2487 6 0 0 22 0 5 0 654796987 162611200 11158 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 39700 11158 180 55 0 37586 0
[pid=12273/tid=12285] ppid=12271 vsize=158800 CPUtime=6.23 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 9322 1249 0 1 622 1 0 0 25 0 5 0 654796993 162611200 11158 33554432000 4194304 4417452 140736488152160 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=158800 CPUtime=6.23 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 8144 1249 0 1 622 1 0 0 25 0 5 0 654796993 162611200 11158 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=158800 CPUtime=6.16 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 8518 1249 0 1 615 1 0 0 25 0 5 0 654796993 162611200 11158 33554432000 4194304 4417452 140736488152160 18446744073709551615 4366082 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=158800 CPUtime=6.23 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 8191 1249 0 1 622 1 0 0 25 0 5 0 654796993 162611200 11158 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364984 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 24.93
Current children cumulated vsize (KiB) 158800

[startup+12.7004 s]
/proc/loadavg: 4.09 4.63 6.20 9/358 12324
/proc/meminfo: memFree=26068808/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=177164 CPUtime=50.45 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 57466 1249 0 1 5036 9 0 0 22 0 5 0 654796987 181415936 15718 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 44291 15718 180 55 0 42177 0
[pid=12273/tid=12285] ppid=12271 vsize=177164 CPUtime=12.63 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 13447 1249 0 1 1262 1 0 0 25 0 5 0 654796993 181415936 15718 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=177164 CPUtime=12.63 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 14271 1249 0 1 1261 2 0 0 25 0 5 0 654796993 181415936 15718 33554432000 4194304 4417452 140736488152160 18446744073709551615 4365062 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=177164 CPUtime=12.49 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 13625 1249 0 1 1247 2 0 0 25 0 5 0 654796993 181415936 15718 33554432000 4194304 4417452 140736488152160 18446744073709551615 4265462 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=177164 CPUtime=12.63 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 13836 1249 0 1 1261 2 0 0 25 0 5 0 654796993 181415936 15718 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 50.45
Current children cumulated vsize (KiB) 177164

[startup+25.5012 s]
/proc/loadavg: 4.96 4.80 6.23 9/358 12324
/proc/meminfo: memFree=26007200/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=213912 CPUtime=101.51 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 86806 1249 0 1 10136 15 0 0 22 0 5 0 654796987 219045888 24791 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 53478 24791 180 55 0 51364 0
[pid=12273/tid=12285] ppid=12271 vsize=213912 CPUtime=25.43 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 23164 1249 0 1 2540 3 0 0 25 0 5 0 654796993 219045888 24791 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=213912 CPUtime=25.43 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 20791 1249 0 1 2540 3 0 0 25 0 5 0 654796993 219045888 24791 33554432000 4194304 4417452 140736488152160 18446744073709551615 4310184 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=213912 CPUtime=25.14 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 22016 1249 0 1 2510 4 0 0 25 0 5 0 654796993 219045888 24791 33554432000 4194304 4417452 140736488152160 18446744073709551615 4307968 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=213912 CPUtime=25.43 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 18548 1249 0 1 2540 3 0 0 25 0 5 0 654796993 219045888 24791 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 101.51
Current children cumulated vsize (KiB) 213912

[startup+51.1006 s]
/proc/loadavg: 6.00 5.05 6.27 9/358 12325
/proc/meminfo: memFree=25956692/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=263564 CPUtime=203.63 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 131606 1249 0 1 20341 22 0 0 22 0 5 0 654796987 269889536 37317 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 65891 37317 180 55 0 63777 0
[pid=12273/tid=12285] ppid=12271 vsize=263564 CPUtime=51.03 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 32684 1249 0 1 5098 5 0 0 25 0 5 0 654796993 269889536 37317 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=263564 CPUtime=51.03 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 32095 1249 0 1 5098 5 0 0 25 0 5 0 654796993 269889536 37317 33554432000 4194304 4417452 140736488152160 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=263564 CPUtime=50.46 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 32984 1249 0 1 5040 6 0 0 25 0 5 0 654796993 269889536 37317 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=263564 CPUtime=51.04 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 31556 1249 0 1 5099 5 0 0 25 0 5 0 654796993 269889536 37317 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 203.63
Current children cumulated vsize (KiB) 263564

[startup+102.313 s]
/proc/loadavg: 7.13 5.51 6.36 9/358 12327
/proc/meminfo: memFree=25871480/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=309644 CPUtime=407.91 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 194897 1249 0 1 40758 33 0 0 22 0 5 0 654796987 317075456 48838 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 77411 48838 180 55 0 75297 0
[pid=12273/tid=12285] ppid=12271 vsize=309644 CPUtime=102.25 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 48910 1249 0 1 10217 8 0 0 25 0 5 0 654796993 317075456 48838 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=309644 CPUtime=102.24 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 48617 1249 0 1 10217 7 0 0 25 0 5 0 654796993 317075456 48838 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=309644 CPUtime=101.11 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 50605 1249 0 1 10102 9 0 0 25 0 5 0 654796993 317075456 48838 33554432000 4194304 4417452 140736488152160 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=309644 CPUtime=102.24 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 44478 1249 0 1 10217 7 0 0 25 0 5 0 654796993 317075456 48838 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 407.91
Current children cumulated vsize (KiB) 309644

[startup+162.301 s]
/proc/loadavg: 7.68 5.96 6.46 9/358 12337
/proc/meminfo: memFree=25752812/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=372724 CPUtime=647.19 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 255549 1249 0 1 64676 43 0 0 22 0 5 0 654796987 381669376 64606 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 93181 64606 180 55 0 91067 0
[pid=12273/tid=12285] ppid=12271 vsize=372724 CPUtime=162.24 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 73333 1249 0 1 16212 12 0 0 25 0 5 0 654796993 381669376 64606 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=372724 CPUtime=162.23 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 57817 1249 0 1 16214 9 0 0 25 0 5 0 654796993 381669376 64606 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=372724 CPUtime=160.43 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 62364 1249 0 1 16033 10 0 0 25 0 5 0 654796993 381669376 64606 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=372724 CPUtime=162.21 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 59748 1249 0 1 16211 10 0 0 25 0 5 0 654796993 381669376 64606 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 647.19
Current children cumulated vsize (KiB) 372724


################
# More data... #
################

[pid=12273/tid=12286] ppid=12271 vsize=794488 CPUtime=702.23 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 151917 1249 0 1 70198 25 0 0 25 0 5 0 654796993 813555712 170049 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=794488 CPUtime=694.46 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 153297 1249 0 1 69421 25 0 0 25 0 5 0 654796993 813555712 170049 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=794488 CPUtime=702.23 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 116041 1249 0 1 70202 21 0 0 25 0 5 0 654796993 813555712 170049 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2800.44
Current children cumulated vsize (KiB) 794488

[startup+762.313 s]
/proc/loadavg: 8.06 7.81 7.24 9/358 12356
/proc/meminfo: memFree=25285760/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=676088 CPUtime=3039.83 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 592787 1249 0 1 303883 100 0 0 22 0 5 0 654796987 692314112 140449 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 169022 140449 180 55 0 166908 0
[pid=12273/tid=12285] ppid=12271 vsize=676088 CPUtime=761.47 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 143300 1249 0 1 76124 23 0 0 25 0 5 0 654796993 692314112 140449 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364903 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=676088 CPUtime=762.24 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 158039 1249 0 1 76198 26 0 0 25 0 5 0 654796993 692314112 140449 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=676088 CPUtime=753.8 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 153297 1249 0 1 75355 25 0 0 25 0 5 0 654796993 692314112 140449 33554432000 4194304 4417452 140736488152160 18446744073709551615 4366400 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=676088 CPUtime=762.24 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 135864 1249 0 1 76199 25 0 0 25 0 5 0 654796993 692314112 140449 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3039.83
Current children cumulated vsize (KiB) 676088

[startup+822.301 s]
/proc/loadavg: 8.05 7.86 7.29 9/358 12358
/proc/meminfo: memFree=25280012/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=663532 CPUtime=3279.11 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 601854 1249 0 1 327810 101 0 0 22 0 5 0 654796987 679456768 137310 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 165883 137310 180 55 0 163769 0
[pid=12273/tid=12285] ppid=12271 vsize=663532 CPUtime=821.46 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 143300 1249 0 1 82123 23 0 0 25 0 5 0 654796993 679456768 137310 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=663532 CPUtime=822.23 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 167106 1249 0 1 82196 27 0 0 25 0 5 0 654796993 679456768 137310 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=663532 CPUtime=813.13 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 153297 1249 0 1 81288 25 0 0 25 0 5 0 654796993 679456768 137310 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=663532 CPUtime=822.23 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 135864 1249 0 1 82198 25 0 0 25 0 5 0 654796993 679456768 137310 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3279.11
Current children cumulated vsize (KiB) 663532

[startup+882.3 s]
/proc/loadavg: 8.13 7.91 7.34 9/358 12359
/proc/meminfo: memFree=25360704/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=640904 CPUtime=3518.45 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 615211 1249 0 1 351742 103 0 0 22 0 5 0 654796987 656285696 131653 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 160226 131653 180 55 0 158112 0
[pid=12273/tid=12285] ppid=12271 vsize=640904 CPUtime=881.45 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 146882 1249 0 1 88122 23 0 0 25 0 5 0 654796993 656285696 131653 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=640904 CPUtime=882.23 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 167106 1249 0 1 88196 27 0 0 25 0 5 0 654796993 656285696 131653 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=640904 CPUtime=872.46 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 160266 1249 0 1 87220 26 0 0 25 0 5 0 654796993 656285696 131653 33554432000 4194304 4417452 140736488152160 18446744073709551615 4306349 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=640904 CPUtime=882.23 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 138670 1249 0 1 88198 25 0 0 25 0 5 0 654796993 656285696 131653 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3518.45
Current children cumulated vsize (KiB) 640904

[startup+942.301 s]
/proc/loadavg: 8.12 7.94 7.38 9/358 12361
/proc/meminfo: memFree=25297060/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=710976 CPUtime=3757.79 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 659946 1249 0 1 375671 108 0 0 22 0 5 0 654796987 728039424 149171 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 177744 149171 180 55 0 175630 0
[pid=12273/tid=12285] ppid=12271 vsize=710976 CPUtime=941.45 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 171179 1249 0 1 94119 26 0 0 25 0 5 0 654796993 728039424 149171 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=710976 CPUtime=942.23 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 176976 1249 0 1 94195 28 0 0 25 0 5 0 654796993 728039424 149171 33554432000 4194304 4417452 140736488152160 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=710976 CPUtime=931.8 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 166287 1249 0 1 93153 27 0 0 25 0 5 0 654796993 728039424 149171 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=710976 CPUtime=942.23 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 143217 1249 0 1 94198 25 0 0 25 0 5 0 654796993 728039424 149171 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3757.79
Current children cumulated vsize (KiB) 710976

[startup+1002.3 s]
/proc/loadavg: 8.23 8.00 7.43 9/358 12362
/proc/meminfo: memFree=25395588/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=642340 CPUtime=3997.13 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 670257 1249 0 1 399603 110 0 0 22 0 5 0 654796987 657756160 132012 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 160585 132012 180 55 0 158471 0
[pid=12273/tid=12285] ppid=12271 vsize=642340 CPUtime=1001.46 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 175919 1249 0 1 100119 27 0 0 25 0 5 0 654796993 657756160 132012 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=642340 CPUtime=1002.23 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 176976 1249 0 1 100195 28 0 0 25 0 5 0 654796993 657756160 132012 33554432000 4194304 4417452 140736488152160 18446744073709551615 4365075 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=642340 CPUtime=991.13 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 170686 1249 0 1 99086 27 0 0 25 0 5 0 654796993 657756160 132012 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=642340 CPUtime=1002.23 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 144389 1249 0 1 100198 25 0 0 25 0 5 0 654796993 657756160 132012 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3997.13
Current children cumulated vsize (KiB) 642340

[startup+1062.3 s]
/proc/loadavg: 8.12 8.01 7.47 9/358 12364
/proc/meminfo: memFree=25256068/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=723632 CPUtime=4236.48 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 724193 1249 0 1 423531 117 0 0 22 0 5 0 654796987 740999168 152367 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 180908 152367 180 55 0 178794 0
[pid=12273/tid=12285] ppid=12271 vsize=723632 CPUtime=1061.46 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 178334 1249 0 1 106118 28 0 0 25 0 5 0 654796993 740999168 152367 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=723632 CPUtime=1062.24 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 206830 1249 0 1 106192 32 0 0 25 0 5 0 654796993 740999168 152367 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=723632 CPUtime=1050.47 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 184398 1249 0 1 105018 29 0 0 25 0 5 0 654796993 740999168 152367 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=723632 CPUtime=1062.23 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 152344 1249 0 1 106197 26 0 0 25 0 5 0 654796993 740999168 152367 33554432000 4194304 4417452 140736488152160 18446744073709551615 4366126 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4236.48
Current children cumulated vsize (KiB) 723632

[startup+1122.3 s]
/proc/loadavg: 8.18 8.05 7.52 9/358 12366
/proc/meminfo: memFree=25168100/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=751316 CPUtime=4475.81 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 751625 1249 0 1 447460 121 0 0 22 0 5 0 654796987 769347584 159288 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 187829 159288 180 55 0 185715 0
[pid=12273/tid=12285] ppid=12271 vsize=751316 CPUtime=1121.47 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 180091 1249 0 1 112119 28 0 0 25 0 5 0 654796993 769347584 159288 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=751316 CPUtime=1122.24 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 229231 1249 0 1 112189 35 0 0 25 0 5 0 654796993 769347584 159288 33554432000 4194304 4417452 140736488152160 18446744073709551615 4365092 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=751316 CPUtime=1109.8 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 184398 1249 0 1 110951 29 0 0 25 0 5 0 654796993 769347584 159288 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=751316 CPUtime=1122.23 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 155618 1249 0 1 112196 27 0 0 25 0 5 0 654796993 769347584 159288 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4475.81
Current children cumulated vsize (KiB) 751316

[startup+1182.3 s]
/proc/loadavg: 8.07 8.04 7.55 9/358 12367
/proc/meminfo: memFree=25148964/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=769004 CPUtime=4715.16 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 769221 1249 0 1 471391 125 0 0 22 0 5 0 654796987 787460096 163678 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 192251 163678 180 55 0 190137 0
[pid=12273/tid=12285] ppid=12271 vsize=769004 CPUtime=1181.47 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 197687 1249 0 1 118116 31 0 0 25 0 5 0 654796993 787460096 163678 33554432000 4194304 4417452 140736488152160 18446744073709551615 4263050 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=769004 CPUtime=1182.24 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 229231 1249 0 1 118189 35 0 0 25 0 5 0 654796993 787460096 163678 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=769004 CPUtime=1169.14 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 184398 1249 0 1 116885 29 0 0 25 0 5 0 654796993 787460096 163678 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=769004 CPUtime=1182.24 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 155618 1249 0 1 118196 28 0 0 25 0 5 0 654796993 787460096 163678 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4715.16
Current children cumulated vsize (KiB) 769004



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.6 s]
/proc/loadavg: 8.12 8.05 7.56 9/358 12368
/proc/meminfo: memFree=25080764/32951124 swapFree=53501236/67111528
[pid=12273] ppid=12271 vsize=821488 CPUtime=4800.14 cores=0,2,4,6
/proc/12273/stat : 12273 (plingeling) S 12271 12273 11874 0 -1 4202496 782342 1249 0 1 479888 126 0 0 22 0 5 0 654796987 841203712 176799 33554432000 4194304 4417452 140736488152160 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/12273/statm: 205372 176799 180 55 0 203258 0
[pid=12273/tid=12285] ppid=12271 vsize=821488 CPUtime=1202.78 cores=0,2,4,6
/proc/12273/task/12285/stat : 12285 (plingeling) R 12271 12273 11874 0 -1 4202560 197687 1249 0 1 120247 31 0 0 25 0 5 0 654796993 841203712 176799 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12273/tid=12286] ppid=12271 vsize=821488 CPUtime=1203.55 cores=0,2,4,6
/proc/12273/task/12286/stat : 12286 (plingeling) R 12271 12273 11874 0 -1 4202560 229231 1249 0 1 120320 35 0 0 25 0 5 0 654796993 841203712 176799 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12273/tid=12287] ppid=12271 vsize=821488 CPUtime=1190.2 cores=0,2,4,6
/proc/12273/task/12287/stat : 12287 (plingeling) R 12271 12273 11874 0 -1 4202560 197519 1249 0 1 118989 31 0 0 25 0 5 0 654796993 841203712 176799 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12273/tid=12288] ppid=12271 vsize=821488 CPUtime=1203.55 cores=0,2,4,6
/proc/12273/task/12288/stat : 12288 (plingeling) R 12271 12273 11874 0 -1 4202560 155618 1249 0 1 120327 28 0 0 25 0 5 0 654796993 841203712 176799 33554432000 4194304 4417452 140736488152160 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.14
Current children cumulated vsize (KiB) 821488

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.65
CPU time (s): 4800.22
CPU user time (s): 4798.92
CPU system time (s): 1.2958
CPU usage (%): 398.806
Max. virtual memory (cumulated for all children) (KiB): 821488

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.92
system time used= 1.2958
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 783591
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 44
involuntary context switches= 22066

runsolver used 3.75043 second user time and 9.40757 second system time

The end

Launcher Data 

Begin job on node103 at 2011-05-15 05:59:04
IDJOB=3371049
IDBENCH=85808
IDSOLVER=1848
FILE ID=node103/3371049-1305431944
RUNJOBID= node103-1305431944-12248
PBS_JOBID= 13324609
Free space on /tmp= 73048 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT05/crafted/sorge05/QG7a/gensys-ukn003.shuffled-as.sat05-3843.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371049-1305431944/watcher-3371049-1305431944 -o /tmp/evaluation-result-3371049-1305431944/solver-3371049-1305431944 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371049-1305431944.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 59fa4424111aa94c942a4bbd786cbd51
RANDOM SEED=79937979

node103.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      26169420 kB
Buffers:        308968 kB
Cached:        3937516 kB
SwapCached:     340192 kB
Active:        1336168 kB
Inactive:      5176064 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      26169420 kB
SwapTotal:    67111528 kB
SwapFree:     53501236 kB
Dirty:            1320 kB
Writeback:           0 kB
AnonPages:     1935796 kB
Mapped:          10588 kB
Slab:           152052 kB
PageTables:      52940 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 78301384 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73048 MiB
End job on node103 at 2011-05-15 06:19:09