Trace number 3394876

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.3 5009.44

General information on the benchmark

NameSAT05/crafted/sorge05/QG7a/
gensys-ukn003.shuffled-as.sat05-3843.cnf
MD5SUM59fa4424111aa94c942a4bbd786cbd51
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark1118.21
Satisfiable
(Un)Satisfiability was proved
Number of variables2969
Number of clauses18767
Sum of the clauses size134137
Maximum clause length13
Minimum clause length1
Number of clauses of size 142
Number of clauses of size 2442
Number of clauses of size 32384
Number of clauses of size 4505
Number of clauses of size 5684
Number of clauses of size over 514710

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394876-1306108751/watcher-3394876-1306108751 -o /tmp/evaluation-result-3394876-1306108751/solver-3394876-1306108751 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394876-1306108751.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.08 8.06 8.01 2/436 12218
/proc/meminfo: memFree=17322692/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=11796 CPUtime=0 cores=0-7
/proc/12218/stat : 12218 (plingeling) R 12216 12218 12067 0 -1 4202496 1042 1254 0 0 0 0 0 0 25 0 1 0 801105799 12079104 971 33554432000 4194304 4417452 140734019072800 18446744073709551615 231895184365 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/12218/statm: 2949 971 148 55 0 835 0

[startup+0.0383889 s]
/proc/loadavg: 8.08 8.06 8.01 2/436 12218
/proc/meminfo: memFree=17322692/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=16080 CPUtime=0.03 cores=0-7
/proc/12218/stat : 12218 (plingeling) R 12216 12218 12067 0 -1 4202496 2162 1254 0 0 3 0 0 0 25 0 1 0 801105799 16465920 2043 33554432000 4194304 4417452 140734019072800 18446744073709551615 4210207 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/12218/statm: 4020 2043 149 55 0 1906 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 16080

[startup+0.100382 s]
/proc/loadavg: 8.08 8.06 8.01 2/436 12218
/proc/meminfo: memFree=17322692/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=83212 CPUtime=0.09 cores=0-7
/proc/12218/stat : 12218 (plingeling) R 12216 12218 12067 0 -1 4202496 4303 1254 0 0 9 0 0 0 25 0 7 0 801105799 85209088 3479 33554432000 4194304 4417452 140734019072800 18446744073709551615 231895533658 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12218/statm: 23364 3479 157 55 0 21250 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 83212

[startup+0.627101 s]
/proc/loadavg: 8.08 8.06 8.01 2/436 12218
/proc/meminfo: memFree=17322692/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=250440 CPUtime=6.5 cores=0-7
/proc/12218/stat : 12218 (plingeling) S 12216 12218 12067 0 -1 4202496 11918 1254 0 0 648 2 0 0 25 0 9 0 801105799 256450560 7424 33554432000 4194304 4417452 140734019072800 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/12218/statm: 62398 7424 180 55 0 60284 0
Current children cumulated CPU time (s) 6.5
Current children cumulated vsize (KiB) 250440

[startup+1.00513 s]
/proc/loadavg: 8.08 8.06 8.01 2/436 12218
/proc/meminfo: memFree=17322692/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=257300 CPUtime=7.3 cores=0-7
/proc/12218/stat : 12218 (plingeling) S 12216 12218 12067 0 -1 4202496 14252 1254 0 0 727 3 0 0 25 0 9 0 801105799 263475200 9366 33554432000 4194304 4417452 140734019072800 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/12218/statm: 64325 9366 180 55 0 62211 0
Current children cumulated CPU time (s) 7.3
Current children cumulated vsize (KiB) 257300

[startup+1.50104 s]
/proc/loadavg: 8.08 8.06 8.01 2/436 12218
/proc/meminfo: memFree=17322692/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=254796 CPUtime=11.27 cores=0-7
/proc/12218/stat : 12218 (plingeling) S 12216 12218 12067 0 -1 4202496 20056 1254 0 0 1124 3 0 0 25 0 9 0 801105799 260911104 8720 33554432000 4194304 4417452 140734019072800 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/12218/statm: 63699 8720 180 55 0 61585 0
Current children cumulated CPU time (s) 11.27
Current children cumulated vsize (KiB) 254796

[startup+3.10074 s]
/proc/loadavg: 8.07 8.06 8.01 9/445 12232
/proc/meminfo: memFree=17265620/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=274184 CPUtime=24.03 cores=0-7
/proc/12218/stat : 12218 (plingeling) S 12216 12218 12067 0 -1 4202496 41424 1254 0 0 2396 7 0 0 25 0 9 0 801105799 280764416 13571 33554432000 4194304 4417452 140734019072800 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/12218/statm: 68546 13571 180 55 0 66432 0
[pid=12218/tid=12225] ppid=12216 vsize=274184 CPUtime=2.99 cores=0-7
/proc/12218/task/12225/stat : 12225 (plingeling) R 12216 12218 12067 0 -1 4202560 4881 1254 0 0 299 0 0 0 25 0 9 0 801105808 280764416 13571 33554432000 4194304 4417452 140734019072800 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12218/tid=12226] ppid=12216 vsize=274184 CPUtime=2.99 cores=0-7
/proc/12218/task/12226/stat : 12226 (plingeling) R 12216 12218 12067 0 -1 4202560 4665 1254 0 0 299 0 0 0 25 0 9 0 801105808 280764416 13571 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=12218/tid=12227] ppid=12216 vsize=274184 CPUtime=2.99 cores=0-7
/proc/12218/task/12227/stat : 12227 (plingeling) R 12216 12218 12067 0 -1 4202560 4400 1254 0 0 299 0 0 0 25 0 9 0 801105808 280764416 13571 33554432000 4194304 4417452 140734019072800 18446744073709551615 4246365 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12218/tid=12228] ppid=12216 vsize=274184 CPUtime=2.99 cores=0-7
/proc/12218/task/12228/stat : 12228 (plingeling) R 12216 12218 12067 0 -1 4202560 5170 1254 0 0 299 0 0 0 25 0 9 0 801105808 280764416 13571 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12218/tid=12229] ppid=12216 vsize=274184 CPUtime=2.97 cores=0-7
/proc/12218/task/12229/stat : 12229 (plingeling) R 12216 12218 12067 0 -1 4202560 4231 1254 0 0 297 0 0 0 25 0 9 0 801105808 280764416 13571 33554432000 4194304 4417452 140734019072800 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12218/tid=12230] ppid=12216 vsize=274184 CPUtime=2.96 cores=0-7
/proc/12218/task/12230/stat : 12230 (plingeling) R 12216 12218 12067 0 -1 4202560 5204 1254 0 0 295 1 0 0 24 0 9 0 801105809 280764416 13571 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12218/tid=12231] ppid=12216 vsize=274184 CPUtime=2.98 cores=0-7
/proc/12218/task/12231/stat : 12231 (plingeling) R 12216 12218 12067 0 -1 4202560 4516 1254 0 0 298 0 0 0 25 0 9 0 801105809 280764416 13571 33554432000 4194304 4417452 140734019072800 18446744073709551615 4365023 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12218/tid=12232] ppid=12216 vsize=274184 CPUtime=2.98 cores=0-7
/proc/12218/task/12232/stat : 12232 (plingeling) R 12216 12218 12067 0 -1 4202560 4065 1254 0 0 298 0 0 0 24 0 9 0 801105809 280764416 13571 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 24.03
Current children cumulated vsize (KiB) 274184

[startup+6.30115 s]
/proc/loadavg: 8.07 8.06 8.01 9/445 12232
/proc/meminfo: memFree=17254296/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=294048 CPUtime=49.59 cores=0-7
/proc/12218/stat : 12218 (plingeling) S 12216 12218 12067 0 -1 4202496 68993 1254 0 0 4947 12 0 0 25 0 9 0 801105799 301105152 18579 33554432000 4194304 4417452 140734019072800 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/12218/statm: 73512 18579 180 55 0 71398 0
[pid=12218/tid=12225] ppid=12216 vsize=294048 CPUtime=6.2 cores=0-7
/proc/12218/task/12225/stat : 12225 (plingeling) R 12216 12218 12067 0 -1 4202560 8158 1254 0 0 619 1 0 0 25 0 9 0 801105808 301105152 18579 33554432000 4194304 4417452 140734019072800 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12218/tid=12226] ppid=12216 vsize=294048 CPUtime=6.2 cores=0-7
/proc/12218/task/12226/stat : 12226 (plingeling) R 12216 12218 12067 0 -1 4202560 7974 1254 0 0 619 1 0 0 25 0 9 0 801105808 301105152 18579 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=12218/tid=12227] ppid=12216 vsize=294048 CPUtime=6.19 cores=0-7
/proc/12218/task/12227/stat : 12227 (plingeling) R 12216 12218 12067 0 -1 4202560 8061 1254 0 0 618 1 0 0 25 0 9 0 801105808 301105152 18579 33554432000 4194304 4417452 140734019072800 18446744073709551615 4353559 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12218/tid=12228] ppid=12216 vsize=294048 CPUtime=6.19 cores=0-7
/proc/12218/task/12228/stat : 12228 (plingeling) R 12216 12218 12067 0 -1 4202560 8353 1254 0 0 618 1 0 0 25 0 9 0 801105808 301105152 18579 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12218/tid=12229] ppid=12216 vsize=294048 CPUtime=6.18 cores=0-7
/proc/12218/task/12229/stat : 12229 (plingeling) R 12216 12218 12067 0 -1 4202560 7465 1254 0 0 617 1 0 0 25 0 9 0 801105808 301105152 18579 33554432000 4194304 4417452 140734019072800 18446744073709551615 4365250 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12218/tid=12230] ppid=12216 vsize=294048 CPUtime=6.11 cores=0-7
/proc/12218/task/12230/stat : 12230 (plingeling) R 12216 12218 12067 0 -1 4202560 8619 1254 0 0 610 1 0 0 25 0 9 0 801105809 301105152 18579 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364675 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12218/tid=12231] ppid=12216 vsize=294048 CPUtime=6.18 cores=0-7
/proc/12218/task/12231/stat : 12231 (plingeling) R 12216 12218 12067 0 -1 4202560 8297 1254 0 0 617 1 0 0 25 0 9 0 801105809 301105152 18579 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12218/tid=12232] ppid=12216 vsize=294048 CPUtime=6.19 cores=0-7
/proc/12218/task/12232/stat : 12232 (plingeling) R 12216 12218 12067 0 -1 4202560 7774 1254 0 0 618 1 0 0 25 0 9 0 801105809 301105152 18579 33554432000 4194304 4417452 140734019072800 18446744073709551615 4354492 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 49.59
Current children cumulated vsize (KiB) 294048

[startup+12.701 s]
/proc/loadavg: 8.06 8.06 8.01 9/445 12233
/proc/meminfo: memFree=17199212/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=342304 CPUtime=100.77 cores=0-7
/proc/12218/stat : 12218 (plingeling) S 12216 12218 12067 0 -1 4202496 114432 1254 0 0 10056 21 0 0 25 0 9 0 801105799 350519296 30682 33554432000 4194304 4417452 140734019072800 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/12218/statm: 85576 30682 180 55 0 83462 0
[pid=12218/tid=12225] ppid=12216 vsize=342304 CPUtime=12.61 cores=0-7
/proc/12218/task/12225/stat : 12225 (plingeling) R 12216 12218 12067 0 -1 4202560 13047 1254 0 0 1259 2 0 0 25 0 9 0 801105808 350519296 30682 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364716 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12218/tid=12226] ppid=12216 vsize=342304 CPUtime=12.61 cores=0-7
/proc/12218/task/12226/stat : 12226 (plingeling) R 12216 12218 12067 0 -1 4202560 14291 1254 0 0 1259 2 0 0 25 0 9 0 801105808 350519296 30682 33554432000 4194304 4417452 140734019072800 18446744073709551615 4352589 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=12218/tid=12227] ppid=12216 vsize=342304 CPUtime=12.61 cores=0-7
/proc/12218/task/12227/stat : 12227 (plingeling) R 12216 12218 12067 0 -1 4202560 12971 1254 0 0 1259 2 0 0 25 0 9 0 801105808 350519296 30682 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12218/tid=12228] ppid=12216 vsize=342304 CPUtime=12.61 cores=0-7
/proc/12218/task/12228/stat : 12228 (plingeling) R 12216 12218 12067 0 -1 4202560 14605 1254 0 0 1258 3 0 0 25 0 9 0 801105808 350519296 30682 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12218/tid=12229] ppid=12216 vsize=342304 CPUtime=12.6 cores=0-7
/proc/12218/task/12229/stat : 12229 (plingeling) R 12216 12218 12067 0 -1 4202560 12836 1254 0 0 1258 2 0 0 25 0 9 0 801105808 350519296 30682 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12218/tid=12230] ppid=12216 vsize=342304 CPUtime=12.43 cores=0-7
/proc/12218/task/12230/stat : 12230 (plingeling) R 12216 12218 12067 0 -1 4202560 14157 1254 0 0 1241 2 0 0 25 0 9 0 801105809 350519296 30682 33554432000 4194304 4417452 140734019072800 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12218/tid=12231] ppid=12216 vsize=342304 CPUtime=12.6 cores=0-7
/proc/12218/task/12231/stat : 12231 (plingeling) R 12216 12218 12067 0 -1 4202560 14415 1254 0 0 1258 2 0 0 25 0 9 0 801105809 350519296 30682 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12218/tid=12232] ppid=12216 vsize=342304 CPUtime=12.6 cores=0-7
/proc/12218/task/12232/stat : 12232 (plingeling) R 12216 12218 12067 0 -1 4202560 13818 1254 0 0 1258 2 0 0 25 0 9 0 801105809 350519296 30682 33554432000 4194304 4417452 140734019072800 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 100.77
Current children cumulated vsize (KiB) 342304

[startup+25.5006 s]
/proc/loadavg: 8.12 8.07 8.01 9/445 12233
/proc/meminfo: memFree=17109376/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=419732 CPUtime=202.92 cores=0-7
/proc/12218/stat : 12218 (plingeling) S 12216 12218 12067 0 -1 4202496 172593 1254 0 0 20257 35 0 0 25 0 9 0 801105799 429805568 50041 33554432000 4194304 4417452 140734019072800 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/12218/statm: 104933 50041 180 55 0 102819 0
[pid=12218/tid=12225] ppid=12216 vsize=421672 CPUtime=25.4 cores=0-7
/proc/12218/task/12225/stat : 12225 (plingeling) R 12216 12218 12067 0 -1 4202560 21429 1254 0 0 2536 4 0 0 25 0 9 0 801105808 431792128 50044 33554432000 4194304 4417452 140734019072800 18446744073709551615 231895184447 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12218/tid=12226] ppid=12216 vsize=421672 CPUtime=25.39 cores=0-7
/proc/12218/task/12226/stat : 12226 (plingeling) R 12216 12218 12067 0 -1 4202560 20996 1254 0 0 2536 3 0 0 25 0 9 0 801105808 431792128 50058 33554432000 4194304 4417452 140734019072800 18446744073709551615 4309982 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=12218/tid=12227] ppid=12216 vsize=421672 CPUtime=25.39 cores=0-7
/proc/12218/task/12227/stat : 12227 (plingeling) R 12216 12218 12067 0 -1 4202560 21249 1254 0 0 2535 4 0 0 25 0 9 0 801105808 431792128 50073 33554432000 4194304 4417452 140734019072800 18446744073709551615 4309221 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12218/tid=12228] ppid=12216 vsize=421672 CPUtime=25.39 cores=0-7
/proc/12218/task/12228/stat : 12228 (plingeling) R 12216 12218 12067 0 -1 4202560 20229 1254 0 0 2535 4 0 0 25 0 9 0 801105808 431792128 50088 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12218/tid=12229] ppid=12216 vsize=421672 CPUtime=25.38 cores=0-7
/proc/12218/task/12229/stat : 12229 (plingeling) R 12216 12218 12067 0 -1 4202560 21846 1254 0 0 2534 4 0 0 25 0 9 0 801105808 431792128 50103 33554432000 4194304 4417452 140734019072800 18446744073709551615 4310641 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12218/tid=12230] ppid=12216 vsize=421672 CPUtime=25.03 cores=0-7
/proc/12218/task/12230/stat : 12230 (plingeling) R 12216 12218 12067 0 -1 4202560 21135 1254 0 0 2500 3 0 0 25 0 9 0 801105809 431792128 50119 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12218/tid=12231] ppid=12216 vsize=421672 CPUtime=25.39 cores=0-7
/proc/12218/task/12231/stat : 12231 (plingeling) R 12216 12218 12067 0 -1 4202560 19524 1254 0 0 2535 4 0 0 25 0 9 0 801105809 431792128 50133 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12218/tid=12232] ppid=12216 vsize=421672 CPUtime=25.39 cores=0-7
/proc/12218/task/12232/stat : 12232 (plingeling) R 12216 12218 12067 0 -1 4202560 21895 1254 0 0 2534 5 0 0 25 0 9 0 801105809 431792128 50149 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 202.92
Current children cumulated vsize (KiB) 419732

[startup+51.1008 s]
/proc/loadavg: 8.08 8.07 8.01 9/445 12234
/proc/meminfo: memFree=17039036/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=515676 CPUtime=407.34 cores=0-7
/proc/12218/stat : 12218 (plingeling) S 12216 12218 12067 0 -1 4202496 266625 1254 0 0 40681 53 0 0 25 0 9 0 801105799 528052224 74072 33554432000 4194304 4417452 140734019072800 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/12218/statm: 128919 74072 180 55 0 126805 0
[pid=12218/tid=12225] ppid=12216 vsize=515676 CPUtime=50.99 cores=0-7
/proc/12218/task/12225/stat : 12225 (plingeling) R 12216 12218 12067 0 -1 4202560 31890 1254 0 0 5093 6 0 0 25 0 9 0 801105808 528052224 74072 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12218/tid=12226] ppid=12216 vsize=515676 CPUtime=50.99 cores=0-7
/proc/12218/task/12226/stat : 12226 (plingeling) R 12216 12218 12067 0 -1 4202560 32074 1254 0 0 5093 6 0 0 25 0 9 0 801105808 528052224 74072 33554432000 4194304 4417452 140734019072800 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=12218/tid=12227] ppid=12216 vsize=515676 CPUtime=50.99 cores=0-7
/proc/12218/task/12227/stat : 12227 (plingeling) R 12216 12218 12067 0 -1 4202560 31850 1254 0 0 5093 6 0 0 25 0 9 0 801105808 528052224 74072 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12218/tid=12228] ppid=12216 vsize=515676 CPUtime=50.99 cores=0-7
/proc/12218/task/12228/stat : 12228 (plingeling) R 12216 12218 12067 0 -1 4202560 33059 1254 0 0 5092 7 0 0 25 0 9 0 801105808 528052224 74072 33554432000 4194304 4417452 140734019072800 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12218/tid=12229] ppid=12216 vsize=515676 CPUtime=50.98 cores=0-7
/proc/12218/task/12229/stat : 12229 (plingeling) R 12216 12218 12067 0 -1 4202560 34318 1254 0 0 5092 6 0 0 25 0 9 0 801105808 528052224 74072 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12218/tid=12230] ppid=12216 vsize=515676 CPUtime=50.27 cores=0-7
/proc/12218/task/12230/stat : 12230 (plingeling) R 12216 12218 12067 0 -1 4202560 36709 1254 0 0 5021 6 0 0 25 0 9 0 801105809 528052224 74072 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12218/tid=12231] ppid=12216 vsize=515676 CPUtime=50.98 cores=0-7
/proc/12218/task/12231/stat : 12231 (plingeling) R 12216 12218 12067 0 -1 4202560 30146 1254 0 0 5093 5 0 0 25 0 9 0 801105809 528052224 74072 33554432000 4194304 4417452 140734019072800 18446744073709551615 4366079 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12218/tid=12232] ppid=12216 vsize=515676 CPUtime=50.98 cores=0-7
/proc/12218/task/12232/stat : 12232 (plingeling) R 12216 12218 12067 0 -1 4202560 32287 1254 0 0 5092 6 0 0 25 0 9 0 801105809 528052224 74072 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 407.34
Current children cumulated vsize (KiB) 515676

[startup+102.301 s]
/proc/loadavg: 8.03 8.05 8.00 9/445 12235

################
# More data... #
################


[startup+4722.3 s]
/proc/loadavg: 8.06 8.07 8.01 9/445 12379
/proc/meminfo: memFree=14302196/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=3229224 CPUtime=37708.2 cores=0-7
/proc/12218/stat : 12218 (plingeling) S 12216 12218 12067 0 -1 4202496 3909172 1254 0 0 3770029 788 0 0 25 0 9 0 801105799 3306725376 753594 33554432000 4194304 4417452 140734019072800 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/12218/statm: 807306 753594 180 55 0 805192 0
[pid=12218/tid=12225] ppid=12216 vsize=3229224 CPUtime=4722.33 cores=0-7
/proc/12218/task/12225/stat : 12225 (plingeling) R 12216 12218 12067 0 -1 4202560 534567 1254 0 0 472130 103 0 0 25 0 9 0 801105808 3306725376 753594 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12218/tid=12226] ppid=12216 vsize=3229224 CPUtime=4722.32 cores=0-7
/proc/12218/task/12226/stat : 12226 (plingeling) R 12216 12218 12067 0 -1 4202560 536604 1254 0 0 472126 106 0 0 25 0 9 0 801105808 3306725376 753594 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=12218/tid=12227] ppid=12216 vsize=3229224 CPUtime=4721.52 cores=0-7
/proc/12218/task/12227/stat : 12227 (plingeling) R 12216 12218 12067 0 -1 4202560 541155 1254 0 0 472051 101 0 0 25 0 9 0 801105808 3306725376 753594 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364809 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12218/tid=12228] ppid=12216 vsize=3229224 CPUtime=4721.12 cores=0-7
/proc/12218/task/12228/stat : 12228 (plingeling) R 12216 12218 12067 0 -1 4202560 337394 1254 0 0 472035 77 0 0 25 0 9 0 801105808 3306725376 753594 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12218/tid=12229] ppid=12216 vsize=3229224 CPUtime=4722.32 cores=0-7
/proc/12218/task/12229/stat : 12229 (plingeling) R 12216 12218 12067 0 -1 4202560 453122 1254 0 0 472138 94 0 0 25 0 9 0 801105808 3306725376 753594 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12218/tid=12230] ppid=12216 vsize=3229224 CPUtime=4653.69 cores=0-7
/proc/12218/task/12230/stat : 12230 (plingeling) R 12216 12218 12067 0 -1 4202560 518885 1254 0 0 465261 108 0 0 25 0 9 0 801105809 3306725376 753594 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12218/tid=12231] ppid=12216 vsize=3229224 CPUtime=4722.35 cores=0-7
/proc/12218/task/12231/stat : 12231 (plingeling) R 12216 12218 12067 0 -1 4202560 498074 1254 0 0 472137 98 0 0 25 0 9 0 801105809 3306725376 753594 33554432000 4194304 4417452 140734019072800 18446744073709551615 4356856 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12218/tid=12232] ppid=12216 vsize=3229224 CPUtime=4722.35 cores=0-7
/proc/12218/task/12232/stat : 12232 (plingeling) R 12216 12218 12067 0 -1 4202560 485079 1254 0 0 472139 96 0 0 25 0 9 0 801105809 3306725376 753594 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37708.2
Current children cumulated vsize (KiB) 3229224

[startup+4782.3 s]
/proc/loadavg: 8.13 8.09 8.02 9/445 12381
/proc/meminfo: memFree=14354108/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=3178004 CPUtime=38187.4 cores=0-7
/proc/12218/stat : 12218 (plingeling) S 12216 12218 12067 0 -1 4202496 3931448 1254 0 0 3817943 794 0 0 25 0 9 0 801105799 3254276096 740789 33554432000 4194304 4417452 140734019072800 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/12218/statm: 794501 740789 180 55 0 792387 0
[pid=12218/tid=12225] ppid=12216 vsize=3178004 CPUtime=4782.35 cores=0-7
/proc/12218/task/12225/stat : 12225 (plingeling) R 12216 12218 12067 0 -1 4202560 534567 1254 0 0 478132 103 0 0 25 0 9 0 801105808 3254276096 740789 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12218/tid=12226] ppid=12216 vsize=3178004 CPUtime=4782.35 cores=0-7
/proc/12218/task/12226/stat : 12226 (plingeling) R 12216 12218 12067 0 -1 4202560 536604 1254 0 0 478128 107 0 0 25 0 9 0 801105808 3254276096 740789 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364690 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=12218/tid=12227] ppid=12216 vsize=3178004 CPUtime=4781.54 cores=0-7
/proc/12218/task/12227/stat : 12227 (plingeling) R 12216 12218 12067 0 -1 4202560 541155 1254 0 0 478053 101 0 0 25 0 9 0 801105808 3254276096 740789 33554432000 4194304 4417452 140734019072800 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12218/tid=12228] ppid=12216 vsize=3178004 CPUtime=4781.12 cores=0-7
/proc/12218/task/12228/stat : 12228 (plingeling) R 12216 12218 12067 0 -1 4202560 337394 1254 0 0 478035 77 0 0 25 0 9 0 801105808 3254276096 740789 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12218/tid=12229] ppid=12216 vsize=3178004 CPUtime=4782.34 cores=0-7
/proc/12218/task/12229/stat : 12229 (plingeling) R 12216 12218 12067 0 -1 4202560 453122 1254 0 0 478140 94 0 0 25 0 9 0 801105808 3254276096 740789 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12218/tid=12230] ppid=12216 vsize=3178004 CPUtime=4712.83 cores=0-7
/proc/12218/task/12230/stat : 12230 (plingeling) R 12216 12218 12067 0 -1 4202560 541161 1254 0 0 471170 113 0 0 25 0 9 0 801105809 3254276096 740789 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12218/tid=12231] ppid=12216 vsize=3178004 CPUtime=4782.37 cores=0-7
/proc/12218/task/12231/stat : 12231 (plingeling) R 12216 12218 12067 0 -1 4202560 498074 1254 0 0 478139 98 0 0 25 0 9 0 801105809 3254276096 740789 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12218/tid=12232] ppid=12216 vsize=3178004 CPUtime=4782.37 cores=0-7
/proc/12218/task/12232/stat : 12232 (plingeling) R 12216 12218 12067 0 -1 4202560 485079 1254 0 0 478141 96 0 0 25 0 9 0 801105809 3254276096 740789 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38187.4
Current children cumulated vsize (KiB) 3178004

[startup+4842.3 s]
/proc/loadavg: 8.12 8.09 8.01 9/445 12382
/proc/meminfo: memFree=14510564/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=3021484 CPUtime=38666.4 cores=0-7
/proc/12218/stat : 12218 (plingeling) S 12216 12218 12067 0 -1 4202496 3993109 1254 0 0 3865840 803 0 0 25 0 9 0 801105799 3093999616 701653 33554432000 4194304 4417452 140734019072800 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/12218/statm: 755371 701653 180 55 0 753257 0
[pid=12218/tid=12225] ppid=12216 vsize=3021484 CPUtime=4842.33 cores=0-7
/proc/12218/task/12225/stat : 12225 (plingeling) R 12216 12218 12067 0 -1 4202560 534567 1254 0 0 484130 103 0 0 25 0 9 0 801105808 3093999616 701653 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12218/tid=12226] ppid=12216 vsize=3021484 CPUtime=4842.33 cores=0-7
/proc/12218/task/12226/stat : 12226 (plingeling) R 12216 12218 12067 0 -1 4202560 566846 1254 0 0 484121 112 0 0 25 0 9 0 801105808 3093999616 701653 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=12218/tid=12227] ppid=12216 vsize=3021484 CPUtime=4841.52 cores=0-7
/proc/12218/task/12227/stat : 12227 (plingeling) R 12216 12218 12067 0 -1 4202560 560049 1254 0 0 484048 104 0 0 25 0 9 0 801105808 3093999616 701653 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12218/tid=12228] ppid=12216 vsize=3021484 CPUtime=4841.1 cores=0-7
/proc/12218/task/12228/stat : 12228 (plingeling) R 12216 12218 12067 0 -1 4202560 349919 1254 0 0 484031 79 0 0 25 0 9 0 801105808 3093999616 701653 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12218/tid=12229] ppid=12216 vsize=3021484 CPUtime=4842.33 cores=0-7
/proc/12218/task/12229/stat : 12229 (plingeling) R 12216 12218 12067 0 -1 4202560 453122 1254 0 0 484139 94 0 0 25 0 9 0 801105808 3093999616 701653 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12218/tid=12230] ppid=12216 vsize=3021484 CPUtime=4771.95 cores=0-7
/proc/12218/task/12230/stat : 12230 (plingeling) R 12216 12218 12067 0 -1 4202560 541161 1254 0 0 477082 113 0 0 25 0 9 0 801105809 3093999616 701653 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12218/tid=12231] ppid=12216 vsize=3021484 CPUtime=4842.35 cores=0-7
/proc/12218/task/12231/stat : 12231 (plingeling) R 12216 12218 12067 0 -1 4202560 498074 1254 0 0 484137 98 0 0 25 0 9 0 801105809 3093999616 701653 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12218/tid=12232] ppid=12216 vsize=3021484 CPUtime=4842.35 cores=0-7
/proc/12218/task/12232/stat : 12232 (plingeling) R 12216 12218 12067 0 -1 4202560 485079 1254 0 0 484139 96 0 0 25 0 9 0 801105809 3093999616 701653 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38666.4
Current children cumulated vsize (KiB) 3021484

[startup+4902.32 s]
/proc/loadavg: 8.04 8.07 8.01 9/445 12384
/proc/meminfo: memFree=14379080/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=3152556 CPUtime=39145.7 cores=0-7
/proc/12218/stat : 12218 (plingeling) S 12216 12218 12067 0 -1 4202496 4025877 1254 0 0 3913761 807 0 0 25 0 9 0 801105799 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/12218/statm: 788139 734421 180 55 0 786025 0
[pid=12218/tid=12225] ppid=12216 vsize=3152556 CPUtime=4902.35 cores=0-7
/proc/12218/task/12225/stat : 12225 (plingeling) R 12216 12218 12067 0 -1 4202560 534567 1254 0 0 490132 103 0 0 25 0 9 0 801105808 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12218/tid=12226] ppid=12216 vsize=3152556 CPUtime=4902.34 cores=0-7
/proc/12218/task/12226/stat : 12226 (plingeling) R 12216 12218 12067 0 -1 4202560 583230 1254 0 0 490120 114 0 0 25 0 9 0 801105808 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=12218/tid=12227] ppid=12216 vsize=3152556 CPUtime=4901.54 cores=0-7
/proc/12218/task/12227/stat : 12227 (plingeling) R 12216 12218 12067 0 -1 4202560 560049 1254 0 0 490050 104 0 0 25 0 9 0 801105808 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12218/tid=12228] ppid=12216 vsize=3152556 CPUtime=4901.09 cores=0-7
/proc/12218/task/12228/stat : 12228 (plingeling) R 12216 12218 12067 0 -1 4202560 366303 1254 0 0 490029 80 0 0 25 0 9 0 801105808 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12218/tid=12229] ppid=12216 vsize=3152556 CPUtime=4902.34 cores=0-7
/proc/12218/task/12229/stat : 12229 (plingeling) R 12216 12218 12067 0 -1 4202560 453122 1254 0 0 490140 94 0 0 25 0 9 0 801105808 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12218/tid=12230] ppid=12216 vsize=3152556 CPUtime=4831.09 cores=0-7
/proc/12218/task/12230/stat : 12230 (plingeling) R 12216 12218 12067 0 -1 4202560 541161 1254 0 0 482996 113 0 0 25 0 9 0 801105809 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12218/tid=12231] ppid=12216 vsize=3152556 CPUtime=4902.37 cores=0-7
/proc/12218/task/12231/stat : 12231 (plingeling) R 12216 12218 12067 0 -1 4202560 498074 1254 0 0 490139 98 0 0 25 0 9 0 801105809 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12218/tid=12232] ppid=12216 vsize=3152556 CPUtime=4902.37 cores=0-7
/proc/12218/task/12232/stat : 12232 (plingeling) R 12216 12218 12067 0 -1 4202560 485079 1254 0 0 490141 96 0 0 25 0 9 0 801105809 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39145.7
Current children cumulated vsize (KiB) 3152556

[startup+4962.3 s]
/proc/loadavg: 8.01 8.05 8.00 9/445 12386
/proc/meminfo: memFree=14378800/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=3152556 CPUtime=39624.7 cores=0-7
/proc/12218/stat : 12218 (plingeling) S 12216 12218 12067 0 -1 4202496 4025877 1254 0 0 3961662 807 0 0 25 0 9 0 801105799 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/12218/statm: 788139 734421 180 55 0 786025 0
[pid=12218/tid=12225] ppid=12216 vsize=3152556 CPUtime=4962.34 cores=0-7
/proc/12218/task/12225/stat : 12225 (plingeling) R 12216 12218 12067 0 -1 4202560 534567 1254 0 0 496131 103 0 0 25 0 9 0 801105808 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12218/tid=12226] ppid=12216 vsize=3152556 CPUtime=4962.33 cores=0-7
/proc/12218/task/12226/stat : 12226 (plingeling) R 12216 12218 12067 0 -1 4202560 583230 1254 0 0 496119 114 0 0 25 0 9 0 801105808 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=12218/tid=12227] ppid=12216 vsize=3152556 CPUtime=4961.53 cores=0-7
/proc/12218/task/12227/stat : 12227 (plingeling) R 12216 12218 12067 0 -1 4202560 560049 1254 0 0 496049 104 0 0 25 0 9 0 801105808 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12218/tid=12228] ppid=12216 vsize=3152556 CPUtime=4961.07 cores=0-7
/proc/12218/task/12228/stat : 12228 (plingeling) R 12216 12218 12067 0 -1 4202560 366303 1254 0 0 496027 80 0 0 25 0 9 0 801105808 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12218/tid=12229] ppid=12216 vsize=3152556 CPUtime=4962.33 cores=0-7
/proc/12218/task/12229/stat : 12229 (plingeling) R 12216 12218 12067 0 -1 4202560 453122 1254 0 0 496139 94 0 0 25 0 9 0 801105808 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12218/tid=12230] ppid=12216 vsize=3152556 CPUtime=4890.21 cores=0-7
/proc/12218/task/12230/stat : 12230 (plingeling) R 12216 12218 12067 0 -1 4202560 541161 1254 0 0 488908 113 0 0 25 0 9 0 801105809 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12218/tid=12231] ppid=12216 vsize=3152556 CPUtime=4962.36 cores=0-7
/proc/12218/task/12231/stat : 12231 (plingeling) R 12216 12218 12067 0 -1 4202560 498074 1254 0 0 496138 98 0 0 25 0 9 0 801105809 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12218/tid=12232] ppid=12216 vsize=3152556 CPUtime=4962.36 cores=0-7
/proc/12218/task/12232/stat : 12232 (plingeling) R 12216 12218 12067 0 -1 4202560 485079 1254 0 0 496140 96 0 0 25 0 9 0 801105809 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39624.7
Current children cumulated vsize (KiB) 3152556



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5009.3 s]
/proc/loadavg: 8.00 8.04 8.00 9/445 12387
/proc/meminfo: memFree=14378628/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=3152556 CPUtime=40000.1 cores=0-7
/proc/12218/stat : 12218 (plingeling) S 12216 12218 12067 0 -1 4202496 4025877 1254 0 0 3999200 807 0 0 25 0 9 0 801105799 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/12218/statm: 788139 734421 180 55 0 786025 0
[pid=12218/tid=12225] ppid=12216 vsize=3152556 CPUtime=5009.35 cores=0-7
/proc/12218/task/12225/stat : 12225 (plingeling) R 12216 12218 12067 0 -1 4202560 534567 1254 0 0 500832 103 0 0 25 0 9 0 801105808 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12218/tid=12226] ppid=12216 vsize=3152556 CPUtime=5009.35 cores=0-7
/proc/12218/task/12226/stat : 12226 (plingeling) R 12216 12218 12067 0 -1 4202560 583230 1254 0 0 500821 114 0 0 25 0 9 0 801105808 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=12218/tid=12227] ppid=12216 vsize=3152556 CPUtime=5008.54 cores=0-7
/proc/12218/task/12227/stat : 12227 (plingeling) R 12216 12218 12067 0 -1 4202560 560049 1254 0 0 500750 104 0 0 25 0 9 0 801105808 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12218/tid=12228] ppid=12216 vsize=3152556 CPUtime=5008.07 cores=0-7
/proc/12218/task/12228/stat : 12228 (plingeling) R 12216 12218 12067 0 -1 4202560 366303 1254 0 0 500727 80 0 0 25 0 9 0 801105808 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12218/tid=12229] ppid=12216 vsize=3152556 CPUtime=5009.35 cores=0-7
/proc/12218/task/12229/stat : 12229 (plingeling) R 12216 12218 12067 0 -1 4202560 453122 1254 0 0 500841 94 0 0 25 0 9 0 801105808 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=12218/tid=12230] ppid=12216 vsize=3152556 CPUtime=4936.53 cores=0-7
/proc/12218/task/12230/stat : 12230 (plingeling) R 12216 12218 12067 0 -1 4202560 541161 1254 0 0 493540 113 0 0 25 0 9 0 801105809 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12218/tid=12231] ppid=12216 vsize=3152556 CPUtime=5009.37 cores=0-7
/proc/12218/task/12231/stat : 12231 (plingeling) R 12216 12218 12067 0 -1 4202560 498074 1254 0 0 500839 98 0 0 25 0 9 0 801105809 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12218/tid=12232] ppid=12216 vsize=3152556 CPUtime=5009.37 cores=0-7
/proc/12218/task/12232/stat : 12232 (plingeling) R 12216 12218 12067 0 -1 4202560 485079 1254 0 0 500841 96 0 0 25 0 9 0 801105809 3228217344 734421 33554432000 4194304 4417452 140734019072800 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 3152556

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5009.42 s]
/proc/loadavg: 8.00 8.04 8.00 9/445 12387
/proc/meminfo: memFree=14378628/32950928 swapFree=60044444/67111528
[pid=12218] ppid=12216 vsize=0 CPUtime=40000.3 cores=0-7
/proc/12218/stat : 12218 (plingeling) Z 12216 12218 12067 0 -1 4203548 4025877 1254 0 0 3999209 817 0 0 18 0 2 0 801105799 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 7 0 0 0
/proc/12218/statm: 0 0 0 0 0 0 0
[pid=12218/tid=12230] ppid=12216 vsize=0 CPUtime=4936.63 cores=0-7
/proc/12218/task/12230/stat : 12230 (plingeling) R 12216 12218 12067 0 -1 4203588 541161 1254 0 0 493540 123 0 0 25 0 2 0 801105809 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.3
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5009.44
CPU time (s): 40000.3
CPU user time (s): 39992.1
CPU system time (s): 8.19075
CPU usage (%): 798.499
Max. virtual memory (cumulated for all children) (KiB): 3229224

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39992.1
system time used= 8.19075
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 4027131
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 376
involuntary context switches= 193982

runsolver used 20.2229 second user time and 52.65 second system time

The end

Launcher Data 

Begin job on node114 at 2011-05-23 01:59:11
IDJOB=3394876
IDBENCH=85808
IDSOLVER=1851
FILE ID=node114/3394876-1306108751
RUNJOBID= node114-1306105432-12081
PBS_JOBID= 13388020
Free space on /tmp= 73096 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT05/crafted/sorge05/QG7a/gensys-ukn003.shuffled-as.sat05-3843.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394876-1306108751/watcher-3394876-1306108751 -o /tmp/evaluation-result-3394876-1306108751/solver-3394876-1306108751 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394876-1306108751.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 59fa4424111aa94c942a4bbd786cbd51
RANDOM SEED=1354898734

node114.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      17323096 kB
Buffers:        222844 kB
Cached:        5277920 kB
SwapCached:      48888 kB
Active:        4051776 kB
Inactive:     11320524 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      17323096 kB
SwapTotal:    67111528 kB
SwapFree:     60044444 kB
Dirty:             816 kB
Writeback:           0 kB
AnonPages:     9829680 kB
Mapped:          27932 kB
Slab:           124040 kB
PageTables:      61840 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 108155360 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73096 MiB
End job on node114 at 2011-05-23 03:22:43