Trace number 3394408

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.6 5010.23

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n130-s30940966-unsat.cnf
MD5SUMca44a78076b8228fbb99ecc0147f157f
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables133
Number of clauses276
Sum of the clauses size828
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3276
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394408-1306095266/watcher-3394408-1306095266 -o /tmp/evaluation-result-3394408-1306095266/solver-3394408-1306095266 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394408-1306095266.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.35 6.14 4.04 1/470 20390
/proc/meminfo: memFree=16493740/32951124 swapFree=67057928/67111528
[pid=20390] ppid=20388 vsize=222300 CPUtime=0.03 cores=0-7
/proc/20390/stat : 20390 (plingeling) S 20388 20390 20285 0 -1 4202496 482 1249 0 0 3 0 0 0 18 0 9 0 701941931 227635200 401 33554432000 4194304 4417452 140735457230208 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/20390/statm: 55575 402 177 55 0 53461 0
[pid=20390/tid=20396] ppid=20388 vsize=222592 CPUtime=0 cores=0-7
/proc/20390/task/20396/stat : 20396 (plingeling) R 20388 20390 20285 0 -1 4202560 26 1249 0 0 0 0 0 0 17 0 9 0 701941932 227934208 487 33554432000 4194304 4417452 140735457230208 18446744073709551615 4351423 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20390/tid=20397] ppid=20388 vsize=222592 CPUtime=0 cores=0-7
/proc/20390/task/20397/stat : 20397 (plingeling) R 20388 20390 20285 0 -1 4202560 27 1249 0 0 0 0 0 0 18 0 9 0 701941932 227934208 487 33554432000 4194304 4417452 140735457230208 18446744073709551615 4232479 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20390/tid=20398] ppid=20388 vsize=222592 CPUtime=0 cores=0-7
/proc/20390/task/20398/stat : 20398 (plingeling) R 20388 20390 20285 0 -1 4202560 34 1249 0 0 0 0 0 0 18 0 9 0 701941932 227934208 487 33554432000 4194304 4417452 140735457230208 18446744073709551615 4236637 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20390/tid=20399] ppid=20388 vsize=222592 CPUtime=0.01 cores=0-7
/proc/20390/task/20399/stat : 20399 (plingeling) R 20388 20390 20285 0 -1 4202560 90 1249 0 0 1 0 0 0 18 0 9 0 701941932 227934208 487 33554432000 4194304 4417452 140735457230208 18446744073709551615 4366126 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20390/tid=20400] ppid=20388 vsize=222592 CPUtime=0 cores=0-7
/proc/20390/task/20400/stat : 20400 (plingeling) R 20388 20390 20285 0 -1 4202560 11 1249 0 0 0 0 0 0 17 0 9 0 701941932 227934208 487 33554432000 4194304 4417452 140735457230208 18446744073709551615 4295251 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20390/tid=20401] ppid=20388 vsize=222592 CPUtime=0 cores=0-7
/proc/20390/task/20401/stat : 20401 (plingeling) R 20388 20390 20285 0 -1 4202560 7 1249 0 0 0 0 0 0 19 0 9 0 701941932 227934208 487 33554432000 4194304 4417452 140735457230208 18446744073709551615 254261721582 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20390/tid=20402] ppid=20388 vsize=222592 CPUtime=0 cores=0-7
/proc/20390/task/20402/stat : 20402 (plingeling) R 20388 20390 20285 0 -1 4202560 11 1249 0 0 0 0 0 0 19 0 9 0 701941932 227934208 487 33554432000 4194304 4417452 140735457230208 18446744073709551615 4232810 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20390/tid=20403] ppid=20388 vsize=222592 CPUtime=0 cores=0-7
/proc/20390/task/20403/stat : 20403 (plingeling) R 20388 20390 20285 0 -1 4202560 5 1249 0 0 0 0 0 0 19 0 9 0 701941932 227934208 487 33554432000 4194304 4417452 140735457230208 18446744073709551615 4272201 0 0 4096 17442 0 0 0 -1 1 0 0 0

[startup+0.0342029 s]
/proc/loadavg: 6.35 6.14 4.04 1/470 20390
/proc/meminfo: memFree=16493740/32951124 swapFree=67057928/67111528
[pid=20390] ppid=20388 vsize=223736 CPUtime=0.17 cores=0-7
/proc/20390/stat : 20390 (plingeling) S 20388 20390 20285 0 -1 4202496 885 1249 0 0 17 0 0 0 18 0 9 0 701941931 229105664 804 33554432000 4194304 4417452 140735457230208 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/20390/statm: 55934 804 178 55 0 53820 0
[pid=20390/tid=20396] ppid=20388 vsize=223736 CPUtime=0.02 cores=0-7
/proc/20390/task/20396/stat : 20396 (plingeling) R 20388 20390 20285 0 -1 4202560 83 1249 0 0 2 0 0 0 18 0 9 0 701941932 229105664 804 33554432000 4194304 4417452 140735457230208 18446744073709551615 4243367 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20390/tid=20397] ppid=20388 vsize=223736 CPUtime=0.02 cores=0-7
/proc/20390/task/20397/stat : 20397 (plingeling) R 20388 20390 20285 0 -1 4202560 76 1249 0 0 2 0 0 0 18 0 9 0 701941932 229105664 804 33554432000 4194304 4417452 140735457230208 18446744073709551615 4277540 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20390/tid=20398] ppid=20388 vsize=223736 CPUtime=0.02 cores=0-7
/proc/20390/task/20398/stat : 20398 (plingeling) R 20388 20390 20285 0 -1 4202560 115 1249 0 0 2 0 0 0 18 0 9 0 701941932 229105664 804 33554432000 4194304 4417452 140735457230208 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20390/tid=20399] ppid=20388 vsize=223736 CPUtime=0.02 cores=0-7
/proc/20390/task/20399/stat : 20399 (plingeling) R 20388 20390 20285 0 -1 4202560 133 1249 0 0 2 0 0 0 18 0 9 0 701941932 229105664 804 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20390/tid=20400] ppid=20388 vsize=223736 CPUtime=0.02 cores=0-7
/proc/20390/task/20400/stat : 20400 (plingeling) R 20388 20390 20285 0 -1 4202560 57 1249 0 0 2 0 0 0 18 0 9 0 701941932 229105664 804 33554432000 4194304 4417452 140735457230208 18446744073709551615 4252507 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20390/tid=20401] ppid=20388 vsize=223736 CPUtime=0 cores=0-7
/proc/20390/task/20401/stat : 20401 (plingeling) R 20388 20390 20285 0 -1 4202560 8 1249 0 0 0 0 0 0 19 0 9 0 701941932 229105664 804 33554432000 4194304 4417452 140735457230208 18446744073709551615 254261721582 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20390/tid=20402] ppid=20388 vsize=223736 CPUtime=0.01 cores=0-7
/proc/20390/task/20402/stat : 20402 (plingeling) R 20388 20390 20285 0 -1 4202560 46 1249 0 0 1 0 0 0 19 0 9 0 701941932 229105664 804 33554432000 4194304 4417452 140735457230208 18446744073709551615 4252349 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20390/tid=20403] ppid=20388 vsize=223736 CPUtime=0.01 cores=0-7
/proc/20390/task/20403/stat : 20403 (plingeling) R 20388 20390 20285 0 -1 4202560 10 1249 0 0 1 0 0 0 19 0 9 0 701941932 229105664 804 33554432000 4194304 4417452 140735457230208 18446744073709551615 254261282328 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 223736

[startup+0.101193 s]
/proc/loadavg: 6.35 6.14 4.04 1/470 20390
/proc/meminfo: memFree=16493740/32951124 swapFree=67057928/67111528
[pid=20390] ppid=20388 vsize=227068 CPUtime=0.64 cores=0-7
/proc/20390/stat : 20390 (plingeling) S 20388 20390 20285 0 -1 4202496 1939 1249 0 0 64 0 0 0 18 0 9 0 701941931 232517632 1670 33554432000 4194304 4417452 140735457230208 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/20390/statm: 56767 1670 179 55 0 54653 0
[pid=20390/tid=20396] ppid=20388 vsize=227068 CPUtime=0.09 cores=0-7
/proc/20390/task/20396/stat : 20396 (plingeling) R 20388 20390 20285 0 -1 4202560 191 1249 0 0 9 0 0 0 18 0 9 0 701941932 232517632 1670 33554432000 4194304 4417452 140735457230208 18446744073709551615 4357902 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20390/tid=20397] ppid=20388 vsize=227068 CPUtime=0.09 cores=0-7
/proc/20390/task/20397/stat : 20397 (plingeling) R 20388 20390 20285 0 -1 4202560 176 1249 0 0 9 0 0 0 18 0 9 0 701941932 232517632 1670 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20390/tid=20398] ppid=20388 vsize=227068 CPUtime=0.09 cores=0-7
/proc/20390/task/20398/stat : 20398 (plingeling) R 20388 20390 20285 0 -1 4202560 294 1249 0 0 9 0 0 0 18 0 9 0 701941932 232517632 1670 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20390/tid=20399] ppid=20388 vsize=227068 CPUtime=0.09 cores=0-7
/proc/20390/task/20399/stat : 20399 (plingeling) R 20388 20390 20285 0 -1 4202560 380 1249 0 0 9 0 0 0 18 0 9 0 701941932 232517632 1670 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364669 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20390/tid=20400] ppid=20388 vsize=227068 CPUtime=0.09 cores=0-7
/proc/20390/task/20400/stat : 20400 (plingeling) R 20388 20390 20285 0 -1 4202560 182 1249 0 0 9 0 0 0 18 0 9 0 701941932 232517632 1670 33554432000 4194304 4417452 140735457230208 18446744073709551615 4357714 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20390/tid=20401] ppid=20388 vsize=227068 CPUtime=0.02 cores=0-7
/proc/20390/task/20401/stat : 20401 (plingeling) R 20388 20390 20285 0 -1 4202560 13 1249 0 0 2 0 0 0 19 0 9 0 701941932 232517632 1670 33554432000 4194304 4417452 140735457230208 18446744073709551615 4257737 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20390/tid=20402] ppid=20388 vsize=227068 CPUtime=0.06 cores=0-7
/proc/20390/task/20402/stat : 20402 (plingeling) R 20388 20390 20285 0 -1 4202560 188 1249 0 0 6 0 0 0 20 0 9 0 701941932 232517632 1670 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364399 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20390/tid=20403] ppid=20388 vsize=227068 CPUtime=0.08 cores=0-7
/proc/20390/task/20403/stat : 20403 (plingeling) R 20388 20390 20285 0 -1 4202560 158 1249 0 0 8 0 0 0 18 0 9 0 701941932 232517632 1670 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 0.64
Current children cumulated vsize (KiB) 227068

[startup+0.33412 s]
/proc/loadavg: 6.35 6.14 4.04 1/470 20390
/proc/meminfo: memFree=16493740/32951124 swapFree=67057928/67111528
[pid=20390] ppid=20388 vsize=232816 CPUtime=2.46 cores=0-7
/proc/20390/stat : 20390 (plingeling) S 20388 20390 20285 0 -1 4202496 6302 1249 0 0 246 0 0 0 18 0 9 0 701941931 238403584 3140 33554432000 4194304 4417452 140735457230208 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/20390/statm: 58204 3140 179 55 0 56090 0
[pid=20390/tid=20396] ppid=20388 vsize=232816 CPUtime=0.32 cores=0-7
/proc/20390/task/20396/stat : 20396 (plingeling) R 20388 20390 20285 0 -1 4202560 847 1249 0 0 32 0 0 0 18 0 9 0 701941932 238403584 3140 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20390/tid=20397] ppid=20388 vsize=232816 CPUtime=0.32 cores=0-7
/proc/20390/task/20397/stat : 20397 (plingeling) R 20388 20390 20285 0 -1 4202560 830 1249 0 0 32 0 0 0 18 0 9 0 701941932 238403584 3140 33554432000 4194304 4417452 140735457230208 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20390/tid=20398] ppid=20388 vsize=232816 CPUtime=0.32 cores=0-7
/proc/20390/task/20398/stat : 20398 (plingeling) R 20388 20390 20285 0 -1 4202560 962 1249 0 0 32 0 0 0 18 0 9 0 701941932 238403584 3140 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364861 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20390/tid=20399] ppid=20388 vsize=232816 CPUtime=0.32 cores=0-7
/proc/20390/task/20399/stat : 20399 (plingeling) R 20388 20390 20285 0 -1 4202560 930 1249 0 0 32 0 0 0 18 0 9 0 701941932 238403584 3140 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364696 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20390/tid=20400] ppid=20388 vsize=232816 CPUtime=0.32 cores=0-7
/proc/20390/task/20400/stat : 20400 (plingeling) R 20388 20390 20285 0 -1 4202560 645 1249 0 0 32 0 0 0 18 0 9 0 701941932 238403584 3140 33554432000 4194304 4417452 140735457230208 18446744073709551615 4366407 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20390/tid=20401] ppid=20388 vsize=232816 CPUtime=0.22 cores=0-7
/proc/20390/task/20401/stat : 20401 (plingeling) R 20388 20390 20285 0 -1 4202560 532 1249 0 0 22 0 0 0 20 0 9 0 701941932 238403584 3140 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=20390/tid=20402] ppid=20388 vsize=232816 CPUtime=0.29 cores=0-7
/proc/20390/task/20402/stat : 20402 (plingeling) R 20388 20390 20285 0 -1 4202560 545 1249 0 0 29 0 0 0 20 0 9 0 701941932 238403584 3140 33554432000 4194304 4417452 140735457230208 18446744073709551615 4356816 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20390/tid=20403] ppid=20388 vsize=232816 CPUtime=0.31 cores=0-7
/proc/20390/task/20403/stat : 20403 (plingeling) R 20388 20390 20285 0 -1 4202560 654 1249 0 0 31 0 0 0 18 0 9 0 701941932 238403584 3140 33554432000 4194304 4417452 140735457230208 18446744073709551615 4366146 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2.46
Current children cumulated vsize (KiB) 232816

[startup+0.701056 s]
/proc/loadavg: 6.35 6.14 4.04 1/470 20390
/proc/meminfo: memFree=16493740/32951124 swapFree=67057928/67111528
[pid=20390] ppid=20388 vsize=238292 CPUtime=5.39 cores=0-7
/proc/20390/stat : 20390 (plingeling) S 20388 20390 20285 0 -1 4202496 12104 1249 0 0 538 1 0 0 18 0 9 0 701941931 244011008 4493 33554432000 4194304 4417452 140735457230208 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/20390/statm: 59573 4493 179 55 0 57459 0
[pid=20390/tid=20396] ppid=20388 vsize=238292 CPUtime=0.68 cores=0-7
/proc/20390/task/20396/stat : 20396 (plingeling) R 20388 20390 20285 0 -1 4202560 1670 1249 0 0 68 0 0 0 18 0 9 0 701941932 244011008 4493 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20390/tid=20397] ppid=20388 vsize=238292 CPUtime=0.69 cores=0-7
/proc/20390/task/20397/stat : 20397 (plingeling) R 20388 20390 20285 0 -1 4202560 1636 1249 0 0 69 0 0 0 18 0 9 0 701941932 244011008 4493 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20390/tid=20398] ppid=20388 vsize=238292 CPUtime=0.69 cores=0-7
/proc/20390/task/20398/stat : 20398 (plingeling) R 20388 20390 20285 0 -1 4202560 1760 1249 0 0 69 0 0 0 18 0 9 0 701941932 244011008 4493 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20390/tid=20399] ppid=20388 vsize=238292 CPUtime=0.69 cores=0-7
/proc/20390/task/20399/stat : 20399 (plingeling) R 20388 20390 20285 0 -1 4202560 1698 1249 0 0 69 0 0 0 18 0 9 0 701941932 244011008 4493 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20390/tid=20400] ppid=20388 vsize=238292 CPUtime=0.68 cores=0-7
/proc/20390/task/20400/stat : 20400 (plingeling) R 20388 20390 20285 0 -1 4202560 1347 1249 0 0 68 0 0 0 18 0 9 0 701941932 244011008 4493 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364873 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20390/tid=20401] ppid=20388 vsize=238292 CPUtime=0.58 cores=0-7
/proc/20390/task/20401/stat : 20401 (plingeling) R 20388 20390 20285 0 -1 4202560 1223 1249 0 0 58 0 0 0 20 0 9 0 701941932 244011008 4493 33554432000 4194304 4417452 140735457230208 18446744073709551615 4278211 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=20390/tid=20402] ppid=20388 vsize=238292 CPUtime=0.66 cores=0-7
/proc/20390/task/20402/stat : 20402 (plingeling) R 20388 20390 20285 0 -1 4202560 1201 1249 0 0 66 0 0 0 20 0 9 0 701941932 244011008 4493 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20390/tid=20403] ppid=20388 vsize=238292 CPUtime=0.68 cores=0-7
/proc/20390/task/20403/stat : 20403 (plingeling) R 20388 20390 20285 0 -1 4202560 1212 1249 0 0 68 0 0 0 18 0 9 0 701941932 244011008 4493 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 5.39
Current children cumulated vsize (KiB) 238292

[startup+1.50091 s]
/proc/loadavg: 6.35 6.14 4.04 9/479 20404
/proc/meminfo: memFree=16473712/32951124 swapFree=67057928/67111528
[pid=20390] ppid=20388 vsize=243492 CPUtime=11.78 cores=0-7
/proc/20390/stat : 20390 (plingeling) S 20388 20390 20285 0 -1 4202496 21824 1249 0 0 1175 3 0 0 18 0 9 0 701941931 249335808 5793 33554432000 4194304 4417452 140735457230208 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/20390/statm: 60873 5793 179 55 0 58759 0
[pid=20390/tid=20396] ppid=20388 vsize=243492 CPUtime=1.48 cores=0-7
/proc/20390/task/20396/stat : 20396 (plingeling) R 20388 20390 20285 0 -1 4202560 2991 1249 0 0 148 0 0 0 19 0 9 0 701941932 249335808 5793 33554432000 4194304 4417452 140735457230208 18446744073709551615 4357235 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20390/tid=20397] ppid=20388 vsize=243492 CPUtime=1.49 cores=0-7
/proc/20390/task/20397/stat : 20397 (plingeling) R 20388 20390 20285 0 -1 4202560 2978 1249 0 0 149 0 0 0 20 0 9 0 701941932 249335808 5793 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20390/tid=20398] ppid=20388 vsize=243492 CPUtime=1.48 cores=0-7
/proc/20390/task/20398/stat : 20398 (plingeling) R 20388 20390 20285 0 -1 4202560 2944 1249 0 0 148 0 0 0 19 0 9 0 701941932 249335808 5793 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20390/tid=20399] ppid=20388 vsize=243492 CPUtime=1.48 cores=0-7
/proc/20390/task/20399/stat : 20399 (plingeling) R 20388 20390 20285 0 -1 4202560 2595 1249 0 0 148 0 0 0 19 0 9 0 701941932 249335808 5793 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20390/tid=20400] ppid=20388 vsize=243492 CPUtime=1.48 cores=0-7
/proc/20390/task/20400/stat : 20400 (plingeling) R 20388 20390 20285 0 -1 4202560 2649 1249 0 0 148 0 0 0 20 0 9 0 701941932 249335808 5793 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20390/tid=20401] ppid=20388 vsize=243492 CPUtime=1.36 cores=0-7
/proc/20390/task/20401/stat : 20401 (plingeling) R 20388 20390 20285 0 -1 4202560 2508 1249 0 0 136 0 0 0 22 0 9 0 701941932 249335808 5793 33554432000 4194304 4417452 140735457230208 18446744073709551615 4250599 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=20390/tid=20402] ppid=20388 vsize=243492 CPUtime=1.46 cores=0-7
/proc/20390/task/20402/stat : 20402 (plingeling) R 20388 20390 20285 0 -1 4202560 2296 1249 0 0 146 0 0 0 23 0 9 0 701941932 249335808 5793 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20390/tid=20403] ppid=20388 vsize=243492 CPUtime=1.48 cores=0-7
/proc/20390/task/20403/stat : 20403 (plingeling) R 20388 20390 20285 0 -1 4202560 2506 1249 0 0 148 0 0 0 19 0 9 0 701941932 249335808 5793 33554432000 4194304 4417452 140735457230208 18446744073709551615 18446744073699065939 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 11.78
Current children cumulated vsize (KiB) 243492

[startup+3.10063 s]
/proc/loadavg: 6.65 6.20 4.07 9/479 20404
/proc/meminfo: memFree=16465592/32951124 swapFree=67057928/67111528
[pid=20390] ppid=20388 vsize=252264 CPUtime=24.56 cores=0-7
/proc/20390/stat : 20390 (plingeling) S 20388 20390 20285 0 -1 4202496 39858 1249 0 0 2449 7 0 0 18 0 9 0 701941931 258318336 7957 33554432000 4194304 4417452 140735457230208 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/20390/statm: 63066 7957 179 55 0 60952 0
[pid=20390/tid=20396] ppid=20388 vsize=252264 CPUtime=3.08 cores=0-7
/proc/20390/task/20396/stat : 20396 (plingeling) R 20388 20390 20285 0 -1 4202560 5166 1249 0 0 308 0 0 0 25 0 9 0 701941932 258318336 7957 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20390/tid=20397] ppid=20388 vsize=252264 CPUtime=3.08 cores=0-7
/proc/20390/task/20397/stat : 20397 (plingeling) R 20388 20390 20285 0 -1 4202560 5668 1249 0 0 308 0 0 0 25 0 9 0 701941932 258318336 7957 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20390/tid=20398] ppid=20388 vsize=252264 CPUtime=3.08 cores=0-7
/proc/20390/task/20398/stat : 20398 (plingeling) R 20388 20390 20285 0 -1 4202560 4977 1249 0 0 308 0 0 0 25 0 9 0 701941932 258318336 7957 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364488 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20390/tid=20399] ppid=20388 vsize=252264 CPUtime=3.09 cores=0-7
/proc/20390/task/20399/stat : 20399 (plingeling) R 20388 20390 20285 0 -1 4202560 5047 1249 0 0 308 1 0 0 25 0 9 0 701941932 258318336 7957 33554432000 4194304 4417452 140735457230208 18446744073709551615 4365075 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20390/tid=20400] ppid=20388 vsize=252264 CPUtime=3.08 cores=0-7
/proc/20390/task/20400/stat : 20400 (plingeling) R 20388 20390 20285 0 -1 4202560 5162 1249 0 0 307 1 0 0 25 0 9 0 701941932 258318336 7957 33554432000 4194304 4417452 140735457230208 18446744073709551615 4366079 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20390/tid=20401] ppid=20388 vsize=252264 CPUtime=2.95 cores=0-7
/proc/20390/task/20401/stat : 20401 (plingeling) R 20388 20390 20285 0 -1 4202560 4447 1249 0 0 294 1 0 0 25 0 9 0 701941932 258318336 7957 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=20390/tid=20402] ppid=20388 vsize=252264 CPUtime=3.06 cores=0-7
/proc/20390/task/20402/stat : 20402 (plingeling) R 20388 20390 20285 0 -1 4202560 4517 1249 0 0 305 1 0 0 25 0 9 0 701941932 258318336 7957 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20390/tid=20403] ppid=20388 vsize=252264 CPUtime=3.07 cores=0-7
/proc/20390/task/20403/stat : 20403 (plingeling) R 20388 20390 20285 0 -1 4202560 4517 1249 0 0 307 0 0 0 25 0 9 0 701941932 258318336 7957 33554432000 4194304 4417452 140735457230208 18446744073709551615 4366126 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 24.56
Current children cumulated vsize (KiB) 252264

[startup+6.30107 s]
/proc/loadavg: 6.65 6.20 4.07 9/479 20404
/proc/meminfo: memFree=16451952/32951124 swapFree=67057928/67111528
[pid=20390] ppid=20388 vsize=264584 CPUtime=50.12 cores=0-7
/proc/20390/stat : 20390 (plingeling) S 20388 20390 20285 0 -1 4202496 70000 1249 0 0 5000 12 0 0 18 0 9 0 701941931 270934016 11044 33554432000 4194304 4417452 140735457230208 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/20390/statm: 66146 11044 179 55 0 64032 0

################
# More data... #
################


[startup+4722.32 s]
/proc/loadavg: 8.07 8.07 8.00 9/479 20545
/proc/meminfo: memFree=14178700/32951124 swapFree=67057928/67111528
[pid=20390] ppid=20388 vsize=2531700 CPUtime=37702.6 cores=0-7
/proc/20390/stat : 20390 (plingeling) S 20388 20390 20285 0 -1 4202496 5867652 1249 0 0 3769129 1128 0 0 18 0 9 0 701941931 2592460800 577813 33554432000 4194304 4417452 140735457230208 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/20390/statm: 632925 577813 179 55 0 630811 0
[pid=20390/tid=20396] ppid=20388 vsize=2531700 CPUtime=4722.11 cores=0-7
/proc/20390/task/20396/stat : 20396 (plingeling) R 20388 20390 20285 0 -1 4202560 623879 1249 0 0 472093 118 0 0 25 0 9 0 701941932 2592460800 577813 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20390/tid=20397] ppid=20388 vsize=2531700 CPUtime=4649.8 cores=0-7
/proc/20390/task/20397/stat : 20397 (plingeling) R 20388 20390 20285 0 -1 4202560 723075 1249 0 0 464838 142 0 0 25 0 9 0 701941932 2592460800 577813 33554432000 4194304 4417452 140735457230208 18446744073709551615 4366099 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=20390/tid=20398] ppid=20388 vsize=2531700 CPUtime=4722.32 cores=0-7
/proc/20390/task/20398/stat : 20398 (plingeling) R 20388 20390 20285 0 -1 4202560 698391 1249 0 0 472106 126 0 0 25 0 9 0 701941932 2592460800 577813 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20390/tid=20399] ppid=20388 vsize=2531700 CPUtime=4722.42 cores=0-7
/proc/20390/task/20399/stat : 20399 (plingeling) R 20388 20390 20285 0 -1 4202560 892475 1249 0 0 472096 146 0 0 25 0 9 0 701941932 2592460800 577813 33554432000 4194304 4417452 140735457230208 18446744073709551615 4365023 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20390/tid=20400] ppid=20388 vsize=2531700 CPUtime=4721.53 cores=0-7
/proc/20390/task/20400/stat : 20400 (plingeling) R 20388 20390 20285 0 -1 4202560 709564 1249 0 0 472008 145 0 0 25 0 9 0 701941932 2592460800 577813 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20390/tid=20401] ppid=20388 vsize=2531700 CPUtime=4720.42 cores=0-7
/proc/20390/task/20401/stat : 20401 (plingeling) R 20388 20390 20285 0 -1 4202560 669238 1249 0 0 471877 165 0 0 25 0 9 0 701941932 2592460800 577813 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20390/tid=20402] ppid=20388 vsize=2531700 CPUtime=4721.51 cores=0-7
/proc/20390/task/20402/stat : 20402 (plingeling) R 20388 20390 20285 0 -1 4202560 744541 1249 0 0 472011 140 0 0 25 0 9 0 701941932 2592460800 577813 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20390/tid=20403] ppid=20388 vsize=2531700 CPUtime=4722.4 cores=0-7
/proc/20390/task/20403/stat : 20403 (plingeling) R 20388 20390 20285 0 -1 4202560 806132 1249 0 0 472097 143 0 0 25 0 9 0 701941932 2592460800 577813 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37702.6
Current children cumulated vsize (KiB) 2531700

[startup+4782.3 s]
/proc/loadavg: 8.02 8.06 8.00 9/479 20547
/proc/meminfo: memFree=14168280/32951124 swapFree=67057928/67111528
[pid=20390] ppid=20388 vsize=2542076 CPUtime=38181.5 cores=0-7
/proc/20390/stat : 20390 (plingeling) S 20388 20390 20285 0 -1 4202496 5878086 1249 0 0 3817021 1129 0 0 18 0 9 0 701941931 2603085824 580407 33554432000 4194304 4417452 140735457230208 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/20390/statm: 635519 580407 179 55 0 633405 0
[pid=20390/tid=20396] ppid=20388 vsize=2542076 CPUtime=4782.09 cores=0-7
/proc/20390/task/20396/stat : 20396 (plingeling) R 20388 20390 20285 0 -1 4202560 623879 1249 0 0 478091 118 0 0 25 0 9 0 701941932 2603085824 580407 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20390/tid=20397] ppid=20388 vsize=2542076 CPUtime=4708.86 cores=0-7
/proc/20390/task/20397/stat : 20397 (plingeling) R 20388 20390 20285 0 -1 4202560 723075 1249 0 0 470744 142 0 0 25 0 9 0 701941932 2603085824 580407 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364558 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=20390/tid=20398] ppid=20388 vsize=2542076 CPUtime=4782.3 cores=0-7
/proc/20390/task/20398/stat : 20398 (plingeling) R 20388 20390 20285 0 -1 4202560 698391 1249 0 0 478104 126 0 0 25 0 9 0 701941932 2603085824 580407 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20390/tid=20399] ppid=20388 vsize=2542076 CPUtime=4782.4 cores=0-7
/proc/20390/task/20399/stat : 20399 (plingeling) R 20388 20390 20285 0 -1 4202560 902909 1249 0 0 478092 148 0 0 25 0 9 0 701941932 2603085824 580407 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20390/tid=20400] ppid=20388 vsize=2542076 CPUtime=4781.51 cores=0-7
/proc/20390/task/20400/stat : 20400 (plingeling) R 20388 20390 20285 0 -1 4202560 709564 1249 0 0 478006 145 0 0 25 0 9 0 701941932 2603085824 580407 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20390/tid=20401] ppid=20388 vsize=2542076 CPUtime=4780.39 cores=0-7
/proc/20390/task/20401/stat : 20401 (plingeling) R 20388 20390 20285 0 -1 4202560 669238 1249 0 0 477874 165 0 0 25 0 9 0 701941932 2603085824 580407 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20390/tid=20402] ppid=20388 vsize=2542076 CPUtime=4781.49 cores=0-7
/proc/20390/task/20402/stat : 20402 (plingeling) R 20388 20390 20285 0 -1 4202560 744541 1249 0 0 478009 140 0 0 25 0 9 0 701941932 2603085824 580407 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20390/tid=20403] ppid=20388 vsize=2542076 CPUtime=4782.39 cores=0-7
/proc/20390/task/20403/stat : 20403 (plingeling) R 20388 20390 20285 0 -1 4202560 806132 1249 0 0 478096 143 0 0 25 0 9 0 701941932 2603085824 580407 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38181.5
Current children cumulated vsize (KiB) 2542076

[startup+4842.3 s]
/proc/loadavg: 8.06 8.06 8.00 9/479 20548
/proc/meminfo: memFree=14152144/32951124 swapFree=67057928/67111528
[pid=20390] ppid=20388 vsize=2558408 CPUtime=38660.6 cores=0-7
/proc/20390/stat : 20390 (plingeling) S 20388 20390 20285 0 -1 4202496 5917161 1249 0 0 3864921 1135 0 0 18 0 9 0 701941931 2619809792 584490 33554432000 4194304 4417452 140735457230208 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/20390/statm: 639602 584490 179 55 0 637488 0
[pid=20390/tid=20396] ppid=20388 vsize=2558408 CPUtime=4842.09 cores=0-7
/proc/20390/task/20396/stat : 20396 (plingeling) R 20388 20390 20285 0 -1 4202560 624021 1249 0 0 484091 118 0 0 25 0 9 0 701941932 2619809792 584490 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20390/tid=20397] ppid=20388 vsize=2558408 CPUtime=4767.92 cores=0-7
/proc/20390/task/20397/stat : 20397 (plingeling) R 20388 20390 20285 0 -1 4202560 752845 1249 0 0 476647 145 0 0 25 0 9 0 701941932 2619809792 584490 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=20390/tid=20398] ppid=20388 vsize=2558408 CPUtime=4842.3 cores=0-7
/proc/20390/task/20398/stat : 20398 (plingeling) R 20388 20390 20285 0 -1 4202560 698391 1249 0 0 484104 126 0 0 25 0 9 0 701941932 2619809792 584490 33554432000 4194304 4417452 140735457230208 18446744073709551615 4366804 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20390/tid=20399] ppid=20388 vsize=2558408 CPUtime=4842.4 cores=0-7
/proc/20390/task/20399/stat : 20399 (plingeling) R 20388 20390 20285 0 -1 4202560 912072 1249 0 0 484091 149 0 0 25 0 9 0 701941932 2619809792 584490 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20390/tid=20400] ppid=20388 vsize=2558408 CPUtime=4841.51 cores=0-7
/proc/20390/task/20400/stat : 20400 (plingeling) R 20388 20390 20285 0 -1 4202560 709564 1249 0 0 484006 145 0 0 25 0 9 0 701941932 2619809792 584490 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20390/tid=20401] ppid=20388 vsize=2558408 CPUtime=4840.38 cores=0-7
/proc/20390/task/20401/stat : 20401 (plingeling) R 20388 20390 20285 0 -1 4202560 669238 1249 0 0 483873 165 0 0 25 0 9 0 701941932 2619809792 584490 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20390/tid=20402] ppid=20388 vsize=2558408 CPUtime=4841.5 cores=0-7
/proc/20390/task/20402/stat : 20402 (plingeling) R 20388 20390 20285 0 -1 4202560 744541 1249 0 0 484010 140 0 0 25 0 9 0 701941932 2619809792 584490 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20390/tid=20403] ppid=20388 vsize=2558408 CPUtime=4842.39 cores=0-7
/proc/20390/task/20403/stat : 20403 (plingeling) R 20388 20390 20285 0 -1 4202560 806132 1249 0 0 484096 143 0 0 25 0 9 0 701941932 2619809792 584490 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38660.6
Current children cumulated vsize (KiB) 2558408

[startup+4902.3 s]
/proc/loadavg: 8.07 8.06 8.00 9/479 20550
/proc/meminfo: memFree=14150032/32951124 swapFree=67057928/67111528
[pid=20390] ppid=20388 vsize=2560648 CPUtime=39139.6 cores=0-7
/proc/20390/stat : 20390 (plingeling) S 20388 20390 20285 0 -1 4202496 5921816 1249 0 0 3912827 1136 0 0 18 0 9 0 701941931 2622103552 585049 33554432000 4194304 4417452 140735457230208 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/20390/statm: 640162 585049 179 55 0 638048 0
[pid=20390/tid=20396] ppid=20388 vsize=2560648 CPUtime=4902.09 cores=0-7
/proc/20390/task/20396/stat : 20396 (plingeling) R 20388 20390 20285 0 -1 4202560 624021 1249 0 0 490091 118 0 0 25 0 9 0 701941932 2622103552 585049 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20390/tid=20397] ppid=20388 vsize=2560648 CPUtime=4826.99 cores=0-7
/proc/20390/task/20397/stat : 20397 (plingeling) R 20388 20390 20285 0 -1 4202560 752845 1249 0 0 482554 145 0 0 25 0 9 0 701941932 2622103552 585049 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=20390/tid=20398] ppid=20388 vsize=2560648 CPUtime=4902.3 cores=0-7
/proc/20390/task/20398/stat : 20398 (plingeling) R 20388 20390 20285 0 -1 4202560 698391 1249 0 0 490104 126 0 0 25 0 9 0 701941932 2622103552 585049 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20390/tid=20399] ppid=20388 vsize=2560648 CPUtime=4902.41 cores=0-7
/proc/20390/task/20399/stat : 20399 (plingeling) R 20388 20390 20285 0 -1 4202560 916563 1249 0 0 490091 150 0 0 25 0 9 0 701941932 2622103552 585049 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20390/tid=20400] ppid=20388 vsize=2560648 CPUtime=4901.51 cores=0-7
/proc/20390/task/20400/stat : 20400 (plingeling) R 20388 20390 20285 0 -1 4202560 709564 1249 0 0 490006 145 0 0 25 0 9 0 701941932 2622103552 585049 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20390/tid=20401] ppid=20388 vsize=2560648 CPUtime=4900.37 cores=0-7
/proc/20390/task/20401/stat : 20401 (plingeling) R 20388 20390 20285 0 -1 4202560 669238 1249 0 0 489871 166 0 0 25 0 9 0 701941932 2622103552 585049 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20390/tid=20402] ppid=20388 vsize=2560648 CPUtime=4901.5 cores=0-7
/proc/20390/task/20402/stat : 20402 (plingeling) R 20388 20390 20285 0 -1 4202560 744541 1249 0 0 490010 140 0 0 25 0 9 0 701941932 2622103552 585049 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20390/tid=20403] ppid=20388 vsize=2560648 CPUtime=4902.39 cores=0-7
/proc/20390/task/20403/stat : 20403 (plingeling) R 20388 20390 20285 0 -1 4202560 806296 1249 0 0 490096 143 0 0 25 0 9 0 701941932 2622103552 585049 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39139.6
Current children cumulated vsize (KiB) 2560648

[startup+4962.3 s]
/proc/loadavg: 8.07 8.06 8.00 9/479 20551
/proc/meminfo: memFree=14107496/32951124 swapFree=67057928/67111528
[pid=20390] ppid=20388 vsize=2603040 CPUtime=39618.7 cores=0-7
/proc/20390/stat : 20390 (plingeling) S 20388 20390 20285 0 -1 4202496 5954677 1249 0 0 3960725 1143 0 0 18 0 9 0 701941931 2665512960 595646 33554432000 4194304 4417452 140735457230208 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/20390/statm: 650760 595646 179 55 0 648646 0
[pid=20390/tid=20396] ppid=20388 vsize=2603040 CPUtime=4962.09 cores=0-7
/proc/20390/task/20396/stat : 20396 (plingeling) R 20388 20390 20285 0 -1 4202560 624021 1249 0 0 496091 118 0 0 25 0 9 0 701941932 2665512960 595646 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20390/tid=20397] ppid=20388 vsize=2603040 CPUtime=4886.05 cores=0-7
/proc/20390/task/20397/stat : 20397 (plingeling) R 20388 20390 20285 0 -1 4202560 752845 1249 0 0 488460 145 0 0 25 0 9 0 701941932 2665512960 595646 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=20390/tid=20398] ppid=20388 vsize=2603040 CPUtime=4962.31 cores=0-7
/proc/20390/task/20398/stat : 20398 (plingeling) R 20388 20390 20285 0 -1 4202560 698391 1249 0 0 496105 126 0 0 25 0 9 0 701941932 2665512960 595646 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20390/tid=20399] ppid=20388 vsize=2603040 CPUtime=4962.4 cores=0-7
/proc/20390/task/20399/stat : 20399 (plingeling) R 20388 20390 20285 0 -1 4202560 928449 1249 0 0 496088 152 0 0 25 0 9 0 701941932 2665512960 595646 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20390/tid=20400] ppid=20388 vsize=2603040 CPUtime=4961.51 cores=0-7
/proc/20390/task/20400/stat : 20400 (plingeling) R 20388 20390 20285 0 -1 4202560 709564 1249 0 0 496006 145 0 0 25 0 9 0 701941932 2665512960 595646 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20390/tid=20401] ppid=20388 vsize=2603040 CPUtime=4960.35 cores=0-7
/proc/20390/task/20401/stat : 20401 (plingeling) R 20388 20390 20285 0 -1 4202560 669238 1249 0 0 495869 166 0 0 25 0 9 0 701941932 2665512960 595646 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20390/tid=20402] ppid=20388 vsize=2603040 CPUtime=4961.5 cores=0-7
/proc/20390/task/20402/stat : 20402 (plingeling) R 20388 20390 20285 0 -1 4202560 765516 1249 0 0 496006 144 0 0 25 0 9 0 701941932 2665512960 595646 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20390/tid=20403] ppid=20388 vsize=2603040 CPUtime=4962.39 cores=0-7
/proc/20390/task/20403/stat : 20403 (plingeling) R 20388 20390 20285 0 -1 4202560 806296 1249 0 0 496096 143 0 0 25 0 9 0 701941932 2665512960 595646 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39618.7
Current children cumulated vsize (KiB) 2603040



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.1 s]
/proc/loadavg: 8.09 8.07 8.00 9/479 20553
/proc/meminfo: memFree=13940588/32951124 swapFree=67057928/67111528
[pid=20390] ppid=20388 vsize=2769648 CPUtime=40000.4 cores=0-7
/proc/20390/stat : 20390 (plingeling) S 20388 20390 20285 0 -1 4202496 6032203 1249 0 0 3998880 1164 0 0 18 0 9 0 701941931 2836119552 637298 33554432000 4194304 4417452 140735457230208 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/20390/statm: 692412 637298 179 55 0 690298 0
[pid=20390/tid=20396] ppid=20388 vsize=2769648 CPUtime=5009.91 cores=0-7
/proc/20390/task/20396/stat : 20396 (plingeling) R 20388 20390 20285 0 -1 4202560 646040 1249 0 0 500866 125 0 0 25 0 9 0 701941932 2836119552 637298 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20390/tid=20397] ppid=20388 vsize=2769648 CPUtime=4933.12 cores=0-7
/proc/20390/task/20397/stat : 20397 (plingeling) R 20388 20390 20285 0 -1 4202560 769229 1249 0 0 493161 151 0 0 25 0 9 0 701941932 2836119552 637298 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=20390/tid=20398] ppid=20388 vsize=2769648 CPUtime=5010.13 cores=0-7
/proc/20390/task/20398/stat : 20398 (plingeling) R 20388 20390 20285 0 -1 4202560 698391 1249 0 0 500887 126 0 0 25 0 9 0 701941932 2836119552 637298 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20390/tid=20399] ppid=20388 vsize=2769648 CPUtime=5010.23 cores=0-7
/proc/20390/task/20399/stat : 20399 (plingeling) R 20388 20390 20285 0 -1 4202560 932545 1249 0 0 500870 153 0 0 25 0 9 0 701941932 2836119552 637298 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20390/tid=20400] ppid=20388 vsize=2769648 CPUtime=5009.33 cores=0-7
/proc/20390/task/20400/stat : 20400 (plingeling) R 20388 20390 20285 0 -1 4202560 709564 1249 0 0 500788 145 0 0 25 0 9 0 701941932 2836119552 637298 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20390/tid=20401] ppid=20388 vsize=2769648 CPUtime=5008.16 cores=0-7
/proc/20390/task/20401/stat : 20401 (plingeling) R 20388 20390 20285 0 -1 4202560 700487 1249 0 0 500643 173 0 0 25 0 9 0 701941932 2836119552 637298 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20390/tid=20402] ppid=20388 vsize=2769648 CPUtime=5009.32 cores=0-7
/proc/20390/task/20402/stat : 20402 (plingeling) R 20388 20390 20285 0 -1 4202560 765516 1249 0 0 500788 144 0 0 25 0 9 0 701941932 2836119552 637298 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20390/tid=20403] ppid=20388 vsize=2769648 CPUtime=5010.22 cores=0-7
/proc/20390/task/20403/stat : 20403 (plingeling) R 20388 20390 20285 0 -1 4202560 810074 1249 0 0 500878 144 0 0 25 0 9 0 701941932 2836119552 637298 33554432000 4194304 4417452 140735457230208 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 2769648

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.22 s]
/proc/loadavg: 8.09 8.07 8.00 9/479 20553
/proc/meminfo: memFree=13940588/32951124 swapFree=67057928/67111528
[pid=20390] ppid=20388 vsize=0 CPUtime=40000.6 cores=0-7
/proc/20390/stat : 20390 (plingeling) Z 20388 20390 20285 0 -1 4203548 6032203 1249 0 0 3998887 1174 0 0 18 0 2 0 701941931 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/20390/statm: 0 0 0 0 0 0 0
[pid=20390/tid=20397] ppid=20388 vsize=0 CPUtime=4933.23 cores=0-7
/proc/20390/task/20397/stat : 20397 (plingeling) R 20388 20390 20285 0 -1 4203588 769229 1249 0 0 493162 161 0 0 25 0 2 0 701941932 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5010.23
CPU time (s): 40000.6
CPU user time (s): 39988.9
CPU system time (s): 11.7442
CPU usage (%): 798.38
Max. virtual memory (cumulated for all children) (KiB): 2769648

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39988.9
system time used= 11.7442
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6033452
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 430
involuntary context switches= 192491

runsolver used 21.2298 second user time and 56.5234 second system time

The end

Launcher Data 

Begin job on node140 at 2011-05-22 22:14:26
IDJOB=3394408
IDBENCH=83148
IDSOLVER=1851
FILE ID=node140/3394408-1306095266
RUNJOBID= node140-1306094852-20299
PBS_JOBID= 13387963
Free space on /tmp= 73224 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n130-s30940966-unsat.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394408-1306095266/watcher-3394408-1306095266 -o /tmp/evaluation-result-3394408-1306095266/solver-3394408-1306095266 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394408-1306095266.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= ca44a78076b8228fbb99ecc0147f157f
RANDOM SEED=1701260264

node140.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.14
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      16494004 kB
Buffers:        358636 kB
Cached:        8823688 kB
SwapCached:       8168 kB
Active:       10803324 kB
Inactive:      5294616 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      16494004 kB
SwapTotal:    67111528 kB
SwapFree:     67057928 kB
Dirty:             260 kB
Writeback:           0 kB
AnonPages:     6912868 kB
Mapped:          37464 kB
Slab:           236156 kB
PageTables:      52196 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 141378140 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73224 MiB
End job on node140 at 2011-05-22 23:37:58