Trace number 3394410

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.6 5011.77

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n280-s93756482-sat.cnf
MD5SUMbadb3b4ac12b29ada7dd8ed96ed75c80
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark68.6746
Satisfiable
(Un)Satisfiability was proved
Number of variables280
Number of clauses672
Sum of the clauses size1680
Maximum clause length5
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2560
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 5112
Number of clauses of size over 50

Solver Data


Verifier Data

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394410-1306095315/watcher-3394410-1306095315 -o /tmp/evaluation-result-3394410-1306095315/solver-3394410-1306095315 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394410-1306095315.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.14 8.10 8.01 1/528 9058
/proc/meminfo: memFree=15528996/32951124 swapFree=51873512/67111528
[pid=9058] ppid=9056 vsize=80844 CPUtime=0 cores=0-7
/proc/9058/stat : 9058 (plingeling) R 9056 9058 8836 0 -1 4202496 408 1253 0 0 0 0 0 0 25 0 8 0 701947749 82784256 334 33554432000 4194304 4417452 140733651132016 18446744073709551615 212039711729 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/9058/statm: 20211 334 168 55 0 18097 0
[pid=9058/tid=9064] ppid=9056 vsize=157180 CPUtime=0.01 cores=0-7
/proc/9058/task/9064/stat : 9064 (plingeling) R 9056 9058 8836 0 -1 4202560 85 1253 0 0 1 0 0 0 25 0 9 0 701947749 160952320 508 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9058/tid=9065] ppid=9056 vsize=157180 CPUtime=0 cores=0-7
/proc/9058/task/9065/stat : 9065 (plingeling) R 9056 9058 8836 0 -1 4202560 5 1253 0 0 0 0 0 0 25 0 9 0 701947749 160952320 510 33554432000 4194304 4417452 140733651132016 18446744073709551615 212039760366 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9058/tid=9066] ppid=9056 vsize=157180 CPUtime=0 cores=0-7
/proc/9058/task/9066/stat : 9066 (plingeling) R 9056 9058 8836 0 -1 4202560 13 1253 0 0 0 0 0 0 25 0 9 0 701947749 160952320 510 33554432000 4194304 4417452 140733651132016 18446744073709551615 212039760366 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9058/tid=9067] ppid=9056 vsize=157180 CPUtime=0 cores=0-7
/proc/9058/task/9067/stat : 9067 (plingeling) R 9056 9058 8836 0 -1 4202560 21 1253 0 0 0 0 0 0 25 0 9 0 701947749 160952320 518 33554432000 4194304 4417452 140733651132016 18446744073709551615 4353568 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9058/tid=9068] ppid=9056 vsize=157180 CPUtime=0 cores=0-7
/proc/9058/task/9068/stat : 9068 (plingeling) R 9056 9058 8836 0 -1 4202560 15 1253 0 0 0 0 0 0 25 0 9 0 701947749 160952320 518 33554432000 4194304 4417452 140733651132016 18446744073709551615 4353896 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9058/tid=9069] ppid=9056 vsize=157180 CPUtime=0 cores=0-7
/proc/9058/task/9069/stat : 9069 (plingeling) S 9056 9058 8836 0 -1 4202560 10 1253 0 0 0 0 0 0 25 0 9 0 701947749 160952320 518 33554432000 4194304 4417452 140733651132016 18446744073709551615 212039760366 0 0 4096 17442 18446604440563631168 0 0 -1 4 0 0 0
[pid=9058/tid=9070] ppid=9056 vsize=157180 CPUtime=0.01 cores=0-7
/proc/9058/task/9070/stat : 9070 (plingeling) R 9056 9058 8836 0 -1 4202560 39 1253 0 0 1 0 0 0 25 0 9 0 701947749 160952320 518 33554432000 4194304 4417452 140733651132016 18446744073709551615 4232612 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9058/tid=9071] ppid=9056 vsize=157180 CPUtime=0.01 cores=0-7
/proc/9058/task/9071/stat : 9071 (plingeling) R 9056 9058 8836 0 -1 4202560 20 1253 0 0 1 0 0 0 17 0 9 0 701947749 160952320 518 33554432000 4194304 4417452 140733651132016 18446744073709551615 4232648 0 0 4096 17442 0 0 0 -1 1 0 0 0

[startup+0.107344 s]
/proc/loadavg: 8.14 8.10 8.01 1/528 9058
/proc/meminfo: memFree=15528996/32951124 swapFree=51873512/67111528
[pid=9058] ppid=9056 vsize=161436 CPUtime=0.73 cores=0-7
/proc/9058/stat : 9058 (plingeling) S 9056 9058 8836 0 -1 4202496 1966 1253 0 0 73 0 0 0 18 0 9 0 701947749 165310464 1602 33554432000 4194304 4417452 140733651132016 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/9058/statm: 40359 1602 180 55 0 38245 0
[pid=9058/tid=9064] ppid=9056 vsize=161436 CPUtime=0.1 cores=0-7
/proc/9058/task/9064/stat : 9064 (plingeling) R 9056 9058 8836 0 -1 4202560 206 1253 0 0 10 0 0 0 25 0 9 0 701947749 165310464 1602 33554432000 4194304 4417452 140733651132016 18446744073709551615 4308814 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9058/tid=9065] ppid=9056 vsize=161436 CPUtime=0.08 cores=0-7
/proc/9058/task/9065/stat : 9065 (plingeling) R 9056 9058 8836 0 -1 4202560 42 1253 0 0 8 0 0 0 25 0 9 0 701947749 165310464 1602 33554432000 4194304 4417452 140733651132016 18446744073709551615 4358355 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9058/tid=9066] ppid=9056 vsize=161436 CPUtime=0.07 cores=0-7
/proc/9058/task/9066/stat : 9066 (plingeling) R 9056 9058 8836 0 -1 4202560 110 1253 0 0 7 0 0 0 25 0 9 0 701947749 165310464 1602 33554432000 4194304 4417452 140733651132016 18446744073709551615 4306294 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9058/tid=9067] ppid=9056 vsize=161436 CPUtime=0.08 cores=0-7
/proc/9058/task/9067/stat : 9067 (plingeling) R 9056 9058 8836 0 -1 4202560 412 1253 0 0 8 0 0 0 25 0 9 0 701947749 165310464 1602 33554432000 4194304 4417452 140733651132016 18446744073709551615 4358956 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9058/tid=9068] ppid=9056 vsize=161436 CPUtime=0.09 cores=0-7
/proc/9058/task/9068/stat : 9068 (plingeling) R 9056 9058 8836 0 -1 4202560 223 1253 0 0 9 0 0 0 25 0 9 0 701947749 165310464 1602 33554432000 4194304 4417452 140733651132016 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9058/tid=9069] ppid=9056 vsize=161436 CPUtime=0.08 cores=0-7
/proc/9058/task/9069/stat : 9069 (plingeling) R 9056 9058 8836 0 -1 4202560 239 1253 0 0 8 0 0 0 18 0 9 0 701947749 165310464 1602 33554432000 4194304 4417452 140733651132016 18446744073709551615 4308643 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9058/tid=9070] ppid=9056 vsize=161436 CPUtime=0.09 cores=0-7
/proc/9058/task/9070/stat : 9070 (plingeling) R 9056 9058 8836 0 -1 4202560 191 1253 0 0 9 0 0 0 25 0 9 0 701947749 165310464 1602 33554432000 4194304 4417452 140733651132016 18446744073709551615 4256146 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9058/tid=9071] ppid=9056 vsize=161436 CPUtime=0.09 cores=0-7
/proc/9058/task/9071/stat : 9071 (plingeling) R 9056 9058 8836 0 -1 4202560 159 1253 0 0 9 0 0 0 18 0 9 0 701947749 165310464 1602 33554432000 4194304 4417452 140733651132016 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 0.73
Current children cumulated vsize (KiB) 161436

[startup+0.201328 s]
/proc/loadavg: 8.14 8.10 8.01 1/528 9058
/proc/meminfo: memFree=15528996/32951124 swapFree=51873512/67111528
[pid=9058] ppid=9056 vsize=164148 CPUtime=1.48 cores=0-7
/proc/9058/stat : 9058 (plingeling) S 9056 9058 8836 0 -1 4202496 3381 1253 0 0 148 0 0 0 18 0 9 0 701947749 168087552 2288 33554432000 4194304 4417452 140733651132016 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/9058/statm: 41037 2288 180 55 0 38923 0
[pid=9058/tid=9064] ppid=9056 vsize=164148 CPUtime=0.19 cores=0-7
/proc/9058/task/9064/stat : 9064 (plingeling) R 9056 9058 8836 0 -1 4202560 363 1253 0 0 19 0 0 0 25 0 9 0 701947749 168087552 2288 33554432000 4194304 4417452 140733651132016 18446744073709551615 4250986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9058/tid=9065] ppid=9056 vsize=164148 CPUtime=0.17 cores=0-7
/proc/9058/task/9065/stat : 9065 (plingeling) R 9056 9058 8836 0 -1 4202560 203 1253 0 0 17 0 0 0 25 0 9 0 701947749 168087552 2288 33554432000 4194304 4417452 140733651132016 18446744073709551615 4237002 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9058/tid=9066] ppid=9056 vsize=164148 CPUtime=0.17 cores=0-7
/proc/9058/task/9066/stat : 9066 (plingeling) R 9056 9058 8836 0 -1 4202560 218 1253 0 0 17 0 0 0 25 0 9 0 701947749 168087552 2288 33554432000 4194304 4417452 140733651132016 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9058/tid=9067] ppid=9056 vsize=164148 CPUtime=0.18 cores=0-7
/proc/9058/task/9067/stat : 9067 (plingeling) R 9056 9058 8836 0 -1 4202560 642 1253 0 0 18 0 0 0 25 0 9 0 701947749 168087552 2288 33554432000 4194304 4417452 140733651132016 18446744073709551615 4358110 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9058/tid=9068] ppid=9056 vsize=164148 CPUtime=0.18 cores=0-7
/proc/9058/task/9068/stat : 9068 (plingeling) R 9056 9058 8836 0 -1 4202560 434 1253 0 0 18 0 0 0 25 0 9 0 701947749 168087552 2288 33554432000 4194304 4417452 140733651132016 18446744073709551615 4237221 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9058/tid=9069] ppid=9056 vsize=164148 CPUtime=0.18 cores=0-7
/proc/9058/task/9069/stat : 9069 (plingeling) R 9056 9058 8836 0 -1 4202560 456 1253 0 0 18 0 0 0 18 0 9 0 701947749 168087552 2288 33554432000 4194304 4417452 140733651132016 18446744073709551615 4353957 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9058/tid=9070] ppid=9056 vsize=164148 CPUtime=0.19 cores=0-7
/proc/9058/task/9070/stat : 9070 (plingeling) R 9056 9058 8836 0 -1 4202560 490 1253 0 0 19 0 0 0 25 0 9 0 701947749 168087552 2288 33554432000 4194304 4417452 140733651132016 18446744073709551615 4310365 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9058/tid=9071] ppid=9056 vsize=164148 CPUtime=0.19 cores=0-7
/proc/9058/task/9071/stat : 9071 (plingeling) R 9056 9058 8836 0 -1 4202560 191 1253 0 0 19 0 0 0 18 0 9 0 701947749 168087552 2288 33554432000 4194304 4417452 140733651132016 18446744073709551615 4356977 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 164148

[startup+0.30131 s]
/proc/loadavg: 8.14 8.10 8.01 1/528 9058
/proc/meminfo: memFree=15528996/32951124 swapFree=51873512/67111528
[pid=9058] ppid=9056 vsize=167612 CPUtime=2.28 cores=0-7
/proc/9058/stat : 9058 (plingeling) S 9056 9058 8836 0 -1 4202496 4972 1253 0 0 228 0 0 0 18 0 9 0 701947749 171634688 3166 33554432000 4194304 4417452 140733651132016 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/9058/statm: 41903 3166 180 55 0 39789 0
[pid=9058/tid=9064] ppid=9056 vsize=167612 CPUtime=0.29 cores=0-7
/proc/9058/task/9064/stat : 9064 (plingeling) R 9056 9058 8836 0 -1 4202560 517 1253 0 0 29 0 0 0 25 0 9 0 701947749 171634688 3166 33554432000 4194304 4417452 140733651132016 18446744073709551615 4306260 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9058/tid=9065] ppid=9056 vsize=167612 CPUtime=0.27 cores=0-7
/proc/9058/task/9065/stat : 9065 (plingeling) R 9056 9058 8836 0 -1 4202560 330 1253 0 0 27 0 0 0 25 0 9 0 701947749 171634688 3166 33554432000 4194304 4417452 140733651132016 18446744073709551615 4309988 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9058/tid=9066] ppid=9056 vsize=167612 CPUtime=0.27 cores=0-7
/proc/9058/task/9066/stat : 9066 (plingeling) R 9056 9058 8836 0 -1 4202560 325 1253 0 0 27 0 0 0 25 0 9 0 701947749 171634688 3166 33554432000 4194304 4417452 140733651132016 18446744073709551615 4366079 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9058/tid=9067] ppid=9056 vsize=167612 CPUtime=0.27 cores=0-7
/proc/9058/task/9067/stat : 9067 (plingeling) R 9056 9058 8836 0 -1 4202560 1101 1253 0 0 27 0 0 0 25 0 9 0 701947749 171634688 3166 33554432000 4194304 4417452 140733651132016 18446744073709551615 4356898 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9058/tid=9068] ppid=9056 vsize=167612 CPUtime=0.28 cores=0-7
/proc/9058/task/9068/stat : 9068 (plingeling) R 9056 9058 8836 0 -1 4202560 713 1253 0 0 28 0 0 0 25 0 9 0 701947749 171634688 3166 33554432000 4194304 4417452 140733651132016 18446744073709551615 4353900 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9058/tid=9069] ppid=9056 vsize=167612 CPUtime=0.28 cores=0-7
/proc/9058/task/9069/stat : 9069 (plingeling) R 9056 9058 8836 0 -1 4202560 672 1253 0 0 28 0 0 0 18 0 9 0 701947749 171634688 3166 33554432000 4194304 4417452 140733651132016 18446744073709551615 4307411 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9058/tid=9070] ppid=9056 vsize=167612 CPUtime=0.29 cores=0-7
/proc/9058/task/9070/stat : 9070 (plingeling) R 9056 9058 8836 0 -1 4202560 564 1253 0 0 29 0 0 0 25 0 9 0 701947749 171634688 3166 33554432000 4194304 4417452 140733651132016 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9058/tid=9071] ppid=9056 vsize=167612 CPUtime=0.29 cores=0-7
/proc/9058/task/9071/stat : 9071 (plingeling) R 9056 9058 8836 0 -1 4202560 366 1253 0 0 29 0 0 0 18 0 9 0 701947749 171634688 3166 33554432000 4194304 4417452 140733651132016 18446744073709551615 4309871 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2.28
Current children cumulated vsize (KiB) 167612

[startup+0.701239 s]
/proc/loadavg: 8.14 8.10 8.01 1/528 9058
/proc/meminfo: memFree=15528996/32951124 swapFree=51873512/67111528
[pid=9058] ppid=9056 vsize=172228 CPUtime=5.48 cores=0-7
/proc/9058/stat : 9058 (plingeling) S 9056 9058 8836 0 -1 4202496 12094 1253 0 0 546 2 0 0 18 0 9 0 701947749 176361472 4331 33554432000 4194304 4417452 140733651132016 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/9058/statm: 43057 4331 180 55 0 40943 0
[pid=9058/tid=9064] ppid=9056 vsize=172228 CPUtime=0.69 cores=0-7
/proc/9058/task/9064/stat : 9064 (plingeling) R 9056 9058 8836 0 -1 4202560 1297 1253 0 0 69 0 0 0 25 0 9 0 701947749 176361472 4331 33554432000 4194304 4417452 140733651132016 18446744073709551615 4251261 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9058/tid=9065] ppid=9056 vsize=172228 CPUtime=0.67 cores=0-7
/proc/9058/task/9065/stat : 9065 (plingeling) R 9056 9058 8836 0 -1 4202560 1122 1253 0 0 67 0 0 0 25 0 9 0 701947749 176361472 4331 33554432000 4194304 4417452 140733651132016 18446744073709551615 4306294 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9058/tid=9066] ppid=9056 vsize=172228 CPUtime=0.67 cores=0-7
/proc/9058/task/9066/stat : 9066 (plingeling) R 9056 9058 8836 0 -1 4202560 1191 1253 0 0 67 0 0 0 25 0 9 0 701947749 176361472 4331 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9058/tid=9067] ppid=9056 vsize=172228 CPUtime=0.67 cores=0-7
/proc/9058/task/9067/stat : 9067 (plingeling) R 9056 9058 8836 0 -1 4202560 2179 1253 0 0 67 0 0 0 25 0 9 0 701947749 176361472 4331 33554432000 4194304 4417452 140733651132016 18446744073709551615 4366385 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9058/tid=9068] ppid=9056 vsize=172228 CPUtime=0.68 cores=0-7
/proc/9058/task/9068/stat : 9068 (plingeling) R 9056 9058 8836 0 -1 4202560 1694 1253 0 0 68 0 0 0 25 0 9 0 701947749 176361472 4331 33554432000 4194304 4417452 140733651132016 18446744073709551615 18446744073699065949 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9058/tid=9069] ppid=9056 vsize=172228 CPUtime=0.68 cores=0-7
/proc/9058/task/9069/stat : 9069 (plingeling) R 9056 9058 8836 0 -1 4202560 1799 1253 0 0 68 0 0 0 18 0 9 0 701947749 176361472 4331 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364453 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9058/tid=9070] ppid=9056 vsize=172228 CPUtime=0.69 cores=0-7
/proc/9058/task/9070/stat : 9070 (plingeling) R 9056 9058 8836 0 -1 4202560 1225 1253 0 0 69 0 0 0 25 0 9 0 701947749 176361472 4331 33554432000 4194304 4417452 140733651132016 18446744073709551615 4305799 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9058/tid=9071] ppid=9056 vsize=172228 CPUtime=0.68 cores=0-7
/proc/9058/task/9071/stat : 9071 (plingeling) R 9056 9058 8836 0 -1 4202560 1203 1253 0 0 68 0 0 0 18 0 9 0 701947749 176361472 4331 33554432000 4194304 4417452 140733651132016 18446744073709551615 4308211 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 5.48
Current children cumulated vsize (KiB) 172228

[startup+1.5011 s]
/proc/loadavg: 8.14 8.10 8.01 9/537 9072
/proc/meminfo: memFree=15506860/32951124 swapFree=51873512/67111528
[pid=9058] ppid=9056 vsize=177064 CPUtime=11.86 cores=0-7
/proc/9058/stat : 9058 (plingeling) S 9056 9058 8836 0 -1 4202496 27394 1253 0 0 1182 4 0 0 18 0 9 0 701947749 181313536 5541 33554432000 4194304 4417452 140733651132016 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/9058/statm: 44266 5541 180 55 0 42152 0
[pid=9058/tid=9064] ppid=9056 vsize=177064 CPUtime=1.49 cores=0-7
/proc/9058/task/9064/stat : 9064 (plingeling) R 9056 9058 8836 0 -1 4202560 3058 1253 0 0 149 0 0 0 25 0 9 0 701947749 181313536 5541 33554432000 4194304 4417452 140733651132016 18446744073709551615 4366105 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9058/tid=9065] ppid=9056 vsize=177064 CPUtime=1.47 cores=0-7
/proc/9058/task/9065/stat : 9065 (plingeling) R 9056 9058 8836 0 -1 4202560 3475 1253 0 0 147 0 0 0 25 0 9 0 701947749 181313536 5541 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9058/tid=9066] ppid=9056 vsize=177064 CPUtime=1.44 cores=0-7
/proc/9058/task/9066/stat : 9066 (plingeling) R 9056 9058 8836 0 -1 4202560 2584 1253 0 0 144 0 0 0 25 0 9 0 701947749 181313536 5541 33554432000 4194304 4417452 140733651132016 18446744073709551615 4269824 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9058/tid=9067] ppid=9056 vsize=177064 CPUtime=1.47 cores=0-7
/proc/9058/task/9067/stat : 9067 (plingeling) R 9056 9058 8836 0 -1 4202560 3973 1253 0 0 147 0 0 0 25 0 9 0 701947749 181313536 5541 33554432000 4194304 4417452 140733651132016 18446744073709551615 4357856 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9058/tid=9068] ppid=9056 vsize=177064 CPUtime=1.48 cores=0-7
/proc/9058/task/9068/stat : 9068 (plingeling) R 9056 9058 8836 0 -1 4202560 3343 1253 0 0 148 0 0 0 25 0 9 0 701947749 181313536 5541 33554432000 4194304 4417452 140733651132016 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9058/tid=9069] ppid=9056 vsize=177064 CPUtime=1.47 cores=0-7
/proc/9058/task/9069/stat : 9069 (plingeling) R 9056 9058 8836 0 -1 4202560 3479 1253 0 0 147 0 0 0 20 0 9 0 701947749 181313536 5541 33554432000 4194304 4417452 140733651132016 18446744073709551615 4354515 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9058/tid=9070] ppid=9056 vsize=177064 CPUtime=1.48 cores=0-7
/proc/9058/task/9070/stat : 9070 (plingeling) R 9056 9058 8836 0 -1 4202560 3516 1253 0 0 148 0 0 0 25 0 9 0 701947749 181313536 5541 33554432000 4194304 4417452 140733651132016 18446744073709551615 4307697 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9058/tid=9071] ppid=9056 vsize=177064 CPUtime=1.48 cores=0-7
/proc/9058/task/9071/stat : 9071 (plingeling) R 9056 9058 8836 0 -1 4202560 3582 1253 0 0 148 0 0 0 20 0 9 0 701947749 181313536 5541 33554432000 4194304 4417452 140733651132016 18446744073709551615 4366520 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 11.86
Current children cumulated vsize (KiB) 177064

[startup+3.10081 s]
/proc/loadavg: 8.13 8.09 8.01 9/537 9072
/proc/meminfo: memFree=15500928/32951124 swapFree=51873512/67111528
[pid=9058] ppid=9056 vsize=189816 CPUtime=24.64 cores=0-7
/proc/9058/stat : 9058 (plingeling) S 9056 9058 8836 0 -1 4202496 56774 1253 0 0 2453 11 0 0 18 0 9 0 701947749 194371584 8729 33554432000 4194304 4417452 140733651132016 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/9058/statm: 47454 8729 180 55 0 45340 0
[pid=9058/tid=9064] ppid=9056 vsize=189816 CPUtime=3.08 cores=0-7
/proc/9058/task/9064/stat : 9064 (plingeling) R 9056 9058 8836 0 -1 4202560 7294 1253 0 0 307 1 0 0 25 0 9 0 701947749 194371584 8729 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9058/tid=9065] ppid=9056 vsize=189816 CPUtime=3.07 cores=0-7
/proc/9058/task/9065/stat : 9065 (plingeling) R 9056 9058 8836 0 -1 4202560 7672 1253 0 0 306 1 0 0 25 0 9 0 701947749 194371584 8729 33554432000 4194304 4417452 140733651132016 18446744073709551615 18446744073699065894 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9058/tid=9066] ppid=9056 vsize=189816 CPUtime=3.02 cores=0-7
/proc/9058/task/9066/stat : 9066 (plingeling) R 9056 9058 8836 0 -1 4202560 6964 1253 0 0 301 1 0 0 25 0 9 0 701947749 194371584 8729 33554432000 4194304 4417452 140733651132016 18446744073709551615 4321391 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9058/tid=9067] ppid=9056 vsize=189816 CPUtime=3.07 cores=0-7
/proc/9058/task/9067/stat : 9067 (plingeling) R 9056 9058 8836 0 -1 4202560 6206 1253 0 0 306 1 0 0 25 0 9 0 701947749 194371584 8729 33554432000 4194304 4417452 140733651132016 18446744073709551615 4357562 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9058/tid=9068] ppid=9056 vsize=189816 CPUtime=3.08 cores=0-7
/proc/9058/task/9068/stat : 9068 (plingeling) R 9056 9058 8836 0 -1 4202560 6428 1253 0 0 307 1 0 0 25 0 9 0 701947749 194371584 8729 33554432000 4194304 4417452 140733651132016 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9058/tid=9069] ppid=9056 vsize=189816 CPUtime=3.08 cores=0-7
/proc/9058/task/9069/stat : 9069 (plingeling) R 9056 9058 8836 0 -1 4202560 7484 1253 0 0 307 1 0 0 25 0 9 0 701947749 194371584 8729 33554432000 4194304 4417452 140733651132016 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9058/tid=9070] ppid=9056 vsize=189816 CPUtime=3.08 cores=0-7
/proc/9058/task/9070/stat : 9070 (plingeling) R 9056 9058 8836 0 -1 4202560 7618 1253 0 0 307 1 0 0 25 0 9 0 701947749 194371584 8729 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9058/tid=9071] ppid=9056 vsize=189816 CPUtime=3.08 cores=0-7
/proc/9058/task/9071/stat : 9071 (plingeling) R 9056 9058 8836 0 -1 4202560 6724 1253 0 0 308 0 0 0 25 0 9 0 701947749 194371584 8729 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 24.64
Current children cumulated vsize (KiB) 189816

[startup+6.30125 s]
/proc/loadavg: 8.13 8.09 8.01 9/537 9072
/proc/meminfo: memFree=15487032/32951124 swapFree=51873512/67111528
[pid=9058] ppid=9056 vsize=195156 CPUtime=50.17 cores=0-7
/proc/9058/stat : 9058 (plingeling) S 9056 9058 8836 0 -1 4202496 115778 1253 0 0 4996 21 0 0 18 0 9 0 701947749 199839744 10031 33554432000 4194304 4417452 140733651132016 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/9058/statm: 48789 10031 180 55 0 46675 0

################
# More data... #
################

/proc/9058/task/9067/stat : 9067 (plingeling) R 9056 9058 8836 0 -1 4202560 1016249 1253 0 0 466008 163 0 0 25 0 9 0 701947749 616497152 111754 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364711 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9058/tid=9068] ppid=9056 vsize=602048 CPUtime=4662.4 cores=0-7
/proc/9058/task/9068/stat : 9068 (plingeling) R 9056 9058 8836 0 -1 4202560 911953 1253 0 0 466083 157 0 0 25 0 9 0 701947749 616497152 111754 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9058/tid=9069] ppid=9056 vsize=602048 CPUtime=4662.38 cores=0-7
/proc/9058/task/9069/stat : 9069 (plingeling) R 9056 9058 8836 0 -1 4202560 870134 1253 0 0 466084 154 0 0 25 0 9 0 701947749 616497152 111754 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364984 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9058/tid=9070] ppid=9056 vsize=602048 CPUtime=4662.27 cores=0-7
/proc/9058/task/9070/stat : 9070 (plingeling) R 9056 9058 8836 0 -1 4202560 895210 1253 0 0 466073 154 0 0 25 0 9 0 701947749 616497152 111754 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364984 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9058/tid=9071] ppid=9056 vsize=602048 CPUtime=4662.37 cores=0-7
/proc/9058/task/9071/stat : 9071 (plingeling) R 9056 9058 8836 0 -1 4202560 861415 1253 0 0 466086 151 0 0 25 0 9 0 701947749 616497152 111754 33554432000 4194304 4417452 140733651132016 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37211.4
Current children cumulated vsize (KiB) 602048

[startup+4722.3 s]
/proc/loadavg: 8.12 8.07 8.03 9/537 9213
/proc/meminfo: memFree=15034664/32951124 swapFree=51873512/67111528
[pid=9058] ppid=9056 vsize=649140 CPUtime=37690.4 cores=0-7
/proc/9058/stat : 9058 (plingeling) S 9056 9058 8836 0 -1 4202496 7287283 1253 0 0 3767770 1269 0 0 18 0 9 0 701947749 664719360 123527 33554432000 4194304 4417452 140733651132016 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/9058/statm: 162285 123527 180 55 0 160171 0
[pid=9058/tid=9064] ppid=9056 vsize=649140 CPUtime=4720.06 cores=0-7
/proc/9058/task/9064/stat : 9064 (plingeling) R 9056 9058 8836 0 -1 4202560 900803 1253 0 0 471841 165 0 0 25 0 9 0 701947749 664719360 123527 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9058/tid=9065] ppid=9056 vsize=649140 CPUtime=4722.41 cores=0-7
/proc/9058/task/9065/stat : 9065 (plingeling) R 9056 9058 8836 0 -1 4202560 949820 1253 0 0 472076 165 0 0 25 0 9 0 701947749 664719360 123527 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9058/tid=9066] ppid=9056 vsize=649140 CPUtime=4636.7 cores=0-7
/proc/9058/task/9066/stat : 9066 (plingeling) R 9056 9058 8836 0 -1 4202560 850380 1253 0 0 463520 150 0 0 25 0 9 0 701947749 664719360 123527 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9058/tid=9067] ppid=9056 vsize=649140 CPUtime=4721.73 cores=0-7
/proc/9058/task/9067/stat : 9067 (plingeling) R 9056 9058 8836 0 -1 4202560 1024733 1253 0 0 472009 164 0 0 25 0 9 0 701947749 664719360 123527 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364885 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9058/tid=9068] ppid=9056 vsize=649140 CPUtime=4722.41 cores=0-7
/proc/9058/task/9068/stat : 9068 (plingeling) R 9056 9058 8836 0 -1 4202560 924315 1253 0 0 472082 159 0 0 25 0 9 0 701947749 664719360 123527 33554432000 4194304 4417452 140733651132016 18446744073709551615 4321714 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9058/tid=9069] ppid=9056 vsize=649140 CPUtime=4722.4 cores=0-7
/proc/9058/task/9069/stat : 9069 (plingeling) R 9056 9058 8836 0 -1 4202560 880223 1253 0 0 472084 156 0 0 25 0 9 0 701947749 664719360 123527 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9058/tid=9070] ppid=9056 vsize=649140 CPUtime=4722.29 cores=0-7
/proc/9058/task/9070/stat : 9070 (plingeling) R 9056 9058 8836 0 -1 4202560 895210 1253 0 0 472075 154 0 0 25 0 9 0 701947749 664719360 123527 33554432000 4194304 4417452 140733651132016 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9058/tid=9071] ppid=9056 vsize=649140 CPUtime=4722.39 cores=0-7
/proc/9058/task/9071/stat : 9071 (plingeling) R 9056 9058 8836 0 -1 4202560 861415 1253 0 0 472088 151 0 0 25 0 9 0 701947749 664719360 123527 33554432000 4194304 4417452 140733651132016 18446744073709551615 4366090 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37690.4
Current children cumulated vsize (KiB) 649140

[startup+4782.3 s]
/proc/loadavg: 8.12 8.07 8.03 9/537 9215
/proc/meminfo: memFree=15122332/32951124 swapFree=51873512/67111528
[pid=9058] ppid=9056 vsize=561612 CPUtime=38169.2 cores=0-7
/proc/9058/stat : 9058 (plingeling) S 9056 9058 8836 0 -1 4202496 7321420 1253 0 0 3815650 1274 0 0 18 0 9 0 701947749 575090688 101645 33554432000 4194304 4417452 140733651132016 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/9058/statm: 140403 101645 180 55 0 138289 0
[pid=9058/tid=9064] ppid=9056 vsize=561612 CPUtime=4780.02 cores=0-7
/proc/9058/task/9064/stat : 9064 (plingeling) R 9056 9058 8836 0 -1 4202560 904645 1253 0 0 477837 165 0 0 25 0 9 0 701947749 575090688 101645 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9058/tid=9065] ppid=9056 vsize=561612 CPUtime=4782.39 cores=0-7
/proc/9058/task/9065/stat : 9065 (plingeling) R 9056 9058 8836 0 -1 4202560 955636 1253 0 0 478074 165 0 0 25 0 9 0 701947749 575090688 101645 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9058/tid=9066] ppid=9056 vsize=561612 CPUtime=4695.61 cores=0-7
/proc/9058/task/9066/stat : 9066 (plingeling) R 9056 9058 8836 0 -1 4202560 856273 1253 0 0 469410 151 0 0 25 0 9 0 701947749 575090688 101645 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9058/tid=9067] ppid=9056 vsize=561612 CPUtime=4781.71 cores=0-7
/proc/9058/task/9067/stat : 9067 (plingeling) R 9056 9058 8836 0 -1 4202560 1027295 1253 0 0 478007 164 0 0 25 0 9 0 701947749 575090688 101645 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9058/tid=9068] ppid=9056 vsize=561612 CPUtime=4782.4 cores=0-7
/proc/9058/task/9068/stat : 9068 (plingeling) R 9056 9058 8836 0 -1 4202560 924315 1253 0 0 478081 159 0 0 25 0 9 0 701947749 575090688 101645 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9058/tid=9069] ppid=9056 vsize=561612 CPUtime=4782.39 cores=0-7
/proc/9058/task/9069/stat : 9069 (plingeling) R 9056 9058 8836 0 -1 4202560 881914 1253 0 0 478082 157 0 0 25 0 9 0 701947749 575090688 101645 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9058/tid=9070] ppid=9056 vsize=561612 CPUtime=4782.27 cores=0-7
/proc/9058/task/9070/stat : 9070 (plingeling) R 9056 9058 8836 0 -1 4202560 900981 1253 0 0 478071 156 0 0 25 0 9 0 701947749 575090688 101645 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9058/tid=9071] ppid=9056 vsize=561612 CPUtime=4782.37 cores=0-7
/proc/9058/task/9071/stat : 9071 (plingeling) R 9056 9058 8836 0 -1 4202560 869977 1253 0 0 478084 153 0 0 25 0 9 0 701947749 575090688 101645 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38169.2
Current children cumulated vsize (KiB) 561612

[startup+4842.32 s]
/proc/loadavg: 8.11 8.07 8.03 9/537 9216
/proc/meminfo: memFree=15075192/32951124 swapFree=51873512/67111528
[pid=9058] ppid=9056 vsize=609032 CPUtime=38648.3 cores=0-7
/proc/9058/stat : 9058 (plingeling) S 9056 9058 8836 0 -1 4202496 7357315 1253 0 0 3863548 1281 0 0 18 0 9 0 701947749 623648768 113500 33554432000 4194304 4417452 140733651132016 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/9058/statm: 152258 113500 180 55 0 150144 0
[pid=9058/tid=9064] ppid=9056 vsize=609032 CPUtime=4840.04 cores=0-7
/proc/9058/task/9064/stat : 9064 (plingeling) R 9056 9058 8836 0 -1 4202560 910372 1253 0 0 483837 167 0 0 25 0 9 0 701947749 623648768 113500 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9058/tid=9065] ppid=9056 vsize=609032 CPUtime=4842.41 cores=0-7
/proc/9058/task/9065/stat : 9065 (plingeling) R 9056 9058 8836 0 -1 4202560 958791 1253 0 0 484075 166 0 0 25 0 9 0 701947749 623648768 113500 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9058/tid=9066] ppid=9056 vsize=609032 CPUtime=4754.53 cores=0-7
/proc/9058/task/9066/stat : 9066 (plingeling) R 9056 9058 8836 0 -1 4202560 858183 1253 0 0 475302 151 0 0 25 0 9 0 701947749 623648768 113500 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9058/tid=9067] ppid=9056 vsize=609032 CPUtime=4841.74 cores=0-7
/proc/9058/task/9067/stat : 9067 (plingeling) R 9056 9058 8836 0 -1 4202560 1033327 1253 0 0 484008 166 0 0 25 0 9 0 701947749 623648768 113500 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9058/tid=9068] ppid=9056 vsize=609032 CPUtime=4842.42 cores=0-7
/proc/9058/task/9068/stat : 9068 (plingeling) R 9056 9058 8836 0 -1 4202560 928135 1253 0 0 484082 160 0 0 25 0 9 0 701947749 623648768 113500 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9058/tid=9069] ppid=9056 vsize=609032 CPUtime=4842.4 cores=0-7
/proc/9058/task/9069/stat : 9069 (plingeling) R 9056 9058 8836 0 -1 4202560 886548 1253 0 0 484083 157 0 0 25 0 9 0 701947749 623648768 113500 33554432000 4194304 4417452 140733651132016 18446744073709551615 4365023 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9058/tid=9070] ppid=9056 vsize=609032 CPUtime=4842.29 cores=0-7
/proc/9058/task/9070/stat : 9070 (plingeling) R 9056 9058 8836 0 -1 4202560 905723 1253 0 0 484072 157 0 0 25 0 9 0 701947749 623648768 113500 33554432000 4194304 4417452 140733651132016 18446744073709551615 4250936 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9058/tid=9071] ppid=9056 vsize=609032 CPUtime=4842.39 cores=0-7
/proc/9058/task/9071/stat : 9071 (plingeling) R 9056 9058 8836 0 -1 4202560 875852 1253 0 0 484085 154 0 0 25 0 9 0 701947749 623648768 113500 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38648.3
Current children cumulated vsize (KiB) 609032

[startup+4902.3 s]
/proc/loadavg: 8.16 8.08 8.04 9/537 9218
/proc/meminfo: memFree=15116624/32951124 swapFree=51873512/67111528
[pid=9058] ppid=9056 vsize=569056 CPUtime=39127 cores=0-7
/proc/9058/stat : 9058 (plingeling) S 9056 9058 8836 0 -1 4202496 7407862 1253 0 0 3911412 1292 0 0 18 0 9 0 701947749 582713344 103506 33554432000 4194304 4417452 140733651132016 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/9058/statm: 142264 103506 180 55 0 140150 0
[pid=9058/tid=9064] ppid=9056 vsize=569056 CPUtime=4899.98 cores=0-7
/proc/9058/task/9064/stat : 9064 (plingeling) R 9056 9058 8836 0 -1 4202560 913002 1253 0 0 489831 167 0 0 25 0 9 0 701947749 582713344 103506 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9058/tid=9065] ppid=9056 vsize=569056 CPUtime=4902.4 cores=0-7
/proc/9058/task/9065/stat : 9065 (plingeling) R 9056 9058 8836 0 -1 4202560 967978 1253 0 0 490072 168 0 0 25 0 9 0 701947749 582713344 103506 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9058/tid=9066] ppid=9056 vsize=569056 CPUtime=4813.43 cores=0-7
/proc/9058/task/9066/stat : 9066 (plingeling) R 9056 9058 8836 0 -1 4202560 871540 1253 0 0 481189 154 0 0 25 0 9 0 701947749 582713344 103506 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9058/tid=9067] ppid=9056 vsize=569056 CPUtime=4901.72 cores=0-7
/proc/9058/task/9067/stat : 9067 (plingeling) R 9056 9058 8836 0 -1 4202560 1041294 1253 0 0 490004 168 0 0 25 0 9 0 701947749 582713344 103506 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9058/tid=9068] ppid=9056 vsize=569056 CPUtime=4902.4 cores=0-7
/proc/9058/task/9068/stat : 9068 (plingeling) R 9056 9058 8836 0 -1 4202560 934050 1253 0 0 490079 161 0 0 25 0 9 0 701947749 582713344 103506 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9058/tid=9069] ppid=9056 vsize=569056 CPUtime=4902.38 cores=0-7
/proc/9058/task/9069/stat : 9069 (plingeling) R 9056 9058 8836 0 -1 4202560 891843 1253 0 0 490080 158 0 0 25 0 9 0 701947749 582713344 103506 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9058/tid=9070] ppid=9056 vsize=569056 CPUtime=4902.27 cores=0-7
/proc/9058/task/9070/stat : 9070 (plingeling) R 9056 9058 8836 0 -1 4202560 908595 1253 0 0 490069 158 0 0 25 0 9 0 701947749 582713344 103506 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9058/tid=9071] ppid=9056 vsize=569056 CPUtime=4902.38 cores=0-7
/proc/9058/task/9071/stat : 9071 (plingeling) R 9056 9058 8836 0 -1 4202560 879176 1253 0 0 490083 155 0 0 25 0 9 0 701947749 582713344 103506 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39127
Current children cumulated vsize (KiB) 569056

[startup+4962.3 s]
/proc/loadavg: 8.12 8.08 8.04 9/537 9219
/proc/meminfo: memFree=15087424/32951124 swapFree=51873512/67111528
[pid=9058] ppid=9056 vsize=596516 CPUtime=39605.9 cores=0-7
/proc/9058/stat : 9058 (plingeling) S 9056 9058 8836 0 -1 4202496 7463538 1253 0 0 3959291 1303 0 0 18 0 9 0 701947749 610832384 110399 33554432000 4194304 4417452 140733651132016 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/9058/statm: 149129 110399 180 55 0 147015 0
[pid=9058/tid=9064] ppid=9056 vsize=596516 CPUtime=4959.97 cores=0-7
/proc/9058/task/9064/stat : 9064 (plingeling) R 9056 9058 8836 0 -1 4202560 924364 1253 0 0 495828 169 0 0 25 0 9 0 701947749 610832384 110399 33554432000 4194304 4417452 140733651132016 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9058/tid=9065] ppid=9056 vsize=596516 CPUtime=4962.4 cores=0-7
/proc/9058/task/9065/stat : 9065 (plingeling) R 9056 9058 8836 0 -1 4202560 975585 1253 0 0 496071 169 0 0 25 0 9 0 701947749 610832384 110399 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9058/tid=9066] ppid=9056 vsize=596516 CPUtime=4872.33 cores=0-7
/proc/9058/task/9066/stat : 9066 (plingeling) R 9056 9058 8836 0 -1 4202560 876699 1253 0 0 487078 155 0 0 25 0 9 0 701947749 610832384 110399 33554432000 4194304 4417452 140733651132016 18446744073709551615 4365075 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9058/tid=9067] ppid=9056 vsize=596516 CPUtime=4961.72 cores=0-7
/proc/9058/task/9067/stat : 9067 (plingeling) R 9056 9058 8836 0 -1 4202560 1045252 1253 0 0 496003 169 0 0 25 0 9 0 701947749 610832384 110399 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364780 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9058/tid=9068] ppid=9056 vsize=596516 CPUtime=4962.4 cores=0-7
/proc/9058/task/9068/stat : 9068 (plingeling) R 9056 9058 8836 0 -1 4202560 940896 1253 0 0 496078 162 0 0 25 0 9 0 701947749 610832384 110399 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9058/tid=9069] ppid=9056 vsize=596516 CPUtime=4962.39 cores=0-7
/proc/9058/task/9069/stat : 9069 (plingeling) R 9056 9058 8836 0 -1 4202560 898142 1253 0 0 496080 159 0 0 25 0 9 0 701947749 610832384 110399 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9058/tid=9070] ppid=9056 vsize=596516 CPUtime=4962.27 cores=0-7
/proc/9058/task/9070/stat : 9070 (plingeling) R 9056 9058 8836 0 -1 4202560 915312 1253 0 0 496068 159 0 0 25 0 9 0 701947749 610832384 110399 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9058/tid=9071] ppid=9056 vsize=596516 CPUtime=4962.38 cores=0-7
/proc/9058/task/9071/stat : 9071 (plingeling) R 9056 9058 8836 0 -1 4202560 886904 1253 0 0 496082 156 0 0 25 0 9 0 701947749 610832384 110399 33554432000 4194304 4417452 140733651132016 18446744073709551615 4366400 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39605.9
Current children cumulated vsize (KiB) 596516



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5011.72 s]
/proc/loadavg: 8.05 8.07 8.03 9/537 9221
/proc/meminfo: memFree=14955856/32951124 swapFree=51873512/67111528
[pid=9058] ppid=9056 vsize=727996 CPUtime=40000.4 cores=0-7
/proc/9058/stat : 9058 (plingeling) S 9056 9058 8836 0 -1 4202496 7508955 1253 0 0 3998730 1313 0 0 18 0 9 0 701947749 745467904 143269 33554432000 4194304 4417452 140733651132016 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/9058/statm: 181999 143269 180 55 0 179885 0
[pid=9058/tid=9064] ppid=9056 vsize=727996 CPUtime=5009.41 cores=0-7
/proc/9058/task/9064/stat : 9064 (plingeling) R 9056 9058 8836 0 -1 4202560 924750 1253 0 0 500771 170 0 0 25 0 9 0 701947749 745467904 143269 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9058/tid=9065] ppid=9056 vsize=727996 CPUtime=5011.83 cores=0-7
/proc/9058/task/9065/stat : 9065 (plingeling) R 9056 9058 8836 0 -1 4202560 983159 1253 0 0 501012 171 0 0 25 0 9 0 701947749 745467904 143269 33554432000 4194304 4417452 140733651132016 18446744073709551615 4366099 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9058/tid=9066] ppid=9056 vsize=727996 CPUtime=4920.85 cores=0-7
/proc/9058/task/9066/stat : 9066 (plingeling) R 9056 9058 8836 0 -1 4202560 878715 1253 0 0 491930 155 0 0 25 0 9 0 701947749 745467904 143269 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9058/tid=9067] ppid=9056 vsize=727996 CPUtime=5011.16 cores=0-7
/proc/9058/task/9067/stat : 9067 (plingeling) R 9056 9058 8836 0 -1 4202560 1050503 1253 0 0 500946 170 0 0 25 0 9 0 701947749 745467904 143269 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9058/tid=9068] ppid=9056 vsize=727996 CPUtime=5011.84 cores=0-7
/proc/9058/task/9068/stat : 9068 (plingeling) R 9056 9058 8836 0 -1 4202560 951817 1253 0 0 501020 164 0 0 25 0 9 0 701947749 745467904 143269 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9058/tid=9069] ppid=9056 vsize=727996 CPUtime=5011.83 cores=0-7
/proc/9058/task/9069/stat : 9069 (plingeling) R 9056 9058 8836 0 -1 4202560 908620 1253 0 0 501022 161 0 0 25 0 9 0 701947749 745467904 143269 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9058/tid=9070] ppid=9056 vsize=727996 CPUtime=5011.71 cores=0-7
/proc/9058/task/9070/stat : 9070 (plingeling) R 9056 9058 8836 0 -1 4202560 921865 1253 0 0 501010 161 0 0 25 0 9 0 701947749 745467904 143269 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9058/tid=9071] ppid=9056 vsize=727996 CPUtime=5011.82 cores=0-7
/proc/9058/task/9071/stat : 9071 (plingeling) R 9056 9058 8836 0 -1 4202560 889142 1253 0 0 501025 157 0 0 25 0 9 0 701947749 745467904 143269 33554432000 4194304 4417452 140733651132016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 727996

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5011.77
CPU time (s): 40000.6
CPU user time (s): 39987.4
CPU system time (s): 13.164
CPU usage (%): 798.133
Max. virtual memory (cumulated for all children) (KiB): 727996

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39987.4
system time used= 13.164
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 7510208
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 707
involuntary context switches= 190669

runsolver used 25.977 second user time and 65.0211 second system time

The end

Launcher Data

Begin job on node146 at 2011-05-22 22:15:15
IDJOB=3394410
IDBENCH=83149
IDSOLVER=1851
FILE ID=node146/3394410-1306095315
RUNJOBID= node146-1306090301-8850
PBS_JOBID= 13387930
Free space on /tmp= 73036 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n280-s93756482-sat.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394410-1306095315/watcher-3394410-1306095315 -o /tmp/evaluation-result-3394410-1306095315/solver-3394410-1306095315 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394410-1306095315.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= badb3b4ac12b29ada7dd8ed96ed75c80
RANDOM SEED=1889658451

node146.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      15529400 kB
Buffers:        311036 kB
Cached:        4765080 kB
SwapCached:      56204 kB
Active:       12504396 kB
Inactive:      4602540 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      15529400 kB
SwapTotal:    67111528 kB
SwapFree:     51873512 kB
Dirty:              44 kB
Writeback:          24 kB
AnonPages:    11981432 kB
Mapped:          29816 kB
Slab:           143880 kB
PageTables:      97460 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 160669572 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73036 MiB
End job on node146 at 2011-05-22 23:38:49