Trace number 3371040

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.47 1204.04

General information on the benchmark

NameSAT03/handmade/anton/UNSAT/
SGI_30_60_28_40_7-log.shuffled-as.sat03-127.cnf
MD5SUMf1f35d90237c4ca07d41a6a9b08e6279
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables140
Number of clauses58376
Sum of the clauses size583480
Maximum clause length10
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 556
Number of clauses of size over 558320

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371040-1305429874/watcher-3371040-1305429874 -o /tmp/evaluation-result-3371040-1305429874/solver-3371040-1305429874 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371040-1305429874.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.12 7.99 6.33 6/368 16243
/proc/meminfo: memFree=30197724/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=10648 CPUtime=0 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) R 16241 16243 8955 0 -1 4202496 755 1253 0 0 0 0 0 0 23 0 1 0 635403708 10903552 685 33554432000 4194304 4417452 140735341181680 18446744073709551615 212039655168 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/16243/statm: 2662 685 148 55 0 548 0

[startup+0.094268 s]
/proc/loadavg: 8.12 7.99 6.33 6/368 16243
/proc/meminfo: memFree=30197724/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=25284 CPUtime=0.08 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) R 16241 16243 8955 0 -1 4202496 4533 1253 0 0 8 0 0 0 23 0 1 0 635403708 25890816 4344 33554432000 4194304 4417452 140735341181680 18446744073709551615 212039274036 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/16243/statm: 6321 4344 148 55 0 4207 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 25284

[startup+0.100263 s]
/proc/loadavg: 8.12 7.99 6.33 6/368 16243
/proc/meminfo: memFree=30197724/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=25284 CPUtime=0.09 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) R 16241 16243 8955 0 -1 4202496 4533 1253 0 0 9 0 0 0 23 0 1 0 635403708 25890816 4344 33554432000 4194304 4417452 140735341181680 18446744073709551615 212039655168 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/16243/statm: 6321 4344 148 55 0 4207 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 25284

[startup+0.30019 s]
/proc/loadavg: 8.12 7.99 6.33 6/368 16243
/proc/meminfo: memFree=30197724/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=89756 CPUtime=0.75 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 12010 1253 0 0 74 1 0 0 23 0 5 0 635403708 91910144 10223 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 22439 10223 177 55 0 20325 0
Current children cumulated CPU time (s) 0.75
Current children cumulated vsize (KiB) 89756

[startup+0.700121 s]
/proc/loadavg: 8.12 7.99 6.33 6/368 16243
/proc/meminfo: memFree=30197724/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=152156 CPUtime=2.36 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 12675 1253 0 0 234 2 0 0 23 0 5 0 635403708 155807744 9441 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 38039 9441 178 55 0 35925 0
Current children cumulated CPU time (s) 2.36
Current children cumulated vsize (KiB) 152156

[startup+1.50098 s]
/proc/loadavg: 8.12 7.99 6.33 10/373 16253
/proc/meminfo: memFree=30156152/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=147504 CPUtime=5.55 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 15673 1253 0 0 553 2 0 0 23 0 5 0 635403708 151044096 8278 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 36876 8278 178 55 0 34762 0
[pid=16243/tid=16250] ppid=16241 vsize=147504 CPUtime=1.35 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 1645 1253 0 0 135 0 0 0 25 0 5 0 635403722 151044096 8278 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=147504 CPUtime=1.33 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 1732 1253 0 0 133 0 0 0 25 0 5 0 635403722 151044096 8278 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=147504 CPUtime=1.35 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 2306 1253 0 0 135 0 0 0 25 0 5 0 635403722 151044096 8278 33554432000 4194304 4417452 140735341181680 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=147504 CPUtime=1.35 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 1349 1253 0 0 135 0 0 0 19 0 5 0 635403722 151044096 8278 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.55
Current children cumulated vsize (KiB) 147504

[startup+3.10069 s]
/proc/loadavg: 8.11 7.99 6.34 10/373 16253
/proc/meminfo: memFree=30141024/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=151724 CPUtime=11.94 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 23380 1253 0 0 1190 4 0 0 23 0 5 0 635403708 155365376 9362 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 37931 9362 178 55 0 35817 0
[pid=16243/tid=16250] ppid=16241 vsize=151724 CPUtime=2.95 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 3678 1253 0 0 295 0 0 0 25 0 5 0 635403722 155365376 9362 33554432000 4194304 4417452 140735341181680 18446744073709551615 4250623 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=151724 CPUtime=2.92 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 3506 1253 0 0 292 0 0 0 25 0 5 0 635403722 155365376 9362 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=151724 CPUtime=2.95 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 4032 1253 0 0 295 0 0 0 25 0 5 0 635403722 155365376 9362 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364812 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=151724 CPUtime=2.95 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 3523 1253 0 0 295 0 0 0 24 0 5 0 635403722 155365376 9362 33554432000 4194304 4417452 140735341181680 18446744073709551615 4357241 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 11.94
Current children cumulated vsize (KiB) 151724

[startup+6.30112 s]
/proc/loadavg: 8.11 7.99 6.34 9/373 16253
/proc/meminfo: memFree=30112020/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=154508 CPUtime=24.69 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 36609 1253 0 0 2463 6 0 0 23 0 5 0 635403708 158216192 10054 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 38627 10054 178 55 0 36513 0
[pid=16243/tid=16250] ppid=16241 vsize=154508 CPUtime=6.15 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 6942 1253 0 0 614 1 0 0 25 0 5 0 635403722 158216192 10054 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=154508 CPUtime=6.09 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 6226 1253 0 0 608 1 0 0 25 0 5 0 635403722 158216192 10054 33554432000 4194304 4417452 140735341181680 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=154508 CPUtime=6.14 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 9038 1253 0 0 613 1 0 0 25 0 5 0 635403722 158216192 10054 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=154508 CPUtime=6.15 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 5762 1253 0 0 614 1 0 0 25 0 5 0 635403722 158216192 10054 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 24.69
Current children cumulated vsize (KiB) 154508

[startup+12.701 s]
/proc/loadavg: 8.17 8.00 6.37 10/373 16253
/proc/meminfo: memFree=30047068/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=160872 CPUtime=50.21 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 59158 1253 0 0 5011 10 0 0 23 0 5 0 635403708 164732928 11620 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 40218 11620 178 55 0 38104 0
[pid=16243/tid=16250] ppid=16241 vsize=160872 CPUtime=12.55 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 12324 1253 0 0 1253 2 0 0 25 0 5 0 635403722 164732928 11620 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=160872 CPUtime=12.4 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 12238 1253 0 0 1239 1 0 0 25 0 5 0 635403722 164732928 11620 33554432000 4194304 4417452 140735341181680 18446744073709551615 4251104 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=160872 CPUtime=12.54 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 15375 1253 0 0 1252 2 0 0 25 0 5 0 635403722 164732928 11620 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=160872 CPUtime=12.54 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 10580 1253 0 0 1253 1 0 0 25 0 5 0 635403722 164732928 11620 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 50.21
Current children cumulated vsize (KiB) 160872

[startup+25.5007 s]
/proc/loadavg: 8.14 8.00 6.38 10/373 16256
/proc/meminfo: memFree=29955924/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=165584 CPUtime=101.25 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 96562 1253 0 0 10109 16 0 0 23 0 5 0 635403708 169558016 12830 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 41396 12830 178 55 0 39282 0
[pid=16243/tid=16250] ppid=16241 vsize=165584 CPUtime=25.35 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 20660 1253 0 0 2532 3 0 0 25 0 5 0 635403722 169558016 12830 33554432000 4194304 4417452 140735341181680 18446744073709551615 4366141 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=165584 CPUtime=25.05 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 20883 1253 0 0 2502 3 0 0 25 0 5 0 635403722 169558016 12830 33554432000 4194304 4417452 140735341181680 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=165584 CPUtime=25.34 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 25754 1253 0 0 2530 4 0 0 25 0 5 0 635403722 169558016 12830 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=165584 CPUtime=25.35 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 20624 1253 0 0 2532 3 0 0 25 0 5 0 635403722 169558016 12830 33554432000 4194304 4417452 140735341181680 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 101.25
Current children cumulated vsize (KiB) 165584

[startup+51.1011 s]
/proc/loadavg: 8.30 8.05 6.44 10/373 16257
/proc/meminfo: memFree=29891848/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=206320 CPUtime=202.54 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 166787 1253 0 0 20226 28 0 0 23 0 5 0 635403708 211271680 22982 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 51580 22982 178 55 0 49466 0
[pid=16243/tid=16250] ppid=16241 vsize=206320 CPUtime=50.94 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 42944 1253 0 0 5086 8 0 0 25 0 5 0 635403722 211271680 22982 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=206320 CPUtime=49.55 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 36494 1253 0 0 4950 5 0 0 25 0 5 0 635403722 211271680 22982 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=206320 CPUtime=50.93 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 44123 1253 0 0 5086 7 0 0 25 0 5 0 635403722 211271680 22982 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=206320 CPUtime=50.95 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 34585 1253 0 0 5090 5 0 0 25 0 5 0 635403722 211271680 22982 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 202.54
Current children cumulated vsize (KiB) 206320

[startup+102.314 s]
/proc/loadavg: 8.20 8.07 6.54 10/373 16259
/proc/meminfo: memFree=29751404/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=210012 CPUtime=406.77 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 264245 1253 0 0 40635 42 0 0 23 0 5 0 635403708 215052288 23905 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 52503 23905 178 55 0 50389 0
[pid=16243/tid=16250] ppid=16241 vsize=210012 CPUtime=102.15 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 68513 1253 0 0 10204 11 0 0 25 0 5 0 635403722 215052288 23905 33554432000 4194304 4417452 140735341181680 18446744073709551615 4366400 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=210012 CPUtime=100.16 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 56880 1253 0 0 10008 8 0 0 25 0 5 0 635403722 215052288 23905 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=210012 CPUtime=102.14 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 70434 1253 0 0 10203 11 0 0 25 0 5 0 635403722 215052288 23905 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=210012 CPUtime=102.17 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 59777 1253 0 0 10207 10 0 0 25 0 5 0 635403722 215052288 23905 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 406.77
Current children cumulated vsize (KiB) 210012

[startup+162.3 s]
/proc/loadavg: 8.15 8.08 6.64 10/373 16260
/proc/meminfo: memFree=29609500/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=209212 CPUtime=646.01 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 366579 1253 0 0 64543 58 0 0 23 0 5 0 635403708 214233088 23705 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 52303 23705 178 55 0 50189 0
[pid=16243/tid=16250] ppid=16241 vsize=209212 CPUtime=162.14 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 100249 1253 0 0 16198 16 0 0 25 0 5 0 635403722 214233088 23705 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=209212 CPUtime=159.44 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 77370 1253 0 0 15933 11 0 0 25 0 5 0 635403722 214233088 23705 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=209212 CPUtime=162.12 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 94585 1253 0 0 16197 15 0 0 25 0 5 0 635403722 214233088 23705 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=209212 CPUtime=162.15 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 85734 1253 0 0 16202 13 0 0 25 0 5 0 635403722 214233088 23705 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364696 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 646.01
Current children cumulated vsize (KiB) 209212


################
# More data... #
################

[pid=16243/tid=16251] ppid=16241 vsize=341072 CPUtime=693.07 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 226852 1253 0 0 69274 33 0 0 25 0 5 0 635403722 349257728 56670 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=341072 CPUtime=702.02 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 246328 1253 0 0 70163 39 0 0 25 0 5 0 635403722 349257728 56670 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=341072 CPUtime=702.17 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 186486 1253 0 0 70188 29 0 0 25 0 5 0 635403722 349257728 56670 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2799.57
Current children cumulated vsize (KiB) 341072

[startup+762.314 s]
/proc/loadavg: 8.11 8.13 7.34 10/373 16276
/proc/meminfo: memFree=28741300/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=382232 CPUtime=3038.9 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 990335 1253 0 0 303736 154 0 0 23 0 5 0 635403708 391405568 66979 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 95558 66979 178 55 0 93444 0
[pid=16243/tid=16250] ppid=16241 vsize=382232 CPUtime=762.15 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 291126 1253 0 0 76171 44 0 0 25 0 5 0 635403722 391405568 66979 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=382232 CPUtime=752.37 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 237012 1253 0 0 75202 35 0 0 25 0 5 0 635403722 391405568 66979 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=382232 CPUtime=762.03 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 257668 1253 0 0 76162 41 0 0 25 0 5 0 635403722 391405568 66979 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=382232 CPUtime=762.18 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 195888 1253 0 0 76187 31 0 0 25 0 5 0 635403722 391405568 66979 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3038.9
Current children cumulated vsize (KiB) 382232

[startup+822.3 s]
/proc/loadavg: 8.08 8.12 7.38 9/373 16278
/proc/meminfo: memFree=28644896/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=383744 CPUtime=3278.13 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 1030336 1253 0 0 327653 160 0 0 23 0 5 0 635403708 392953856 67357 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 95936 67357 178 55 0 93822 0
[pid=16243/tid=16250] ppid=16241 vsize=383744 CPUtime=822.14 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 304403 1253 0 0 82168 46 0 0 25 0 5 0 635403722 392953856 67357 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=383744 CPUtime=811.66 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 246988 1253 0 0 81129 37 0 0 25 0 5 0 635403722 392953856 67357 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=383744 CPUtime=822 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 272058 1253 0 0 82157 43 0 0 25 0 5 0 635403722 392953856 67357 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=383744 CPUtime=822.17 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 198246 1253 0 0 82185 32 0 0 25 0 5 0 635403722 392953856 67357 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364542 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3278.13
Current children cumulated vsize (KiB) 383744

[startup+882.301 s]
/proc/loadavg: 8.07 8.11 7.42 10/373 16280
/proc/meminfo: memFree=28649440/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=406152 CPUtime=3517.42 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 1070020 1253 0 0 351577 165 0 0 23 0 5 0 635403708 415899648 72959 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 101538 72959 178 55 0 99424 0
[pid=16243/tid=16250] ppid=16241 vsize=406152 CPUtime=882.14 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 313453 1253 0 0 88167 47 0 0 25 0 5 0 635403722 415899648 72959 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=406152 CPUtime=870.94 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 261231 1253 0 0 87056 38 0 0 25 0 5 0 635403722 415899648 72959 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=406152 CPUtime=882 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 284815 1253 0 0 88155 45 0 0 25 0 5 0 635403722 415899648 72959 33554432000 4194304 4417452 140735341181680 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=406152 CPUtime=882.17 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 201880 1253 0 0 88185 32 0 0 25 0 5 0 635403722 415899648 72959 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3517.42
Current children cumulated vsize (KiB) 406152

[startup+942.301 s]
/proc/loadavg: 8.11 8.12 7.47 9/373 16281
/proc/meminfo: memFree=28442048/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=376340 CPUtime=3756.71 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 1128682 1253 0 0 375499 172 0 0 23 0 5 0 635403708 385372160 65487 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 94085 65487 178 55 0 91971 0
[pid=16243/tid=16250] ppid=16241 vsize=376340 CPUtime=942.13 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 324802 1253 0 0 94165 48 0 0 25 0 5 0 635403722 385372160 65487 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=376340 CPUtime=930.24 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 276402 1253 0 0 92984 40 0 0 25 0 5 0 635403722 385372160 65487 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=376340 CPUtime=941.99 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 294180 1253 0 0 94153 46 0 0 25 0 5 0 635403722 385372160 65487 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=376340 CPUtime=942.17 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 224657 1253 0 0 94183 34 0 0 25 0 5 0 635403722 385372160 65487 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3756.71
Current children cumulated vsize (KiB) 376340

[startup+1002.3 s]
/proc/loadavg: 8.04 8.09 7.50 9/373 16283
/proc/meminfo: memFree=28376044/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=389364 CPUtime=3995.98 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 1164097 1253 0 0 399421 177 0 0 23 0 5 0 635403708 398708736 68766 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 97341 68766 178 55 0 95227 0
[pid=16243/tid=16250] ppid=16241 vsize=389364 CPUtime=1002.14 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 335394 1253 0 0 100164 50 0 0 25 0 5 0 635403722 398708736 68766 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=389364 CPUtime=989.53 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 290720 1253 0 0 98911 42 0 0 25 0 5 0 635403722 398708736 68766 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=389364 CPUtime=1001.97 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 303699 1253 0 0 100150 47 0 0 25 0 5 0 635403722 398708736 68766 33554432000 4194304 4417452 140735341181680 18446744073709551615 4366806 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=389364 CPUtime=1002.17 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 225643 1253 0 0 100182 35 0 0 25 0 5 0 635403722 398708736 68766 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3995.98
Current children cumulated vsize (KiB) 389364

[startup+1062.3 s]
/proc/loadavg: 8.09 8.10 7.54 10/373 16284
/proc/meminfo: memFree=28286832/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=366212 CPUtime=4235.27 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 1205420 1253 0 0 423344 183 0 0 23 0 5 0 635403708 375001088 62955 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 91553 62955 178 55 0 89439 0
[pid=16243/tid=16250] ppid=16241 vsize=366212 CPUtime=1062.14 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 350191 1253 0 0 106162 52 0 0 25 0 5 0 635403722 375001088 62955 33554432000 4194304 4417452 140735341181680 18446744073709551615 4321590 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=366212 CPUtime=1048.83 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 302365 1253 0 0 104839 44 0 0 25 0 5 0 635403722 375001088 62955 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=366212 CPUtime=1061.96 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 316623 1253 0 0 106147 49 0 0 25 0 5 0 635403722 375001088 62955 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=366212 CPUtime=1062.17 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 227600 1253 0 0 106182 35 0 0 25 0 5 0 635403722 375001088 62955 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4235.27
Current children cumulated vsize (KiB) 366212

[startup+1122.3 s]
/proc/loadavg: 8.03 8.08 7.56 9/373 16286
/proc/meminfo: memFree=28599760/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=416500 CPUtime=4474.56 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 1241930 1253 0 0 447268 188 0 0 23 0 5 0 635403708 426496000 75527 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 104125 75527 178 55 0 102011 0
[pid=16243/tid=16250] ppid=16241 vsize=416500 CPUtime=1122.14 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 363197 1253 0 0 112160 54 0 0 25 0 5 0 635403722 426496000 75527 33554432000 4194304 4417452 140735341181680 18446744073709551615 4353830 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=416500 CPUtime=1108.12 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 307587 1253 0 0 110767 45 0 0 25 0 5 0 635403722 426496000 75527 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=416500 CPUtime=1121.95 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 326697 1253 0 0 112145 50 0 0 25 0 5 0 635403722 426496000 75527 33554432000 4194304 4417452 140735341181680 18446744073709551615 4354497 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=416500 CPUtime=1122.18 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 235808 1253 0 0 112182 36 0 0 25 0 5 0 635403722 426496000 75527 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4474.56
Current children cumulated vsize (KiB) 416500

[startup+1182.3 s]
/proc/loadavg: 8.08 8.09 7.59 10/373 16287
/proc/meminfo: memFree=28336196/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=464276 CPUtime=4713.84 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 1281814 1253 0 0 471190 194 0 0 23 0 5 0 635403708 475418624 87471 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 116069 87471 178 55 0 113955 0
[pid=16243/tid=16250] ppid=16241 vsize=464276 CPUtime=1182.13 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 373819 1253 0 0 118159 54 0 0 25 0 5 0 635403722 475418624 87471 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=464276 CPUtime=1167.42 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 314837 1253 0 0 116696 46 0 0 25 0 5 0 635403722 475418624 87471 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=464276 CPUtime=1181.94 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 336505 1253 0 0 118142 52 0 0 25 0 5 0 635403722 475418624 87471 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=464276 CPUtime=1182.18 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 248012 1253 0 0 118180 38 0 0 25 0 5 0 635403722 475418624 87471 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4713.84
Current children cumulated vsize (KiB) 464276



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1204 s]
/proc/loadavg: 8.17 8.11 7.61 5/367 16289
/proc/meminfo: memFree=29848284/32951124 swapFree=51873460/67111528
[pid=16243] ppid=16241 vsize=466752 CPUtime=4800.42 cores=0,2,4,6
/proc/16243/stat : 16243 (plingeling) S 16241 16243 8955 0 -1 4202496 1291234 1253 0 0 479847 195 0 0 23 0 5 0 635403708 477954048 88090 33554432000 4194304 4417452 140735341181680 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/16243/statm: 116688 88090 178 55 0 114574 0
[pid=16243/tid=16250] ppid=16241 vsize=466752 CPUtime=1203.85 cores=0,2,4,6
/proc/16243/task/16250/stat : 16250 (plingeling) R 16241 16243 8955 0 -1 4202560 376703 1253 0 0 120330 55 0 0 25 0 5 0 635403722 477954048 88090 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16243/tid=16251] ppid=16241 vsize=466752 CPUtime=1188.86 cores=0,2,4,6
/proc/16243/task/16251/stat : 16251 (plingeling) R 16241 16243 8955 0 -1 4202560 318777 1253 0 0 118840 46 0 0 25 0 5 0 635403722 477954048 88090 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16243/tid=16252] ppid=16241 vsize=466752 CPUtime=1203.65 cores=0,2,4,6
/proc/16243/task/16252/stat : 16252 (plingeling) R 16241 16243 8955 0 -1 4202560 339101 1253 0 0 120313 52 0 0 25 0 5 0 635403722 477954048 88090 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16243/tid=16253] ppid=16241 vsize=466752 CPUtime=1203.89 cores=0,2,4,6
/proc/16243/task/16253/stat : 16253 (plingeling) R 16241 16243 8955 0 -1 4202560 248012 1253 0 0 120351 38 0 0 25 0 5 0 635403722 477954048 88090 33554432000 4194304 4417452 140735341181680 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.42
Current children cumulated vsize (KiB) 466752

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1204.04
CPU time (s): 4800.47
CPU user time (s): 4798.5
CPU system time (s): 1.9697
CPU usage (%): 398.698
Max. virtual memory (cumulated for all children) (KiB): 475008

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.5
system time used= 1.9697
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1292487
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 73
involuntary context switches= 25484

runsolver used 3.9624 second user time and 10.2834 second system time

The end

Launcher Data 

Begin job on node146 at 2011-05-15 05:24:34
IDJOB=3371040
IDBENCH=83965
IDSOLVER=1848
FILE ID=node146/3371040-1305429874
RUNJOBID= node146-1305428668-16108
PBS_JOBID= 13324589
Free space on /tmp= 73332 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT03/handmade/anton/UNSAT/SGI_30_60_28_40_7-log.shuffled-as.sat03-127.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371040-1305429874/watcher-3371040-1305429874 -o /tmp/evaluation-result-3371040-1305429874/solver-3371040-1305429874 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371040-1305429874.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f1f35d90237c4ca07d41a6a9b08e6279
RANDOM SEED=1833966236

node146.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30197756 kB
Buffers:        100436 kB
Cached:         172044 kB
SwapCached:      56148 kB
Active:         777692 kB
Inactive:      1745716 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30197756 kB
SwapTotal:    67111528 kB
SwapFree:     51873460 kB
Dirty:            2740 kB
Writeback:           0 kB
AnonPages:     2201168 kB
Mapped:          17320 kB
Slab:           104024 kB
PageTables:      59284 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 81825120 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73332 MiB
End job on node146 at 2011-05-15 05:44:40