Trace number 3370895

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.51 1203.29

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m37-q505.75-n35-p30-s59841049.cnf
MD5SUMc86ce640ddb0138bf41ccc37bc6fcf28
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.84587
Satisfiable
(Un)Satisfiability was proved
Number of variables1295
Number of clauses148400
Sum of the clauses size298025
Maximum clause length37
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2148365
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 535

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3370895-1305433079/watcher-3370895-1305433079 -o /tmp/evaluation-result-3370895-1305433079/solver-3370895-1305433079 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3370895-1305433079.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.78 5.45 6.59 3/304 29877
/proc/meminfo: memFree=17993108/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=10056 CPUtime=0 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) R 29875 29877 20898 0 -1 4202496 606 1261 0 0 0 0 0 0 22 0 1 0 654907555 10297344 536 33554432000 4194304 4417452 140734120844016 18446744073709551615 274473967360 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/29877/statm: 2514 536 147 55 0 400 0

[startup+0.0802971 s]
/proc/loadavg: 3.78 5.45 6.59 3/304 29877
/proc/meminfo: memFree=17993108/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=25864 CPUtime=0.07 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) R 29875 29877 20898 0 -1 4202496 4641 1261 0 0 7 0 0 0 22 0 1 0 654907555 26484736 4489 33554432000 4194304 4417452 140734120844016 18446744073709551615 4231283 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/29877/statm: 6466 4489 148 55 0 4352 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 25864

[startup+0.109527 s]
/proc/loadavg: 3.78 5.45 6.59 3/304 29877
/proc/meminfo: memFree=17993108/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=198208 CPUtime=0.13 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 4732 1261 0 0 13 0 0 0 22 0 5 0 654907555 202964992 4576 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 49552 4576 171 55 0 47438 0
Current children cumulated CPU time (s) 0.13
Current children cumulated vsize (KiB) 198208

[startup+0.300211 s]
/proc/loadavg: 3.78 5.45 6.59 3/304 29877
/proc/meminfo: memFree=17993108/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=136452 CPUtime=0.89 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 9424 1261 0 0 88 1 0 0 22 0 5 0 654907555 139726848 5564 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 34113 5564 176 55 0 31999 0
Current children cumulated CPU time (s) 0.89
Current children cumulated vsize (KiB) 136452

[startup+0.70014 s]
/proc/loadavg: 3.78 5.45 6.59 3/304 29877
/proc/meminfo: memFree=17993108/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=144972 CPUtime=2.49 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 12313 1261 0 0 248 1 0 0 22 0 5 0 654907555 148451328 7665 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 36243 7665 179 55 0 34129 0
Current children cumulated CPU time (s) 2.49
Current children cumulated vsize (KiB) 144972

[startup+1.501 s]
/proc/loadavg: 3.78 5.45 6.59 9/316 29902
/proc/meminfo: memFree=17814644/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=148084 CPUtime=5.69 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 18639 1261 0 0 566 3 0 0 22 0 5 0 654907555 151638016 8443 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 37021 8443 179 55 0 34907 0
[pid=29877/tid=29884] ppid=29875 vsize=148084 CPUtime=1.39 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 1799 1261 0 0 139 0 0 0 25 0 5 0 654907565 151638016 8443 33554432000 4194304 4417452 140734120844016 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=148084 CPUtime=1.39 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 2065 1261 0 0 139 0 0 0 25 0 5 0 654907565 151638016 8443 33554432000 4194304 4417452 140734120844016 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=148084 CPUtime=1.4 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 1328 1261 0 0 140 0 0 0 25 0 5 0 654907565 151638016 8443 33554432000 4194304 4417452 140734120844016 18446744073709551615 4258401 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=148084 CPUtime=1.38 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 8794 1261 0 0 137 1 0 0 25 0 5 0 654907565 151638016 8443 33554432000 4194304 4417452 140734120844016 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 5.69
Current children cumulated vsize (KiB) 148084

[startup+3.10072 s]
/proc/loadavg: 4.12 5.50 6.59 9/316 29902
/proc/meminfo: memFree=17774644/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=166552 CPUtime=12.08 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 27499 1261 0 0 1204 4 0 0 22 0 5 0 654907555 170549248 13060 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 41638 13060 179 55 0 39524 0
[pid=29877/tid=29884] ppid=29875 vsize=166552 CPUtime=2.99 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 1799 1261 0 0 299 0 0 0 25 0 5 0 654907565 170549248 13060 33554432000 4194304 4417452 140734120844016 18446744073709551615 4258410 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=166552 CPUtime=2.99 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 2065 1261 0 0 299 0 0 0 25 0 5 0 654907565 170549248 13060 33554432000 4194304 4417452 140734120844016 18446744073709551615 4249143 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=166552 CPUtime=3 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 1570 1261 0 0 300 0 0 0 25 0 5 0 654907565 170549248 13060 33554432000 4194304 4417452 140734120844016 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=166552 CPUtime=2.97 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 17412 1261 0 0 295 2 0 0 25 0 5 0 654907565 170549248 13060 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 12.08
Current children cumulated vsize (KiB) 166552

[startup+6.30115 s]
/proc/loadavg: 4.12 5.50 6.59 9/316 29903
/proc/meminfo: memFree=17698020/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=170448 CPUtime=24.84 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 40868 1261 0 0 2478 6 0 0 22 0 5 0 654907555 174538752 14034 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 42612 14034 179 55 0 40498 0
[pid=29877/tid=29884] ppid=29875 vsize=170448 CPUtime=6.19 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 1799 1261 0 0 619 0 0 0 25 0 5 0 654907565 174538752 14034 33554432000 4194304 4417452 140734120844016 18446744073709551615 4258491 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=170448 CPUtime=6.19 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 2731 1261 0 0 619 0 0 0 25 0 5 0 654907565 174538752 14034 33554432000 4194304 4417452 140734120844016 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=170448 CPUtime=6.19 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 2104 1261 0 0 619 0 0 0 25 0 5 0 654907565 174538752 14034 33554432000 4194304 4417452 140734120844016 18446744073709551615 4251060 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=170448 CPUtime=6.14 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 29581 1261 0 0 610 4 0 0 25 0 5 0 654907565 174538752 14034 33554432000 4194304 4417452 140734120844016 18446744073709551615 4251172 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.84
Current children cumulated vsize (KiB) 170448

[startup+12.701 s]
/proc/loadavg: 4.72 5.58 6.61 9/316 29903
/proc/meminfo: memFree=17569368/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=199504 CPUtime=50.38 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 58372 1261 0 0 5029 9 0 0 22 0 5 0 654907555 204292096 21330 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 49876 21330 179 55 0 47762 0
[pid=29877/tid=29884] ppid=29875 vsize=199504 CPUtime=12.59 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 1799 1261 0 0 1259 0 0 0 25 0 5 0 654907565 204292096 21330 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=199504 CPUtime=12.59 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 2731 1261 0 0 1259 0 0 0 25 0 5 0 654907565 204292096 21330 33554432000 4194304 4417452 140734120844016 18446744073709551615 4356796 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=199504 CPUtime=12.59 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 4873 1261 0 0 1259 0 0 0 25 0 5 0 654907565 204292096 21330 33554432000 4194304 4417452 140734120844016 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=199504 CPUtime=12.48 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 44316 1261 0 0 1241 7 0 0 25 0 5 0 654907565 204292096 21330 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.38
Current children cumulated vsize (KiB) 199504

[startup+25.5008 s]
/proc/loadavg: 5.22 5.66 6.62 9/316 29903
/proc/meminfo: memFree=17526412/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=263688 CPUtime=101.46 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 98822 1261 0 0 10131 15 0 0 22 0 5 0 654907555 270016512 37357 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 65922 37357 179 55 0 63808 0
[pid=29877/tid=29884] ppid=29875 vsize=263688 CPUtime=25.4 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 6016 1261 0 0 2539 1 0 0 25 0 5 0 654907565 270016512 37357 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=263688 CPUtime=25.39 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 8773 1261 0 0 2538 1 0 0 25 0 5 0 654907565 270016512 37357 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=263688 CPUtime=25.4 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 10353 1261 0 0 2539 1 0 0 25 0 5 0 654907565 270016512 37357 33554432000 4194304 4417452 140734120844016 18446744073709551615 4249943 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=263688 CPUtime=25.16 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 69027 1261 0 0 2505 11 0 0 25 0 5 0 654907565 270016512 37357 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.46
Current children cumulated vsize (KiB) 263688

[startup+51.1003 s]
/proc/loadavg: 6.17 5.84 6.66 9/316 29904
/proc/meminfo: memFree=17271000/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=388152 CPUtime=203.6 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 186788 1261 0 0 20332 28 0 0 22 0 5 0 654907555 397467648 68474 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 97038 68474 180 55 0 94924 0
[pid=29877/tid=29884] ppid=29875 vsize=388152 CPUtime=50.99 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 25783 1261 0 0 5096 3 0 0 25 0 5 0 654907565 397467648 68474 33554432000 4194304 4417452 140734120844016 18446744073709551615 4305912 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=388152 CPUtime=50.98 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 24309 1261 0 0 5095 3 0 0 25 0 5 0 654907565 397467648 68474 33554432000 4194304 4417452 140734120844016 18446744073709551615 4250524 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=388152 CPUtime=50.99 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 26255 1261 0 0 5097 2 0 0 25 0 5 0 654907565 397467648 68474 33554432000 4194304 4417452 140734120844016 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=388152 CPUtime=50.5 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 105788 1261 0 0 5034 16 0 0 25 0 5 0 654907565 397467648 68474 33554432000 4194304 4417452 140734120844016 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 203.6
Current children cumulated vsize (KiB) 388152

[startup+102.311 s]
/proc/loadavg: 7.36 6.23 6.74 9/316 29905
/proc/meminfo: memFree=16960732/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=526552 CPUtime=407.93 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 343480 1261 0 0 40742 51 0 0 22 0 5 0 654907555 539189248 103074 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 131638 103074 180 55 0 129524 0
[pid=29877/tid=29884] ppid=29875 vsize=526552 CPUtime=102.2 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 48949 1261 0 0 10214 6 0 0 25 0 5 0 654907565 539189248 103074 33554432000 4194304 4417452 140734120844016 18446744073709551615 4263109 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=526552 CPUtime=102.19 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 45845 1261 0 0 10213 6 0 0 25 0 5 0 654907565 539189248 103074 33554432000 4194304 4417452 140734120844016 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=526552 CPUtime=102.21 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 83196 1261 0 0 10211 10 0 0 25 0 5 0 654907565 539189248 103074 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=526552 CPUtime=101.19 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 160837 1261 0 0 10093 26 0 0 25 0 5 0 654907565 539189248 103074 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 407.93
Current children cumulated vsize (KiB) 526552

[startup+162.301 s]
/proc/loadavg: 7.85 6.58 6.83 9/316 29907
/proc/meminfo: memFree=16768736/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=668292 CPUtime=647.28 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 506252 1261 0 0 64653 75 0 0 22 0 5 0 654907555 684331008 138509 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 167073 138509 180 55 0 164959 0
[pid=29877/tid=29884] ppid=29875 vsize=668292 CPUtime=162.19 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 72487 1261 0 0 16209 10 0 0 25 0 5 0 654907565 684331008 138509 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=668292 CPUtime=162.18 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 67895 1261 0 0 16209 9 0 0 25 0 5 0 654907565 684331008 138509 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=668292 CPUtime=162.19 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 145239 1261 0 0 16202 17 0 0 25 0 5 0 654907565 684331008 138509 33554432000 4194304 4417452 140734120844016 18446744073709551615 4356962 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=668292 CPUtime=160.58 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 215978 1261 0 0 16022 36 0 0 25 0 5 0 654907565 684331008 138509 33554432000 4194304 4417452 140734120844016 18446744073709551615 4366134 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 647.28
Current children cumulated vsize (KiB) 668292


################
# More data... #
################

[pid=29877/tid=29885] ppid=29875 vsize=1386304 CPUtime=702.17 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 379384 1261 0 0 70159 58 0 0 25 0 5 0 654907565 1419575296 318012 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=1386304 CPUtime=702.21 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 622856 1261 0 0 70132 89 0 0 25 0 5 0 654907565 1419575296 318012 33554432000 4194304 4417452 140734120844016 18446744073709551615 4354517 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=1386304 CPUtime=695.12 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 429715 1261 0 0 69433 79 0 0 25 0 5 0 654907565 1419575296 318012 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2801.82
Current children cumulated vsize (KiB) 1386304

[startup+762.312 s]
/proc/loadavg: 8.18 7.95 7.45 9/316 29923
/proc/meminfo: memFree=14455924/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=1543304 CPUtime=3041.26 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 1908715 1261 0 0 303827 299 0 0 22 0 5 0 654907555 1580343296 357262 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 385826 357262 180 55 0 383712 0
[pid=29877/tid=29884] ppid=29875 vsize=1543304 CPUtime=762.23 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 400419 1261 0 0 76162 61 0 0 25 0 5 0 654907565 1580343296 357262 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=1543304 CPUtime=762.17 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 395768 1261 0 0 76158 59 0 0 25 0 5 0 654907565 1580343296 357262 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=1543304 CPUtime=762.22 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 663676 1261 0 0 76128 94 0 0 25 0 5 0 654907565 1580343296 357262 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=1543304 CPUtime=754.51 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 444199 1261 0 0 75369 82 0 0 25 0 5 0 654907565 1580343296 357262 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364711 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3041.26
Current children cumulated vsize (KiB) 1543304

[startup+822.301 s]
/proc/loadavg: 8.24 8.00 7.49 9/316 29924
/proc/meminfo: memFree=14607164/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=1565864 CPUtime=3280.62 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 2008326 1261 0 0 327750 312 0 0 22 0 5 0 654907555 1603444736 362902 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 391466 362902 180 55 0 389352 0
[pid=29877/tid=29884] ppid=29875 vsize=1565864 CPUtime=822.21 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 424995 1261 0 0 82158 63 0 0 25 0 5 0 654907565 1603444736 362902 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=1565864 CPUtime=822.17 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 403960 1261 0 0 82157 60 0 0 25 0 5 0 654907565 1603444736 362902 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=1565864 CPUtime=822.21 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 727682 1261 0 0 82117 104 0 0 25 0 5 0 654907565 1603444736 362902 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=1565864 CPUtime=813.9 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 447036 1261 0 0 81308 82 0 0 25 0 5 0 654907565 1603444736 362902 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3280.62
Current children cumulated vsize (KiB) 1565864

[startup+882.301 s]
/proc/loadavg: 8.13 8.01 7.53 9/316 29926
/proc/meminfo: memFree=14410828/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=1682320 CPUtime=3520.02 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 2154761 1261 0 0 351671 331 0 0 22 0 5 0 654907555 1722695680 392016 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 420580 392016 180 55 0 418466 0
[pid=29877/tid=29884] ppid=29875 vsize=1682320 CPUtime=882.21 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 458571 1261 0 0 88154 67 0 0 25 0 5 0 654907565 1722695680 392016 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=1682320 CPUtime=882.17 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 467667 1261 0 0 88148 69 0 0 25 0 5 0 654907565 1722695680 392016 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=1682320 CPUtime=882.21 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 776834 1261 0 0 88112 109 0 0 25 0 5 0 654907565 1722695680 392016 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=1682320 CPUtime=873.29 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 447036 1261 0 0 87247 82 0 0 25 0 5 0 654907565 1722695680 392016 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3520.02
Current children cumulated vsize (KiB) 1682320

[startup+942.3 s]
/proc/loadavg: 8.15 8.04 7.57 9/316 29927
/proc/meminfo: memFree=14098564/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=1872632 CPUtime=3759.41 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 2292450 1261 0 0 375590 351 0 0 22 0 5 0 654907555 1917575168 439593 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 468158 439593 180 55 0 466044 0
[pid=29877/tid=29884] ppid=29875 vsize=1872632 CPUtime=942.21 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 491339 1261 0 0 94150 71 0 0 25 0 5 0 654907565 1917575168 439593 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=1872632 CPUtime=942.17 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 508627 1261 0 0 94141 76 0 0 25 0 5 0 654907565 1917575168 439593 33554432000 4194304 4417452 140734120844016 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=1872632 CPUtime=942.22 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 822421 1261 0 0 94106 116 0 0 25 0 5 0 654907565 1917575168 439593 33554432000 4194304 4417452 140734120844016 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=1872632 CPUtime=932.69 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 465410 1261 0 0 93183 86 0 0 25 0 5 0 654907565 1917575168 439593 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3759.41
Current children cumulated vsize (KiB) 1872632

[startup+1002.3 s]
/proc/loadavg: 8.05 8.03 7.59 9/316 29929
/proc/meminfo: memFree=14127784/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=1708568 CPUtime=3998.79 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 2331338 1261 0 0 399522 357 0 0 22 0 5 0 654907555 1749573632 398577 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 427142 398577 180 55 0 425028 0
[pid=29877/tid=29884] ppid=29875 vsize=1708568 CPUtime=1002.22 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 499531 1261 0 0 100149 73 0 0 25 0 5 0 654907565 1749573632 398577 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=1708568 CPUtime=1002.16 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 522899 1261 0 0 100138 78 0 0 25 0 5 0 654907565 1749573632 398577 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=1708568 CPUtime=1002.21 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 830613 1261 0 0 100105 116 0 0 25 0 5 0 654907565 1749573632 398577 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=1708568 CPUtime=992.08 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 473642 1261 0 0 99120 88 0 0 25 0 5 0 654907565 1749573632 398577 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3998.79
Current children cumulated vsize (KiB) 1708568

[startup+1062.3 s]
/proc/loadavg: 8.10 8.05 7.62 9/316 29931
/proc/meminfo: memFree=13531916/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=1820720 CPUtime=4238.19 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 2408389 1261 0 0 423449 370 0 0 22 0 5 0 654907555 1864417280 426615 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 455180 426615 180 55 0 453066 0
[pid=29877/tid=29884] ppid=29875 vsize=1820720 CPUtime=1062.22 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 570491 1261 0 0 106138 84 0 0 25 0 5 0 654907565 1864417280 426615 33554432000 4194304 4417452 140734120844016 18446744073709551615 4310641 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=1820720 CPUtime=1062.17 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 523875 1261 0 0 106138 79 0 0 25 0 5 0 654907565 1864417280 426615 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=1820720 CPUtime=1062.21 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 830613 1261 0 0 106105 116 0 0 25 0 5 0 654907565 1864417280 426615 33554432000 4194304 4417452 140734120844016 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=1820720 CPUtime=1051.48 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 478757 1261 0 0 105059 89 0 0 25 0 5 0 654907565 1864417280 426615 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4238.19
Current children cumulated vsize (KiB) 1820720

[startup+1122.3 s]
/proc/loadavg: 8.17 8.07 7.66 9/316 29932
/proc/meminfo: memFree=14448104/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=1533136 CPUtime=4477.59 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 2589255 1261 0 0 447358 401 0 0 22 0 5 0 654907555 1569931264 354719 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 383284 354719 180 55 0 381170 0
[pid=29877/tid=29884] ppid=29875 vsize=1533136 CPUtime=1122.22 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 622866 1261 0 0 112129 93 0 0 25 0 5 0 654907565 1569931264 354719 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=1533136 CPUtime=1122.16 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 586075 1261 0 0 112128 88 0 0 25 0 5 0 654907565 1569931264 354719 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=1533136 CPUtime=1122.22 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 835467 1261 0 0 112104 118 0 0 25 0 5 0 654907565 1569931264 354719 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=1533136 CPUtime=1110.86 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 540194 1261 0 0 110986 100 0 0 25 0 5 0 654907565 1569931264 354719 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4477.59
Current children cumulated vsize (KiB) 1533136

[startup+1182.3 s]
/proc/loadavg: 8.18 8.09 7.68 9/316 29934
/proc/meminfo: memFree=14287100/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=1668652 CPUtime=4716.99 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 2705054 1261 0 0 471281 418 0 0 22 0 5 0 654907555 1708699648 388598 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 417163 388598 180 55 0 415049 0
[pid=29877/tid=29884] ppid=29875 vsize=1668652 CPUtime=1182.22 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 647442 1261 0 0 118126 96 0 0 25 0 5 0 654907565 1708699648 388598 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=1668652 CPUtime=1182.17 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 643419 1261 0 0 118120 97 0 0 25 0 5 0 654907565 1708699648 388598 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=1668652 CPUtime=1182.22 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 852356 1261 0 0 118101 121 0 0 25 0 5 0 654907565 1708699648 388598 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=1668652 CPUtime=1170.26 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 557184 1261 0 0 116923 103 0 0 25 0 5 0 654907565 1708699648 388598 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4716.99
Current children cumulated vsize (KiB) 1668652



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.2 s]
/proc/loadavg: 8.29 8.11 7.70 9/316 29935
/proc/meminfo: memFree=14083356/32950928 swapFree=67111348/67111528
[pid=29877] ppid=29875 vsize=1672428 CPUtime=4800.39 cores=0,2,4,6
/proc/29877/stat : 29877 (plingeling) S 29875 29877 20898 0 -1 4202496 2738766 1261 0 0 479616 423 0 0 22 0 5 0 654907555 1712566272 389542 33554432000 4194304 4417452 140734120844016 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29877/statm: 418107 389542 180 55 0 415993 0
[pid=29877/tid=29884] ppid=29875 vsize=1672428 CPUtime=1203.13 cores=0,2,4,6
/proc/29877/task/29884/stat : 29884 (plingeling) R 29875 29877 20898 0 -1 4202560 647442 1261 0 0 120217 96 0 0 25 0 5 0 654907565 1712566272 389542 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29877/tid=29885] ppid=29875 vsize=1672428 CPUtime=1203.08 cores=0,2,4,6
/proc/29877/task/29885/stat : 29885 (plingeling) R 29875 29877 20898 0 -1 4202560 667995 1261 0 0 120208 100 0 0 25 0 5 0 654907565 1712566272 389542 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29877/tid=29886] ppid=29875 vsize=1672428 CPUtime=1203.13 cores=0,2,4,6
/proc/29877/task/29886/stat : 29886 (plingeling) R 29875 29877 20898 0 -1 4202560 852356 1261 0 0 120192 121 0 0 25 0 5 0 654907565 1712566272 389542 33554432000 4194304 4417452 140734120844016 18446744073709551615 4310644 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29877/tid=29887] ppid=29875 vsize=1672428 CPUtime=1190.94 cores=0,2,4,6
/proc/29877/task/29887/stat : 29887 (plingeling) R 29875 29877 20898 0 -1 4202560 566320 1261 0 0 118990 104 0 0 25 0 5 0 654907565 1712566272 389542 33554432000 4194304 4417452 140734120844016 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.39
Current children cumulated vsize (KiB) 1672428

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.29
CPU time (s): 4800.51
CPU user time (s): 4796.2
CPU system time (s): 4.30835
CPU usage (%): 398.949
Max. virtual memory (cumulated for all children) (KiB): 1945396

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.2
system time used= 4.30835
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2740027
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 89
involuntary context switches= 25644

runsolver used 3.55046 second user time and 8.45371 second system time

The end

Launcher Data 

Begin job on node125 at 2011-05-15 06:17:59
IDJOB=3370895
IDBENCH=88218
IDSOLVER=1848
FILE ID=node125/3370895-1305433079
RUNJOBID= node125-1305433079-29863
PBS_JOBID= 13324587
Free space on /tmp= 73228 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m37-q505.75-n35-p30-s59841049.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3370895-1305433079/watcher-3370895-1305433079 -o /tmp/evaluation-result-3370895-1305433079/solver-3370895-1305433079 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3370895-1305433079.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= c86ce640ddb0138bf41ccc37bc6fcf28
RANDOM SEED=630014773

node125.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5266.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      17993156 kB
Buffers:        267436 kB
Cached:        4258312 kB
SwapCached:          0 kB
Active:       11175424 kB
Inactive:      3409020 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      17993156 kB
SwapTotal:    67111528 kB
SwapFree:     67111348 kB
Dirty:            3504 kB
Writeback:           8 kB
AnonPages:    10058300 kB
Mapped:          28972 kB
Slab:           274268 kB
PageTables:      35644 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 50990728 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73216 MiB
End job on node125 at 2011-05-15 06:38:04