Trace number 3372257

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.5 5008.37

General information on the benchmark

NameAPPLICATIONS/crypto/
md5gen/gus-md5-12.cnf
MD5SUM0621e557bffdc18ca509447665adecda
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables69553
Number of clauses226752
Sum of the clauses size587624
Maximum clause length3
Minimum clause length1
Number of clauses of size 1129
Number of clauses of size 292374
Number of clauses of size 3134249
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372257-1305437039/watcher-3372257-1305437039 -o /tmp/evaluation-result-3372257-1305437039/solver-3372257-1305437039 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3372257-1305437039.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.52 3.87 3.62 1/359 17275
/proc/meminfo: memFree=18236696/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=27204 CPUtime=0 cores=0-7
/proc/17275/stat : 17275 (plingeling) R 17273 17275 17249 0 -1 4202496 4879 1252 0 0 0 0 0 0 22 0 1 0 733934567 27856896 4809 33554432000 4194304 4417452 140734291067360 18446744073709551615 224284619747 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/17275/statm: 6801 4809 148 55 0 4687 0

[startup+0.066495 s]
/proc/loadavg: 3.52 3.87 3.62 1/359 17275
/proc/meminfo: memFree=18236696/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=37952 CPUtime=0.05 cores=0-7
/proc/17275/stat : 17275 (plingeling) R 17273 17275 17249 0 -1 4202496 7616 1252 0 0 5 0 0 0 22 0 1 0 733934567 38862848 7481 33554432000 4194304 4417452 140734291067360 18446744073709551615 4264920 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/17275/statm: 9488 7481 149 55 0 7374 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 37952

[startup+0.100489 s]
/proc/loadavg: 3.52 3.87 3.62 1/359 17275
/proc/meminfo: memFree=18236696/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=49380 CPUtime=0.09 cores=0-7
/proc/17275/stat : 17275 (plingeling) R 17273 17275 17249 0 -1 4202496 10505 1252 0 0 8 1 0 0 22 0 1 0 733934567 50565120 10370 33554432000 4194304 4417452 140734291067360 18446744073709551615 4231277 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/17275/statm: 12345 10370 149 55 0 10231 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 49380

[startup+0.300465 s]
/proc/loadavg: 3.52 3.87 3.62 1/359 17275
/proc/meminfo: memFree=18236696/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=72308 CPUtime=0.28 cores=0-7
/proc/17275/stat : 17275 (plingeling) R 17273 17275 17249 0 -1 4202496 16238 1252 0 0 27 1 0 0 23 0 1 0 733934567 74043392 16102 33554432000 4194304 4417452 140734291067360 18446744073709551615 224284925696 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/17275/statm: 18077 16102 149 55 0 15963 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 72308

[startup+0.700384 s]
/proc/loadavg: 3.52 3.87 3.62 1/359 17275
/proc/meminfo: memFree=18236696/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=272880 CPUtime=0.83 cores=0-7
/proc/17275/stat : 17275 (plingeling) S 17273 17275 17249 0 -1 4202496 38117 1252 0 0 79 4 0 0 18 0 9 0 733934567 279429120 29261 33554432000 4194304 4417452 140734291067360 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/17275/statm: 68220 29288 159 55 0 66106 0
Current children cumulated CPU time (s) 0.83
Current children cumulated vsize (KiB) 272880

[startup+1.50122 s]
/proc/loadavg: 3.52 3.87 3.62 9/368 17289
/proc/meminfo: memFree=18086592/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=347872 CPUtime=7.15 cores=0-7
/proc/17275/stat : 17275 (plingeling) S 17273 17275 17249 0 -1 4202496 62360 1252 0 0 708 7 0 0 18 0 9 0 733934567 356220928 36514 33554432000 4194304 4417452 140734291067360 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/17275/statm: 86968 36514 174 55 0 84854 0
[pid=17275/tid=17282] ppid=17273 vsize=347872 CPUtime=0.82 cores=0-7
/proc/17275/task/17282/stat : 17282 (plingeling) R 17273 17275 17249 0 -1 4202560 1900 1252 0 0 82 0 0 0 25 0 9 0 733934633 356220928 36514 33554432000 4194304 4417452 140734291067360 18446744073709551615 4287111 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17275/tid=17283] ppid=17273 vsize=347872 CPUtime=0.77 cores=0-7
/proc/17275/task/17283/stat : 17283 (plingeling) R 17273 17275 17249 0 -1 4202560 4081 1252 0 0 77 0 0 0 25 0 9 0 733934633 356220928 36514 33554432000 4194304 4417452 140734291067360 18446744073709551615 4286255 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17275/tid=17284] ppid=17273 vsize=347872 CPUtime=0.83 cores=0-7
/proc/17275/task/17284/stat : 17284 (plingeling) R 17273 17275 17249 0 -1 4202560 6462 1252 0 0 82 1 0 0 18 0 9 0 733934633 356220928 36514 33554432000 4194304 4417452 140734291067360 18446744073709551615 4290744 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17275/tid=17285] ppid=17273 vsize=347872 CPUtime=0.82 cores=0-7
/proc/17275/task/17285/stat : 17285 (plingeling) R 17273 17275 17249 0 -1 4202560 1962 1252 0 0 82 0 0 0 21 0 9 0 733934633 356220928 36514 33554432000 4194304 4417452 140734291067360 18446744073709551615 4250972 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=17275/tid=17286] ppid=17273 vsize=347872 CPUtime=0.77 cores=0-7
/proc/17275/task/17286/stat : 17286 (plingeling) R 17273 17275 17249 0 -1 4202560 2842 1252 0 0 77 0 0 0 21 0 9 0 733934634 356220928 36514 33554432000 4194304 4417452 140734291067360 18446744073709551615 4301776 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17275/tid=17287] ppid=17273 vsize=347872 CPUtime=0.8 cores=0-7
/proc/17275/task/17287/stat : 17287 (plingeling) R 17273 17275 17249 0 -1 4202560 5060 1252 0 0 80 0 0 0 19 0 9 0 733934634 356220928 36514 33554432000 4194304 4417452 140734291067360 18446744073709551615 4301955 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17275/tid=17288] ppid=17273 vsize=347872 CPUtime=0.8 cores=0-7
/proc/17275/task/17288/stat : 17288 (plingeling) R 17273 17275 17249 0 -1 4202560 2286 1252 0 0 80 0 0 0 21 0 9 0 733934634 356220928 36514 33554432000 4194304 4417452 140734291067360 18446744073709551615 4302058 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17275/tid=17289] ppid=17273 vsize=347872 CPUtime=0.82 cores=0-7
/proc/17275/task/17289/stat : 17289 (plingeling) R 17273 17275 17249 0 -1 4202560 1828 1252 0 0 82 0 0 0 21 0 9 0 733934635 356220928 36514 33554432000 4194304 4417452 140734291067360 18446744073709551615 4234277 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 7.15
Current children cumulated vsize (KiB) 347872

[startup+3.10095 s]
/proc/loadavg: 3.88 3.94 3.65 9/368 17289
/proc/meminfo: memFree=18085368/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=360552 CPUtime=19.93 cores=0-7
/proc/17275/stat : 17275 (plingeling) S 17273 17275 17249 0 -1 4202496 79529 1252 0 0 1984 9 0 0 18 0 9 0 733934567 369205248 37795 33554432000 4194304 4417452 140734291067360 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/17275/statm: 90138 37795 178 55 0 88024 0
[pid=17275/tid=17282] ppid=17273 vsize=360552 CPUtime=2.42 cores=0-7
/proc/17275/task/17282/stat : 17282 (plingeling) R 17273 17275 17249 0 -1 4202560 3459 1252 0 0 242 0 0 0 25 0 9 0 733934633 369205248 37795 33554432000 4194304 4417452 140734291067360 18446744073709551615 4265600 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17275/tid=17283] ppid=17273 vsize=360552 CPUtime=2.35 cores=0-7
/proc/17275/task/17283/stat : 17283 (plingeling) R 17273 17275 17249 0 -1 4202560 5640 1252 0 0 235 0 0 0 25 0 9 0 733934633 369205248 37795 33554432000 4194304 4417452 140734291067360 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17275/tid=17284] ppid=17273 vsize=360552 CPUtime=2.43 cores=0-7
/proc/17275/task/17284/stat : 17284 (plingeling) R 17273 17275 17249 0 -1 4202560 12229 1252 0 0 241 2 0 0 22 0 9 0 733934633 369205248 37795 33554432000 4194304 4417452 140734291067360 18446744073709551615 4265940 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17275/tid=17285] ppid=17273 vsize=360552 CPUtime=2.42 cores=0-7
/proc/17275/task/17285/stat : 17285 (plingeling) R 17273 17275 17249 0 -1 4202560 1962 1252 0 0 242 0 0 0 25 0 9 0 733934633 369205248 37795 33554432000 4194304 4417452 140734291067360 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=17275/tid=17286] ppid=17273 vsize=360552 CPUtime=2.37 cores=0-7
/proc/17275/task/17286/stat : 17286 (plingeling) R 17273 17275 17249 0 -1 4202560 4398 1252 0 0 237 0 0 0 25 0 9 0 733934634 369205248 37795 33554432000 4194304 4417452 140734291067360 18446744073709551615 4296272 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17275/tid=17287] ppid=17273 vsize=360552 CPUtime=2.4 cores=0-7
/proc/17275/task/17287/stat : 17287 (plingeling) R 17273 17275 17249 0 -1 4202560 9735 1252 0 0 239 1 0 0 22 0 9 0 733934634 369205248 37795 33554432000 4194304 4417452 140734291067360 18446744073709551615 4277584 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17275/tid=17288] ppid=17273 vsize=360552 CPUtime=2.4 cores=0-7
/proc/17275/task/17288/stat : 17288 (plingeling) R 17273 17275 17249 0 -1 4202560 2783 1252 0 0 240 0 0 0 25 0 9 0 733934634 369205248 37795 33554432000 4194304 4417452 140734291067360 18446744073709551615 4302058 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17275/tid=17289] ppid=17273 vsize=360552 CPUtime=2.42 cores=0-7
/proc/17275/task/17289/stat : 17289 (plingeling) R 17273 17275 17249 0 -1 4202560 3384 1252 0 0 242 0 0 0 25 0 9 0 733934635 369205248 37795 33554432000 4194304 4417452 140734291067360 18446744073709551615 4301776 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 19.93
Current children cumulated vsize (KiB) 360552

[startup+6.30042 s]
/proc/loadavg: 3.88 3.94 3.65 9/368 17289
/proc/meminfo: memFree=18031320/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=404716 CPUtime=45.49 cores=0-7
/proc/17275/stat : 17275 (plingeling) S 17273 17275 17249 0 -1 4202496 110026 1252 0 0 4534 15 0 0 18 0 9 0 733934567 414429184 50828 33554432000 4194304 4417452 140734291067360 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/17275/statm: 101179 50828 179 55 0 99065 0
[pid=17275/tid=17282] ppid=17273 vsize=404716 CPUtime=5.61 cores=0-7
/proc/17275/task/17282/stat : 17282 (plingeling) R 17273 17275 17249 0 -1 4202560 5513 1252 0 0 561 0 0 0 25 0 9 0 733934633 414429184 50828 33554432000 4194304 4417452 140734291067360 18446744073709551615 4236319 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17275/tid=17283] ppid=17273 vsize=404716 CPUtime=5.51 cores=0-7
/proc/17275/task/17283/stat : 17283 (plingeling) R 17273 17275 17249 0 -1 4202560 13755 1252 0 0 550 1 0 0 25 0 9 0 733934633 414429184 50828 33554432000 4194304 4417452 140734291067360 18446744073709551615 4252499 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17275/tid=17284] ppid=17273 vsize=404716 CPUtime=5.63 cores=0-7
/proc/17275/task/17284/stat : 17284 (plingeling) R 17273 17275 17249 0 -1 4202560 12230 1252 0 0 561 2 0 0 25 0 9 0 733934633 414429184 50828 33554432000 4194304 4417452 140734291067360 18446744073709551615 4251096 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17275/tid=17285] ppid=17273 vsize=404716 CPUtime=5.62 cores=0-7
/proc/17275/task/17285/stat : 17285 (plingeling) R 17273 17275 17249 0 -1 4202560 2287 1252 0 0 562 0 0 0 25 0 9 0 733934633 414429184 50828 33554432000 4194304 4417452 140734291067360 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=17275/tid=17286] ppid=17273 vsize=404716 CPUtime=5.57 cores=0-7
/proc/17275/task/17286/stat : 17286 (plingeling) R 17273 17275 17249 0 -1 4202560 7492 1252 0 0 556 1 0 0 25 0 9 0 733934634 414429184 50828 33554432000 4194304 4417452 140734291067360 18446744073709551615 4272314 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17275/tid=17287] ppid=17273 vsize=404716 CPUtime=5.6 cores=0-7
/proc/17275/task/17287/stat : 17287 (plingeling) R 17273 17275 17249 0 -1 4202560 16109 1252 0 0 558 2 0 0 25 0 9 0 733934634 414429184 50828 33554432000 4194304 4417452 140734291067360 18446744073709551615 4252365 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17275/tid=17288] ppid=17273 vsize=404716 CPUtime=5.6 cores=0-7
/proc/17275/task/17288/stat : 17288 (plingeling) R 17273 17275 17249 0 -1 4202560 2783 1252 0 0 560 0 0 0 25 0 9 0 733934634 414429184 50828 33554432000 4194304 4417452 140734291067360 18446744073709551615 4236874 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17275/tid=17289] ppid=17273 vsize=404716 CPUtime=5.61 cores=0-7
/proc/17275/task/17289/stat : 17289 (plingeling) R 17273 17275 17249 0 -1 4202560 13918 1252 0 0 559 2 0 0 25 0 9 0 733934635 414429184 50828 33554432000 4194304 4417452 140734291067360 18446744073709551615 4271570 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 45.49
Current children cumulated vsize (KiB) 404716

[startup+12.7003 s]
/proc/loadavg: 4.51 4.07 3.69 9/368 17289
/proc/meminfo: memFree=18032764/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=403976 CPUtime=96.61 cores=0-7
/proc/17275/stat : 17275 (plingeling) S 17273 17275 17249 0 -1 4202496 110026 1252 0 0 9646 15 0 0 18 0 9 0 733934567 413671424 50643 33554432000 4194304 4417452 140734291067360 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/17275/statm: 100994 50643 179 55 0 98880 0
[pid=17275/tid=17282] ppid=17273 vsize=403976 CPUtime=12.01 cores=0-7
/proc/17275/task/17282/stat : 17282 (plingeling) R 17273 17275 17249 0 -1 4202560 5513 1252 0 0 1201 0 0 0 25 0 9 0 733934633 413671424 50643 33554432000 4194304 4417452 140734291067360 18446744073709551615 4250972 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17275/tid=17283] ppid=17273 vsize=403976 CPUtime=11.83 cores=0-7
/proc/17275/task/17283/stat : 17283 (plingeling) R 17273 17275 17249 0 -1 4202560 13755 1252 0 0 1182 1 0 0 25 0 9 0 733934633 413671424 50643 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17275/tid=17284] ppid=17273 vsize=403976 CPUtime=12.03 cores=0-7
/proc/17275/task/17284/stat : 17284 (plingeling) R 17273 17275 17249 0 -1 4202560 12230 1252 0 0 1201 2 0 0 25 0 9 0 733934633 413671424 50643 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364542 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17275/tid=17285] ppid=17273 vsize=403976 CPUtime=12.02 cores=0-7
/proc/17275/task/17285/stat : 17285 (plingeling) R 17273 17275 17249 0 -1 4202560 2287 1252 0 0 1202 0 0 0 25 0 9 0 733934633 413671424 50643 33554432000 4194304 4417452 140734291067360 18446744073709551615 4250432 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=17275/tid=17286] ppid=17273 vsize=403976 CPUtime=11.97 cores=0-7
/proc/17275/task/17286/stat : 17286 (plingeling) R 17273 17275 17249 0 -1 4202560 7492 1252 0 0 1196 1 0 0 25 0 9 0 733934634 413671424 50643 33554432000 4194304 4417452 140734291067360 18446744073709551615 4365996 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17275/tid=17287] ppid=17273 vsize=403976 CPUtime=12 cores=0-7
/proc/17275/task/17287/stat : 17287 (plingeling) R 17273 17275 17249 0 -1 4202560 16109 1252 0 0 1198 2 0 0 25 0 9 0 733934634 413671424 50643 33554432000 4194304 4417452 140734291067360 18446744073709551615 4352329 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17275/tid=17288] ppid=17273 vsize=403976 CPUtime=12 cores=0-7
/proc/17275/task/17288/stat : 17288 (plingeling) R 17273 17275 17249 0 -1 4202560 2783 1252 0 0 1200 0 0 0 25 0 9 0 733934634 413671424 50643 33554432000 4194304 4417452 140734291067360 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17275/tid=17289] ppid=17273 vsize=403976 CPUtime=12.01 cores=0-7
/proc/17275/task/17289/stat : 17289 (plingeling) R 17273 17275 17249 0 -1 4202560 13918 1252 0 0 1199 2 0 0 25 0 9 0 733934635 413671424 50643 33554432000 4194304 4417452 140734291067360 18446744073709551615 4250615 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 96.61
Current children cumulated vsize (KiB) 403976

[startup+25.5012 s]
/proc/loadavg: 5.05 4.20 3.74 9/368 17291
/proc/meminfo: memFree=18010068/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=429468 CPUtime=198.86 cores=0-7
/proc/17275/stat : 17275 (plingeling) S 17273 17275 17249 0 -1 4202496 116424 1252 0 0 19870 16 0 0 18 0 9 0 733934567 439775232 57016 33554432000 4194304 4417452 140734291067360 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/17275/statm: 107367 57016 179 55 0 105253 0
[pid=17275/tid=17282] ppid=17273 vsize=429468 CPUtime=24.82 cores=0-7
/proc/17275/task/17282/stat : 17282 (plingeling) R 17273 17275 17249 0 -1 4202560 5538 1252 0 0 2482 0 0 0 25 0 9 0 733934633 439775232 57016 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17275/tid=17283] ppid=17273 vsize=429468 CPUtime=24.48 cores=0-7
/proc/17275/task/17283/stat : 17283 (plingeling) R 17273 17275 17249 0 -1 4202560 14714 1252 0 0 2446 2 0 0 25 0 9 0 733934633 439775232 57016 33554432000 4194304 4417452 140734291067360 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17275/tid=17284] ppid=17273 vsize=429468 CPUtime=24.83 cores=0-7
/proc/17275/task/17284/stat : 17284 (plingeling) R 17273 17275 17249 0 -1 4202560 12230 1252 0 0 2481 2 0 0 25 0 9 0 733934633 439775232 57016 33554432000 4194304 4417452 140734291067360 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17275/tid=17285] ppid=17273 vsize=429468 CPUtime=24.82 cores=0-7
/proc/17275/task/17285/stat : 17285 (plingeling) R 17273 17275 17249 0 -1 4202560 3680 1252 0 0 2482 0 0 0 25 0 9 0 733934633 439775232 57016 33554432000 4194304 4417452 140734291067360 18446744073709551615 4355852 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=17275/tid=17286] ppid=17273 vsize=429468 CPUtime=24.77 cores=0-7
/proc/17275/task/17286/stat : 17286 (plingeling) R 17273 17275 17249 0 -1 4202560 8029 1252 0 0 2476 1 0 0 25 0 9 0 733934634 439775232 57016 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17275/tid=17287] ppid=17273 vsize=429468 CPUtime=24.8 cores=0-7
/proc/17275/task/17287/stat : 17287 (plingeling) R 17273 17275 17249 0 -1 4202560 16704 1252 0 0 2478 2 0 0 25 0 9 0 733934634 439775232 57016 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17275/tid=17288] ppid=17273 vsize=429468 CPUtime=24.8 cores=0-7
/proc/17275/task/17288/stat : 17288 (plingeling) R 17273 17275 17249 0 -1 4202560 3194 1252 0 0 2480 0 0 0 25 0 9 0 733934634 439775232 57016 33554432000 4194304 4417452 140734291067360 18446744073709551615 4352496 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17275/tid=17289] ppid=17273 vsize=429468 CPUtime=24.81 cores=0-7
/proc/17275/task/17289/stat : 17289 (plingeling) R 17273 17275 17249 0 -1 4202560 16396 1252 0 0 2479 2 0 0 25 0 9 0 733934635 439775232 57016 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 198.86
Current children cumulated vsize (KiB) 429468

[startup+51.1009 s]
/proc/loadavg: 6.12 4.52 3.85 9/368 17291
/proc/meminfo: memFree=17975020/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=465092 CPUtime=403.36 cores=0-7
/proc/17275/stat : 17275 (plingeling) S 17273 17275 17249 0 -1 4202496 133679 1252 0 0 40317 19 0 0 18 0 9 0 733934567 476254208 65923 33554432000 4194304 4417452 140734291067360 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/17275/statm: 116273 65923 180 55 0 114159 0
[pid=17275/tid=17282] ppid=17273 vsize=465092 CPUtime=50.42 cores=0-7
/proc/17275/task/17282/stat : 17282 (plingeling) R 17273 17275 17249 0 -1 4202560 9863 1252 0 0 5041 1 0 0 25 0 9 0 733934633 476254208 65923 33554432000 4194304 4417452 140734291067360 18446744073709551615 4252064 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17275/tid=17283] ppid=17273 vsize=465092 CPUtime=49.78 cores=0-7
/proc/17275/task/17283/stat : 17283 (plingeling) R 17273 17275 17249 0 -1 4202560 19957 1252 0 0 4976 2 0 0 25 0 9 0 733934633 476254208 65923 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17275/tid=17284] ppid=17273 vsize=465092 CPUtime=50.43 cores=0-7

################
# More data... #
################

/proc/17275/task/17285/stat : 17285 (plingeling) R 17273 17275 17249 0 -1 4202560 400953 1252 0 0 472096 72 0 0 25 0 9 0 733934633 968200192 186039 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=17275/tid=17286] ppid=17273 vsize=945508 CPUtime=4721 cores=0-7
/proc/17275/task/17286/stat : 17286 (plingeling) R 17273 17275 17249 0 -1 4202560 527479 1252 0 0 471991 109 0 0 25 0 9 0 733934634 968200192 186039 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364675 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17275/tid=17287] ppid=17273 vsize=945508 CPUtime=4720.79 cores=0-7
/proc/17275/task/17287/stat : 17287 (plingeling) R 17273 17275 17249 0 -1 4202560 589822 1252 0 0 471953 126 0 0 25 0 9 0 733934634 968200192 186039 33554432000 4194304 4417452 140734291067360 18446744073709551615 4352540 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17275/tid=17288] ppid=17273 vsize=945508 CPUtime=4721.54 cores=0-7
/proc/17275/task/17288/stat : 17288 (plingeling) R 17273 17275 17249 0 -1 4202560 459245 1252 0 0 472055 99 0 0 25 0 9 0 733934634 968200192 186039 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17275/tid=17289] ppid=17273 vsize=945508 CPUtime=4720.77 cores=0-7
/proc/17275/task/17289/stat : 17289 (plingeling) R 17273 17275 17249 0 -1 4202560 559644 1252 0 0 471954 123 0 0 25 0 9 0 733934635 968200192 186039 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37715.7
Current children cumulated vsize (KiB) 945508

[startup+4782.3 s]
/proc/loadavg: 8.02 8.08 8.00 9/368 17432
/proc/meminfo: memFree=17652076/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=787096 CPUtime=38194.9 cores=0-7
/proc/17275/stat : 17275 (plingeling) S 17273 17275 17249 0 -1 4202496 4345864 1252 0 0 3818577 913 0 0 18 0 9 0 733934567 805986304 146436 33554432000 4194304 4417452 140734291067360 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/17275/statm: 196774 146436 180 55 0 194660 0
[pid=17275/tid=17282] ppid=17273 vsize=787096 CPUtime=4781.68 cores=0-7
/proc/17275/task/17282/stat : 17282 (plingeling) R 17273 17275 17249 0 -1 4202560 623417 1252 0 0 478032 136 0 0 25 0 9 0 733934633 805986304 146436 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17275/tid=17283] ppid=17273 vsize=787096 CPUtime=4725.13 cores=0-7
/proc/17275/task/17283/stat : 17283 (plingeling) R 17273 17275 17249 0 -1 4202560 602779 1252 0 0 472385 128 0 0 25 0 9 0 733934633 805986304 146436 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17275/tid=17284] ppid=17273 vsize=787096 CPUtime=4781.68 cores=0-7
/proc/17275/task/17284/stat : 17284 (plingeling) R 17273 17275 17249 0 -1 4202560 520123 1252 0 0 478059 109 0 0 25 0 9 0 733934633 805986304 146436 33554432000 4194304 4417452 140734291067360 18446744073709551615 4354469 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17275/tid=17285] ppid=17273 vsize=787096 CPUtime=4781.66 cores=0-7
/proc/17275/task/17285/stat : 17285 (plingeling) R 17273 17275 17249 0 -1 4202560 408902 1252 0 0 478093 73 0 0 25 0 9 0 733934633 805986304 146436 33554432000 4194304 4417452 140734291067360 18446744073709551615 4366400 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=17275/tid=17286] ppid=17273 vsize=787096 CPUtime=4780.98 cores=0-7
/proc/17275/task/17286/stat : 17286 (plingeling) R 17273 17275 17249 0 -1 4202560 534070 1252 0 0 477988 110 0 0 25 0 9 0 733934634 805986304 146436 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17275/tid=17287] ppid=17273 vsize=787096 CPUtime=4780.77 cores=0-7
/proc/17275/task/17287/stat : 17287 (plingeling) R 17273 17275 17249 0 -1 4202560 590753 1252 0 0 477950 127 0 0 25 0 9 0 733934634 805986304 146436 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17275/tid=17288] ppid=17273 vsize=787096 CPUtime=4781.52 cores=0-7
/proc/17275/task/17288/stat : 17288 (plingeling) R 17273 17275 17249 0 -1 4202560 462226 1252 0 0 478053 99 0 0 25 0 9 0 733934634 805986304 146436 33554432000 4194304 4417452 140734291067360 18446744073709551615 4352320 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17275/tid=17289] ppid=17273 vsize=787096 CPUtime=4780.76 cores=0-7
/proc/17275/task/17289/stat : 17289 (plingeling) R 17273 17275 17249 0 -1 4202560 567655 1252 0 0 477952 124 0 0 25 0 9 0 733934635 805986304 146436 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38194.9
Current children cumulated vsize (KiB) 787096

[startup+4842.3 s]
/proc/loadavg: 8.04 8.08 8.00 9/368 17434
/proc/meminfo: memFree=17509692/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=932092 CPUtime=38674.2 cores=0-7
/proc/17275/stat : 17275 (plingeling) S 17273 17275 17249 0 -1 4202496 4402141 1252 0 0 3866496 922 0 0 18 0 9 0 733934567 954462208 182685 33554432000 4194304 4417452 140734291067360 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/17275/statm: 233023 182685 180 55 0 230909 0
[pid=17275/tid=17282] ppid=17273 vsize=932092 CPUtime=4841.68 cores=0-7
/proc/17275/task/17282/stat : 17282 (plingeling) R 17273 17275 17249 0 -1 4202560 632306 1252 0 0 484030 138 0 0 25 0 9 0 733934633 954462208 182685 33554432000 4194304 4417452 140734291067360 18446744073709551615 4250927 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17275/tid=17283] ppid=17273 vsize=932092 CPUtime=4784.41 cores=0-7
/proc/17275/task/17283/stat : 17283 (plingeling) R 17273 17275 17249 0 -1 4202560 610113 1252 0 0 478312 129 0 0 25 0 9 0 733934633 954462208 182685 33554432000 4194304 4417452 140734291067360 18446744073709551615 4354159 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17275/tid=17284] ppid=17273 vsize=932092 CPUtime=4841.68 cores=0-7
/proc/17275/task/17284/stat : 17284 (plingeling) R 17273 17275 17249 0 -1 4202560 526096 1252 0 0 484058 110 0 0 25 0 9 0 733934633 954462208 182685 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17275/tid=17285] ppid=17273 vsize=932092 CPUtime=4841.67 cores=0-7
/proc/17275/task/17285/stat : 17285 (plingeling) R 17273 17275 17249 0 -1 4202560 417909 1252 0 0 484093 74 0 0 25 0 9 0 733934633 954462208 182685 33554432000 4194304 4417452 140734291067360 18446744073709551615 4356852 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=17275/tid=17286] ppid=17273 vsize=932092 CPUtime=4840.98 cores=0-7
/proc/17275/task/17286/stat : 17286 (plingeling) R 17273 17275 17249 0 -1 4202560 541682 1252 0 0 483987 111 0 0 25 0 9 0 733934634 954462208 182685 33554432000 4194304 4417452 140734291067360 18446744073709551615 4353830 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17275/tid=17287] ppid=17273 vsize=932092 CPUtime=4840.75 cores=0-7
/proc/17275/task/17287/stat : 17287 (plingeling) R 17273 17275 17249 0 -1 4202560 598760 1252 0 0 483948 127 0 0 25 0 9 0 733934634 954462208 182685 33554432000 4194304 4417452 140734291067360 18446744073709551615 4352385 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17275/tid=17288] ppid=17273 vsize=932092 CPUtime=4841.52 cores=0-7
/proc/17275/task/17288/stat : 17288 (plingeling) R 17273 17275 17249 0 -1 4202560 466565 1252 0 0 484052 100 0 0 25 0 9 0 733934634 954462208 182685 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17275/tid=17289] ppid=17273 vsize=932092 CPUtime=4840.75 cores=0-7
/proc/17275/task/17289/stat : 17289 (plingeling) R 17273 17275 17249 0 -1 4202560 572771 1252 0 0 483951 124 0 0 25 0 9 0 733934635 954462208 182685 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38674.2
Current children cumulated vsize (KiB) 932092

[startup+4902.3 s]
/proc/loadavg: 8.09 8.08 8.00 9/368 17436
/proc/meminfo: memFree=17430944/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=1007796 CPUtime=39153.5 cores=0-7
/proc/17275/stat : 17275 (plingeling) S 17273 17275 17249 0 -1 4202496 4463441 1252 0 0 3914415 933 0 0 18 0 9 0 733934567 1031983104 201611 33554432000 4194304 4417452 140734291067360 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/17275/statm: 251949 201611 180 55 0 249835 0
[pid=17275/tid=17282] ppid=17273 vsize=1007796 CPUtime=4901.68 cores=0-7
/proc/17275/task/17282/stat : 17282 (plingeling) R 17273 17275 17249 0 -1 4202560 641604 1252 0 0 490029 139 0 0 25 0 9 0 733934633 1031983104 201611 33554432000 4194304 4417452 140734291067360 18446744073709551615 4354150 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17275/tid=17283] ppid=17273 vsize=1007796 CPUtime=4843.7 cores=0-7
/proc/17275/task/17283/stat : 17283 (plingeling) R 17273 17275 17249 0 -1 4202560 625817 1252 0 0 484237 133 0 0 25 0 9 0 733934633 1031983104 201611 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17275/tid=17284] ppid=17273 vsize=1007796 CPUtime=4901.67 cores=0-7
/proc/17275/task/17284/stat : 17284 (plingeling) R 17273 17275 17249 0 -1 4202560 531659 1252 0 0 490057 110 0 0 25 0 9 0 733934633 1031983104 201611 33554432000 4194304 4417452 140734291067360 18446744073709551615 4352347 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17275/tid=17285] ppid=17273 vsize=1007796 CPUtime=4901.67 cores=0-7
/proc/17275/task/17285/stat : 17285 (plingeling) R 17273 17275 17249 0 -1 4202560 426749 1252 0 0 490091 76 0 0 25 0 9 0 733934633 1031983104 201611 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=17275/tid=17286] ppid=17273 vsize=1007796 CPUtime=4900.99 cores=0-7
/proc/17275/task/17286/stat : 17286 (plingeling) R 17273 17275 17249 0 -1 4202560 547217 1252 0 0 489987 112 0 0 25 0 9 0 733934634 1031983104 201611 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17275/tid=17287] ppid=17273 vsize=1007796 CPUtime=4900.75 cores=0-7
/proc/17275/task/17287/stat : 17287 (plingeling) R 17273 17275 17249 0 -1 4202560 606213 1252 0 0 489946 129 0 0 25 0 9 0 733934634 1031983104 201611 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17275/tid=17288] ppid=17273 vsize=1007796 CPUtime=4901.53 cores=0-7
/proc/17275/task/17288/stat : 17288 (plingeling) R 17273 17275 17249 0 -1 4202560 474490 1252 0 0 490051 102 0 0 25 0 9 0 733934634 1031983104 201611 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17275/tid=17289] ppid=17273 vsize=1007796 CPUtime=4900.76 cores=0-7
/proc/17275/task/17289/stat : 17289 (plingeling) R 17273 17275 17249 0 -1 4202560 573753 1252 0 0 489951 125 0 0 25 0 9 0 733934635 1031983104 201611 33554432000 4194304 4417452 140734291067360 18446744073709551615 4352402 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39153.5
Current children cumulated vsize (KiB) 1007796

[startup+4962.3 s]
/proc/loadavg: 8.03 8.06 8.00 9/368 17436
/proc/meminfo: memFree=17490804/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=947928 CPUtime=39632.7 cores=0-7
/proc/17275/stat : 17275 (plingeling) S 17273 17275 17249 0 -1 4202496 4512108 1252 0 0 3962334 940 0 0 18 0 9 0 733934567 970678272 186644 33554432000 4194304 4417452 140734291067360 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/17275/statm: 236982 186644 180 55 0 234868 0
[pid=17275/tid=17282] ppid=17273 vsize=947928 CPUtime=4961.68 cores=0-7
/proc/17275/task/17282/stat : 17282 (plingeling) R 17273 17275 17249 0 -1 4202560 648522 1252 0 0 496028 140 0 0 25 0 9 0 733934633 970678272 186644 33554432000 4194304 4417452 140734291067360 18446744073709551615 4307420 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17275/tid=17283] ppid=17273 vsize=947928 CPUtime=4902.99 cores=0-7
/proc/17275/task/17283/stat : 17283 (plingeling) R 17273 17275 17249 0 -1 4202560 627699 1252 0 0 490166 133 0 0 25 0 9 0 733934633 970678272 186644 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17275/tid=17284] ppid=17273 vsize=947928 CPUtime=4961.67 cores=0-7
/proc/17275/task/17284/stat : 17284 (plingeling) R 17273 17275 17249 0 -1 4202560 540139 1252 0 0 496055 112 0 0 25 0 9 0 733934633 970678272 186644 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17275/tid=17285] ppid=17273 vsize=947928 CPUtime=4961.67 cores=0-7
/proc/17275/task/17285/stat : 17285 (plingeling) R 17273 17275 17249 0 -1 4202560 431496 1252 0 0 496090 77 0 0 25 0 9 0 733934633 970678272 186644 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364583 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=17275/tid=17286] ppid=17273 vsize=947928 CPUtime=4960.99 cores=0-7
/proc/17275/task/17286/stat : 17286 (plingeling) R 17273 17275 17249 0 -1 4202560 547548 1252 0 0 495987 112 0 0 25 0 9 0 733934634 970678272 186644 33554432000 4194304 4417452 140734291067360 18446744073709551615 4271602 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17275/tid=17287] ppid=17273 vsize=947928 CPUtime=4960.73 cores=0-7
/proc/17275/task/17287/stat : 17287 (plingeling) R 17273 17275 17249 0 -1 4202560 615800 1252 0 0 495943 130 0 0 25 0 9 0 733934634 970678272 186644 33554432000 4194304 4417452 140734291067360 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17275/tid=17288] ppid=17273 vsize=947928 CPUtime=4961.52 cores=0-7
/proc/17275/task/17288/stat : 17288 (plingeling) R 17273 17275 17249 0 -1 4202560 486457 1252 0 0 496049 103 0 0 25 0 9 0 733934634 970678272 186644 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364442 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17275/tid=17289] ppid=17273 vsize=947928 CPUtime=4960.76 cores=0-7
/proc/17275/task/17289/stat : 17289 (plingeling) R 17273 17275 17249 0 -1 4202560 578508 1252 0 0 495950 126 0 0 25 0 9 0 733934635 970678272 186644 33554432000 4194304 4417452 140734291067360 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39632.7
Current children cumulated vsize (KiB) 947928



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5008.31 s]
/proc/loadavg: 8.09 8.07 8.01 9/368 17438
/proc/meminfo: memFree=17501952/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=936276 CPUtime=40000.4 cores=0-7
/proc/17275/stat : 17275 (plingeling) S 17273 17275 17249 0 -1 4202496 4561265 1252 0 0 3999090 950 0 0 18 0 9 0 733934567 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/17275/statm: 234069 183763 180 55 0 231955 0
[pid=17275/tid=17282] ppid=17273 vsize=936276 CPUtime=5007.7 cores=0-7
/proc/17275/task/17282/stat : 17282 (plingeling) R 17273 17275 17249 0 -1 4202560 658797 1252 0 0 500629 141 0 0 25 0 9 0 733934633 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 4250936 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17275/tid=17283] ppid=17273 vsize=936276 CPUtime=4948.46 cores=0-7
/proc/17275/task/17283/stat : 17283 (plingeling) R 17273 17275 17249 0 -1 4202560 628505 1252 0 0 494712 134 0 0 25 0 9 0 733934633 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17275/tid=17284] ppid=17273 vsize=936276 CPUtime=5007.7 cores=0-7
/proc/17275/task/17284/stat : 17284 (plingeling) R 17273 17275 17249 0 -1 4202560 547275 1252 0 0 500657 113 0 0 25 0 9 0 733934633 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17275/tid=17285] ppid=17273 vsize=936276 CPUtime=5007.7 cores=0-7
/proc/17275/task/17285/stat : 17285 (plingeling) R 17273 17275 17249 0 -1 4202560 443688 1252 0 0 500691 79 0 0 25 0 9 0 733934633 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 4355860 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=17275/tid=17286] ppid=17273 vsize=936276 CPUtime=5007.01 cores=0-7
/proc/17275/task/17286/stat : 17286 (plingeling) R 17273 17275 17249 0 -1 4202560 552114 1252 0 0 500589 112 0 0 25 0 9 0 733934634 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 4352376 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17275/tid=17287] ppid=17273 vsize=936276 CPUtime=5006.75 cores=0-7
/proc/17275/task/17287/stat : 17287 (plingeling) R 17273 17275 17249 0 -1 4202560 623879 1252 0 0 500543 132 0 0 25 0 9 0 733934634 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 4250947 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17275/tid=17288] ppid=17273 vsize=936276 CPUtime=5007.55 cores=0-7
/proc/17275/task/17288/stat : 17288 (plingeling) R 17273 17275 17249 0 -1 4202560 488948 1252 0 0 500651 104 0 0 25 0 9 0 733934634 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 4366005 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17275/tid=17289] ppid=17273 vsize=936276 CPUtime=5006.78 cores=0-7
/proc/17275/task/17289/stat : 17289 (plingeling) R 17273 17275 17249 0 -1 4202560 582120 1252 0 0 500552 126 0 0 25 0 9 0 733934635 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 4353776 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 936276

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5008.31 s]
/proc/loadavg: 8.09 8.07 8.01 9/368 17438
/proc/meminfo: memFree=17501952/32951124 swapFree=45193364/67111528
[pid=17275] ppid=17273 vsize=936276 CPUtime=40000.4 cores=0-7
/proc/17275/stat : 17275 (plingeling) S 17273 17275 17249 0 -1 4202496 4561265 1252 0 0 3999090 950 0 0 18 0 9 0 733934567 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/17275/statm: 234069 183763 180 55 0 231955 0
[pid=17275/tid=17282] ppid=17273 vsize=936276 CPUtime=5007.7 cores=0-7
/proc/17275/task/17282/stat : 17282 (plingeling) R 17273 17275 17249 0 -1 4202560 658797 1252 0 0 500629 141 0 0 25 0 9 0 733934633 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 4250936 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17275/tid=17283] ppid=17273 vsize=936276 CPUtime=4948.46 cores=0-7
/proc/17275/task/17283/stat : 17283 (plingeling) R 17273 17275 17249 0 -1 4202560 628505 1252 0 0 494712 134 0 0 25 0 9 0 733934633 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17275/tid=17284] ppid=17273 vsize=936276 CPUtime=5007.7 cores=0-7
/proc/17275/task/17284/stat : 17284 (plingeling) R 17273 17275 17249 0 -1 4202560 547275 1252 0 0 500657 113 0 0 25 0 9 0 733934633 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17275/tid=17285] ppid=17273 vsize=936276 CPUtime=5007.7 cores=0-7
/proc/17275/task/17285/stat : 17285 (plingeling) R 17273 17275 17249 0 -1 4202560 443688 1252 0 0 500691 79 0 0 25 0 9 0 733934633 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 4355860 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=17275/tid=17286] ppid=17273 vsize=936276 CPUtime=5007.01 cores=0-7
/proc/17275/task/17286/stat : 17286 (plingeling) R 17273 17275 17249 0 -1 4202560 552114 1252 0 0 500589 112 0 0 25 0 9 0 733934634 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 4352376 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17275/tid=17287] ppid=17273 vsize=936276 CPUtime=5006.75 cores=0-7
/proc/17275/task/17287/stat : 17287 (plingeling) R 17273 17275 17249 0 -1 4202560 623879 1252 0 0 500543 132 0 0 25 0 9 0 733934634 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 4250947 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17275/tid=17288] ppid=17273 vsize=936276 CPUtime=5007.55 cores=0-7
/proc/17275/task/17288/stat : 17288 (plingeling) R 17273 17275 17249 0 -1 4202560 488948 1252 0 0 500651 104 0 0 25 0 9 0 733934634 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 4366005 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17275/tid=17289] ppid=17273 vsize=936276 CPUtime=5006.78 cores=0-7
/proc/17275/task/17289/stat : 17289 (plingeling) R 17273 17275 17249 0 -1 4202560 582120 1252 0 0 500552 126 0 0 25 0 9 0 733934635 958746624 183763 33554432000 4194304 4417452 140734291067360 18446744073709551615 4353776 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 936276

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5008.37
CPU time (s): 40000.5
CPU user time (s): 39990.9
CPU system time (s): 9.54355
CPU usage (%): 798.673
Max. virtual memory (cumulated for all children) (KiB): 1109852

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39990.9
system time used= 9.54355
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 4562517
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 368
involuntary context switches= 191862

runsolver used 16.7205 second user time and 42.4935 second system time

The end

Launcher Data 

Begin job on node118 at 2011-05-15 07:23:59
IDJOB=3372257
IDBENCH=70829
IDSOLVER=1851
FILE ID=node118/3372257-1305437039
RUNJOBID= node118-1305437039-17263
PBS_JOBID= 13324763
Free space on /tmp= 73324 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT09/APPLICATIONS/crypto/md5gen/gus-md5-12.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372257-1305437039/watcher-3372257-1305437039 -o /tmp/evaluation-result-3372257-1305437039/solver-3372257-1305437039 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3372257-1305437039.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 0621e557bffdc18ca509447665adecda
RANDOM SEED=1018573700

node118.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5374.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      18236976 kB
Buffers:        286996 kB
Cached:        4428600 kB
SwapCached:      22084 kB
Active:       11782632 kB
Inactive:      2627204 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      18236976 kB
SwapTotal:    67111528 kB
SwapFree:     45193364 kB
Dirty:            4464 kB
Writeback:           0 kB
AnonPages:     9685368 kB
Mapped:          14520 kB
Slab:           147672 kB
PageTables:      89704 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 88626884 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73324 MiB
End job on node118 at 2011-05-15 08:47:29