Trace number 3372260

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.2 5005.84

General information on the benchmark

NameAPPLICATIONS/crypto/
md5gen/gus-md5-11.cnf
MD5SUM49867705b76fc90285dd6fc48de2613c
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark1376.76
Satisfiable
(Un)Satisfiability was proved
Number of variables69561
Number of clauses226787
Sum of the clauses size587727
Maximum clause length3
Minimum clause length1
Number of clauses of size 1129
Number of clauses of size 292376
Number of clauses of size 3134282
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372260-1305437047/watcher-3372260-1305437047 -o /tmp/evaluation-result-3372260-1305437047/solver-3372260-1305437047 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3372260-1305437047.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.09 7.81 6.70 2/255 26485
/proc/meminfo: memFree=23062280/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=18420 CPUtime=0 cores=0-7
/proc/26485/stat : 26485 (plingeling) R 26483 26485 26398 0 -1 4202496 2464 1261 0 0 0 0 0 0 21 0 1 0 655302951 18862080 2395 33554432000 4194304 4417452 140736607358672 18446744073709551615 246516528221 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26485/statm: 4605 2408 144 55 0 2491 0

[startup+0.10736 s]
/proc/loadavg: 8.09 7.81 6.70 2/255 26485
/proc/meminfo: memFree=23062280/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=49380 CPUtime=0.1 cores=0-7
/proc/26485/stat : 26485 (plingeling) R 26483 26485 26398 0 -1 4202496 10504 1261 0 0 9 1 0 0 21 0 1 0 655302951 50565120 10370 33554432000 4194304 4417452 140736607358672 18446744073709551615 4264940 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26485/statm: 12345 10370 149 55 0 10231 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 49380

[startup+0.200349 s]
/proc/loadavg: 8.09 7.81 6.70 2/255 26485
/proc/meminfo: memFree=23062280/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=66164 CPUtime=0.19 cores=0-7
/proc/26485/stat : 26485 (plingeling) R 26483 26485 26398 0 -1 4202496 14462 1261 0 0 18 1 0 0 22 0 1 0 655302951 67751936 14327 33554432000 4194304 4417452 140736607358672 18446744073709551615 246516528221 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26485/statm: 16541 14354 149 55 0 14427 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 66164

[startup+0.300324 s]
/proc/loadavg: 8.09 7.81 6.70 2/255 26485
/proc/meminfo: memFree=23062280/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=72308 CPUtime=0.28 cores=0-7
/proc/26485/stat : 26485 (plingeling) R 26483 26485 26398 0 -1 4202496 16237 1261 0 0 27 1 0 0 22 0 1 0 655302951 74043392 16102 33554432000 4194304 4417452 140736607358672 18446744073709551615 4235968 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26485/statm: 18077 16102 149 55 0 15963 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 72308

[startup+0.700218 s]
/proc/loadavg: 8.09 7.81 6.70 2/255 26485
/proc/meminfo: memFree=23062280/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=206056 CPUtime=0.88 cores=0-7
/proc/26485/stat : 26485 (plingeling) S 26483 26485 26398 0 -1 4202496 37817 1261 0 0 84 4 0 0 18 0 9 0 655302951 211001344 29045 33554432000 4194304 4417452 140736607358672 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/26485/statm: 51514 29045 159 55 0 49400 0
Current children cumulated CPU time (s) 0.88
Current children cumulated vsize (KiB) 206056

[startup+1.50107 s]
/proc/loadavg: 8.09 7.81 6.70 9/264 26499
/proc/meminfo: memFree=22915468/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=287216 CPUtime=7.12 cores=0-7
/proc/26485/stat : 26485 (plingeling) S 26483 26485 26398 0 -1 4202496 56622 1261 0 0 705 7 0 0 18 0 9 0 655302951 294109184 37768 33554432000 4194304 4417452 140736607358672 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/26485/statm: 71804 37768 173 55 0 69690 0
[pid=26485/tid=26492] ppid=26483 vsize=287216 CPUtime=0.82 cores=0-7
/proc/26485/task/26492/stat : 26492 (plingeling) R 26483 26485 26398 0 -1 4202560 1898 1261 0 0 82 0 0 0 25 0 9 0 655303017 294109184 37768 33554432000 4194304 4417452 140736607358672 18446744073709551615 4301955 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26485/tid=26493] ppid=26483 vsize=287216 CPUtime=0.75 cores=0-7
/proc/26485/task/26493/stat : 26493 (plingeling) R 26483 26485 26398 0 -1 4202560 1866 1261 0 0 75 0 0 0 25 0 9 0 655303017 294109184 37768 33554432000 4194304 4417452 140736607358672 18446744073709551615 4282467 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26485/tid=26494] ppid=26483 vsize=287216 CPUtime=0.77 cores=0-7
/proc/26485/task/26494/stat : 26494 (plingeling) R 26483 26485 26398 0 -1 4202560 1908 1261 0 0 77 0 0 0 21 0 9 0 655303017 294109184 37768 33554432000 4194304 4417452 140736607358672 18446744073709551615 4286467 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26485/tid=26495] ppid=26483 vsize=287216 CPUtime=0.82 cores=0-7
/proc/26485/task/26495/stat : 26495 (plingeling) R 26483 26485 26398 0 -1 4202560 2007 1261 0 0 82 0 0 0 18 0 9 0 655303017 294109184 37768 33554432000 4194304 4417452 140736607358672 18446744073709551615 4232791 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26485/tid=26496] ppid=26483 vsize=287216 CPUtime=0.82 cores=0-7
/proc/26485/task/26496/stat : 26496 (plingeling) R 26483 26485 26398 0 -1 4202560 1830 1261 0 0 82 0 0 0 21 0 9 0 655303017 294109184 37768 33554432000 4194304 4417452 140736607358672 18446744073709551615 4284514 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26485/tid=26497] ppid=26483 vsize=287216 CPUtime=0.79 cores=0-7
/proc/26485/task/26497/stat : 26497 (plingeling) R 26483 26485 26398 0 -1 4202560 1829 1261 0 0 79 0 0 0 21 0 9 0 655303017 294109184 37768 33554432000 4194304 4417452 140736607358672 18446744073709551615 4287486 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26485/tid=26498] ppid=26483 vsize=287216 CPUtime=0.82 cores=0-7
/proc/26485/task/26498/stat : 26498 (plingeling) R 26483 26485 26398 0 -1 4202560 4641 1261 0 0 82 0 0 0 18 0 9 0 655303017 294109184 37768 33554432000 4194304 4417452 140736607358672 18446744073709551615 4287187 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26485/tid=26499] ppid=26483 vsize=287216 CPUtime=0.81 cores=0-7
/proc/26485/task/26499/stat : 26499 (plingeling) R 26483 26485 26398 0 -1 4202560 4705 1261 0 0 81 0 0 0 22 0 9 0 655303017 294109184 37768 33554432000 4194304 4417452 140736607358672 18446744073709551615 4234242 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 7.12
Current children cumulated vsize (KiB) 287216

[startup+3.10078 s]
/proc/loadavg: 8.09 7.81 6.70 9/264 26499
/proc/meminfo: memFree=22907160/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=291276 CPUtime=19.91 cores=0-7
/proc/26485/stat : 26485 (plingeling) S 26483 26485 26398 0 -1 4202496 69827 1261 0 0 1983 8 0 0 18 0 9 0 655302951 298266624 38986 33554432000 4194304 4417452 140736607358672 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/26485/statm: 72819 38986 176 55 0 70705 0
[pid=26485/tid=26492] ppid=26483 vsize=291276 CPUtime=2.42 cores=0-7
/proc/26485/task/26492/stat : 26492 (plingeling) R 26483 26485 26398 0 -1 4202560 1898 1261 0 0 242 0 0 0 25 0 9 0 655303017 298266624 38986 33554432000 4194304 4417452 140736607358672 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26485/tid=26493] ppid=26483 vsize=291276 CPUtime=2.35 cores=0-7
/proc/26485/task/26493/stat : 26493 (plingeling) R 26483 26485 26398 0 -1 4202560 3423 1261 0 0 235 0 0 0 25 0 9 0 655303017 298266624 38986 33554432000 4194304 4417452 140736607358672 18446744073709551615 4237157 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26485/tid=26494] ppid=26483 vsize=291276 CPUtime=2.37 cores=0-7
/proc/26485/task/26494/stat : 26494 (plingeling) R 26483 26485 26398 0 -1 4202560 5464 1261 0 0 237 0 0 0 25 0 9 0 655303017 298266624 38986 33554432000 4194304 4417452 140736607358672 18446744073709551615 4274422 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26485/tid=26495] ppid=26483 vsize=291276 CPUtime=2.42 cores=0-7
/proc/26485/task/26495/stat : 26495 (plingeling) R 26483 26485 26398 0 -1 4202560 2007 1261 0 0 242 0 0 0 22 0 9 0 655303017 298266624 38986 33554432000 4194304 4417452 140736607358672 18446744073709551615 4236941 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26485/tid=26496] ppid=26483 vsize=291276 CPUtime=2.42 cores=0-7
/proc/26485/task/26496/stat : 26496 (plingeling) R 26483 26485 26398 0 -1 4202560 3388 1261 0 0 242 0 0 0 25 0 9 0 655303017 298266624 38986 33554432000 4194304 4417452 140736607358672 18446744073709551615 4237169 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26485/tid=26497] ppid=26483 vsize=291276 CPUtime=2.39 cores=0-7
/proc/26485/task/26497/stat : 26497 (plingeling) R 26483 26485 26398 0 -1 4202560 1829 1261 0 0 239 0 0 0 25 0 9 0 655303017 298266624 38986 33554432000 4194304 4417452 140736607358672 18446744073709551615 4264405 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26485/tid=26498] ppid=26483 vsize=291276 CPUtime=2.41 cores=0-7
/proc/26485/task/26498/stat : 26498 (plingeling) R 26483 26485 26398 0 -1 4202560 6366 1261 0 0 241 0 0 0 22 0 9 0 655303017 298266624 38986 33554432000 4194304 4417452 140736607358672 18446744073709551615 4265832 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26485/tid=26499] ppid=26483 vsize=291276 CPUtime=2.4 cores=0-7
/proc/26485/task/26499/stat : 26499 (plingeling) R 26483 26485 26398 0 -1 4202560 9514 1261 0 0 239 1 0 0 25 0 9 0 655303017 298266624 38986 33554432000 4194304 4417452 140736607358672 18446744073709551615 246516528089 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 19.91
Current children cumulated vsize (KiB) 291276

[startup+6.3012 s]
/proc/loadavg: 8.08 7.81 6.70 10/264 26499
/proc/meminfo: memFree=22862528/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=345412 CPUtime=45.49 cores=0-7
/proc/26485/stat : 26485 (plingeling) S 26483 26485 26398 0 -1 4202496 87531 1261 0 0 4537 12 0 0 18 0 9 0 655302951 353701888 49428 33554432000 4194304 4417452 140736607358672 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/26485/statm: 86353 49428 179 55 0 84239 0
[pid=26485/tid=26492] ppid=26483 vsize=345412 CPUtime=5.62 cores=0-7
/proc/26485/task/26492/stat : 26492 (plingeling) R 26483 26485 26398 0 -1 4202560 4977 1261 0 0 562 0 0 0 25 0 9 0 655303017 353701888 49428 33554432000 4194304 4417452 140736607358672 18446744073709551615 4273098 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26485/tid=26493] ppid=26483 vsize=345412 CPUtime=5.56 cores=0-7
/proc/26485/task/26493/stat : 26493 (plingeling) R 26483 26485 26398 0 -1 4202560 5494 1261 0 0 555 1 0 0 25 0 9 0 655303017 353701888 49428 33554432000 4194304 4417452 140736607358672 18446744073709551615 4253075 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26485/tid=26494] ppid=26483 vsize=345412 CPUtime=5.56 cores=0-7
/proc/26485/task/26494/stat : 26494 (plingeling) R 26483 26485 26398 0 -1 4202560 6299 1261 0 0 556 0 0 0 25 0 9 0 655303017 353701888 49428 33554432000 4194304 4417452 140736607358672 18446744073709551615 4352402 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26485/tid=26495] ppid=26483 vsize=345412 CPUtime=5.62 cores=0-7
/proc/26485/task/26495/stat : 26495 (plingeling) R 26483 26485 26398 0 -1 4202560 2008 1261 0 0 562 0 0 0 25 0 9 0 655303017 353701888 49428 33554432000 4194304 4417452 140736607358672 18446744073709551615 4366024 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26485/tid=26496] ppid=26483 vsize=345412 CPUtime=5.61 cores=0-7
/proc/26485/task/26496/stat : 26496 (plingeling) R 26483 26485 26398 0 -1 4202560 7673 1261 0 0 561 0 0 0 25 0 9 0 655303017 353701888 49428 33554432000 4194304 4417452 140736607358672 18446744073709551615 4253336 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26485/tid=26497] ppid=26483 vsize=345412 CPUtime=5.59 cores=0-7
/proc/26485/task/26497/stat : 26497 (plingeling) R 26483 26485 26398 0 -1 4202560 4907 1261 0 0 559 0 0 0 25 0 9 0 655303017 353701888 49428 33554432000 4194304 4417452 140736607358672 18446744073709551615 4253152 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26485/tid=26498] ppid=26483 vsize=345412 CPUtime=5.62 cores=0-7
/proc/26485/task/26498/stat : 26498 (plingeling) R 26483 26485 26398 0 -1 4202560 8155 1261 0 0 561 1 0 0 25 0 9 0 655303017 353701888 49428 33554432000 4194304 4417452 140736607358672 18446744073709551615 4250945 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26485/tid=26499] ppid=26483 vsize=345412 CPUtime=5.57 cores=0-7
/proc/26485/task/26499/stat : 26499 (plingeling) R 26483 26485 26398 0 -1 4202560 12080 1261 0 0 556 1 0 0 25 0 9 0 655303017 353701888 49428 33554432000 4194304 4417452 140736607358672 18446744073709551615 4258545 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 45.49
Current children cumulated vsize (KiB) 345412

[startup+12.701 s]
/proc/loadavg: 8.07 7.81 6.71 9/264 26499
/proc/meminfo: memFree=22863292/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=344672 CPUtime=96.63 cores=0-7
/proc/26485/stat : 26485 (plingeling) S 26483 26485 26398 0 -1 4202496 87555 1261 0 0 9651 12 0 0 18 0 9 0 655302951 352944128 49243 33554432000 4194304 4417452 140736607358672 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/26485/statm: 86168 49243 179 55 0 84054 0
[pid=26485/tid=26492] ppid=26483 vsize=344672 CPUtime=12.02 cores=0-7
/proc/26485/task/26492/stat : 26492 (plingeling) R 26483 26485 26398 0 -1 4202560 4977 1261 0 0 1202 0 0 0 25 0 9 0 655303017 352944128 49243 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26485/tid=26493] ppid=26483 vsize=344672 CPUtime=11.96 cores=0-7
/proc/26485/task/26493/stat : 26493 (plingeling) R 26483 26485 26398 0 -1 4202560 5518 1261 0 0 1195 1 0 0 25 0 9 0 655303017 352944128 49243 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26485/tid=26494] ppid=26483 vsize=344672 CPUtime=11.96 cores=0-7
/proc/26485/task/26494/stat : 26494 (plingeling) R 26483 26485 26398 0 -1 4202560 6299 1261 0 0 1196 0 0 0 25 0 9 0 655303017 352944128 49243 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26485/tid=26495] ppid=26483 vsize=344672 CPUtime=12.02 cores=0-7
/proc/26485/task/26495/stat : 26495 (plingeling) R 26483 26485 26398 0 -1 4202560 2008 1261 0 0 1202 0 0 0 25 0 9 0 655303017 352944128 49243 33554432000 4194304 4417452 140736607358672 18446744073709551615 4251010 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26485/tid=26496] ppid=26483 vsize=344672 CPUtime=12.01 cores=0-7
/proc/26485/task/26496/stat : 26496 (plingeling) R 26483 26485 26398 0 -1 4202560 7673 1261 0 0 1201 0 0 0 25 0 9 0 655303017 352944128 49243 33554432000 4194304 4417452 140736607358672 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26485/tid=26497] ppid=26483 vsize=344672 CPUtime=11.99 cores=0-7
/proc/26485/task/26497/stat : 26497 (plingeling) R 26483 26485 26398 0 -1 4202560 4907 1261 0 0 1199 0 0 0 25 0 9 0 655303017 352944128 49243 33554432000 4194304 4417452 140736607358672 18446744073709551615 4353900 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26485/tid=26498] ppid=26483 vsize=344672 CPUtime=12.02 cores=0-7
/proc/26485/task/26498/stat : 26498 (plingeling) R 26483 26485 26398 0 -1 4202560 8155 1261 0 0 1201 1 0 0 25 0 9 0 655303017 352944128 49243 33554432000 4194304 4417452 140736607358672 18446744073709551615 4353572 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26485/tid=26499] ppid=26483 vsize=344672 CPUtime=11.92 cores=0-7
/proc/26485/task/26499/stat : 26499 (plingeling) R 26483 26485 26398 0 -1 4202560 12080 1261 0 0 1191 1 0 0 25 0 9 0 655303017 352944128 49243 33554432000 4194304 4417452 140736607358672 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 96.63
Current children cumulated vsize (KiB) 344672

[startup+25.5007 s]
/proc/loadavg: 8.12 7.84 6.74 9/264 26500
/proc/meminfo: memFree=22853340/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=358312 CPUtime=198.92 cores=0-7
/proc/26485/stat : 26485 (plingeling) S 26483 26485 26398 0 -1 4202496 90966 1261 0 0 19880 12 0 0 18 0 9 0 655302951 366911488 52654 33554432000 4194304 4417452 140736607358672 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/26485/statm: 89578 52654 180 55 0 87464 0
[pid=26485/tid=26492] ppid=26483 vsize=358312 CPUtime=24.82 cores=0-7
/proc/26485/task/26492/stat : 26492 (plingeling) R 26483 26485 26398 0 -1 4202560 4977 1261 0 0 2482 0 0 0 25 0 9 0 655303017 366911488 52654 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26485/tid=26493] ppid=26483 vsize=358312 CPUtime=24.75 cores=0-7
/proc/26485/task/26493/stat : 26493 (plingeling) R 26483 26485 26398 0 -1 4202560 5800 1261 0 0 2474 1 0 0 25 0 9 0 655303017 366911488 52654 33554432000 4194304 4417452 140736607358672 18446744073709551615 4366469 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26485/tid=26494] ppid=26483 vsize=358312 CPUtime=24.76 cores=0-7
/proc/26485/task/26494/stat : 26494 (plingeling) R 26483 26485 26398 0 -1 4202560 6299 1261 0 0 2476 0 0 0 25 0 9 0 655303017 366911488 52654 33554432000 4194304 4417452 140736607358672 18446744073709551615 4352496 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26485/tid=26495] ppid=26483 vsize=358312 CPUtime=24.82 cores=0-7
/proc/26485/task/26495/stat : 26495 (plingeling) R 26483 26485 26398 0 -1 4202560 2723 1261 0 0 2482 0 0 0 25 0 9 0 655303017 366911488 52654 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26485/tid=26496] ppid=26483 vsize=358312 CPUtime=24.82 cores=0-7
/proc/26485/task/26496/stat : 26496 (plingeling) R 26483 26485 26398 0 -1 4202560 8881 1261 0 0 2481 1 0 0 25 0 9 0 655303017 366911488 52654 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26485/tid=26497] ppid=26483 vsize=358312 CPUtime=24.79 cores=0-7
/proc/26485/task/26497/stat : 26497 (plingeling) R 26483 26485 26398 0 -1 4202560 5649 1261 0 0 2478 1 0 0 25 0 9 0 655303017 366911488 52654 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26485/tid=26498] ppid=26483 vsize=358312 CPUtime=24.82 cores=0-7
/proc/26485/task/26498/stat : 26498 (plingeling) R 26483 26485 26398 0 -1 4202560 8155 1261 0 0 2481 1 0 0 25 0 9 0 655303017 366911488 52654 33554432000 4194304 4417452 140736607358672 18446744073709551615 4365999 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26485/tid=26499] ppid=26483 vsize=358312 CPUtime=24.61 cores=0-7
/proc/26485/task/26499/stat : 26499 (plingeling) R 26483 26485 26398 0 -1 4202560 12544 1261 0 0 2460 1 0 0 25 0 9 0 655303017 366911488 52654 33554432000 4194304 4417452 140736607358672 18446744073709551615 4365150 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 198.92
Current children cumulated vsize (KiB) 358312

[startup+51.101 s]
/proc/loadavg: 8.15 7.86 6.77 9/264 26500
/proc/meminfo: memFree=22811604/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=399972 CPUtime=403.51 cores=0-7
/proc/26485/stat : 26485 (plingeling) S 26483 26485 26398 0 -1 4202496 104708 1261 0 0 40337 14 0 0 18 0 9 0 655302951 409571328 63069 33554432000 4194304 4417452 140736607358672 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/26485/statm: 99993 63069 180 55 0 97879 0
[pid=26485/tid=26492] ppid=26483 vsize=399972 CPUtime=50.42 cores=0-7
/proc/26485/task/26492/stat : 26492 (plingeling) R 26483 26485 26398 0 -1 4202560 5001 1261 0 0 5042 0 0 0 25 0 9 0 655303017 409571328 63069 33554432000 4194304 4417452 140736607358672 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26485/tid=26493] ppid=26483 vsize=399972 CPUtime=50.35 cores=0-7
/proc/26485/task/26493/stat : 26493 (plingeling) R 26483 26485 26398 0 -1 4202560 7490 1261 0 0 5034 1 0 0 25 0 9 0 655303017 409571328 63069 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26485/tid=26494] ppid=26483 vsize=399972 CPUtime=50.36 cores=0-7

################
# More data... #
################

/proc/26485/task/26495/stat : 26495 (plingeling) R 26483 26485 26398 0 -1 4202560 331997 1261 0 0 472117 59 0 0 25 0 9 0 655303017 668614656 126836 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26485/tid=26496] ppid=26483 vsize=652944 CPUtime=4721.11 cores=0-7
/proc/26485/task/26496/stat : 26496 (plingeling) R 26483 26485 26398 0 -1 4202560 399848 1261 0 0 472029 82 0 0 25 0 9 0 655303017 668614656 126836 33554432000 4194304 4417452 140736607358672 18446744073709551615 4352402 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26485/tid=26497] ppid=26483 vsize=652944 CPUtime=4721.72 cores=0-7
/proc/26485/task/26497/stat : 26497 (plingeling) R 26483 26485 26398 0 -1 4202560 384853 1261 0 0 472094 78 0 0 25 0 9 0 655303017 668614656 126836 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364597 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26485/tid=26498] ppid=26483 vsize=652944 CPUtime=4721.77 cores=0-7
/proc/26485/task/26498/stat : 26498 (plingeling) R 26483 26485 26398 0 -1 4202560 382203 1261 0 0 472101 76 0 0 25 0 9 0 655303017 668614656 126836 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26485/tid=26499] ppid=26483 vsize=652944 CPUtime=4683.14 cores=0-7
/proc/26485/task/26499/stat : 26499 (plingeling) R 26483 26485 26398 0 -1 4202560 383466 1261 0 0 468241 73 0 0 25 0 9 0 655303017 668614656 126836 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 37734.4
Current children cumulated vsize (KiB) 652944

[startup+4782.3 s]
/proc/loadavg: 8.11 8.11 8.09 10/264 26638
/proc/meminfo: memFree=22543524/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=665900 CPUtime=38213.9 cores=0-7
/proc/26485/stat : 26485 (plingeling) S 26483 26485 26398 0 -1 4202496 3162379 1261 0 0 3820775 614 0 0 18 0 9 0 655302951 681881600 130075 33554432000 4194304 4417452 140736607358672 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/26485/statm: 166475 130075 180 55 0 164361 0
[pid=26485/tid=26492] ppid=26483 vsize=665900 CPUtime=4781.76 cores=0-7
/proc/26485/task/26492/stat : 26492 (plingeling) R 26483 26485 26398 0 -1 4202560 366083 1261 0 0 478106 70 0 0 25 0 9 0 655303017 681881600 130075 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26485/tid=26493] ppid=26483 vsize=665900 CPUtime=4781.55 cores=0-7
/proc/26485/task/26493/stat : 26493 (plingeling) R 26483 26485 26398 0 -1 4202560 414258 1261 0 0 478071 84 0 0 25 0 9 0 655303017 681881600 130075 33554432000 4194304 4417452 140736607358672 18446744073709551615 4236816 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26485/tid=26494] ppid=26483 vsize=665900 CPUtime=4780.87 cores=0-7
/proc/26485/task/26494/stat : 26494 (plingeling) R 26483 26485 26398 0 -1 4202560 449156 1261 0 0 478005 82 0 0 25 0 9 0 655303017 681881600 130075 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364752 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26485/tid=26495] ppid=26483 vsize=665900 CPUtime=4781.75 cores=0-7
/proc/26485/task/26495/stat : 26495 (plingeling) R 26483 26485 26398 0 -1 4202560 333919 1261 0 0 478116 59 0 0 25 0 9 0 655303017 681881600 130075 33554432000 4194304 4417452 140736607358672 18446744073709551615 4250476 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26485/tid=26496] ppid=26483 vsize=665900 CPUtime=4781.1 cores=0-7
/proc/26485/task/26496/stat : 26496 (plingeling) R 26483 26485 26398 0 -1 4202560 402077 1261 0 0 478028 82 0 0 25 0 9 0 655303017 681881600 130075 33554432000 4194304 4417452 140736607358672 18446744073709551615 4356995 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26485/tid=26497] ppid=26483 vsize=665900 CPUtime=4781.71 cores=0-7
/proc/26485/task/26497/stat : 26497 (plingeling) R 26483 26485 26398 0 -1 4202560 384853 1261 0 0 478093 78 0 0 25 0 9 0 655303017 681881600 130075 33554432000 4194304 4417452 140736607358672 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26485/tid=26498] ppid=26483 vsize=665900 CPUtime=4781.76 cores=0-7
/proc/26485/task/26498/stat : 26498 (plingeling) R 26483 26485 26398 0 -1 4202560 384075 1261 0 0 478100 76 0 0 25 0 9 0 655303017 681881600 130075 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26485/tid=26499] ppid=26483 vsize=665900 CPUtime=4742.64 cores=0-7
/proc/26485/task/26499/stat : 26499 (plingeling) R 26483 26485 26398 0 -1 4202560 392020 1261 0 0 474190 74 0 0 25 0 9 0 655303017 681881600 130075 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38213.9
Current children cumulated vsize (KiB) 665900

[startup+4842.31 s]
/proc/loadavg: 8.31 8.17 8.11 9/264 26640
/proc/meminfo: memFree=22465032/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=743972 CPUtime=38693.5 cores=0-7
/proc/26485/stat : 26485 (plingeling) S 26483 26485 26398 0 -1 4202496 3210373 1261 0 0 3868724 623 0 0 18 0 9 0 655302951 761827328 149593 33554432000 4194304 4417452 140736607358672 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/26485/statm: 185993 149593 180 55 0 183879 0
[pid=26485/tid=26492] ppid=26483 vsize=743972 CPUtime=4841.77 cores=0-7
/proc/26485/task/26492/stat : 26492 (plingeling) R 26483 26485 26398 0 -1 4202560 378683 1261 0 0 484106 71 0 0 25 0 9 0 655303017 761827328 149593 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364440 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26485/tid=26493] ppid=26483 vsize=743972 CPUtime=4841.56 cores=0-7
/proc/26485/task/26493/stat : 26493 (plingeling) R 26483 26485 26398 0 -1 4202560 422155 1261 0 0 484071 85 0 0 25 0 9 0 655303017 761827328 149593 33554432000 4194304 4417452 140736607358672 18446744073709551615 4344151 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26485/tid=26494] ppid=26483 vsize=743972 CPUtime=4840.87 cores=0-7
/proc/26485/task/26494/stat : 26494 (plingeling) R 26483 26485 26398 0 -1 4202560 455277 1261 0 0 484004 83 0 0 25 0 9 0 655303017 761827328 149593 33554432000 4194304 4417452 140736607358672 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26485/tid=26495] ppid=26483 vsize=743972 CPUtime=4841.76 cores=0-7
/proc/26485/task/26495/stat : 26495 (plingeling) R 26483 26485 26398 0 -1 4202560 341255 1261 0 0 484115 61 0 0 25 0 9 0 655303017 761827328 149593 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26485/tid=26496] ppid=26483 vsize=743972 CPUtime=4841.12 cores=0-7
/proc/26485/task/26496/stat : 26496 (plingeling) R 26483 26485 26398 0 -1 4202560 402936 1261 0 0 484029 83 0 0 25 0 9 0 655303017 761827328 149593 33554432000 4194304 4417452 140736607358672 18446744073709551615 4354159 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26485/tid=26497] ppid=26483 vsize=743972 CPUtime=4841.73 cores=0-7
/proc/26485/task/26497/stat : 26497 (plingeling) R 26483 26485 26398 0 -1 4202560 385550 1261 0 0 484094 79 0 0 25 0 9 0 655303017 761827328 149593 33554432000 4194304 4417452 140736607358672 18446744073709551615 4366407 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26485/tid=26498] ppid=26483 vsize=743972 CPUtime=4841.78 cores=0-7
/proc/26485/task/26498/stat : 26498 (plingeling) R 26483 26485 26398 0 -1 4202560 391548 1261 0 0 484099 79 0 0 25 0 9 0 655303017 761827328 149593 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26485/tid=26499] ppid=26483 vsize=743972 CPUtime=4802.16 cores=0-7
/proc/26485/task/26499/stat : 26499 (plingeling) R 26483 26485 26398 0 -1 4202560 397031 1261 0 0 480141 75 0 0 25 0 9 0 655303017 761827328 149593 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38693.5
Current children cumulated vsize (KiB) 743972

[startup+4902.3 s]
/proc/loadavg: 8.16 8.15 8.10 9/264 26642
/proc/meminfo: memFree=22425228/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=758628 CPUtime=39172.9 cores=0-7
/proc/26485/stat : 26485 (plingeling) S 26483 26485 26398 0 -1 4202496 3250998 1261 0 0 3916660 631 0 0 18 0 9 0 655302951 776835072 153257 33554432000 4194304 4417452 140736607358672 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/26485/statm: 189657 153257 180 55 0 187543 0
[pid=26485/tid=26492] ppid=26483 vsize=758628 CPUtime=4901.76 cores=0-7
/proc/26485/task/26492/stat : 26492 (plingeling) R 26483 26485 26398 0 -1 4202560 387333 1261 0 0 490104 72 0 0 25 0 9 0 655303017 776835072 153257 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364772 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26485/tid=26493] ppid=26483 vsize=758628 CPUtime=4901.56 cores=0-7
/proc/26485/task/26493/stat : 26493 (plingeling) R 26483 26485 26398 0 -1 4202560 432746 1261 0 0 490067 89 0 0 25 0 9 0 655303017 776835072 153257 33554432000 4194304 4417452 140736607358672 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26485/tid=26494] ppid=26483 vsize=758628 CPUtime=4900.85 cores=0-7
/proc/26485/task/26494/stat : 26494 (plingeling) R 26483 26485 26398 0 -1 4202560 455277 1261 0 0 490002 83 0 0 25 0 9 0 655303017 776835072 153257 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26485/tid=26495] ppid=26483 vsize=758628 CPUtime=4901.75 cores=0-7
/proc/26485/task/26495/stat : 26495 (plingeling) R 26483 26485 26398 0 -1 4202560 344590 1261 0 0 490114 61 0 0 25 0 9 0 655303017 776835072 153257 33554432000 4194304 4417452 140736607358672 18446744073709551615 4365949 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26485/tid=26496] ppid=26483 vsize=758628 CPUtime=4901.11 cores=0-7
/proc/26485/task/26496/stat : 26496 (plingeling) R 26483 26485 26398 0 -1 4202560 406677 1261 0 0 490028 83 0 0 25 0 9 0 655303017 776835072 153257 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26485/tid=26497] ppid=26483 vsize=758628 CPUtime=4901.72 cores=0-7
/proc/26485/task/26497/stat : 26497 (plingeling) R 26483 26485 26398 0 -1 4202560 386451 1261 0 0 490093 79 0 0 25 0 9 0 655303017 776835072 153257 33554432000 4194304 4417452 140736607358672 18446744073709551615 4353830 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26485/tid=26498] ppid=26483 vsize=758628 CPUtime=4901.77 cores=0-7
/proc/26485/task/26498/stat : 26498 (plingeling) R 26483 26485 26398 0 -1 4202560 400113 1261 0 0 490097 80 0 0 25 0 9 0 655303017 776835072 153257 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26485/tid=26499] ppid=26483 vsize=758628 CPUtime=4861.67 cores=0-7
/proc/26485/task/26499/stat : 26499 (plingeling) R 26483 26485 26398 0 -1 4202560 401873 1261 0 0 486091 76 0 0 25 0 9 0 655303017 776835072 153257 33554432000 4194304 4417452 140736607358672 18446744073709551615 4365952 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39172.9
Current children cumulated vsize (KiB) 758628

[startup+4962.3 s]
/proc/loadavg: 8.17 8.15 8.10 9/264 26643
/proc/meminfo: memFree=22498748/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=710432 CPUtime=39652.4 cores=0-7
/proc/26485/stat : 26485 (plingeling) S 26483 26485 26398 0 -1 4202496 3273777 1261 0 0 3964608 635 0 0 18 0 9 0 655302951 727482368 141208 33554432000 4194304 4417452 140736607358672 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/26485/statm: 177608 141208 180 55 0 175494 0
[pid=26485/tid=26492] ppid=26483 vsize=710432 CPUtime=4961.76 cores=0-7
/proc/26485/task/26492/stat : 26492 (plingeling) R 26483 26485 26398 0 -1 4202560 392143 1261 0 0 496103 73 0 0 25 0 9 0 655303017 727482368 141208 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26485/tid=26493] ppid=26483 vsize=710432 CPUtime=4961.55 cores=0-7
/proc/26485/task/26493/stat : 26493 (plingeling) R 26483 26485 26398 0 -1 4202560 435744 1261 0 0 496066 89 0 0 25 0 9 0 655303017 727482368 141208 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26485/tid=26494] ppid=26483 vsize=710432 CPUtime=4960.85 cores=0-7
/proc/26485/task/26494/stat : 26494 (plingeling) R 26483 26485 26398 0 -1 4202560 455277 1261 0 0 496002 83 0 0 25 0 9 0 655303017 727482368 141208 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26485/tid=26495] ppid=26483 vsize=710432 CPUtime=4961.76 cores=0-7
/proc/26485/task/26495/stat : 26495 (plingeling) R 26483 26485 26398 0 -1 4202560 347680 1261 0 0 496114 62 0 0 25 0 9 0 655303017 727482368 141208 33554432000 4194304 4417452 140736607358672 18446744073709551615 4236874 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26485/tid=26496] ppid=26483 vsize=710432 CPUtime=4961.1 cores=0-7
/proc/26485/task/26496/stat : 26496 (plingeling) R 26483 26485 26398 0 -1 4202560 408247 1261 0 0 496027 83 0 0 25 0 9 0 655303017 727482368 141208 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26485/tid=26497] ppid=26483 vsize=710432 CPUtime=4961.72 cores=0-7
/proc/26485/task/26497/stat : 26497 (plingeling) R 26483 26485 26398 0 -1 4202560 387722 1261 0 0 496093 79 0 0 25 0 9 0 655303017 727482368 141208 33554432000 4194304 4417452 140736607358672 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26485/tid=26498] ppid=26483 vsize=710432 CPUtime=4961.77 cores=0-7
/proc/26485/task/26498/stat : 26498 (plingeling) R 26483 26485 26398 0 -1 4202560 409153 1261 0 0 496096 81 0 0 25 0 9 0 655303017 727482368 141208 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364861 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26485/tid=26499] ppid=26483 vsize=710432 CPUtime=4921.17 cores=0-7
/proc/26485/task/26499/stat : 26499 (plingeling) R 26483 26485 26398 0 -1 4202560 401873 1261 0 0 492041 76 0 0 25 0 9 0 655303017 727482368 141208 33554432000 4194304 4417452 140736607358672 18446744073709551615 4353830 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39652.4
Current children cumulated vsize (KiB) 710432



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5005.8 s]
/proc/loadavg: 8.14 8.14 8.10 9/264 26644
/proc/meminfo: memFree=22480640/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=728416 CPUtime=40000.1 cores=0-7
/proc/26485/stat : 26485 (plingeling) S 26483 26485 26398 0 -1 4202496 3295985 1261 0 0 3999374 639 0 0 18 0 9 0 655302951 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/26485/statm: 182104 145704 180 55 0 179990 0
[pid=26485/tid=26492] ppid=26483 vsize=728416 CPUtime=5005.27 cores=0-7
/proc/26485/task/26492/stat : 26492 (plingeling) R 26483 26485 26398 0 -1 4202560 392998 1261 0 0 500454 73 0 0 25 0 9 0 655303017 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 4236816 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26485/tid=26493] ppid=26483 vsize=728416 CPUtime=5005.07 cores=0-7
/proc/26485/task/26493/stat : 26493 (plingeling) R 26483 26485 26398 0 -1 4202560 439210 1261 0 0 500417 90 0 0 25 0 9 0 655303017 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26485/tid=26494] ppid=26483 vsize=728416 CPUtime=5004.36 cores=0-7
/proc/26485/task/26494/stat : 26494 (plingeling) R 26483 26485 26398 0 -1 4202560 458719 1261 0 0 500352 84 0 0 25 0 9 0 655303017 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26485/tid=26495] ppid=26483 vsize=728416 CPUtime=5005.26 cores=0-7
/proc/26485/task/26495/stat : 26495 (plingeling) R 26483 26485 26398 0 -1 4202560 350985 1261 0 0 500464 62 0 0 25 0 9 0 655303017 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26485/tid=26496] ppid=26483 vsize=728416 CPUtime=5004.62 cores=0-7
/proc/26485/task/26496/stat : 26496 (plingeling) R 26483 26485 26398 0 -1 4202560 408247 1261 0 0 500379 83 0 0 25 0 9 0 655303017 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26485/tid=26497] ppid=26483 vsize=728416 CPUtime=5005.23 cores=0-7
/proc/26485/task/26497/stat : 26497 (plingeling) R 26483 26485 26398 0 -1 4202560 388558 1261 0 0 500444 79 0 0 25 0 9 0 655303017 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 4353559 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26485/tid=26498] ppid=26483 vsize=728416 CPUtime=5005.28 cores=0-7
/proc/26485/task/26498/stat : 26498 (plingeling) R 26483 26485 26398 0 -1 4202560 409153 1261 0 0 500447 81 0 0 25 0 9 0 655303017 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26485/tid=26499] ppid=26483 vsize=728416 CPUtime=4964.32 cores=0-7
/proc/26485/task/26499/stat : 26499 (plingeling) R 26483 26485 26398 0 -1 4202560 412177 1261 0 0 496353 79 0 0 25 0 9 0 655303017 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 4252389 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 728416

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5005.8 s]
/proc/loadavg: 8.14 8.14 8.10 9/264 26644
/proc/meminfo: memFree=22480640/32951124 swapFree=61565116/67111528
[pid=26485] ppid=26483 vsize=728416 CPUtime=40000.1 cores=0-7
/proc/26485/stat : 26485 (plingeling) S 26483 26485 26398 0 -1 4202496 3295985 1261 0 0 3999374 639 0 0 18 0 9 0 655302951 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/26485/statm: 182104 145704 180 55 0 179990 0
[pid=26485/tid=26492] ppid=26483 vsize=728416 CPUtime=5005.27 cores=0-7
/proc/26485/task/26492/stat : 26492 (plingeling) R 26483 26485 26398 0 -1 4202560 392998 1261 0 0 500454 73 0 0 25 0 9 0 655303017 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 4236816 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26485/tid=26493] ppid=26483 vsize=728416 CPUtime=5005.07 cores=0-7
/proc/26485/task/26493/stat : 26493 (plingeling) R 26483 26485 26398 0 -1 4202560 439210 1261 0 0 500417 90 0 0 25 0 9 0 655303017 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26485/tid=26494] ppid=26483 vsize=728416 CPUtime=5004.36 cores=0-7
/proc/26485/task/26494/stat : 26494 (plingeling) R 26483 26485 26398 0 -1 4202560 458719 1261 0 0 500352 84 0 0 25 0 9 0 655303017 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26485/tid=26495] ppid=26483 vsize=728416 CPUtime=5005.26 cores=0-7
/proc/26485/task/26495/stat : 26495 (plingeling) R 26483 26485 26398 0 -1 4202560 350985 1261 0 0 500464 62 0 0 25 0 9 0 655303017 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26485/tid=26496] ppid=26483 vsize=728416 CPUtime=5004.62 cores=0-7
/proc/26485/task/26496/stat : 26496 (plingeling) R 26483 26485 26398 0 -1 4202560 408247 1261 0 0 500379 83 0 0 25 0 9 0 655303017 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26485/tid=26497] ppid=26483 vsize=728416 CPUtime=5005.23 cores=0-7
/proc/26485/task/26497/stat : 26497 (plingeling) R 26483 26485 26398 0 -1 4202560 388558 1261 0 0 500444 79 0 0 25 0 9 0 655303017 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 4353559 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26485/tid=26498] ppid=26483 vsize=728416 CPUtime=5005.28 cores=0-7
/proc/26485/task/26498/stat : 26498 (plingeling) R 26483 26485 26398 0 -1 4202560 409153 1261 0 0 500447 81 0 0 25 0 9 0 655303017 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26485/tid=26499] ppid=26483 vsize=728416 CPUtime=4964.32 cores=0-7
/proc/26485/task/26499/stat : 26499 (plingeling) R 26483 26485 26398 0 -1 4202560 412177 1261 0 0 496353 79 0 0 25 0 9 0 655303017 745897984 145704 33554432000 4194304 4417452 140736607358672 18446744073709551615 4252389 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 728416

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5005.84
CPU time (s): 40000.2
CPU user time (s): 39993.8
CPU system time (s): 6.42702
CPU usage (%): 799.071
Max. virtual memory (cumulated for all children) (KiB): 811548

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39993.8
system time used= 6.42702
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3297246
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 243
involuntary context switches= 347754

runsolver used 11.4633 second user time and 29.5005 second system time

The end

Launcher Data 

Begin job on node134 at 2011-05-15 07:24:08
IDJOB=3372260
IDBENCH=70830
IDSOLVER=1851
FILE ID=node134/3372260-1305437047
RUNJOBID= node134-1305436261-26412
PBS_JOBID= 13324747
Free space on /tmp= 73504 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT09/APPLICATIONS/crypto/md5gen/gus-md5-11.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372260-1305437047/watcher-3372260-1305437047 -o /tmp/evaluation-result-3372260-1305437047/solver-3372260-1305437047 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3372260-1305437047.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 49867705b76fc90285dd6fc48de2613c
RANDOM SEED=720736704

node134.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 7186.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23062568 kB
Buffers:        268564 kB
Cached:        6071024 kB
SwapCached:      13768 kB
Active:         949080 kB
Inactive:      8722892 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23062568 kB
SwapTotal:    67111528 kB
SwapFree:     61565116 kB
Dirty:            4264 kB
Writeback:           0 kB
AnonPages:     3326248 kB
Mapped:          11852 kB
Slab:           122796 kB
PageTables:      30716 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 38873152 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73504 MiB
End job on node134 at 2011-05-15 08:47:35