Trace number 3309807

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 40000.5 5004.09

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-top24.cnf
MD5SUM1f485d737caaaca49e2c467c83cbdcec
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark585.525
Satisfiable
(Un)Satisfiability was proved
Number of variables65983
Number of clauses209942
Sum of the clauses size513194
Maximum clause length2578
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 2127592
Number of clauses of size 380970
Number of clauses of size 4299
Number of clauses of size 522
Number of clauses of size over 51058

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3309807-1304009299/watcher-3309807-1304009299 -o /tmp/evaluation-result-3309807-1304009299/solver-3309807-1304009299 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3309807-1304009299.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.42 7.88 7.93 2/170 29168
/proc/meminfo: memFree=23728176/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=15664 CPUtime=0 cores=0-7
/proc/29168/stat : 29168 (plingeling) R 29166 29168 29142 0 -1 4202496 1718 1253 0 0 0 0 0 0 25 0 1 0 591160562 16039936 1648 33554432000 4194304 4417332 140735722445936 18446744073709551615 260905088093 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/29168/statm: 3916 1660 140 55 0 1802 0

[startup+0.078666 s]
/proc/loadavg: 7.42 7.88 7.93 2/170 29168
/proc/meminfo: memFree=23728176/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=38860 CPUtime=0.06 cores=0-7
/proc/29168/stat : 29168 (plingeling) R 29166 29168 29142 0 -1 4202496 7776 1253 0 0 6 0 0 0 25 0 1 0 591160562 39792640 7706 33554432000 4194304 4417332 140735722445936 18446744073709551615 4231477 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/29168/statm: 9715 7706 148 55 0 7601 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 38860

[startup+0.100664 s]
/proc/loadavg: 7.42 7.88 7.93 2/170 29168
/proc/meminfo: memFree=23728176/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=50024 CPUtime=0.09 cores=0-7
/proc/29168/stat : 29168 (plingeling) R 29166 29168 29142 0 -1 4202496 10599 1253 0 0 8 1 0 0 25 0 1 0 591160562 51224576 10529 33554432000 4194304 4417332 140735722445936 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/29168/statm: 12506 10529 148 55 0 10392 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 50024

[startup+0.300626 s]
/proc/loadavg: 7.42 7.88 7.93 2/170 29168
/proc/meminfo: memFree=23728176/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=73220 CPUtime=0.28 cores=0-7
/proc/29168/stat : 29168 (plingeling) R 29166 29168 29142 0 -1 4202496 16398 1253 0 0 27 1 0 0 25 0 1 0 591160562 74977280 16328 33554432000 4194304 4417332 140735722445936 18446744073709551615 4264994 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/29168/statm: 18305 16328 148 55 0 16191 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 73220

[startup+0.700557 s]
/proc/loadavg: 7.42 7.88 7.93 2/170 29168
/proc/meminfo: memFree=23728176/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=243852 CPUtime=2.07 cores=0-7
/proc/29168/stat : 29168 (plingeling) S 29166 29168 29142 0 -1 4202496 47808 1253 0 0 201 6 0 0 25 0 9 0 591160562 249704448 27440 33554432000 4194304 4417332 140735722445936 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29168/statm: 60963 27440 173 55 0 58849 0
Current children cumulated CPU time (s) 2.07
Current children cumulated vsize (KiB) 243852

[startup+1.50041 s]
/proc/loadavg: 7.42 7.88 7.93 9/179 29182
/proc/meminfo: memFree=23615768/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=239648 CPUtime=8.45 cores=0-7
/proc/29168/stat : 29168 (plingeling) S 29166 29168 29142 0 -1 4202496 58551 1253 0 0 838 7 0 0 25 0 9 0 591160562 245399552 27009 33554432000 4194304 4417332 140735722445936 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29168/statm: 59912 27009 175 55 0 57798 0
[pid=29168/tid=29175] ppid=29166 vsize=239648 CPUtime=0.99 cores=0-7
/proc/29168/task/29175/stat : 29175 (plingeling) R 29166 29168 29142 0 -1 4202560 6402 1253 0 0 99 0 0 0 19 0 9 0 591160612 245399552 27009 33554432000 4194304 4417332 140735722445936 18446744073709551615 4270947 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29168/tid=29176] ppid=29166 vsize=239648 CPUtime=0.99 cores=0-7
/proc/29168/task/29176/stat : 29176 (plingeling) R 29166 29168 29142 0 -1 4202560 4190 1253 0 0 99 0 0 0 25 0 9 0 591160612 245399552 27009 33554432000 4194304 4417332 140735722445936 18446744073709551615 4284514 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29168/tid=29177] ppid=29166 vsize=239648 CPUtime=0.99 cores=0-7
/proc/29168/task/29177/stat : 29177 (plingeling) R 29166 29168 29142 0 -1 4202560 3440 1253 0 0 99 0 0 0 25 0 9 0 591160612 245399552 27009 33554432000 4194304 4417332 140735722445936 18446744073709551615 4284595 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29168/tid=29178] ppid=29166 vsize=239648 CPUtime=0.98 cores=0-7
/proc/29168/task/29178/stat : 29178 (plingeling) R 29166 29168 29142 0 -1 4202560 1121 1253 0 0 98 0 0 0 25 0 9 0 591160612 245399552 27009 33554432000 4194304 4417332 140735722445936 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29168/tid=29179] ppid=29166 vsize=239648 CPUtime=0.99 cores=0-7
/proc/29168/task/29179/stat : 29179 (plingeling) R 29166 29168 29142 0 -1 4202560 5164 1253 0 0 99 0 0 0 25 0 9 0 591160612 245399552 27009 33554432000 4194304 4417332 140735722445936 18446744073709551615 4270935 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29168/tid=29180] ppid=29166 vsize=239648 CPUtime=0.99 cores=0-7
/proc/29168/task/29180/stat : 29180 (plingeling) R 29166 29168 29142 0 -1 4202560 4050 1253 0 0 99 0 0 0 25 0 9 0 591160612 245399552 27009 33554432000 4194304 4417332 140735722445936 18446744073709551615 4292084 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29168/tid=29181] ppid=29166 vsize=239648 CPUtime=0.99 cores=0-7
/proc/29168/task/29181/stat : 29181 (plingeling) R 29166 29168 29142 0 -1 4202560 1735 1253 0 0 99 0 0 0 25 0 9 0 591160612 245399552 27009 33554432000 4194304 4417332 140735722445936 18446744073709551615 4281485 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29168/tid=29182] ppid=29166 vsize=239648 CPUtime=0.97 cores=0-7
/proc/29168/task/29182/stat : 29182 (plingeling) R 29166 29168 29142 0 -1 4202560 4542 1253 0 0 97 0 0 0 25 0 9 0 591160612 245399552 27009 33554432000 4194304 4417332 140735722445936 18446744073709551615 4288259 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 8.45
Current children cumulated vsize (KiB) 239648

[startup+3.10112 s]
/proc/loadavg: 7.42 7.88 7.93 9/179 29182
/proc/meminfo: memFree=23612544/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=269580 CPUtime=21.25 cores=0-7
/proc/29168/stat : 29168 (plingeling) S 29166 29168 29142 0 -1 4202496 79225 1253 0 0 2115 10 0 0 25 0 9 0 591160562 276049920 30074 33554432000 4194304 4417332 140735722445936 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29168/statm: 67395 30074 179 55 0 65281 0
[pid=29168/tid=29175] ppid=29166 vsize=269580 CPUtime=2.59 cores=0-7
/proc/29168/task/29175/stat : 29175 (plingeling) R 29166 29168 29142 0 -1 4202560 9589 1253 0 0 258 1 0 0 22 0 9 0 591160612 276049920 30074 33554432000 4194304 4417332 140735722445936 18446744073709551615 4346564 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29168/tid=29176] ppid=29166 vsize=269580 CPUtime=2.59 cores=0-7
/proc/29168/task/29176/stat : 29176 (plingeling) R 29166 29168 29142 0 -1 4202560 9065 1253 0 0 259 0 0 0 25 0 9 0 591160612 276049920 30074 33554432000 4194304 4417332 140735722445936 18446744073709551615 4301995 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29168/tid=29177] ppid=29166 vsize=269580 CPUtime=2.59 cores=0-7
/proc/29168/task/29177/stat : 29177 (plingeling) R 29166 29168 29142 0 -1 4202560 4414 1253 0 0 259 0 0 0 25 0 9 0 591160612 276049920 30074 33554432000 4194304 4417332 140735722445936 18446744073709551615 4233447 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29168/tid=29178] ppid=29166 vsize=269580 CPUtime=2.58 cores=0-7
/proc/29168/task/29178/stat : 29178 (plingeling) R 29166 29168 29142 0 -1 4202560 2515 1253 0 0 258 0 0 0 25 0 9 0 591160612 276049920 30074 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29168/tid=29179] ppid=29166 vsize=269580 CPUtime=2.59 cores=0-7
/proc/29168/task/29179/stat : 29179 (plingeling) R 29166 29168 29142 0 -1 4202560 7771 1253 0 0 258 1 0 0 25 0 9 0 591160612 276049920 30074 33554432000 4194304 4417332 140735722445936 18446744073709551615 4346390 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29168/tid=29180] ppid=29166 vsize=269580 CPUtime=2.58 cores=0-7
/proc/29168/task/29180/stat : 29180 (plingeling) R 29166 29168 29142 0 -1 4202560 8599 1253 0 0 258 0 0 0 25 0 9 0 591160612 276049920 30074 33554432000 4194304 4417332 140735722445936 18446744073709551615 4346278 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29168/tid=29181] ppid=29166 vsize=269580 CPUtime=2.59 cores=0-7
/proc/29168/task/29181/stat : 29181 (plingeling) R 29166 29168 29142 0 -1 4202560 1735 1253 0 0 259 0 0 0 25 0 9 0 591160612 276049920 30074 33554432000 4194304 4417332 140735722445936 18446744073709551615 4330209 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29168/tid=29182] ppid=29166 vsize=269580 CPUtime=2.57 cores=0-7
/proc/29168/task/29182/stat : 29182 (plingeling) R 29166 29168 29142 0 -1 4202560 7630 1253 0 0 256 1 0 0 25 0 9 0 591160612 276049920 30074 33554432000 4194304 4417332 140735722445936 18446744073709551615 4346456 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 21.25
Current children cumulated vsize (KiB) 269580

[startup+6.30053 s]
/proc/loadavg: 7.46 7.88 7.93 9/179 29182
/proc/meminfo: memFree=23548188/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=325392 CPUtime=46.84 cores=0-7
/proc/29168/stat : 29168 (plingeling) S 29166 29168 29142 0 -1 4202496 94293 1253 0 0 4671 13 0 0 25 0 9 0 591160562 333201408 44926 33554432000 4194304 4417332 140735722445936 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29168/statm: 81348 44926 181 55 0 79234 0
[pid=29168/tid=29175] ppid=29166 vsize=325392 CPUtime=5.79 cores=0-7
/proc/29168/task/29175/stat : 29175 (plingeling) R 29166 29168 29142 0 -1 4202560 11497 1253 0 0 578 1 0 0 25 0 9 0 591160612 333201408 44926 33554432000 4194304 4417332 140735722445936 18446744073709551615 4253237 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29168/tid=29176] ppid=29166 vsize=325392 CPUtime=5.79 cores=0-7
/proc/29168/task/29176/stat : 29176 (plingeling) R 29166 29168 29142 0 -1 4202560 11707 1253 0 0 578 1 0 0 25 0 9 0 591160612 333201408 44926 33554432000 4194304 4417332 140735722445936 18446744073709551615 4239395 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29168/tid=29177] ppid=29166 vsize=325392 CPUtime=5.79 cores=0-7
/proc/29168/task/29177/stat : 29177 (plingeling) R 29166 29168 29142 0 -1 4202560 6520 1253 0 0 579 0 0 0 25 0 9 0 591160612 333201408 44926 33554432000 4194304 4417332 140735722445936 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29168/tid=29178] ppid=29166 vsize=325392 CPUtime=5.76 cores=0-7
/proc/29168/task/29178/stat : 29178 (plingeling) R 29166 29168 29142 0 -1 4202560 2515 1253 0 0 576 0 0 0 25 0 9 0 591160612 333201408 44926 33554432000 4194304 4417332 140735722445936 18446744073709551615 4237298 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29168/tid=29179] ppid=29166 vsize=325392 CPUtime=5.79 cores=0-7
/proc/29168/task/29179/stat : 29179 (plingeling) R 29166 29168 29142 0 -1 4202560 9954 1253 0 0 578 1 0 0 25 0 9 0 591160612 333201408 44926 33554432000 4194304 4417332 140735722445936 18446744073709551615 4252951 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29168/tid=29180] ppid=29166 vsize=325392 CPUtime=5.79 cores=0-7
/proc/29168/task/29180/stat : 29180 (plingeling) R 29166 29168 29142 0 -1 4202560 11674 1253 0 0 578 1 0 0 25 0 9 0 591160612 333201408 44926 33554432000 4194304 4417332 140735722445936 18446744073709551615 4271682 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29168/tid=29181] ppid=29166 vsize=325392 CPUtime=5.79 cores=0-7
/proc/29168/task/29181/stat : 29181 (plingeling) R 29166 29168 29142 0 -1 4202560 3271 1253 0 0 579 0 0 0 25 0 9 0 591160612 333201408 44926 33554432000 4194304 4417332 140735722445936 18446744073709551615 4377212 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29168/tid=29182] ppid=29166 vsize=325392 CPUtime=5.77 cores=0-7
/proc/29168/task/29182/stat : 29182 (plingeling) R 29166 29168 29142 0 -1 4202560 9248 1253 0 0 576 1 0 0 25 0 9 0 591160612 333201408 44926 33554432000 4194304 4417332 140735722445936 18446744073709551615 4253228 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 46.84
Current children cumulated vsize (KiB) 325392

[startup+12.7004 s]
/proc/loadavg: 7.50 7.89 7.93 9/179 29182
/proc/meminfo: memFree=23560976/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=306088 CPUtime=98 cores=0-7
/proc/29168/stat : 29168 (plingeling) S 29166 29168 29142 0 -1 4202496 94756 1253 0 0 9787 13 0 0 25 0 9 0 591160562 313434112 39748 33554432000 4194304 4417332 140735722445936 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29168/statm: 76522 39748 181 55 0 74408 0
[pid=29168/tid=29175] ppid=29166 vsize=306088 CPUtime=12.19 cores=0-7
/proc/29168/task/29175/stat : 29175 (plingeling) R 29166 29168 29142 0 -1 4202560 11497 1253 0 0 1218 1 0 0 25 0 9 0 591160612 313434112 39748 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29168/tid=29176] ppid=29166 vsize=306088 CPUtime=12.19 cores=0-7
/proc/29168/task/29176/stat : 29176 (plingeling) R 29166 29168 29142 0 -1 4202560 11707 1253 0 0 1218 1 0 0 25 0 9 0 591160612 313434112 39748 33554432000 4194304 4417332 140735722445936 18446744073709551615 4357173 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29168/tid=29177] ppid=29166 vsize=306088 CPUtime=12.19 cores=0-7
/proc/29168/task/29177/stat : 29177 (plingeling) R 29166 29168 29142 0 -1 4202560 6541 1253 0 0 1219 0 0 0 25 0 9 0 591160612 313434112 39748 33554432000 4194304 4417332 140735722445936 18446744073709551615 4254468 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29168/tid=29178] ppid=29166 vsize=306088 CPUtime=12.13 cores=0-7
/proc/29168/task/29178/stat : 29178 (plingeling) R 29166 29168 29142 0 -1 4202560 2515 1253 0 0 1213 0 0 0 25 0 9 0 591160612 313434112 39748 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29168/tid=29179] ppid=29166 vsize=306088 CPUtime=12.19 cores=0-7
/proc/29168/task/29179/stat : 29179 (plingeling) R 29166 29168 29142 0 -1 4202560 10210 1253 0 0 1218 1 0 0 25 0 9 0 591160612 313434112 39748 33554432000 4194304 4417332 140735722445936 18446744073709551615 4250003 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29168/tid=29180] ppid=29166 vsize=306088 CPUtime=12.19 cores=0-7
/proc/29168/task/29180/stat : 29180 (plingeling) R 29166 29168 29142 0 -1 4202560 11674 1253 0 0 1218 1 0 0 25 0 9 0 591160612 313434112 39748 33554432000 4194304 4417332 140735722445936 18446744073709551615 4258627 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29168/tid=29181] ppid=29166 vsize=306088 CPUtime=12.19 cores=0-7
/proc/29168/task/29181/stat : 29181 (plingeling) R 29166 29168 29142 0 -1 4202560 3457 1253 0 0 1219 0 0 0 25 0 9 0 591160612 313434112 39748 33554432000 4194304 4417332 140735722445936 18446744073709551615 4236224 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29168/tid=29182] ppid=29166 vsize=306088 CPUtime=12.17 cores=0-7
/proc/29168/task/29182/stat : 29182 (plingeling) R 29166 29168 29142 0 -1 4202560 9248 1253 0 0 1216 1 0 0 25 0 9 0 591160612 313434112 39748 33554432000 4194304 4417332 140735722445936 18446744073709551615 4366066 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 98
Current children cumulated vsize (KiB) 306088

[startup+25.501 s]
/proc/loadavg: 7.61 7.89 7.93 9/179 29183
/proc/meminfo: memFree=23577796/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=297588 CPUtime=200.34 cores=0-7
/proc/29168/stat : 29168 (plingeling) S 29166 29168 29142 0 -1 4202496 94906 1253 0 0 20021 13 0 0 25 0 9 0 591160562 304730112 37635 33554432000 4194304 4417332 140735722445936 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29168/statm: 74397 37635 181 55 0 72283 0
[pid=29168/tid=29175] ppid=29166 vsize=297588 CPUtime=24.99 cores=0-7
/proc/29168/task/29175/stat : 29175 (plingeling) R 29166 29168 29142 0 -1 4202560 11518 1253 0 0 2498 1 0 0 25 0 9 0 591160612 304730112 37635 33554432000 4194304 4417332 140735722445936 18446744073709551615 4258491 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29168/tid=29176] ppid=29166 vsize=297588 CPUtime=24.99 cores=0-7
/proc/29168/task/29176/stat : 29176 (plingeling) R 29166 29168 29142 0 -1 4202560 11727 1253 0 0 2498 1 0 0 25 0 9 0 591160612 304730112 37635 33554432000 4194304 4417332 140735722445936 18446744073709551615 4253152 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29168/tid=29177] ppid=29166 vsize=297588 CPUtime=24.99 cores=0-7
/proc/29168/task/29177/stat : 29177 (plingeling) R 29166 29168 29142 0 -1 4202560 6564 1253 0 0 2499 0 0 0 25 0 9 0 591160612 304730112 37635 33554432000 4194304 4417332 140735722445936 18446744073709551615 4257580 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29168/tid=29178] ppid=29166 vsize=297588 CPUtime=24.86 cores=0-7
/proc/29168/task/29178/stat : 29178 (plingeling) R 29166 29168 29142 0 -1 4202560 2515 1253 0 0 2486 0 0 0 25 0 9 0 591160612 304730112 37635 33554432000 4194304 4417332 140735722445936 18446744073709551615 4236722 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29168/tid=29179] ppid=29166 vsize=297588 CPUtime=24.99 cores=0-7
/proc/29168/task/29179/stat : 29179 (plingeling) R 29166 29168 29142 0 -1 4202560 10231 1253 0 0 2498 1 0 0 25 0 9 0 591160612 304730112 37635 33554432000 4194304 4417332 140735722445936 18446744073709551615 4249570 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29168/tid=29180] ppid=29166 vsize=297588 CPUtime=24.99 cores=0-7
/proc/29168/task/29180/stat : 29180 (plingeling) R 29166 29168 29142 0 -1 4202560 11695 1253 0 0 2498 1 0 0 25 0 9 0 591160612 304730112 37635 33554432000 4194304 4417332 140735722445936 18446744073709551615 4239395 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29168/tid=29181] ppid=29166 vsize=297588 CPUtime=24.99 cores=0-7
/proc/29168/task/29181/stat : 29181 (plingeling) R 29166 29168 29142 0 -1 4202560 3480 1253 0 0 2499 0 0 0 25 0 9 0 591160612 304730112 37635 33554432000 4194304 4417332 140735722445936 18446744073709551615 4271867 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29168/tid=29182] ppid=29166 vsize=297588 CPUtime=24.97 cores=0-7
/proc/29168/task/29182/stat : 29182 (plingeling) R 29166 29168 29142 0 -1 4202560 9269 1253 0 0 2496 1 0 0 25 0 9 0 591160612 304730112 37635 33554432000 4194304 4417332 140735722445936 18446744073709551615 4366324 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 200.34
Current children cumulated vsize (KiB) 297588

[startup+51.1013 s]
/proc/loadavg: 7.74 7.90 7.93 9/179 29183
/proc/meminfo: memFree=23533976/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=338668 CPUtime=405.01 cores=0-7
/proc/29168/stat : 29168 (plingeling) S 29166 29168 29142 0 -1 4202496 123751 1253 0 0 40484 17 0 0 25 0 9 0 591160562 346796032 47393 33554432000 4194304 4417332 140735722445936 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29168/statm: 84667 47393 181 55 0 82553 0
[pid=29168/tid=29175] ppid=29166 vsize=338668 CPUtime=50.59 cores=0-7
/proc/29168/task/29175/stat : 29175 (plingeling) R 29166 29168 29142 0 -1 4202560 18520 1253 0 0 5057 2 0 0 25 0 9 0 591160612 346796032 47393 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29168/tid=29176] ppid=29166 vsize=338668 CPUtime=50.6 cores=0-7
/proc/29168/task/29176/stat : 29176 (plingeling) R 29166 29168 29142 0 -1 4202560 16526 1253 0 0 5058 2 0 0 25 0 9 0 591160612 346796032 47393 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29168/tid=29177] ppid=29166 vsize=338668 CPUtime=50.59 cores=0-7

################
# More data... #
################


[startup+4722.31 s]
/proc/loadavg: 8.00 8.01 8.00 9/179 29329
/proc/meminfo: memFree=19386072/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=4486060 CPUtime=37748.9 cores=0-7
/proc/29168/stat : 29168 (plingeling) S 29166 29168 29142 0 -1 4202496 6360761 1253 0 0 3773721 1169 0 0 25 0 9 0 591160562 4593725440 1084241 33554432000 4194304 4417332 140735722445936 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29168/statm: 1121515 1084241 181 55 0 1119401 0
[pid=29168/tid=29175] ppid=29166 vsize=4486060 CPUtime=4721.92 cores=0-7
/proc/29168/task/29175/stat : 29175 (plingeling) R 29166 29168 29142 0 -1 4202560 777941 1253 0 0 472047 145 0 0 25 0 9 0 591160612 4593725440 1084241 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29168/tid=29176] ppid=29166 vsize=4486060 CPUtime=4721.94 cores=0-7
/proc/29168/task/29176/stat : 29176 (plingeling) R 29166 29168 29142 0 -1 4202560 766933 1253 0 0 472049 145 0 0 25 0 9 0 591160612 4593725440 1084241 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29168/tid=29177] ppid=29166 vsize=4486060 CPUtime=4721.94 cores=0-7
/proc/29168/task/29177/stat : 29177 (plingeling) R 29166 29168 29142 0 -1 4202560 794685 1253 0 0 472049 145 0 0 25 0 9 0 591160612 4593725440 1084241 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29168/tid=29178] ppid=29166 vsize=4486060 CPUtime=4696.22 cores=0-7
/proc/29168/task/29178/stat : 29178 (plingeling) R 29166 29168 29142 0 -1 4202560 667259 1253 0 0 469516 106 0 0 25 0 9 0 591160612 4593725440 1084241 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364873 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29168/tid=29179] ppid=29166 vsize=4486060 CPUtime=4721.76 cores=0-7
/proc/29168/task/29179/stat : 29179 (plingeling) R 29166 29168 29142 0 -1 4202560 872025 1253 0 0 472000 176 0 0 25 0 9 0 591160612 4593725440 1084241 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364899 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29168/tid=29180] ppid=29166 vsize=4486060 CPUtime=4720.84 cores=0-7
/proc/29168/task/29180/stat : 29180 (plingeling) R 29166 29168 29142 0 -1 4202560 901154 1253 0 0 471916 168 0 0 25 0 9 0 591160612 4593725440 1084241 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29168/tid=29181] ppid=29166 vsize=4486060 CPUtime=4721.79 cores=0-7
/proc/29168/task/29181/stat : 29181 (plingeling) R 29166 29168 29142 0 -1 4202560 691761 1253 0 0 472050 129 0 0 25 0 9 0 591160612 4593725440 1084241 33554432000 4194304 4417332 140735722445936 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29168/tid=29182] ppid=29166 vsize=4486060 CPUtime=4721.92 cores=0-7
/proc/29168/task/29182/stat : 29182 (plingeling) R 29166 29168 29142 0 -1 4202560 861096 1253 0 0 472044 148 0 0 25 0 9 0 591160612 4593725440 1084241 33554432000 4194304 4417332 140735722445936 18446744073709551615 4366297 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37748.9
Current children cumulated vsize (KiB) 4486060

[startup+4782.3 s]
/proc/loadavg: 8.00 8.01 8.00 9/179 29330
/proc/meminfo: memFree=19689340/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=4182692 CPUtime=38228.6 cores=0-7
/proc/29168/stat : 29168 (plingeling) S 29166 29168 29142 0 -1 4202496 6394050 1253 0 0 3821680 1175 0 0 25 0 9 0 591160562 4283076608 1008399 33554432000 4194304 4417332 140735722445936 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29168/statm: 1045673 1008399 181 55 0 1043559 0
[pid=29168/tid=29175] ppid=29166 vsize=4182692 CPUtime=4781.91 cores=0-7
/proc/29168/task/29175/stat : 29175 (plingeling) R 29166 29168 29142 0 -1 4202560 777941 1253 0 0 478046 145 0 0 25 0 9 0 591160612 4283076608 1008399 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29168/tid=29176] ppid=29166 vsize=4182692 CPUtime=4781.94 cores=0-7
/proc/29168/task/29176/stat : 29176 (plingeling) R 29166 29168 29142 0 -1 4202560 766933 1253 0 0 478049 145 0 0 25 0 9 0 591160612 4283076608 1008399 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29168/tid=29177] ppid=29166 vsize=4182692 CPUtime=4781.94 cores=0-7
/proc/29168/task/29177/stat : 29177 (plingeling) R 29166 29168 29142 0 -1 4202560 822504 1253 0 0 478044 150 0 0 25 0 9 0 591160612 4283076608 1008399 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364671 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29168/tid=29178] ppid=29166 vsize=4182692 CPUtime=4755.9 cores=0-7
/proc/29168/task/29178/stat : 29178 (plingeling) R 29166 29168 29142 0 -1 4202560 672729 1253 0 0 475483 107 0 0 25 0 9 0 591160612 4283076608 1008399 33554432000 4194304 4417332 140735722445936 18446744073709551615 4365018 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29168/tid=29179] ppid=29166 vsize=4182692 CPUtime=4781.76 cores=0-7
/proc/29168/task/29179/stat : 29179 (plingeling) R 29166 29168 29142 0 -1 4202560 872025 1253 0 0 478000 176 0 0 25 0 9 0 591160612 4283076608 1008399 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29168/tid=29180] ppid=29166 vsize=4182692 CPUtime=4780.83 cores=0-7
/proc/29168/task/29180/stat : 29180 (plingeling) R 29166 29168 29142 0 -1 4202560 901154 1253 0 0 477915 168 0 0 25 0 9 0 591160612 4283076608 1008399 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29168/tid=29181] ppid=29166 vsize=4182692 CPUtime=4781.79 cores=0-7
/proc/29168/task/29181/stat : 29181 (plingeling) R 29166 29168 29142 0 -1 4202560 691761 1253 0 0 478050 129 0 0 25 0 9 0 591160612 4283076608 1008399 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29168/tid=29182] ppid=29166 vsize=4182692 CPUtime=4781.92 cores=0-7
/proc/29168/task/29182/stat : 29182 (plingeling) R 29166 29168 29142 0 -1 4202560 861096 1253 0 0 478044 148 0 0 25 0 9 0 591160612 4283076608 1008399 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38228.6
Current children cumulated vsize (KiB) 4182692

[startup+4842.3 s]
/proc/loadavg: 8.04 8.02 8.00 9/179 29332
/proc/meminfo: memFree=19820116/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=4051168 CPUtime=38708.2 cores=0-7
/proc/29168/stat : 29168 (plingeling) S 29166 29168 29142 0 -1 4202496 6405393 1253 0 0 3869645 1177 0 0 25 0 9 0 591160562 4148396032 975518 33554432000 4194304 4417332 140735722445936 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29168/statm: 1012792 975518 181 55 0 1010678 0
[pid=29168/tid=29175] ppid=29166 vsize=4051168 CPUtime=4841.91 cores=0-7
/proc/29168/task/29175/stat : 29175 (plingeling) R 29166 29168 29142 0 -1 4202560 777941 1253 0 0 484046 145 0 0 25 0 9 0 591160612 4148396032 975518 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29168/tid=29176] ppid=29166 vsize=4051168 CPUtime=4841.94 cores=0-7
/proc/29168/task/29176/stat : 29176 (plingeling) R 29166 29168 29142 0 -1 4202560 766933 1253 0 0 484049 145 0 0 25 0 9 0 591160612 4148396032 975518 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29168/tid=29177] ppid=29166 vsize=4051168 CPUtime=4841.94 cores=0-7
/proc/29168/task/29177/stat : 29177 (plingeling) R 29166 29168 29142 0 -1 4202560 822504 1253 0 0 484044 150 0 0 25 0 9 0 591160612 4148396032 975518 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29168/tid=29178] ppid=29166 vsize=4051168 CPUtime=4815.57 cores=0-7
/proc/29168/task/29178/stat : 29178 (plingeling) R 29166 29168 29142 0 -1 4202560 684072 1253 0 0 481449 108 0 0 25 0 9 0 591160612 4148396032 975518 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364370 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29168/tid=29179] ppid=29166 vsize=4051168 CPUtime=4841.76 cores=0-7
/proc/29168/task/29179/stat : 29179 (plingeling) R 29166 29168 29142 0 -1 4202560 872025 1253 0 0 484000 176 0 0 25 0 9 0 591160612 4148396032 975518 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29168/tid=29180] ppid=29166 vsize=4051168 CPUtime=4840.82 cores=0-7
/proc/29168/task/29180/stat : 29180 (plingeling) R 29166 29168 29142 0 -1 4202560 901154 1253 0 0 483914 168 0 0 25 0 9 0 591160612 4148396032 975518 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364544 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29168/tid=29181] ppid=29166 vsize=4051168 CPUtime=4841.79 cores=0-7
/proc/29168/task/29181/stat : 29181 (plingeling) R 29166 29168 29142 0 -1 4202560 691761 1253 0 0 484050 129 0 0 25 0 9 0 591160612 4148396032 975518 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29168/tid=29182] ppid=29166 vsize=4051168 CPUtime=4841.92 cores=0-7
/proc/29168/task/29182/stat : 29182 (plingeling) R 29166 29168 29142 0 -1 4202560 861096 1253 0 0 484044 148 0 0 25 0 9 0 591160612 4148396032 975518 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38708.2
Current children cumulated vsize (KiB) 4051168

[startup+4902.3 s]
/proc/loadavg: 8.05 8.03 8.00 9/179 29334
/proc/meminfo: memFree=19847852/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=4023640 CPUtime=39187.9 cores=0-7
/proc/29168/stat : 29168 (plingeling) S 29166 29168 29142 0 -1 4202496 6410294 1253 0 0 3917613 1178 0 0 25 0 9 0 591160562 4120207360 968636 33554432000 4194304 4417332 140735722445936 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29168/statm: 1005910 968636 181 55 0 1003796 0
[pid=29168/tid=29175] ppid=29166 vsize=4023640 CPUtime=4901.92 cores=0-7
/proc/29168/task/29175/stat : 29175 (plingeling) R 29166 29168 29142 0 -1 4202560 777941 1253 0 0 490047 145 0 0 25 0 9 0 591160612 4120207360 968636 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29168/tid=29176] ppid=29166 vsize=4023640 CPUtime=4901.94 cores=0-7
/proc/29168/task/29176/stat : 29176 (plingeling) R 29166 29168 29142 0 -1 4202560 766933 1253 0 0 490049 145 0 0 25 0 9 0 591160612 4120207360 968636 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29168/tid=29177] ppid=29166 vsize=4023640 CPUtime=4901.94 cores=0-7
/proc/29168/task/29177/stat : 29177 (plingeling) R 29166 29168 29142 0 -1 4202560 822504 1253 0 0 490044 150 0 0 25 0 9 0 591160612 4120207360 968636 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29168/tid=29178] ppid=29166 vsize=4023640 CPUtime=4875.25 cores=0-7
/proc/29168/task/29178/stat : 29178 (plingeling) R 29166 29168 29142 0 -1 4202560 688973 1253 0 0 487416 109 0 0 25 0 9 0 591160612 4120207360 968636 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29168/tid=29179] ppid=29166 vsize=4023640 CPUtime=4901.76 cores=0-7
/proc/29168/task/29179/stat : 29179 (plingeling) R 29166 29168 29142 0 -1 4202560 872025 1253 0 0 490000 176 0 0 25 0 9 0 591160612 4120207360 968636 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29168/tid=29180] ppid=29166 vsize=4023640 CPUtime=4900.82 cores=0-7
/proc/29168/task/29180/stat : 29180 (plingeling) R 29166 29168 29142 0 -1 4202560 901154 1253 0 0 489914 168 0 0 25 0 9 0 591160612 4120207360 968636 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364671 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29168/tid=29181] ppid=29166 vsize=4023640 CPUtime=4901.79 cores=0-7
/proc/29168/task/29181/stat : 29181 (plingeling) R 29166 29168 29142 0 -1 4202560 691761 1253 0 0 490050 129 0 0 25 0 9 0 591160612 4120207360 968636 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29168/tid=29182] ppid=29166 vsize=4023640 CPUtime=4901.92 cores=0-7
/proc/29168/task/29182/stat : 29182 (plingeling) R 29166 29168 29142 0 -1 4202560 861096 1253 0 0 490044 148 0 0 25 0 9 0 591160612 4120207360 968636 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39187.9
Current children cumulated vsize (KiB) 4023640

[startup+4962.3 s]
/proc/loadavg: 8.05 8.03 8.00 9/179 29335
/proc/meminfo: memFree=19743772/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=4127968 CPUtime=39667.6 cores=0-7
/proc/29168/stat : 29168 (plingeling) S 29166 29168 29142 0 -1 4202496 6436375 1253 0 0 3965578 1180 0 0 25 0 9 0 591160562 4227039232 994717 33554432000 4194304 4417332 140735722445936 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29168/statm: 1031992 994717 181 55 0 1029878 0
[pid=29168/tid=29175] ppid=29166 vsize=4127968 CPUtime=4961.92 cores=0-7
/proc/29168/task/29175/stat : 29175 (plingeling) R 29166 29168 29142 0 -1 4202560 777941 1253 0 0 496047 145 0 0 25 0 9 0 591160612 4227039232 994717 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29168/tid=29176] ppid=29166 vsize=4127968 CPUtime=4961.94 cores=0-7
/proc/29168/task/29176/stat : 29176 (plingeling) R 29166 29168 29142 0 -1 4202560 766933 1253 0 0 496049 145 0 0 25 0 9 0 591160612 4227039232 994717 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29168/tid=29177] ppid=29166 vsize=4127968 CPUtime=4961.95 cores=0-7
/proc/29168/task/29177/stat : 29177 (plingeling) R 29166 29168 29142 0 -1 4202560 841050 1253 0 0 496043 152 0 0 25 0 9 0 591160612 4227039232 994717 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29168/tid=29178] ppid=29166 vsize=4127968 CPUtime=4934.92 cores=0-7
/proc/29168/task/29178/stat : 29178 (plingeling) R 29166 29168 29142 0 -1 4202560 696508 1253 0 0 493382 110 0 0 25 0 9 0 591160612 4227039232 994717 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29168/tid=29179] ppid=29166 vsize=4127968 CPUtime=4961.76 cores=0-7
/proc/29168/task/29179/stat : 29179 (plingeling) R 29166 29168 29142 0 -1 4202560 872025 1253 0 0 496000 176 0 0 25 0 9 0 591160612 4227039232 994717 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364942 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29168/tid=29180] ppid=29166 vsize=4127968 CPUtime=4960.82 cores=0-7
/proc/29168/task/29180/stat : 29180 (plingeling) R 29166 29168 29142 0 -1 4202560 901154 1253 0 0 495914 168 0 0 25 0 9 0 591160612 4227039232 994717 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29168/tid=29181] ppid=29166 vsize=4127968 CPUtime=4961.79 cores=0-7
/proc/29168/task/29181/stat : 29181 (plingeling) R 29166 29168 29142 0 -1 4202560 691761 1253 0 0 496050 129 0 0 25 0 9 0 591160612 4227039232 994717 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29168/tid=29182] ppid=29166 vsize=4127968 CPUtime=4961.92 cores=0-7
/proc/29168/task/29182/stat : 29182 (plingeling) R 29166 29168 29142 0 -1 4202560 861096 1253 0 0 496044 148 0 0 25 0 9 0 591160612 4227039232 994717 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39667.6
Current children cumulated vsize (KiB) 4127968



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5003.91 s]
/proc/loadavg: 8.02 8.03 8.00 9/179 29336
/proc/meminfo: memFree=19482716/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=4389084 CPUtime=40000.2 cores=0-7
/proc/29168/stat : 29168 (plingeling) S 29166 29168 29142 0 -1 4202496 6513557 1253 0 0 3998830 1195 0 0 25 0 9 0 591160562 4494422016 1059997 33554432000 4194304 4417332 140735722445936 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/29168/statm: 1097271 1059997 181 55 0 1095157 0
[pid=29168/tid=29175] ppid=29166 vsize=4389084 CPUtime=5003.53 cores=0-7
/proc/29168/task/29175/stat : 29175 (plingeling) R 29166 29168 29142 0 -1 4202560 810709 1253 0 0 500198 155 0 0 25 0 9 0 591160612 4494422016 1059997 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29168/tid=29176] ppid=29166 vsize=4389084 CPUtime=5003.56 cores=0-7
/proc/29168/task/29176/stat : 29176 (plingeling) R 29166 29168 29142 0 -1 4202560 766933 1253 0 0 500211 145 0 0 25 0 9 0 591160612 4494422016 1059997 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29168/tid=29177] ppid=29166 vsize=4389084 CPUtime=5003.56 cores=0-7
/proc/29168/task/29177/stat : 29177 (plingeling) R 29166 29168 29142 0 -1 4202560 848264 1253 0 0 500203 153 0 0 25 0 9 0 591160612 4494422016 1059997 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29168/tid=29178] ppid=29166 vsize=4389084 CPUtime=4976.3 cores=0-7
/proc/29168/task/29178/stat : 29178 (plingeling) R 29166 29168 29142 0 -1 4202560 700940 1253 0 0 497519 111 0 0 25 0 9 0 591160612 4494422016 1059997 33554432000 4194304 4417332 140735722445936 18446744073709551615 4236670 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29168/tid=29179] ppid=29166 vsize=4389084 CPUtime=5003.38 cores=0-7
/proc/29168/task/29179/stat : 29179 (plingeling) R 29166 29168 29142 0 -1 4202560 904793 1253 0 0 500158 180 0 0 25 0 9 0 591160612 4494422016 1059997 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29168/tid=29180] ppid=29166 vsize=4389084 CPUtime=5002.43 cores=0-7
/proc/29168/task/29180/stat : 29180 (plingeling) R 29166 29168 29142 0 -1 4202560 901154 1253 0 0 500075 168 0 0 25 0 9 0 591160612 4494422016 1059997 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364502 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29168/tid=29181] ppid=29166 vsize=4389084 CPUtime=5003.4 cores=0-7
/proc/29168/task/29181/stat : 29181 (plingeling) R 29166 29168 29142 0 -1 4202560 691761 1253 0 0 500211 129 0 0 25 0 9 0 591160612 4494422016 1059997 33554432000 4194304 4417332 140735722445936 18446744073709551615 4263155 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=29168/tid=29182] ppid=29166 vsize=4389084 CPUtime=5003.53 cores=0-7
/proc/29168/task/29182/stat : 29182 (plingeling) R 29166 29168 29142 0 -1 4202560 861096 1253 0 0 500205 148 0 0 25 0 9 0 591160612 4494422016 1059997 33554432000 4194304 4417332 140735722445936 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40000.2
Current children cumulated vsize (KiB) 4389084

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5004.01 s]
/proc/loadavg: 8.02 8.03 8.00 9/179 29336
/proc/meminfo: memFree=19482716/32950928 swapFree=67071984/67111528
[pid=29168] ppid=29166 vsize=0 CPUtime=40000.4 cores=0-7
/proc/29168/stat : 29168 (plingeling) Z 29166 29168 29142 0 -1 4203548 6513557 1253 0 0 3998834 1204 0 0 18 0 2 0 591160562 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/29168/statm: 0 0 0 0 0 0 0
[pid=29168/tid=29178] ppid=29166 vsize=0 CPUtime=4976.4 cores=0-7
/proc/29168/task/29178/stat : 29178 (plingeling) R 29166 29168 29142 0 -1 4203588 700940 1253 0 0 497520 120 0 0 25 0 2 0 591160612 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5004.09
CPU time (s): 40000.5
CPU user time (s): 39988.3
CPU system time (s): 12.1262
CPU usage (%): 799.355
Max. virtual memory (cumulated for all children) (KiB): 4545064

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39988.3
system time used= 12.1262
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6514810
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 365
involuntary context switches= 192929

runsolver used 7.78082 second user time and 19.48 second system time

The end

Launcher Data 

Begin job on node116 at 2011-04-28 18:48:19
IDJOB=3309807
IDBENCH=83116
IDSOLVER=1695
FILE ID=node116/3309807-1304009299
RUNJOBID= node116-1304009299-29156
PBS_JOBID= 13168851
Free space on /tmp= 72512 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-top24.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3309807-1304009299/watcher-3309807-1304009299 -o /tmp/evaluation-result-3309807-1304009299/solver-3309807-1304009299 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3309807-1304009299.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 1f485d737caaaca49e2c467c83cbdcec
RANDOM SEED=924359495

node116.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      23728456 kB
Buffers:       1890196 kB
Cached:        6869944 kB
SwapCached:       9732 kB
Active:        4111188 kB
Inactive:      4687364 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      23728456 kB
SwapTotal:    67111528 kB
SwapFree:     67071984 kB
Dirty:            3976 kB
Writeback:           0 kB
AnonPages:       32244 kB
Mapped:          13328 kB
Slab:           358300 kB
PageTables:       4260 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   187188 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72512 MiB
End job on node116 at 2011-04-28 20:11:45