Trace number 3309381

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 40000.8 5004.24

General information on the benchmark

NameAPPLICATIONS/crypto/
md5gen/gus-md5-11.cnf
MD5SUM49867705b76fc90285dd6fc48de2613c
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark1376.76
Satisfiable
(Un)Satisfiability was proved
Number of variables69561
Number of clauses226787
Sum of the clauses size587727
Maximum clause length3
Minimum clause length1
Number of clauses of size 1129
Number of clauses of size 292376
Number of clauses of size 3134282
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3309381-1304002669/watcher-3309381-1304002669 -o /tmp/evaluation-result-3309381-1304002669/solver-3309381-1304002669 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3309381-1304002669.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.96 1.00 1.02 2/170 8621
/proc/meminfo: memFree=26386720/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=14316 CPUtime=0 cores=0-7
/proc/8621/stat : 8621 (plingeling) R 8619 8621 8595 0 -1 4202496 1348 1260 0 0 0 0 0 0 25 0 1 0 511866578 14659584 1278 33554432000 4194304 4417332 140734139130304 18446744073709551615 208079926365 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/8621/statm: 3579 1291 143 55 0 1465 0

[startup+0.0520671 s]
/proc/loadavg: 0.96 1.00 1.02 2/170 8621
/proc/meminfo: memFree=26386720/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=37952 CPUtime=0.04 cores=0-7
/proc/8621/stat : 8621 (plingeling) R 8619 8621 8595 0 -1 4202496 7615 1260 0 0 3 1 0 0 25 0 1 0 511866578 38862848 7479 33554432000 4194304 4417332 140734139130304 18446744073709551615 4264950 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/8621/statm: 9488 7479 148 55 0 7374 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 37952

[startup+0.101058 s]
/proc/loadavg: 0.96 1.00 1.02 2/170 8621
/proc/meminfo: memFree=26386720/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=49380 CPUtime=0.09 cores=0-7
/proc/8621/stat : 8621 (plingeling) R 8619 8621 8595 0 -1 4202496 10504 1260 0 0 8 1 0 0 25 0 1 0 511866578 50565120 10368 33554432000 4194304 4417332 140734139130304 18446744073709551615 4264978 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/8621/statm: 12345 10368 148 55 0 10231 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 49380

[startup+0.301019 s]
/proc/loadavg: 0.96 1.00 1.02 2/170 8621
/proc/meminfo: memFree=26386720/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=72308 CPUtime=0.29 cores=0-7
/proc/8621/stat : 8621 (plingeling) R 8619 8621 8595 0 -1 4202496 16237 1260 0 0 28 1 0 0 25 0 1 0 511866578 74043392 16100 33554432000 4194304 4417332 140734139130304 18446744073709551615 4274442 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/8621/statm: 18077 16100 148 55 0 15963 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 72308

[startup+0.700953 s]
/proc/loadavg: 0.96 1.00 1.02 2/170 8621
/proc/meminfo: memFree=26386720/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=274536 CPUtime=0.82 cores=0-7
/proc/8621/stat : 8621 (plingeling) S 8619 8621 8595 0 -1 4202496 36926 1260 0 0 78 4 0 0 25 0 9 0 511866578 281124864 29830 33554432000 4194304 4417332 140734139130304 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/8621/statm: 68634 29830 159 55 0 66520 0
Current children cumulated CPU time (s) 0.82
Current children cumulated vsize (KiB) 274536

[startup+1.5008 s]
/proc/loadavg: 0.96 1.00 1.02 9/179 8635
/proc/meminfo: memFree=26234632/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=285584 CPUtime=7.2 cores=0-7
/proc/8621/stat : 8621 (plingeling) S 8619 8621 8595 0 -1 4202496 60535 1260 0 0 713 7 0 0 25 0 9 0 511866578 292438016 37341 33554432000 4194304 4417332 140734139130304 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/8621/statm: 71396 37341 172 55 0 69282 0
[pid=8621/tid=8628] ppid=8619 vsize=285584 CPUtime=0.82 cores=0-7
/proc/8621/task/8628/stat : 8628 (plingeling) R 8619 8621 8595 0 -1 4202560 6228 1260 0 0 82 0 0 0 19 0 9 0 511866645 292438016 37341 33554432000 4194304 4417332 140734139130304 18446744073709551615 4286467 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8621/tid=8629] ppid=8619 vsize=285584 CPUtime=0.81 cores=0-7
/proc/8621/task/8629/stat : 8629 (plingeling) R 8619 8621 8595 0 -1 4202560 1901 1260 0 0 81 0 0 0 25 0 9 0 511866645 292438016 37341 33554432000 4194304 4417332 140734139130304 18446744073709551615 4287223 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8621/tid=8630] ppid=8619 vsize=285584 CPUtime=0.81 cores=0-7
/proc/8621/task/8630/stat : 8630 (plingeling) R 8619 8621 8595 0 -1 4202560 1898 1260 0 0 81 0 0 0 25 0 9 0 511866645 292438016 37341 33554432000 4194304 4417332 140734139130304 18446744073709551615 4286859 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8621/tid=8631] ppid=8619 vsize=285584 CPUtime=0.81 cores=0-7
/proc/8621/task/8631/stat : 8631 (plingeling) R 8619 8621 8595 0 -1 4202560 1754 1260 0 0 81 0 0 0 25 0 9 0 511866645 292438016 37341 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364452 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8621/tid=8632] ppid=8619 vsize=285584 CPUtime=0.81 cores=0-7
/proc/8621/task/8632/stat : 8632 (plingeling) R 8619 8621 8595 0 -1 4202560 1866 1260 0 0 81 0 0 0 25 0 9 0 511866645 292438016 37341 33554432000 4194304 4417332 140734139130304 18446744073709551615 4284626 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8621/tid=8633] ppid=8619 vsize=285584 CPUtime=0.81 cores=0-7
/proc/8621/task/8633/stat : 8633 (plingeling) R 8619 8621 8595 0 -1 4202560 2774 1260 0 0 81 0 0 0 25 0 9 0 511866645 292438016 37341 33554432000 4194304 4417332 140734139130304 18446744073709551615 4292124 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8621/tid=8634] ppid=8619 vsize=285584 CPUtime=0.8 cores=0-7
/proc/8621/task/8634/stat : 8634 (plingeling) R 8619 8621 8595 0 -1 4202560 6414 1260 0 0 80 0 0 0 25 0 9 0 511866645 292438016 37341 33554432000 4194304 4417332 140734139130304 18446744073709551615 4282356 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8621/tid=8635] ppid=8619 vsize=285584 CPUtime=0.78 cores=0-7
/proc/8621/task/8635/stat : 8635 (plingeling) R 8619 8621 8595 0 -1 4202560 1762 1260 0 0 78 0 0 0 25 0 9 0 511866646 292438016 37341 33554432000 4194304 4417332 140734139130304 18446744073709551615 4283557 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 7.2
Current children cumulated vsize (KiB) 285584

[startup+3.1005 s]
/proc/loadavg: 0.96 1.00 1.02 9/179 8635
/proc/meminfo: memFree=26227440/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=291756 CPUtime=19.99 cores=0-7
/proc/8621/stat : 8621 (plingeling) S 8619 8621 8595 0 -1 4202496 81707 1260 0 0 1989 10 0 0 25 0 9 0 511866578 298758144 36349 33554432000 4194304 4417332 140734139130304 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/8621/statm: 72939 36349 174 55 0 70825 0
[pid=8621/tid=8628] ppid=8619 vsize=291756 CPUtime=2.43 cores=0-7
/proc/8621/task/8628/stat : 8628 (plingeling) R 8619 8621 8595 0 -1 4202560 12420 1260 0 0 242 1 0 0 22 0 9 0 511866645 298758144 36349 33554432000 4194304 4417332 140734139130304 18446744073709551615 4276255 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8621/tid=8629] ppid=8619 vsize=291756 CPUtime=2.41 cores=0-7
/proc/8621/task/8629/stat : 8629 (plingeling) R 8619 8621 8595 0 -1 4202560 3459 1260 0 0 241 0 0 0 25 0 9 0 511866645 298758144 36349 33554432000 4194304 4417332 140734139130304 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8621/tid=8630] ppid=8619 vsize=291756 CPUtime=2.41 cores=0-7
/proc/8621/task/8630/stat : 8630 (plingeling) R 8619 8621 8595 0 -1 4202560 1899 1260 0 0 241 0 0 0 25 0 9 0 511866645 298758144 36349 33554432000 4194304 4417332 140734139130304 18446744073709551615 4345860 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8621/tid=8631] ppid=8619 vsize=291756 CPUtime=2.4 cores=0-7
/proc/8621/task/8631/stat : 8631 (plingeling) R 8619 8621 8595 0 -1 4202560 1754 1260 0 0 240 0 0 0 25 0 9 0 511866645 298758144 36349 33554432000 4194304 4417332 140734139130304 18446744073709551615 4264796 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8621/tid=8632] ppid=8619 vsize=291756 CPUtime=2.41 cores=0-7
/proc/8621/task/8632/stat : 8632 (plingeling) R 8619 8621 8595 0 -1 4202560 1867 1260 0 0 241 0 0 0 25 0 9 0 511866645 298758144 36349 33554432000 4194304 4417332 140734139130304 18446744073709551615 4236891 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8621/tid=8633] ppid=8619 vsize=291756 CPUtime=2.41 cores=0-7
/proc/8621/task/8633/stat : 8633 (plingeling) R 8619 8621 8595 0 -1 4202560 4329 1260 0 0 241 0 0 0 25 0 9 0 511866645 298758144 36349 33554432000 4194304 4417332 140734139130304 18446744073709551615 4265465 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8621/tid=8634] ppid=8619 vsize=291756 CPUtime=2.41 cores=0-7
/proc/8621/task/8634/stat : 8634 (plingeling) R 8619 8621 8595 0 -1 4202560 10943 1260 0 0 240 1 0 0 25 0 9 0 511866645 298758144 36349 33554432000 4194304 4417332 140734139130304 18446744073709551615 4234242 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8621/tid=8635] ppid=8619 vsize=291756 CPUtime=2.38 cores=0-7
/proc/8621/task/8635/stat : 8635 (plingeling) R 8619 8621 8595 0 -1 4202560 9098 1260 0 0 237 1 0 0 25 0 9 0 511866646 298758144 36349 33554432000 4194304 4417332 140734139130304 18446744073709551615 4326602 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 19.99
Current children cumulated vsize (KiB) 291756

[startup+6.30088 s]
/proc/loadavg: 1.52 1.11 1.06 10/179 8635
/proc/meminfo: memFree=26186808/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=336916 CPUtime=45.57 cores=0-7
/proc/8621/stat : 8621 (plingeling) S 8619 8621 8595 0 -1 4202496 101101 1260 0 0 4544 13 0 0 25 0 9 0 511866578 345001984 49341 33554432000 4194304 4417332 140734139130304 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/8621/statm: 84229 49341 180 55 0 82115 0
[pid=8621/tid=8628] ppid=8619 vsize=336916 CPUtime=5.62 cores=0-7
/proc/8621/task/8628/stat : 8628 (plingeling) R 8619 8621 8595 0 -1 4202560 17419 1260 0 0 561 1 0 0 25 0 9 0 511866645 345001984 49341 33554432000 4194304 4417332 140734139130304 18446744073709551615 4252352 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8621/tid=8629] ppid=8619 vsize=336916 CPUtime=5.61 cores=0-7
/proc/8621/task/8629/stat : 8629 (plingeling) R 8619 8621 8595 0 -1 4202560 7049 1260 0 0 560 1 0 0 25 0 9 0 511866645 345001984 49341 33554432000 4194304 4417332 140734139130304 18446744073709551615 4253272 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8621/tid=8630] ppid=8619 vsize=336916 CPUtime=5.61 cores=0-7
/proc/8621/task/8630/stat : 8630 (plingeling) R 8619 8621 8595 0 -1 4202560 1903 1260 0 0 561 0 0 0 25 0 9 0 511866645 345001984 49341 33554432000 4194304 4417332 140734139130304 18446744073709551615 4365894 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8621/tid=8631] ppid=8619 vsize=336916 CPUtime=5.58 cores=0-7
/proc/8621/task/8631/stat : 8631 (plingeling) R 8619 8621 8595 0 -1 4202560 1756 1260 0 0 558 0 0 0 25 0 9 0 511866645 345001984 49341 33554432000 4194304 4417332 140734139130304 18446744073709551615 4236549 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8621/tid=8632] ppid=8619 vsize=336916 CPUtime=5.61 cores=0-7
/proc/8621/task/8632/stat : 8632 (plingeling) R 8619 8621 8595 0 -1 4202560 6532 1260 0 0 560 1 0 0 25 0 9 0 511866645 345001984 49341 33554432000 4194304 4417332 140734139130304 18446744073709551615 4252240 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8621/tid=8633] ppid=8619 vsize=336916 CPUtime=5.61 cores=0-7
/proc/8621/task/8633/stat : 8633 (plingeling) R 8619 8621 8595 0 -1 4202560 6895 1260 0 0 560 1 0 0 25 0 9 0 511866645 345001984 49341 33554432000 4194304 4417332 140734139130304 18446744073709551615 4273771 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8621/tid=8634] ppid=8619 vsize=336916 CPUtime=5.6 cores=0-7
/proc/8621/task/8634/stat : 8634 (plingeling) R 8619 8621 8595 0 -1 4202560 12457 1260 0 0 559 1 0 0 25 0 9 0 511866645 345001984 49341 33554432000 4194304 4417332 140734139130304 18446744073709551615 4236601 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8621/tid=8635] ppid=8619 vsize=336916 CPUtime=5.58 cores=0-7
/proc/8621/task/8635/stat : 8635 (plingeling) R 8619 8621 8595 0 -1 4202560 11152 1260 0 0 557 1 0 0 25 0 9 0 511866646 345001984 49341 33554432000 4194304 4417332 140734139130304 18446744073709551615 4252491 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 45.57
Current children cumulated vsize (KiB) 336916

[startup+12.7007 s]
/proc/loadavg: 2.04 1.23 1.09 9/179 8636
/proc/meminfo: memFree=26190060/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=337796 CPUtime=96.73 cores=0-7
/proc/8621/stat : 8621 (plingeling) S 8619 8621 8595 0 -1 4202496 101506 1260 0 0 9660 13 0 0 25 0 9 0 511866578 345903104 48938 33554432000 4194304 4417332 140734139130304 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/8621/statm: 84449 48938 180 55 0 82335 0
[pid=8621/tid=8628] ppid=8619 vsize=337796 CPUtime=12.02 cores=0-7
/proc/8621/task/8628/stat : 8628 (plingeling) R 8619 8621 8595 0 -1 4202560 17419 1260 0 0 1201 1 0 0 25 0 9 0 511866645 345903104 48938 33554432000 4194304 4417332 140734139130304 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8621/tid=8629] ppid=8619 vsize=337796 CPUtime=12.01 cores=0-7
/proc/8621/task/8629/stat : 8629 (plingeling) R 8619 8621 8595 0 -1 4202560 7049 1260 0 0 1200 1 0 0 25 0 9 0 511866645 345903104 48938 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8621/tid=8630] ppid=8619 vsize=337796 CPUtime=12.01 cores=0-7
/proc/8621/task/8630/stat : 8630 (plingeling) R 8619 8621 8595 0 -1 4202560 1903 1260 0 0 1201 0 0 0 25 0 9 0 511866645 345903104 48938 33554432000 4194304 4417332 140734139130304 18446744073709551615 4353660 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8621/tid=8631] ppid=8619 vsize=337796 CPUtime=11.95 cores=0-7
/proc/8621/task/8631/stat : 8631 (plingeling) R 8619 8621 8595 0 -1 4202560 2161 1260 0 0 1195 0 0 0 25 0 9 0 511866645 345903104 48938 33554432000 4194304 4417332 140734139130304 18446744073709551615 4250969 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8621/tid=8632] ppid=8619 vsize=337796 CPUtime=12.01 cores=0-7
/proc/8621/task/8632/stat : 8632 (plingeling) R 8619 8621 8595 0 -1 4202560 6532 1260 0 0 1200 1 0 0 25 0 9 0 511866645 345903104 48938 33554432000 4194304 4417332 140734139130304 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8621/tid=8633] ppid=8619 vsize=337796 CPUtime=12.01 cores=0-7
/proc/8621/task/8633/stat : 8633 (plingeling) R 8619 8621 8595 0 -1 4202560 6895 1260 0 0 1200 1 0 0 25 0 9 0 511866645 345903104 48938 33554432000 4194304 4417332 140734139130304 18446744073709551615 4357111 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8621/tid=8634] ppid=8619 vsize=337796 CPUtime=12 cores=0-7
/proc/8621/task/8634/stat : 8634 (plingeling) R 8619 8621 8595 0 -1 4202560 12457 1260 0 0 1199 1 0 0 25 0 9 0 511866645 345903104 48938 33554432000 4194304 4417332 140734139130304 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8621/tid=8635] ppid=8619 vsize=337796 CPUtime=11.98 cores=0-7
/proc/8621/task/8635/stat : 8635 (plingeling) R 8619 8621 8595 0 -1 4202560 11152 1260 0 0 1197 1 0 0 25 0 9 0 511866646 345903104 48938 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 96.73
Current children cumulated vsize (KiB) 337796

[startup+25.5012 s]
/proc/loadavg: 3.36 1.56 1.20 9/179 8636
/proc/meminfo: memFree=26166360/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=363164 CPUtime=199.07 cores=0-7
/proc/8621/stat : 8621 (plingeling) S 8619 8621 8595 0 -1 4202496 107864 1260 0 0 19893 14 0 0 25 0 9 0 511866578 371879936 55280 33554432000 4194304 4417332 140734139130304 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/8621/statm: 90791 55280 180 55 0 88677 0
[pid=8621/tid=8628] ppid=8619 vsize=363164 CPUtime=24.82 cores=0-7
/proc/8621/task/8628/stat : 8628 (plingeling) R 8619 8621 8595 0 -1 4202560 19179 1260 0 0 2481 1 0 0 25 0 9 0 511866645 371879936 55280 33554432000 4194304 4417332 140734139130304 18446744073709551615 4366319 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8621/tid=8629] ppid=8619 vsize=363164 CPUtime=24.82 cores=0-7
/proc/8621/task/8629/stat : 8629 (plingeling) R 8619 8621 8595 0 -1 4202560 7049 1260 0 0 2481 1 0 0 25 0 9 0 511866645 371879936 55280 33554432000 4194304 4417332 140734139130304 18446744073709551615 4250762 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8621/tid=8630] ppid=8619 vsize=363164 CPUtime=24.81 cores=0-7
/proc/8621/task/8630/stat : 8630 (plingeling) R 8619 8621 8595 0 -1 4202560 1903 1260 0 0 2481 0 0 0 25 0 9 0 511866645 371879936 55280 33554432000 4194304 4417332 140734139130304 18446744073709551615 4250943 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8621/tid=8631] ppid=8619 vsize=363164 CPUtime=24.68 cores=0-7
/proc/8621/task/8631/stat : 8631 (plingeling) R 8619 8621 8595 0 -1 4202560 3720 1260 0 0 2468 0 0 0 25 0 9 0 511866645 371879936 55280 33554432000 4194304 4417332 140734139130304 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8621/tid=8632] ppid=8619 vsize=363164 CPUtime=24.81 cores=0-7
/proc/8621/task/8632/stat : 8632 (plingeling) R 8619 8621 8595 0 -1 4202560 7503 1260 0 0 2480 1 0 0 25 0 9 0 511866645 371879936 55280 33554432000 4194304 4417332 140734139130304 18446744073709551615 4357111 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8621/tid=8633] ppid=8619 vsize=363164 CPUtime=24.81 cores=0-7
/proc/8621/task/8633/stat : 8633 (plingeling) R 8619 8621 8595 0 -1 4202560 7586 1260 0 0 2480 1 0 0 25 0 9 0 511866645 371879936 55280 33554432000 4194304 4417332 140734139130304 18446744073709551615 4353957 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8621/tid=8634] ppid=8619 vsize=363164 CPUtime=24.81 cores=0-7
/proc/8621/task/8634/stat : 8634 (plingeling) R 8619 8621 8595 0 -1 4202560 13409 1260 0 0 2479 2 0 0 25 0 9 0 511866645 371879936 55280 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364375 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8621/tid=8635] ppid=8619 vsize=363164 CPUtime=24.78 cores=0-7
/proc/8621/task/8635/stat : 8635 (plingeling) R 8619 8621 8595 0 -1 4202560 11577 1260 0 0 2477 1 0 0 25 0 9 0 511866646 371879936 55280 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364463 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 199.07
Current children cumulated vsize (KiB) 363164

[startup+51.1013 s]
/proc/loadavg: 5.01 2.09 1.39 9/179 8637
/proc/meminfo: memFree=26134772/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=395936 CPUtime=403.73 cores=0-7
/proc/8621/stat : 8621 (plingeling) S 8619 8621 8595 0 -1 4202496 121008 1260 0 0 40357 16 0 0 25 0 9 0 511866578 405438464 63350 33554432000 4194304 4417332 140734139130304 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/8621/statm: 98984 63350 181 55 0 96870 0
[pid=8621/tid=8628] ppid=8619 vsize=395936 CPUtime=50.42 cores=0-7
/proc/8621/task/8628/stat : 8628 (plingeling) R 8619 8621 8595 0 -1 4202560 20626 1260 0 0 5041 1 0 0 25 0 9 0 511866645 405438464 63350 33554432000 4194304 4417332 140734139130304 18446744073709551615 4278271 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8621/tid=8629] ppid=8619 vsize=395936 CPUtime=50.41 cores=0-7
/proc/8621/task/8629/stat : 8629 (plingeling) R 8619 8621 8595 0 -1 4202560 7176 1260 0 0 5040 1 0 0 25 0 9 0 511866645 405438464 63350 33554432000 4194304 4417332 140734139130304 18446744073709551615 4251172 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8621/tid=8630] ppid=8619 vsize=395936 CPUtime=50.41 cores=0-7

################
# More data... #
################

/proc/8621/task/8631/stat : 8631 (plingeling) R 8619 8621 8595 0 -1 4202560 392532 1260 0 0 463556 81 0 0 25 0 9 0 511866645 790396928 157378 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8621/tid=8632] ppid=8619 vsize=771872 CPUtime=4661.12 cores=0-7
/proc/8621/task/8632/stat : 8632 (plingeling) R 8619 8621 8595 0 -1 4202560 419153 1260 0 0 466029 83 0 0 25 0 9 0 511866645 790396928 157378 33554432000 4194304 4417332 140734139130304 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8621/tid=8633] ppid=8619 vsize=771872 CPUtime=4661.79 cores=0-7
/proc/8621/task/8633/stat : 8633 (plingeling) R 8619 8621 8595 0 -1 4202560 441000 1260 0 0 466089 90 0 0 25 0 9 0 511866645 790396928 157378 33554432000 4194304 4417332 140734139130304 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8621/tid=8634] ppid=8619 vsize=771872 CPUtime=4661.78 cores=0-7
/proc/8621/task/8634/stat : 8634 (plingeling) R 8619 8621 8595 0 -1 4202560 421278 1260 0 0 466102 76 0 0 25 0 9 0 511866645 790396928 157378 33554432000 4194304 4417332 140734139130304 18446744073709551615 4357111 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8621/tid=8635] ppid=8619 vsize=771872 CPUtime=4661.46 cores=0-7
/proc/8621/task/8635/stat : 8635 (plingeling) R 8619 8621 8595 0 -1 4202560 447456 1260 0 0 466054 92 0 0 25 0 9 0 511866646 790396928 157378 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364919 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37267.3
Current children cumulated vsize (KiB) 771872

[startup+4722.31 s]
/proc/loadavg: 8.07 8.06 7.97 9/179 8785
/proc/meminfo: memFree=25835916/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=692864 CPUtime=37747.1 cores=0-7
/proc/8621/stat : 8621 (plingeling) S 8619 8621 8595 0 -1 4202496 3600304 1260 0 0 3773985 720 0 0 25 0 9 0 511866578 709492736 137626 33554432000 4194304 4417332 140734139130304 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/8621/statm: 173216 137626 181 55 0 171102 0
[pid=8621/tid=8628] ppid=8619 vsize=692864 CPUtime=4721.81 cores=0-7
/proc/8621/task/8628/stat : 8628 (plingeling) R 8619 8621 8595 0 -1 4202560 510172 1260 0 0 472082 99 0 0 25 0 9 0 511866645 709492736 137626 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364899 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8621/tid=8629] ppid=8619 vsize=692864 CPUtime=4721.62 cores=0-7
/proc/8621/task/8629/stat : 8629 (plingeling) R 8619 8621 8595 0 -1 4202560 429711 1260 0 0 472077 85 0 0 25 0 9 0 511866645 709492736 137626 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364568 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8621/tid=8630] ppid=8619 vsize=692864 CPUtime=4720.67 cores=0-7
/proc/8621/task/8630/stat : 8630 (plingeling) R 8619 8621 8595 0 -1 4202560 466708 1260 0 0 471969 98 0 0 25 0 9 0 511866645 709492736 137626 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8621/tid=8631] ppid=8619 vsize=692864 CPUtime=4696.05 cores=0-7
/proc/8621/task/8631/stat : 8631 (plingeling) R 8619 8621 8595 0 -1 4202560 407823 1260 0 0 469521 84 0 0 25 0 9 0 511866645 709492736 137626 33554432000 4194304 4417332 140734139130304 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8621/tid=8632] ppid=8619 vsize=692864 CPUtime=4721.12 cores=0-7
/proc/8621/task/8632/stat : 8632 (plingeling) R 8619 8621 8595 0 -1 4202560 420583 1260 0 0 472029 83 0 0 25 0 9 0 511866645 709492736 137626 33554432000 4194304 4417332 140734139130304 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8621/tid=8633] ppid=8619 vsize=692864 CPUtime=4721.8 cores=0-7
/proc/8621/task/8633/stat : 8633 (plingeling) R 8619 8621 8595 0 -1 4202560 444314 1260 0 0 472089 91 0 0 25 0 9 0 511866645 709492736 137626 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8621/tid=8634] ppid=8619 vsize=692864 CPUtime=4721.78 cores=0-7
/proc/8621/task/8634/stat : 8634 (plingeling) R 8619 8621 8595 0 -1 4202560 430260 1260 0 0 472100 78 0 0 25 0 9 0 511866645 709492736 137626 33554432000 4194304 4417332 140734139130304 18446744073709551615 4352400 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8621/tid=8635] ppid=8619 vsize=692864 CPUtime=4721.46 cores=0-7
/proc/8621/task/8635/stat : 8635 (plingeling) R 8619 8621 8595 0 -1 4202560 454795 1260 0 0 472052 94 0 0 25 0 9 0 511866646 709492736 137626 33554432000 4194304 4417332 140734139130304 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37747.1
Current children cumulated vsize (KiB) 692864

[startup+4782.3 s]
/proc/loadavg: 8.08 8.06 7.98 9/179 8787
/proc/meminfo: memFree=25899728/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=629356 CPUtime=38226.7 cores=0-7
/proc/8621/stat : 8621 (plingeling) S 8619 8621 8595 0 -1 4202496 3624780 1260 0 0 3821945 725 0 0 25 0 9 0 511866578 644460544 121749 33554432000 4194304 4417332 140734139130304 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/8621/statm: 157339 121749 181 55 0 155225 0
[pid=8621/tid=8628] ppid=8619 vsize=629356 CPUtime=4781.8 cores=0-7
/proc/8621/task/8628/stat : 8628 (plingeling) R 8619 8621 8595 0 -1 4202560 515056 1260 0 0 478081 99 0 0 25 0 9 0 511866645 644460544 121749 33554432000 4194304 4417332 140734139130304 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8621/tid=8629] ppid=8619 vsize=629356 CPUtime=4781.61 cores=0-7
/proc/8621/task/8629/stat : 8629 (plingeling) R 8619 8621 8595 0 -1 4202560 431493 1260 0 0 478076 85 0 0 25 0 9 0 511866645 644460544 121749 33554432000 4194304 4417332 140734139130304 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8621/tid=8630] ppid=8619 vsize=629356 CPUtime=4780.67 cores=0-7
/proc/8621/task/8630/stat : 8630 (plingeling) R 8619 8621 8595 0 -1 4202560 466708 1260 0 0 477969 98 0 0 25 0 9 0 511866645 644460544 121749 33554432000 4194304 4417332 140734139130304 18446744073709551615 4358014 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8621/tid=8631] ppid=8619 vsize=629356 CPUtime=4755.72 cores=0-7
/proc/8621/task/8631/stat : 8631 (plingeling) R 8619 8621 8595 0 -1 4202560 414177 1260 0 0 475487 85 0 0 25 0 9 0 511866645 644460544 121749 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8621/tid=8632] ppid=8619 vsize=629356 CPUtime=4781.13 cores=0-7
/proc/8621/task/8632/stat : 8632 (plingeling) R 8619 8621 8595 0 -1 4202560 421593 1260 0 0 478029 84 0 0 25 0 9 0 511866645 644460544 121749 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8621/tid=8633] ppid=8619 vsize=629356 CPUtime=4781.8 cores=0-7
/proc/8621/task/8633/stat : 8633 (plingeling) R 8619 8621 8595 0 -1 4202560 449469 1260 0 0 478087 93 0 0 25 0 9 0 511866645 644460544 121749 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8621/tid=8634] ppid=8619 vsize=629356 CPUtime=4781.78 cores=0-7
/proc/8621/task/8634/stat : 8634 (plingeling) R 8619 8621 8595 0 -1 4202560 431100 1260 0 0 478100 78 0 0 25 0 9 0 511866645 644460544 121749 33554432000 4194304 4417332 140734139130304 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8621/tid=8635] ppid=8619 vsize=629356 CPUtime=4781.46 cores=0-7
/proc/8621/task/8635/stat : 8635 (plingeling) R 8619 8621 8595 0 -1 4202560 459246 1260 0 0 478051 95 0 0 25 0 9 0 511866646 644460544 121749 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38226.7
Current children cumulated vsize (KiB) 629356

[startup+4842.3 s]
/proc/loadavg: 8.07 8.06 7.98 9/179 8788
/proc/meminfo: memFree=25834488/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=694304 CPUtime=38706.4 cores=0-7
/proc/8621/stat : 8621 (plingeling) S 8619 8621 8595 0 -1 4202496 3650978 1260 0 0 3869909 729 0 0 25 0 9 0 511866578 710967296 137986 33554432000 4194304 4417332 140734139130304 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/8621/statm: 173576 137986 181 55 0 171462 0
[pid=8621/tid=8628] ppid=8619 vsize=694304 CPUtime=4841.81 cores=0-7
/proc/8621/task/8628/stat : 8628 (plingeling) R 8619 8621 8595 0 -1 4202560 517456 1260 0 0 484081 100 0 0 25 0 9 0 511866645 710967296 137986 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364590 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8621/tid=8629] ppid=8619 vsize=694304 CPUtime=4841.62 cores=0-7
/proc/8621/task/8629/stat : 8629 (plingeling) R 8619 8621 8595 0 -1 4202560 436429 1260 0 0 484076 86 0 0 25 0 9 0 511866645 710967296 137986 33554432000 4194304 4417332 140734139130304 18446744073709551615 4353734 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8621/tid=8630] ppid=8619 vsize=694304 CPUtime=4840.67 cores=0-7
/proc/8621/task/8630/stat : 8630 (plingeling) R 8619 8621 8595 0 -1 4202560 471994 1260 0 0 483968 99 0 0 25 0 9 0 511866645 710967296 137986 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8621/tid=8631] ppid=8619 vsize=694304 CPUtime=4815.39 cores=0-7
/proc/8621/task/8631/stat : 8631 (plingeling) R 8619 8621 8595 0 -1 4202560 417288 1260 0 0 481454 85 0 0 25 0 9 0 511866645 710967296 137986 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8621/tid=8632] ppid=8619 vsize=694304 CPUtime=4841.13 cores=0-7
/proc/8621/task/8632/stat : 8632 (plingeling) R 8619 8621 8595 0 -1 4202560 423078 1260 0 0 484029 84 0 0 25 0 9 0 511866645 710967296 137986 33554432000 4194304 4417332 140734139130304 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8621/tid=8633] ppid=8619 vsize=694304 CPUtime=4841.8 cores=0-7
/proc/8621/task/8633/stat : 8633 (plingeling) R 8619 8621 8595 0 -1 4202560 451195 1260 0 0 484087 93 0 0 25 0 9 0 511866645 710967296 137986 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8621/tid=8634] ppid=8619 vsize=694304 CPUtime=4841.78 cores=0-7
/proc/8621/task/8634/stat : 8634 (plingeling) R 8619 8621 8595 0 -1 4202560 437394 1260 0 0 484098 80 0 0 25 0 9 0 511866645 710967296 137986 33554432000 4194304 4417332 140734139130304 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8621/tid=8635] ppid=8619 vsize=694304 CPUtime=4841.46 cores=0-7
/proc/8621/task/8635/stat : 8635 (plingeling) R 8619 8621 8595 0 -1 4202560 460206 1260 0 0 484051 95 0 0 25 0 9 0 511866646 710967296 137986 33554432000 4194304 4417332 140734139130304 18446744073709551615 4250977 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38706.4
Current children cumulated vsize (KiB) 694304

[startup+4902.3 s]
/proc/loadavg: 8.07 8.06 7.99 9/179 8790
/proc/meminfo: memFree=25802088/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=726676 CPUtime=39186.1 cores=0-7
/proc/8621/stat : 8621 (plingeling) S 8619 8621 8595 0 -1 4202496 3683606 1260 0 0 3917872 734 0 0 25 0 9 0 511866578 744116224 146079 33554432000 4194304 4417332 140734139130304 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/8621/statm: 181669 146079 181 55 0 179555 0
[pid=8621/tid=8628] ppid=8619 vsize=726676 CPUtime=4901.8 cores=0-7
/proc/8621/task/8628/stat : 8628 (plingeling) R 8619 8621 8595 0 -1 4202560 524315 1260 0 0 490080 100 0 0 25 0 9 0 511866645 744116224 146079 33554432000 4194304 4417332 140734139130304 18446744073709551615 4353974 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8621/tid=8629] ppid=8619 vsize=726676 CPUtime=4901.62 cores=0-7
/proc/8621/task/8629/stat : 8629 (plingeling) R 8619 8621 8595 0 -1 4202560 443576 1260 0 0 490075 87 0 0 25 0 9 0 511866645 744116224 146079 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8621/tid=8630] ppid=8619 vsize=726676 CPUtime=4900.66 cores=0-7
/proc/8621/task/8630/stat : 8630 (plingeling) R 8619 8621 8595 0 -1 4202560 477725 1260 0 0 489966 100 0 0 25 0 9 0 511866645 744116224 146079 33554432000 4194304 4417332 140734139130304 18446744073709551615 4237332 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8621/tid=8631] ppid=8619 vsize=726676 CPUtime=4875.08 cores=0-7
/proc/8621/task/8631/stat : 8631 (plingeling) R 8619 8621 8595 0 -1 4202560 418506 1260 0 0 487422 86 0 0 25 0 9 0 511866645 744116224 146079 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8621/tid=8632] ppid=8619 vsize=726676 CPUtime=4901.13 cores=0-7
/proc/8621/task/8632/stat : 8632 (plingeling) R 8619 8621 8595 0 -1 4202560 424954 1260 0 0 490029 84 0 0 25 0 9 0 511866645 744116224 146079 33554432000 4194304 4417332 140734139130304 18446744073709551615 4366166 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8621/tid=8633] ppid=8619 vsize=726676 CPUtime=4901.8 cores=0-7
/proc/8621/task/8633/stat : 8633 (plingeling) R 8619 8621 8595 0 -1 4202560 456662 1260 0 0 490087 93 0 0 25 0 9 0 511866645 744116224 146079 33554432000 4194304 4417332 140734139130304 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8621/tid=8634] ppid=8619 vsize=726676 CPUtime=4901.78 cores=0-7
/proc/8621/task/8634/stat : 8634 (plingeling) R 8619 8621 8595 0 -1 4202560 441724 1260 0 0 490097 81 0 0 25 0 9 0 511866645 744116224 146079 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8621/tid=8635] ppid=8619 vsize=726676 CPUtime=4901.46 cores=0-7
/proc/8621/task/8635/stat : 8635 (plingeling) R 8619 8621 8595 0 -1 4202560 460206 1260 0 0 490051 95 0 0 25 0 9 0 511866646 744116224 146079 33554432000 4194304 4417332 140734139130304 18446744073709551615 4352400 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39186.1
Current children cumulated vsize (KiB) 726676

[startup+4962.3 s]
/proc/loadavg: 8.06 8.06 8.00 9/179 8792
/proc/meminfo: memFree=25866156/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=663276 CPUtime=39665.8 cores=0-7
/proc/8621/stat : 8621 (plingeling) S 8619 8621 8595 0 -1 4202496 3724458 1260 0 0 3965833 742 0 0 25 0 9 0 511866578 679194624 130197 33554432000 4194304 4417332 140734139130304 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/8621/statm: 165819 130197 181 55 0 163705 0
[pid=8621/tid=8628] ppid=8619 vsize=663276 CPUtime=4961.81 cores=0-7
/proc/8621/task/8628/stat : 8628 (plingeling) R 8619 8621 8595 0 -1 4202560 524818 1260 0 0 496080 101 0 0 25 0 9 0 511866645 679194624 130197 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8621/tid=8629] ppid=8619 vsize=663276 CPUtime=4961.62 cores=0-7
/proc/8621/task/8629/stat : 8629 (plingeling) R 8619 8621 8595 0 -1 4202560 452107 1260 0 0 496073 89 0 0 25 0 9 0 511866645 679194624 130197 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8621/tid=8630] ppid=8619 vsize=663276 CPUtime=4960.66 cores=0-7
/proc/8621/task/8630/stat : 8630 (plingeling) R 8619 8621 8595 0 -1 4202560 486646 1260 0 0 495964 102 0 0 25 0 9 0 511866645 679194624 130197 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364720 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8621/tid=8631] ppid=8619 vsize=663276 CPUtime=4934.75 cores=0-7
/proc/8621/task/8631/stat : 8631 (plingeling) R 8619 8621 8595 0 -1 4202560 419253 1260 0 0 493389 86 0 0 25 0 9 0 511866645 679194624 130197 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364547 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8621/tid=8632] ppid=8619 vsize=663276 CPUtime=4961.13 cores=0-7
/proc/8621/task/8632/stat : 8632 (plingeling) R 8619 8621 8595 0 -1 4202560 428054 1260 0 0 496028 85 0 0 25 0 9 0 511866645 679194624 130197 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364671 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8621/tid=8633] ppid=8619 vsize=663276 CPUtime=4961.8 cores=0-7
/proc/8621/task/8633/stat : 8633 (plingeling) R 8619 8621 8595 0 -1 4202560 468150 1260 0 0 496084 96 0 0 25 0 9 0 511866645 679194624 130197 33554432000 4194304 4417332 140734139130304 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8621/tid=8634] ppid=8619 vsize=663276 CPUtime=4961.78 cores=0-7
/proc/8621/task/8634/stat : 8634 (plingeling) R 8619 8621 8595 0 -1 4202560 446477 1260 0 0 496097 81 0 0 25 0 9 0 511866645 679194624 130197 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8621/tid=8635] ppid=8619 vsize=663276 CPUtime=4961.47 cores=0-7
/proc/8621/task/8635/stat : 8635 (plingeling) R 8619 8621 8595 0 -1 4202560 463015 1260 0 0 496052 95 0 0 25 0 9 0 511866646 679194624 130197 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39665.8
Current children cumulated vsize (KiB) 663276



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5004.2 s]
/proc/loadavg: 8.08 8.07 8.00 9/179 8793
/proc/meminfo: memFree=25853212/32950928 swapFree=67111364/67111528
[pid=8621] ppid=8619 vsize=675608 CPUtime=40000.8 cores=0-7
/proc/8621/stat : 8621 (plingeling) S 8619 8621 8595 0 -1 4202496 3745339 1260 0 0 3999331 745 0 0 25 0 9 0 511866578 691822592 133280 33554432000 4194304 4417332 140734139130304 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/8621/statm: 168902 133280 181 55 0 166788 0
[pid=8621/tid=8628] ppid=8619 vsize=675608 CPUtime=5003.71 cores=0-7
/proc/8621/task/8628/stat : 8628 (plingeling) R 8619 8621 8595 0 -1 4202560 528989 1260 0 0 500270 101 0 0 25 0 9 0 511866645 691822592 133280 33554432000 4194304 4417332 140734139130304 18446744073709551615 4321915 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=8621/tid=8629] ppid=8619 vsize=675608 CPUtime=5003.52 cores=0-7
/proc/8621/task/8629/stat : 8629 (plingeling) R 8619 8621 8595 0 -1 4202560 452271 1260 0 0 500263 89 0 0 25 0 9 0 511866645 691822592 133280 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364602 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=8621/tid=8630] ppid=8619 vsize=675608 CPUtime=5002.56 cores=0-7
/proc/8621/task/8630/stat : 8630 (plingeling) R 8619 8621 8595 0 -1 4202560 490335 1260 0 0 500153 103 0 0 25 0 9 0 511866645 691822592 133280 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=8621/tid=8631] ppid=8619 vsize=675608 CPUtime=4976.42 cores=0-7
/proc/8621/task/8631/stat : 8631 (plingeling) R 8619 8621 8595 0 -1 4202560 421286 1260 0 0 497556 86 0 0 25 0 9 0 511866645 691822592 133280 33554432000 4194304 4417332 140734139130304 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8621/tid=8632] ppid=8619 vsize=675608 CPUtime=5003.04 cores=0-7
/proc/8621/task/8632/stat : 8632 (plingeling) R 8619 8621 8595 0 -1 4202560 428828 1260 0 0 500219 85 0 0 25 0 9 0 511866645 691822592 133280 33554432000 4194304 4417332 140734139130304 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8621/tid=8633] ppid=8619 vsize=675608 CPUtime=5003.71 cores=0-7
/proc/8621/task/8633/stat : 8633 (plingeling) R 8619 8621 8595 0 -1 4202560 468150 1260 0 0 500275 96 0 0 25 0 9 0 511866645 691822592 133280 33554432000 4194304 4417332 140734139130304 18446744073709551615 4365900 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8621/tid=8634] ppid=8619 vsize=675608 CPUtime=5003.69 cores=0-7
/proc/8621/task/8634/stat : 8634 (plingeling) R 8619 8621 8595 0 -1 4202560 450916 1260 0 0 500288 81 0 0 25 0 9 0 511866645 691822592 133280 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=8621/tid=8635] ppid=8619 vsize=675608 CPUtime=5003.38 cores=0-7
/proc/8621/task/8635/stat : 8635 (plingeling) R 8619 8621 8595 0 -1 4202560 468626 1260 0 0 500242 96 0 0 25 0 9 0 511866646 691822592 133280 33554432000 4194304 4417332 140734139130304 18446744073709551615 4364919 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 675608

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5004.24
CPU time (s): 40000.8
CPU user time (s): 39993.4
CPU system time (s): 7.48286
CPU usage (%): 799.339
Max. virtual memory (cumulated for all children) (KiB): 878972

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39993.4
system time used= 7.48286
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3746599
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 237
involuntary context switches= 191004

runsolver used 8.21575 second user time and 19.0751 second system time

The end

Launcher Data 

Begin job on node127 at 2011-04-28 16:57:49
IDJOB=3309381
IDBENCH=70830
IDSOLVER=1695
FILE ID=node127/3309381-1304002669
RUNJOBID= node127-1304002669-8609
PBS_JOBID= 13168797
Free space on /tmp= 71816 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT09/APPLICATIONS/crypto/md5gen/gus-md5-11.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3309381-1304002669/watcher-3309381-1304002669 -o /tmp/evaluation-result-3309381-1304002669/solver-3309381-1304002669 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3309381-1304002669.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 49867705b76fc90285dd6fc48de2613c
RANDOM SEED=1873684325

node127.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.60
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      26387248 kB
Buffers:       1366976 kB
Cached:        4659556 kB
SwapCached:          0 kB
Active:        1666468 kB
Inactive:      4423152 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      26387248 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:            4328 kB
Writeback:           0 kB
AnonPages:       63328 kB
Mapped:          14248 kB
Slab:           409360 kB
PageTables:       3928 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   182804 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71816 MiB
End job on node127 at 2011-04-28 18:21:15