Trace number 3438596

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 40001.2 5013.25

General information on the benchmark

NameCRAFTED/rbsat/crafted/
forced/rbsat-v2400c266431gyes2.cnf
MD5SUMaac2bd287b8cd6b96cbf7ecc463acf19
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2400
Number of clauses266431
Sum of the clauses size535112
Maximum clause length32
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2266356
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 575

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3438596-1306920438/watcher-3438596-1306920438 -o /tmp/evaluation-result-3438596-1306920438/solver-3438596-1306920438 -C 40000 -W 5100 -M 31000 SArTagnan HOME/instance-3438596-1306920438.cnf -threads=8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.86 2.02 1.99 2/568 5992
/proc/meminfo: memFree=24005456/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=25164 CPUtime=0 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 505 0 1 0 0 0 0 0 17 0 1 0 803641551 25767936 443 33554432000 4194304 4625164 140735867741680 18446744073709551615 212687675136 0 0 4096 2 0 0 0 17 4 0 0 0
/proc/5992/statm: 6291 448 296 106 0 1794 0

[startup+0.123855 s]
/proc/loadavg: 1.86 2.02 1.99 2/568 5992
/proc/meminfo: memFree=24005456/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=467004 CPUtime=1.8 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 17984 0 1 0 177 3 0 0 21 0 8 0 803641551 478212096 15363 33554432000 4194304 4625164 140735867741680 18446744073709551615 212687718535 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 116751 14851 349 106 0 112254 0
Current children cumulated CPU time (s) 1.8
Current children cumulated vsize (KiB) 467004

[startup+0.996759 s]
/proc/loadavg: 1.86 2.02 1.99 2/568 5992
/proc/meminfo: memFree=24005456/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=472524 CPUtime=7.33 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 32767 0 1 0 728 5 0 0 23 0 8 0 803641551 483864576 26876 33554432000 4194304 4625164 140735867741680 18446744073709551615 4512315 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 118131 26877 349 106 0 113634 0
Current children cumulated CPU time (s) 7.33
Current children cumulated vsize (KiB) 472524

[startup+1.07029 s]
/proc/loadavg: 1.86 2.02 1.99 2/568 5992
/proc/meminfo: memFree=24005456/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=483572 CPUtime=7.92 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 35448 0 1 0 787 5 0 0 24 0 8 0 803641551 495177728 28874 33554432000 4194304 4625164 140735867741680 18446744073709551615 212687718535 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 120893 28362 349 106 0 116396 0
Current children cumulated CPU time (s) 7.92
Current children cumulated vsize (KiB) 483572

[startup+1.13021 s]
/proc/loadavg: 1.86 2.02 1.99 2/568 5992
/proc/meminfo: memFree=24005456/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=480160 CPUtime=15.33 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 45268 0 1 0 1526 7 0 0 25 0 8 0 803641551 491683840 34748 33554432000 4194304 4625164 140735867741680 18446744073709551615 4512312 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 120040 34748 349 106 0 115543 0
Current children cumulated CPU time (s) 15.33
Current children cumulated vsize (KiB) 480160

[startup+2.09756 s]
/proc/loadavg: 1.86 2.02 1.99 2/568 5992
/proc/meminfo: memFree=24005456/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=490804 CPUtime=16.13 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 49215 0 1 0 1606 7 0 0 25 0 8 0 803641551 502583296 35932 33554432000 4194304 4625164 140735867741680 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 122701 35932 349 106 0 118204 0
Current children cumulated CPU time (s) 16.13
Current children cumulated vsize (KiB) 490804

[startup+3.10038 s]
/proc/loadavg: 2.35 2.12 2.03 9/576 6000
/proc/meminfo: memFree=23846060/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=490804 CPUtime=24.13 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 57909 0 1 0 2404 9 0 0 25 0 8 0 803641551 502583296 43422 33554432000 4194304 4625164 140735867741680 18446744073709551615 4513505 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 122701 43422 349 106 0 118204 0
[pid=5992/tid=5994] ppid=5990 vsize=490804 CPUtime=3.01 cores=0-7
/proc/5992/task/5994/stat : 5994 (SArTagnan) R 5990 5992 5966 0 -1 4202560 3826 0 0 0 301 0 0 0 25 0 8 0 803641559 502583296 43422 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5992/tid=5995] ppid=5990 vsize=490804 CPUtime=3.01 cores=0-7
/proc/5992/task/5995/stat : 5995 (SArTagnan) R 5990 5992 5966 0 -1 4202560 4222 0 0 0 300 1 0 0 25 0 8 0 803641559 502583296 43423 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=5992/tid=5996] ppid=5990 vsize=490804 CPUtime=3 cores=0-7
/proc/5992/task/5996/stat : 5996 (SArTagnan) R 5990 5992 5966 0 -1 4202560 7950 0 0 0 300 0 0 0 25 0 8 0 803641559 502583296 43424 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5992/tid=5997] ppid=5990 vsize=490804 CPUtime=3 cores=0-7
/proc/5992/task/5997/stat : 5997 (SArTagnan) R 5990 5992 5966 0 -1 4202560 4532 0 0 0 300 0 0 0 25 0 8 0 803641559 502583296 43424 33554432000 4194304 4625164 140735867741680 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5992/tid=5998] ppid=5990 vsize=490804 CPUtime=2.97 cores=0-7
/proc/5992/task/5998/stat : 5998 (SArTagnan) R 5990 5992 5966 0 -1 4202560 3502 0 0 0 297 0 0 0 21 0 8 0 803641559 502583296 43424 33554432000 4194304 4625164 140735867741680 18446744073709551615 4438690 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5992/tid=5999] ppid=5990 vsize=490804 CPUtime=3 cores=0-7
/proc/5992/task/5999/stat : 5999 (SArTagnan) R 5990 5992 5966 0 -1 4202560 5391 0 0 0 300 0 0 0 25 0 8 0 803641559 502583296 43424 33554432000 4194304 4625164 140735867741680 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=5992/tid=6000] ppid=5990 vsize=490804 CPUtime=2.98 cores=0-7
/proc/5992/task/6000/stat : 6000 (SArTagnan) R 5990 5992 5966 0 -1 4202560 4124 0 0 0 298 0 0 0 25 0 8 0 803641559 502583296 43424 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 24.13
Current children cumulated vsize (KiB) 490804

[startup+6.3008 s]
/proc/loadavg: 2.35 2.12 2.03 9/576 6001
/proc/meminfo: memFree=23776992/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=499092 CPUtime=49.67 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 89100 0 1 0 4952 15 0 0 25 0 8 0 803641551 511070208 59607 33554432000 4194304 4625164 140735867741680 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 124773 59607 355 106 0 120276 0
[pid=5992/tid=5994] ppid=5990 vsize=499092 CPUtime=6.2 cores=0-7
/proc/5992/task/5994/stat : 5994 (SArTagnan) R 5990 5992 5966 0 -1 4202560 6081 0 0 0 620 0 0 0 25 0 8 0 803641559 511070208 59607 33554432000 4194304 4625164 140735867741680 18446744073709551615 4352010 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5992/tid=5995] ppid=5990 vsize=499092 CPUtime=6.21 cores=0-7
/proc/5992/task/5995/stat : 5995 (SArTagnan) R 5990 5992 5966 0 -1 4202560 6252 0 0 0 620 1 0 0 25 0 8 0 803641559 511070208 59607 33554432000 4194304 4625164 140735867741680 18446744073709551615 4464328 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=5992/tid=5996] ppid=5990 vsize=499092 CPUtime=6.2 cores=0-7
/proc/5992/task/5996/stat : 5996 (SArTagnan) R 5990 5992 5966 0 -1 4202560 18230 0 0 0 618 2 0 0 25 0 8 0 803641559 511070208 59607 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351731 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5992/tid=5997] ppid=5990 vsize=499092 CPUtime=6.2 cores=0-7
/proc/5992/task/5997/stat : 5997 (SArTagnan) R 5990 5992 5966 0 -1 4202560 6710 0 0 0 619 1 0 0 25 0 8 0 803641559 511070208 59607 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5992/tid=5998] ppid=5990 vsize=499092 CPUtime=6.11 cores=0-7
/proc/5992/task/5998/stat : 5998 (SArTagnan) R 5990 5992 5966 0 -1 4202560 6111 0 0 0 610 1 0 0 25 0 8 0 803641559 511070208 59608 33554432000 4194304 4625164 140735867741680 18446744073709551615 4318490 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5992/tid=5999] ppid=5990 vsize=499092 CPUtime=6.2 cores=0-7
/proc/5992/task/5999/stat : 5999 (SArTagnan) R 5990 5992 5966 0 -1 4202560 8714 0 0 0 619 1 0 0 25 0 8 0 803641559 511070208 59608 33554432000 4194304 4625164 140735867741680 18446744073709551615 4339626 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=5992/tid=6000] ppid=5990 vsize=499092 CPUtime=6.18 cores=0-7
/proc/5992/task/6000/stat : 6000 (SArTagnan) R 5990 5992 5966 0 -1 4202560 4411 0 0 0 618 0 0 0 25 0 8 0 803641559 511070208 59608 33554432000 4194304 4625164 140735867741680 18446744073709551615 4288861 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 49.67
Current children cumulated vsize (KiB) 499092

[startup+12.7006 s]
/proc/loadavg: 3.22 2.31 2.09 9/576 6001
/proc/meminfo: memFree=23669592/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=563044 CPUtime=100.73 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 124550 0 1 0 10051 22 0 0 25 0 8 0 803641551 576557056 84052 33554432000 4194304 4625164 140735867741680 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 140761 84052 369 106 0 136264 0
[pid=5992/tid=5994] ppid=5990 vsize=563044 CPUtime=12.61 cores=0-7
/proc/5992/task/5994/stat : 5994 (SArTagnan) R 5990 5992 5966 0 -1 4202560 9268 0 0 0 1260 1 0 0 25 0 8 0 803641559 576557056 84052 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351989 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5992/tid=5995] ppid=5990 vsize=563044 CPUtime=12.6 cores=0-7
/proc/5992/task/5995/stat : 5995 (SArTagnan) R 5990 5992 5966 0 -1 4202560 9075 0 0 0 1259 1 0 0 25 0 8 0 803641559 576557056 84052 33554432000 4194304 4625164 140735867741680 18446744073709551615 4352827 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=5992/tid=5996] ppid=5990 vsize=563044 CPUtime=12.61 cores=0-7
/proc/5992/task/5996/stat : 5996 (SArTagnan) R 5990 5992 5966 0 -1 4202560 26219 0 0 0 1257 4 0 0 25 0 8 0 803641559 576557056 84052 33554432000 4194304 4625164 140735867741680 18446744073709551615 4334466 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5992/tid=5997] ppid=5990 vsize=563044 CPUtime=12.61 cores=0-7
/proc/5992/task/5997/stat : 5997 (SArTagnan) R 5990 5992 5966 0 -1 4202560 9461 0 0 0 1259 2 0 0 25 0 8 0 803641559 576557056 84052 33554432000 4194304 4625164 140735867741680 18446744073709551615 4317920 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5992/tid=5998] ppid=5990 vsize=563044 CPUtime=12.38 cores=0-7
/proc/5992/task/5998/stat : 5998 (SArTagnan) R 5990 5992 5966 0 -1 4202560 9253 0 0 0 1237 1 0 0 25 0 8 0 803641559 576557056 84052 33554432000 4194304 4625164 140735867741680 18446744073709551615 4349024 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5992/tid=5999] ppid=5990 vsize=563044 CPUtime=12.6 cores=0-7
/proc/5992/task/5999/stat : 5999 (SArTagnan) R 5990 5992 5966 0 -1 4202560 12697 0 0 0 1258 2 0 0 25 0 8 0 803641559 576557056 84052 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=5992/tid=6000] ppid=5990 vsize=563044 CPUtime=12.57 cores=0-7
/proc/5992/task/6000/stat : 6000 (SArTagnan) R 5990 5992 5966 0 -1 4202560 4435 0 0 0 1257 0 0 0 25 0 8 0 803641559 576557056 84052 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 100.73
Current children cumulated vsize (KiB) 563044

[startup+25.5013 s]
/proc/loadavg: 3.95 2.50 2.15 9/576 6001
/proc/meminfo: memFree=23534772/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=644200 CPUtime=202.91 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 217278 0 1 0 20252 39 0 0 25 0 8 0 803641551 659660800 119362 33554432000 4194304 4625164 140735867741680 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 161050 119362 546 106 0 156553 0
[pid=5992/tid=5994] ppid=5990 vsize=644200 CPUtime=25.41 cores=0-7
/proc/5992/task/5994/stat : 5994 (SArTagnan) R 5990 5992 5966 0 -1 4202560 13618 0 0 0 2539 2 0 0 25 0 8 0 803641559 659660800 119362 33554432000 4194304 4625164 140735867741680 18446744073709551615 4352296 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5992/tid=5995] ppid=5990 vsize=644200 CPUtime=25.4 cores=0-7
/proc/5992/task/5995/stat : 5995 (SArTagnan) R 5990 5992 5966 0 -1 4202560 10352 0 0 0 2538 2 0 0 25 0 8 0 803641559 659660800 119362 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351975 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=5992/tid=5996] ppid=5990 vsize=644200 CPUtime=25.41 cores=0-7
/proc/5992/task/5996/stat : 5996 (SArTagnan) R 5990 5992 5966 0 -1 4202560 80565 0 0 0 2528 13 0 0 25 0 8 0 803641559 659660800 119362 33554432000 4194304 4625164 140735867741680 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5992/tid=5997] ppid=5990 vsize=644200 CPUtime=25.41 cores=0-7
/proc/5992/task/5997/stat : 5997 (SArTagnan) R 5990 5992 5966 0 -1 4202560 13973 0 0 0 2538 3 0 0 25 0 8 0 803641559 659660800 119362 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5992/tid=5998] ppid=5990 vsize=644200 CPUtime=24.95 cores=0-7
/proc/5992/task/5998/stat : 5998 (SArTagnan) R 5990 5992 5966 0 -1 4202560 13925 0 0 0 2493 2 0 0 25 0 8 0 803641559 659660800 119362 33554432000 4194304 4625164 140735867741680 18446744073709551615 4348804 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5992/tid=5999] ppid=5990 vsize=644200 CPUtime=25.4 cores=0-7
/proc/5992/task/5999/stat : 5999 (SArTagnan) R 5990 5992 5966 0 -1 4202560 18767 0 0 0 2537 3 0 0 25 0 8 0 803641559 659660800 119362 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351987 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=5992/tid=6000] ppid=5990 vsize=644200 CPUtime=25.38 cores=0-7
/proc/5992/task/6000/stat : 6000 (SArTagnan) R 5990 5992 5966 0 -1 4202560 4617 0 0 0 2537 1 0 0 18 0 8 0 803641559 659660800 119362 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 202.91
Current children cumulated vsize (KiB) 644200

[startup+51.1226 s]
/proc/loadavg: 5.40 2.95 2.31 9/576 6002
/proc/meminfo: memFree=23341336/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=1061624 CPUtime=407.35 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 345254 0 1 0 40674 61 0 0 25 0 8 0 803641551 1087102976 166034 33554432000 4194304 4625164 140735867741680 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 265406 166034 709 106 0 260909 0
[pid=5992/tid=5994] ppid=5990 vsize=1061624 CPUtime=51.03 cores=0-7
/proc/5992/task/5994/stat : 5994 (SArTagnan) R 5990 5992 5966 0 -1 4202560 22218 0 0 0 5100 3 0 0 25 0 8 0 803641559 1087102976 166034 33554432000 4194304 4625164 140735867741680 18446744073709551615 4352157 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5992/tid=5995] ppid=5990 vsize=1061624 CPUtime=51.01 cores=0-7
/proc/5992/task/5995/stat : 5995 (SArTagnan) R 5990 5992 5966 0 -1 4202560 12083 0 0 0 5098 3 0 0 25 0 8 0 803641559 1087102976 166034 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=5992/tid=5996] ppid=5990 vsize=1061624 CPUtime=51.03 cores=0-7
/proc/5992/task/5996/stat : 5996 (SArTagnan) R 5990 5992 5966 0 -1 4202560 147135 0 0 0 5079 24 0 0 25 0 8 0 803641559 1087102976 166034 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351975 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5992/tid=5997] ppid=5990 vsize=1061624 CPUtime=51.02 cores=0-7
/proc/5992/task/5997/stat : 5997 (SArTagnan) R 5990 5992 5966 0 -1 4202560 22189 0 0 0 5098 4 0 0 25 0 8 0 803641559 1087102976 166034 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5992/tid=5998] ppid=5990 vsize=1061624 CPUtime=50.07 cores=0-7
/proc/5992/task/5998/stat : 5998 (SArTagnan) R 5990 5992 5966 0 -1 4202560 20280 0 0 0 5003 4 0 0 25 0 8 0 803641559 1087102976 166034 33554432000 4194304 4625164 140735867741680 18446744073709551615 4349019 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5992/tid=5999] ppid=5990 vsize=1061624 CPUtime=51.02 cores=0-7
/proc/5992/task/5999/stat : 5999 (SArTagnan) R 5990 5992 5966 0 -1 4202560 27909 0 0 0 5097 5 0 0 25 0 8 0 803641559 1087102976 166034 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=5992/tid=6000] ppid=5990 vsize=1061624 CPUtime=50.99 cores=0-7
/proc/5992/task/6000/stat : 6000 (SArTagnan) R 5990 5992 5966 0 -1 4202560 5116 0 0 0 5098 1 0 0 25 0 8 0 803641559 1087102976 166034 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 407.35
Current children cumulated vsize (KiB) 1061624

[startup+102.301 s]
/proc/loadavg: 6.87 3.73 2.61 9/576 6003
/proc/meminfo: memFree=23098772/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=1229484 CPUtime=815.78 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 806331 0 1 0 81442 136 0 0 25 0 8 0 803641551 1258991616 227268 33554432000 4194304 4625164 140735867741680 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 307371 227268 407 106 0 302874 0
[pid=5992/tid=5994] ppid=5990 vsize=1229484 CPUtime=102.21 cores=0-7
/proc/5992/task/5994/stat : 5994 (SArTagnan) R 5990 5992 5966 0 -1 4202560 33305 0 0 0 10216 5 0 0 25 0 8 0 803641559 1258991616 227268 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5992/tid=5995] ppid=5990 vsize=1229484 CPUtime=102.17 cores=0-7
/proc/5992/task/5995/stat : 5995 (SArTagnan) R 5990 5992 5966 0 -1 4202560 19243 0 0 0 10213 4 0 0 25 0 8 0 803641559 1258991616 227268 33554432000 4194304 4625164 140735867741680 18446744073709551615 4288870 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=5992/tid=5996] ppid=5990 vsize=1229484 CPUtime=102.2 cores=0-7
/proc/5992/task/5996/stat : 5996 (SArTagnan) R 5990 5992 5966 0 -1 4202560 516589 0 0 0 10139 81 0 0 25 0 8 0 803641559 1258991616 227268 33554432000 4194304 4625164 140735867741680 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 0 0 0 0

################
# More data... #
################

Current children cumulated vsize (KiB) 8063576

[startup+4722.3 s]
/proc/loadavg: 8.00 8.00 7.94 9/576 6138
/proc/meminfo: memFree=17679928/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=8063576 CPUtime=37681.8 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 39526849 0 1 0 3761845 6330 0 0 25 0 8 0 803641551 8257101824 1578357 33554432000 4194304 4625164 140735867741680 18446744073709551615 4393061 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 2015894 1578357 349 106 0 2011397 0
[pid=5992/tid=5994] ppid=5990 vsize=8063576 CPUtime=4722.35 cores=0-7
/proc/5992/task/5994/stat : 5994 (SArTagnan) R 5990 5992 5966 0 -1 4202560 174404 0 0 0 472198 37 0 0 25 0 8 0 803641559 8257101824 1578358 33554432000 4194304 4625164 140735867741680 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5992/tid=5995] ppid=5990 vsize=8063576 CPUtime=4720.63 cores=0-7
/proc/5992/task/5995/stat : 5995 (SArTagnan) R 5990 5992 5966 0 -1 4202560 340734 0 0 0 471784 279 0 0 25 0 8 0 803641559 8257101824 1578358 33554432000 4194304 4625164 140735867741680 18446744073709551615 212687336481 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=5992/tid=5996] ppid=5990 vsize=8063576 CPUtime=4722.1 cores=0-7
/proc/5992/task/5996/stat : 5996 (SArTagnan) R 5990 5992 5966 0 -1 4202560 34935115 0 0 0 466899 5311 0 0 25 0 8 0 803641559 8257101824 1578358 33554432000 4194304 4625164 140735867741680 18446744073709551615 4335407 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5992/tid=5997] ppid=5990 vsize=8063576 CPUtime=4722.29 cores=0-7
/proc/5992/task/5997/stat : 5997 (SArTagnan) R 5990 5992 5966 0 -1 4202560 181952 0 0 0 472189 40 0 0 25 0 8 0 803641559 8257101824 1578358 33554432000 4194304 4625164 140735867741680 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5992/tid=5998] ppid=5990 vsize=8063576 CPUtime=4628.16 cores=0-7
/proc/5992/task/5998/stat : 5998 (SArTagnan) R 5990 5992 5966 0 -1 4202560 117419 0 0 0 462789 27 0 0 25 0 8 0 803641559 8257101824 1578358 33554432000 4194304 4625164 140735867741680 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5992/tid=5999] ppid=5990 vsize=8063576 CPUtime=4722.31 cores=0-7
/proc/5992/task/5999/stat : 5999 (SArTagnan) R 5990 5992 5966 0 -1 4202560 177934 0 0 0 472187 44 0 0 25 0 8 0 803641559 8257101824 1578358 33554432000 4194304 4625164 140735867741680 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=5992/tid=6000] ppid=5990 vsize=8063576 CPUtime=4722.24 cores=0-7
/proc/5992/task/6000/stat : 6000 (SArTagnan) R 5990 5992 5966 0 -1 4202560 130800 0 0 0 472153 71 0 0 25 0 8 0 803641559 8257101824 1578358 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 37681.8
Current children cumulated vsize (KiB) 8063576

[startup+4782.3 s]
/proc/loadavg: 8.03 8.01 7.95 9/576 6140
/proc/meminfo: memFree=17629812/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=8202424 CPUtime=38160.5 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 39987198 0 1 0 3809649 6405 0 0 25 0 8 0 803641551 8399282176 1591130 33554432000 4194304 4625164 140735867741680 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 2050606 1591130 349 106 0 2046109 0
[pid=5992/tid=5994] ppid=5990 vsize=8202424 CPUtime=4782.35 cores=0-7
/proc/5992/task/5994/stat : 5994 (SArTagnan) R 5990 5992 5966 0 -1 4202560 174543 0 0 0 478198 37 0 0 25 0 8 0 803641559 8399282176 1591130 33554432000 4194304 4625164 140735867741680 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5992/tid=5995] ppid=5990 vsize=8202424 CPUtime=4780.61 cores=0-7
/proc/5992/task/5995/stat : 5995 (SArTagnan) R 5990 5992 5966 0 -1 4202560 344214 0 0 0 477778 283 0 0 25 0 8 0 803641559 8399282176 1591130 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=5992/tid=5996] ppid=5990 vsize=8202424 CPUtime=4782.1 cores=0-7
/proc/5992/task/5996/stat : 5996 (SArTagnan) R 5990 5992 5966 0 -1 4202560 35386303 0 0 0 472830 5380 0 0 25 0 8 0 803641559 8399282176 1591130 33554432000 4194304 4625164 140735867741680 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5992/tid=5997] ppid=5990 vsize=8202424 CPUtime=4782.3 cores=0-7
/proc/5992/task/5997/stat : 5997 (SArTagnan) R 5990 5992 5966 0 -1 4202560 182083 0 0 0 478189 41 0 0 25 0 8 0 803641559 8399282176 1591130 33554432000 4194304 4625164 140735867741680 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5992/tid=5998] ppid=5990 vsize=8202424 CPUtime=4686.96 cores=0-7
/proc/5992/task/5998/stat : 5998 (SArTagnan) R 5990 5992 5966 0 -1 4202560 117739 0 0 0 468668 28 0 0 25 0 8 0 803641559 8399282176 1591130 33554432000 4194304 4625164 140735867741680 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5992/tid=5999] ppid=5990 vsize=8202424 CPUtime=4782.31 cores=0-7
/proc/5992/task/5999/stat : 5999 (SArTagnan) R 5990 5992 5966 0 -1 4202560 178070 0 0 0 478187 44 0 0 25 0 8 0 803641559 8399282176 1591130 33554432000 4194304 4625164 140735867741680 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=5992/tid=6000] ppid=5990 vsize=8202424 CPUtime=4782.25 cores=0-7
/proc/5992/task/6000/stat : 6000 (SArTagnan) R 5990 5992 5966 0 -1 4202560 131306 0 0 0 478153 72 0 0 25 0 8 0 803641559 8399282176 1591130 33554432000 4194304 4625164 140735867741680 18446744073709551615 4352192 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38160.5
Current children cumulated vsize (KiB) 8202424

[startup+4842.3 s]
/proc/loadavg: 8.08 8.03 7.96 9/576 6142
/proc/meminfo: memFree=17585772/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=8269168 CPUtime=38639.4 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 40437240 0 1 0 3857458 6477 0 0 25 0 8 0 803641551 8467628032 1601743 33554432000 4194304 4625164 140735867741680 18446744073709551615 4393101 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 2067292 1601743 349 106 0 2062795 0
[pid=5992/tid=5994] ppid=5990 vsize=8269168 CPUtime=4842.35 cores=0-7
/proc/5992/task/5994/stat : 5994 (SArTagnan) R 5990 5992 5966 0 -1 4202560 174723 0 0 0 484198 37 0 0 25 0 8 0 803641559 8467628032 1601743 33554432000 4194304 4625164 140735867741680 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5992/tid=5995] ppid=5990 vsize=8269168 CPUtime=4840.6 cores=0-7
/proc/5992/task/5995/stat : 5995 (SArTagnan) R 5990 5992 5966 0 -1 4202560 346513 0 0 0 483774 286 0 0 25 0 8 0 803641559 8467628032 1601743 33554432000 4194304 4625164 140735867741680 18446744073709551615 4352280 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=5992/tid=5996] ppid=5990 vsize=8269168 CPUtime=4842.1 cores=0-7
/proc/5992/task/5996/stat : 5996 (SArTagnan) R 5990 5992 5966 0 -1 4202560 35825431 0 0 0 478763 5447 0 0 25 0 8 0 803641559 8467628032 1601743 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5992/tid=5997] ppid=5990 vsize=8269168 CPUtime=4842.3 cores=0-7
/proc/5992/task/5997/stat : 5997 (SArTagnan) R 5990 5992 5966 0 -1 4202560 182354 0 0 0 484189 41 0 0 25 0 8 0 803641559 8467628032 1601743 33554432000 4194304 4625164 140735867741680 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5992/tid=5998] ppid=5990 vsize=8269168 CPUtime=4745.77 cores=0-7
/proc/5992/task/5998/stat : 5998 (SArTagnan) R 5990 5992 5966 0 -1 4202560 117891 0 0 0 474549 28 0 0 25 0 8 0 803641559 8467628032 1601743 33554432000 4194304 4625164 140735867741680 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5992/tid=5999] ppid=5990 vsize=8269168 CPUtime=4842.31 cores=0-7
/proc/5992/task/5999/stat : 5999 (SArTagnan) R 5990 5992 5966 0 -1 4202560 178814 0 0 0 484187 44 0 0 25 0 8 0 803641559 8467628032 1601743 33554432000 4194304 4625164 140735867741680 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=5992/tid=6000] ppid=5990 vsize=8269168 CPUtime=4842.25 cores=0-7
/proc/5992/task/6000/stat : 6000 (SArTagnan) R 5990 5992 5966 0 -1 4202560 131383 0 0 0 484153 72 0 0 25 0 8 0 803641559 8467628032 1601743 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38639.3
Current children cumulated vsize (KiB) 8269168

[startup+4902.3 s]
/proc/loadavg: 8.05 8.04 7.96 9/576 6143
/proc/meminfo: memFree=17566524/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=8270632 CPUtime=39118.1 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 40713527 0 1 0 3905292 6521 0 0 25 0 8 0 803641551 8469127168 1608238 33554432000 4194304 4625164 140735867741680 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 2067658 1608238 349 106 0 2063161 0
[pid=5992/tid=5994] ppid=5990 vsize=8270632 CPUtime=4902.35 cores=0-7
/proc/5992/task/5994/stat : 5994 (SArTagnan) R 5990 5992 5966 0 -1 4202560 174895 0 0 0 490198 37 0 0 25 0 8 0 803641559 8469127168 1608238 33554432000 4194304 4625164 140735867741680 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5992/tid=5995] ppid=5990 vsize=8270632 CPUtime=4900.57 cores=0-7
/proc/5992/task/5995/stat : 5995 (SArTagnan) R 5990 5992 5966 0 -1 4202560 346551 0 0 0 489771 286 0 0 25 0 8 0 803641559 8469127168 1608238 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351844 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=5992/tid=5996] ppid=5990 vsize=8270632 CPUtime=4902.1 cores=0-7
/proc/5992/task/5996/stat : 5996 (SArTagnan) R 5990 5992 5966 0 -1 4202560 36061762 0 0 0 484726 5484 0 0 25 0 8 0 803641559 8469127168 1608238 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5992/tid=5997] ppid=5990 vsize=8270632 CPUtime=4902.29 cores=0-7
/proc/5992/task/5997/stat : 5997 (SArTagnan) R 5990 5992 5966 0 -1 4202560 182689 0 0 0 490188 41 0 0 25 0 8 0 803641559 8469127168 1608238 33554432000 4194304 4625164 140735867741680 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5992/tid=5998] ppid=5990 vsize=8270632 CPUtime=4804.56 cores=0-7
/proc/5992/task/5998/stat : 5998 (SArTagnan) R 5990 5992 5966 0 -1 4202560 118182 0 0 0 480428 28 0 0 25 0 8 0 803641559 8469127168 1608238 33554432000 4194304 4625164 140735867741680 18446744073709551615 4460412 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5992/tid=5999] ppid=5990 vsize=8270632 CPUtime=4902.31 cores=0-7
/proc/5992/task/5999/stat : 5999 (SArTagnan) R 5990 5992 5966 0 -1 4202560 179012 0 0 0 490187 44 0 0 25 0 8 0 803641559 8469127168 1608238 33554432000 4194304 4625164 140735867741680 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=5992/tid=6000] ppid=5990 vsize=8270632 CPUtime=4902.25 cores=0-7
/proc/5992/task/6000/stat : 6000 (SArTagnan) R 5990 5992 5966 0 -1 4202560 131462 0 0 0 490153 72 0 0 25 0 8 0 803641559 8469127168 1608238 33554432000 4194304 4625164 140735867741680 18446744073709551615 4464328 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39118.1
Current children cumulated vsize (KiB) 8270632

[startup+4962.3 s]
/proc/loadavg: 8.02 8.03 7.96 9/576 6145
/proc/meminfo: memFree=17523100/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=8290564 CPUtime=39596.9 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 41323825 0 1 0 3953075 6619 0 0 25 0 8 0 803641551 8489537536 1615764 33554432000 4194304 4625164 140735867741680 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 2072641 1615764 349 106 0 2068144 0
[pid=5992/tid=5994] ppid=5990 vsize=8290564 CPUtime=4962.35 cores=0-7
/proc/5992/task/5994/stat : 5994 (SArTagnan) R 5990 5992 5966 0 -1 4202560 175055 0 0 0 496198 37 0 0 25 0 8 0 803641559 8489537536 1615764 33554432000 4194304 4625164 140735867741680 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5992/tid=5995] ppid=5990 vsize=8290564 CPUtime=4960.56 cores=0-7
/proc/5992/task/5995/stat : 5995 (SArTagnan) R 5990 5992 5966 0 -1 4202560 352156 0 0 0 495767 289 0 0 25 0 8 0 803641559 8489537536 1615764 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=5992/tid=5996] ppid=5990 vsize=8290564 CPUtime=4962.11 cores=0-7
/proc/5992/task/5996/stat : 5996 (SArTagnan) R 5990 5992 5966 0 -1 4202560 36630351 0 0 0 490637 5574 0 0 25 0 8 0 803641559 8489537536 1615764 33554432000 4194304 4625164 140735867741680 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5992/tid=5997] ppid=5990 vsize=8290564 CPUtime=4962.3 cores=0-7
/proc/5992/task/5997/stat : 5997 (SArTagnan) R 5990 5992 5966 0 -1 4202560 182940 0 0 0 496189 41 0 0 25 0 8 0 803641559 8489537536 1615765 33554432000 4194304 4625164 140735867741680 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5992/tid=5998] ppid=5990 vsize=8290564 CPUtime=4863.37 cores=0-7
/proc/5992/task/5998/stat : 5998 (SArTagnan) R 5990 5992 5966 0 -1 4202560 118383 0 0 0 486309 28 0 0 25 0 8 0 803641559 8489537536 1615765 33554432000 4194304 4625164 140735867741680 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5992/tid=5999] ppid=5990 vsize=8290564 CPUtime=4962.31 cores=0-7
/proc/5992/task/5999/stat : 5999 (SArTagnan) R 5990 5992 5966 0 -1 4202560 179556 0 0 0 496186 45 0 0 25 0 8 0 803641559 8489537536 1615765 33554432000 4194304 4625164 140735867741680 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=5992/tid=6000] ppid=5990 vsize=8290564 CPUtime=4962.25 cores=0-7
/proc/5992/task/6000/stat : 6000 (SArTagnan) R 5990 5992 5966 0 -1 4202560 131591 0 0 0 496153 72 0 0 25 0 8 0 803641559 8489537536 1615765 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39596.9
Current children cumulated vsize (KiB) 8290564



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5012.9 s]
/proc/loadavg: 8.01 8.02 7.96 9/576 6146
/proc/meminfo: memFree=17491832/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=8290564 CPUtime=40000.8 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) R 5990 5992 5966 0 -1 4202496 41735961 0 1 0 3993394 6684 0 0 25 0 8 0 803641551 8489537536 1624846 33554432000 4194304 4625164 140735867741680 18446744073709551615 4392981 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 2072641 1624846 349 106 0 2068144 0
[pid=5992/tid=5994] ppid=5990 vsize=8290564 CPUtime=5012.98 cores=0-7
/proc/5992/task/5994/stat : 5994 (SArTagnan) R 5990 5992 5966 0 -1 4202560 175247 0 0 0 501260 38 0 0 25 0 8 0 803641559 8489537536 1622958 33554432000 4194304 4625164 140735867741680 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5992/tid=5995] ppid=5990 vsize=8290564 CPUtime=5011.17 cores=0-7
/proc/5992/task/5995/stat : 5995 (SArTagnan) R 5990 5992 5966 0 -1 4202560 353398 0 0 0 500827 290 0 0 25 0 8 0 803641559 8489537536 1622958 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=5992/tid=5996] ppid=5990 vsize=8290564 CPUtime=5012.73 cores=0-7
/proc/5992/task/5996/stat : 5996 (SArTagnan) R 5990 5992 5966 0 -1 4202560 37011476 0 0 0 495640 5633 0 0 25 0 8 0 803641559 8489537536 1622958 33554432000 4194304 4625164 140735867741680 18446744073709551615 4547606 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5992/tid=5997] ppid=5990 vsize=8290564 CPUtime=5012.92 cores=0-7
/proc/5992/task/5997/stat : 5997 (SArTagnan) R 5990 5992 5966 0 -1 4202560 183073 0 0 0 501251 41 0 0 25 0 8 0 803641559 8489537536 1622958 33554432000 4194304 4625164 140735867741680 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5992/tid=5998] ppid=5990 vsize=8290564 CPUtime=4912.96 cores=0-7
/proc/5992/task/5998/stat : 5998 (SArTagnan) R 5990 5992 5966 0 -1 4202560 118553 0 0 0 491268 28 0 0 25 0 8 0 803641559 8489537536 1622958 33554432000 4194304 4625164 140735867741680 18446744073709551615 4329673 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5992/tid=5999] ppid=5990 vsize=8290564 CPUtime=5012.93 cores=0-7
/proc/5992/task/5999/stat : 5999 (SArTagnan) R 5990 5992 5966 0 -1 4202560 179900 0 0 0 501248 45 0 0 25 0 8 0 803641559 8489537536 1622958 33554432000 4194304 4625164 140735867741680 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=5992/tid=6000] ppid=5990 vsize=8290564 CPUtime=5012.88 cores=0-7
/proc/5992/task/6000/stat : 6000 (SArTagnan) R 5990 5992 5966 0 -1 4202560 131625 0 0 0 501216 72 0 0 25 0 8 0 803641559 8489537536 1622958 33554432000 4194304 4625164 140735867741680 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 8290564

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5013.03 s]
/proc/loadavg: 8.01 8.02 7.96 9/576 6146
/proc/meminfo: memFree=17491832/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=0 CPUtime=40001 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) Z 5990 5992 5966 0 -1 4203548 41736072 0 1 0 3993408 6694 0 0 25 0 2 0 803641551 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 0 0 0 0 0 0 0
[pid=5992/tid=5998] ppid=5990 vsize=0 CPUtime=4913.07 cores=0-7
/proc/5992/task/5998/stat : 5998 (SArTagnan) R 5990 5992 5966 0 -1 4203588 118553 0 0 0 491269 38 0 0 25 0 2 0 803641559 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40001
Current children cumulated vsize (KiB) 0

[startup+5013.22 s]
/proc/loadavg: 8.01 8.02 7.96 2/571 6147
/proc/meminfo: memFree=20634776/32951124 swapFree=58940260/67111528
[pid=5992] ppid=5990 vsize=0 CPUtime=40001.2 cores=0-7
/proc/5992/stat : 5992 (SArTagnan) Z 5990 5992 5966 0 -1 4203548 41736072 0 1 0 3993408 6713 0 0 25 0 2 0 803641551 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5992/statm: 0 0 0 0 0 0 0
[pid=5992/tid=5998] ppid=5990 vsize=0 CPUtime=4913.27 cores=0-7
/proc/5992/task/5998/stat : 5998 (SArTagnan) R 5990 5992 5966 0 -1 4203588 118553 0 0 0 491269 58 0 0 25 0 2 0 803641559 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40001.2
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5013.25
CPU time (s): 40001.2
CPU user time (s): 39934.1
CPU system time (s): 67.1618
CPU usage (%): 797.911
Max. virtual memory (cumulated for all children) (KiB): 10222468

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39934.1
system time used= 67.1618
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 41736072
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 12740
involuntary context switches= 168217

runsolver used 27.5358 second user time and 72.494 second system time

The end

Launcher Data 

Begin job on node139 at 2011-06-01 11:27:18
IDJOB=3438596
IDBENCH=71612
IDSOLVER=1936
FILE ID=node139/3438596-1306920438
RUNJOBID= node139-1306920438-5980
PBS_JOBID= 13488547
Free space on /tmp= 73220 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/forced/rbsat-v2400c266431gyes2.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3438596-1306920438/watcher-3438596-1306920438 -o /tmp/evaluation-result-3438596-1306920438/solver-3438596-1306920438 -C 40000 -W 5100 -M 31000  SArTagnan HOME/instance-3438596-1306920438.cnf -threads=8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= aac2bd287b8cd6b96cbf7ecc463acf19
RANDOM SEED=1259204750

node139.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5321.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      24005852 kB
Buffers:        126940 kB
Cached:         313544 kB
SwapCached:       3628 kB
Active:        8347016 kB
Inactive:       297700 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      24005852 kB
SwapTotal:    67111528 kB
SwapFree:     58940260 kB
Dirty:            3924 kB
Writeback:          28 kB
AnonPages:     8202468 kB
Mapped:          32200 kB
Slab:           148804 kB
PageTables:      78648 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 170869204 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73220 MiB
End job on node139 at 2011-06-01 12:50:53