Trace number 3439796

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 40001 5013.98

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m52-q1041.25-n50-p15-s99099953.cnf
MD5SUMdda220d08783aabf4e7a9e99eeeccae6
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2444
Number of clauses498568
Sum of the clauses size999480
Maximum clause length52
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2498518
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 550

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3439796-1306965836/watcher-3439796-1306965836 -o /tmp/evaluation-result-3439796-1306965836/solver-3439796-1306965836 -C 40000 -W 5100 -M 31000 SArTagnan HOME/instance-3439796-1306965836.cnf -threads=8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.44 7.95 7.97 1/510 26336
/proc/meminfo: memFree=20834672/32950928 swapFree=34979748/67111528
[pid=26336] ppid=26334 vsize=31612 CPUtime=0.01 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 1176 0 0 0 1 0 0 0 24 0 1 0 808180331 32370688 668 33554432000 4194304 4625164 140737106952048 18446744073709551615 4339447 0 0 4096 2 0 0 0 17 4 0 0 0
/proc/26336/statm: 7903 668 296 106 0 3406 0

[startup+0.099032 s]
/proc/loadavg: 7.44 7.95 7.97 1/510 26336
/proc/meminfo: memFree=20834672/32950928 swapFree=34979748/67111528
[pid=26336] ppid=26334 vsize=43524 CPUtime=0.09 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 5235 0 0 0 9 0 0 0 24 0 1 0 808180331 44568576 2652 33554432000 4194304 4625164 140737106952048 18446744073709551615 254500389333 0 0 4096 2 0 0 0 17 4 0 0 0
/proc/26336/statm: 10881 2675 296 106 0 6384 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 43524

[startup+0.101028 s]
/proc/loadavg: 7.44 7.95 7.97 1/510 26336
/proc/meminfo: memFree=20834672/32950928 swapFree=34979748/67111528
[pid=26336] ppid=26334 vsize=38352 CPUtime=0.09 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 5640 0 0 0 9 0 0 0 24 0 1 0 808180331 39272448 1764 33554432000 4194304 4625164 140737106952048 18446744073709551615 254500691712 0 0 4096 2 0 0 0 17 4 0 0 0
/proc/26336/statm: 9588 1765 296 106 0 5091 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 38352

[startup+0.558424 s]
/proc/loadavg: 7.44 7.95 7.97 1/510 26336
/proc/meminfo: memFree=20834672/32950928 swapFree=34979748/67111528
[pid=26336] ppid=26334 vsize=568200 CPUtime=5.35 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 95794 0 0 0 523 12 0 0 18 0 8 0 808180331 581836800 63404 33554432000 4194304 4625164 140737106952048 18446744073709551615 254500353057 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/26336/statm: 163192 63404 555 106 0 158695 0
Current children cumulated CPU time (s) 5.35
Current children cumulated vsize (KiB) 568200

[startup+0.913471 s]
/proc/loadavg: 7.44 7.95 7.97 1/510 26336
/proc/meminfo: memFree=20834672/32950928 swapFree=34979748/67111528
[pid=26336] ppid=26334 vsize=603628 CPUtime=6.7 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 112668 0 0 0 654 16 0 0 18 0 8 0 808180331 618115072 65684 33554432000 4194304 4625164 140737106952048 18446744073709551615 254500735159 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/26336/statm: 186504 91472 526 106 0 182007 0
Current children cumulated CPU time (s) 6.7
Current children cumulated vsize (KiB) 603628

[startup+1.6452 s]
/proc/loadavg: 7.44 7.95 7.97 1/510 26336
/proc/meminfo: memFree=20834672/32950928 swapFree=34979748/67111528
[pid=26336] ppid=26334 vsize=737108 CPUtime=12.16 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 171200 0 0 0 1189 27 0 0 18 0 8 0 808180331 754798592 97711 33554432000 4194304 4625164 140737106952048 18446744073709551615 254500353057 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/26336/statm: 184277 97713 564 106 0 179780 0
Current children cumulated CPU time (s) 12.16
Current children cumulated vsize (KiB) 737108

[startup+3.10035 s]
/proc/loadavg: 7.49 7.95 7.97 9/518 26344
/proc/meminfo: memFree=20329984/32950928 swapFree=34979748/67111528
[pid=26336] ppid=26334 vsize=818756 CPUtime=23.75 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 207683 0 0 0 2339 36 0 0 18 0 8 0 808180331 838406144 126275 33554432000 4194304 4625164 140737106952048 18446744073709551615 4360345 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/26336/statm: 204689 126275 819 106 0 200192 0
[pid=26336/tid=26338] ppid=26334 vsize=818756 CPUtime=2.96 cores=0-7
/proc/26336/task/26338/stat : 26338 (SArTagnan) R 26334 26336 26075 0 -1 4202560 8588 0 0 0 294 2 0 0 18 0 8 0 808180345 838406144 126275 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=26336/tid=26339] ppid=26334 vsize=818756 CPUtime=2.92 cores=0-7
/proc/26336/task/26339/stat : 26339 (SArTagnan) R 26334 26336 26075 0 -1 4202560 10428 0 0 0 290 2 0 0 18 0 8 0 808180345 838406144 126275 33554432000 4194304 4625164 140737106952048 18446744073709551615 4292266 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=26336/tid=26340] ppid=26334 vsize=818756 CPUtime=2.92 cores=0-7
/proc/26336/task/26340/stat : 26340 (SArTagnan) R 26334 26336 26075 0 -1 4202560 11531 0 0 0 291 1 0 0 18 0 8 0 808180345 838406144 126275 33554432000 4194304 4625164 140737106952048 18446744073709551615 4547614 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=26336/tid=26341] ppid=26334 vsize=818756 CPUtime=2.96 cores=0-7
/proc/26336/task/26341/stat : 26341 (SArTagnan) R 26334 26336 26075 0 -1 4202560 10793 0 0 0 294 2 0 0 18 0 8 0 808180345 838406144 126275 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=26336/tid=26342] ppid=26334 vsize=818756 CPUtime=2.96 cores=0-7
/proc/26336/task/26342/stat : 26342 (SArTagnan) R 26334 26336 26075 0 -1 4202560 9063 0 0 0 295 1 0 0 18 0 8 0 808180345 838406144 126275 33554432000 4194304 4625164 140737106952048 18446744073709551615 4349036 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=26336/tid=26343] ppid=26334 vsize=818756 CPUtime=2.96 cores=0-7
/proc/26336/task/26343/stat : 26343 (SArTagnan) R 26334 26336 26075 0 -1 4202560 8887 0 0 0 294 2 0 0 18 0 8 0 808180345 838406144 126275 33554432000 4194304 4625164 140737106952048 18446744073709551615 4375311 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=26336/tid=26344] ppid=26334 vsize=818756 CPUtime=2.96 cores=0-7
/proc/26336/task/26344/stat : 26344 (SArTagnan) R 26334 26336 26075 0 -1 4202560 8503 0 0 0 295 1 0 0 18 0 8 0 808180345 838406144 126275 33554432000 4194304 4625164 140737106952048 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 23.75
Current children cumulated vsize (KiB) 818756

[startup+6.30072 s]
/proc/loadavg: 7.49 7.95 7.97 9/518 26344
/proc/meminfo: memFree=20264160/32950928 swapFree=34979748/67111528
[pid=26336] ppid=26334 vsize=963620 CPUtime=49.15 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 276988 0 0 0 4870 45 0 0 24 0 8 0 808180331 986746880 152733 33554432000 4194304 4625164 140737106952048 18446744073709551615 4511874 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/26336/statm: 240905 152733 816 106 0 236408 0
[pid=26336/tid=26338] ppid=26334 vsize=963620 CPUtime=6.14 cores=0-7
/proc/26336/task/26338/stat : 26338 (SArTagnan) R 26334 26336 26075 0 -1 4202560 9263 0 0 0 612 2 0 0 25 0 8 0 808180345 986746880 152733 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=26336/tid=26339] ppid=26334 vsize=963620 CPUtime=6.11 cores=0-7
/proc/26336/task/26339/stat : 26339 (SArTagnan) R 26334 26336 26075 0 -1 4202560 13542 0 0 0 608 3 0 0 23 0 8 0 808180345 986746880 152733 33554432000 4194304 4625164 140737106952048 18446744073709551615 4352296 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=26336/tid=26340] ppid=26334 vsize=963620 CPUtime=6.06 cores=0-7
/proc/26336/task/26340/stat : 26340 (SArTagnan) R 26334 26336 26075 0 -1 4202560 18531 0 0 0 604 2 0 0 25 0 8 0 808180345 986746880 152733 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=26336/tid=26341] ppid=26334 vsize=963620 CPUtime=6.14 cores=0-7
/proc/26336/task/26341/stat : 26341 (SArTagnan) R 26334 26336 26075 0 -1 4202560 15772 0 0 0 611 3 0 0 25 0 8 0 808180345 986746880 152733 33554432000 4194304 4625164 140737106952048 18446744073709551615 4345252 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=26336/tid=26342] ppid=26334 vsize=963620 CPUtime=6.14 cores=0-7
/proc/26336/task/26342/stat : 26342 (SArTagnan) R 26334 26336 26075 0 -1 4202560 16718 0 0 0 612 2 0 0 25 0 8 0 808180345 986746880 152733 33554432000 4194304 4625164 140737106952048 18446744073709551615 4348903 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=26336/tid=26343] ppid=26334 vsize=963620 CPUtime=6.14 cores=0-7
/proc/26336/task/26343/stat : 26343 (SArTagnan) R 26334 26336 26075 0 -1 4202560 16101 0 0 0 611 3 0 0 25 0 8 0 808180345 986746880 152733 33554432000 4194304 4625164 140737106952048 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=26336/tid=26344] ppid=26334 vsize=963620 CPUtime=6.14 cores=0-7
/proc/26336/task/26344/stat : 26344 (SArTagnan) R 26334 26336 26075 0 -1 4202560 8933 0 0 0 613 1 0 0 25 0 8 0 808180345 986746880 152733 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 49.15
Current children cumulated vsize (KiB) 963620

[startup+12.7004 s]
/proc/loadavg: 7.53 7.95 7.97 9/518 26344
/proc/meminfo: memFree=20036172/32950928 swapFree=34979748/67111528
[pid=26336] ppid=26334 vsize=1268536 CPUtime=100.23 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 404203 0 0 0 9959 64 0 0 25 0 8 0 808180331 1298980864 205084 33554432000 4194304 4625164 140737106952048 18446744073709551615 4393061 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/26336/statm: 317134 205084 738 106 0 312637 0
[pid=26336/tid=26338] ppid=26334 vsize=1268536 CPUtime=12.54 cores=0-7
/proc/26336/task/26338/stat : 26338 (SArTagnan) R 26334 26336 26075 0 -1 4202560 15317 0 0 0 1251 3 0 0 25 0 8 0 808180345 1298980864 205084 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351887 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=26336/tid=26339] ppid=26334 vsize=1268536 CPUtime=12.5 cores=0-7
/proc/26336/task/26339/stat : 26339 (SArTagnan) R 26334 26336 26075 0 -1 4202560 15168 0 0 0 1247 3 0 0 25 0 8 0 808180345 1298980864 205084 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=26336/tid=26340] ppid=26334 vsize=1268536 CPUtime=12.35 cores=0-7
/proc/26336/task/26340/stat : 26340 (SArTagnan) R 26334 26336 26075 0 -1 4202560 37454 0 0 0 1229 6 0 0 25 0 8 0 808180345 1298980864 205084 33554432000 4194304 4625164 140737106952048 18446744073709551615 4328194 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=26336/tid=26341] ppid=26334 vsize=1268536 CPUtime=12.54 cores=0-7
/proc/26336/task/26341/stat : 26341 (SArTagnan) R 26334 26336 26075 0 -1 4202560 18836 0 0 0 1251 3 0 0 25 0 8 0 808180345 1298980864 205084 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=26336/tid=26342] ppid=26334 vsize=1268536 CPUtime=12.55 cores=0-7
/proc/26336/task/26342/stat : 26342 (SArTagnan) R 26334 26336 26075 0 -1 4202560 26346 0 0 0 1251 4 0 0 25 0 8 0 808180345 1298980864 205084 33554432000 4194304 4625164 140737106952048 18446744073709551615 254500350389 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=26336/tid=26343] ppid=26334 vsize=1268536 CPUtime=12.53 cores=0-7
/proc/26336/task/26343/stat : 26343 (SArTagnan) R 26334 26336 26075 0 -1 4202560 25044 0 0 0 1249 4 0 0 25 0 8 0 808180345 1298980864 205084 33554432000 4194304 4625164 140737106952048 18446744073709551615 4352005 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=26336/tid=26344] ppid=26334 vsize=1268536 CPUtime=12.54 cores=0-7
/proc/26336/task/26344/stat : 26344 (SArTagnan) R 26334 26336 26075 0 -1 4202560 12486 0 0 0 1252 2 0 0 25 0 8 0 808180345 1298980864 205084 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 100.23
Current children cumulated vsize (KiB) 1268536

[startup+25.5009 s]
/proc/loadavg: 7.63 7.96 7.97 9/518 26345
/proc/meminfo: memFree=19551072/32950928 swapFree=34979748/67111528
[pid=26336] ppid=26334 vsize=1894728 CPUtime=202.33 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 804173 0 0 0 20112 121 0 0 25 0 8 0 808180331 1940201472 323883 33554432000 4194304 4625164 140737106952048 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/26336/statm: 473682 323883 1517 106 0 469185 0
[pid=26336/tid=26338] ppid=26334 vsize=1894728 CPUtime=25.33 cores=0-7
/proc/26336/task/26338/stat : 26338 (SArTagnan) R 26334 26336 26075 0 -1 4202560 24994 0 0 0 2528 5 0 0 25 0 8 0 808180345 1940201472 323883 33554432000 4194304 4625164 140737106952048 18446744073709551615 4469816 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=26336/tid=26339] ppid=26334 vsize=1894728 CPUtime=25.3 cores=0-7
/proc/26336/task/26339/stat : 26339 (SArTagnan) R 26334 26336 26075 0 -1 4202560 25484 0 0 0 2525 5 0 0 25 0 8 0 808180345 1940201472 323883 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=26336/tid=26340] ppid=26334 vsize=1894728 CPUtime=25.1 cores=0-7
/proc/26336/task/26340/stat : 26340 (SArTagnan) R 26334 26336 26075 0 -1 4202560 129742 0 0 0 2489 21 0 0 25 0 8 0 808180345 1940201472 323883 33554432000 4194304 4625164 140737106952048 18446744073709551615 4334268 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=26336/tid=26341] ppid=26334 vsize=1894728 CPUtime=25.14 cores=0-7
/proc/26336/task/26341/stat : 26341 (SArTagnan) R 26334 26336 26075 0 -1 4202560 28022 0 0 0 2509 5 0 0 25 0 8 0 808180345 1940201472 323883 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=26336/tid=26342] ppid=26334 vsize=1894728 CPUtime=25.32 cores=0-7
/proc/26336/task/26342/stat : 26342 (SArTagnan) R 26334 26336 26075 0 -1 4202560 39653 0 0 0 2526 6 0 0 25 0 8 0 808180345 1940201472 323883 33554432000 4194304 4625164 140737106952048 18446744073709551615 4349019 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=26336/tid=26343] ppid=26334 vsize=1894728 CPUtime=25.31 cores=0-7
/proc/26336/task/26343/stat : 26343 (SArTagnan) R 26334 26336 26075 0 -1 4202560 38380 0 0 0 2525 6 0 0 25 0 8 0 808180345 1940201472 323883 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351998 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=26336/tid=26344] ppid=26334 vsize=1894728 CPUtime=25.33 cores=0-7
/proc/26336/task/26344/stat : 26344 (SArTagnan) R 26334 26336 26075 0 -1 4202560 15136 0 0 0 2530 3 0 0 18 0 8 0 808180345 1940201472 323883 33554432000 4194304 4625164 140737106952048 18446744073709551615 4353820 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 202.33
Current children cumulated vsize (KiB) 1894728

[startup+51.1008 s]
/proc/loadavg: 7.98 8.01 7.99 10/519 26348
/proc/meminfo: memFree=19209788/32950928 swapFree=34979748/67111528
[pid=26336] ppid=26334 vsize=2124148 CPUtime=406.39 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 1059936 0 0 0 40471 168 0 0 25 0 8 0 808180331 2175127552 403356 33554432000 4194304 4625164 140737106952048 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/26336/statm: 531037 403356 941 106 0 526540 0
[pid=26336/tid=26338] ppid=26334 vsize=2124148 CPUtime=50.93 cores=0-7
/proc/26336/task/26338/stat : 26338 (SArTagnan) R 26334 26336 26075 0 -1 4202560 40365 0 0 0 5086 7 0 0 25 0 8 0 808180345 2175127552 403357 33554432000 4194304 4625164 140737106952048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=26336/tid=26339] ppid=26334 vsize=2124148 CPUtime=50.79 cores=0-7
/proc/26336/task/26339/stat : 26339 (SArTagnan) R 26334 26336 26075 0 -1 4202560 34151 0 0 0 5068 11 0 0 25 0 8 0 808180345 2175127552 403358 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351987 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=26336/tid=26340] ppid=26334 vsize=2124148 CPUtime=50.69 cores=0-7
/proc/26336/task/26340/stat : 26340 (SArTagnan) R 26334 26336 26075 0 -1 4202560 262514 0 0 0 5027 42 0 0 25 0 8 0 808180345 2175127552 403358 33554432000 4194304 4625164 140737106952048 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=26336/tid=26341] ppid=26334 vsize=2124148 CPUtime=50.12 cores=0-7
/proc/26336/task/26341/stat : 26341 (SArTagnan) R 26334 26336 26075 0 -1 4202560 39079 0 0 0 5005 7 0 0 25 0 8 0 808180345 2175127552 403360 33554432000 4194304 4625164 140737106952048 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=26336/tid=26342] ppid=26334 vsize=2124148 CPUtime=50.92 cores=0-7
/proc/26336/task/26342/stat : 26342 (SArTagnan) R 26334 26336 26075 0 -1 4202560 46179 0 0 0 5084 8 0 0 25 0 8 0 808180345 2175127552 403360 33554432000 4194304 4625164 140737106952048 18446744073709551615 4330074 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=26336/tid=26343] ppid=26334 vsize=2124148 CPUtime=50.91 cores=0-7
/proc/26336/task/26343/stat : 26343 (SArTagnan) R 26334 26336 26075 0 -1 4202560 53823 0 0 0 5082 9 0 0 25 0 8 0 808180345 2175127552 403363 33554432000 4194304 4625164 140737106952048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=26336/tid=26344] ppid=26334 vsize=2124148 CPUtime=50.93 cores=0-7
/proc/26336/task/26344/stat : 26344 (SArTagnan) R 26334 26336 26075 0 -1 4202560 22441 0 0 0 5088 5 0 0 25 0 8 0 808180345 2175127552 403364 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 406.39
Current children cumulated vsize (KiB) 2124148

[startup+102.301 s]
/proc/loadavg: 8.11 8.05 8.00 9/518 26350
/proc/meminfo: memFree=18835876/32950928 swapFree=34979748/67111528
[pid=26336] ppid=26334 vsize=2642080 CPUtime=814.46 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 1568515 0 0 0 81181 265 0 0 25 0 8 0 808180331 2705489920 499797 33554432000 4194304 4625164 140737106952048 18446744073709551615 4393112 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/26336/statm: 660520 499797 1014 106 0 656023 0
[pid=26336/tid=26338] ppid=26334 vsize=2642080 CPUtime=102.13 cores=0-7
/proc/26336/task/26338/stat : 26338 (SArTagnan) R 26334 26336 26075 0 -1 4202560 42070 0 0 0 10205 8 0 0 25 0 8 0 808180345 2705489920 499797 33554432000 4194304 4625164 140737106952048 18446744073709551615 254500796959 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=26336/tid=26339] ppid=26334 vsize=2642080 CPUtime=101.96 cores=0-7
/proc/26336/task/26339/stat : 26339 (SArTagnan) R 26334 26336 26075 0 -1 4202560 56030 0 0 0 10168 28 0 0 25 0 8 0 808180345 2705489920 499797 33554432000 4194304 4625164 140737106952048 18446744073709551615 254500796959 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=26336/tid=26340] ppid=26334 vsize=2642080 CPUtime=101.35 cores=0-7
/proc/26336/task/26340/stat : 26340 (SArTagnan) R 26334 26336 26075 0 -1 4202560 496365 0 0 0 10052 83 0 0 25 0 8 0 808180345 2705489920 499797 33554432000 4194304 4625164 140737106952048 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 5 0 0 0

################
# More data... #
################

Current children cumulated vsize (KiB) 20907268

[startup+4722.32 s]
/proc/loadavg: 8.05 8.06 8.04 9/518 26485
/proc/meminfo: memFree=5200868/32950928 swapFree=34259960/67111528
[pid=26336] ppid=26334 vsize=20907268 CPUtime=37677.6 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 41310784 0 0 0 3759647 8110 0 0 25 0 8 0 808180331 21409042432 4072110 33554432000 4194304 4625164 140737106952048 18446744073709551615 4393089 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 241
/proc/26336/statm: 5226817 4072110 349 106 0 5222320 0
[pid=26336/tid=26338] ppid=26334 vsize=20907268 CPUtime=4720.91 cores=0-7
/proc/26336/task/26338/stat : 26338 (SArTagnan) R 26334 26336 26075 0 -1 4202560 350935 0 0 0 472025 66 0 0 25 0 8 0 808180345 21409042432 4072110 33554432000 4194304 4625164 140737106952048 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 3 0 0 3
[pid=26336/tid=26339] ppid=26334 vsize=20907268 CPUtime=4658.22 cores=0-7
/proc/26336/task/26339/stat : 26339 (SArTagnan) R 26334 26336 26075 0 -1 4202560 978139 0 0 0 464939 883 0 0 25 0 8 0 808180345 21409042432 4072110 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 5
[pid=26336/tid=26340] ppid=26334 vsize=20907268 CPUtime=4713.33 cores=0-7
/proc/26336/task/26340/stat : 26340 (SArTagnan) R 26334 26336 26075 0 -1 4202560 26770799 0 0 0 466517 4816 0 0 25 0 8 0 808180345 21409042432 4072110 33554432000 4194304 4625164 140737106952048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 7 0 0 148
[pid=26336/tid=26341] ppid=26334 vsize=20907268 CPUtime=4713.75 cores=0-7
/proc/26336/task/26341/stat : 26341 (SArTagnan) R 26334 26336 26075 0 -1 4202560 441520 0 0 0 471297 78 0 0 25 0 8 0 808180345 21409042432 4072110 33554432000 4194304 4625164 140737106952048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=26336/tid=26342] ppid=26334 vsize=20907268 CPUtime=4721.57 cores=0-7
/proc/26336/task/26342/stat : 26342 (SArTagnan) R 26334 26336 26075 0 -1 4202560 339157 0 0 0 472087 70 0 0 25 0 8 0 808180345 21409042432 4072110 33554432000 4194304 4625164 140737106952048 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 8
[pid=26336/tid=26343] ppid=26334 vsize=20907268 CPUtime=4720.76 cores=0-7
/proc/26336/task/26343/stat : 26343 (SArTagnan) R 26334 26336 26075 0 -1 4202560 453864 0 0 0 471979 97 0 0 25 0 8 0 808180345 21409042432 4072110 33554432000 4194304 4625164 140737106952048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=26336/tid=26344] ppid=26334 vsize=20907268 CPUtime=4711.53 cores=0-7
/proc/26336/task/26344/stat : 26344 (SArTagnan) R 26334 26336 26075 0 -1 4202560 185282 0 0 0 471060 93 0 0 25 0 8 0 808180345 21409042432 4072110 33554432000 4194304 4625164 140737106952048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 37677.6
Current children cumulated vsize (KiB) 20907268

[startup+4782.3 s]
/proc/loadavg: 8.07 8.07 8.04 9/518 26486
/proc/meminfo: memFree=5072724/32950928 swapFree=34259960/67111528
[pid=26336] ppid=26334 vsize=21234532 CPUtime=38156.3 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 41492395 0 0 0 3807485 8148 0 0 25 0 8 0 808180331 21744160768 4104183 33554432000 4194304 4625164 140737106952048 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 241
/proc/26336/statm: 5308633 4104183 349 106 0 5304136 0
[pid=26336/tid=26338] ppid=26334 vsize=21234532 CPUtime=4780.9 cores=0-7
/proc/26336/task/26338/stat : 26338 (SArTagnan) R 26334 26336 26075 0 -1 4202560 351183 0 0 0 478023 67 0 0 25 0 8 0 808180345 21744160768 4104183 33554432000 4194304 4625164 140737106952048 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 3 0 0 3
[pid=26336/tid=26339] ppid=26334 vsize=21234532 CPUtime=4718.2 cores=0-7
/proc/26336/task/26339/stat : 26339 (SArTagnan) R 26334 26336 26075 0 -1 4202560 985383 0 0 0 470931 889 0 0 25 0 8 0 808180345 21744160768 4104183 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 4 0 0 5
[pid=26336/tid=26340] ppid=26334 vsize=21234532 CPUtime=4772.24 cores=0-7
/proc/26336/task/26340/stat : 26340 (SArTagnan) R 26334 26336 26075 0 -1 4202560 26771028 0 0 0 472407 4817 0 0 25 0 8 0 808180345 21744160768 4104183 33554432000 4194304 4625164 140737106952048 18446744073709551615 4578811 0 0 4096 2 0 0 0 -1 7 0 0 148
[pid=26336/tid=26341] ppid=26334 vsize=21234532 CPUtime=4773.74 cores=0-7
/proc/26336/task/26341/stat : 26341 (SArTagnan) R 26334 26336 26075 0 -1 4202560 441764 0 0 0 477295 79 0 0 25 0 8 0 808180345 21744160768 4104183 33554432000 4194304 4625164 140737106952048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=26336/tid=26342] ppid=26334 vsize=21234532 CPUtime=4781.53 cores=0-7
/proc/26336/task/26342/stat : 26342 (SArTagnan) R 26334 26336 26075 0 -1 4202560 339484 0 0 0 478083 70 0 0 25 0 8 0 808180345 21744160768 4104183 33554432000 4194304 4625164 140737106952048 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 8
[pid=26336/tid=26343] ppid=26334 vsize=21234532 CPUtime=4780.75 cores=0-7
/proc/26336/task/26343/stat : 26343 (SArTagnan) R 26334 26336 26075 0 -1 4202560 598347 0 0 0 477953 122 0 0 25 0 8 0 808180345 21744160768 4104183 33554432000 4194304 4625164 140737106952048 18446744073709551615 4330793 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=26336/tid=26344] ppid=26334 vsize=21234532 CPUtime=4771.52 cores=0-7
/proc/26336/task/26344/stat : 26344 (SArTagnan) R 26334 26336 26075 0 -1 4202560 185297 0 0 0 477059 93 0 0 25 0 8 0 808180345 21744160768 4104183 33554432000 4194304 4625164 140737106952048 18446744073709551615 4488068 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38156.3
Current children cumulated vsize (KiB) 21234532

[startup+4842.3 s]
/proc/loadavg: 8.07 8.06 8.04 9/518 26488
/proc/meminfo: memFree=5040356/32950928 swapFree=34259960/67111528
[pid=26336] ppid=26334 vsize=21234532 CPUtime=38635.2 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 41938646 0 0 0 3855289 8235 0 0 25 0 8 0 808180331 21744160768 4114192 33554432000 4194304 4625164 140737106952048 18446744073709551615 4393074 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 241
/proc/26336/statm: 5308633 4114192 359 106 0 5304136 0
[pid=26336/tid=26338] ppid=26334 vsize=21234532 CPUtime=4840.9 cores=0-7
/proc/26336/task/26338/stat : 26338 (SArTagnan) R 26334 26336 26075 0 -1 4202560 351415 0 0 0 484023 67 0 0 25 0 8 0 808180345 21744160768 4114192 33554432000 4194304 4625164 140737106952048 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 3 0 0 3
[pid=26336/tid=26339] ppid=26334 vsize=21234532 CPUtime=4778.21 cores=0-7
/proc/26336/task/26339/stat : 26339 (SArTagnan) R 26334 26336 26075 0 -1 4202560 985856 0 0 0 476931 890 0 0 25 0 8 0 808180345 21744160768 4114192 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 5
[pid=26336/tid=26340] ppid=26334 vsize=21234532 CPUtime=4831.14 cores=0-7
/proc/26336/task/26340/stat : 26340 (SArTagnan) R 26334 26336 26075 0 -1 4202560 27209937 0 0 0 478213 4901 0 0 25 0 8 0 808180345 21744160768 4114192 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 148
[pid=26336/tid=26341] ppid=26334 vsize=21234532 CPUtime=4833.75 cores=0-7
/proc/26336/task/26341/stat : 26341 (SArTagnan) R 26334 26336 26075 0 -1 4202560 442144 0 0 0 483296 79 0 0 25 0 8 0 808180345 21744160768 4114192 33554432000 4194304 4625164 140737106952048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=26336/tid=26342] ppid=26334 vsize=21234532 CPUtime=4841.52 cores=0-7
/proc/26336/task/26342/stat : 26342 (SArTagnan) R 26334 26336 26075 0 -1 4202560 339719 0 0 0 484082 70 0 0 25 0 8 0 808180345 21744160768 4114192 33554432000 4194304 4625164 140737106952048 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 8
[pid=26336/tid=26343] ppid=26334 vsize=21234532 CPUtime=4840.75 cores=0-7
/proc/26336/task/26343/stat : 26343 (SArTagnan) R 26334 26336 26075 0 -1 4202560 598892 0 0 0 483953 122 0 0 25 0 8 0 808180345 21744160768 4114192 33554432000 4194304 4625164 140737106952048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=26336/tid=26344] ppid=26334 vsize=21234532 CPUtime=4831.52 cores=0-7
/proc/26336/task/26344/stat : 26344 (SArTagnan) R 26334 26336 26075 0 -1 4202560 185313 0 0 0 483059 93 0 0 25 0 8 0 808180345 21744160768 4114192 33554432000 4194304 4625164 140737106952048 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38635.2
Current children cumulated vsize (KiB) 21234532

[startup+4902.3 s]
/proc/loadavg: 8.11 8.08 8.05 9/518 26490
/proc/meminfo: memFree=4976224/32950928 swapFree=34259960/67111528
[pid=26336] ppid=26334 vsize=21240576 CPUtime=39114.2 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 42397605 0 0 0 3903091 8326 0 0 25 0 8 0 808180331 21750349824 4126662 33554432000 4194304 4625164 140737106952048 18446744073709551615 4393061 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 241
/proc/26336/statm: 5310144 4126662 349 106 0 5305647 0
[pid=26336/tid=26338] ppid=26334 vsize=21240576 CPUtime=4900.9 cores=0-7
/proc/26336/task/26338/stat : 26338 (SArTagnan) R 26334 26336 26075 0 -1 4202560 351662 0 0 0 490023 67 0 0 25 0 8 0 808180345 21750349824 4126662 33554432000 4194304 4625164 140737106952048 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 3 0 0 3
[pid=26336/tid=26339] ppid=26334 vsize=21240576 CPUtime=4838.21 cores=0-7
/proc/26336/task/26339/stat : 26339 (SArTagnan) R 26334 26336 26075 0 -1 4202560 987971 0 0 0 482930 891 0 0 25 0 8 0 808180345 21750349824 4126662 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 4 0 0 5
[pid=26336/tid=26340] ppid=26334 vsize=21240576 CPUtime=4890.07 cores=0-7
/proc/26336/task/26340/stat : 26340 (SArTagnan) R 26334 26336 26075 0 -1 4202560 27594333 0 0 0 484030 4977 0 0 25 0 8 0 808180345 21750349824 4126662 33554432000 4194304 4625164 140737106952048 18446744073709551615 4557254 0 0 4096 2 0 0 0 -1 7 0 0 148
[pid=26336/tid=26341] ppid=26334 vsize=21240576 CPUtime=4893.75 cores=0-7
/proc/26336/task/26341/stat : 26341 (SArTagnan) R 26334 26336 26075 0 -1 4202560 442729 0 0 0 489296 79 0 0 25 0 8 0 808180345 21750349824 4126662 33554432000 4194304 4625164 140737106952048 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=26336/tid=26342] ppid=26334 vsize=21240576 CPUtime=4901.5 cores=0-7
/proc/26336/task/26342/stat : 26342 (SArTagnan) R 26334 26336 26075 0 -1 4202560 339944 0 0 0 490080 70 0 0 25 0 8 0 808180345 21750349824 4126662 33554432000 4194304 4625164 140737106952048 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 8
[pid=26336/tid=26343] ppid=26334 vsize=21240576 CPUtime=4900.75 cores=0-7
/proc/26336/task/26343/stat : 26343 (SArTagnan) R 26334 26336 26075 0 -1 4202560 599226 0 0 0 489953 122 0 0 25 0 8 0 808180345 21750349824 4126662 33554432000 4194304 4625164 140737106952048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=26336/tid=26344] ppid=26334 vsize=21240576 CPUtime=4891.52 cores=0-7
/proc/26336/task/26344/stat : 26344 (SArTagnan) R 26334 26336 26075 0 -1 4202560 185328 0 0 0 489059 93 0 0 25 0 8 0 808180345 21750349824 4126662 33554432000 4194304 4625164 140737106952048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39114.2
Current children cumulated vsize (KiB) 21240576

[startup+4962.3 s]
/proc/loadavg: 8.07 8.08 8.05 9/518 26491
/proc/meminfo: memFree=4909840/32950928 swapFree=34259960/67111528
[pid=26336] ppid=26334 vsize=21306112 CPUtime=39593.1 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 42879466 0 0 0 3950873 8433 0 0 25 0 8 0 808180331 21817458688 4146860 33554432000 4194304 4625164 140737106952048 18446744073709551615 4393110 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 241
/proc/26336/statm: 5326528 4146860 349 106 0 5322031 0
[pid=26336/tid=26338] ppid=26334 vsize=21306112 CPUtime=4960.9 cores=0-7
/proc/26336/task/26338/stat : 26338 (SArTagnan) R 26334 26336 26075 0 -1 4202560 351999 0 0 0 496023 67 0 0 25 0 8 0 808180345 21817458688 4146860 33554432000 4194304 4625164 140737106952048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 3
[pid=26336/tid=26339] ppid=26334 vsize=21306112 CPUtime=4898.2 cores=0-7
/proc/26336/task/26339/stat : 26339 (SArTagnan) R 26334 26336 26075 0 -1 4202560 994359 0 0 0 488925 895 0 0 25 0 8 0 808180345 21817458688 4146860 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 4 0 0 5
[pid=26336/tid=26340] ppid=26334 vsize=21306112 CPUtime=4948.97 cores=0-7
/proc/26336/task/26340/stat : 26340 (SArTagnan) R 26334 26336 26075 0 -1 4202560 27985669 0 0 0 489836 5061 0 0 25 0 8 0 808180345 21817458688 4146860 33554432000 4194304 4625164 140737106952048 18446744073709551615 4328594 0 0 4096 2 0 0 0 -1 7 0 0 148
[pid=26336/tid=26341] ppid=26334 vsize=21306112 CPUtime=4953.74 cores=0-7
/proc/26336/task/26341/stat : 26341 (SArTagnan) R 26334 26336 26075 0 -1 4202560 442992 0 0 0 495295 79 0 0 25 0 8 0 808180345 21817458688 4146860 33554432000 4194304 4625164 140737106952048 18446744073709551615 4330793 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=26336/tid=26342] ppid=26334 vsize=21306112 CPUtime=4961.49 cores=0-7
/proc/26336/task/26342/stat : 26342 (SArTagnan) R 26334 26336 26075 0 -1 4202560 340907 0 0 0 496078 71 0 0 25 0 8 0 808180345 21817458688 4146860 33554432000 4194304 4625164 140737106952048 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 8
[pid=26336/tid=26343] ppid=26334 vsize=21306112 CPUtime=4960.75 cores=0-7
/proc/26336/task/26343/stat : 26343 (SArTagnan) R 26334 26336 26075 0 -1 4202560 599532 0 0 0 495953 122 0 0 25 0 8 0 808180345 21817458688 4146860 33554432000 4194304 4625164 140737106952048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=26336/tid=26344] ppid=26334 vsize=21306112 CPUtime=4951.52 cores=0-7
/proc/26336/task/26344/stat : 26344 (SArTagnan) R 26334 26336 26075 0 -1 4202560 185342 0 0 0 495059 93 0 0 25 0 8 0 808180345 21817458688 4146860 33554432000 4194304 4625164 140737106952048 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39593.1
Current children cumulated vsize (KiB) 21306112



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5013.3 s]
/proc/loadavg: 8.03 8.06 8.04 9/518 26493
/proc/meminfo: memFree=4869868/32950928 swapFree=34259960/67111528
[pid=26336] ppid=26334 vsize=21306112 CPUtime=40000.3 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) R 26334 26336 26075 0 -1 4202496 43286825 0 0 0 3991502 8526 0 0 25 0 8 0 808180331 21817458688 4154366 33554432000 4194304 4625164 140737106952048 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 241
/proc/26336/statm: 5326528 4154366 349 106 0 5322031 0
[pid=26336/tid=26338] ppid=26334 vsize=21306112 CPUtime=5011.92 cores=0-7
/proc/26336/task/26338/stat : 26338 (SArTagnan) R 26334 26336 26075 0 -1 4202560 352190 0 0 0 501125 67 0 0 25 0 8 0 808180345 21817458688 4154366 33554432000 4194304 4625164 140737106952048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 3
[pid=26336/tid=26339] ppid=26334 vsize=21306112 CPUtime=4949.23 cores=0-7
/proc/26336/task/26339/stat : 26339 (SArTagnan) R 26334 26336 26075 0 -1 4202560 998709 0 0 0 494022 901 0 0 25 0 8 0 808180345 21817458688 4154366 33554432000 4194304 4625164 140737106952048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 5
[pid=26336/tid=26340] ppid=26334 vsize=21306112 CPUtime=4999.05 cores=0-7
/proc/26336/task/26340/stat : 26340 (SArTagnan) R 26334 26336 26075 0 -1 4202560 28321568 0 0 0 494769 5136 0 0 25 0 8 0 808180345 21817458688 4154366 33554432000 4194304 4625164 140737106952048 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 7 0 0 148
[pid=26336/tid=26341] ppid=26334 vsize=21306112 CPUtime=5004.77 cores=0-7
/proc/26336/task/26341/stat : 26341 (SArTagnan) R 26334 26336 26075 0 -1 4202560 443668 0 0 0 500397 80 0 0 25 0 8 0 808180345 21817458688 4154366 33554432000 4194304 4625164 140737106952048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=26336/tid=26342] ppid=26334 vsize=21306112 CPUtime=5012.5 cores=0-7
/proc/26336/task/26342/stat : 26342 (SArTagnan) R 26334 26336 26075 0 -1 4202560 341114 0 0 0 501179 71 0 0 25 0 8 0 808180345 21817458688 4154366 33554432000 4194304 4625164 140737106952048 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 8
[pid=26336/tid=26343] ppid=26334 vsize=21306112 CPUtime=5011.77 cores=0-7
/proc/26336/task/26343/stat : 26343 (SArTagnan) R 26334 26336 26075 0 -1 4202560 600115 0 0 0 501055 122 0 0 25 0 8 0 808180345 21817458688 4154366 33554432000 4194304 4625164 140737106952048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=26336/tid=26344] ppid=26334 vsize=21306112 CPUtime=5002.55 cores=0-7
/proc/26336/task/26344/stat : 26344 (SArTagnan) R 26334 26336 26075 0 -1 4202560 185355 0 0 0 500162 93 0 0 25 0 8 0 808180345 21817458688 4154366 33554432000 4194304 4625164 140737106952048 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 40000.3
Current children cumulated vsize (KiB) 21306112

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5013.7 s]
/proc/loadavg: 8.03 8.06 8.04 2/513 26494
/proc/meminfo: memFree=13720872/32950928 swapFree=34259960/67111528
[pid=26336] ppid=26334 vsize=0 CPUtime=40000.7 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) Z 26334 26336 26075 0 -1 4203548 43286825 0 0 0 3991507 8565 0 0 25 0 2 0 808180331 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 241
/proc/26336/statm: 0 0 0 0 0 0 0
[pid=26336/tid=26340] ppid=26334 vsize=0 CPUtime=4999.45 cores=0-7
/proc/26336/task/26340/stat : 26340 (SArTagnan) R 26334 26336 26075 0 -1 4203588 28321568 0 0 0 494769 5176 0 0 25 0 2 0 808180345 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 -1 5 0 0 148
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 0

[startup+5013.9 s]
/proc/loadavg: 8.03 8.06 8.04 2/513 26494
/proc/meminfo: memFree=13720872/32950928 swapFree=34259960/67111528
[pid=26336] ppid=26334 vsize=0 CPUtime=40000.9 cores=0-7
/proc/26336/stat : 26336 (SArTagnan) Z 26334 26336 26075 0 -1 4203548 43286825 0 0 0 3991507 8583 0 0 25 0 2 0 808180331 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 241
/proc/26336/statm: 0 0 0 0 0 0 0
[pid=26336/tid=26340] ppid=26334 vsize=0 CPUtime=4999.63 cores=0-7
/proc/26336/task/26340/stat : 26340 (SArTagnan) R 26334 26336 26075 0 -1 4203588 28321568 0 0 0 494769 5194 0 0 25 0 2 0 808180345 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 -1 5 0 0 148
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5013.98
CPU time (s): 40001
CPU user time (s): 39915.1
CPU system time (s): 85.9119
CPU usage (%): 797.789
Max. virtual memory (cumulated for all children) (KiB): 27146644

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39915.1
system time used= 85.9119
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 43286825
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 32883
involuntary context switches= 168036

runsolver used 23.9474 second user time and 67.0388 second system time

The end

Launcher Data 

Begin job on node149 at 2011-06-02 00:03:56
IDJOB=3439796
IDBENCH=88226
IDSOLVER=1936
FILE ID=node149/3439796-1306965836
RUNJOBID= node149-1306960404-26089
PBS_JOBID= 13497602
Free space on /tmp= 73512 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m52-q1041.25-n50-p15-s99099953.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3439796-1306965836/watcher-3439796-1306965836 -o /tmp/evaluation-result-3439796-1306965836/solver-3439796-1306965836 -C 40000 -W 5100 -M 31000  SArTagnan HOME/instance-3439796-1306965836.cnf -threads=8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= dda220d08783aabf4e7a9e99eeeccae6
RANDOM SEED=665992453

node149.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      20835200 kB
Buffers:        299232 kB
Cached:        3331168 kB
SwapCached:      84504 kB
Active:        2055876 kB
Inactive:      9729708 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      20835200 kB
SwapTotal:    67111528 kB
SwapFree:     34979748 kB
Dirty:            6988 kB
Writeback:          32 kB
AnonPages:     8095104 kB
Mapped:          17408 kB
Slab:           140028 kB
PageTables:     117944 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 173081848 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73512 MiB
End job on node149 at 2011-06-02 01:27:32