Trace number 3438371

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 40000.9 5011.07

General information on the benchmark

NameCRAFTED/rbsat/random/unforced/
rbsat-v2400c266431g3.cnf
MD5SUM231f625a25c77518fbce98fe99f807b5
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2400
Number of clauses266431
Sum of the clauses size535112
Maximum clause length32
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2266356
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 575

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3438371-1306915609/watcher-3438371-1306915609 -o /tmp/evaluation-result-3438371-1306915609/solver-3438371-1306915609 -C 40000 -W 5100 -M 31000 SArTagnan HOME/instance-3438371-1306915609.cnf -threads=8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.16 2.11 2.03 2/446 4664
/proc/meminfo: memFree=10881024/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=25348 CPUtime=0 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 674 0 0 0 0 0 0 0 25 0 1 0 803158811 25956352 494 33554432000 4194304 4625164 140734073300448 18446744073709551615 271550537472 0 0 4096 2 0 0 0 17 6 0 0 0
/proc/4664/statm: 6337 495 296 106 0 1840 0

[startup+0.049775 s]
/proc/loadavg: 2.16 2.11 2.03 2/446 4664
/proc/meminfo: memFree=10881024/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=28820 CPUtime=0.04 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 2915 0 0 0 4 0 0 0 25 0 1 0 803158811 29511680 1064 33554432000 4194304 4625164 140734073300448 18446744073709551615 271550156288 0 0 4096 2 0 0 0 17 6 0 0 0
/proc/4664/statm: 7205 1065 296 106 0 2708 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 28820

[startup+0.10076 s]
/proc/loadavg: 2.16 2.11 2.03 2/446 4664
/proc/meminfo: memFree=10881024/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=127128 CPUtime=0.22 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 9264 0 0 0 22 0 0 0 19 0 8 0 803158811 130179072 7412 33554432000 4194304 4625164 140734073300448 18446744073709551615 47301434357682 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/4664/statm: 31782 7412 301 106 0 27285 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 127128

[startup+0.362678 s]
/proc/loadavg: 2.16 2.11 2.03 2/446 4664
/proc/meminfo: memFree=10881024/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=484840 CPUtime=1.96 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 38089 0 0 0 190 6 0 0 18 0 8 0 803158811 496476160 24118 33554432000 4194304 4625164 140734073300448 18446744073709551615 4361099 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4664/statm: 121210 24118 351 106 0 116713 0
Current children cumulated CPU time (s) 1.96
Current children cumulated vsize (KiB) 484840

[startup+1.13971 s]
/proc/loadavg: 2.16 2.11 2.03 2/446 4664
/proc/meminfo: memFree=10881024/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=555452 CPUtime=8.16 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 49547 0 0 0 807 9 0 0 19 0 8 0 803158811 568782848 35045 33554432000 4194304 4625164 140734073300448 18446744073709551615 271550198817 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4664/statm: 138863 35045 389 106 0 134366 0
Current children cumulated CPU time (s) 8.16
Current children cumulated vsize (KiB) 555452

[startup+1.51747 s]
/proc/loadavg: 2.71 2.23 2.07 9/454 4675
/proc/meminfo: memFree=10735020/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=556604 CPUtime=11.17 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 53520 0 0 0 1107 10 0 0 19 0 8 0 803158811 569962496 38008 33554432000 4194304 4625164 140734073300448 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4664/statm: 139151 38008 389 106 0 134654 0
[pid=4664/tid=4667] ppid=4661 vsize=556604 CPUtime=1.43 cores=0-7
/proc/4664/task/4667/stat : 4667 (SArTagnan) R 4661 4664 4590 0 -1 4202560 3080 0 0 0 143 0 0 0 19 0 8 0 803158819 569962496 38008 33554432000 4194304 4625164 140734073300448 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4664/tid=4668] ppid=4661 vsize=556604 CPUtime=1.19 cores=0-7
/proc/4664/task/4668/stat : 4668 (SArTagnan) R 4661 4664 4590 0 -1 4202560 1776 0 0 0 119 0 0 0 19 0 8 0 803158819 569962496 38008 33554432000 4194304 4625164 140734073300448 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4664/tid=4670] ppid=4661 vsize=556604 CPUtime=1.39 cores=0-7
/proc/4664/task/4670/stat : 4670 (SArTagnan) R 4661 4664 4590 0 -1 4202560 3649 0 0 0 139 0 0 0 19 0 8 0 803158819 569962496 38008 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4664/tid=4671] ppid=4661 vsize=556604 CPUtime=1.32 cores=0-7
/proc/4664/task/4671/stat : 4671 (SArTagnan) R 4661 4664 4590 0 -1 4202560 2447 0 0 0 132 0 0 0 19 0 8 0 803158819 569962496 38008 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351984 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4664/tid=4672] ppid=4661 vsize=556604 CPUtime=1.42 cores=0-7
/proc/4664/task/4672/stat : 4672 (SArTagnan) R 4661 4664 4590 0 -1 4202560 2593 0 0 0 142 0 0 0 19 0 8 0 803158819 569962496 38008 33554432000 4194304 4625164 140734073300448 18446744073709551615 4349226 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4664/tid=4673] ppid=4661 vsize=556604 CPUtime=1.43 cores=0-7
/proc/4664/task/4673/stat : 4673 (SArTagnan) R 4661 4664 4590 0 -1 4202560 2953 0 0 0 143 0 0 0 19 0 8 0 803158819 569962496 38008 33554432000 4194304 4625164 140734073300448 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4664/tid=4675] ppid=4661 vsize=556604 CPUtime=1.42 cores=0-7
/proc/4664/task/4675/stat : 4675 (SArTagnan) R 4661 4664 4590 0 -1 4202560 4108 0 0 0 142 0 0 0 19 0 8 0 803158819 569962496 38008 33554432000 4194304 4625164 140734073300448 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 11.17
Current children cumulated vsize (KiB) 556604

[startup+3.10118 s]
/proc/loadavg: 2.71 2.23 2.07 9/454 4675
/proc/meminfo: memFree=10721380/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=561852 CPUtime=23.82 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 65540 0 0 0 2370 12 0 0 24 0 8 0 803158811 575336448 43655 33554432000 4194304 4625164 140734073300448 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4664/statm: 140463 43655 390 106 0 135966 0
[pid=4664/tid=4667] ppid=4661 vsize=561852 CPUtime=3.01 cores=0-7
/proc/4664/task/4667/stat : 4667 (SArTagnan) R 4661 4664 4590 0 -1 4202560 4846 0 0 0 301 0 0 0 22 0 8 0 803158819 575336448 43655 33554432000 4194304 4625164 140734073300448 18446744073709551615 4352047 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4664/tid=4668] ppid=4661 vsize=561852 CPUtime=2.78 cores=0-7
/proc/4664/task/4668/stat : 4668 (SArTagnan) R 4661 4664 4590 0 -1 4202560 3014 0 0 0 277 1 0 0 22 0 8 0 803158819 575336448 43655 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4664/tid=4670] ppid=4661 vsize=561852 CPUtime=2.98 cores=0-7
/proc/4664/task/4670/stat : 4670 (SArTagnan) R 4661 4664 4590 0 -1 4202560 6393 0 0 0 297 1 0 0 22 0 8 0 803158819 575336448 43655 33554432000 4194304 4625164 140734073300448 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4664/tid=4671] ppid=4661 vsize=561852 CPUtime=2.9 cores=0-7
/proc/4664/task/4671/stat : 4671 (SArTagnan) R 4661 4664 4590 0 -1 4202560 3614 0 0 0 289 1 0 0 22 0 8 0 803158819 575336448 43655 33554432000 4194304 4625164 140734073300448 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4664/tid=4672] ppid=4661 vsize=561852 CPUtime=3.01 cores=0-7
/proc/4664/task/4672/stat : 4672 (SArTagnan) R 4661 4664 4590 0 -1 4202560 4346 0 0 0 301 0 0 0 22 0 8 0 803158819 575336448 43655 33554432000 4194304 4625164 140734073300448 18446744073709551615 4348903 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4664/tid=4673] ppid=4661 vsize=561852 CPUtime=3.01 cores=0-7
/proc/4664/task/4673/stat : 4673 (SArTagnan) R 4661 4664 4590 0 -1 4202560 5739 0 0 0 301 0 0 0 24 0 8 0 803158819 575336448 43655 33554432000 4194304 4625164 140734073300448 18446744073709551615 47301434358310 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4664/tid=4675] ppid=4661 vsize=561852 CPUtime=3 cores=0-7
/proc/4664/task/4675/stat : 4675 (SArTagnan) R 4661 4664 4590 0 -1 4202560 4403 0 0 0 300 0 0 0 23 0 8 0 803158819 575336448 43655 33554432000 4194304 4625164 140734073300448 18446744073709551615 4352855 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 23.82
Current children cumulated vsize (KiB) 561852

[startup+6.3006 s]
/proc/loadavg: 3.21 2.34 2.11 10/454 4675
/proc/meminfo: memFree=10662996/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=572704 CPUtime=49.37 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 89264 0 0 0 4921 16 0 0 25 0 8 0 803158811 586448896 58896 33554432000 4194304 4625164 140734073300448 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4664/statm: 143176 58896 382 106 0 138679 0
[pid=4664/tid=4667] ppid=4661 vsize=572704 CPUtime=6.21 cores=0-7
/proc/4664/task/4667/stat : 4667 (SArTagnan) R 4661 4664 4590 0 -1 4202560 6863 0 0 0 620 1 0 0 25 0 8 0 803158819 586448896 58896 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4664/tid=4668] ppid=4661 vsize=572704 CPUtime=5.97 cores=0-7
/proc/4664/task/4668/stat : 4668 (SArTagnan) R 4661 4664 4590 0 -1 4202560 5271 0 0 0 596 1 0 0 25 0 8 0 803158819 586448896 58896 33554432000 4194304 4625164 140734073300448 18446744073709551615 4352017 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4664/tid=4670] ppid=4661 vsize=572704 CPUtime=6.18 cores=0-7
/proc/4664/task/4670/stat : 4670 (SArTagnan) R 4661 4664 4590 0 -1 4202560 13013 0 0 0 616 2 0 0 25 0 8 0 803158819 586448896 58896 33554432000 4194304 4625164 140734073300448 18446744073709551615 4547828 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4664/tid=4671] ppid=4661 vsize=572704 CPUtime=6.05 cores=0-7
/proc/4664/task/4671/stat : 4671 (SArTagnan) R 4661 4664 4590 0 -1 4202560 5220 0 0 0 604 1 0 0 25 0 8 0 803158819 586448896 58896 33554432000 4194304 4625164 140734073300448 18446744073709551615 4352014 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4664/tid=4672] ppid=4661 vsize=572704 CPUtime=6.21 cores=0-7
/proc/4664/task/4672/stat : 4672 (SArTagnan) R 4661 4664 4590 0 -1 4202560 7567 0 0 0 620 1 0 0 25 0 8 0 803158819 586448896 58896 33554432000 4194304 4625164 140734073300448 18446744073709551615 4339643 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4664/tid=4673] ppid=4661 vsize=572704 CPUtime=6.22 cores=0-7
/proc/4664/task/4673/stat : 4673 (SArTagnan) R 4661 4664 4590 0 -1 4202560 8693 0 0 0 621 1 0 0 25 0 8 0 803158819 586448896 58896 33554432000 4194304 4625164 140734073300448 18446744073709551615 47301434358310 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4664/tid=4675] ppid=4661 vsize=572704 CPUtime=6.2 cores=0-7
/proc/4664/task/4675/stat : 4675 (SArTagnan) R 4661 4664 4590 0 -1 4202560 6781 0 0 0 619 1 0 0 25 0 8 0 803158819 586448896 58896 33554432000 4194304 4625164 140734073300448 18446744073709551615 4463849 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 49.37
Current children cumulated vsize (KiB) 572704

[startup+12.7004 s]
/proc/loadavg: 3.59 2.43 2.14 9/454 4676
/proc/meminfo: memFree=10566960/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=680500 CPUtime=100.46 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 129859 0 0 0 10023 23 0 0 25 0 8 0 803158811 696832000 79377 33554432000 4194304 4625164 140734073300448 18446744073709551615 4512017 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4664/statm: 170125 79378 427 106 0 165628 0
[pid=4664/tid=4667] ppid=4661 vsize=680500 CPUtime=12.61 cores=0-7
/proc/4664/task/4667/stat : 4667 (SArTagnan) R 4661 4664 4590 0 -1 4202560 10659 0 0 0 1259 2 0 0 25 0 8 0 803158819 696832000 79378 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4664/tid=4668] ppid=4661 vsize=680500 CPUtime=12.37 cores=0-7
/proc/4664/task/4668/stat : 4668 (SArTagnan) R 4661 4664 4590 0 -1 4202560 7958 0 0 0 1236 1 0 0 25 0 8 0 803158819 696832000 79378 33554432000 4194304 4625164 140734073300448 18446744073709551615 4352098 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4664/tid=4670] ppid=4661 vsize=680500 CPUtime=12.58 cores=0-7
/proc/4664/task/4670/stat : 4670 (SArTagnan) R 4661 4664 4590 0 -1 4202560 28066 0 0 0 1253 5 0 0 25 0 8 0 803158819 696832000 79379 33554432000 4194304 4625164 140734073300448 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4664/tid=4671] ppid=4661 vsize=680500 CPUtime=12.37 cores=0-7
/proc/4664/task/4671/stat : 4671 (SArTagnan) R 4661 4664 4590 0 -1 4202560 5608 0 0 0 1236 1 0 0 25 0 8 0 803158819 696832000 79379 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351818 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=4664/tid=4672] ppid=4661 vsize=680500 CPUtime=12.61 cores=0-7
/proc/4664/task/4672/stat : 4672 (SArTagnan) R 4661 4664 4590 0 -1 4202560 9514 0 0 0 1260 1 0 0 25 0 8 0 803158819 696832000 79379 33554432000 4194304 4625164 140734073300448 18446744073709551615 4318960 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4664/tid=4673] ppid=4661 vsize=680500 CPUtime=12.61 cores=0-7
/proc/4664/task/4673/stat : 4673 (SArTagnan) R 4661 4664 4590 0 -1 4202560 13700 0 0 0 1260 1 0 0 19 0 8 0 803158819 696832000 79380 33554432000 4194304 4625164 140734073300448 18446744073709551615 4352085 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4664/tid=4675] ppid=4661 vsize=680500 CPUtime=12.57 cores=0-7
/proc/4664/task/4675/stat : 4675 (SArTagnan) R 4661 4664 4590 0 -1 4202560 7448 0 0 0 1256 1 0 0 25 0 8 0 803158819 696832000 79380 33554432000 4194304 4625164 140734073300448 18446744073709551615 4339646 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 100.46
Current children cumulated vsize (KiB) 680500

[startup+25.5011 s]
/proc/loadavg: 4.27 2.62 2.20 9/454 4676
/proc/meminfo: memFree=10487008/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=714412 CPUtime=202.68 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 222603 0 0 0 20230 38 0 0 25 0 8 0 803158811 731557888 99184 33554432000 4194304 4625164 140734073300448 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4664/statm: 178603 99184 479 106 0 174106 0
[pid=4664/tid=4667] ppid=4661 vsize=714412 CPUtime=25.42 cores=0-7
/proc/4664/task/4667/stat : 4667 (SArTagnan) R 4661 4664 4590 0 -1 4202560 14941 0 0 0 2539 3 0 0 25 0 8 0 803158819 731557888 99184 33554432000 4194304 4625164 140734073300448 18446744073709551615 4339559 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4664/tid=4668] ppid=4661 vsize=714412 CPUtime=25.17 cores=0-7
/proc/4664/task/4668/stat : 4668 (SArTagnan) R 4661 4664 4590 0 -1 4202560 9161 0 0 0 2515 2 0 0 25 0 8 0 803158819 731557888 99184 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4664/tid=4670] ppid=4661 vsize=714412 CPUtime=25.38 cores=0-7
/proc/4664/task/4670/stat : 4670 (SArTagnan) R 4661 4664 4590 0 -1 4202560 86122 0 0 0 2526 12 0 0 25 0 8 0 803158819 731557888 99184 33554432000 4194304 4625164 140734073300448 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4664/tid=4671] ppid=4661 vsize=714412 CPUtime=25.18 cores=0-7
/proc/4664/task/4671/stat : 4671 (SArTagnan) R 4661 4664 4590 0 -1 4202560 8349 0 0 0 2516 2 0 0 25 0 8 0 803158819 731557888 99184 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351975 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=4664/tid=4672] ppid=4661 vsize=714412 CPUtime=25.41 cores=0-7
/proc/4664/task/4672/stat : 4672 (SArTagnan) R 4661 4664 4590 0 -1 4202560 14018 0 0 0 2539 2 0 0 25 0 8 0 803158819 731557888 99184 33554432000 4194304 4625164 140734073300448 18446744073709551615 4348934 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4664/tid=4673] ppid=4661 vsize=714412 CPUtime=25.41 cores=0-7
/proc/4664/task/4673/stat : 4673 (SArTagnan) R 4661 4664 4590 0 -1 4202560 15560 0 0 0 2539 2 0 0 25 0 8 0 803158819 731557888 99184 33554432000 4194304 4625164 140734073300448 18446744073709551615 4339591 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4664/tid=4675] ppid=4661 vsize=714412 CPUtime=25.19 cores=0-7
/proc/4664/task/4675/stat : 4675 (SArTagnan) R 4661 4664 4590 0 -1 4202560 10480 0 0 0 2517 2 0 0 25 0 8 0 803158819 731557888 99184 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 202.68
Current children cumulated vsize (KiB) 714412

[startup+51.1005 s]
/proc/loadavg: 5.81 3.15 2.39 9/454 4677
/proc/meminfo: memFree=10266888/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=923160 CPUtime=407.07 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 381953 0 0 0 40640 67 0 0 25 0 8 0 803158811 945315840 155208 33554432000 4194304 4625164 140734073300448 18446744073709551615 4512012 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4664/statm: 230790 155208 743 106 0 226293 0
[pid=4664/tid=4667] ppid=4661 vsize=923160 CPUtime=51.01 cores=0-7
/proc/4664/task/4667/stat : 4667 (SArTagnan) R 4661 4664 4590 0 -1 4202560 21472 0 0 0 5097 4 0 0 25 0 8 0 803158819 945315840 155208 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4664/tid=4668] ppid=4661 vsize=923160 CPUtime=50.77 cores=0-7
/proc/4664/task/4668/stat : 4668 (SArTagnan) R 4661 4664 4590 0 -1 4202560 15194 0 0 0 5074 3 0 0 25 0 8 0 803158819 945315840 155208 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4664/tid=4670] ppid=4661 vsize=923160 CPUtime=50.97 cores=0-7
/proc/4664/task/4670/stat : 4670 (SArTagnan) R 4661 4664 4590 0 -1 4202560 181562 0 0 0 5069 28 0 0 25 0 8 0 803158819 945315840 155208 33554432000 4194304 4625164 140734073300448 18446744073709551615 4352280 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4664/tid=4671] ppid=4661 vsize=923160 CPUtime=50.78 cores=0-7
/proc/4664/task/4671/stat : 4671 (SArTagnan) R 4661 4664 4590 0 -1 4202560 10119 0 0 0 5075 3 0 0 25 0 8 0 803158819 945315840 155208 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=4664/tid=4672] ppid=4661 vsize=923160 CPUtime=51.01 cores=0-7
/proc/4664/task/4672/stat : 4672 (SArTagnan) R 4661 4664 4590 0 -1 4202560 22421 0 0 0 5097 4 0 0 25 0 8 0 803158819 945315840 155208 33554432000 4194304 4625164 140734073300448 18446744073709551615 4349019 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4664/tid=4673] ppid=4661 vsize=923160 CPUtime=51.01 cores=0-7
/proc/4664/task/4673/stat : 4673 (SArTagnan) R 4661 4664 4590 0 -1 4202560 25625 0 0 0 5097 4 0 0 25 0 8 0 803158819 945315840 155208 33554432000 4194304 4625164 140734073300448 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4664/tid=4675] ppid=4661 vsize=923160 CPUtime=50.39 cores=0-7
/proc/4664/task/4675/stat : 4675 (SArTagnan) R 4661 4664 4590 0 -1 4202560 15008 0 0 0 5036 3 0 0 25 0 8 0 803158819 945315840 155208 33554432000 4194304 4625164 140734073300448 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 407.07

################
# More data... #
################

Current children cumulated vsize (KiB) 4925380

[startup+4722.3 s]
/proc/loadavg: 8.21 8.20 8.14 9/454 4962
/proc/meminfo: memFree=28322536/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=4925380 CPUtime=37697 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 32691256 0 0 0 3764208 5494 0 0 25 0 8 0 803158811 5043589120 975321 33554432000 4194304 4625164 140734073300448 18446744073709551615 4329151 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4664/statm: 1231345 975321 352 106 0 1226848 0
[pid=4664/tid=4667] ppid=4661 vsize=4925380 CPUtime=4720.69 cores=0-7
/proc/4664/task/4667/stat : 4667 (SArTagnan) R 4661 4664 4590 0 -1 4202560 117498 0 0 0 472043 26 0 0 25 0 8 0 803158819 5043589120 975321 33554432000 4194304 4625164 140734073300448 18446744073709551615 4330822 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4664/tid=4668] ppid=4661 vsize=4925380 CPUtime=4714.61 cores=0-7
/proc/4664/task/4668/stat : 4668 (SArTagnan) R 4661 4664 4590 0 -1 4202560 221503 0 0 0 471285 176 0 0 25 0 8 0 803158819 5043589120 975321 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4664/tid=4670] ppid=4661 vsize=4925380 CPUtime=4721.06 cores=0-7
/proc/4664/task/4670/stat : 4670 (SArTagnan) R 4661 4664 4590 0 -1 4202560 30907669 0 0 0 467209 4897 0 0 25 0 8 0 803158819 5043589120 975321 33554432000 4194304 4625164 140734073300448 18446744073709551615 4335165 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4664/tid=4671] ppid=4661 vsize=4925380 CPUtime=4721.83 cores=0-7
/proc/4664/task/4671/stat : 4671 (SArTagnan) R 4661 4664 4590 0 -1 4202560 92538 0 0 0 472136 47 0 0 25 0 8 0 803158819 5043589120 975321 33554432000 4194304 4625164 140734073300448 18446744073709551615 4464984 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=4664/tid=4672] ppid=4661 vsize=4925380 CPUtime=4721.79 cores=0-7
/proc/4664/task/4672/stat : 4672 (SArTagnan) R 4661 4664 4590 0 -1 4202560 170852 0 0 0 472143 36 0 0 25 0 8 0 803158819 5043589120 975321 33554432000 4194304 4625164 140734073300448 18446744073709551615 4460412 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4664/tid=4673] ppid=4661 vsize=4925380 CPUtime=4721.13 cores=0-7
/proc/4664/task/4673/stat : 4673 (SArTagnan) R 4661 4664 4590 0 -1 4202560 171492 0 0 0 472072 41 0 0 25 0 8 0 803158819 5043589120 975321 33554432000 4194304 4625164 140734073300448 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4664/tid=4675] ppid=4661 vsize=4925380 CPUtime=4653.72 cores=0-7
/proc/4664/task/4675/stat : 4675 (SArTagnan) R 4661 4664 4590 0 -1 4202560 121374 0 0 0 465289 83 0 0 25 0 8 0 803158819 5043589120 975321 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37697
Current children cumulated vsize (KiB) 4925380

[startup+4782.3 s]
/proc/loadavg: 8.24 8.22 8.15 9/454 4964
/proc/meminfo: memFree=28282076/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=4925380 CPUtime=38176.1 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 33195311 0 0 0 3812032 5576 0 0 25 0 8 0 803158811 5043589120 983628 33554432000 4194304 4625164 140734073300448 18446744073709551615 4512818 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4664/statm: 1231345 983628 352 106 0 1226848 0
[pid=4664/tid=4667] ppid=4661 vsize=4925380 CPUtime=4780.69 cores=0-7
/proc/4664/task/4667/stat : 4667 (SArTagnan) R 4661 4664 4590 0 -1 4202560 117700 0 0 0 478043 26 0 0 25 0 8 0 803158819 5043589120 983628 33554432000 4194304 4625164 140734073300448 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4664/tid=4668] ppid=4661 vsize=4925380 CPUtime=4773.67 cores=0-7
/proc/4664/task/4668/stat : 4668 (SArTagnan) R 4661 4664 4590 0 -1 4202560 221544 0 0 0 477191 176 0 0 25 0 8 0 803158819 5043589120 983628 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4664/tid=4670] ppid=4661 vsize=4925380 CPUtime=4781.06 cores=0-7
/proc/4664/task/4670/stat : 4670 (SArTagnan) R 4661 4664 4590 0 -1 4202560 31400367 0 0 0 473127 4979 0 0 25 0 8 0 803158819 5043589120 983628 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4664/tid=4671] ppid=4661 vsize=4925380 CPUtime=4781.83 cores=0-7
/proc/4664/task/4671/stat : 4671 (SArTagnan) R 4661 4664 4590 0 -1 4202560 92558 0 0 0 478136 47 0 0 25 0 8 0 803158819 5043589120 983628 33554432000 4194304 4625164 140734073300448 18446744073709551615 4352014 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=4664/tid=4672] ppid=4661 vsize=4925380 CPUtime=4781.79 cores=0-7
/proc/4664/task/4672/stat : 4672 (SArTagnan) R 4661 4664 4590 0 -1 4202560 170965 0 0 0 478143 36 0 0 25 0 8 0 803158819 5043589120 983628 33554432000 4194304 4625164 140734073300448 18446744073709551615 4329721 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4664/tid=4673] ppid=4661 vsize=4925380 CPUtime=4781.14 cores=0-7
/proc/4664/task/4673/stat : 4673 (SArTagnan) R 4661 4664 4590 0 -1 4202560 171693 0 0 0 478073 41 0 0 25 0 8 0 803158819 5043589120 983628 33554432000 4194304 4625164 140734073300448 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4664/tid=4675] ppid=4661 vsize=4925380 CPUtime=4713.72 cores=0-7
/proc/4664/task/4675/stat : 4675 (SArTagnan) R 4661 4664 4590 0 -1 4202560 121390 0 0 0 471289 83 0 0 25 0 8 0 803158819 5043589120 983628 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38176.1
Current children cumulated vsize (KiB) 4925380

[startup+4842.3 s]
/proc/loadavg: 8.19 8.21 8.15 9/454 4965
/proc/meminfo: memFree=28264708/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=4930744 CPUtime=38655.2 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 33201975 0 0 0 3859942 5579 0 0 25 0 8 0 803158811 5049081856 988175 33554432000 4194304 4625164 140734073300448 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4664/statm: 1232686 988175 352 106 0 1228189 0
[pid=4664/tid=4667] ppid=4661 vsize=4930744 CPUtime=4840.71 cores=0-7
/proc/4664/task/4667/stat : 4667 (SArTagnan) R 4661 4664 4590 0 -1 4202560 117901 0 0 0 484045 26 0 0 25 0 8 0 803158819 5049081856 988175 33554432000 4194304 4625164 140734073300448 18446744073709551615 4330844 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4664/tid=4668] ppid=4661 vsize=4930744 CPUtime=4832.76 cores=0-7
/proc/4664/task/4668/stat : 4668 (SArTagnan) R 4661 4664 4590 0 -1 4202560 221714 0 0 0 483099 177 0 0 25 0 8 0 803158819 5049081856 988175 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4664/tid=4670] ppid=4661 vsize=4930744 CPUtime=4841.06 cores=0-7
/proc/4664/task/4670/stat : 4670 (SArTagnan) R 4661 4664 4590 0 -1 4202560 31403618 0 0 0 479127 4979 0 0 25 0 8 0 803158819 5049081856 988175 33554432000 4194304 4625164 140734073300448 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4664/tid=4671] ppid=4661 vsize=4930744 CPUtime=4841.84 cores=0-7
/proc/4664/task/4671/stat : 4671 (SArTagnan) R 4661 4664 4590 0 -1 4202560 93149 0 0 0 484136 48 0 0 25 0 8 0 803158819 5049081856 988175 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=4664/tid=4672] ppid=4661 vsize=4930744 CPUtime=4841.81 cores=0-7
/proc/4664/task/4672/stat : 4672 (SArTagnan) R 4661 4664 4590 0 -1 4202560 171106 0 0 0 484145 36 0 0 25 0 8 0 803158819 5049081856 988175 33554432000 4194304 4625164 140734073300448 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4664/tid=4673] ppid=4661 vsize=4930744 CPUtime=4841.15 cores=0-7
/proc/4664/task/4673/stat : 4673 (SArTagnan) R 4661 4664 4590 0 -1 4202560 171819 0 0 0 484074 41 0 0 25 0 8 0 803158819 5049081856 988175 33554432000 4194304 4625164 140734073300448 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4664/tid=4675] ppid=4661 vsize=4930744 CPUtime=4773.74 cores=0-7
/proc/4664/task/4675/stat : 4675 (SArTagnan) R 4661 4664 4590 0 -1 4202560 121428 0 0 0 477291 83 0 0 25 0 8 0 803158819 5049081856 988175 33554432000 4194304 4625164 140734073300448 18446744073709551615 4352017 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38655.2
Current children cumulated vsize (KiB) 4930744

[startup+4902.3 s]
/proc/loadavg: 8.23 8.21 8.15 9/454 4967
/proc/meminfo: memFree=28229332/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=4996280 CPUtime=39134.2 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 33345394 0 0 0 3907809 5613 0 0 25 0 8 0 803158811 5116190720 997548 33554432000 4194304 4625164 140734073300448 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4664/statm: 1249070 997548 352 106 0 1244573 0
[pid=4664/tid=4667] ppid=4661 vsize=4996280 CPUtime=4900.69 cores=0-7
/proc/4664/task/4667/stat : 4667 (SArTagnan) R 4661 4664 4590 0 -1 4202560 118068 0 0 0 490043 26 0 0 25 0 8 0 803158819 5116190720 997548 33554432000 4194304 4625164 140734073300448 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4664/tid=4668] ppid=4661 vsize=4996280 CPUtime=4891.83 cores=0-7
/proc/4664/task/4668/stat : 4668 (SArTagnan) R 4661 4664 4590 0 -1 4202560 226211 0 0 0 488996 187 0 0 25 0 8 0 803158819 5116190720 997548 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4664/tid=4670] ppid=4661 vsize=4996280 CPUtime=4901.02 cores=0-7
/proc/4664/task/4670/stat : 4670 (SArTagnan) R 4661 4664 4590 0 -1 4202560 31537953 0 0 0 485100 5002 0 0 25 0 8 0 803158819 5116190720 997548 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4664/tid=4671] ppid=4661 vsize=4996280 CPUtime=4901.82 cores=0-7
/proc/4664/task/4671/stat : 4671 (SArTagnan) R 4661 4664 4590 0 -1 4202560 93455 0 0 0 490134 48 0 0 25 0 8 0 803158819 5116190720 997548 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=4664/tid=4672] ppid=4661 vsize=4996280 CPUtime=4901.8 cores=0-7
/proc/4664/task/4672/stat : 4672 (SArTagnan) R 4661 4664 4590 0 -1 4202560 171254 0 0 0 490143 37 0 0 25 0 8 0 803158819 5116190720 997548 33554432000 4194304 4625164 140734073300448 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4664/tid=4673] ppid=4661 vsize=4996280 CPUtime=4901.14 cores=0-7
/proc/4664/task/4673/stat : 4673 (SArTagnan) R 4661 4664 4590 0 -1 4202560 172337 0 0 0 490073 41 0 0 25 0 8 0 803158819 5116190720 997548 33554432000 4194304 4625164 140734073300448 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4664/tid=4675] ppid=4661 vsize=4996280 CPUtime=4833.73 cores=0-7
/proc/4664/task/4675/stat : 4675 (SArTagnan) R 4661 4664 4590 0 -1 4202560 121448 0 0 0 483290 83 0 0 25 0 8 0 803158819 5116190720 997548 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39134.2
Current children cumulated vsize (KiB) 4996280

[startup+4962.32 s]
/proc/loadavg: 8.23 8.20 8.15 9/454 4969
/proc/meminfo: memFree=28205248/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=4996280 CPUtime=39613.4 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 33633375 0 0 0 3955684 5658 0 0 25 0 8 0 803158811 5116190720 1001887 33554432000 4194304 4625164 140734073300448 18446744073709551615 4393074 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4664/statm: 1249070 1001887 352 106 0 1244573 0
[pid=4664/tid=4667] ppid=4661 vsize=4996280 CPUtime=4960.71 cores=0-7
/proc/4664/task/4667/stat : 4667 (SArTagnan) R 4661 4664 4590 0 -1 4202560 118244 0 0 0 496044 27 0 0 25 0 8 0 803158819 5116190720 1001887 33554432000 4194304 4625164 140734073300448 18446744073709551615 4330793 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4664/tid=4668] ppid=4661 vsize=4996280 CPUtime=4950.92 cores=0-7
/proc/4664/task/4668/stat : 4668 (SArTagnan) R 4661 4664 4590 0 -1 4202560 226287 0 0 0 494905 187 0 0 25 0 8 0 803158819 5116190720 1001887 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4664/tid=4670] ppid=4661 vsize=4996280 CPUtime=4961.03 cores=0-7
/proc/4664/task/4670/stat : 4670 (SArTagnan) R 4661 4664 4590 0 -1 4202560 31818116 0 0 0 491058 5045 0 0 25 0 8 0 803158819 5116190720 1001887 33554432000 4194304 4625164 140734073300448 18446744073709551615 4547221 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4664/tid=4671] ppid=4661 vsize=4996280 CPUtime=4961.84 cores=0-7
/proc/4664/task/4671/stat : 4671 (SArTagnan) R 4661 4664 4590 0 -1 4202560 93486 0 0 0 496136 48 0 0 25 0 8 0 803158819 5116190720 1001887 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=4664/tid=4672] ppid=4661 vsize=4996280 CPUtime=4961.82 cores=0-7
/proc/4664/task/4672/stat : 4672 (SArTagnan) R 4661 4664 4590 0 -1 4202560 171404 0 0 0 496145 37 0 0 25 0 8 0 803158819 5116190720 1001887 33554432000 4194304 4625164 140734073300448 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4664/tid=4673] ppid=4661 vsize=4996280 CPUtime=4961.15 cores=0-7
/proc/4664/task/4673/stat : 4673 (SArTagnan) R 4661 4664 4590 0 -1 4202560 172480 0 0 0 496074 41 0 0 25 0 8 0 803158819 5116190720 1001887 33554432000 4194304 4625164 140734073300448 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4664/tid=4675] ppid=4661 vsize=4996280 CPUtime=4893.75 cores=0-7
/proc/4664/task/4675/stat : 4675 (SArTagnan) R 4661 4664 4590 0 -1 4202560 121488 0 0 0 489292 83 0 0 25 0 8 0 803158819 5116190720 1001887 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39613.4
Current children cumulated vsize (KiB) 4996280



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.8 s]
/proc/loadavg: 8.29 8.22 8.16 9/454 4970
/proc/meminfo: memFree=28172476/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=5001032 CPUtime=40000.6 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) R 4661 4664 4590 0 -1 4202496 34042379 0 0 0 3994339 5720 0 0 25 0 8 0 803158811 5121056768 1011044 33554432000 4194304 4625164 140734073300448 18446744073709551615 4512364 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4664/statm: 1250258 1011044 352 106 0 1245761 0
[pid=4664/tid=4667] ppid=4661 vsize=5001032 CPUtime=5009.22 cores=0-7
/proc/4664/task/4667/stat : 4667 (SArTagnan) R 4661 4664 4590 0 -1 4202560 118356 0 0 0 500895 27 0 0 25 0 8 0 803158819 5121056768 1011044 33554432000 4194304 4625164 140734073300448 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4664/tid=4668] ppid=4661 vsize=5001032 CPUtime=4998.65 cores=0-7
/proc/4664/task/4668/stat : 4668 (SArTagnan) R 4661 4664 4590 0 -1 4202560 230458 0 0 0 499675 190 0 0 25 0 8 0 803158819 5121056768 1011044 33554432000 4194304 4625164 140734073300448 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4664/tid=4670] ppid=4661 vsize=5001032 CPUtime=5009.52 cores=0-7
/proc/4664/task/4670/stat : 4670 (SArTagnan) R 4661 4664 4590 0 -1 4202560 32217096 0 0 0 495850 5102 0 0 25 0 8 0 803158819 5121056768 1011044 33554432000 4194304 4625164 140734073300448 18446744073709551615 4288765 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4664/tid=4671] ppid=4661 vsize=5001032 CPUtime=5010.34 cores=0-7
/proc/4664/task/4671/stat : 4671 (SArTagnan) R 4661 4664 4590 0 -1 4202560 93555 0 0 0 500986 48 0 0 25 0 8 0 803158819 5121056768 1011044 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=4664/tid=4672] ppid=4661 vsize=5001032 CPUtime=5010.32 cores=0-7
/proc/4664/task/4672/stat : 4672 (SArTagnan) R 4661 4664 4590 0 -1 4202560 171546 0 0 0 500995 37 0 0 25 0 8 0 803158819 5121056768 1011044 33554432000 4194304 4625164 140734073300448 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4664/tid=4673] ppid=4661 vsize=5001032 CPUtime=5009.66 cores=0-7
/proc/4664/task/4673/stat : 4673 (SArTagnan) R 4661 4664 4590 0 -1 4202560 172800 0 0 0 500925 41 0 0 25 0 8 0 803158819 5121056768 1011044 33554432000 4194304 4625164 140734073300448 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4664/tid=4675] ppid=4661 vsize=5001032 CPUtime=4942.25 cores=0-7
/proc/4664/task/4675/stat : 4675 (SArTagnan) R 4661 4664 4590 0 -1 4202560 121526 0 0 0 494142 83 0 0 25 0 8 0 803158819 5121056768 1011044 33554432000 4194304 4625164 140734073300448 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 5001032

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.92 s]
/proc/loadavg: 8.29 8.22 8.16 9/454 4970
/proc/meminfo: memFree=28172476/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=0 CPUtime=40000.8 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) Z 4661 4664 4590 0 -1 4203548 34042379 0 0 0 3994350 5730 0 0 25 0 2 0 803158811 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4664/statm: 0 0 0 0 0 0 0
[pid=4664/tid=4668] ppid=4661 vsize=0 CPUtime=4998.75 cores=0-7
/proc/4664/task/4668/stat : 4668 (SArTagnan) R 4661 4664 4590 0 -1 4203588 230458 0 0 0 499675 200 0 0 25 0 2 0 803158819 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 0

[startup+5011 s]
/proc/loadavg: 8.29 8.22 8.16 9/454 4970
/proc/meminfo: memFree=28172476/32951124 swapFree=55639044/67111528
[pid=4664] ppid=4661 vsize=0 CPUtime=40000.9 cores=0-7
/proc/4664/stat : 4664 (SArTagnan) Z 4661 4664 4590 0 -1 4203548 34042379 0 0 0 3994350 5738 0 0 25 0 2 0 803158811 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4664/statm: 0 0 0 0 0 0 0
[pid=4664/tid=4668] ppid=4661 vsize=0 CPUtime=4998.83 cores=0-7
/proc/4664/task/4668/stat : 4668 (SArTagnan) R 4661 4664 4590 0 -1 4203588 230458 0 0 0 499675 208 0 0 25 0 2 0 803158819 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5011.07
CPU time (s): 40000.9
CPU user time (s): 39943.5
CPU system time (s): 57.4493
CPU usage (%): 798.252
Max. virtual memory (cumulated for all children) (KiB): 5433492

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39943.5
system time used= 57.4493
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 34042379
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 8560
involuntary context switches= 396614

runsolver used 20.7049 second user time and 56.5844 second system time

The end

Launcher Data 

Begin job on node133 at 2011-06-01 10:06:49
IDJOB=3438371
IDBENCH=71269
IDSOLVER=1936
FILE ID=node133/3438371-1306915609
RUNJOBID= node133-1306915609-4605
PBS_JOBID= 13488512
Free space on /tmp= 300 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/random/unforced/rbsat-v2400c266431g3.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3438371-1306915609/watcher-3438371-1306915609 -o /tmp/evaluation-result-3438371-1306915609/solver-3438371-1306915609 -C 40000 -W 5100 -M 31000  SArTagnan HOME/instance-3438371-1306915609.cnf -threads=8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 231f625a25c77518fbce98fe99f807b5
RANDOM SEED=1247525436

node133.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6741.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      10881188 kB
Buffers:        113660 kB
Cached:       21143872 kB
SwapCached:      10520 kB
Active:        5445060 kB
Inactive:     15833064 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      10881188 kB
SwapTotal:    67111528 kB
SwapFree:     55639044 kB
Dirty:            3888 kB
Writeback:           0 kB
AnonPages:       18072 kB
Mapped:          12692 kB
Slab:           667856 kB
PageTables:      53808 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 114888876 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73720 MiB
End job on node133 at 2011-06-01 11:30:22