Trace number 3386092

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 4094.08 1300.06

General information on the benchmark

NameCRAFTED/rbsat/crafted/forced/
rbsat-v945c61409gyes7.cnf
MD5SUM6200c727c8a5a324974090a1f9a6d7ab
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2.03669
Satisfiable
(Un)Satisfiability was proved
Number of variables945
Number of clauses61409
Sum of the clauses size123673
Maximum clause length21
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 261364
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386092-1305578948/watcher-3386092-1305578948 -o /tmp/evaluation-result-3386092-1305578948/solver-3386092-1305578948 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3386092-1305578948.cnf -threads=4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 10.35 10.17 10.13 7/373 28022
/proc/meminfo: memFree=14001388/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=20728 CPUtime=0.01 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 939 0 0 0 1 0 0 0 25 0 1 0 669494414 21225472 564 33554432000 4194304 4625164 140734666494016 18446744073709551615 274664427056 0 0 4096 2 0 0 0 17 4 0 0 0
/proc/28022/statm: 5182 564 296 106 0 685 0

[startup+0.139383 s]
/proc/loadavg: 10.35 10.17 10.13 7/373 28022
/proc/meminfo: memFree=14001388/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=188928 CPUtime=0.37 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 3279 0 0 0 37 0 0 0 25 0 4 0 669494414 193462272 2656 33554432000 4194304 4625164 140734666494016 18446744073709551615 4512312 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/28022/statm: 47232 2656 345 106 0 42735 0
Current children cumulated CPU time (s) 0.37
Current children cumulated vsize (KiB) 188928

[startup+0.200373 s]
/proc/loadavg: 10.35 10.17 10.13 7/373 28022
/proc/meminfo: memFree=14001388/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=189280 CPUtime=0.56 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 3714 0 0 0 56 0 0 0 25 0 4 0 669494414 193822720 3091 33554432000 4194304 4625164 140734666494016 18446744073709551615 4511757 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/28022/statm: 47320 3091 345 106 0 42823 0
Current children cumulated CPU time (s) 0.56
Current children cumulated vsize (KiB) 189280

[startup+0.300356 s]
/proc/loadavg: 10.35 10.17 10.13 7/373 28022
/proc/meminfo: memFree=14001388/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=191008 CPUtime=0.85 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 4323 0 0 0 85 0 0 0 25 0 4 0 669494414 195592192 3484 33554432000 4194304 4625164 140734666494016 18446744073709551615 4511938 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/28022/statm: 47752 3484 345 106 0 43255 0
Current children cumulated CPU time (s) 0.85
Current children cumulated vsize (KiB) 191008

[startup+0.700286 s]
/proc/loadavg: 10.35 10.17 10.13 7/373 28022
/proc/meminfo: memFree=14001388/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=194016 CPUtime=2.05 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 6611 0 0 0 205 0 0 0 25 0 4 0 669494414 198672384 4814 33554432000 4194304 4625164 140734666494016 18446744073709551615 4511874 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/28022/statm: 48504 4814 345 106 0 44007 0
Current children cumulated CPU time (s) 2.05
Current children cumulated vsize (KiB) 194016

[startup+1.51415 s]
/proc/loadavg: 10.35 10.17 10.13 11/377 28026
/proc/meminfo: memFree=13976644/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=263572 CPUtime=4.11 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 10109 0 0 0 409 2 0 0 25 0 4 0 669494414 269897728 6158 33554432000 4194304 4625164 140734666494016 18446744073709551615 4511973 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/28022/statm: 65893 6158 345 106 0 61396 0
[pid=28022/tid=28024] ppid=28020 vsize=263572 CPUtime=1.07 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 1476 0 0 0 107 0 0 0 25 0 4 0 669494416 269897728 6158 33554432000 4194304 4625164 140734666494016 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=263572 CPUtime=0.81 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 1589 0 0 0 81 0 0 0 25 0 4 0 669494416 269897728 6158 33554432000 4194304 4625164 140734666494016 18446744073709551615 4463778 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=263572 CPUtime=0.99 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 2957 0 0 0 99 0 0 0 25 0 4 0 669494416 269897728 6158 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4.11
Current children cumulated vsize (KiB) 263572

[startup+3.10087 s]
/proc/loadavg: 10.35 10.17 10.13 11/377 28026
/proc/meminfo: memFree=13966104/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=265976 CPUtime=8.74 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 16313 0 0 0 871 3 0 0 25 0 4 0 669494414 272359424 8740 33554432000 4194304 4625164 140734666494016 18446744073709551615 4519328 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/28022/statm: 66494 8740 346 106 0 61997 0
[pid=28022/tid=28024] ppid=28020 vsize=265976 CPUtime=2.44 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 2788 0 0 0 244 0 0 0 25 0 4 0 669494416 272359424 8740 33554432000 4194304 4625164 140734666494016 18446744073709551615 4462375 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=265976 CPUtime=1.61 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 2919 0 0 0 161 0 0 0 25 0 4 0 669494416 272359424 8740 33554432000 4194304 4625164 140734666494016 18446744073709551615 4352010 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=265976 CPUtime=1.87 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 6387 0 0 0 186 1 0 0 25 0 4 0 669494416 272359424 8741 33554432000 4194304 4625164 140734666494016 18446744073709551615 4334485 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 8.74
Current children cumulated vsize (KiB) 265976

[startup+6.30032 s]
/proc/loadavg: 10.32 10.16 10.13 11/377 28026
/proc/meminfo: memFree=13947496/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=273196 CPUtime=18.24 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 36625 0 0 0 1817 7 0 0 25 0 4 0 669494414 279752704 13330 33554432000 4194304 4625164 140734666494016 18446744073709551615 4511963 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/28022/statm: 68299 13330 346 106 0 63802 0
[pid=28022/tid=28024] ppid=28020 vsize=273196 CPUtime=3.86 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 4251 0 0 0 385 1 0 0 25 0 4 0 669494416 279752704 13330 33554432000 4194304 4625164 140734666494016 18446744073709551615 4291687 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=273196 CPUtime=4.77 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 6959 0 0 0 476 1 0 0 25 0 4 0 669494416 279752704 13330 33554432000 4194304 4625164 140734666494016 18446744073709551615 4339865 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=273196 CPUtime=3.6 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 20374 0 0 0 356 4 0 0 25 0 4 0 669494416 279752704 13330 33554432000 4194304 4625164 140734666494016 18446744073709551615 274664505440 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 18.24
Current children cumulated vsize (KiB) 273196

[startup+12.7002 s]
/proc/loadavg: 10.30 10.16 10.13 11/377 28026
/proc/meminfo: memFree=13913056/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=288460 CPUtime=37.35 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 64303 0 0 0 3722 13 0 0 25 0 4 0 669494414 295383040 20841 33554432000 4194304 4625164 140734666494016 18446744073709551615 4392981 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/28022/statm: 72115 20841 349 106 0 67618 0
[pid=28022/tid=28024] ppid=28020 vsize=288460 CPUtime=9.53 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 8435 0 0 0 952 1 0 0 25 0 4 0 669494416 295383040 20841 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=288460 CPUtime=8.31 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 10633 0 0 0 828 3 0 0 25 0 4 0 669494416 295383040 20841 33554432000 4194304 4625164 140734666494016 18446744073709551615 4291811 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=288460 CPUtime=7.21 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 38145 0 0 0 715 6 0 0 25 0 4 0 669494416 295383040 20841 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37.35
Current children cumulated vsize (KiB) 288460

[startup+25.501 s]
/proc/loadavg: 10.23 10.15 10.13 11/377 28026
/proc/meminfo: memFree=13866432/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=318412 CPUtime=75.59 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 92654 0 0 0 7540 19 0 0 25 0 4 0 669494414 326053888 31199 33554432000 4194304 4625164 140734666494016 18446744073709551615 4512156 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/28022/statm: 79603 31199 349 106 0 75106 0
[pid=28022/tid=28024] ppid=28020 vsize=318412 CPUtime=17.08 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 12346 0 0 0 1706 2 0 0 25 0 4 0 669494416 326053888 31199 33554432000 4194304 4625164 140734666494016 18446744073709551615 4339554 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=318412 CPUtime=16.61 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 15820 0 0 0 1656 5 0 0 25 0 4 0 669494416 326053888 31199 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=318412 CPUtime=16.8 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 52609 0 0 0 1672 8 0 0 25 0 4 0 669494416 326053888 31199 33554432000 4194304 4625164 140734666494016 18446744073709551615 4579418 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 75.59
Current children cumulated vsize (KiB) 318412

[startup+51.1006 s]
/proc/loadavg: 10.23 10.16 10.13 11/377 28027
/proc/meminfo: memFree=13777372/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=350156 CPUtime=151.8 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 281251 0 0 0 15130 50 0 0 25 0 4 0 669494414 358559744 49238 33554432000 4194304 4625164 140734666494016 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28022/statm: 87539 49238 346 106 0 83042 0
[pid=28022/tid=28024] ppid=28020 vsize=350156 CPUtime=38.19 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 19461 0 0 0 3815 4 0 0 25 0 4 0 669494416 358559744 49238 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=350156 CPUtime=34.76 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 22703 0 0 0 3467 9 0 0 25 0 4 0 669494416 358559744 49238 33554432000 4194304 4625164 140734666494016 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=350156 CPUtime=31.89 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 214113 0 0 0 3157 32 0 0 25 0 4 0 669494416 358559744 49238 33554432000 4194304 4625164 140734666494016 18446744073709551615 4563696 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 151.8
Current children cumulated vsize (KiB) 350156

[startup+102.301 s]
/proc/loadavg: 10.17 10.16 10.13 11/377 28028
/proc/meminfo: memFree=13656716/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=427424 CPUtime=304.71 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 416647 0 0 0 30398 73 0 0 25 0 4 0 669494414 437682176 72756 33554432000 4194304 4625164 140734666494016 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28022/statm: 106856 72756 359 106 0 102359 0
[pid=28022/tid=28024] ppid=28020 vsize=427424 CPUtime=83.7 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 30835 0 0 0 8362 8 0 0 25 0 4 0 669494416 437682176 72756 33554432000 4194304 4625164 140734666494016 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=427424 CPUtime=60.01 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 30005 0 0 0 5991 10 0 0 25 0 4 0 669494416 437682176 72756 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=427424 CPUtime=67.79 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 306502 0 0 0 6732 47 0 0 25 0 4 0 669494416 437682176 72756 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 304.71
Current children cumulated vsize (KiB) 427424

[startup+162.3 s]
/proc/loadavg: 10.16 10.17 10.13 11/377 28030
/proc/meminfo: memFree=13509892/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=533436 CPUtime=483.97 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 503311 0 0 0 48309 88 0 0 25 0 4 0 669494414 546238464 86839 33554432000 4194304 4625164 140734666494016 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28022/statm: 133359 86839 382 106 0 128862 0
[pid=28022/tid=28024] ppid=28020 vsize=533436 CPUtime=143.69 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 32859 0 0 0 14360 9 0 0 25 0 4 0 669494416 546238464 86839 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=533436 CPUtime=89.71 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 30505 0 0 0 8961 10 0 0 25 0 4 0 669494416 546238464 86839 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=533436 CPUtime=98.09 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 368420 0 0 0 9751 58 0 0 25 0 4 0 669494416 546238464 86839 33554432000 4194304 4625164 140734666494016 18446744073709551615 4352053 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 483.97
Current children cumulated vsize (KiB) 533436

[startup+222.301 s]
/proc/loadavg: 10.06 10.14 10.12 11/377 28032
/proc/meminfo: memFree=13451856/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=549772 CPUtime=663.23 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 800652 0 0 0 66191 132 0 0 25 0 4 0 669494414 562966528 94814 33554432000 4194304 4625164 140734666494016 18446744073709551615 4393112 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28022/statm: 137443 94814 382 106 0 132946 0
[pid=28022/tid=28024] ppid=28020 vsize=549772 CPUtime=203.68 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 43771 0 0 0 20358 10 0 0 25 0 4 0 669494416 562966528 94814 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=549772 CPUtime=119.42 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 31337 0 0 0 11931 11 0 0 25 0 4 0 669494416 562966528 94814 33554432000 4194304 4625164 140734666494016 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=549772 CPUtime=128.38 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 653552 0 0 0 12739 99 0 0 25 0 4 0 669494416 562966528 94814 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351866 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 663.23
Current children cumulated vsize (KiB) 549772

[startup+282.301 s]

################
# More data... #
################

[pid=28022] ppid=28020 vsize=1076272 CPUtime=2324.15 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 4314867 0 0 0 231769 646 0 0 25 0 4 0 669494414 1102102528 199647 33554432000 4194304 4625164 140734666494016 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28022/statm: 269068 199647 418 106 0 264571 0
[pid=28022/tid=28024] ppid=28020 vsize=1076272 CPUtime=718.62 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 104303 0 0 0 71836 26 0 0 25 0 4 0 669494416 1102102528 199647 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=1076272 CPUtime=428.8 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 59709 0 0 0 42859 21 0 0 25 0 4 0 669494416 1102102528 199647 33554432000 4194304 4625164 140734666494016 18446744073709551615 4339591 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=1076272 CPUtime=441.17 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 3977687 0 0 0 43558 559 0 0 25 0 4 0 669494416 1102102528 199647 33554432000 4194304 4625164 140734666494016 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2324.15
Current children cumulated vsize (KiB) 1076272

[startup+822.314 s]
/proc/loadavg: 10.45 10.22 10.14 8/371 28121
/proc/meminfo: memFree=13791064/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=1117832 CPUtime=2541.91 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 4584388 0 0 0 253493 698 0 0 25 0 4 0 669494414 1144659968 211684 33554432000 4194304 4625164 140734666494016 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28022/statm: 279458 211684 463 106 0 274961 0
[pid=28022/tid=28024] ppid=28020 vsize=1117832 CPUtime=778.63 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 105214 0 0 0 77834 29 0 0 25 0 4 0 669494416 1144659968 211684 33554432000 4194304 4625164 140734666494016 18446744073709551615 4339569 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=1117832 CPUtime=488.72 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 60106 0 0 0 48848 24 0 0 25 0 4 0 669494416 1144659968 211684 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=1117832 CPUtime=479.73 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 4243351 0 0 0 47373 600 0 0 25 0 4 0 669494416 1144659968 211684 33554432000 4194304 4625164 140734666494016 18446744073709551615 4335165 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2541.91
Current children cumulated vsize (KiB) 1117832

[startup+882.301 s]
/proc/loadavg: 10.87 10.35 10.18 11/377 28239
/proc/meminfo: memFree=13827580/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=1122500 CPUtime=2757.47 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 4590211 0 0 0 275041 706 0 0 25 0 4 0 669494414 1149440000 217089 33554432000 4194304 4625164 140734666494016 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/28022/statm: 280625 217089 463 106 0 276128 0
[pid=28022/tid=28024] ppid=28020 vsize=1122500 CPUtime=838.62 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 106878 0 0 0 83831 31 0 0 25 0 4 0 669494416 1149440000 217089 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=1122500 CPUtime=548.71 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 60314 0 0 0 54846 25 0 0 25 0 4 0 669494416 1149440000 217089 33554432000 4194304 4625164 140734666494016 18446744073709551615 4352098 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=1122500 CPUtime=528.02 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 4243696 0 0 0 52200 602 0 0 25 0 4 0 669494416 1149440000 217089 33554432000 4194304 4625164 140734666494016 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2757.47
Current children cumulated vsize (KiB) 1122500

[startup+942.3 s]
/proc/loadavg: 10.52 10.33 10.19 11/377 28271
/proc/meminfo: memFree=13274492/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=1122900 CPUtime=2991.96 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 4598651 0 0 0 298486 710 0 0 25 0 4 0 669494414 1149849600 224970 33554432000 4194304 4625164 140734666494016 18446744073709551615 4512092 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/28022/statm: 280725 224970 461 106 0 276228 0
[pid=28022/tid=28024] ppid=28020 vsize=1122900 CPUtime=893.83 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 110568 0 0 0 89350 33 0 0 25 0 4 0 669494416 1149849600 224970 33554432000 4194304 4625164 140734666494016 18446744073709551615 4339536 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=1122900 CPUtime=608.7 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 60537 0 0 0 60845 25 0 0 25 0 4 0 669494416 1149849600 224970 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=1122900 CPUtime=587.31 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 4244415 0 0 0 58128 603 0 0 25 0 4 0 669494416 1149849600 224970 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2991.96
Current children cumulated vsize (KiB) 1122900

[startup+1002.3 s]
/proc/loadavg: 10.30 10.30 10.18 11/377 28273
/proc/meminfo: memFree=13197856/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=1220000 CPUtime=3205.06 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 4721326 0 0 0 319777 729 0 0 25 0 4 0 669494414 1249280000 238618 33554432000 4194304 4625164 140734666494016 18446744073709551615 4393124 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/28022/statm: 305000 238618 392 106 0 300503 0
[pid=28022/tid=28024] ppid=28020 vsize=1220000 CPUtime=953.83 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 114537 0 0 0 95346 37 0 0 25 0 4 0 669494416 1249280000 238618 33554432000 4194304 4625164 140734666494016 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=1220000 CPUtime=652.32 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 60631 0 0 0 65206 26 0 0 25 0 4 0 669494416 1249280000 238618 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=1220000 CPUtime=646.6 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 4306562 0 0 0 64049 611 0 0 25 0 4 0 669494416 1249280000 238618 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3205.06
Current children cumulated vsize (KiB) 1220000

[startup+1062.3 s]
/proc/loadavg: 10.49 10.36 10.21 11/377 28277
/proc/meminfo: memFree=13152852/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=1243264 CPUtime=3382.94 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 5128126 0 0 0 337499 795 0 0 25 0 4 0 669494414 1273102336 243051 33554432000 4194304 4625164 140734666494016 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/28022/statm: 310816 243051 352 106 0 306319 0
[pid=28022/tid=28024] ppid=28020 vsize=1243264 CPUtime=995.23 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 116631 0 0 0 99482 41 0 0 25 0 4 0 669494416 1273102336 243051 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=1243264 CPUtime=698.75 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 61059 0 0 0 69848 27 0 0 25 0 4 0 669494416 1273102336 243051 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=1243264 CPUtime=695.96 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 4697300 0 0 0 68926 670 0 0 25 0 4 0 669494416 1273102336 243051 33554432000 4194304 4625164 140734666494016 18446744073709551615 4547606 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3382.94
Current children cumulated vsize (KiB) 1243264

[startup+1122.3 s]
/proc/loadavg: 10.32 10.34 10.21 11/377 28279
/proc/meminfo: memFree=13091472/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=1346784 CPUtime=3562.22 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 5441449 0 0 0 355383 839 0 0 25 0 4 0 669494414 1379106816 258296 33554432000 4194304 4625164 140734666494016 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/28022/statm: 336696 258296 352 106 0 332199 0
[pid=28022/tid=28024] ppid=28020 vsize=1346784 CPUtime=1029.68 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 154048 0 0 0 102921 47 0 0 25 0 4 0 669494416 1379106816 258296 33554432000 4194304 4625164 140734666494016 18446744073709551615 4352047 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=1346784 CPUtime=758.63 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 61549 0 0 0 75836 27 0 0 25 0 4 0 669494416 1379106816 258296 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=1346784 CPUtime=725.64 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 4970690 0 0 0 71856 708 0 0 25 0 4 0 669494416 1379106816 258296 33554432000 4194304 4625164 140734666494016 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3562.22
Current children cumulated vsize (KiB) 1346784

[startup+1182.3 s]
/proc/loadavg: 10.15 10.29 10.19 11/377 28280
/proc/meminfo: memFree=13056504/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=1346784 CPUtime=3741.5 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 5765909 0 0 0 373265 885 0 0 25 0 4 0 669494414 1379106816 264511 33554432000 4194304 4625164 140734666494016 18446744073709551615 4393080 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/28022/statm: 336696 264511 352 106 0 332199 0
[pid=28022/tid=28024] ppid=28020 vsize=1346784 CPUtime=1059.37 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 156492 0 0 0 105888 49 0 0 25 0 4 0 669494416 1379106816 264511 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=1346784 CPUtime=815.62 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 61793 0 0 0 81535 27 0 0 25 0 4 0 669494416 1379106816 264511 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=1346784 CPUtime=758.23 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 5290954 0 0 0 75073 750 0 0 25 0 4 0 669494416 1379106816 264511 33554432000 4194304 4625164 140734666494016 18446744073709551615 4334485 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3741.5
Current children cumulated vsize (KiB) 1346784

[startup+1242.3 s]
/proc/loadavg: 10.18 10.28 10.19 11/377 28282
/proc/meminfo: memFree=13009380/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=1346784 CPUtime=3921.57 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 6079590 0 0 0 391227 930 0 0 25 0 4 0 669494414 1379106816 268420 33554432000 4194304 4625164 140734666494016 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/28022/statm: 336696 268420 352 106 0 332199 0
[pid=28022/tid=28024] ppid=28020 vsize=1346784 CPUtime=1090.17 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 158107 0 0 0 108965 52 0 0 25 0 4 0 669494416 1379106816 268420 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=1346784 CPUtime=874.84 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 62030 0 0 0 87457 27 0 0 25 0 4 0 669494416 1379106816 268420 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=1346784 CPUtime=788.28 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 5601141 0 0 0 78036 792 0 0 25 0 4 0 669494416 1379106816 268420 33554432000 4194304 4625164 140734666494016 18446744073709551615 4335584 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3921.57
Current children cumulated vsize (KiB) 1346784



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 10.13 10.25 10.19 11/377 28283
/proc/meminfo: memFree=12947384/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=1346784 CPUtime=4094.02 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 6391071 0 0 0 408428 974 0 0 25 0 4 0 669494414 1379106816 273186 33554432000 4194304 4625164 140734666494016 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/28022/statm: 336696 273186 355 106 0 332199 0
[pid=28022/tid=28024] ppid=28020 vsize=1346784 CPUtime=1119.57 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 159542 0 0 0 111903 54 0 0 25 0 4 0 669494416 1379106816 273186 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=1346784 CPUtime=931.75 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 62531 0 0 0 93147 28 0 0 25 0 4 0 669494416 1379106816 273186 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=1346784 CPUtime=816.71 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 5908931 0 0 0 80838 833 0 0 25 0 4 0 669494416 1379106816 273186 33554432000 4194304 4625164 140734666494016 18446744073709551615 4334485 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4094.02
Current children cumulated vsize (KiB) 1346784

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1300 s]
/proc/loadavg: 10.13 10.25 10.19 11/377 28283
/proc/meminfo: memFree=12947384/32951132 swapFree=67093824/67111528
[pid=28022] ppid=28020 vsize=1346784 CPUtime=4094.02 cores=0,2,4,6
/proc/28022/stat : 28022 (SArTagnan) R 28020 28022 27834 0 -1 4202496 6391071 0 0 0 408428 974 0 0 25 0 4 0 669494414 1379106816 273186 33554432000 4194304 4625164 140734666494016 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/28022/statm: 336696 273186 355 106 0 332199 0
[pid=28022/tid=28024] ppid=28020 vsize=1346784 CPUtime=1119.57 cores=0,2,4,6
/proc/28022/task/28024/stat : 28024 (SArTagnan) R 28020 28022 27834 0 -1 4202560 159542 0 0 0 111903 54 0 0 25 0 4 0 669494416 1379106816 273186 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28022/tid=28025] ppid=28020 vsize=1346784 CPUtime=931.75 cores=0,2,4,6
/proc/28022/task/28025/stat : 28025 (SArTagnan) R 28020 28022 27834 0 -1 4202560 62531 0 0 0 93147 28 0 0 25 0 4 0 669494416 1379106816 273186 33554432000 4194304 4625164 140734666494016 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28022/tid=28026] ppid=28020 vsize=1346784 CPUtime=816.71 cores=0,2,4,6
/proc/28022/task/28026/stat : 28026 (SArTagnan) R 28020 28022 27834 0 -1 4202560 5908931 0 0 0 80838 833 0 0 25 0 4 0 669494416 1379106816 273186 33554432000 4194304 4625164 140734666494016 18446744073709551615 4334485 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4094.02
Current children cumulated vsize (KiB) 1346784

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1300.06
CPU time (s): 4094.08
CPU user time (s): 4084.3
CPU system time (s): 9.78351
CPU usage (%): 314.914
Max. virtual memory (cumulated for all children) (KiB): 1457084

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4084.3
system time used= 9.78351
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6391071
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3043
involuntary context switches= 40454

runsolver used 4.42433 second user time and 11.3703 second system time

The end

Launcher Data 

Begin job on node131 at 2011-05-16 22:49:08
IDJOB=3386092
IDBENCH=71540
IDSOLVER=1871
FILE ID=node131/3386092-1305578948
RUNJOBID= node131-1305576906-27852
PBS_JOBID= 13350211
Free space on /tmp= 73572 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/forced/rbsat-v945c61409gyes7.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386092-1305578948/watcher-3386092-1305578948 -o /tmp/evaluation-result-3386092-1305578948/solver-3386092-1305578948 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3386092-1305578948.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 6200c727c8a5a324974090a1f9a6d7ab
RANDOM SEED=1092594408

node131.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      14001560 kB
Buffers:        308904 kB
Cached:        4718644 kB
SwapCached:       1504 kB
Active:       14981216 kB
Inactive:      3686864 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      14001560 kB
SwapTotal:    67111528 kB
SwapFree:     67093824 kB
Dirty:            1228 kB
Writeback:           0 kB
AnonPages:    13639464 kB
Mapped:          33108 kB
Slab:           170580 kB
PageTables:      47736 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS: 66766648 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73564 MiB
End job on node131 at 2011-05-16 23:10:50