Trace number 3439226

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 40000.7 5010.86

General information on the benchmark

NameSAT03/handmade/gomes/bqwh/
bqwh.50.825.shuffled-as.sat03-1648.cnf
MD5SUM05a191325d89de504821b55dea51ca7f
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2.04769
Satisfiable
(Un)Satisfiability was proved
Number of variables4979
Number of clauses43754
Sum of the clauses size97507
Maximum clause length12
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 241316
Number of clauses of size 3117
Number of clauses of size 4287
Number of clauses of size 5525
Number of clauses of size over 51509

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3439226-1306960406/watcher-3439226-1306960406 -o /tmp/evaluation-result-3439226-1306960406/solver-3439226-1306960406 -C 40000 -W 5100 -M 31000 SArTagnan HOME/instance-3439226-1306960406.cnf -threads=8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.00 0.00 0.00 1/474 30017
/proc/meminfo: memFree=27586628/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=26004 CPUtime=0.01 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 2012 0 0 0 1 0 0 0 25 0 1 0 807640275 26628096 1709 33554432000 4194304 4625164 140736155157216 18446744073709551615 208079929552 0 0 4096 2 0 0 0 17 2 0 0 0
/proc/30017/statm: 6501 1720 298 106 0 2004 0

[startup+0.0355661 s]
/proc/loadavg: 0.00 0.00 0.00 1/474 30017
/proc/meminfo: memFree=27586628/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=362568 CPUtime=0.13 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 3054 0 0 0 13 0 0 0 25 0 8 0 807640275 371269632 2745 33554432000 4194304 4625164 140736155157216 18446744073709551615 208079899650 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 90642 2745 348 106 0 86145 0
Current children cumulated CPU time (s) 0.13
Current children cumulated vsize (KiB) 362568

[startup+0.113459 s]
/proc/loadavg: 0.00 0.00 0.00 1/474 30017
/proc/meminfo: memFree=27586628/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=507872 CPUtime=0.75 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 14521 0 0 0 74 1 0 0 25 0 8 0 807640275 520060928 9420 33554432000 4194304 4625164 140736155157216 18446744073709551615 4339569 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 126968 9448 349 106 0 122471 0
Current children cumulated CPU time (s) 0.75
Current children cumulated vsize (KiB) 507872

[startup+0.300168 s]
/proc/loadavg: 0.00 0.00 0.00 1/474 30017
/proc/meminfo: memFree=27586628/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=512944 CPUtime=2.25 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 19530 0 0 0 223 2 0 0 25 0 8 0 807640275 525254656 10637 33554432000 4194304 4625164 140736155157216 18446744073709551615 4513092 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 128236 10637 352 106 0 123739 0
Current children cumulated CPU time (s) 2.25
Current children cumulated vsize (KiB) 512944

[startup+0.701091 s]
/proc/loadavg: 0.00 0.00 0.00 1/474 30017
/proc/meminfo: memFree=27586628/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=513420 CPUtime=5.46 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 22872 0 0 0 543 3 0 0 25 0 8 0 807640275 525742080 12553 33554432000 4194304 4625164 140736155157216 18446744073709551615 4425328 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 128355 12553 352 106 0 123858 0
Current children cumulated CPU time (s) 5.46
Current children cumulated vsize (KiB) 513420

[startup+1.50094 s]
/proc/loadavg: 0.00 0.00 0.00 9/482 30025
/proc/meminfo: memFree=27532384/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=522192 CPUtime=11.84 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 34460 0 0 0 1180 4 0 0 25 0 8 0 807640275 534724608 16709 33554432000 4194304 4625164 140736155157216 18446744073709551615 4338055 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 130548 16709 353 106 0 126051 0
[pid=30017/tid=30019] ppid=30015 vsize=522192 CPUtime=1.47 cores=0-7
/proc/30017/task/30019/stat : 30019 (SArTagnan) R 30015 30017 29991 0 -1 4202560 2982 0 0 0 147 0 0 0 25 0 8 0 807640277 534724608 16709 33554432000 4194304 4625164 140736155157216 18446744073709551615 4278640 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=30017/tid=30020] ppid=30015 vsize=522192 CPUtime=1.47 cores=0-7
/proc/30017/task/30020/stat : 30020 (SArTagnan) R 30015 30017 29991 0 -1 4202560 2900 0 0 0 147 0 0 0 25 0 8 0 807640277 534724608 16710 33554432000 4194304 4625164 140736155157216 18446744073709551615 4352577 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=30017/tid=30021] ppid=30015 vsize=522192 CPUtime=1.47 cores=0-7
/proc/30017/task/30021/stat : 30021 (SArTagnan) R 30015 30017 29991 0 -1 4202560 11034 0 0 0 146 1 0 0 25 0 8 0 807640277 534724608 16710 33554432000 4194304 4625164 140736155157216 18446744073709551615 4334485 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=30017/tid=30022] ppid=30015 vsize=522192 CPUtime=1.47 cores=0-7
/proc/30017/task/30022/stat : 30022 (SArTagnan) R 30015 30017 29991 0 -1 4202560 2710 0 0 0 147 0 0 0 25 0 8 0 807640277 534724608 16710 33554432000 4194304 4625164 140736155157216 18446744073709551615 208079893189 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=30017/tid=30023] ppid=30015 vsize=522192 CPUtime=1.47 cores=0-7
/proc/30017/task/30023/stat : 30023 (SArTagnan) R 30015 30017 29991 0 -1 4202560 2702 0 0 0 147 0 0 0 25 0 8 0 807640277 534724608 16710 33554432000 4194304 4625164 140736155157216 18446744073709551615 4348903 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=30017/tid=30024] ppid=30015 vsize=522192 CPUtime=1.48 cores=0-7
/proc/30017/task/30024/stat : 30024 (SArTagnan) R 30015 30017 29991 0 -1 4202560 2154 0 0 0 148 0 0 0 25 0 8 0 807640277 534724608 16711 33554432000 4194304 4625164 140736155157216 18446744073709551615 4352532 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=30017/tid=30025] ppid=30015 vsize=522192 CPUtime=1.45 cores=0-7
/proc/30017/task/30025/stat : 30025 (SArTagnan) R 30015 30017 29991 0 -1 4202560 2789 0 0 0 145 0 0 0 25 0 8 0 807640277 534724608 16711 33554432000 4194304 4625164 140736155157216 18446744073709551615 4352072 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.84
Current children cumulated vsize (KiB) 522192

[startup+3.10064 s]
/proc/loadavg: 0.64 0.13 0.04 9/482 30026
/proc/meminfo: memFree=27504380/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=531088 CPUtime=24.62 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 53165 0 0 0 2454 8 0 0 25 0 8 0 807640275 543834112 26416 33554432000 4194304 4625164 140736155157216 18446744073709551615 4337665 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 132772 26416 353 106 0 128275 0
[pid=30017/tid=30019] ppid=30015 vsize=531088 CPUtime=3.07 cores=0-7
/proc/30017/task/30019/stat : 30019 (SArTagnan) R 30015 30017 29991 0 -1 4202560 4002 0 0 0 307 0 0 0 25 0 8 0 807640277 543834112 26416 33554432000 4194304 4625164 140736155157216 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=30017/tid=30020] ppid=30015 vsize=531088 CPUtime=3.07 cores=0-7
/proc/30017/task/30020/stat : 30020 (SArTagnan) R 30015 30017 29991 0 -1 4202560 3658 0 0 0 307 0 0 0 25 0 8 0 807640277 543834112 26416 33554432000 4194304 4625164 140736155157216 18446744073709551615 4351942 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=30017/tid=30021] ppid=30015 vsize=531088 CPUtime=3.07 cores=0-7
/proc/30017/task/30021/stat : 30021 (SArTagnan) R 30015 30017 29991 0 -1 4202560 23459 0 0 0 304 3 0 0 25 0 8 0 807640277 543834112 26416 33554432000 4194304 4625164 140736155157216 18446744073709551615 4334466 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=30017/tid=30022] ppid=30015 vsize=531088 CPUtime=3.07 cores=0-7
/proc/30017/task/30022/stat : 30022 (SArTagnan) R 30015 30017 29991 0 -1 4202560 3269 0 0 0 307 0 0 0 25 0 8 0 807640277 543834112 26416 33554432000 4194304 4625164 140736155157216 18446744073709551615 4352280 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=30017/tid=30023] ppid=30015 vsize=531088 CPUtime=3.08 cores=0-7
/proc/30017/task/30023/stat : 30023 (SArTagnan) R 30015 30017 29991 0 -1 4202560 3417 0 0 0 307 1 0 0 25 0 8 0 807640277 543834112 26416 33554432000 4194304 4625164 140736155157216 18446744073709551615 4374304 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=30017/tid=30024] ppid=30015 vsize=531088 CPUtime=3.08 cores=0-7
/proc/30017/task/30024/stat : 30024 (SArTagnan) R 30015 30017 29991 0 -1 4202560 3105 0 0 0 308 0 0 0 25 0 8 0 807640277 543834112 26416 33554432000 4194304 4625164 140736155157216 18446744073709551615 208079893537 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=30017/tid=30025] ppid=30015 vsize=531088 CPUtime=3.04 cores=0-7
/proc/30017/task/30025/stat : 30025 (SArTagnan) R 30015 30017 29991 0 -1 4202560 3650 0 0 0 303 1 0 0 25 0 8 0 807640277 543834112 26416 33554432000 4194304 4625164 140736155157216 18446744073709551615 4352285 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.62
Current children cumulated vsize (KiB) 531088

[startup+6.30102 s]
/proc/loadavg: 0.64 0.13 0.04 9/482 30026
/proc/meminfo: memFree=27431832/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=549964 CPUtime=50.16 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 91093 0 0 0 4999 17 0 0 25 0 8 0 807640275 563163136 42896 33554432000 4194304 4625164 140736155157216 18446744073709551615 4338085 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 137491 42897 353 106 0 132994 0
[pid=30017/tid=30019] ppid=30015 vsize=549964 CPUtime=6.28 cores=0-7
/proc/30017/task/30019/stat : 30019 (SArTagnan) R 30015 30017 29991 0 -1 4202560 6106 0 0 0 627 1 0 0 25 0 8 0 807640277 563163136 42897 33554432000 4194304 4625164 140736155157216 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=30017/tid=30020] ppid=30015 vsize=549964 CPUtime=6.27 cores=0-7
/proc/30017/task/30020/stat : 30020 (SArTagnan) R 30015 30017 29991 0 -1 4202560 6159 0 0 0 626 1 0 0 25 0 8 0 807640277 563163136 42898 33554432000 4194304 4625164 140736155157216 18446744073709551615 208079893537 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=30017/tid=30021] ppid=30015 vsize=549964 CPUtime=6.27 cores=0-7
/proc/30017/task/30021/stat : 30021 (SArTagnan) R 30015 30017 29991 0 -1 4202560 46023 0 0 0 620 7 0 0 25 0 8 0 807640277 563163136 42898 33554432000 4194304 4625164 140736155157216 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=30017/tid=30022] ppid=30015 vsize=549964 CPUtime=6.27 cores=0-7
/proc/30017/task/30022/stat : 30022 (SArTagnan) R 30015 30017 29991 0 -1 4202560 5284 0 0 0 626 1 0 0 25 0 8 0 807640277 563163136 42898 33554432000 4194304 4625164 140736155157216 18446744073709551615 208079893537 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=30017/tid=30023] ppid=30015 vsize=549964 CPUtime=6.27 cores=0-7
/proc/30017/task/30023/stat : 30023 (SArTagnan) R 30015 30017 29991 0 -1 4202560 5514 0 0 0 626 1 0 0 25 0 8 0 807640277 563163136 42898 33554432000 4194304 4625164 140736155157216 18446744073709551615 4348864 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=30017/tid=30024] ppid=30015 vsize=549964 CPUtime=6.27 cores=0-7
/proc/30017/task/30024/stat : 30024 (SArTagnan) R 30015 30017 29991 0 -1 4202560 4845 0 0 0 627 0 0 0 25 0 8 0 807640277 563163136 42898 33554432000 4194304 4625164 140736155157216 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=30017/tid=30025] ppid=30015 vsize=549964 CPUtime=6.19 cores=0-7
/proc/30017/task/30025/stat : 30025 (SArTagnan) R 30015 30017 29991 0 -1 4202560 6314 0 0 0 617 2 0 0 25 0 8 0 807640277 563163136 42898 33554432000 4194304 4625164 140736155157216 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.16
Current children cumulated vsize (KiB) 549964

[startup+12.7008 s]
/proc/loadavg: 1.77 0.39 0.13 9/482 30026
/proc/meminfo: memFree=27312804/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=589240 CPUtime=101.25 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 165444 0 0 0 10092 33 0 0 25 0 8 0 807640275 603381760 71707 33554432000 4194304 4625164 140736155157216 18446744073709551615 4396555 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 147310 71707 353 106 0 142813 0
[pid=30017/tid=30019] ppid=30015 vsize=589240 CPUtime=12.67 cores=0-7
/proc/30017/task/30019/stat : 30019 (SArTagnan) R 30015 30017 29991 0 -1 4202560 9856 0 0 0 1266 1 0 0 25 0 8 0 807640277 603381760 71707 33554432000 4194304 4625164 140736155157216 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=30017/tid=30020] ppid=30015 vsize=589240 CPUtime=12.66 cores=0-7
/proc/30017/task/30020/stat : 30020 (SArTagnan) R 30015 30017 29991 0 -1 4202560 10608 0 0 0 1264 2 0 0 25 0 8 0 807640277 603381760 71707 33554432000 4194304 4625164 140736155157216 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=30017/tid=30021] ppid=30015 vsize=589240 CPUtime=12.67 cores=0-7
/proc/30017/task/30021/stat : 30021 (SArTagnan) R 30015 30017 29991 0 -1 4202560 92531 0 0 0 1253 14 0 0 25 0 8 0 807640277 603381760 71707 33554432000 4194304 4625164 140736155157216 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=30017/tid=30022] ppid=30015 vsize=589240 CPUtime=12.67 cores=0-7
/proc/30017/task/30022/stat : 30022 (SArTagnan) R 30015 30017 29991 0 -1 4202560 9272 0 0 0 1265 2 0 0 25 0 8 0 807640277 603381760 71707 33554432000 4194304 4625164 140736155157216 18446744073709551615 4464349 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=30017/tid=30023] ppid=30015 vsize=589240 CPUtime=12.67 cores=0-7
/proc/30017/task/30023/stat : 30023 (SArTagnan) R 30015 30017 29991 0 -1 4202560 9353 0 0 0 1264 3 0 0 25 0 8 0 807640277 603381760 71707 33554432000 4194304 4625164 140736155157216 18446744073709551615 4348821 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=30017/tid=30024] ppid=30015 vsize=589240 CPUtime=12.68 cores=0-7
/proc/30017/task/30024/stat : 30024 (SArTagnan) R 30015 30017 29991 0 -1 4202560 8572 0 0 0 1266 2 0 0 25 0 8 0 807640277 603381760 71707 33554432000 4194304 4625164 140736155157216 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=30017/tid=30025] ppid=30015 vsize=589240 CPUtime=12.48 cores=0-7
/proc/30017/task/30025/stat : 30025 (SArTagnan) R 30015 30017 29991 0 -1 4202560 10335 0 0 0 1245 3 0 0 25 0 8 0 807640277 603381760 71707 33554432000 4194304 4625164 140736155157216 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.25
Current children cumulated vsize (KiB) 589240

[startup+25.5004 s]
/proc/loadavg: 2.81 0.66 0.22 9/482 30026
/proc/meminfo: memFree=27104636/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=704600 CPUtime=203.42 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 290989 0 0 0 20283 59 0 0 25 0 8 0 807640275 721510400 121580 33554432000 4194304 4625164 140736155157216 18446744073709551615 4490199 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 176150 121580 353 106 0 171653 0
[pid=30017/tid=30019] ppid=30015 vsize=704600 CPUtime=25.47 cores=0-7
/proc/30017/task/30019/stat : 30019 (SArTagnan) R 30015 30017 29991 0 -1 4202560 16937 0 0 0 2543 4 0 0 25 0 8 0 807640277 721510400 121580 33554432000 4194304 4625164 140736155157216 18446744073709551615 4351906 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=30017/tid=30020] ppid=30015 vsize=704600 CPUtime=25.46 cores=0-7
/proc/30017/task/30020/stat : 30020 (SArTagnan) R 30015 30017 29991 0 -1 4202560 17828 0 0 0 2542 4 0 0 25 0 8 0 807640277 721510400 121580 33554432000 4194304 4625164 140736155157216 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=30017/tid=30021] ppid=30015 vsize=704600 CPUtime=25.47 cores=0-7
/proc/30017/task/30021/stat : 30021 (SArTagnan) R 30015 30017 29991 0 -1 4202560 175310 0 0 0 2521 26 0 0 25 0 8 0 807640277 721510400 121580 33554432000 4194304 4625164 140736155157216 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=30017/tid=30022] ppid=30015 vsize=704600 CPUtime=25.48 cores=0-7
/proc/30017/task/30022/stat : 30022 (SArTagnan) R 30015 30017 29991 0 -1 4202560 14993 0 0 0 2543 5 0 0 25 0 8 0 807640277 721510400 121580 33554432000 4194304 4625164 140736155157216 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=30017/tid=30023] ppid=30015 vsize=704600 CPUtime=25.47 cores=0-7
/proc/30017/task/30023/stat : 30023 (SArTagnan) R 30015 30017 29991 0 -1 4202560 15697 0 0 0 2542 5 0 0 25 0 8 0 807640277 721510400 121580 33554432000 4194304 4625164 140736155157216 18446744073709551615 4348858 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=30017/tid=30024] ppid=30015 vsize=704600 CPUtime=25.48 cores=0-7
/proc/30017/task/30024/stat : 30024 (SArTagnan) R 30015 30017 29991 0 -1 4202560 14663 0 0 0 2543 5 0 0 25 0 8 0 807640277 721510400 121581 33554432000 4194304 4625164 140736155157216 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=30017/tid=30025] ppid=30015 vsize=704600 CPUtime=25.06 cores=0-7
/proc/30017/task/30025/stat : 30025 (SArTagnan) R 30015 30017 29991 0 -1 4202560 15482 0 0 0 2502 4 0 0 25 0 8 0 807640277 721510400 121582 33554432000 4194304 4625164 140736155157216 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.42
Current children cumulated vsize (KiB) 704600

[startup+51.1005 s]
/proc/loadavg: 4.65 1.26 0.43 9/482 30027
/proc/meminfo: memFree=26794724/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=1118304 CPUtime=407.81 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 520334 0 0 0 40673 108 0 0 25 0 8 0 807640275 1145143296 200795 33554432000 4194304 4625164 140736155157216 18446744073709551615 4393101 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 279576 200796 353 106 0 275079 0
[pid=30017/tid=30019] ppid=30015 vsize=1118304 CPUtime=51.07 cores=0-7
/proc/30017/task/30019/stat : 30019 (SArTagnan) R 30015 30017 29991 0 -1 4202560 28898 0 0 0 5098 9 0 0 25 0 8 0 807640277 1145143296 200796 33554432000 4194304 4625164 140736155157216 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=30017/tid=30020] ppid=30015 vsize=1118304 CPUtime=51.07 cores=0-7
/proc/30017/task/30020/stat : 30020 (SArTagnan) R 30015 30017 29991 0 -1 4202560 28065 0 0 0 5098 9 0 0 25 0 8 0 807640277 1145143296 200797 33554432000 4194304 4625164 140736155157216 18446744073709551615 208079893537 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=30017/tid=30021] ppid=30015 vsize=1118304 CPUtime=51.06 cores=0-7
/proc/30017/task/30021/stat : 30021 (SArTagnan) R 30015 30017 29991 0 -1 4202560 332155 0 0 0 5059 47 0 0 25 0 8 0 807640277 1145143296 200798 33554432000 4194304 4625164 140736155157216 18446744073709551615 4547221 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=30017/tid=30022] ppid=30015 vsize=1118304 CPUtime=51.07 cores=0-7
/proc/30017/task/30022/stat : 30022 (SArTagnan) R 30015 30017 29991 0 -1 4202560 23592 0 0 0 5099 8 0 0 25 0 8 0 807640277 1145143296 200799 33554432000 4194304 4625164 140736155157216 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=30017/tid=30023] ppid=30015 vsize=1118304 CPUtime=51.07 cores=0-7
/proc/30017/task/30023/stat : 30023 (SArTagnan) R 30015 30017 29991 0 -1 4202560 24921 0 0 0 5098 9 0 0 25 0 8 0 807640277 1145143296 200799 33554432000 4194304 4625164 140736155157216 18446744073709551615 208079893537 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=30017/tid=30024] ppid=30015 vsize=1118304 CPUtime=51.08 cores=0-7
/proc/30017/task/30024/stat : 30024 (SArTagnan) R 30015 30017 29991 0 -1 4202560 26225 0 0 0 5099 9 0 0 25 0 8 0 807640277 1145143296 200799 33554432000 4194304 4625164 140736155157216 18446744073709551615 4351814 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=30017/tid=30025] ppid=30015 vsize=1118304 CPUtime=50.25 cores=0-7
/proc/30017/task/30025/stat : 30025 (SArTagnan) R 30015 30017 29991 0 -1 4202560 24030 0 0 0 5017 8 0 0 25 0 8 0 807640277 1145143296 200799 33554432000 4194304 4625164 140736155157216 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 407.81

################
# More data... #
################

Current children cumulated vsize (KiB) 6814532

[startup+4722.32 s]
/proc/loadavg: 8.05 8.04 8.00 9/482 30163
/proc/meminfo: memFree=21578432/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=6814532 CPUtime=37699.4 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 21810486 0 0 0 3765334 4606 0 0 25 0 8 0 807640275 6978080768 1498694 33554432000 4194304 4625164 140736155157216 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 1703633 1498694 353 106 0 1699136 0
[pid=30017/tid=30019] ppid=30015 vsize=6814532 CPUtime=4722.46 cores=0-7
/proc/30017/task/30019/stat : 30019 (SArTagnan) R 30015 30017 29991 0 -1 4202560 119444 0 0 0 472207 39 0 0 25 0 8 0 807640277 6978080768 1498694 33554432000 4194304 4625164 140736155157216 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=30017/tid=30020] ppid=30015 vsize=6814532 CPUtime=4722.46 cores=0-7
/proc/30017/task/30020/stat : 30020 (SArTagnan) R 30015 30017 29991 0 -1 4202560 123223 0 0 0 472198 48 0 0 25 0 8 0 807640277 6978080768 1498694 33554432000 4194304 4625164 140736155157216 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=30017/tid=30021] ppid=30015 vsize=6814532 CPUtime=4722.21 cores=0-7
/proc/30017/task/30021/stat : 30021 (SArTagnan) R 30015 30017 29991 0 -1 4202560 19709948 0 0 0 468298 3923 0 0 25 0 8 0 807640277 6978080768 1498695 33554432000 4194304 4625164 140736155157216 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=30017/tid=30022] ppid=30015 vsize=6814532 CPUtime=4722.02 cores=0-7
/proc/30017/task/30022/stat : 30022 (SArTagnan) R 30015 30017 29991 0 -1 4202560 126477 0 0 0 472129 73 0 0 25 0 8 0 807640277 6978080768 1498695 33554432000 4194304 4625164 140736155157216 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=30017/tid=30023] ppid=30015 vsize=6814532 CPUtime=4722.45 cores=0-7
/proc/30017/task/30023/stat : 30023 (SArTagnan) R 30015 30017 29991 0 -1 4202560 147179 0 0 0 472172 73 0 0 25 0 8 0 807640277 6978080768 1498695 33554432000 4194304 4625164 140736155157216 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=30017/tid=30024] ppid=30015 vsize=6814532 CPUtime=4649.01 cores=0-7
/proc/30017/task/30024/stat : 30024 (SArTagnan) R 30015 30017 29991 0 -1 4202560 196313 0 0 0 464801 100 0 0 25 0 8 0 807640277 6978080768 1498695 33554432000 4194304 4625164 140736155157216 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=30017/tid=30025] ppid=30015 vsize=6814532 CPUtime=4717.56 cores=0-7
/proc/30017/task/30025/stat : 30025 (SArTagnan) R 30015 30017 29991 0 -1 4202560 147654 0 0 0 471692 64 0 0 25 0 8 0 807640277 6978080768 1498695 33554432000 4194304 4625164 140736155157216 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 37699.4
Current children cumulated vsize (KiB) 6814532

[startup+4782.3 s]
/proc/loadavg: 8.07 8.05 8.00 9/482 30165
/proc/meminfo: memFree=21557704/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=6814532 CPUtime=38178.2 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 22129865 0 0 0 3813151 4673 0 0 25 0 8 0 807640275 6978080768 1504362 33554432000 4194304 4625164 140736155157216 18446744073709551615 4396545 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 1703633 1504362 353 106 0 1699136 0
[pid=30017/tid=30019] ppid=30015 vsize=6814532 CPUtime=4782.45 cores=0-7
/proc/30017/task/30019/stat : 30019 (SArTagnan) R 30015 30017 29991 0 -1 4202560 119614 0 0 0 478206 39 0 0 25 0 8 0 807640277 6978080768 1504362 33554432000 4194304 4625164 140736155157216 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=30017/tid=30020] ppid=30015 vsize=6814532 CPUtime=4782.45 cores=0-7
/proc/30017/task/30020/stat : 30020 (SArTagnan) R 30015 30017 29991 0 -1 4202560 123371 0 0 0 478197 48 0 0 25 0 8 0 807640277 6978080768 1504362 33554432000 4194304 4625164 140736155157216 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=30017/tid=30021] ppid=30015 vsize=6814532 CPUtime=4782.2 cores=0-7
/proc/30017/task/30021/stat : 30021 (SArTagnan) R 30015 30017 29991 0 -1 4202560 20014639 0 0 0 474235 3985 0 0 25 0 8 0 807640277 6978080768 1504281 33554432000 4194304 4625164 140736155157216 18446744073709551615 208080275815 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=30017/tid=30022] ppid=30015 vsize=6814532 CPUtime=4782 cores=0-7
/proc/30017/task/30022/stat : 30022 (SArTagnan) R 30015 30017 29991 0 -1 4202560 126640 0 0 0 478127 73 0 0 25 0 8 0 807640277 6978080768 1504281 33554432000 4194304 4625164 140736155157216 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=30017/tid=30023] ppid=30015 vsize=6814532 CPUtime=4782.43 cores=0-7
/proc/30017/task/30023/stat : 30023 (SArTagnan) R 30015 30017 29991 0 -1 4202560 147396 0 0 0 478170 73 0 0 25 0 8 0 807640277 6978080768 1504281 33554432000 4194304 4625164 140736155157216 18446744073709551615 4329739 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=30017/tid=30024] ppid=30015 vsize=6814532 CPUtime=4708.01 cores=0-7
/proc/30017/task/30024/stat : 30024 (SArTagnan) R 30015 30017 29991 0 -1 4202560 196474 0 0 0 470701 100 0 0 25 0 8 0 807640277 6978080768 1504281 33554432000 4194304 4625164 140736155157216 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=30017/tid=30025] ppid=30015 vsize=6814532 CPUtime=4777.54 cores=0-7
/proc/30017/task/30025/stat : 30025 (SArTagnan) R 30015 30017 29991 0 -1 4202560 147807 0 0 0 477690 64 0 0 25 0 8 0 807640277 6978080768 1504281 33554432000 4194304 4625164 140736155157216 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38178.2
Current children cumulated vsize (KiB) 6814532

[startup+4842.3 s]
/proc/loadavg: 8.07 8.05 8.00 9/482 30167
/proc/meminfo: memFree=21551220/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=6814532 CPUtime=38657.2 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 22493541 0 0 0 3860979 4744 0 0 25 0 8 0 807640275 6978080768 1506475 33554432000 4194304 4625164 140736155157216 18446744073709551615 4338055 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 1703633 1506475 353 106 0 1699136 0
[pid=30017/tid=30019] ppid=30015 vsize=6814532 CPUtime=4842.45 cores=0-7
/proc/30017/task/30019/stat : 30019 (SArTagnan) R 30015 30017 29991 0 -1 4202560 119791 0 0 0 484206 39 0 0 25 0 8 0 807640277 6978080768 1506475 33554432000 4194304 4625164 140736155157216 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=30017/tid=30020] ppid=30015 vsize=6814532 CPUtime=4842.45 cores=0-7
/proc/30017/task/30020/stat : 30020 (SArTagnan) R 30015 30017 29991 0 -1 4202560 123524 0 0 0 484197 48 0 0 25 0 8 0 807640277 6978080768 1506475 33554432000 4194304 4625164 140736155157216 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=30017/tid=30021] ppid=30015 vsize=6814532 CPUtime=4842.19 cores=0-7
/proc/30017/task/30021/stat : 30021 (SArTagnan) R 30015 30017 29991 0 -1 4202560 20376779 0 0 0 480164 4055 0 0 25 0 8 0 807640277 6978080768 1506475 33554432000 4194304 4625164 140736155157216 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=30017/tid=30022] ppid=30015 vsize=6814532 CPUtime=4842 cores=0-7
/proc/30017/task/30022/stat : 30022 (SArTagnan) R 30015 30017 29991 0 -1 4202560 126806 0 0 0 484127 73 0 0 25 0 8 0 807640277 6978080768 1506475 33554432000 4194304 4625164 140736155157216 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=30017/tid=30023] ppid=30015 vsize=6814532 CPUtime=4842.43 cores=0-7
/proc/30017/task/30023/stat : 30023 (SArTagnan) R 30015 30017 29991 0 -1 4202560 147585 0 0 0 484170 73 0 0 25 0 8 0 807640277 6978080768 1506475 33554432000 4194304 4625164 140736155157216 18446744073709551615 4329721 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=30017/tid=30024] ppid=30015 vsize=6814532 CPUtime=4767.02 cores=0-7
/proc/30017/task/30024/stat : 30024 (SArTagnan) R 30015 30017 29991 0 -1 4202560 196867 0 0 0 476601 101 0 0 25 0 8 0 807640277 6978080768 1506475 33554432000 4194304 4625164 140736155157216 18446744073709551615 4330793 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=30017/tid=30025] ppid=30015 vsize=6814532 CPUtime=4837.56 cores=0-7
/proc/30017/task/30025/stat : 30025 (SArTagnan) R 30015 30017 29991 0 -1 4202560 148039 0 0 0 483691 65 0 0 25 0 8 0 807640277 6978080768 1506475 33554432000 4194304 4625164 140736155157216 18446744073709551615 4488020 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38657.2
Current children cumulated vsize (KiB) 6814532

[startup+4902.3 s]
/proc/loadavg: 8.15 8.08 8.01 9/482 30168
/proc/meminfo: memFree=21536080/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=6814532 CPUtime=39136.2 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 22810931 0 0 0 3908813 4812 0 0 25 0 8 0 807640275 6978080768 1509661 33554432000 4194304 4625164 140736155157216 18446744073709551615 4338085 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 1703633 1509661 353 106 0 1699136 0
[pid=30017/tid=30019] ppid=30015 vsize=6814532 CPUtime=4902.45 cores=0-7
/proc/30017/task/30019/stat : 30019 (SArTagnan) R 30015 30017 29991 0 -1 4202560 119946 0 0 0 490206 39 0 0 25 0 8 0 807640277 6978080768 1509661 33554432000 4194304 4625164 140736155157216 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=30017/tid=30020] ppid=30015 vsize=6814532 CPUtime=4902.45 cores=0-7
/proc/30017/task/30020/stat : 30020 (SArTagnan) R 30015 30017 29991 0 -1 4202560 123658 0 0 0 490197 48 0 0 25 0 8 0 807640277 6978080768 1509661 33554432000 4194304 4625164 140736155157216 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=30017/tid=30021] ppid=30015 vsize=6814532 CPUtime=4902.19 cores=0-7
/proc/30017/task/30021/stat : 30021 (SArTagnan) R 30015 30017 29991 0 -1 4202560 20693101 0 0 0 486097 4122 0 0 25 0 8 0 807640277 6978080768 1509661 33554432000 4194304 4625164 140736155157216 18446744073709551615 4579373 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=30017/tid=30022] ppid=30015 vsize=6814532 CPUtime=4902 cores=0-7
/proc/30017/task/30022/stat : 30022 (SArTagnan) R 30015 30017 29991 0 -1 4202560 126968 0 0 0 490127 73 0 0 25 0 8 0 807640277 6978080768 1509661 33554432000 4194304 4625164 140736155157216 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=30017/tid=30023] ppid=30015 vsize=6814532 CPUtime=4902.43 cores=0-7
/proc/30017/task/30023/stat : 30023 (SArTagnan) R 30015 30017 29991 0 -1 4202560 147725 0 0 0 490170 73 0 0 25 0 8 0 807640277 6978080768 1509661 33554432000 4194304 4625164 140736155157216 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=30017/tid=30024] ppid=30015 vsize=6814532 CPUtime=4826.05 cores=0-7
/proc/30017/task/30024/stat : 30024 (SArTagnan) R 30015 30017 29991 0 -1 4202560 196949 0 0 0 482504 101 0 0 25 0 8 0 807640277 6978080768 1509661 33554432000 4194304 4625164 140736155157216 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=30017/tid=30025] ppid=30015 vsize=6814532 CPUtime=4897.56 cores=0-7
/proc/30017/task/30025/stat : 30025 (SArTagnan) R 30015 30017 29991 0 -1 4202560 148188 0 0 0 489691 65 0 0 25 0 8 0 807640277 6978080768 1509661 33554432000 4194304 4625164 140736155157216 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39136.2
Current children cumulated vsize (KiB) 6814532

[startup+4962.3 s]
/proc/loadavg: 8.14 8.10 8.02 9/482 30170
/proc/meminfo: memFree=21509524/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=6853540 CPUtime=39615.3 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 23105333 0 0 0 3956651 4875 0 0 25 0 8 0 807640275 7018024960 1516297 33554432000 4194304 4625164 140736155157216 18446744073709551615 4338055 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 1713385 1516297 353 106 0 1708888 0
[pid=30017/tid=30019] ppid=30015 vsize=6853540 CPUtime=4962.45 cores=0-7
/proc/30017/task/30019/stat : 30019 (SArTagnan) R 30015 30017 29991 0 -1 4202560 120119 0 0 0 496206 39 0 0 25 0 8 0 807640277 7018024960 1516297 33554432000 4194304 4625164 140736155157216 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=30017/tid=30020] ppid=30015 vsize=6853540 CPUtime=4962.45 cores=0-7
/proc/30017/task/30020/stat : 30020 (SArTagnan) R 30015 30017 29991 0 -1 4202560 123785 0 0 0 496197 48 0 0 25 0 8 0 807640277 7018024960 1516297 33554432000 4194304 4625164 140736155157216 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=30017/tid=30021] ppid=30015 vsize=6853540 CPUtime=4962.2 cores=0-7
/proc/30017/task/30021/stat : 30021 (SArTagnan) R 30015 30017 29991 0 -1 4202560 20976523 0 0 0 492037 4183 0 0 25 0 8 0 807640277 7018024960 1516297 33554432000 4194304 4625164 140736155157216 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=30017/tid=30022] ppid=30015 vsize=6853540 CPUtime=4962 cores=0-7
/proc/30017/task/30022/stat : 30022 (SArTagnan) R 30015 30017 29991 0 -1 4202560 127111 0 0 0 496127 73 0 0 25 0 8 0 807640277 7018024960 1516297 33554432000 4194304 4625164 140736155157216 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=30017/tid=30023] ppid=30015 vsize=6853540 CPUtime=4962.43 cores=0-7
/proc/30017/task/30023/stat : 30023 (SArTagnan) R 30015 30017 29991 0 -1 4202560 147862 0 0 0 496170 73 0 0 25 0 8 0 807640277 7018024960 1516297 33554432000 4194304 4625164 140736155157216 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=30017/tid=30024] ppid=30015 vsize=6853540 CPUtime=4885.04 cores=0-7
/proc/30017/task/30024/stat : 30024 (SArTagnan) R 30015 30017 29991 0 -1 4202560 206773 0 0 0 488402 102 0 0 25 0 8 0 807640277 7018024960 1516297 33554432000 4194304 4625164 140736155157216 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=30017/tid=30025] ppid=30015 vsize=6853540 CPUtime=4957.56 cores=0-7
/proc/30017/task/30025/stat : 30025 (SArTagnan) R 30015 30017 29991 0 -1 4202560 148348 0 0 0 495691 65 0 0 25 0 8 0 807640277 7018024960 1516297 33554432000 4194304 4625164 140736155157216 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39615.3
Current children cumulated vsize (KiB) 6853540



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.52 s]
/proc/loadavg: 8.12 8.09 8.02 9/482 30171
/proc/meminfo: memFree=21487312/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=6958756 CPUtime=40000.3 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) R 30015 30017 29991 0 -1 4202496 23357440 0 0 0 3995105 4928 0 0 25 0 8 0 807640275 7125766144 1521773 33554432000 4194304 4625164 140736155157216 18446744073709551615 4338055 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 1739689 1521773 353 106 0 1735192 0
[pid=30017/tid=30019] ppid=30015 vsize=6958756 CPUtime=5010.69 cores=0-7
/proc/30017/task/30019/stat : 30019 (SArTagnan) R 30015 30017 29991 0 -1 4202560 120235 0 0 0 501030 39 0 0 25 0 8 0 807640277 7125766144 1521773 33554432000 4194304 4625164 140736155157216 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=30017/tid=30020] ppid=30015 vsize=6958756 CPUtime=5010.69 cores=0-7
/proc/30017/task/30020/stat : 30020 (SArTagnan) R 30015 30017 29991 0 -1 4202560 123888 0 0 0 501021 48 0 0 25 0 8 0 807640277 7125766144 1521773 33554432000 4194304 4625164 140736155157216 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=30017/tid=30021] ppid=30015 vsize=6958756 CPUtime=5010.43 cores=0-7
/proc/30017/task/30021/stat : 30021 (SArTagnan) R 30015 30017 29991 0 -1 4202560 21217843 0 0 0 496809 4234 0 0 25 0 8 0 807640277 7125766144 1521773 33554432000 4194304 4625164 140736155157216 18446744073709551615 4334923 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=30017/tid=30022] ppid=30015 vsize=6958756 CPUtime=5010.24 cores=0-7
/proc/30017/task/30022/stat : 30022 (SArTagnan) R 30015 30017 29991 0 -1 4202560 127230 0 0 0 500951 73 0 0 25 0 8 0 807640277 7125766144 1521773 33554432000 4194304 4625164 140736155157216 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=30017/tid=30023] ppid=30015 vsize=6958756 CPUtime=5010.68 cores=0-7
/proc/30017/task/30023/stat : 30023 (SArTagnan) R 30015 30017 29991 0 -1 4202560 157741 0 0 0 500993 75 0 0 25 0 8 0 807640277 7125766144 1521773 33554432000 4194304 4625164 140736155157216 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=30017/tid=30024] ppid=30015 vsize=6958756 CPUtime=4932.46 cores=0-7
/proc/30017/task/30024/stat : 30024 (SArTagnan) R 30015 30017 29991 0 -1 4202560 206846 0 0 0 493144 102 0 0 25 0 8 0 807640277 7125766144 1521773 33554432000 4194304 4625164 140736155157216 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=30017/tid=30025] ppid=30015 vsize=6958756 CPUtime=5005.79 cores=0-7
/proc/30017/task/30025/stat : 30025 (SArTagnan) R 30015 30017 29991 0 -1 4202560 148455 0 0 0 500514 65 0 0 25 0 8 0 807640277 7125766144 1521773 33554432000 4194304 4625164 140736155157216 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.3
Current children cumulated vsize (KiB) 6958756

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.62 s]
/proc/loadavg: 8.12 8.09 8.02 9/482 30171
/proc/meminfo: memFree=21487312/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=0 CPUtime=40000.5 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) Z 30015 30017 29991 0 -1 4203548 23357440 0 0 0 3995109 4937 0 0 25 0 2 0 807640275 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 0 0 0 0 0 0 0
[pid=30017/tid=30024] ppid=30015 vsize=0 CPUtime=4932.55 cores=0-7
/proc/30017/task/30024/stat : 30024 (SArTagnan) R 30015 30017 29991 0 -1 4203588 206846 0 0 0 493144 111 0 0 25 0 2 0 807640277 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 0

[startup+5010.8 s]
/proc/loadavg: 8.12 8.09 8.02 9/482 30171
/proc/meminfo: memFree=21487312/32950928 swapFree=39617324/67111528
[pid=30017] ppid=30015 vsize=0 CPUtime=40000.6 cores=0-7
/proc/30017/stat : 30017 (SArTagnan) Z 30015 30017 29991 0 -1 4203548 23357440 0 0 0 3995109 4954 0 0 25 0 2 0 807640275 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/30017/statm: 0 0 0 0 0 0 0
[pid=30017/tid=30024] ppid=30015 vsize=0 CPUtime=4932.72 cores=0-7
/proc/30017/task/30024/stat : 30024 (SArTagnan) R 30015 30017 29991 0 -1 4203588 206846 0 0 0 493144 128 0 0 25 0 2 0 807640277 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5010.86
CPU time (s): 40000.7
CPU user time (s): 39951.1
CPU system time (s): 49.6095
CPU usage (%): 798.279
Max. virtual memory (cumulated for all children) (KiB): 6997768

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39951.1
system time used= 49.6095
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 23357440
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 5768
involuntary context switches= 164068

runsolver used 21.1128 second user time and 61.5506 second system time

The end

Launcher Data 

Begin job on node127 at 2011-06-01 22:33:26
IDJOB=3439226
IDBENCH=83796
IDSOLVER=1936
FILE ID=node127/3439226-1306960406
RUNJOBID= node127-1306960406-30005
PBS_JOBID= 13497624
Free space on /tmp= 73932 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT03/handmade/gomes/bqwh/bqwh.50.825.shuffled-as.sat03-1648.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3439226-1306960406/watcher-3439226-1306960406 -o /tmp/evaluation-result-3439226-1306960406/solver-3439226-1306960406 -C 40000 -W 5100 -M 31000  SArTagnan HOME/instance-3439226-1306960406.cnf -threads=8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 05a191325d89de504821b55dea51ca7f
RANDOM SEED=643754986

node127.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.60
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      27587040 kB
Buffers:        290892 kB
Cached:         780912 kB
SwapCached:      59160 kB
Active:        3067176 kB
Inactive:      2011160 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      27587040 kB
SwapTotal:    67111528 kB
SwapFree:     39617324 kB
Dirty:            1184 kB
Writeback:           0 kB
AnonPages:     3954604 kB
Mapped:          21456 kB
Slab:           119412 kB
PageTables:      95480 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 119722204 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73932 MiB
End job on node127 at 2011-06-01 23:56:59