Trace number 3438476

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 40000.8 5008.12

General information on the benchmark

NameCRAFTED/rbsat/random/
forced/rbsat-v2640c305320gyes5.cnf
MD5SUMf892a2bda6fe6aec6f2fcb4e1c541df8
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2640
Number of clauses305320
Sum of the clauses size613120
Maximum clause length33
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2305240
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 580

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3438476-1306918148/watcher-3438476-1306918148 -o /tmp/evaluation-result-3438476-1306918148/solver-3438476-1306918148 -C 40000 -W 5100 -M 31000 SArTagnan HOME/instance-3438476-1306918148.cnf -threads=8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.00 2.02 2.00 1/366 28224
/proc/meminfo: memFree=23033896/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=26560 CPUtime=0 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 874 0 0 0 0 0 0 0 25 0 1 0 803415263 27197440 572 33554432000 4194304 4625164 140733775929104 18446744073709551615 230809165568 0 0 4096 2 0 0 0 17 2 0 0 0
/proc/28224/statm: 6640 572 296 106 0 2143 0

[startup+0.0997179 s]
/proc/loadavg: 2.00 2.02 2.00 1/366 28224
/proc/meminfo: memFree=23033896/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=333048 CPUtime=0.13 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 10695 0 0 0 12 1 0 0 25 0 8 0 803415263 341041152 8357 33554432000 4194304 4625164 140733775929104 18446744073709551615 4499398 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28224/statm: 83262 8357 337 106 0 78765 0
Current children cumulated CPU time (s) 0.13
Current children cumulated vsize (KiB) 333048

[startup+0.100709 s]
/proc/loadavg: 2.00 2.02 2.00 1/366 28224
/proc/meminfo: memFree=23033896/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=333048 CPUtime=0.13 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 10702 0 0 0 12 1 0 0 25 0 8 0 803415263 341041152 8363 33554432000 4194304 4625164 140733775929104 18446744073709551615 4500243 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28224/statm: 83262 8363 337 106 0 78765 0
Current children cumulated CPU time (s) 0.13
Current children cumulated vsize (KiB) 333048

[startup+0.300636 s]
/proc/loadavg: 2.00 2.02 2.00 1/366 28224
/proc/meminfo: memFree=23033896/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=482692 CPUtime=1.64 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 27243 0 0 0 160 4 0 0 18 0 8 0 803415263 494276608 18855 33554432000 4194304 4625164 140733775929104 18446744073709551615 4281116 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28224/statm: 120673 18855 346 106 0 116176 0
Current children cumulated CPU time (s) 1.64
Current children cumulated vsize (KiB) 482692

[startup+0.700565 s]
/proc/loadavg: 2.00 2.02 2.00 1/366 28224
/proc/meminfo: memFree=23033896/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=548228 CPUtime=4.83 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 34269 0 0 0 478 5 0 0 18 0 8 0 803415263 561385472 25278 33554432000 4194304 4625164 140733775929104 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28224/statm: 137057 25278 352 106 0 132560 0
Current children cumulated CPU time (s) 4.83
Current children cumulated vsize (KiB) 548228

[startup+1.50042 s]
/proc/loadavg: 2.48 2.12 2.03 9/374 28232
/proc/meminfo: memFree=22915928/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=560924 CPUtime=11.22 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 48660 0 0 0 1114 8 0 0 19 0 8 0 803415263 574386176 32603 33554432000 4194304 4625164 140733775929104 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28224/statm: 140231 32603 353 106 0 135734 0
[pid=28224/tid=28226] ppid=28222 vsize=560924 CPUtime=1.38 cores=0-7
/proc/28224/task/28226/stat : 28226 (SArTagnan) R 28222 28224 28198 0 -1 4202560 3251 0 0 0 138 0 0 0 19 0 8 0 803415272 574386176 32603 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28224/tid=28227] ppid=28222 vsize=560924 CPUtime=1.4 cores=0-7
/proc/28224/task/28227/stat : 28227 (SArTagnan) R 28222 28224 28198 0 -1 4202560 3464 0 0 0 140 0 0 0 19 0 8 0 803415272 574386176 32604 33554432000 4194304 4625164 140733775929104 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28224/tid=28228] ppid=28222 vsize=560924 CPUtime=1.39 cores=0-7
/proc/28224/task/28228/stat : 28228 (SArTagnan) R 28222 28224 28198 0 -1 4202560 4105 0 0 0 139 0 0 0 19 0 8 0 803415272 574386176 32604 33554432000 4194304 4625164 140733775929104 18446744073709551615 4375467 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28224/tid=28229] ppid=28222 vsize=560924 CPUtime=1.36 cores=0-7
/proc/28224/task/28229/stat : 28229 (SArTagnan) R 28222 28224 28198 0 -1 4202560 2687 0 0 0 136 0 0 0 25 0 8 0 803415272 574386176 32604 33554432000 4194304 4625164 140733775929104 18446744073709551615 230808832986 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28224/tid=28230] ppid=28222 vsize=560924 CPUtime=1.38 cores=0-7
/proc/28224/task/28230/stat : 28230 (SArTagnan) R 28222 28224 28198 0 -1 4202560 3787 0 0 0 138 0 0 0 19 0 8 0 803415272 574386176 32604 33554432000 4194304 4625164 140733775929104 18446744073709551615 4440008 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28224/tid=28231] ppid=28222 vsize=560924 CPUtime=1.35 cores=0-7
/proc/28224/task/28231/stat : 28231 (SArTagnan) R 28222 28224 28198 0 -1 4202560 3686 0 0 0 135 0 0 0 25 0 8 0 803415272 574386176 32604 33554432000 4194304 4625164 140733775929104 18446744073709551615 4480048 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28224/tid=28232] ppid=28222 vsize=560924 CPUtime=1.39 cores=0-7
/proc/28224/task/28232/stat : 28232 (SArTagnan) R 28222 28224 28198 0 -1 4202560 2814 0 0 0 139 0 0 0 19 0 8 0 803415272 574386176 32604 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351901 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 11.22
Current children cumulated vsize (KiB) 560924

[startup+3.10113 s]
/proc/loadavg: 2.48 2.12 2.03 9/374 28232
/proc/meminfo: memFree=22883440/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=575468 CPUtime=24.02 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 65830 0 0 0 2391 11 0 0 23 0 8 0 803415263 589279232 42658 33554432000 4194304 4625164 140733775929104 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28224/statm: 143867 42658 355 106 0 139370 0
[pid=28224/tid=28226] ppid=28222 vsize=575468 CPUtime=2.97 cores=0-7
/proc/28224/task/28226/stat : 28226 (SArTagnan) R 28222 28224 28198 0 -1 4202560 3967 0 0 0 297 0 0 0 24 0 8 0 803415272 589279232 42658 33554432000 4194304 4625164 140733775929104 18446744073709551615 4467259 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28224/tid=28227] ppid=28222 vsize=575468 CPUtime=3 cores=0-7
/proc/28224/task/28227/stat : 28227 (SArTagnan) R 28222 28224 28198 0 -1 4202560 3929 0 0 0 300 0 0 0 23 0 8 0 803415272 589279232 42658 33554432000 4194304 4625164 140733775929104 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28224/tid=28228] ppid=28222 vsize=575468 CPUtime=2.99 cores=0-7
/proc/28224/task/28228/stat : 28228 (SArTagnan) R 28222 28224 28198 0 -1 4202560 6611 0 0 0 299 0 0 0 25 0 8 0 803415272 589279232 42658 33554432000 4194304 4625164 140733775929104 18446744073709551615 4334238 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28224/tid=28229] ppid=28222 vsize=575468 CPUtime=2.96 cores=0-7
/proc/28224/task/28229/stat : 28229 (SArTagnan) R 28222 28224 28198 0 -1 4202560 2931 0 0 0 296 0 0 0 25 0 8 0 803415272 589279232 42658 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28224/tid=28230] ppid=28222 vsize=575468 CPUtime=2.99 cores=0-7
/proc/28224/task/28230/stat : 28230 (SArTagnan) R 28222 28224 28198 0 -1 4202560 6172 0 0 0 298 1 0 0 18 0 8 0 803415272 589279232 42658 33554432000 4194304 4625164 140733775929104 18446744073709551615 4348821 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28224/tid=28231] ppid=28222 vsize=575468 CPUtime=2.96 cores=0-7
/proc/28224/task/28231/stat : 28231 (SArTagnan) R 28222 28224 28198 0 -1 4202560 6069 0 0 0 295 1 0 0 25 0 8 0 803415272 589279232 42658 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28224/tid=28232] ppid=28222 vsize=575468 CPUtime=3 cores=0-7
/proc/28224/task/28232/stat : 28232 (SArTagnan) R 28222 28224 28198 0 -1 4202560 4210 0 0 0 299 1 0 0 23 0 8 0 803415272 589279232 42658 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 24.02
Current children cumulated vsize (KiB) 575468

[startup+6.30054 s]
/proc/loadavg: 2.48 2.12 2.03 9/374 28232
/proc/meminfo: memFree=22804328/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=604036 CPUtime=49.56 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 97441 0 0 0 4940 16 0 0 25 0 8 0 803415263 618532864 59900 33554432000 4194304 4625164 140733775929104 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28224/statm: 151009 59900 364 106 0 146512 0
[pid=28224/tid=28226] ppid=28222 vsize=604036 CPUtime=6.13 cores=0-7
/proc/28224/task/28226/stat : 28226 (SArTagnan) R 28222 28224 28198 0 -1 4202560 6024 0 0 0 612 1 0 0 25 0 8 0 803415272 618532864 59900 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28224/tid=28227] ppid=28222 vsize=604036 CPUtime=6.19 cores=0-7
/proc/28224/task/28227/stat : 28227 (SArTagnan) R 28222 28224 28198 0 -1 4202560 5873 0 0 0 619 0 0 0 25 0 8 0 803415272 618532864 59900 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28224/tid=28228] ppid=28222 vsize=604036 CPUtime=6.19 cores=0-7
/proc/28224/task/28228/stat : 28228 (SArTagnan) R 28222 28224 28198 0 -1 4202560 13216 0 0 0 618 1 0 0 25 0 8 0 803415272 618532864 59900 33554432000 4194304 4625164 140733775929104 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28224/tid=28229] ppid=28222 vsize=604036 CPUtime=6.16 cores=0-7
/proc/28224/task/28229/stat : 28229 (SArTagnan) R 28222 28224 28198 0 -1 4202560 5029 0 0 0 616 0 0 0 25 0 8 0 803415272 618532864 59900 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28224/tid=28230] ppid=28222 vsize=604036 CPUtime=6.19 cores=0-7
/proc/28224/task/28230/stat : 28230 (SArTagnan) R 28222 28224 28198 0 -1 4202560 9063 0 0 0 617 2 0 0 25 0 8 0 803415272 618532864 59900 33554432000 4194304 4625164 140733775929104 18446744073709551615 4348903 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28224/tid=28231] ppid=28222 vsize=604036 CPUtime=6.16 cores=0-7
/proc/28224/task/28231/stat : 28231 (SArTagnan) R 28222 28224 28198 0 -1 4202560 9184 0 0 0 614 2 0 0 25 0 8 0 803415272 618532864 59900 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28224/tid=28232] ppid=28222 vsize=604036 CPUtime=6.2 cores=0-7
/proc/28224/task/28232/stat : 28232 (SArTagnan) R 28222 28224 28198 0 -1 4202560 5930 0 0 0 619 1 0 0 25 0 8 0 803415272 618532864 59900 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 49.56
Current children cumulated vsize (KiB) 604036

[startup+12.7004 s]
/proc/loadavg: 3.33 2.31 2.10 9/374 28233
/proc/meminfo: memFree=22688716/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=670236 CPUtime=100.69 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 137968 0 0 0 10044 25 0 0 25 0 8 0 803415263 686321664 87652 33554432000 4194304 4625164 140733775929104 18446744073709551615 4512315 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28224/statm: 167559 87652 406 106 0 163062 0
[pid=28224/tid=28226] ppid=28222 vsize=670236 CPUtime=12.45 cores=0-7
/proc/28224/task/28226/stat : 28226 (SArTagnan) R 28222 28224 28198 0 -1 4202560 10037 0 0 0 1243 2 0 0 25 0 8 0 803415272 686321664 87652 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28224/tid=28227] ppid=28222 vsize=670236 CPUtime=12.6 cores=0-7
/proc/28224/task/28227/stat : 28227 (SArTagnan) R 28222 28224 28198 0 -1 4202560 8549 0 0 0 1259 1 0 0 25 0 8 0 803415272 686321664 87652 33554432000 4194304 4625164 140733775929104 18446744073709551615 4462369 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28224/tid=28228] ppid=28222 vsize=670236 CPUtime=12.59 cores=0-7
/proc/28224/task/28228/stat : 28228 (SArTagnan) R 28222 28224 28198 0 -1 4202560 30755 0 0 0 1255 4 0 0 25 0 8 0 803415272 686321664 87652 33554432000 4194304 4625164 140733775929104 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28224/tid=28229] ppid=28222 vsize=670236 CPUtime=12.56 cores=0-7
/proc/28224/task/28229/stat : 28229 (SArTagnan) R 28222 28224 28198 0 -1 4202560 8085 0 0 0 1255 1 0 0 25 0 8 0 803415272 686321664 87652 33554432000 4194304 4625164 140733775929104 18446744073709551615 4352014 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28224/tid=28230] ppid=28222 vsize=670236 CPUtime=12.58 cores=0-7
/proc/28224/task/28230/stat : 28230 (SArTagnan) R 28222 28224 28198 0 -1 4202560 14163 0 0 0 1256 2 0 0 25 0 8 0 803415272 686321664 87652 33554432000 4194304 4625164 140733775929104 18446744073709551615 4348903 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28224/tid=28231] ppid=28222 vsize=670236 CPUtime=12.55 cores=0-7
/proc/28224/task/28231/stat : 28231 (SArTagnan) R 28222 28224 28198 0 -1 4202560 10920 0 0 0 1253 2 0 0 25 0 8 0 803415272 686321664 87652 33554432000 4194304 4625164 140733775929104 18446744073709551615 4328194 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28224/tid=28232] ppid=28222 vsize=670236 CPUtime=12.59 cores=0-7
/proc/28224/task/28232/stat : 28232 (SArTagnan) R 28222 28224 28198 0 -1 4202560 6521 0 0 0 1258 1 0 0 25 0 8 0 803415272 686321664 87652 33554432000 4194304 4625164 140733775929104 18446744073709551615 4352010 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 100.69
Current children cumulated vsize (KiB) 670236

[startup+25.501 s]
/proc/loadavg: 4.05 2.50 2.16 9/374 28233
/proc/meminfo: memFree=22580204/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=846580 CPUtime=202.92 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 203196 0 0 0 20258 34 0 0 25 0 8 0 803415263 866897920 114121 33554432000 4194304 4625164 140733775929104 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28224/statm: 211645 114121 466 106 0 207148 0
[pid=28224/tid=28226] ppid=28222 vsize=846580 CPUtime=25.09 cores=0-7
/proc/28224/task/28226/stat : 28226 (SArTagnan) R 28222 28224 28198 0 -1 4202560 14654 0 0 0 2507 2 0 0 25 0 8 0 803415272 866897920 114121 33554432000 4194304 4625164 140733775929104 18446744073709551615 4339591 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28224/tid=28227] ppid=28222 vsize=846580 CPUtime=25.39 cores=0-7
/proc/28224/task/28227/stat : 28227 (SArTagnan) R 28222 28224 28198 0 -1 4202560 9150 0 0 0 2538 1 0 0 25 0 8 0 803415272 866897920 114121 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28224/tid=28228] ppid=28222 vsize=846580 CPUtime=25.39 cores=0-7
/proc/28224/task/28228/stat : 28228 (SArTagnan) R 28222 28224 28198 0 -1 4202560 61106 0 0 0 2531 8 0 0 25 0 8 0 803415272 866897920 114121 33554432000 4194304 4625164 140733775929104 18446744073709551615 4579399 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28224/tid=28229] ppid=28222 vsize=846580 CPUtime=25.36 cores=0-7
/proc/28224/task/28229/stat : 28229 (SArTagnan) R 28222 28224 28198 0 -1 4202560 9739 0 0 0 2535 1 0 0 25 0 8 0 803415272 866897920 114121 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351989 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28224/tid=28230] ppid=28222 vsize=846580 CPUtime=25.39 cores=0-7
/proc/28224/task/28230/stat : 28230 (SArTagnan) R 28222 28224 28198 0 -1 4202560 15838 0 0 0 2536 3 0 0 25 0 8 0 803415272 866897920 114121 33554432000 4194304 4625164 140733775929104 18446744073709551615 4349047 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28224/tid=28231] ppid=28222 vsize=846580 CPUtime=25.35 cores=0-7
/proc/28224/task/28231/stat : 28231 (SArTagnan) R 28222 28224 28198 0 -1 4202560 16976 0 0 0 2532 3 0 0 25 0 8 0 803415272 866897920 114121 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28224/tid=28232] ppid=28222 vsize=846580 CPUtime=25.4 cores=0-7
/proc/28224/task/28232/stat : 28232 (SArTagnan) R 28222 28224 28198 0 -1 4202560 9167 0 0 0 2538 2 0 0 25 0 8 0 803415272 866897920 114121 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 202.92
Current children cumulated vsize (KiB) 846580

[startup+51.1004 s]
/proc/loadavg: 5.39 2.94 2.31 9/374 28234
/proc/meminfo: memFree=22434864/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=1009496 CPUtime=407.43 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 302795 0 0 0 40694 49 0 0 25 0 8 0 803415263 1033723904 150763 33554432000 4194304 4625164 140733775929104 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28224/statm: 252374 150763 706 106 0 247877 0
[pid=28224/tid=28226] ppid=28222 vsize=1009496 CPUtime=50.4 cores=0-7
/proc/28224/task/28226/stat : 28226 (SArTagnan) R 28222 28224 28198 0 -1 4202560 21129 0 0 0 5037 3 0 0 25 0 8 0 803415272 1033723904 150763 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28224/tid=28227] ppid=28222 vsize=1009496 CPUtime=50.99 cores=0-7
/proc/28224/task/28227/stat : 28227 (SArTagnan) R 28222 28224 28198 0 -1 4202560 9652 0 0 0 5098 1 0 0 25 0 8 0 803415272 1033723904 150763 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351975 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28224/tid=28228] ppid=28222 vsize=1009496 CPUtime=50.99 cores=0-7
/proc/28224/task/28228/stat : 28228 (SArTagnan) R 28222 28224 28198 0 -1 4202560 104256 0 0 0 5085 14 0 0 25 0 8 0 803415272 1033723904 150763 33554432000 4194304 4625164 140733775929104 18446744073709551615 4579376 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28224/tid=28229] ppid=28222 vsize=1009496 CPUtime=50.96 cores=0-7
/proc/28224/task/28229/stat : 28229 (SArTagnan) R 28222 28224 28198 0 -1 4202560 13349 0 0 0 5094 2 0 0 25 0 8 0 803415272 1033723904 150763 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351996 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28224/tid=28230] ppid=28222 vsize=1009496 CPUtime=50.99 cores=0-7
/proc/28224/task/28230/stat : 28230 (SArTagnan) R 28222 28224 28198 0 -1 4202560 24114 0 0 0 5095 4 0 0 25 0 8 0 803415272 1033723904 150763 33554432000 4194304 4625164 140733775929104 18446744073709551615 4348903 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28224/tid=28231] ppid=28222 vsize=1009496 CPUtime=50.95 cores=0-7
/proc/28224/task/28231/stat : 28231 (SArTagnan) R 28222 28224 28198 0 -1 4202560 26412 0 0 0 5090 5 0 0 25 0 8 0 803415272 1033723904 150763 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28224/tid=28232] ppid=28222 vsize=1009496 CPUtime=51 cores=0-7
/proc/28224/task/28232/stat : 28232 (SArTagnan) R 28222 28224 28198 0 -1 4202560 10656 0 0 0 5098 2 0 0 25 0 8 0 803415272 1033723904 150763 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 407.43

################
# More data... #
################

/proc/28224/task/28231/stat : 28231 (SArTagnan) R 28222 28224 28198 0 -1 4202560 174572 0 0 0 466194 36 0 0 25 0 8 0 803415272 6556164096 1227264 33554432000 4194304 4625164 140733775929104 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28224/tid=28232] ppid=28222 vsize=6402504 CPUtime=4646.62 cores=0-7
/proc/28224/task/28232/stat : 28232 (SArTagnan) R 28222 28224 28198 0 -1 4202560 121186 0 0 0 464595 67 0 0 25 0 8 0 803415272 6556164096 1227264 33554432000 4194304 4625164 140733775929104 18446744073709551615 4352014 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37239.7
Current children cumulated vsize (KiB) 6402504

[startup+4722.31 s]
/proc/loadavg: 8.11 8.10 8.01 9/374 28382
/proc/meminfo: memFree=18100480/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=6403160 CPUtime=37719.1 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 33364272 0 0 0 3766288 5621 0 0 25 0 8 0 803415263 6556835840 1231601 33554432000 4194304 4625164 140733775929104 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/28224/statm: 1600790 1231601 352 106 0 1596293 0
[pid=28224/tid=28226] ppid=28222 vsize=6403160 CPUtime=4719.74 cores=0-7
/proc/28224/task/28226/stat : 28226 (SArTagnan) R 28222 28224 28198 0 -1 4202560 135409 0 0 0 471947 27 0 0 25 0 8 0 803415272 6556835840 1231601 33554432000 4194304 4625164 140733775929104 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28224/tid=28227] ppid=28222 vsize=6403160 CPUtime=4721.48 cores=0-7
/proc/28224/task/28227/stat : 28227 (SArTagnan) R 28222 28224 28198 0 -1 4202560 281555 0 0 0 471875 273 0 0 25 0 8 0 803415272 6556835840 1231601 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28224/tid=28228] ppid=28222 vsize=6403160 CPUtime=4688 cores=0-7
/proc/28224/task/28228/stat : 28228 (SArTagnan) R 28222 28224 28198 0 -1 4202560 30149459 0 0 0 464013 4787 0 0 25 0 8 0 803415272 6556835840 1231601 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351792 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28224/tid=28229] ppid=28222 vsize=6403160 CPUtime=4722.24 cores=0-7
/proc/28224/task/28229/stat : 28229 (SArTagnan) R 28222 28224 28198 0 -1 4202560 91809 0 0 0 472187 37 0 0 25 0 8 0 803415272 6556835840 1231601 33554432000 4194304 4625164 140733775929104 18446744073709551615 4352005 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28224/tid=28230] ppid=28222 vsize=6403160 CPUtime=4722.33 cores=0-7
/proc/28224/task/28230/stat : 28230 (SArTagnan) R 28222 28224 28198 0 -1 4202560 181154 0 0 0 472197 36 0 0 25 0 8 0 803415272 6556835840 1231601 33554432000 4194304 4625164 140733775929104 18446744073709551615 4460412 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28224/tid=28231] ppid=28222 vsize=6403160 CPUtime=4722.31 cores=0-7
/proc/28224/task/28231/stat : 28231 (SArTagnan) R 28222 28224 28198 0 -1 4202560 175494 0 0 0 472195 36 0 0 25 0 8 0 803415272 6556835840 1231601 33554432000 4194304 4625164 140733775929104 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28224/tid=28232] ppid=28222 vsize=6403160 CPUtime=4706.64 cores=0-7
/proc/28224/task/28232/stat : 28232 (SArTagnan) R 28222 28224 28198 0 -1 4202560 121195 0 0 0 470597 67 0 0 25 0 8 0 803415272 6556835840 1231601 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351818 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37719.1
Current children cumulated vsize (KiB) 6403160

[startup+4782.3 s]
/proc/loadavg: 8.04 8.08 8.00 9/374 28384
/proc/meminfo: memFree=18065252/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=6403160 CPUtime=38198.3 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 33769428 0 0 0 3814142 5687 0 0 25 0 8 0 803415263 6556835840 1240514 33554432000 4194304 4625164 140733775929104 18446744073709551615 4393000 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/28224/statm: 1600790 1240514 352 106 0 1596293 0
[pid=28224/tid=28226] ppid=28222 vsize=6403160 CPUtime=4779.73 cores=0-7
/proc/28224/task/28226/stat : 28226 (SArTagnan) R 28222 28224 28198 0 -1 4202560 135595 0 0 0 477946 27 0 0 25 0 8 0 803415272 6556835840 1240514 33554432000 4194304 4625164 140733775929104 18446744073709551615 4330976 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28224/tid=28227] ppid=28222 vsize=6403160 CPUtime=4781.47 cores=0-7
/proc/28224/task/28227/stat : 28227 (SArTagnan) R 28222 28224 28198 0 -1 4202560 283905 0 0 0 477869 278 0 0 25 0 8 0 803415272 6556835840 1240514 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28224/tid=28228] ppid=28222 vsize=6403160 CPUtime=4747.98 cores=0-7
/proc/28224/task/28228/stat : 28228 (SArTagnan) R 28222 28224 28198 0 -1 4202560 30547138 0 0 0 469952 4846 0 0 25 0 8 0 803415272 6556835840 1240514 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28224/tid=28229] ppid=28222 vsize=6403160 CPUtime=4782.22 cores=0-7
/proc/28224/task/28229/stat : 28229 (SArTagnan) R 28222 28224 28198 0 -1 4202560 91819 0 0 0 478185 37 0 0 25 0 8 0 803415272 6556835840 1240514 33554432000 4194304 4625164 140733775929104 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28224/tid=28230] ppid=28222 vsize=6403160 CPUtime=4782.32 cores=0-7
/proc/28224/task/28230/stat : 28230 (SArTagnan) R 28222 28224 28198 0 -1 4202560 181334 0 0 0 478196 36 0 0 25 0 8 0 803415272 6556835840 1240514 33554432000 4194304 4625164 140733775929104 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28224/tid=28231] ppid=28222 vsize=6403160 CPUtime=4782.3 cores=0-7
/proc/28224/task/28231/stat : 28231 (SArTagnan) R 28222 28224 28198 0 -1 4202560 175577 0 0 0 478194 36 0 0 25 0 8 0 803415272 6556835840 1240514 33554432000 4194304 4625164 140733775929104 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28224/tid=28232] ppid=28222 vsize=6403160 CPUtime=4766.62 cores=0-7
/proc/28224/task/28232/stat : 28232 (SArTagnan) R 28222 28224 28198 0 -1 4202560 122922 0 0 0 476594 68 0 0 25 0 8 0 803415272 6556835840 1240514 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38198.3
Current children cumulated vsize (KiB) 6403160

[startup+4842.3 s]
/proc/loadavg: 8.08 8.08 8.00 9/374 28385
/proc/meminfo: memFree=18025804/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=6403160 CPUtime=38677.6 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 34285889 0 0 0 3861977 5779 0 0 25 0 8 0 803415263 6556835840 1252073 33554432000 4194304 4625164 140733775929104 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/28224/statm: 1600790 1252073 352 106 0 1596293 0
[pid=28224/tid=28226] ppid=28222 vsize=6403160 CPUtime=4839.73 cores=0-7
/proc/28224/task/28226/stat : 28226 (SArTagnan) R 28222 28224 28198 0 -1 4202560 135798 0 0 0 483946 27 0 0 25 0 8 0 803415272 6556835840 1252073 33554432000 4194304 4625164 140733775929104 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28224/tid=28227] ppid=28222 vsize=6403160 CPUtime=4841.47 cores=0-7
/proc/28224/task/28227/stat : 28227 (SArTagnan) R 28222 28224 28198 0 -1 4202560 287504 0 0 0 483863 284 0 0 25 0 8 0 803415272 6556835840 1252073 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28224/tid=28228] ppid=28222 vsize=6403160 CPUtime=4807.96 cores=0-7
/proc/28224/task/28228/stat : 28228 (SArTagnan) R 28222 28224 28198 0 -1 4202560 31053442 0 0 0 475865 4931 0 0 25 0 8 0 803415272 6556835840 1252073 33554432000 4194304 4625164 140733775929104 18446744073709551615 4335156 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28224/tid=28229] ppid=28222 vsize=6403160 CPUtime=4842.22 cores=0-7
/proc/28224/task/28229/stat : 28229 (SArTagnan) R 28222 28224 28198 0 -1 4202560 91914 0 0 0 484185 37 0 0 25 0 8 0 803415272 6556835840 1252073 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28224/tid=28230] ppid=28222 vsize=6403160 CPUtime=4842.32 cores=0-7
/proc/28224/task/28230/stat : 28230 (SArTagnan) R 28222 28224 28198 0 -1 4202560 181529 0 0 0 484196 36 0 0 25 0 8 0 803415272 6556835840 1252073 33554432000 4194304 4625164 140733775929104 18446744073709551615 4460432 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28224/tid=28231] ppid=28222 vsize=6403160 CPUtime=4842.3 cores=0-7
/proc/28224/task/28231/stat : 28231 (SArTagnan) R 28222 28224 28198 0 -1 4202560 176794 0 0 0 484194 36 0 0 25 0 8 0 803415272 6556835840 1252073 33554432000 4194304 4625164 140733775929104 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28224/tid=28232] ppid=28222 vsize=6403160 CPUtime=4826.62 cores=0-7
/proc/28224/task/28232/stat : 28232 (SArTagnan) R 28222 28224 28198 0 -1 4202560 123030 0 0 0 482594 68 0 0 25 0 8 0 803415272 6556835840 1252073 33554432000 4194304 4625164 140733775929104 18446744073709551615 4352014 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38677.6
Current children cumulated vsize (KiB) 6403160

[startup+4902.3 s]
/proc/loadavg: 8.09 8.08 8.00 9/374 28387
/proc/meminfo: memFree=17985608/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=6403160 CPUtime=39156.9 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 34850782 0 0 0 3909805 5882 0 0 25 0 8 0 803415263 6556835840 1258815 33554432000 4194304 4625164 140733775929104 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/28224/statm: 1600790 1258815 352 106 0 1596293 0
[pid=28224/tid=28226] ppid=28222 vsize=6403160 CPUtime=4899.74 cores=0-7
/proc/28224/task/28226/stat : 28226 (SArTagnan) R 28222 28224 28198 0 -1 4202560 136048 0 0 0 489947 27 0 0 25 0 8 0 803415272 6556835840 1258815 33554432000 4194304 4625164 140733775929104 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28224/tid=28227] ppid=28222 vsize=6403160 CPUtime=4901.46 cores=0-7
/proc/28224/task/28227/stat : 28227 (SArTagnan) R 28222 28224 28198 0 -1 4202560 290916 0 0 0 489856 290 0 0 25 0 8 0 803415272 6556835840 1258815 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28224/tid=28228] ppid=28222 vsize=6403160 CPUtime=4867.96 cores=0-7
/proc/28224/task/28228/stat : 28228 (SArTagnan) R 28222 28224 28198 0 -1 4202560 31609391 0 0 0 481770 5026 0 0 25 0 8 0 803415272 6556835840 1258815 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28224/tid=28229] ppid=28222 vsize=6403160 CPUtime=4902.22 cores=0-7
/proc/28224/task/28229/stat : 28229 (SArTagnan) R 28222 28224 28198 0 -1 4202560 91983 0 0 0 490185 37 0 0 25 0 8 0 803415272 6556835840 1258815 33554432000 4194304 4625164 140733775929104 18446744073709551615 4339643 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28224/tid=28230] ppid=28222 vsize=6403160 CPUtime=4902.32 cores=0-7
/proc/28224/task/28230/stat : 28230 (SArTagnan) R 28222 28224 28198 0 -1 4202560 181707 0 0 0 490196 36 0 0 25 0 8 0 803415272 6556835840 1258815 33554432000 4194304 4625164 140733775929104 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28224/tid=28231] ppid=28222 vsize=6403160 CPUtime=4902.31 cores=0-7
/proc/28224/task/28231/stat : 28231 (SArTagnan) R 28222 28224 28198 0 -1 4202560 176974 0 0 0 490194 37 0 0 25 0 8 0 803415272 6556835840 1258815 33554432000 4194304 4625164 140733775929104 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28224/tid=28232] ppid=28222 vsize=6403160 CPUtime=4886.63 cores=0-7
/proc/28224/task/28232/stat : 28232 (SArTagnan) R 28222 28224 28198 0 -1 4202560 123043 0 0 0 488594 69 0 0 25 0 8 0 803415272 6556835840 1258815 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39156.9
Current children cumulated vsize (KiB) 6403160

[startup+4962.3 s]
/proc/loadavg: 8.09 8.08 8.00 9/374 28389
/proc/meminfo: memFree=17954104/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=6403160 CPUtime=39636.1 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 35391514 0 0 0 3957642 5973 0 0 25 0 8 0 803415263 6556835840 1268674 33554432000 4194304 4625164 140733775929104 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/28224/statm: 1600790 1268674 352 106 0 1596293 0
[pid=28224/tid=28226] ppid=28222 vsize=6403160 CPUtime=4959.74 cores=0-7
/proc/28224/task/28226/stat : 28226 (SArTagnan) R 28222 28224 28198 0 -1 4202560 136265 0 0 0 495947 27 0 0 25 0 8 0 803415272 6556835840 1268674 33554432000 4194304 4625164 140733775929104 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28224/tid=28227] ppid=28222 vsize=6403160 CPUtime=4961.46 cores=0-7
/proc/28224/task/28227/stat : 28227 (SArTagnan) R 28222 28224 28198 0 -1 4202560 294630 0 0 0 495853 293 0 0 25 0 8 0 803415272 6556835840 1268674 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28224/tid=28228] ppid=28222 vsize=6403160 CPUtime=4927.95 cores=0-7
/proc/28224/task/28228/stat : 28228 (SArTagnan) R 28222 28224 28198 0 -1 4202560 32111860 0 0 0 487688 5107 0 0 25 0 8 0 803415272 6556835840 1268674 33554432000 4194304 4625164 140733775929104 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28224/tid=28229] ppid=28222 vsize=6403160 CPUtime=4962.22 cores=0-7
/proc/28224/task/28229/stat : 28229 (SArTagnan) R 28222 28224 28198 0 -1 4202560 92007 0 0 0 496185 37 0 0 25 0 8 0 803415272 6556835840 1268674 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28224/tid=28230] ppid=28222 vsize=6403160 CPUtime=4962.32 cores=0-7
/proc/28224/task/28230/stat : 28230 (SArTagnan) R 28222 28224 28198 0 -1 4202560 181935 0 0 0 496196 36 0 0 25 0 8 0 803415272 6556835840 1268674 33554432000 4194304 4625164 140733775929104 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28224/tid=28231] ppid=28222 vsize=6403160 CPUtime=4962.31 cores=0-7
/proc/28224/task/28231/stat : 28231 (SArTagnan) R 28222 28224 28198 0 -1 4202560 177726 0 0 0 496194 37 0 0 25 0 8 0 803415272 6556835840 1268674 33554432000 4194304 4625164 140733775929104 18446744073709551615 4488064 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28224/tid=28232] ppid=28222 vsize=6403160 CPUtime=4946.63 cores=0-7
/proc/28224/task/28232/stat : 28232 (SArTagnan) R 28222 28224 28198 0 -1 4202560 124595 0 0 0 494593 70 0 0 25 0 8 0 803415272 6556835840 1268674 33554432000 4194304 4625164 140733775929104 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39636.2
Current children cumulated vsize (KiB) 6403160



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5007.9 s]
/proc/loadavg: 8.11 8.08 8.00 9/374 28390
/proc/meminfo: memFree=17915400/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=6469656 CPUtime=40000.5 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4202496 35723034 0 0 0 3994017 6032 0 0 25 0 8 0 803415263 6624927744 1277601 33554432000 4194304 4625164 140733775929104 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/28224/statm: 1617414 1277601 352 106 0 1612917 0
[pid=28224/tid=28226] ppid=28222 vsize=6469656 CPUtime=5005.35 cores=0-7
/proc/28224/task/28226/stat : 28226 (SArTagnan) R 28222 28224 28198 0 -1 4202560 136446 0 0 0 500508 27 0 0 25 0 8 0 803415272 6624927744 1277601 33554432000 4194304 4625164 140733775929104 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28224/tid=28227] ppid=28222 vsize=6469656 CPUtime=5007.08 cores=0-7
/proc/28224/task/28227/stat : 28227 (SArTagnan) R 28222 28224 28198 0 -1 4202560 297269 0 0 0 500411 297 0 0 25 0 8 0 803415272 6624927744 1277601 33554432000 4194304 4625164 140733775929104 18446744073709551615 4339564 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28224/tid=28228] ppid=28222 vsize=6469656 CPUtime=4973.55 cores=0-7
/proc/28224/task/28228/stat : 28228 (SArTagnan) R 28222 28224 28198 0 -1 4202560 32414529 0 0 0 492199 5156 0 0 25 0 8 0 803415272 6624927744 1277601 33554432000 4194304 4625164 140733775929104 18446744073709551615 4335407 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28224/tid=28229] ppid=28222 vsize=6469656 CPUtime=5007.83 cores=0-7
/proc/28224/task/28229/stat : 28229 (SArTagnan) R 28222 28224 28198 0 -1 4202560 92093 0 0 0 500746 37 0 0 25 0 8 0 803415272 6624927744 1277601 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28224/tid=28230] ppid=28222 vsize=6469656 CPUtime=5007.95 cores=0-7
/proc/28224/task/28230/stat : 28230 (SArTagnan) R 28222 28224 28198 0 -1 4202560 182035 0 0 0 500758 37 0 0 25 0 8 0 803415272 6624927744 1277601 33554432000 4194304 4625164 140733775929104 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28224/tid=28231] ppid=28222 vsize=6469656 CPUtime=5007.92 cores=0-7
/proc/28224/task/28231/stat : 28231 (SArTagnan) R 28222 28224 28198 0 -1 4202560 178108 0 0 0 500755 37 0 0 25 0 8 0 803415272 6624927744 1277601 33554432000 4194304 4625164 140733775929104 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28224/tid=28232] ppid=28222 vsize=6469656 CPUtime=4992.24 cores=0-7
/proc/28224/task/28232/stat : 28232 (SArTagnan) R 28222 28224 28198 0 -1 4202560 125569 0 0 0 499153 71 0 0 25 0 8 0 803415272 6624927744 1277601 33554432000 4194304 4625164 140733775929104 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 6469656

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5008.02 s]
/proc/loadavg: 8.11 8.08 8.00 9/374 28390
/proc/meminfo: memFree=17915400/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=0 CPUtime=40000.6 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4203524 35723040 0 0 0 3994023 6042 0 0 25 0 1 0 803415263 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/28224/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 0

[startup+5008.1 s]
/proc/loadavg: 8.11 8.08 8.00 9/374 28390
/proc/meminfo: memFree=17915400/32951124 swapFree=37148216/67111528
[pid=28224] ppid=28222 vsize=0 CPUtime=40000.7 cores=0-7
/proc/28224/stat : 28224 (SArTagnan) R 28222 28224 28198 0 -1 4203524 35723040 0 0 0 3994023 6050 0 0 25 0 1 0 803415263 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/28224/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5008.12
CPU time (s): 40000.8
CPU user time (s): 39940.2
CPU system time (s): 60.5278
CPU usage (%): 798.718
Max. virtual memory (cumulated for all children) (KiB): 7725812

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39940.2
system time used= 60.5278
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 35723040
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 9716
involuntary context switches= 170430

runsolver used 15.6996 second user time and 44.2113 second system time

The end

Launcher Data 

Begin job on node104 at 2011-06-01 10:49:08
IDJOB=3438476
IDBENCH=71408
IDSOLVER=1936
FILE ID=node104/3438476-1306918148
RUNJOBID= node104-1306918148-28212
PBS_JOBID= 13488524
Free space on /tmp= 73856 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/random/forced/rbsat-v2640c305320gyes5.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3438476-1306918148/watcher-3438476-1306918148 -o /tmp/evaluation-result-3438476-1306918148/solver-3438476-1306918148 -C 40000 -W 5100 -M 31000  SArTagnan HOME/instance-3438476-1306918148.cnf -threads=8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= f892a2bda6fe6aec6f2fcb4e1c541df8
RANDOM SEED=1928582870

node104.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5293.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23034176 kB
Buffers:        390764 kB
Cached:        7422812 kB
SwapCached:     163872 kB
Active:        3266452 kB
Inactive:      6320520 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23034176 kB
SwapTotal:    67111528 kB
SwapFree:     37148216 kB
Dirty:            4252 kB
Writeback:           0 kB
AnonPages:     1633136 kB
Mapped:          10156 kB
Slab:           173220 kB
PageTables:      88604 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 87697976 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73856 MiB
End job on node104 at 2011-06-01 12:12:38