Trace number 3386295

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 3875.23 1300.09

General information on the benchmark

NameSAT02/industrial/aloul/
Homer/homer14.shuffled.cnf
MD5SUM1d9569aa70fee033c68469753053c259
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark0.403937
Satisfiable
(Un)Satisfiability was proved
Number of variables300
Number of clauses2130
Sum of the clauses size4500
Maximum clause length10
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 22100
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 530

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386295-1305583435/watcher-3386295-1305583435 -o /tmp/evaluation-result-3386295-1305583435/solver-3386295-1305583435 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3386295-1305583435.cnf -threads=4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 9.67 9.95 8.08 8/503 5656
/proc/meminfo: memFree=17614876/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=49356 CPUtime=0 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 468 0 0 0 0 0 0 0 25 0 4 0 748574146 50540544 405 33554432000 4194304 4625164 140734052355536 18446744073709551615 47525143844780 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/5656/statm: 12339 405 300 106 0 7842 0
[pid=5656/tid=5657] ppid=5654 vsize=49356 CPUtime=0 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 4 0 748574147 50540544 405 33554432000 4194304 4625164 140734052355536 18446744073709551615 47525143844790 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=49356 CPUtime=0 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 4 0 748574147 50540544 405 33554432000 4194304 4625164 140734052355536 18446744073709551615 47525143844780 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=49356 CPUtime=0 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 4 0 748574147 50540544 405 33554432000 4194304 4625164 140734052355536 18446744073709551615 251367603185 0 0 4096 2 0 0 0 -1 1 0 0 0

[startup+0.1715 s]
/proc/loadavg: 9.67 9.95 8.08 8/503 5656
/proc/meminfo: memFree=17614876/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=247752 CPUtime=0.3 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 1844 0 0 0 30 0 0 0 25 0 4 0 748574146 253698048 1552 33554432000 4194304 4625164 140734052355536 18446744073709551615 4519369 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/5656/statm: 61938 1552 345 106 0 57441 0
[pid=5656/tid=5657] ppid=5654 vsize=247752 CPUtime=0.1 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 354 0 0 0 10 0 0 0 25 0 4 0 748574147 253698048 1552 33554432000 4194304 4625164 140734052355536 18446744073709551615 4291783 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=247752 CPUtime=0.13 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 700 0 0 0 13 0 0 0 25 0 4 0 748574147 253698048 1552 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=247752 CPUtime=0.02 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 133 0 0 0 2 0 0 0 25 0 4 0 748574147 253698048 1553 33554432000 4194304 4625164 140734052355536 18446744073709551615 251367651822 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 0.3
Current children cumulated vsize (KiB) 247752

[startup+0.200493 s]
/proc/loadavg: 9.67 9.95 8.08 8/503 5656
/proc/meminfo: memFree=17614876/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=248740 CPUtime=0.35 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 2352 0 0 0 35 0 0 0 25 0 4 0 748574146 254709760 1773 33554432000 4194304 4625164 140734052355536 18446744073709551615 251367243976 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/5656/statm: 62185 1773 345 106 0 57688 0
[pid=5656/tid=5657] ppid=5654 vsize=248740 CPUtime=0.11 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 527 0 0 0 11 0 0 0 25 0 4 0 748574147 254709760 1773 33554432000 4194304 4625164 140734052355536 18446744073709551615 251367207969 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=248740 CPUtime=0.15 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 914 0 0 0 15 0 0 0 25 0 4 0 748574147 254709760 1773 33554432000 4194304 4625164 140734052355536 18446744073709551615 4462330 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=248740 CPUtime=0.04 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 163 0 0 0 4 0 0 0 25 0 4 0 748574147 254709760 1773 33554432000 4194304 4625164 140734052355536 18446744073709551615 4547887 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 0.35
Current children cumulated vsize (KiB) 248740

[startup+0.300477 s]
/proc/loadavg: 9.67 9.95 8.08 8/503 5656
/proc/meminfo: memFree=17614876/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=249720 CPUtime=0.53 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) D 5654 5656 5390 0 -1 4202496 2940 0 0 0 53 0 0 0 25 0 4 0 748574146 255713280 2155 33554432000 4194304 4625164 140734052355536 18446744073709551615 4518211 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/5656/statm: 62430 2155 345 106 0 57933 0
[pid=5656/tid=5657] ppid=5654 vsize=249720 CPUtime=0.14 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 569 0 0 0 14 0 0 0 25 0 4 0 748574147 255713280 2155 33554432000 4194304 4625164 140734052355536 18446744073709551615 251367590071 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=249720 CPUtime=0.21 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 1201 0 0 0 21 0 0 0 25 0 4 0 748574147 255713280 2155 33554432000 4194304 4625164 140734052355536 18446744073709551615 251367207969 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=249720 CPUtime=0.11 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 316 0 0 0 11 0 0 0 25 0 4 0 748574147 255713280 2155 33554432000 4194304 4625164 140734052355536 18446744073709551615 4547221 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 0.53
Current children cumulated vsize (KiB) 249720

[startup+0.912041 s]
/proc/loadavg: 9.67 9.95 8.08 8/503 5656
/proc/meminfo: memFree=17614876/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=252208 CPUtime=1.83 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 6112 0 0 0 182 1 0 0 25 0 4 0 748574146 258260992 4002 33554432000 4194304 4625164 140734052355536 18446744073709551615 4393110 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/5656/statm: 63052 4002 345 106 0 58555 0
[pid=5656/tid=5657] ppid=5654 vsize=252208 CPUtime=0.47 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 1988 0 0 0 47 0 0 0 18 0 4 0 748574147 258260992 4002 33554432000 4194304 4625164 140734052355536 18446744073709551615 251367590071 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=252208 CPUtime=0.49 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 1965 0 0 0 49 0 0 0 25 0 4 0 748574147 258260992 4002 33554432000 4194304 4625164 140734052355536 18446744073709551615 4352728 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=252208 CPUtime=0.4 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 1029 0 0 0 40 0 0 0 25 0 4 0 748574147 258260992 4002 33554432000 4194304 4625164 140734052355536 18446744073709551615 251383530523 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1.83
Current children cumulated vsize (KiB) 252208

[startup+1.50125 s]
/proc/loadavg: 9.67 9.95 8.08 11/507 5660
/proc/meminfo: memFree=17621044/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=254348 CPUtime=3.22 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 8792 0 0 0 320 2 0 0 25 0 4 0 748574146 260452352 5482 33554432000 4194304 4625164 140734052355536 18446744073709551615 4512335 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/5656/statm: 63587 5482 346 106 0 59090 0
[pid=5656/tid=5657] ppid=5654 vsize=254348 CPUtime=1.04 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 3129 0 0 0 103 1 0 0 18 0 4 0 748574147 260452352 5482 33554432000 4194304 4625164 140734052355536 18446744073709551615 4462375 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=254348 CPUtime=0.67 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 2601 0 0 0 67 0 0 0 25 0 4 0 748574147 260452352 5482 33554432000 4194304 4625164 140734052355536 18446744073709551615 4466846 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=254348 CPUtime=0.61 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 1629 0 0 0 61 0 0 0 25 0 4 0 748574147 260452352 5482 33554432000 4194304 4625164 140734052355536 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3.22
Current children cumulated vsize (KiB) 254348

[startup+3.10096 s]
/proc/loadavg: 9.69 9.95 8.09 11/507 5660
/proc/meminfo: memFree=17626260/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=264024 CPUtime=7.1 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 14968 0 0 0 707 3 0 0 25 0 4 0 748574146 270360576 9346 33554432000 4194304 4625164 140734052355536 18446744073709551615 4512125 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/5656/statm: 66006 9346 363 106 0 61509 0
[pid=5656/tid=5657] ppid=5654 vsize=264024 CPUtime=1.89 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 5001 0 0 0 188 1 0 0 20 0 4 0 748574147 270360576 9346 33554432000 4194304 4625164 140734052355536 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=264024 CPUtime=1.39 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 3904 0 0 0 139 0 0 0 25 0 4 0 748574147 270360576 9346 33554432000 4194304 4625164 140734052355536 18446744073709551615 4339639 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=264024 CPUtime=1.33 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 3696 0 0 0 133 0 0 0 25 0 4 0 748574147 270360576 9346 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 7.1
Current children cumulated vsize (KiB) 264024

[startup+6.30037 s]
/proc/loadavg: 9.69 9.95 8.09 11/507 5660
/proc/meminfo: memFree=17603576/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=271148 CPUtime=15.14 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 20784 0 0 0 1510 4 0 0 25 0 4 0 748574146 277655552 12782 33554432000 4194304 4625164 140734052355536 18446744073709551615 4519568 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/5656/statm: 67787 12782 357 106 0 63290 0
[pid=5656/tid=5657] ppid=5654 vsize=271148 CPUtime=3.39 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 6994 0 0 0 338 1 0 0 25 0 4 0 748574147 277655552 12782 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351811 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=271148 CPUtime=3.09 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 5782 0 0 0 308 1 0 0 25 0 4 0 748574147 277655552 12782 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351909 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=271148 CPUtime=2.98 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 5154 0 0 0 297 1 0 0 25 0 4 0 748574147 277655552 12782 33554432000 4194304 4625164 140734052355536 18446744073709551615 4552410 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 15.14
Current children cumulated vsize (KiB) 271148

[startup+12.7012 s]
/proc/loadavg: 9.74 9.95 8.11 11/507 5660
/proc/meminfo: memFree=17576540/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=288880 CPUtime=34.23 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 41767 0 0 0 3415 8 0 0 25 0 4 0 748574146 295813120 19567 33554432000 4194304 4625164 140734052355536 18446744073709551615 4519341 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/5656/statm: 72220 19567 349 106 0 67723 0
[pid=5656/tid=5657] ppid=5654 vsize=288880 CPUtime=6.79 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 13899 0 0 0 677 2 0 0 25 0 4 0 748574147 295813120 19567 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=288880 CPUtime=6.3 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 9244 0 0 0 629 1 0 0 25 0 4 0 748574147 295813120 19567 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=288880 CPUtime=9.05 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 14781 0 0 0 903 2 0 0 25 0 4 0 748574147 295813120 19567 33554432000 4194304 4625164 140734052355536 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 34.23
Current children cumulated vsize (KiB) 288880

[startup+25.5008 s]
/proc/loadavg: 9.78 9.95 8.13 11/507 5661
/proc/meminfo: memFree=17485536/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=317024 CPUtime=72.45 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 79061 0 0 0 7230 15 0 0 25 0 4 0 748574146 324632576 30673 33554432000 4194304 4625164 140734052355536 18446744073709551615 4512891 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/5656/statm: 79256 30673 473 106 0 74759 0
[pid=5656/tid=5657] ppid=5654 vsize=317024 CPUtime=13.33 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 21743 0 0 0 1329 4 0 0 25 0 4 0 748574147 324632576 30675 33554432000 4194304 4625164 140734052355536 18446744073709551615 4352728 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=317024 CPUtime=12.59 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 14219 0 0 0 1256 3 0 0 25 0 4 0 748574147 324632576 30675 33554432000 4194304 4625164 140734052355536 18446744073709551615 4352285 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=317024 CPUtime=21.65 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 37449 0 0 0 2160 5 0 0 25 0 4 0 748574147 324632576 30675 33554432000 4194304 4625164 140734052355536 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 72.45
Current children cumulated vsize (KiB) 317024

[startup+51.1011 s]
/proc/loadavg: 9.93 9.97 8.18 11/507 5661
/proc/meminfo: memFree=17398584/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=353604 CPUtime=148.78 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 112571 0 0 0 14855 23 0 0 25 0 4 0 748574146 362090496 49993 33554432000 4194304 4625164 140734052355536 18446744073709551615 4512156 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/5656/statm: 88401 49993 869 106 0 83904 0
[pid=5656/tid=5657] ppid=5654 vsize=353604 CPUtime=26.22 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 32615 0 0 0 2615 7 0 0 25 0 4 0 748574147 362090496 49993 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=353604 CPUtime=25.29 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 22520 0 0 0 2523 6 0 0 25 0 4 0 748574147 362090496 49993 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=353604 CPUtime=46.82 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 49802 0 0 0 4674 8 0 0 25 0 4 0 748574147 362090496 49993 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 148.78
Current children cumulated vsize (KiB) 353604

[startup+102.301 s]
/proc/loadavg: 9.97 9.97 8.28 11/507 5663
/proc/meminfo: memFree=17267576/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=477852 CPUtime=301.56 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 364294 0 0 0 30093 63 0 0 25 0 4 0 748574146 489320448 80760 33554432000 4194304 4625164 140734052355536 18446744073709551615 4512007 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/5656/statm: 119463 80760 2058 106 0 114966 0
[pid=5656/tid=5657] ppid=5654 vsize=477852 CPUtime=51.88 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 46900 0 0 0 5179 9 0 0 25 0 4 0 748574147 489320448 80760 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=477852 CPUtime=50.81 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 31322 0 0 0 5073 8 0 0 25 0 4 0 748574147 489320448 80760 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=477852 CPUtime=97.2 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 263082 0 0 0 9681 39 0 0 25 0 4 0 748574147 489320448 80760 33554432000 4194304 4625164 140734052355536 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 301.56
Current children cumulated vsize (KiB) 477852

[startup+162.3 s]

################
# More data... #
################

[pid=5656] ppid=5654 vsize=1587040 CPUtime=2270.78 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 2188534 0 0 0 226729 349 0 0 25 0 4 0 748574146 1625128960 304227 33554432000 4194304 4625164 140734052355536 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/5656/statm: 396760 304227 421 106 0 392263 0
[pid=5656/tid=5657] ppid=5654 vsize=1587040 CPUtime=638.4 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 221219 0 0 0 63801 39 0 0 25 0 4 0 748574147 1625128960 304227 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=1587040 CPUtime=411.32 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 75219 0 0 0 41104 28 0 0 25 0 4 0 748574147 1625128960 304227 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351958 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=1587040 CPUtime=581.75 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 1468590 0 0 0 57960 215 0 0 25 0 4 0 748574147 1625128960 304227 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2270.78
Current children cumulated vsize (KiB) 1587040

[startup+822.3 s]
/proc/loadavg: 10.24 10.12 9.26 11/507 5682
/proc/meminfo: memFree=15876832/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=1993940 CPUtime=2449.8 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 2413731 0 0 0 244601 379 0 0 25 0 4 0 748574146 2041794560 354532 33554432000 4194304 4625164 140734052355536 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/5656/statm: 498485 354532 421 106 0 493988 0
[pid=5656/tid=5657] ppid=5654 vsize=1993940 CPUtime=673.18 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 228120 0 0 0 67278 40 0 0 25 0 4 0 748574147 2041794560 354532 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=1993940 CPUtime=441.83 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 76365 0 0 0 44153 30 0 0 25 0 4 0 748574147 2041794560 354532 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=1993940 CPUtime=640.76 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 1520732 0 0 0 63854 222 0 0 25 0 4 0 748574147 2041794560 354532 33554432000 4194304 4625164 140734052355536 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2449.8
Current children cumulated vsize (KiB) 1993940

[startup+882.301 s]
/proc/loadavg: 10.23 10.14 9.32 11/507 5684
/proc/meminfo: memFree=15792628/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=1993940 CPUtime=2628.83 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 2557401 0 0 0 262483 400 0 0 25 0 4 0 748574146 2041794560 364132 33554432000 4194304 4625164 140734052355536 18446744073709551615 4511967 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/5656/statm: 498485 364132 421 106 0 493988 0
[pid=5656/tid=5657] ppid=5654 vsize=1993940 CPUtime=723.38 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 228674 0 0 0 72297 41 0 0 25 0 4 0 748574147 2041794560 364132 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=1993940 CPUtime=471.53 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 76770 0 0 0 47123 30 0 0 25 0 4 0 748574147 2041794560 364132 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=1993940 CPUtime=685.27 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 1650620 0 0 0 68287 240 0 0 25 0 4 0 748574147 2041794560 364132 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2628.83
Current children cumulated vsize (KiB) 1993940

[startup+942.301 s]
/proc/loadavg: 8.06 9.58 9.18 7/498 5702
/proc/meminfo: memFree=17470004/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=2060544 CPUtime=2807.85 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 2951312 0 0 0 280335 450 0 0 25 0 4 0 748574146 2109997056 372686 33554432000 4194304 4625164 140734052355536 18446744073709551615 4511967 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/5656/statm: 515136 372686 422 106 0 510639 0
[pid=5656/tid=5657] ppid=5654 vsize=2060544 CPUtime=756.28 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 228999 0 0 0 75587 41 0 0 25 0 4 0 748574147 2109997056 372686 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=2060544 CPUtime=514.43 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 76978 0 0 0 51413 30 0 0 25 0 4 0 748574147 2109997056 372686 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=2060544 CPUtime=741.55 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 2036614 0 0 0 73866 289 0 0 25 0 4 0 748574147 2109997056 372686 33554432000 4194304 4625164 140734052355536 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2807.85
Current children cumulated vsize (KiB) 2060544

[startup+1002.3 s]
/proc/loadavg: 6.69 8.88 8.96 7/498 5704
/proc/meminfo: memFree=17433900/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=2060544 CPUtime=2986.9 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 3391674 0 0 0 298177 513 0 0 25 0 4 0 748574146 2109997056 381749 33554432000 4194304 4625164 140734052355536 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/5656/statm: 515136 381749 433 106 0 510639 0
[pid=5656/tid=5657] ppid=5654 vsize=2060544 CPUtime=786.27 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 229514 0 0 0 78586 41 0 0 25 0 4 0 748574147 2109997056 381749 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=2060544 CPUtime=574.45 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 79437 0 0 0 57412 33 0 0 25 0 4 0 748574147 2109997056 381749 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=2060544 CPUtime=800.59 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 2465378 0 0 0 79712 347 0 0 25 0 4 0 748574147 2109997056 381749 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2986.9
Current children cumulated vsize (KiB) 2060544

[startup+1062.3 s]
/proc/loadavg: 6.27 8.39 8.79 7/498 5705
/proc/meminfo: memFree=17393456/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=2101884 CPUtime=3165.89 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 3760900 0 0 0 316024 565 0 0 25 0 4 0 748574146 2152329216 391055 33554432000 4194304 4625164 140734052355536 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/5656/statm: 525471 391055 347 106 0 520974 0
[pid=5656/tid=5657] ppid=5654 vsize=2101884 CPUtime=816.36 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 230887 0 0 0 81595 41 0 0 25 0 4 0 748574147 2152329216 391055 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=2101884 CPUtime=622.69 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 80192 0 0 0 62233 36 0 0 25 0 4 0 748574147 2152329216 391055 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=2101884 CPUtime=859.59 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 2827737 0 0 0 85564 395 0 0 25 0 4 0 748574147 2152329216 391055 33554432000 4194304 4625164 140734052355536 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3165.89
Current children cumulated vsize (KiB) 2101884

[startup+1122.32 s]
/proc/loadavg: 6.09 7.92 8.60 7/498 5707
/proc/meminfo: memFree=17366412/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=2101884 CPUtime=3344.97 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 4215118 0 0 0 333872 625 0 0 25 0 4 0 748574146 2152329216 398553 33554432000 4194304 4625164 140734052355536 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/5656/statm: 525471 398553 409 106 0 520974 0
[pid=5656/tid=5657] ppid=5654 vsize=2101884 CPUtime=846.39 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 231679 0 0 0 84597 42 0 0 25 0 4 0 748574147 2152329216 398553 33554432000 4194304 4625164 140734052355536 18446744073709551615 4352053 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=2101884 CPUtime=652.69 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 81114 0 0 0 65232 37 0 0 25 0 4 0 748574147 2152329216 398553 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=2101884 CPUtime=918.63 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 3275640 0 0 0 91410 453 0 0 25 0 4 0 748574147 2152329216 398553 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3344.97
Current children cumulated vsize (KiB) 2101884

[startup+1182.3 s]
/proc/loadavg: 6.13 7.62 8.45 7/498 5709
/proc/meminfo: memFree=17324608/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=2196012 CPUtime=3523.94 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 4598282 0 0 0 351713 681 0 0 25 0 4 0 748574146 2248716288 408048 33554432000 4194304 4625164 140734052355536 18446744073709551615 4512012 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/5656/statm: 549003 408048 347 106 0 544506 0
[pid=5656/tid=5657] ppid=5654 vsize=2196012 CPUtime=891.85 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 233186 0 0 0 89142 43 0 0 25 0 4 0 748574147 2248716288 408048 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=2196012 CPUtime=682.46 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 82020 0 0 0 68207 39 0 0 25 0 4 0 748574147 2248716288 408048 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=2196012 CPUtime=962.39 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 3602931 0 0 0 95739 500 0 0 25 0 4 0 748574147 2248716288 408048 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351958 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3523.94
Current children cumulated vsize (KiB) 2196012

[startup+1242.3 s]
/proc/loadavg: 6.08 7.32 8.29 7/498 5710
/proc/meminfo: memFree=17306236/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=2196012 CPUtime=3702.96 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 4780346 0 0 0 369583 713 0 0 25 0 4 0 748574146 2248716288 413444 33554432000 4194304 4625164 140734052355536 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/5656/statm: 549003 413444 347 106 0 544506 0
[pid=5656/tid=5657] ppid=5654 vsize=2196012 CPUtime=951.86 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 235601 0 0 0 95141 45 0 0 25 0 4 0 748574147 2248716288 413444 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=2196012 CPUtime=717.07 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 83174 0 0 0 71663 44 0 0 25 0 4 0 748574147 2248716288 413444 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351818 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=2196012 CPUtime=996.71 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 3779646 0 0 0 99147 524 0 0 25 0 4 0 748574147 2248716288 413444 33554432000 4194304 4625164 140734052355536 18446744073709551615 4335407 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3702.96
Current children cumulated vsize (KiB) 2196012



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 6.07 7.11 8.16 7/498 5712
/proc/meminfo: memFree=17259212/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=2203256 CPUtime=3875.13 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 4983963 0 0 0 386765 748 0 0 25 0 4 0 748574146 2256134144 425068 33554432000 4194304 4625164 140734052355536 18446744073709551615 4511874 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/5656/statm: 550814 425068 390 106 0 546317 0
[pid=5656/tid=5657] ppid=5654 vsize=2203256 CPUtime=990.94 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 237506 0 0 0 99045 49 0 0 25 0 4 0 748574147 2256134144 425068 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=2203256 CPUtime=767.82 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 84829 0 0 0 76735 47 0 0 25 0 4 0 748574147 2256134144 425068 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=2203256 CPUtime=1026.03 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 3972302 0 0 0 102052 551 0 0 25 0 4 0 748574147 2256134144 425068 33554432000 4194304 4625164 140734052355536 18446744073709551615 4335407 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3875.13
Current children cumulated vsize (KiB) 2203256

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1300 s]
/proc/loadavg: 6.07 7.11 8.16 7/498 5712
/proc/meminfo: memFree=17259212/32950928 swapFree=38710912/67111528
[pid=5656] ppid=5654 vsize=2203256 CPUtime=3875.13 cores=1,3,5,7
/proc/5656/stat : 5656 (SArTagnan) R 5654 5656 5390 0 -1 4202496 4983963 0 0 0 386765 748 0 0 25 0 4 0 748574146 2256134144 425068 33554432000 4194304 4625164 140734052355536 18446744073709551615 4511874 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/5656/statm: 550814 425068 390 106 0 546317 0
[pid=5656/tid=5657] ppid=5654 vsize=2203256 CPUtime=990.94 cores=1,3,5,7
/proc/5656/task/5657/stat : 5657 (SArTagnan) R 5654 5656 5390 0 -1 4202560 237506 0 0 0 99045 49 0 0 25 0 4 0 748574147 2256134144 425068 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=5656/tid=5658] ppid=5654 vsize=2203256 CPUtime=767.82 cores=1,3,5,7
/proc/5656/task/5658/stat : 5658 (SArTagnan) R 5654 5656 5390 0 -1 4202560 84829 0 0 0 76735 47 0 0 25 0 4 0 748574147 2256134144 425068 33554432000 4194304 4625164 140734052355536 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=5656/tid=5659] ppid=5654 vsize=2203256 CPUtime=1026.03 cores=1,3,5,7
/proc/5656/task/5659/stat : 5659 (SArTagnan) R 5654 5656 5390 0 -1 4202560 3972302 0 0 0 102052 551 0 0 25 0 4 0 748574147 2256134144 425068 33554432000 4194304 4625164 140734052355536 18446744073709551615 4335407 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3875.13
Current children cumulated vsize (KiB) 2203256

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1300.09
CPU time (s): 3875.23
CPU user time (s): 3867.69
CPU system time (s): 7.54185
CPU usage (%): 298.075
Max. virtual memory (cumulated for all children) (KiB): 2380352

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3867.69
system time used= 7.54185
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 4983963
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2862
involuntary context switches= 43782

runsolver used 5.72213 second user time and 15.5736 second system time

The end

Launcher Data 

Begin job on node113 at 2011-05-17 00:03:55
IDJOB=3386295
IDBENCH=87935
IDSOLVER=1871
FILE ID=node113/3386295-1305583435
RUNJOBID= node113-1305582132-5409
PBS_JOBID= 13350303
Free space on /tmp= 73296 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT02/industrial/aloul/Homer/homer14.shuffled.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386295-1305583435/watcher-3386295-1305583435 -o /tmp/evaluation-result-3386295-1305583435/solver-3386295-1305583435 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3386295-1305583435.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 1d9569aa70fee033c68469753053c259
RANDOM SEED=1293309560

node113.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      17615008 kB
Buffers:        305892 kB
Cached:        4059868 kB
SwapCached:      93692 kB
Active:        7432276 kB
Inactive:      7584032 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      17615008 kB
SwapTotal:    67111528 kB
SwapFree:     38710912 kB
Dirty:          115440 kB
Writeback:           0 kB
AnonPages:    10569220 kB
Mapped:          26320 kB
Slab:           131532 kB
PageTables:     119220 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 157083480 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73408 MiB
End job on node113 at 2011-05-17 00:25:37