Trace number 3386209

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 3981.02 1300.08

General information on the benchmark

NameSAT02/industrial/aloul/
Homer/homer16.shuffled.cnf
MD5SUMf98b1abe53ae4846a5f9102253ed913e
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark0.484926
Satisfiable
(Un)Satisfiability was proved
Number of variables264
Number of clauses1476
Sum of the clauses size3168
Maximum clause length11
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21452
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 524

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386209-1305583435/watcher-3386209-1305583435 -o /tmp/evaluation-result-3386209-1305583435/solver-3386209-1305583435 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3386209-1305583435.cnf -threads=4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 9.06 9.00 7.10 6/473 32610
/proc/meminfo: memFree=12981420/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=49344 CPUtime=0.02 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 462 0 0 0 2 0 0 0 25 0 4 0 748573953 50528256 400 33554432000 4194304 4625164 140733821260144 18446744073709551615 47898193668018 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/32610/statm: 12336 400 300 106 0 7839 0
[pid=32610/tid=32611] ppid=32608 vsize=49344 CPUtime=0 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 4 0 748573954 50528256 400 33554432000 4194304 4625164 140733821260144 18446744073709551615 47898193668018 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=49344 CPUtime=0 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 4 0 748573954 50528256 400 33554432000 4194304 4625164 140733821260144 18446744073709551615 252458122225 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=49344 CPUtime=0 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 4 0 748573954 50528256 400 33554432000 4194304 4625164 140733821260144 18446744073709551615 47898193668018 0 0 4096 2 0 0 0 -1 3 0 0 0

[startup+0.116687 s]
/proc/loadavg: 9.06 9.00 7.10 6/473 32610
/proc/meminfo: memFree=12981420/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=181928 CPUtime=0.32 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 1341 0 0 0 32 0 0 0 25 0 4 0 748573953 186294272 1083 33554432000 4194304 4625164 140733821260144 18446744073709551615 4392915 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/32610/statm: 45482 1083 345 106 0 40985 0
[pid=32610/tid=32611] ppid=32608 vsize=181928 CPUtime=0.05 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 6 0 0 0 5 0 0 0 25 0 4 0 748573954 186294272 1083 33554432000 4194304 4625164 140733821260144 18446744073709551615 252458170862 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=181928 CPUtime=0.05 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 358 0 0 0 5 0 0 0 25 0 4 0 748573954 186294272 1083 33554432000 4194304 4625164 140733821260144 18446744073709551615 252457727009 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=181928 CPUtime=0.1 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 216 0 0 0 10 0 0 0 25 0 4 0 748573954 186294272 1083 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351839 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 0.32
Current children cumulated vsize (KiB) 181928

[startup+0.200674 s]
/proc/loadavg: 9.06 9.00 7.10 6/473 32610
/proc/meminfo: memFree=12981420/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=183256 CPUtime=0.57 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 2125 0 0 0 57 0 0 0 25 0 4 0 748573953 187654144 1687 33554432000 4194304 4625164 140733821260144 18446744073709551615 4512156 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/32610/statm: 45814 1687 345 106 0 41317 0
[pid=32610/tid=32611] ppid=32608 vsize=183256 CPUtime=0.06 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 56 0 0 0 6 0 0 0 25 0 4 0 748573954 187654144 1687 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351798 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=183256 CPUtime=0.12 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 708 0 0 0 12 0 0 0 25 0 4 0 748573954 187654144 1687 33554432000 4194304 4625164 140733821260144 18446744073709551615 4354402 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=183256 CPUtime=0.18 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 445 0 0 0 18 0 0 0 25 0 4 0 748573954 187654144 1687 33554432000 4194304 4625164 140733821260144 18446744073709551615 4552340 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 0.57
Current children cumulated vsize (KiB) 183256

[startup+0.300655 s]
/proc/loadavg: 9.06 9.00 7.10 6/473 32610
/proc/meminfo: memFree=12981420/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=184888 CPUtime=0.87 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 2985 0 0 0 87 0 0 0 25 0 4 0 748573953 189325312 2239 33554432000 4194304 4625164 140733821260144 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/32610/statm: 46222 2239 345 106 0 41725 0
[pid=32610/tid=32611] ppid=32608 vsize=184888 CPUtime=0.14 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 346 0 0 0 14 0 0 0 25 0 4 0 748573954 189325312 2239 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=184888 CPUtime=0.13 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 877 0 0 0 13 0 0 0 25 0 4 0 748573954 189325312 2239 33554432000 4194304 4625164 140733821260144 18446744073709551615 4352647 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=184888 CPUtime=0.28 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 687 0 0 0 28 0 0 0 25 0 4 0 748573954 189325312 2239 33554432000 4194304 4625164 140733821260144 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 0.87
Current children cumulated vsize (KiB) 184888

[startup+0.700578 s]
/proc/loadavg: 9.06 9.00 7.10 6/473 32610
/proc/meminfo: memFree=12981420/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=253124 CPUtime=2.04 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 6467 0 0 0 204 0 0 0 25 0 4 0 748573953 259198976 3752 33554432000 4194304 4625164 140733821260144 18446744073709551615 4512475 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/32610/statm: 63281 3752 345 106 0 58784 0
[pid=32610/tid=32611] ppid=32608 vsize=253124 CPUtime=0.43 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 1153 0 0 0 43 0 0 0 18 0 4 0 748573954 259198976 3752 33554432000 4194304 4625164 140733821260144 18446744073709551615 252457727009 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=253124 CPUtime=0.41 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 2079 0 0 0 41 0 0 0 25 0 4 0 748573954 259198976 3752 33554432000 4194304 4625164 140733821260144 18446744073709551615 4352186 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=253124 CPUtime=0.49 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 1858 0 0 0 49 0 0 0 25 0 4 0 748573954 259198976 3752 33554432000 4194304 4625164 140733821260144 18446744073709551615 252457727009 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2.04
Current children cumulated vsize (KiB) 253124

[startup+1.50043 s]
/proc/loadavg: 9.06 9.00 7.10 10/477 32614
/proc/meminfo: memFree=12957836/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=255088 CPUtime=4.42 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 11017 0 0 0 441 1 0 0 25 0 4 0 748573953 261210112 5976 33554432000 4194304 4625164 140733821260144 18446744073709551615 252457714892 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/32610/statm: 63772 5976 346 106 0 59275 0
[pid=32610/tid=32611] ppid=32608 vsize=255088 CPUtime=1.23 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 2757 0 0 0 123 0 0 0 19 0 4 0 748573954 261210112 5976 33554432000 4194304 4625164 140733821260144 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=255088 CPUtime=1.01 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 3448 0 0 0 101 0 0 0 25 0 4 0 748573954 261210112 5976 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351827 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=255088 CPUtime=0.7 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 2827 0 0 0 70 0 0 0 25 0 4 0 748573954 261210112 5976 33554432000 4194304 4625164 140733821260144 18446744073709551615 4552399 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4.42
Current children cumulated vsize (KiB) 255088

[startup+3.10114 s]
/proc/loadavg: 9.05 9.00 7.10 10/477 32614
/proc/meminfo: memFree=12948288/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=267016 CPUtime=9.2 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 18670 0 0 0 918 2 0 0 25 0 4 0 748573953 273424384 9079 33554432000 4194304 4625164 140733821260144 18446744073709551615 4512854 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/32610/statm: 66754 9079 356 106 0 62257 0
[pid=32610/tid=32611] ppid=32608 vsize=267016 CPUtime=2.13 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 3988 0 0 0 213 0 0 0 20 0 4 0 748573954 273424384 9079 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=267016 CPUtime=2.51 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 6436 0 0 0 250 1 0 0 25 0 4 0 748573954 273424384 9079 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=267016 CPUtime=1.5 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 5447 0 0 0 150 0 0 0 25 0 4 0 748573954 273424384 9079 33554432000 4194304 4625164 140733821260144 18446744073709551615 4552348 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 9.2
Current children cumulated vsize (KiB) 267016

[startup+6.30054 s]
/proc/loadavg: 9.05 9.00 7.10 10/477 32614
/proc/meminfo: memFree=12930680/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=273492 CPUtime=18.75 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 40170 0 0 0 1870 5 0 0 25 0 4 0 748573953 280055808 14050 33554432000 4194304 4625164 140733821260144 18446744073709551615 4511874 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/32610/statm: 68373 14050 383 106 0 63876 0
[pid=32610/tid=32611] ppid=32608 vsize=273492 CPUtime=3.63 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 5721 0 0 0 363 0 0 0 24 0 4 0 748573954 280055808 14050 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=273492 CPUtime=5.71 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 10290 0 0 0 569 2 0 0 25 0 4 0 748573954 280055808 14050 33554432000 4194304 4625164 140733821260144 18446744073709551615 4278661 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=273492 CPUtime=3.2 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 20198 0 0 0 318 2 0 0 25 0 4 0 748573954 280055808 14050 33554432000 4194304 4625164 140733821260144 18446744073709551615 4547807 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 18.75
Current children cumulated vsize (KiB) 273492

[startup+12.7003 s]
/proc/loadavg: 8.72 8.93 7.10 10/477 32646
/proc/meminfo: memFree=17306728/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=288964 CPUtime=37.85 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 77505 0 0 0 3773 12 0 0 25 0 4 0 748573953 295899136 21497 33554432000 4194304 4625164 140733821260144 18446744073709551615 4396555 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/32610/statm: 72241 21497 440 106 0 67744 0
[pid=32610/tid=32611] ppid=32608 vsize=288964 CPUtime=7.74 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 8663 0 0 0 773 1 0 0 25 0 4 0 748573954 295899136 21497 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=288964 CPUtime=9.49 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 12032 0 0 0 947 2 0 0 25 0 4 0 748573954 295899136 21497 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351827 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=288964 CPUtime=9.09 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 51158 0 0 0 902 7 0 0 25 0 4 0 748573954 295899136 21497 33554432000 4194304 4625164 140733821260144 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37.85
Current children cumulated vsize (KiB) 288964

[startup+25.501 s]
/proc/loadavg: 8.77 8.94 7.12 10/477 32646
/proc/meminfo: memFree=17192356/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=311704 CPUtime=76.05 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 138970 0 0 0 7582 23 0 0 25 0 4 0 748573953 319184896 30403 33554432000 4194304 4625164 140733821260144 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/32610/statm: 77926 30403 444 106 0 73429 0
[pid=32610/tid=32611] ppid=32608 vsize=311704 CPUtime=17.53 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 12377 0 0 0 1752 1 0 0 25 0 4 0 748573954 319184896 30403 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=311704 CPUtime=21.59 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 22525 0 0 0 2155 4 0 0 25 0 4 0 748573954 319184896 30403 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=311704 CPUtime=15.9 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 97564 0 0 0 1575 15 0 0 25 0 4 0 748573954 319184896 30403 33554432000 4194304 4625164 140733821260144 18446744073709551615 4579399 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 76.05
Current children cumulated vsize (KiB) 311704

[startup+51.1012 s]
/proc/loadavg: 8.92 8.96 7.18 10/477 32646
/proc/meminfo: memFree=17000388/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=337448 CPUtime=152.46 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 238042 0 0 0 15207 39 0 0 25 0 4 0 748573953 345546752 46018 33554432000 4194304 4625164 140733821260144 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/32610/statm: 84362 46018 472 106 0 79865 0
[pid=32610/tid=32611] ppid=32608 vsize=337448 CPUtime=29.84 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 13592 0 0 0 2982 2 0 0 25 0 4 0 748573954 345546752 46018 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351818 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=337448 CPUtime=35.2 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 25592 0 0 0 3514 6 0 0 25 0 4 0 748573954 345546752 46018 33554432000 4194304 4625164 140733821260144 18446744073709551615 4464695 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=337448 CPUtime=40.79 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 179341 0 0 0 4052 27 0 0 25 0 4 0 748573954 345546752 46018 33554432000 4194304 4625164 140733821260144 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 152.46
Current children cumulated vsize (KiB) 337448

[startup+102.301 s]
/proc/loadavg: 9.08 9.00 7.29 10/477 32648
/proc/meminfo: memFree=16651220/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=409824 CPUtime=305.31 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 321880 0 0 0 30480 51 0 0 25 0 4 0 748573953 419659776 63353 33554432000 4194304 4625164 140733821260144 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/32610/statm: 102456 63353 536 106 0 97959 0
[pid=32610/tid=32611] ppid=32608 vsize=409824 CPUtime=75.75 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 19566 0 0 0 7572 3 0 0 25 0 4 0 748573954 419659776 63353 33554432000 4194304 4625164 140733821260144 18446744073709551615 4352047 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=409824 CPUtime=60.6 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 34009 0 0 0 6053 7 0 0 25 0 4 0 748573954 419659776 63353 33554432000 4194304 4625164 140733821260144 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=409824 CPUtime=71.12 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 236801 0 0 0 7076 36 0 0 25 0 4 0 748573954 419659776 63353 33554432000 4194304 4625164 140733821260144 18446744073709551615 4579601 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 305.31
Current children cumulated vsize (KiB) 409824

[startup+162.301 s]

################
# More data... #
################

[pid=32610] ppid=32608 vsize=1237092 CPUtime=2369.32 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 5783168 0 0 0 236022 910 0 0 25 0 4 0 748573953 1266782208 224706 33554432000 4194304 4625164 140733821260144 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/32610/statm: 309273 224706 347 106 0 304776 0
[pid=32610/tid=32611] ppid=32608 vsize=1237092 CPUtime=584.7 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 69720 0 0 0 58449 21 0 0 25 0 4 0 748573954 1266782208 224707 33554432000 4194304 4625164 140733821260144 18446744073709551615 4352827 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=1237092 CPUtime=519.73 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 105335 0 0 0 51945 28 0 0 25 0 4 0 748573954 1266782208 224708 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=1237092 CPUtime=570.22 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 4572042 0 0 0 56342 680 0 0 25 0 4 0 748573954 1266782208 224708 33554432000 4194304 4625164 140733821260144 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2369.32
Current children cumulated vsize (KiB) 1237092

[startup+822.301 s]
/proc/loadavg: 9.09 9.12 8.28 10/477 32697
/proc/meminfo: memFree=14599112/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=1347008 CPUtime=2555.16 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 6355848 0 0 0 254513 1003 0 0 25 0 4 0 748573953 1379336192 239810 33554432000 4194304 4625164 140733821260144 18446744073709551615 4512156 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/32610/statm: 336752 239810 414 106 0 332255 0
[pid=32610/tid=32611] ppid=32608 vsize=1347008 CPUtime=621.64 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 70780 0 0 0 62142 22 0 0 25 0 4 0 748573954 1379336192 239810 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351792 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=1347008 CPUtime=572.84 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 106052 0 0 0 57256 28 0 0 25 0 4 0 748573954 1379336192 239810 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351987 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=1347008 CPUtime=606.94 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 5078671 0 0 0 59931 763 0 0 25 0 4 0 748573954 1379336192 239810 33554432000 4194304 4625164 140733821260144 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2555.16
Current children cumulated vsize (KiB) 1347008

[startup+882.319 s]
/proc/loadavg: 9.07 9.11 8.32 10/477 32699
/proc/meminfo: memFree=14538452/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=1347008 CPUtime=2734.28 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 6653386 0 0 0 272375 1053 0 0 25 0 4 0 748573953 1379336192 245491 33554432000 4194304 4625164 140733821260144 18446744073709551615 4392981 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/32610/statm: 336752 245491 349 106 0 332255 0
[pid=32610/tid=32611] ppid=32608 vsize=1347008 CPUtime=659.7 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 71654 0 0 0 65947 23 0 0 25 0 4 0 748573954 1379336192 245491 33554432000 4194304 4625164 140733821260144 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=1347008 CPUtime=626.22 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 107244 0 0 0 62593 29 0 0 25 0 4 0 748573954 1379336192 245491 33554432000 4194304 4625164 140733821260144 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=1347008 CPUtime=643.41 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 5338684 0 0 0 63535 806 0 0 25 0 4 0 748573954 1379336192 245491 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351981 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2734.28
Current children cumulated vsize (KiB) 1347008

[startup+942.3 s]
/proc/loadavg: 9.12 9.11 8.37 10/477 32700
/proc/meminfo: memFree=14459204/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=1412544 CPUtime=2913.31 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 7507598 0 0 0 290152 1179 0 0 25 0 4 0 748573953 1446445056 254695 33554432000 4194304 4625164 140733821260144 18446744073709551615 4329151 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/32610/statm: 353136 254695 348 106 0 348639 0
[pid=32610/tid=32611] ppid=32608 vsize=1412544 CPUtime=719.68 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 72444 0 0 0 71945 23 0 0 25 0 4 0 748573954 1446445056 254695 33554432000 4194304 4625164 140733821260144 18446744073709551615 4352728 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=1412544 CPUtime=656.21 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 108423 0 0 0 65592 29 0 0 25 0 4 0 748573954 1446445056 254695 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351878 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=1412544 CPUtime=702.46 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 6156639 0 0 0 69321 925 0 0 25 0 4 0 748573954 1446445056 254695 33554432000 4194304 4625164 140733821260144 18446744073709551615 252457763143 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2913.31
Current children cumulated vsize (KiB) 1412544

[startup+1002.3 s]
/proc/loadavg: 9.15 9.12 8.42 10/477 32702
/proc/meminfo: memFree=14365812/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=1470836 CPUtime=3092.38 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 8353202 0 0 0 307931 1307 0 0 25 0 4 0 748573953 1506136064 266218 33554432000 4194304 4625164 140733821260144 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/32610/statm: 367709 266218 365 106 0 363212 0
[pid=32610/tid=32611] ppid=32608 vsize=1470836 CPUtime=754.33 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 73726 0 0 0 75409 24 0 0 25 0 4 0 748573954 1506136064 266218 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=1470836 CPUtime=686.38 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 109731 0 0 0 68608 30 0 0 25 0 4 0 748573954 1506136064 266218 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=1470836 CPUtime=761.54 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 6961638 0 0 0 75107 1047 0 0 25 0 4 0 748573954 1506136064 266218 33554432000 4194304 4625164 140733821260144 18446744073709551615 4352285 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3092.38
Current children cumulated vsize (KiB) 1470836

[startup+1062.3 s]
/proc/loadavg: 9.18 9.13 8.46 10/477 32703
/proc/meminfo: memFree=14306632/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=1470836 CPUtime=3271.47 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 9175424 0 0 0 325715 1432 0 0 25 0 4 0 748573953 1506136064 274388 33554432000 4194304 4625164 140733821260144 18446744073709551615 4512315 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/32610/statm: 367709 274388 434 106 0 363212 0
[pid=32610/tid=32611] ppid=32608 vsize=1470836 CPUtime=784.28 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 74772 0 0 0 78404 24 0 0 25 0 4 0 748573954 1506136064 274388 33554432000 4194304 4625164 140733821260144 18446744073709551615 4352728 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=1470836 CPUtime=735.68 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 111327 0 0 0 73537 31 0 0 25 0 4 0 748573954 1506136064 274388 33554432000 4194304 4625164 140733821260144 18446744073709551615 4352005 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=1470836 CPUtime=820.62 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 7684411 0 0 0 80909 1153 0 0 25 0 4 0 748573954 1506136064 274388 33554432000 4194304 4625164 140733821260144 18446744073709551615 4335119 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3271.47
Current children cumulated vsize (KiB) 1470836

[startup+1122.3 s]
/proc/loadavg: 9.13 9.12 8.50 10/477 32705
/proc/meminfo: memFree=14220060/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=1470836 CPUtime=3450.54 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 9958826 0 0 0 343502 1552 0 0 25 0 4 0 748573953 1506136064 286108 33554432000 4194304 4625164 140733821260144 18446744073709551615 4511863 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/32610/statm: 367709 286108 347 106 0 363212 0
[pid=32610/tid=32611] ppid=32608 vsize=1470836 CPUtime=816.49 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 74837 0 0 0 81625 24 0 0 25 0 4 0 748573954 1506136064 286108 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=1470836 CPUtime=795.69 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 113964 0 0 0 79536 33 0 0 25 0 4 0 748573954 1506136064 286108 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=1470836 CPUtime=877.47 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 8301444 0 0 0 86503 1244 0 0 25 0 4 0 748573954 1506136064 286108 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3450.54
Current children cumulated vsize (KiB) 1470836

[startup+1182.3 s]
/proc/loadavg: 9.16 9.12 8.54 10/477 32707
/proc/meminfo: memFree=14137576/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=1536372 CPUtime=3629.62 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 10288843 0 0 0 361357 1605 0 0 25 0 4 0 748573953 1573244928 296573 33554432000 4194304 4625164 140733821260144 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/32610/statm: 384093 296573 347 106 0 379596 0
[pid=32610/tid=32611] ppid=32608 vsize=1536372 CPUtime=876.49 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 75669 0 0 0 87624 25 0 0 25 0 4 0 748573954 1573244928 296573 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=1536372 CPUtime=837.18 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 115163 0 0 0 83685 33 0 0 25 0 4 0 748573954 1573244928 296573 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=1536372 CPUtime=925.16 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 8570074 0 0 0 91230 1286 0 0 25 0 4 0 748573954 1573244928 296573 33554432000 4194304 4625164 140733821260144 18446744073709551615 4552315 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3629.62
Current children cumulated vsize (KiB) 1536372

[startup+1242.3 s]
/proc/loadavg: 9.06 9.10 8.56 10/477 32708
/proc/meminfo: memFree=13898496/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=1640508 CPUtime=3808.69 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 10760338 0 0 0 379190 1679 0 0 25 0 4 0 748573953 1679880192 310198 33554432000 4194304 4625164 140733821260144 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/32610/statm: 410127 310198 412 106 0 405630 0
[pid=32610/tid=32611] ppid=32608 vsize=1640508 CPUtime=934.12 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 75914 0 0 0 93387 25 0 0 25 0 4 0 748573954 1679880192 310198 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=1640508 CPUtime=884.92 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 117002 0 0 0 88458 34 0 0 25 0 4 0 748573954 1679880192 310198 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351827 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=1640508 CPUtime=959.16 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 8949725 0 0 0 94575 1341 0 0 25 0 4 0 748573954 1679880192 310198 33554432000 4194304 4625164 140733821260144 18446744073709551615 4579643 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3808.69
Current children cumulated vsize (KiB) 1640508



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 9.14 9.12 8.60 10/477 32710
/proc/meminfo: memFree=13717300/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=1640508 CPUtime=3980.94 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 11132031 0 0 0 396361 1733 0 0 25 0 4 0 748573953 1679880192 314457 33554432000 4194304 4625164 140733821260144 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/32610/statm: 410127 314457 347 106 0 405630 0
[pid=32610/tid=32611] ppid=32608 vsize=1640508 CPUtime=991.84 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 76748 0 0 0 99158 26 0 0 25 0 4 0 748573954 1679880192 314457 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351786 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=1640508 CPUtime=913.74 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 118016 0 0 0 91339 35 0 0 25 0 4 0 748573954 1679880192 314457 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351814 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=1640508 CPUtime=988.05 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 9202775 0 0 0 97431 1374 0 0 25 0 4 0 748573954 1679880192 314457 33554432000 4194304 4625164 140733821260144 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3980.94
Current children cumulated vsize (KiB) 1640508

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1300 s]
/proc/loadavg: 9.14 9.12 8.60 10/477 32710
/proc/meminfo: memFree=13717300/32951124 swapFree=57414336/67111528
[pid=32610] ppid=32608 vsize=1640508 CPUtime=3980.94 cores=1,3,5,7
/proc/32610/stat : 32610 (SArTagnan) R 32608 32610 32434 0 -1 4202496 11132031 0 0 0 396361 1733 0 0 25 0 4 0 748573953 1679880192 314457 33554432000 4194304 4625164 140733821260144 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/32610/statm: 410127 314457 347 106 0 405630 0
[pid=32610/tid=32611] ppid=32608 vsize=1640508 CPUtime=991.84 cores=1,3,5,7
/proc/32610/task/32611/stat : 32611 (SArTagnan) R 32608 32610 32434 0 -1 4202560 76748 0 0 0 99158 26 0 0 25 0 4 0 748573954 1679880192 314457 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351786 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=32610/tid=32612] ppid=32608 vsize=1640508 CPUtime=913.74 cores=1,3,5,7
/proc/32610/task/32612/stat : 32612 (SArTagnan) R 32608 32610 32434 0 -1 4202560 118016 0 0 0 91339 35 0 0 25 0 4 0 748573954 1679880192 314457 33554432000 4194304 4625164 140733821260144 18446744073709551615 4351814 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=32610/tid=32613] ppid=32608 vsize=1640508 CPUtime=988.05 cores=1,3,5,7
/proc/32610/task/32613/stat : 32613 (SArTagnan) R 32608 32610 32434 0 -1 4202560 9202775 0 0 0 97431 1374 0 0 25 0 4 0 748573954 1679880192 314457 33554432000 4194304 4625164 140733821260144 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3980.94
Current children cumulated vsize (KiB) 1640508

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1300.08
CPU time (s): 3981.02
CPU user time (s): 3963.63
CPU system time (s): 17.3894
CPU usage (%): 306.215
Max. virtual memory (cumulated for all children) (KiB): 1651852

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3963.63
system time used= 17.3894
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 11132031
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2096
involuntary context switches= 32091

runsolver used 4.91825 second user time and 15.2317 second system time

The end

Launcher Data 

Begin job on node111 at 2011-05-17 00:03:55
IDJOB=3386209
IDBENCH=87943
IDSOLVER=1871
FILE ID=node111/3386209-1305583435
RUNJOBID= node111-1305582132-32453
PBS_JOBID= 13350305
Free space on /tmp= 73552 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT02/industrial/aloul/Homer/homer16.shuffled.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386209-1305583435/watcher-3386209-1305583435 -o /tmp/evaluation-result-3386209-1305583435/solver-3386209-1305583435 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3386209-1305583435.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f98b1abe53ae4846a5f9102253ed913e
RANDOM SEED=1293309560

node111.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5428.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      12981436 kB
Buffers:        312260 kB
Cached:        6396760 kB
SwapCached:      19200 kB
Active:       12338972 kB
Inactive:      7352516 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      12981436 kB
SwapTotal:    67111528 kB
SwapFree:     57414336 kB
Dirty:             400 kB
Writeback:           0 kB
AnonPages:    12972364 kB
Mapped:          28368 kB
Slab:           134180 kB
PageTables:      75624 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 154716536 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73540 MiB
End job on node111 at 2011-05-17 00:25:37