Trace number 3386104

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 3933.91 1300.11

General information on the benchmark

NameCRAFTED/rbsat/random/
forced/rbsat-v2100c215164gyes2.cnf
MD5SUM5301921bbf5d6337937a0fa8d34aaf48
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2100
Number of clauses215164
Sum of the clauses size432288
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2215094
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 570

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386104-1305578590/watcher-3386104-1305578590 -o /tmp/evaluation-result-3386104-1305578590/solver-3386104-1305578590 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3386104-1305578590.cnf -threads=4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 9.62 10.00 8.15 7/348 5905
/proc/meminfo: memFree=16830460/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=24008 CPUtime=0 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 662 0 0 0 0 0 0 0 25 0 1 0 669457250 24584192 484 33554432000 4194304 4625164 140734412031072 18446744073709551615 246516834048 0 0 4096 2 0 0 0 17 2 0 0 0
/proc/5905/statm: 6002 484 296 106 0 1505 0

[startup+0.170444 s]
/proc/loadavg: 9.62 10.00 8.15 7/348 5905
/proc/meminfo: memFree=16830460/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=201760 CPUtime=0.37 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 5847 0 0 0 37 0 0 0 25 0 4 0 669457250 206602240 3995 33554432000 4194304 4625164 140734412031072 18446744073709551615 4512970 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5905/statm: 50440 3997 345 106 0 45943 0
Current children cumulated CPU time (s) 0.37
Current children cumulated vsize (KiB) 201760

[startup+0.200439 s]
/proc/loadavg: 9.62 10.00 8.15 7/348 5905
/proc/meminfo: memFree=16830460/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=202232 CPUtime=0.46 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 6202 0 0 0 46 0 0 0 25 0 4 0 669457250 207085568 4119 33554432000 4194304 4625164 140734412031072 18446744073709551615 4512057 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5905/statm: 50558 4119 345 106 0 46061 0
Current children cumulated CPU time (s) 0.46
Current children cumulated vsize (KiB) 202232

[startup+0.300421 s]
/proc/loadavg: 9.62 10.00 8.15 7/348 5905
/proc/meminfo: memFree=16830460/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=204088 CPUtime=0.76 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 6929 0 0 0 76 0 0 0 25 0 4 0 669457250 208986112 4836 33554432000 4194304 4625164 140734412031072 18446744073709551615 4511795 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5905/statm: 51022 4836 345 106 0 46525 0
Current children cumulated CPU time (s) 0.76
Current children cumulated vsize (KiB) 204088

[startup+0.700349 s]
/proc/loadavg: 9.62 10.00 8.15 7/348 5905
/proc/meminfo: memFree=16830460/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=207752 CPUtime=1.96 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 9093 0 0 0 195 1 0 0 25 0 4 0 669457250 212738048 7000 33554432000 4194304 4625164 140734412031072 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5905/statm: 51938 7000 345 106 0 47441 0
Current children cumulated CPU time (s) 1.96
Current children cumulated vsize (KiB) 207752

[startup+1.5002 s]
/proc/loadavg: 9.62 10.00 8.15 12/352 5909
/proc/meminfo: memFree=16767704/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=213724 CPUtime=4.34 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 12377 0 0 0 432 2 0 0 25 0 4 0 669457250 218853376 10284 33554432000 4194304 4625164 140734412031072 18446744073709551615 4339447 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5905/statm: 53431 10284 345 106 0 48934 0
[pid=5905/tid=5907] ppid=5903 vsize=213724 CPUtime=1.24 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 1944 0 0 0 124 0 0 0 25 0 4 0 669457256 218853376 10284 33554432000 4194304 4625164 140734412031072 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=213724 CPUtime=0.71 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 1234 0 0 0 71 0 0 0 25 0 4 0 669457256 218853376 10284 33554432000 4194304 4625164 140734412031072 18446744073709551615 4462528 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=213724 CPUtime=0.9 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 1141 0 0 0 90 0 0 0 25 0 4 0 669457256 218853376 10284 33554432000 4194304 4625164 140734412031072 18446744073709551615 4352035 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4.34
Current children cumulated vsize (KiB) 213724

[startup+3.10091 s]
/proc/loadavg: 9.62 10.00 8.15 11/352 5909
/proc/meminfo: memFree=16732048/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=224984 CPUtime=9.13 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 18872 0 0 0 910 3 0 0 25 0 4 0 669457250 230383616 15059 33554432000 4194304 4625164 140734412031072 18446744073709551615 4512017 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5905/statm: 56246 15059 348 106 0 51749 0
[pid=5905/tid=5907] ppid=5903 vsize=224984 CPUtime=2.83 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 2925 0 0 0 283 0 0 0 25 0 4 0 669457256 230383616 15059 33554432000 4194304 4625164 140734412031072 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=224984 CPUtime=1.7 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 2967 0 0 0 170 0 0 0 25 0 4 0 669457256 230383616 15059 33554432000 4194304 4625164 140734412031072 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=224984 CPUtime=2.5 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 3454 0 0 0 250 0 0 0 25 0 4 0 669457256 230383616 15059 33554432000 4194304 4625164 140734412031072 18446744073709551615 4334224 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 9.13
Current children cumulated vsize (KiB) 224984

[startup+6.30033 s]
/proc/loadavg: 9.65 10.00 8.16 13/352 5910
/proc/meminfo: memFree=16669924/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=245932 CPUtime=18.68 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 28840 0 0 0 1863 5 0 0 25 0 4 0 669457250 251834368 21531 33554432000 4194304 4625164 140734412031072 18446744073709551615 4521565 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5905/statm: 61483 21531 346 106 0 56986 0
[pid=5905/tid=5907] ppid=5903 vsize=245932 CPUtime=5.38 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 5451 0 0 0 537 1 0 0 25 0 4 0 669457256 251834368 21531 33554432000 4194304 4625164 140734412031072 18446744073709551615 4352296 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=245932 CPUtime=3.4 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 4931 0 0 0 340 0 0 0 25 0 4 0 669457256 251834368 21531 33554432000 4194304 4625164 140734412031072 18446744073709551615 4339643 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=245932 CPUtime=4.66 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 5722 0 0 0 466 0 0 0 25 0 4 0 669457256 251834368 21531 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 18.68
Current children cumulated vsize (KiB) 245932

[startup+12.7012 s]
/proc/loadavg: 9.75 10.02 8.18 11/352 5910
/proc/meminfo: memFree=16582644/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=260916 CPUtime=37.79 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 49931 0 0 0 3770 9 0 0 25 0 4 0 669457250 267177984 28890 33554432000 4194304 4625164 140734412031072 18446744073709551615 4512968 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5905/statm: 65229 28890 346 106 0 60732 0
[pid=5905/tid=5907] ppid=5903 vsize=260916 CPUtime=10.65 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 9186 0 0 0 1063 2 0 0 25 0 4 0 669457256 267177984 28891 33554432000 4194304 4625164 140734412031072 18446744073709551615 4340391 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=260916 CPUtime=7.79 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 8503 0 0 0 778 1 0 0 25 0 4 0 669457256 267177984 28891 33554432000 4194304 4625164 140734412031072 18446744073709551615 4339554 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=260916 CPUtime=7.79 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 17028 0 0 0 777 2 0 0 25 0 4 0 669457256 267177984 28891 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351975 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37.79
Current children cumulated vsize (KiB) 260916

[startup+25.5008 s]
/proc/loadavg: 9.88 10.03 8.21 11/352 5910
/proc/meminfo: memFree=16446620/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=299596 CPUtime=76.04 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 106884 0 0 0 7584 20 0 0 25 0 4 0 669457250 306786304 44882 33554432000 4194304 4625164 140734412031072 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5905/statm: 74899 44882 346 106 0 70402 0
[pid=5905/tid=5907] ppid=5903 vsize=299596 CPUtime=20.01 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 12615 0 0 0 1998 3 0 0 25 0 4 0 669457256 306786304 44882 33554432000 4194304 4625164 140734412031072 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=299596 CPUtime=20.37 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 19021 0 0 0 2033 4 0 0 25 0 4 0 669457256 306786304 44882 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=299596 CPUtime=14.31 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 57754 0 0 0 1423 8 0 0 25 0 4 0 669457256 306786304 44882 33554432000 4194304 4625164 140734412031072 18446744073709551615 47393430481446 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 76.04
Current children cumulated vsize (KiB) 299596

[startup+51.1012 s]
/proc/loadavg: 9.98 10.04 8.26 11/352 5911
/proc/meminfo: memFree=16222280/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=360232 CPUtime=152.54 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 238766 0 0 0 15212 42 0 0 25 0 4 0 669457250 368877568 68186 33554432000 4194304 4625164 140734412031072 18446744073709551615 4511874 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/5905/statm: 90058 68186 346 106 0 85561 0
[pid=5905/tid=5907] ppid=5903 vsize=360232 CPUtime=41.38 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 20183 0 0 0 4133 5 0 0 25 0 4 0 669457256 368877568 68186 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351860 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=360232 CPUtime=41 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 26317 0 0 0 4093 7 0 0 25 0 4 0 669457256 368877568 68186 33554432000 4194304 4625164 140734412031072 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=360232 CPUtime=31.8 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 153723 0 0 0 3158 22 0 0 25 0 4 0 669457256 368877568 68186 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 152.54
Current children cumulated vsize (KiB) 360232

[startup+102.301 s]
/proc/loadavg: 10.21 10.10 8.37 11/352 5912
/proc/meminfo: memFree=15815672/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=714432 CPUtime=305.34 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 350921 0 0 0 30461 73 0 0 25 0 4 0 669457250 731578368 115891 33554432000 4194304 4625164 140734412031072 18446744073709551615 4393101 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/5905/statm: 178608 115891 348 106 0 174111 0
[pid=5905/tid=5907] ppid=5903 vsize=714432 CPUtime=81.57 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 30925 0 0 0 8146 11 0 0 25 0 4 0 669457256 731578368 115891 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=714432 CPUtime=75.13 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 42921 0 0 0 7495 18 0 0 25 0 4 0 669457256 731578368 115891 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351998 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=714432 CPUtime=76.49 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 173685 0 0 0 7623 26 0 0 25 0 4 0 669457256 731578368 115891 33554432000 4194304 4625164 140734412031072 18446744073709551615 4334251 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 305.34
Current children cumulated vsize (KiB) 714432

[startup+162.301 s]
/proc/loadavg: 10.22 10.12 8.49 11/352 5914
/proc/meminfo: memFree=15428528/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=825552 CPUtime=484.43 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 437865 0 0 0 48345 98 0 0 25 0 4 0 669457250 845365248 161559 33554432000 4194304 4625164 140734412031072 18446744073709551615 4511874 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5905/statm: 206388 161559 348 106 0 201891 0
[pid=5905/tid=5907] ppid=5903 vsize=825552 CPUtime=115.9 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 38412 0 0 0 11574 16 0 0 25 0 4 0 669457256 845365248 161559 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=825552 CPUtime=108.29 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 59812 0 0 0 10800 29 0 0 25 0 4 0 669457256 845365248 161559 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351987 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=825552 CPUtime=133.32 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 184049 0 0 0 13304 28 0 0 25 0 4 0 669457256 845365248 161559 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 484.43
Current children cumulated vsize (KiB) 825552

[startup+222.313 s]
/proc/loadavg: 10.34 10.17 8.60 11/352 5915
/proc/meminfo: memFree=15115036/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=1134148 CPUtime=663.64 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 544298 0 0 0 66243 121 0 0 25 0 4 0 669457250 1161367552 204351 33554432000 4194304 4625164 140734412031072 18446744073709551615 4393086 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/5905/statm: 283537 204351 353 106 0 279040 0
[pid=5905/tid=5907] ppid=5903 vsize=1134148 CPUtime=146.1 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 49555 0 0 0 14590 20 0 0 25 0 4 0 669457256 1161367552 204351 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=1134148 CPUtime=140.08 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 68490 0 0 0 13972 36 0 0 25 0 4 0 669457256 1161367552 204351 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=1134148 CPUtime=193.32 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 193381 0 0 0 19302 30 0 0 25 0 4 0 669457256 1161367552 204351 33554432000 4194304 4625164 140734412031072 18446744073709551615 4547365 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 663.64
Current children cumulated vsize (KiB) 1134148

[startup+282.301 s]

################
# More data... #
################

[pid=5905] ppid=5903 vsize=1883860 CPUtime=2277.44 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 2421703 0 0 0 227306 438 0 0 25 0 4 0 669457250 1929072640 372328 33554432000 4194304 4625164 140734412031072 18446744073709551615 4511795 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/5905/statm: 470965 372328 346 106 0 466468 0
[pid=5905/tid=5907] ppid=5903 vsize=1883860 CPUtime=590.45 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 80033 0 0 0 59010 35 0 0 25 0 4 0 669457256 1929072640 372328 33554432000 4194304 4625164 140734412031072 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=1883860 CPUtime=509.32 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 119143 0 0 0 50870 62 0 0 25 0 4 0 669457256 1929072640 372328 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=1883860 CPUtime=572.46 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 1691255 0 0 0 56980 266 0 0 25 0 4 0 669457256 1929072640 372328 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2277.44
Current children cumulated vsize (KiB) 1883860

[startup+822.301 s]
/proc/loadavg: 10.25 10.22 9.37 12/352 5931
/proc/meminfo: memFree=13641448/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=1923268 CPUtime=2463.65 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 2669597 0 0 0 245889 476 0 0 25 0 4 0 669457250 1969426432 382313 33554432000 4194304 4625164 140734412031072 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/5905/statm: 480817 382313 346 106 0 476320 0
[pid=5905/tid=5907] ppid=5903 vsize=1923268 CPUtime=650.45 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 80243 0 0 0 65009 36 0 0 25 0 4 0 669457256 1969426432 382313 33554432000 4194304 4625164 140734412031072 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=1923268 CPUtime=545.8 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 119382 0 0 0 54518 62 0 0 25 0 4 0 669457256 1969426432 382313 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=1923268 CPUtime=602.2 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 1929880 0 0 0 59918 302 0 0 25 0 4 0 669457256 1969426432 382313 33554432000 4194304 4625164 140734412031072 18446744073709551615 4334251 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2463.65
Current children cumulated vsize (KiB) 1923268

[startup+882.314 s]
/proc/loadavg: 10.38 10.26 9.43 11/352 5933
/proc/meminfo: memFree=13567276/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=1923780 CPUtime=2673.02 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 2679500 0 0 0 266818 484 0 0 25 0 4 0 669457250 1969950720 391644 33554432000 4194304 4625164 140734412031072 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/5905/statm: 480945 391644 346 106 0 476448 0
[pid=5905/tid=5907] ppid=5903 vsize=1923780 CPUtime=710.46 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 80482 0 0 0 71010 36 0 0 25 0 4 0 669457256 1969950720 391644 33554432000 4194304 4625164 140734412031072 18446744073709551615 4331211 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=1923780 CPUtime=605.14 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 124718 0 0 0 60446 68 0 0 25 0 4 0 669457256 1969950720 391644 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=1923780 CPUtime=632.2 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 1929951 0 0 0 62918 302 0 0 25 0 4 0 669457256 1969950720 391644 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2673.02
Current children cumulated vsize (KiB) 1923780

[startup+942.301 s]
/proc/loadavg: 10.25 10.24 9.47 11/352 5935
/proc/meminfo: memFree=13495956/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=1989972 CPUtime=2859.2 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 2839184 0 0 0 285406 514 0 0 25 0 4 0 669457250 2037731328 403393 33554432000 4194304 4625164 140734412031072 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/5905/statm: 497493 403393 346 106 0 492996 0
[pid=5905/tid=5907] ppid=5903 vsize=1989972 CPUtime=770.44 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 80712 0 0 0 77008 36 0 0 25 0 4 0 669457256 2037731328 403393 33554432000 4194304 4625164 140734412031072 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=1989972 CPUtime=641.65 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 129611 0 0 0 64090 75 0 0 25 0 4 0 669457256 2037731328 403393 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=1989972 CPUtime=661.89 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 2080017 0 0 0 65865 324 0 0 25 0 4 0 669457256 2037731328 403393 33554432000 4194304 4625164 140734412031072 18446744073709551615 4334485 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2859.2
Current children cumulated vsize (KiB) 1989972

[startup+1002.3 s]
/proc/loadavg: 10.40 10.27 9.53 11/352 5936
/proc/meminfo: memFree=13420408/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=2055508 CPUtime=3038.62 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 3062827 0 0 0 303306 556 0 0 25 0 4 0 669457250 2104840192 414059 33554432000 4194304 4625164 140734412031072 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/5905/statm: 513877 414059 346 106 0 509380 0
[pid=5905/tid=5907] ppid=5903 vsize=2055508 CPUtime=830.43 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 80899 0 0 0 83007 36 0 0 25 0 4 0 669457256 2104840192 414059 33554432000 4194304 4625164 140734412031072 18446744073709551615 4330603 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=2055508 CPUtime=671.49 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 133258 0 0 0 67068 81 0 0 25 0 4 0 669457256 2104840192 414059 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=2055508 CPUtime=691.49 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 2295339 0 0 0 68791 358 0 0 25 0 4 0 669457256 2104840192 414059 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3038.62
Current children cumulated vsize (KiB) 2055508

[startup+1062.3 s]
/proc/loadavg: 10.25 10.25 9.57 11/352 5938
/proc/meminfo: memFree=13353920/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=2086748 CPUtime=3217.95 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 3300748 0 0 0 321199 596 0 0 25 0 4 0 669457250 2136829952 425440 33554432000 4194304 4625164 140734412031072 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/5905/statm: 521687 425440 346 106 0 517190 0
[pid=5905/tid=5907] ppid=5903 vsize=2086748 CPUtime=890.42 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 81046 0 0 0 89006 36 0 0 25 0 4 0 669457256 2136829952 425440 33554432000 4194304 4625164 140734412031072 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=2086748 CPUtime=701.15 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 136897 0 0 0 70029 86 0 0 25 0 4 0 669457256 2136829952 425440 33554432000 4194304 4625164 140734412031072 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=2086748 CPUtime=721.19 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 2509084 0 0 0 71727 392 0 0 25 0 4 0 669457256 2136829952 425440 33554432000 4194304 4625164 140734412031072 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3217.95
Current children cumulated vsize (KiB) 2086748

[startup+1122.3 s]
/proc/loadavg: 10.30 10.26 9.61 12/352 5948
/proc/meminfo: memFree=13202096/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=2116236 CPUtime=3397.26 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 3547392 0 0 0 339090 636 0 0 25 0 4 0 669457250 2167025664 436188 33554432000 4194304 4625164 140734412031072 18446744073709551615 4393080 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/5905/statm: 529059 436188 346 106 0 524562 0
[pid=5905/tid=5907] ppid=5903 vsize=2116236 CPUtime=950.41 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 97061 0 0 0 95003 38 0 0 25 0 4 0 669457256 2167025664 436188 33554432000 4194304 4625164 140734412031072 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=2116236 CPUtime=730.79 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 139704 0 0 0 72989 90 0 0 25 0 4 0 669457256 2167025664 436188 33554432000 4194304 4625164 140734412031072 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=2116236 CPUtime=750.88 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 2732839 0 0 0 74664 424 0 0 25 0 4 0 669457256 2167025664 436188 33554432000 4194304 4625164 140734412031072 18446744073709551615 4547221 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3397.26
Current children cumulated vsize (KiB) 2116236

[startup+1182.3 s]
/proc/loadavg: 10.20 10.24 9.64 11/352 5950
/proc/meminfo: memFree=13124240/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=2116236 CPUtime=3577.75 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 3766808 0 0 0 357102 673 0 0 25 0 4 0 669457250 2167025664 447279 33554432000 4194304 4625164 140734412031072 18446744073709551615 4512046 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/5905/statm: 529059 447279 346 106 0 524562 0
[pid=5905/tid=5907] ppid=5903 vsize=2116236 CPUtime=1010.39 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 97359 0 0 0 101001 38 0 0 25 0 4 0 669457256 2167025664 447279 33554432000 4194304 4625164 140734412031072 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=2116236 CPUtime=761.69 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 142432 0 0 0 76075 94 0 0 25 0 4 0 669457256 2167025664 447279 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=2116236 CPUtime=780.49 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 2942695 0 0 0 77594 455 0 0 25 0 4 0 669457256 2167025664 447279 33554432000 4194304 4625164 140734412031072 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3577.75
Current children cumulated vsize (KiB) 2116236

[startup+1242.3 s]
/proc/loadavg: 10.16 10.22 9.67 11/352 5952
/proc/meminfo: memFree=12960400/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=2444612 CPUtime=3760.58 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 4162319 0 0 0 375328 730 0 0 25 0 4 0 669457250 2503282688 480039 33554432000 4194304 4625164 140734412031072 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/5905/statm: 611153 480039 346 106 0 606656 0
[pid=5905/tid=5907] ppid=5903 vsize=2444612 CPUtime=1069.2 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 97520 0 0 0 106882 38 0 0 25 0 4 0 669457256 2503282688 480039 33554432000 4194304 4625164 140734412031072 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=2444612 CPUtime=796.02 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 146555 0 0 0 79503 99 0 0 25 0 4 0 669457256 2503282688 480039 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351987 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=2444612 CPUtime=810.48 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 3147633 0 0 0 80561 487 0 0 25 0 4 0 669457256 2503282688 480039 33554432000 4194304 4625164 140734412031072 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3760.58
Current children cumulated vsize (KiB) 2444612



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 10.48 10.29 9.73 8/351 5957
/proc/meminfo: memFree=14797976/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=2576784 CPUtime=3933.79 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 4404543 0 0 0 392608 771 0 0 25 0 4 0 669457250 2638626816 490027 33554432000 4194304 4625164 140734412031072 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/5905/statm: 644196 490027 346 106 0 639699 0
[pid=5905/tid=5907] ppid=5903 vsize=2576784 CPUtime=1123.32 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 97693 0 0 0 112294 38 0 0 25 0 4 0 669457256 2638626816 490027 33554432000 4194304 4625164 140734412031072 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=2576784 CPUtime=825.58 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 149726 0 0 0 82455 103 0 0 25 0 4 0 669457256 2638626816 490027 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=2576784 CPUtime=843.59 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 3383202 0 0 0 83835 524 0 0 25 0 4 0 669457256 2638626816 490027 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3933.79
Current children cumulated vsize (KiB) 2576784

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1300 s]
/proc/loadavg: 10.48 10.29 9.73 8/351 5957
/proc/meminfo: memFree=14797976/32951124 swapFree=54450792/67111528
[pid=5905] ppid=5903 vsize=2576784 CPUtime=3933.79 cores=0,2,4,6
/proc/5905/stat : 5905 (SArTagnan) R 5903 5905 5768 0 -1 4202496 4404543 0 0 0 392608 771 0 0 25 0 4 0 669457250 2638626816 490027 33554432000 4194304 4625164 140734412031072 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/5905/statm: 644196 490027 346 106 0 639699 0
[pid=5905/tid=5907] ppid=5903 vsize=2576784 CPUtime=1123.32 cores=0,2,4,6
/proc/5905/task/5907/stat : 5907 (SArTagnan) R 5903 5905 5768 0 -1 4202560 97693 0 0 0 112294 38 0 0 25 0 4 0 669457256 2638626816 490027 33554432000 4194304 4625164 140734412031072 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=5905/tid=5908] ppid=5903 vsize=2576784 CPUtime=825.58 cores=0,2,4,6
/proc/5905/task/5908/stat : 5908 (SArTagnan) R 5903 5905 5768 0 -1 4202560 149726 0 0 0 82455 103 0 0 25 0 4 0 669457256 2638626816 490027 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=5905/tid=5909] ppid=5903 vsize=2576784 CPUtime=843.59 cores=0,2,4,6
/proc/5905/task/5909/stat : 5909 (SArTagnan) R 5903 5905 5768 0 -1 4202560 3383202 0 0 0 83835 524 0 0 25 0 4 0 669457256 2638626816 490027 33554432000 4194304 4625164 140734412031072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3933.79
Current children cumulated vsize (KiB) 2576784

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1300.11
CPU time (s): 3933.91
CPU user time (s): 3926.11
CPU system time (s): 7.80581
CPU usage (%): 302.584
Max. virtual memory (cumulated for all children) (KiB): 3078524

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3926.11
system time used= 7.80581
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 4404544
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 5422
involuntary context switches= 86824

runsolver used 3.9254 second user time and 10.3224 second system time

The end

Launcher Data 

Begin job on node134 at 2011-05-16 22:43:11
IDJOB=3386104
IDBENCH=71428
IDSOLVER=1871
FILE ID=node134/3386104-1305578590
RUNJOBID= node134-1305577288-5786
PBS_JOBID= 13350221
Free space on /tmp= 73576 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/random/forced/rbsat-v2100c215164gyes2.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386104-1305578590/watcher-3386104-1305578590 -o /tmp/evaluation-result-3386104-1305578590/solver-3386104-1305578590 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3386104-1305578590.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 5301921bbf5d6337937a0fa8d34aaf48
RANDOM SEED=1429592824

node134.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 7186.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      16830740 kB
Buffers:        165092 kB
Cached:         693172 kB
SwapCached:      16428 kB
Active:        4791156 kB
Inactive:     11090988 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      16830740 kB
SwapTotal:    67111528 kB
SwapFree:     54450792 kB
Dirty:            7468 kB
Writeback:          28 kB
AnonPages:    15015340 kB
Mapped:          16440 kB
Slab:            98188 kB
PageTables:      76744 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 72802556 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73580 MiB
End job on node134 at 2011-05-16 23:04:53