Trace number 3261960

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-03-24? (TO) 4800.43 1201.76

General information on the benchmark

Namecrafted/kullmann/VanDerWaerden/VanderWaerden_pd_3k/
VanDerWaerden_pd_2-3-24_567.cnf
MD5SUMcb7c9df8144e5ba66f8009983efe791f
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark14.0229
Satisfiable
(Un)Satisfiability was proved
Number of variables284
Number of clauses42826
Sum of the clauses size191271
Maximum clause length24
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2377
Number of clauses of size 339422
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 53027

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3261960-1304093552/watcher-3261960-1304093552 -o /tmp/evaluation-result-3261960-1304093552/solver-3261960-1304093552 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3261960-1304093552.cnf -threads=4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.11 8.11 8.03 6/180 28596
/proc/meminfo: memFree=18297416/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=19652 CPUtime=0 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 502 0 0 0 0 0 0 0 23 0 1 0 520951876 20123648 440 33554432000 4194304 4623224 140734455479248 18446744073709551615 4366247 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/28596/statm: 4913 440 296 105 0 417 0

[startup+0.0328859 s]
/proc/loadavg: 8.11 8.11 8.03 6/180 28596
/proc/meminfo: memFree=18297416/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=59828 CPUtime=0.02 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 4007 0 0 0 2 0 0 0 24 0 4 0 520951876 61263872 2873 33554432000 4194304 4623224 140734455479248 18446744073709551615 47734303082418 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 14957 2873 302 105 0 10461 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 59828

[startup+0.100802 s]
/proc/loadavg: 8.11 8.11 8.03 6/180 28596
/proc/meminfo: memFree=18297416/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=197096 CPUtime=0.29 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 6875 0 0 0 29 0 0 0 25 0 4 0 520951876 201826304 4139 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 49274 4140 337 105 0 44778 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 197096

[startup+0.300764 s]
/proc/loadavg: 8.11 8.11 8.03 6/180 28596
/proc/meminfo: memFree=18297416/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=198764 CPUtime=1.09 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 10261 0 0 0 108 1 0 0 25 0 4 0 520951876 203534336 4560 33554432000 4194304 4623224 140734455479248 18446744073709551615 4414846 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 49691 4560 337 105 0 45195 0
Current children cumulated CPU time (s) 1.09
Current children cumulated vsize (KiB) 198764

[startup+0.700684 s]
/proc/loadavg: 8.11 8.11 8.03 6/180 28596
/proc/meminfo: memFree=18297416/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=200772 CPUtime=2.69 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 15240 0 0 0 268 1 0 0 25 0 4 0 520951876 205590528 5436 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510298 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 50193 5436 337 105 0 45697 0
Current children cumulated CPU time (s) 2.69
Current children cumulated vsize (KiB) 200772

[startup+1.50053 s]
/proc/loadavg: 8.10 8.11 8.02 9/184 28600
/proc/meminfo: memFree=18264284/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=202772 CPUtime=5.89 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 24727 0 0 0 586 3 0 0 25 0 4 0 520951876 207638528 6997 33554432000 4194304 4623224 140734455479248 18446744073709551615 4414885 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 50693 6997 337 105 0 46197 0
[pid=28596/tid=28598] ppid=28594 vsize=202772 CPUtime=1.46 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 2075 0 0 0 146 0 0 0 25 0 4 0 520951879 207638528 6997 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376643 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=202772 CPUtime=1.45 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 2000 0 0 0 145 0 0 0 25 0 4 0 520951879 207638528 6997 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376749 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=202772 CPUtime=1.45 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 15767 0 0 0 144 1 0 0 25 0 4 0 520951879 207638528 6997 33554432000 4194304 4623224 140734455479248 18446744073709551615 254500389049 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 5.89
Current children cumulated vsize (KiB) 202772

[startup+3.10021 s]
/proc/loadavg: 8.10 8.11 8.02 9/184 28600
/proc/meminfo: memFree=18245828/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=205828 CPUtime=12.28 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 39293 0 0 0 1223 5 0 0 25 0 4 0 520951876 210767872 9546 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510217 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 51457 9546 341 105 0 46961 0
[pid=28596/tid=28598] ppid=28594 vsize=205828 CPUtime=3.06 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 3528 0 0 0 306 0 0 0 25 0 4 0 520951879 210767872 9546 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376708 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=205828 CPUtime=3.05 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 2308 0 0 0 305 0 0 0 25 0 4 0 520951879 210767872 9546 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376869 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=205828 CPUtime=3.06 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 28317 0 0 0 302 4 0 0 25 0 4 0 520951879 210767872 9546 33554432000 4194304 4623224 140734455479248 18446744073709551615 4545376 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 12.28
Current children cumulated vsize (KiB) 205828

[startup+6.30057 s]
/proc/loadavg: 8.10 8.11 8.02 9/184 28630
/proc/meminfo: memFree=18224696/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=208224 CPUtime=25.06 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 69987 0 0 0 2496 10 0 0 25 0 4 0 520951876 213221376 14122 33554432000 4194304 4623224 140734455479248 18446744073709551615 4385133 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 52056 14122 342 105 0 47560 0
[pid=28596/tid=28598] ppid=28594 vsize=208224 CPUtime=6.26 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 5768 0 0 0 626 0 0 0 25 0 4 0 520951879 213221376 14122 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376678 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=208224 CPUtime=6.23 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 4052 0 0 0 623 0 0 0 25 0 4 0 520951879 213221376 14122 33554432000 4194304 4623224 140734455479248 18446744073709551615 4377587 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=208224 CPUtime=6.26 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 54511 0 0 0 618 8 0 0 25 0 4 0 520951879 213221376 14122 33554432000 4194304 4623224 140734455479248 18446744073709551615 4550465 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 25.06
Current children cumulated vsize (KiB) 208224

[startup+12.7003 s]
/proc/loadavg: 8.08 8.10 8.02 9/184 28630
/proc/meminfo: memFree=18155264/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=218044 CPUtime=50.63 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 109564 0 0 0 5047 16 0 0 25 0 4 0 520951876 223277056 21824 33554432000 4194304 4623224 140734455479248 18446744073709551615 4384231 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 54511 21824 348 105 0 50015 0
[pid=28596/tid=28598] ppid=28594 vsize=218044 CPUtime=12.65 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 6557 0 0 0 1265 0 0 0 25 0 4 0 520951879 223277056 21824 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376672 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=218044 CPUtime=12.59 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 6353 0 0 0 1259 0 0 0 25 0 4 0 520951879 223277056 21824 33554432000 4194304 4623224 140734455479248 18446744073709551615 4330616 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=218044 CPUtime=12.66 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 90103 0 0 0 1253 13 0 0 25 0 4 0 520951879 223277056 21824 33554432000 4194304 4623224 140734455479248 18446744073709551615 4354624 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 50.63
Current children cumulated vsize (KiB) 218044

[startup+25.5008 s]
/proc/loadavg: 8.07 8.10 8.02 9/184 28631
/proc/meminfo: memFree=18061388/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=265148 CPUtime=101.75 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 162556 0 0 0 10149 26 0 0 25 0 4 0 520951876 271511552 33462 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 66287 33462 357 105 0 61791 0
[pid=28596/tid=28598] ppid=28594 vsize=265148 CPUtime=25.46 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 14247 0 0 0 2544 2 0 0 25 0 4 0 520951879 271511552 33462 33554432000 4194304 4623224 140734455479248 18446744073709551615 4366439 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=265148 CPUtime=25.32 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 10052 0 0 0 2531 1 0 0 25 0 4 0 520951879 271511552 33462 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=265148 CPUtime=25.46 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 130004 0 0 0 2526 20 0 0 25 0 4 0 520951879 271511552 33462 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 101.75
Current children cumulated vsize (KiB) 265148

[startup+51.1007 s]
/proc/loadavg: 8.18 8.12 8.03 9/184 28631
/proc/meminfo: memFree=17937604/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=309460 CPUtime=204.02 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 331364 0 0 0 20351 51 0 0 25 0 4 0 520951876 316887040 48627 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510206 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 77365 48627 345 105 0 72869 0
[pid=28596/tid=28598] ppid=28594 vsize=309460 CPUtime=51.06 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 22338 0 0 0 5103 3 0 0 25 0 4 0 520951879 316887040 48627 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376835 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=309460 CPUtime=50.78 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 14605 0 0 0 5076 2 0 0 25 0 4 0 520951879 316887040 48627 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=309460 CPUtime=51.06 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 284146 0 0 0 5063 43 0 0 25 0 4 0 520951879 316887040 48627 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 204.02
Current children cumulated vsize (KiB) 309460

[startup+102.306 s]
/proc/loadavg: 8.20 8.13 8.04 9/184 28633
/proc/meminfo: memFree=17728772/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=475096 CPUtime=408.56 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 644275 0 0 0 40756 100 0 0 25 0 4 0 520951876 486498304 72849 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510090 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 118774 72849 407 105 0 114278 0
[pid=28596/tid=28598] ppid=28594 vsize=475096 CPUtime=102.26 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 33532 0 0 0 10222 4 0 0 25 0 4 0 520951879 486498304 72851 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=475096 CPUtime=101.7 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 23229 0 0 0 10167 3 0 0 25 0 4 0 520951879 486498304 72851 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376678 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=475096 CPUtime=102.27 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 570924 0 0 0 10138 89 0 0 25 0 4 0 520951879 486498304 72851 33554432000 4194304 4623224 140734455479248 18446744073709551615 4577678 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 408.56
Current children cumulated vsize (KiB) 475096

[startup+162.301 s]
/proc/loadavg: 8.19 8.14 8.04 9/184 28634
/proc/meminfo: memFree=17480260/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=623216 CPUtime=648.22 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 907074 0 0 0 64681 141 0 0 25 0 4 0 520951876 638173184 101916 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 155804 101916 867 105 0 151308 0
[pid=28596/tid=28598] ppid=28594 vsize=623216 CPUtime=162.26 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 39295 0 0 0 16221 5 0 0 25 0 4 0 520951879 638173184 101916 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376579 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=623216 CPUtime=161.37 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 32897 0 0 0 16132 5 0 0 25 0 4 0 520951879 638173184 101916 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376666 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=623216 CPUtime=162.26 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 804898 0 0 0 16102 124 0 0 25 0 4 0 520951879 638173184 101916 33554432000 4194304 4623224 140734455479248 18446744073709551615 4364911 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 648.22
Current children cumulated vsize (KiB) 623216

[startup+222.301 s]
/proc/loadavg: 8.13 8.12 8.04 9/184 28636
/proc/meminfo: memFree=17365668/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=683132 CPUtime=887.91 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 1062827 0 0 0 88623 168 0 0 25 0 4 0 520951876 699527168 119136 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510090 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 170783 119136 834 105 0 166287 0
[pid=28596/tid=28598] ppid=28594 vsize=683132 CPUtime=222.27 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 53569 0 0 0 22219 8 0 0 25 0 4 0 520951879 699527168 119136 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=683132 CPUtime=221.04 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 36300 0 0 0 22098 6 0 0 25 0 4 0 520951879 699527168 119136 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=683132 CPUtime=222.26 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 938752 0 0 0 22079 147 0 0 25 0 4 0 520951879 699527168 119136 33554432000 4194304 4623224 140734455479248 18446744073709551615 4366412 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 887.91
Current children cumulated vsize (KiB) 683132

[startup+282.301 s]

################
# More data... #
################

[pid=28596] ppid=28594 vsize=1209812 CPUtime=2805.38 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 2999161 0 0 0 280072 466 0 0 25 0 4 0 520951876 1238847488 238566 33554432000 4194304 4623224 140734455479248 18446744073709551615 4414885 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 302453 238566 593 105 0 297957 0
[pid=28596/tid=28598] ppid=28594 vsize=1209812 CPUtime=702.29 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 91165 0 0 0 70213 16 0 0 25 0 4 0 520951879 1238847488 238566 33554432000 4194304 4623224 140734455479248 18446744073709551615 4377587 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=1209812 CPUtime=698.44 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 80233 0 0 0 69829 15 0 0 25 0 4 0 520951879 1238847488 238566 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=1209812 CPUtime=702.29 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 2709054 0 0 0 69816 413 0 0 25 0 4 0 520951879 1238847488 238566 33554432000 4194304 4623224 140734455479248 18446744073709551615 4364911 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2805.38
Current children cumulated vsize (KiB) 1209812

[startup+762.307 s]
/proc/loadavg: 8.04 8.06 8.03 9/184 28650
/proc/meminfo: memFree=16406588/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=1264924 CPUtime=3045.08 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 3482462 0 0 0 303965 543 0 0 25 0 4 0 520951876 1295282176 248817 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 316231 248817 593 105 0 311735 0
[pid=28596/tid=28598] ppid=28594 vsize=1264924 CPUtime=762.3 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 122180 0 0 0 76210 20 0 0 25 0 4 0 520951879 1295282176 248817 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=1264924 CPUtime=758.13 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 81172 0 0 0 75797 16 0 0 25 0 4 0 520951879 1295282176 248817 33554432000 4194304 4623224 140734455479248 18446744073709551615 4366336 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=1264924 CPUtime=762.3 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 3151400 0 0 0 75747 483 0 0 25 0 4 0 520951879 1295282176 248817 33554432000 4194304 4623224 140734455479248 18446744073709551615 4577493 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3045.08
Current children cumulated vsize (KiB) 1264924

[startup+822.301 s]
/proc/loadavg: 8.20 8.09 8.04 9/184 28652
/proc/meminfo: memFree=16308484/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=1264924 CPUtime=3284.74 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 3922385 0 0 0 327863 611 0 0 25 0 4 0 520951876 1295282176 253594 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 316231 253594 565 105 0 311735 0
[pid=28596/tid=28598] ppid=28594 vsize=1264924 CPUtime=822.3 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 123442 0 0 0 82209 21 0 0 25 0 4 0 520951879 1295282176 253594 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376672 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=1264924 CPUtime=817.8 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 81633 0 0 0 81764 16 0 0 25 0 4 0 520951879 1295282176 253594 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376678 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=1264924 CPUtime=822.3 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 3581730 0 0 0 81682 548 0 0 25 0 4 0 520951879 1295282176 253594 33554432000 4194304 4623224 140734455479248 18446744073709551615 4545376 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3284.74
Current children cumulated vsize (KiB) 1264924

[startup+882.3 s]
/proc/loadavg: 8.21 8.10 8.04 9/184 28653
/proc/meminfo: memFree=16265312/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=1330460 CPUtime=3524.42 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 4401923 0 0 0 351758 684 0 0 25 0 4 0 520951876 1362391040 259488 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 332615 259488 342 105 0 328119 0
[pid=28596/tid=28598] ppid=28594 vsize=1330460 CPUtime=882.3 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 124319 0 0 0 88209 21 0 0 25 0 4 0 520951879 1362391040 259488 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376625 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=1330460 CPUtime=877.46 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 83860 0 0 0 87730 16 0 0 25 0 4 0 520951879 1362391040 259488 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376640 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=1330460 CPUtime=882.29 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 4043692 0 0 0 87613 616 0 0 25 0 4 0 520951879 1362391040 259488 33554432000 4194304 4623224 140734455479248 18446744073709551615 4364911 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3524.42
Current children cumulated vsize (KiB) 1330460

[startup+942.3 s]
/proc/loadavg: 8.14 8.10 8.04 9/184 28655
/proc/meminfo: memFree=16193124/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=1356832 CPUtime=3764.1 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 4753399 0 0 0 375675 735 0 0 25 0 4 0 520951876 1389395968 271142 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 339208 271142 358 105 0 334712 0
[pid=28596/tid=28598] ppid=28594 vsize=1356832 CPUtime=942.3 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 126936 0 0 0 94208 22 0 0 25 0 4 0 520951879 1389395968 271143 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=1356832 CPUtime=937.14 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 86072 0 0 0 93697 17 0 0 25 0 4 0 520951879 1389395968 271147 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376612 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=1356832 CPUtime=942.3 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 4373881 0 0 0 93567 663 0 0 25 0 4 0 520951879 1389395968 271147 33554432000 4194304 4623224 140734455479248 18446744073709551615 4577678 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3764.1
Current children cumulated vsize (KiB) 1356832

[startup+1002.3 s]
/proc/loadavg: 8.11 8.09 8.04 9/184 28657
/proc/meminfo: memFree=16090680/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=1388464 CPUtime=4003.78 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 5235376 0 0 0 399570 808 0 0 25 0 4 0 520951876 1421787136 278954 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 347116 278957 360 105 0 342620 0
[pid=28596/tid=28598] ppid=28594 vsize=1388464 CPUtime=1002.3 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 127545 0 0 0 100208 22 0 0 25 0 4 0 520951879 1421787136 278957 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=1388464 CPUtime=996.82 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 87599 0 0 0 99664 18 0 0 25 0 4 0 520951879 1421787136 278957 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=1388464 CPUtime=1002.3 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 4827393 0 0 0 99501 729 0 0 25 0 4 0 520951879 1421787136 278957 33554432000 4194304 4623224 140734455479248 18446744073709551615 4577476 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4003.78
Current children cumulated vsize (KiB) 1388464

[startup+1062.3 s]
/proc/loadavg: 8.16 8.10 8.04 9/184 28658
/proc/meminfo: memFree=16024440/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=1452512 CPUtime=4243.47 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 5714319 0 0 0 423472 875 0 0 25 0 4 0 520951876 1487372288 289106 33554432000 4194304 4623224 140734455479248 18446744073709551615 4517479 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 363128 289106 342 105 0 358632 0
[pid=28596/tid=28598] ppid=28594 vsize=1452512 CPUtime=1062.3 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 128240 0 0 0 106208 22 0 0 25 0 4 0 520951879 1487372288 289106 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376678 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=1452512 CPUtime=1056.49 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 116638 0 0 0 105627 22 0 0 25 0 4 0 520951879 1487372288 289106 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376612 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=1452512 CPUtime=1062.31 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 5255857 0 0 0 105443 788 0 0 25 0 4 0 520951879 1487372288 289106 33554432000 4194304 4623224 140734455479248 18446744073709551615 4377576 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4243.47
Current children cumulated vsize (KiB) 1452512

[startup+1122.3 s]
/proc/loadavg: 8.17 8.11 8.05 9/184 28660
/proc/meminfo: memFree=15983380/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=1459704 CPUtime=4483.15 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 5983923 0 0 0 447402 913 0 0 25 0 4 0 520951876 1494736896 296632 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 364926 296632 342 105 0 360430 0
[pid=28596/tid=28598] ppid=28594 vsize=1459704 CPUtime=1122.31 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 129182 0 0 0 112208 23 0 0 25 0 4 0 520951879 1494736896 296632 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376612 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=1459704 CPUtime=1116.17 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 116981 0 0 0 111595 22 0 0 25 0 4 0 520951879 1494736896 296632 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376675 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=1459704 CPUtime=1122.31 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 5509867 0 0 0 111408 823 0 0 25 0 4 0 520951879 1494736896 296632 33554432000 4194304 4623224 140734455479248 18446744073709551615 4545702 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4483.15
Current children cumulated vsize (KiB) 1459704

[startup+1182.3 s]
/proc/loadavg: 8.16 8.12 8.05 9/184 28661
/proc/meminfo: memFree=15916648/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=1463956 CPUtime=4722.83 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 6347699 0 0 0 471316 967 0 0 25 0 4 0 520951876 1499090944 307995 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 365989 307995 389 105 0 361493 0
[pid=28596/tid=28598] ppid=28594 vsize=1463956 CPUtime=1182.31 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 131388 0 0 0 118208 23 0 0 25 0 4 0 520951879 1499090944 307995 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=1463956 CPUtime=1175.85 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 117591 0 0 0 117562 23 0 0 25 0 4 0 520951879 1499090944 307995 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376672 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=1463956 CPUtime=1182.31 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 5862748 0 0 0 117356 875 0 0 25 0 4 0 520951879 1499090944 307995 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4722.83
Current children cumulated vsize (KiB) 1463956



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 8.19 8.13 8.06 9/184 28662
/proc/meminfo: memFree=15902756/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=1470068 CPUtime=4800.35 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 6481498 0 0 0 479045 990 0 0 25 0 4 0 520951876 1505349632 308552 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510090 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 367517 308552 343 105 0 363021 0
[pid=28596/tid=28598] ppid=28594 vsize=1470068 CPUtime=1201.71 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 131497 0 0 0 120148 23 0 0 25 0 4 0 520951879 1505349632 308552 33554432000 4194304 4623224 140734455479248 18446744073709551615 4366412 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=1470068 CPUtime=1195.14 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 117700 0 0 0 119491 23 0 0 25 0 4 0 520951879 1505349632 308552 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376625 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=1470068 CPUtime=1201.71 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 5993273 0 0 0 119275 896 0 0 25 0 4 0 520951879 1505349632 308552 33554432000 4194304 4623224 140734455479248 18446744073709551615 4364911 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.35
Current children cumulated vsize (KiB) 1470068

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 8.19 8.13 8.06 9/184 28662
/proc/meminfo: memFree=15902756/32950928 swapFree=67111528/67111528
[pid=28596] ppid=28594 vsize=1470068 CPUtime=4800.35 cores=1,3,5,7
/proc/28596/stat : 28596 (SArTagnan) R 28594 28596 28449 0 -1 4202496 6481498 0 0 0 479045 990 0 0 25 0 4 0 520951876 1505349632 308552 33554432000 4194304 4623224 140734455479248 18446744073709551615 4510090 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/28596/statm: 367517 308552 343 105 0 363021 0
[pid=28596/tid=28598] ppid=28594 vsize=1470068 CPUtime=1201.71 cores=1,3,5,7
/proc/28596/task/28598/stat : 28598 (SArTagnan) R 28594 28596 28449 0 -1 4202560 131497 0 0 0 120148 23 0 0 25 0 4 0 520951879 1505349632 308552 33554432000 4194304 4623224 140734455479248 18446744073709551615 4366412 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28596/tid=28599] ppid=28594 vsize=1470068 CPUtime=1195.14 cores=1,3,5,7
/proc/28596/task/28599/stat : 28599 (SArTagnan) R 28594 28596 28449 0 -1 4202560 117700 0 0 0 119491 23 0 0 25 0 4 0 520951879 1505349632 308552 33554432000 4194304 4623224 140734455479248 18446744073709551615 4376625 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28596/tid=28600] ppid=28594 vsize=1470068 CPUtime=1201.71 cores=1,3,5,7
/proc/28596/task/28600/stat : 28600 (SArTagnan) R 28594 28596 28449 0 -1 4202560 5993273 0 0 0 119275 896 0 0 25 0 4 0 520951879 1505349632 308552 33554432000 4194304 4623224 140734455479248 18446744073709551615 4364911 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.35
Current children cumulated vsize (KiB) 1470068

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.76
CPU time (s): 4800.43
CPU user time (s): 4790.48
CPU system time (s): 9.95349
CPU usage (%): 399.449
Max. virtual memory (cumulated for all children) (KiB): 1552092

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4790.48
system time used= 9.95349
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6481498
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1485
involuntary context switches= 31861

runsolver used 1.82972 second user time and 4.73428 second system time

The end

Launcher Data 

Begin job on node149 at 2011-04-29 18:12:32
IDJOB=3261960
IDBENCH=83266
IDSOLVER=1656
FILE ID=node149/3261960-1304093552
RUNJOBID= node149-1304092347-28468
PBS_JOBID= 13170582
Free space on /tmp= 71932 MiB

SOLVER NAME= SArTagnan 2011-03-24
BENCH NAME= SAT11/crafted/kullmann/VanDerWaerden/VanderWaerden_pd_3k/VanDerWaerden_pd_2-3-24_567.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3261960-1304093552/watcher-3261960-1304093552 -o /tmp/evaluation-result-3261960-1304093552/solver-3261960-1304093552 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3261960-1304093552.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= cb7c9df8144e5ba66f8009983efe791f
RANDOM SEED=1270592388

node149.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      18297696 kB
Buffers:       2225112 kB
Cached:       11803108 kB
SwapCached:          0 kB
Active:        6065728 kB
Inactive:      8037860 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      18297696 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            1964 kB
Writeback:           0 kB
AnonPages:       74820 kB
Mapped:          15480 kB
Slab:           485872 kB
PageTables:       4592 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   252224 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71932 MiB
End job on node149 at 2011-04-29 18:32:36