Trace number 3386021

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 4800.44 1203.82

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m32-q326.25-n30-p30-s48700942.cnf
MD5SUM1bb1eb2a3bb0b22f560ab6712b727e98
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.385941
Satisfiable
(Un)Satisfiability was proved
Number of variables943
Number of clauses93431
Sum of the clauses size187745
Maximum clause length32
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 293401
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 530

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386021-1305583644/watcher-3386021-1305583644 -o /tmp/evaluation-result-3386021-1305583644/solver-3386021-1305583644 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3386021-1305583644.cnf -threads=4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.86 7.90 7.82 1/370 6033
/proc/meminfo: memFree=27871848/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=21212 CPUtime=0 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 759 0 0 0 0 0 0 0 22 0 1 0 669964850 21721088 522 33554432000 4194304 4625164 140733440712832 18446744073709551615 4339447 0 0 4096 2 0 0 0 17 2 0 0 0
/proc/6033/statm: 5303 522 296 106 0 806 0

[startup+0.0745201 s]
/proc/loadavg: 6.86 7.90 7.82 1/370 6033
/proc/meminfo: memFree=27871848/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=255972 CPUtime=0.2 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 3824 0 0 0 20 0 0 0 24 0 4 0 669964850 262115328 2935 33554432000 4194304 4625164 140733440712832 18446744073709551615 4499479 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 63993 2935 345 106 0 59496 0
Current children cumulated CPU time (s) 0.2
Current children cumulated vsize (KiB) 255972

[startup+0.100516 s]
/proc/loadavg: 6.86 7.90 7.82 1/370 6033
/proc/meminfo: memFree=27871848/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=257708 CPUtime=0.31 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 4319 0 0 0 31 0 0 0 24 0 4 0 669964850 263892992 3031 33554432000 4194304 4625164 140733440712832 18446744073709551615 4393061 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 64427 3031 345 106 0 59930 0
Current children cumulated CPU time (s) 0.31
Current children cumulated vsize (KiB) 257708

[startup+0.30048 s]
/proc/loadavg: 6.86 7.90 7.82 1/370 6033
/proc/meminfo: memFree=27871848/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=259992 CPUtime=1.11 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 6594 0 0 0 110 1 0 0 25 0 4 0 669964850 266231808 4934 33554432000 4194304 4625164 140733440712832 18446744073709551615 4511874 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 64998 4936 345 106 0 60501 0
Current children cumulated CPU time (s) 1.11
Current children cumulated vsize (KiB) 259992

[startup+0.700408 s]
/proc/loadavg: 6.86 7.90 7.82 1/370 6033
/proc/meminfo: memFree=27871848/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=265552 CPUtime=2.71 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 9972 0 0 0 270 1 0 0 25 0 4 0 669964850 271925248 7589 33554432000 4194304 4625164 140733440712832 18446744073709551615 4509158 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 66388 7589 352 106 0 61891 0
Current children cumulated CPU time (s) 2.71
Current children cumulated vsize (KiB) 265552

[startup+1.50026 s]
/proc/loadavg: 6.86 7.90 7.82 10/380 6046
/proc/meminfo: memFree=27807176/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=272504 CPUtime=5.89 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 13222 0 0 0 588 1 0 0 25 0 4 0 669964850 279044096 9982 33554432000 4194304 4625164 140733440712832 18446744073709551615 4521612 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 68126 9982 345 106 0 63629 0
[pid=6033/tid=6035] ppid=6031 vsize=272504 CPUtime=1.45 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 2709 0 0 0 145 0 0 0 25 0 4 0 669964853 279044096 9982 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=272504 CPUtime=1.47 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 2823 0 0 0 147 0 0 0 25 0 4 0 669964853 279044096 9982 33554432000 4194304 4625164 140733440712832 18446744073709551615 4352672 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=272504 CPUtime=1.47 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 1751 0 0 0 147 0 0 0 25 0 4 0 669964853 279044096 9982 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351719 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 5.89
Current children cumulated vsize (KiB) 272504

[startup+3.10097 s]
/proc/loadavg: 6.86 7.90 7.82 9/380 6046
/proc/meminfo: memFree=27777092/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=281312 CPUtime=12.28 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 24456 0 0 0 1225 3 0 0 25 0 4 0 669964850 288063488 14568 33554432000 4194304 4625164 140733440712832 18446744073709551615 4511986 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 70328 14568 346 106 0 65831 0
[pid=6033/tid=6035] ppid=6031 vsize=281312 CPUtime=3.03 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 4842 0 0 0 303 0 0 0 25 0 4 0 669964853 288063488 14568 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=281312 CPUtime=3.07 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 4803 0 0 0 306 1 0 0 25 0 4 0 669964853 288063488 14568 33554432000 4194304 4625164 140733440712832 18446744073709551615 4345240 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=281312 CPUtime=3.06 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 7212 0 0 0 306 0 0 0 25 0 4 0 669964853 288063488 14568 33554432000 4194304 4625164 140733440712832 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 12.28
Current children cumulated vsize (KiB) 281312

[startup+6.30038 s]
/proc/loadavg: 6.95 7.90 7.82 9/380 6046
/proc/meminfo: memFree=27717328/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=297828 CPUtime=25.04 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 46191 0 0 0 2497 7 0 0 25 0 4 0 669964850 304975872 20402 33554432000 4194304 4625164 140733440712832 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 74457 20402 347 106 0 69960 0
[pid=6033/tid=6035] ppid=6031 vsize=297828 CPUtime=6.2 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 7966 0 0 0 619 1 0 0 25 0 4 0 669964853 304975872 20402 33554432000 4194304 4625164 140733440712832 18446744073709551615 4352010 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=297828 CPUtime=6.26 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 8960 0 0 0 625 1 0 0 25 0 4 0 669964853 304975872 20402 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=297828 CPUtime=6.27 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 19987 0 0 0 625 2 0 0 25 0 4 0 669964853 304975872 20402 33554432000 4194304 4625164 140733440712832 18446744073709551615 4547248 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 25.04
Current children cumulated vsize (KiB) 297828

[startup+12.7002 s]
/proc/loadavg: 7.12 7.92 7.83 9/380 6046
/proc/meminfo: memFree=27617380/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=330144 CPUtime=50.56 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 89789 0 0 0 5041 15 0 0 25 0 4 0 669964850 338067456 32092 33554432000 4194304 4625164 140733440712832 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 82536 32092 351 106 0 78039 0
[pid=6033/tid=6035] ppid=6031 vsize=330144 CPUtime=12.52 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 15531 0 0 0 1249 3 0 0 25 0 4 0 669964853 338067456 32092 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=330144 CPUtime=12.67 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 15649 0 0 0 1264 3 0 0 25 0 4 0 669964853 338067456 32092 33554432000 4194304 4625164 140733440712832 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=330144 CPUtime=12.66 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 45757 0 0 0 1260 6 0 0 25 0 4 0 669964853 338067456 32092 33554432000 4194304 4625164 140733440712832 18446744073709551615 4334258 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 50.56
Current children cumulated vsize (KiB) 330144

[startup+25.5009 s]
/proc/loadavg: 7.31 7.92 7.83 9/380 6046
/proc/meminfo: memFree=27507152/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=349724 CPUtime=101.61 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 190844 0 0 0 10131 30 0 0 25 0 4 0 669964850 358117376 44811 33554432000 4194304 4625164 140733440712832 18446744073709551615 4513499 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 87431 44811 351 106 0 82934 0
[pid=6033/tid=6035] ppid=6031 vsize=349724 CPUtime=25.17 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 23328 0 0 0 2511 6 0 0 25 0 4 0 669964853 358117376 44811 33554432000 4194304 4625164 140733440712832 18446744073709551615 4352647 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=349724 CPUtime=25.46 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 21673 0 0 0 2542 4 0 0 25 0 4 0 669964853 358117376 44811 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=349724 CPUtime=25.47 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 131361 0 0 0 2530 17 0 0 25 0 4 0 669964853 358117376 44811 33554432000 4194304 4625164 140733440712832 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 101.61
Current children cumulated vsize (KiB) 349724

[startup+51.1012 s]
/proc/loadavg: 7.55 7.93 7.83 9/380 6047
/proc/meminfo: memFree=27314688/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=384544 CPUtime=203.71 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 278899 0 0 0 20325 46 0 0 25 0 4 0 669964850 393773056 64506 33554432000 4194304 4625164 140733440712832 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 96136 64506 412 106 0 91639 0
[pid=6033/tid=6035] ppid=6031 vsize=384544 CPUtime=50.47 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 35068 0 0 0 5037 10 0 0 25 0 4 0 669964853 393773056 64506 33554432000 4194304 4625164 140733440712832 18446744073709551615 4353239 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=384544 CPUtime=51.07 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 34025 0 0 0 5100 7 0 0 25 0 4 0 669964853 393773056 64506 33554432000 4194304 4625164 140733440712832 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=384544 CPUtime=51.05 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 190471 0 0 0 5081 24 0 0 25 0 4 0 669964853 393773056 64506 33554432000 4194304 4625164 140733440712832 18446744073709551615 4547335 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 203.71
Current children cumulated vsize (KiB) 384544

[startup+102.315 s]
/proc/loadavg: 7.90 7.97 7.85 9/380 6048
/proc/meminfo: memFree=26996864/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=582208 CPUtime=407.94 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 396594 0 0 0 40729 65 0 0 25 0 4 0 669964850 596180992 103094 33554432000 4194304 4625164 140733440712832 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 145552 103094 449 106 0 141055 0
[pid=6033/tid=6035] ppid=6031 vsize=582208 CPUtime=101.07 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 49026 0 0 0 10093 14 0 0 25 0 4 0 669964853 596180992 103094 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=582208 CPUtime=102.28 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 47283 0 0 0 10218 10 0 0 25 0 4 0 669964853 596180992 103094 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=582208 CPUtime=102.26 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 245074 0 0 0 10194 32 0 0 25 0 4 0 669964853 596180992 103094 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 407.94
Current children cumulated vsize (KiB) 582208

[startup+162.301 s]
/proc/loadavg: 7.96 7.97 7.85 9/380 6050
/proc/meminfo: memFree=26677548/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=841556 CPUtime=647.18 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 804966 0 0 0 64597 121 0 0 25 0 4 0 669964850 861753344 142396 33554432000 4194304 4625164 140733440712832 18446744073709551615 4512763 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 210389 142396 481 106 0 205892 0
[pid=6033/tid=6035] ppid=6031 vsize=841556 CPUtime=160.35 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 69694 0 0 0 16016 19 0 0 25 0 4 0 669964853 861753344 142397 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=841556 CPUtime=162.27 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 65416 0 0 0 16214 13 0 0 25 0 4 0 669964853 861753344 142397 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=841556 CPUtime=162.23 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 561558 0 0 0 16151 72 0 0 25 0 4 0 669964853 861753344 142397 33554432000 4194304 4625164 140733440712832 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 647.18
Current children cumulated vsize (KiB) 841556

[startup+222.301 s]
/proc/loadavg: 8.02 7.99 7.86 9/380 6051
/proc/meminfo: memFree=26390712/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=868212 CPUtime=886.46 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 1302331 0 0 0 88454 192 0 0 25 0 4 0 669964850 889049088 170683 33554432000 4194304 4625164 140733440712832 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 217053 170684 504 106 0 212556 0
[pid=6033/tid=6035] ppid=6031 vsize=868212 CPUtime=219.64 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 74694 0 0 0 21941 23 0 0 25 0 4 0 669964853 889049088 170684 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351860 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=868212 CPUtime=222.27 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 71812 0 0 0 22211 16 0 0 25 0 4 0 669964853 889049088 170685 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=868212 CPUtime=222.22 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 1035766 0 0 0 22090 132 0 0 25 0 4 0 669964853 889049088 170685 33554432000 4194304 4625164 140733440712832 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 886.46
Current children cumulated vsize (KiB) 868212

[startup+282.301 s]

################
# More data... #
################

[pid=6033] ppid=6031 vsize=2035344 CPUtime=2800.7 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 5406006 0 0 0 279284 786 0 0 25 0 4 0 669964850 2084192256 400986 33554432000 4194304 4625164 140733440712832 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 508836 400986 346 106 0 504339 0
[pid=6033/tid=6035] ppid=6031 vsize=2035344 CPUtime=693.98 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 164028 0 0 0 69347 51 0 0 25 0 4 0 669964853 2084192256 400986 33554432000 4194304 4625164 140733440712832 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=2035344 CPUtime=702.28 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 158083 0 0 0 70192 36 0 0 25 0 4 0 669964853 2084192256 400986 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=2035344 CPUtime=702.1 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 4531815 0 0 0 69598 612 0 0 25 0 4 0 669964853 2084192256 400986 33554432000 4194304 4625164 140733440712832 18446744073709551615 4328206 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2800.7
Current children cumulated vsize (KiB) 2035344

[startup+762.314 s]
/proc/loadavg: 8.08 8.09 7.97 9/380 6066
/proc/meminfo: memFree=24618836/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=2259660 CPUtime=3040.04 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 5572396 0 0 0 303195 809 0 0 25 0 4 0 669964850 2313891840 420801 33554432000 4194304 4625164 140733440712832 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 564915 420801 346 106 0 560418 0
[pid=6033/tid=6035] ppid=6031 vsize=2259660 CPUtime=753.28 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 164061 0 0 0 75277 51 0 0 25 0 4 0 669964853 2313891840 420801 33554432000 4194304 4625164 140733440712832 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=2259660 CPUtime=762.3 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 159881 0 0 0 76192 38 0 0 25 0 4 0 669964853 2313891840 420801 33554432000 4194304 4625164 140733440712832 18446744073709551615 4352280 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=2259660 CPUtime=762.11 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 4532908 0 0 0 75598 613 0 0 25 0 4 0 669964853 2313891840 420801 33554432000 4194304 4625164 140733440712832 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3040.04
Current children cumulated vsize (KiB) 2259660

[startup+822.301 s]
/proc/loadavg: 8.14 8.10 7.98 9/380 6067
/proc/meminfo: memFree=24537856/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=2265116 CPUtime=3279.27 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 5792391 0 0 0 327081 846 0 0 25 0 4 0 669964850 2319478784 431951 33554432000 4194304 4625164 140733440712832 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 566279 431951 347 106 0 561782 0
[pid=6033/tid=6035] ppid=6031 vsize=2265116 CPUtime=812.57 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 164157 0 0 0 81205 52 0 0 25 0 4 0 669964853 2319478784 431951 33554432000 4194304 4625164 140733440712832 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=2265116 CPUtime=822.28 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 162918 0 0 0 82187 41 0 0 25 0 4 0 669964853 2319478784 431951 33554432000 4194304 4625164 140733440712832 18446744073709551615 4352105 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=2265116 CPUtime=822.08 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 4740569 0 0 0 81564 644 0 0 25 0 4 0 669964853 2319478784 431951 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3279.27
Current children cumulated vsize (KiB) 2265116

[startup+882.301 s]
/proc/loadavg: 8.14 8.11 7.99 9/380 6069
/proc/meminfo: memFree=24455996/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=2265116 CPUtime=3518.55 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 6214924 0 0 0 350944 911 0 0 25 0 4 0 669964850 2319478784 439526 33554432000 4194304 4625164 140733440712832 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 566279 439526 346 106 0 561782 0
[pid=6033/tid=6035] ppid=6031 vsize=2265116 CPUtime=871.86 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 164521 0 0 0 87134 52 0 0 25 0 4 0 669964853 2319478784 439526 33554432000 4194304 4625164 140733440712832 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=2265116 CPUtime=882.29 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 166467 0 0 0 88184 45 0 0 25 0 4 0 669964853 2319478784 439526 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=2265116 CPUtime=882.07 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 5156659 0 0 0 87501 706 0 0 25 0 4 0 669964853 2319478784 439526 33554432000 4194304 4625164 140733440712832 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3518.55
Current children cumulated vsize (KiB) 2265116

[startup+942.301 s]
/proc/loadavg: 8.14 8.12 8.00 9/380 6071
/proc/meminfo: memFree=24261420/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=2330652 CPUtime=3757.84 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 6635922 0 0 0 374812 972 0 0 25 0 4 0 669964850 2386587648 463938 33554432000 4194304 4625164 140733440712832 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 582663 463958 346 106 0 578166 0
[pid=6033/tid=6035] ppid=6031 vsize=2330652 CPUtime=931.16 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 164785 0 0 0 93064 52 0 0 25 0 4 0 669964853 2386587648 463968 33554432000 4194304 4625164 140733440712832 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=2330652 CPUtime=942.29 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 168991 0 0 0 94182 47 0 0 25 0 4 0 669964853 2386587648 463980 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=2330652 CPUtime=942.06 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 5553275 0 0 0 93447 759 0 0 25 0 4 0 669964853 2386587648 463994 33554432000 4194304 4625164 140733440712832 18446744073709551615 230808863179 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3757.84
Current children cumulated vsize (KiB) 2330652

[startup+1002.3 s]
/proc/loadavg: 8.24 8.15 8.01 9/380 6072
/proc/meminfo: memFree=24040180/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=2396188 CPUtime=3997.11 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 7082944 0 0 0 398671 1040 0 0 25 0 4 0 669964850 2453696512 509611 33554432000 4194304 4625164 140733440712832 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 599047 509611 346 106 0 594550 0
[pid=6033/tid=6035] ppid=6031 vsize=2396188 CPUtime=990.45 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 164872 0 0 0 98993 52 0 0 25 0 4 0 669964853 2453696512 509611 33554432000 4194304 4625164 140733440712832 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=2396188 CPUtime=1002.29 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 171346 0 0 0 100181 48 0 0 25 0 4 0 669964853 2453696512 509611 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=2396188 CPUtime=1002.04 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 5938156 0 0 0 99390 814 0 0 25 0 4 0 669964853 2453696512 509611 33554432000 4194304 4625164 140733440712832 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3997.11
Current children cumulated vsize (KiB) 2396188

[startup+1062.3 s]
/proc/loadavg: 8.18 8.15 8.02 9/380 6074
/proc/meminfo: memFree=23823792/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=2995660 CPUtime=4236.38 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 7780945 0 0 0 422494 1144 0 0 25 0 4 0 669964850 3067555840 554504 33554432000 4194304 4625164 140733440712832 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 748915 554524 346 106 0 744418 0
[pid=6033/tid=6035] ppid=6031 vsize=2995660 CPUtime=1049.74 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 164937 0 0 0 104922 52 0 0 25 0 4 0 669964853 3067555840 554534 33554432000 4194304 4625164 140733440712832 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=2995660 CPUtime=1062.27 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 221699 0 0 0 106171 56 0 0 25 0 4 0 669964853 3067555840 554544 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=2995660 CPUtime=1062.01 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 6347035 0 0 0 105322 879 0 0 25 0 4 0 669964853 3067555840 554557 33554432000 4194304 4625164 140733440712832 18446744073709551615 230808863107 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4236.38
Current children cumulated vsize (KiB) 2995660

[startup+1122.3 s]
/proc/loadavg: 8.10 8.14 8.02 9/380 6075
/proc/meminfo: memFree=23700388/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=2995660 CPUtime=4475.65 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 8288465 0 0 0 446344 1221 0 0 25 0 4 0 669964850 3067555840 576260 33554432000 4194304 4625164 140733440712832 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 748915 576261 346 106 0 744418 0
[pid=6033/tid=6035] ppid=6031 vsize=2995660 CPUtime=1109.03 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 165157 0 0 0 110851 52 0 0 25 0 4 0 669964853 3067555840 576261 33554432000 4194304 4625164 140733440712832 18446744073709551615 4488020 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=2995660 CPUtime=1122.28 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 225604 0 0 0 112168 60 0 0 25 0 4 0 669964853 3067555840 576261 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=2995660 CPUtime=1122 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 6805291 0 0 0 111256 944 0 0 25 0 4 0 669964853 3067555840 576261 33554432000 4194304 4625164 140733440712832 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4475.65
Current children cumulated vsize (KiB) 2995660

[startup+1182.3 s]
/proc/loadavg: 8.10 8.14 8.02 9/380 6077
/proc/meminfo: memFree=23596584/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=3032412 CPUtime=4714.94 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 8730746 0 0 0 470207 1287 0 0 25 0 4 0 669964850 3105189888 598562 33554432000 4194304 4625164 140733440712832 18446744073709551615 4393051 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 758103 598562 346 106 0 753606 0
[pid=6033/tid=6035] ppid=6031 vsize=3032412 CPUtime=1168.33 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 165445 0 0 0 116781 52 0 0 25 0 4 0 669964853 3105189888 598562 33554432000 4194304 4625164 140733440712832 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=3032412 CPUtime=1182.28 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 228208 0 0 0 118166 62 0 0 25 0 4 0 669964853 3105189888 598562 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=3032412 CPUtime=1181.98 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 7210813 0 0 0 117199 999 0 0 25 0 4 0 669964853 3105189888 598562 33554432000 4194304 4625164 140733440712832 18446744073709551615 4352280 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4714.94
Current children cumulated vsize (KiB) 3032412



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.7 s]
/proc/loadavg: 8.07 8.12 8.02 9/380 6078
/proc/meminfo: memFree=23577360/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=3032412 CPUtime=4800.31 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 8905795 0 0 0 478719 1312 0 0 25 0 4 0 669964850 3105189888 601262 33554432000 4194304 4625164 140733440712832 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 758103 601262 346 106 0 753606 0
[pid=6033/tid=6035] ppid=6031 vsize=3032412 CPUtime=1189.47 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 165447 0 0 0 118895 52 0 0 25 0 4 0 669964853 3105189888 601262 33554432000 4194304 4625164 140733440712832 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=3032412 CPUtime=1203.7 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 229062 0 0 0 120307 63 0 0 25 0 4 0 669964853 3105189888 601262 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=3032412 CPUtime=1203.39 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 7371788 0 0 0 119318 1021 0 0 25 0 4 0 669964853 3105189888 601262 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.31
Current children cumulated vsize (KiB) 3032412

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.7 s]
/proc/loadavg: 8.07 8.12 8.02 9/380 6078
/proc/meminfo: memFree=23577360/32951124 swapFree=37160408/67111528
[pid=6033] ppid=6031 vsize=3032412 CPUtime=4800.31 cores=0,2,4,6
/proc/6033/stat : 6033 (SArTagnan) R 6031 6033 5006 0 -1 4202496 8905795 0 0 0 478719 1312 0 0 25 0 4 0 669964850 3105189888 601262 33554432000 4194304 4625164 140733440712832 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/6033/statm: 758103 601262 346 106 0 753606 0
[pid=6033/tid=6035] ppid=6031 vsize=3032412 CPUtime=1189.47 cores=0,2,4,6
/proc/6033/task/6035/stat : 6035 (SArTagnan) R 6031 6033 5006 0 -1 4202560 165447 0 0 0 118895 52 0 0 25 0 4 0 669964853 3105189888 601262 33554432000 4194304 4625164 140733440712832 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=6033/tid=6036] ppid=6031 vsize=3032412 CPUtime=1203.7 cores=0,2,4,6
/proc/6033/task/6036/stat : 6036 (SArTagnan) R 6031 6033 5006 0 -1 4202560 229062 0 0 0 120307 63 0 0 25 0 4 0 669964853 3105189888 601262 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=6033/tid=6037] ppid=6031 vsize=3032412 CPUtime=1203.39 cores=0,2,4,6
/proc/6033/task/6037/stat : 6037 (SArTagnan) R 6031 6033 5006 0 -1 4202560 7371788 0 0 0 119318 1021 0 0 25 0 4 0 669964853 3105189888 601262 33554432000 4194304 4625164 140733440712832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.31
Current children cumulated vsize (KiB) 3032412

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.82
CPU time (s): 4800.44
CPU user time (s): 4787.22
CPU system time (s): 13.22
CPU usage (%): 398.769
Max. virtual memory (cumulated for all children) (KiB): 3528096

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4787.22
system time used= 13.22
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 8905795
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2413
involuntary context switches= 22724

runsolver used 4.00039 second user time and 10.2574 second system time

The end

Launcher Data 

Begin job on node104 at 2011-05-17 00:07:24
IDJOB=3386021
IDBENCH=88214
IDSOLVER=1871
FILE ID=node104/3386021-1305583644
RUNJOBID= node104-1305583644-6019
PBS_JOBID= 13350212
Free space on /tmp= 73272 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m32-q326.25-n30-p30-s48700942.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386021-1305583644/watcher-3386021-1305583644 -o /tmp/evaluation-result-3386021-1305583644/solver-3386021-1305583644 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3386021-1305583644.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 1bb1eb2a3bb0b22f560ab6712b727e98
RANDOM SEED=501810637

node104.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5293.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      27872136 kB
Buffers:        302484 kB
Cached:        2720704 kB
SwapCached:     152312 kB
Active:        1258092 kB
Inactive:      3539180 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      27872136 kB
SwapTotal:    67111528 kB
SwapFree:     37160408 kB
Dirty:            1692 kB
Writeback:           0 kB
AnonPages:     1634772 kB
Mapped:          10408 kB
Slab:           127848 kB
PageTables:      89076 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 87701824 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73268 MiB
End job on node104 at 2011-05-17 00:27:30