Trace number 3385934

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 3164.33 1300.06

General information on the benchmark

NameSAT02/handmade/pehoushek/plainoldcnf/
5cnf_4300_4300_110.shuffled.cnf
MD5SUM34f8b747bfe8e7b589b8832e56ae8363
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.022996
Satisfiable
(Un)Satisfiability was proved
Number of variables110
Number of clauses1892
Sum of the clauses size9308
Maximum clause length5
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 33
Number of clauses of size 4146
Number of clauses of size 51743
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385934-1305583433/watcher-3385934-1305583433 -o /tmp/evaluation-result-3385934-1305583433/solver-3385934-1305583433 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3385934-1305583433.cnf -threads=4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 11.57 11.91 10.12 9/469 8390
/proc/meminfo: memFree=21952428/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=49672 CPUtime=0 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 546 0 0 0 0 0 0 0 25 0 4 0 669941297 50864128 483 33554432000 4194304 4625164 140736943350208 18446744073709551615 47259370207148 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/8390/statm: 12418 483 300 106 0 7921 0
[pid=8390/tid=8391] ppid=8388 vsize=49892 CPUtime=0 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 4 0 669941297 51089408 565 33554432000 4194304 4625164 140736943350208 18446744073709551615 47259370207148 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=49892 CPUtime=0 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 4 0 669941297 51089408 565 33554432000 4194304 4625164 140736943350208 18446744073709551615 47259370207154 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=49892 CPUtime=0 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 46 0 0 0 0 0 0 0 25 0 4 0 669941297 51089408 565 33554432000 4194304 4625164 140736943350208 18446744073709551615 4352072 0 0 4096 2 0 0 0 -1 4 0 0 0

[startup+0.083439 s]
/proc/loadavg: 11.57 11.91 10.12 9/469 8390
/proc/meminfo: memFree=21952428/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=116040 CPUtime=0.14 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 987 0 0 0 14 0 0 0 25 0 4 0 669941297 118824960 802 33554432000 4194304 4625164 140736943350208 18446744073709551615 228244046016 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/8390/statm: 29010 814 338 106 0 24513 0
[pid=8390/tid=8391] ppid=8388 vsize=116040 CPUtime=0 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) D 8388 8390 8244 0 -1 4202560 2 0 0 0 0 0 0 0 25 0 4 0 669941297 118824960 814 33554432000 4194304 4625164 140736943350208 18446744073709551615 228244009669 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=116040 CPUtime=0 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) S 8388 8390 8244 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 4 0 669941297 118824960 814 33554432000 4194304 4625164 140736943350208 18446744073709551615 228244453870 0 0 4096 2 18446604453290764160 0 0 -1 4 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=116040 CPUtime=0.06 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 261 0 0 0 6 0 0 0 25 0 4 0 669941297 118824960 814 33554432000 4194304 4625164 140736943350208 18446744073709551615 228244045579 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 0.14
Current children cumulated vsize (KiB) 116040

[startup+0.183952 s]
/proc/loadavg: 11.57 11.91 10.12 9/469 8390
/proc/meminfo: memFree=21952428/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=117316 CPUtime=0.34 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 1531 0 0 0 34 0 0 0 25 0 4 0 669941297 120131584 1226 33554432000 4194304 4625164 140736943350208 18446744073709551615 4512315 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/8390/statm: 29329 1226 340 106 0 24832 0
[pid=8390/tid=8391] ppid=8388 vsize=117316 CPUtime=0.1 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 308 0 0 0 10 0 0 0 25 0 4 0 669941297 120131584 1226 33554432000 4194304 4625164 140736943350208 18446744073709551615 4352112 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=117316 CPUtime=0.07 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 210 0 0 0 7 0 0 0 25 0 4 0 669941297 120131584 1226 33554432000 4194304 4625164 140736943350208 18446744073709551615 4353450 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=117316 CPUtime=0.06 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 265 0 0 0 6 0 0 0 25 0 4 0 669941297 120131584 1226 33554432000 4194304 4625164 140736943350208 18446744073709551615 228244453870 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 0.34
Current children cumulated vsize (KiB) 117316

[startup+0.300932 s]
/proc/loadavg: 11.57 11.91 10.12 9/469 8390
/proc/meminfo: memFree=21952428/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=118192 CPUtime=0.57 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 2130 0 0 0 57 0 0 0 25 0 4 0 669941297 121028608 1499 33554432000 4194304 4625164 140736943350208 18446744073709551615 4512066 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/8390/statm: 29548 1499 340 106 0 25051 0
[pid=8390/tid=8391] ppid=8388 vsize=118192 CPUtime=0.18 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 558 0 0 0 18 0 0 0 25 0 4 0 669941297 121028608 1499 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351958 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=118192 CPUtime=0.1 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 311 0 0 0 10 0 0 0 25 0 4 0 669941297 121028608 1499 33554432000 4194304 4625164 140736943350208 18446744073709551615 4464202 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=118192 CPUtime=0.09 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 386 0 0 0 9 0 0 0 25 0 4 0 669941297 121028608 1499 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351795 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 0.57
Current children cumulated vsize (KiB) 118192

[startup+1.28199 s]
/proc/loadavg: 11.57 11.91 10.12 9/469 8390
/proc/meminfo: memFree=21952428/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=254624 CPUtime=3.04 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 6148 0 0 0 303 1 0 0 25 0 4 0 669941297 260734976 4117 33554432000 4194304 4625164 140736943350208 18446744073709551615 4512156 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/8390/statm: 63656 4117 341 106 0 59159 0
[pid=8390/tid=8391] ppid=8388 vsize=254624 CPUtime=1.1 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 2152 0 0 0 110 0 0 0 19 0 4 0 669941297 260734976 4117 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=254624 CPUtime=0.25 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 828 0 0 0 25 0 0 0 25 0 4 0 669941297 260734976 4117 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351731 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=254624 CPUtime=1.05 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 1831 0 0 0 105 0 0 0 18 0 4 0 669941297 260734976 4117 33554432000 4194304 4625164 140736943350208 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3.04
Current children cumulated vsize (KiB) 254624

[startup+1.50072 s]
/proc/loadavg: 11.57 11.91 10.12 9/469 8390
/proc/meminfo: memFree=21952428/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=254760 CPUtime=3.34 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 6255 0 0 0 333 1 0 0 25 0 4 0 669941297 260874240 4224 33554432000 4194304 4625164 140736943350208 18446744073709551615 4337645 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/8390/statm: 63690 4224 341 106 0 59193 0
[pid=8390/tid=8391] ppid=8388 vsize=254760 CPUtime=1.21 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 2164 0 0 0 121 0 0 0 19 0 4 0 669941297 260874240 4224 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351792 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=254760 CPUtime=0.29 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 891 0 0 0 29 0 0 0 25 0 4 0 669941297 260874240 4224 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=254760 CPUtime=1.11 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 1849 0 0 0 111 0 0 0 19 0 4 0 669941297 260874240 4224 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351918 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3.34
Current children cumulated vsize (KiB) 254760

[startup+3.10044 s]
/proc/loadavg: 11.57 11.91 10.12 13/473 8394
/proc/meminfo: memFree=21902728/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=258088 CPUtime=6.56 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 10399 0 0 0 654 2 0 0 25 0 4 0 669941297 264282112 6374 33554432000 4194304 4625164 140736943350208 18446744073709551615 4339447 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/8390/statm: 64522 6374 341 106 0 60025 0
[pid=8390/tid=8391] ppid=8388 vsize=258088 CPUtime=2.12 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 3984 0 0 0 212 0 0 0 20 0 4 0 669941297 264282112 6374 33554432000 4194304 4625164 140736943350208 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=258088 CPUtime=0.98 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 2135 0 0 0 97 1 0 0 25 0 4 0 669941297 264282112 6374 33554432000 4194304 4625164 140736943350208 18446744073709551615 228244046221 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=258088 CPUtime=2.02 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 2674 0 0 0 202 0 0 0 20 0 4 0 669941297 264282112 6374 33554432000 4194304 4625164 140736943350208 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 6.56
Current children cumulated vsize (KiB) 258088

[startup+6.30088 s]
/proc/loadavg: 11.60 11.91 10.13 13/473 8394
/proc/meminfo: memFree=21871936/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=264884 CPUtime=12.92 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 14456 0 0 0 1288 4 0 0 25 0 4 0 669941297 271241216 9178 33554432000 4194304 4625164 140736943350208 18446744073709551615 228244004821 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/8390/statm: 66221 9178 347 106 0 61724 0
[pid=8390/tid=8391] ppid=8388 vsize=264884 CPUtime=3.7 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 4767 0 0 0 369 1 0 0 25 0 4 0 669941297 271241216 9178 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=264884 CPUtime=2.52 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 3646 0 0 0 251 1 0 0 25 0 4 0 669941297 271241216 9178 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351818 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=264884 CPUtime=3.69 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 3833 0 0 0 368 1 0 0 25 0 4 0 669941297 271241216 9178 33554432000 4194304 4625164 140736943350208 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 12.92
Current children cumulated vsize (KiB) 264884

[startup+12.7008 s]
/proc/loadavg: 11.63 11.91 10.14 13/473 8395
/proc/meminfo: memFree=21827440/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=273392 CPUtime=25.63 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 21391 0 0 0 2557 6 0 0 25 0 4 0 669941297 279953408 13073 33554432000 4194304 4625164 140736943350208 18446744073709551615 4396736 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/8390/statm: 68348 13073 356 106 0 63851 0
[pid=8390/tid=8391] ppid=8388 vsize=273392 CPUtime=6.8 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 7065 0 0 0 678 2 0 0 25 0 4 0 669941297 279953408 13073 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=273392 CPUtime=5.59 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 5983 0 0 0 558 1 0 0 25 0 4 0 669941297 279953408 13073 33554432000 4194304 4625164 140736943350208 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=273392 CPUtime=6.86 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 5560 0 0 0 685 1 0 0 25 0 4 0 669941297 279953408 13073 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 25.63
Current children cumulated vsize (KiB) 273392

[startup+25.5005 s]
/proc/loadavg: 11.79 11.93 10.18 13/473 8397
/proc/meminfo: memFree=21766920/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=282464 CPUtime=51.01 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 30688 0 0 0 5093 8 0 0 25 0 4 0 669941297 289243136 19190 33554432000 4194304 4625164 140736943350208 18446744073709551615 4512854 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/8390/statm: 70616 19190 398 106 0 66119 0
[pid=8390/tid=8391] ppid=8388 vsize=282464 CPUtime=13.1 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 10844 0 0 0 1307 3 0 0 25 0 4 0 669941297 289243136 19190 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=282464 CPUtime=11.99 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 9625 0 0 0 1196 3 0 0 25 0 4 0 669941297 289243136 19190 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351827 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=282464 CPUtime=13.16 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 6591 0 0 0 1315 1 0 0 25 0 4 0 669941297 289243136 19190 33554432000 4194304 4625164 140736943350208 18446744073709551615 4579312 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 51.01
Current children cumulated vsize (KiB) 282464

[startup+51.101 s]
/proc/loadavg: 12.29 12.03 10.26 14/474 8399
/proc/meminfo: memFree=21645456/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=306644 CPUtime=101.76 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 48725 0 0 0 10164 12 0 0 25 0 4 0 669941297 314003456 27692 33554432000 4194304 4625164 140736943350208 18446744073709551615 4329151 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/8390/statm: 76661 27692 459 106 0 72164 0
[pid=8390/tid=8391] ppid=8388 vsize=306644 CPUtime=25.56 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 15590 0 0 0 2551 5 0 0 25 0 4 0 669941297 314003456 27692 33554432000 4194304 4625164 140736943350208 18446744073709551615 228244010017 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=306644 CPUtime=24.68 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) D 8388 8390 8244 0 -1 4202560 17740 0 0 0 2464 4 0 0 25 0 4 0 669941297 314003456 27692 33554432000 4194304 4625164 140736943350208 18446744073709551615 228244392119 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=306644 CPUtime=25.78 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 9847 0 0 0 2576 2 0 0 25 0 4 0 669941297 314003456 27692 33554432000 4194304 4625164 140736943350208 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.76
Current children cumulated vsize (KiB) 306644

[startup+102.301 s]
/proc/loadavg: 12.33 12.08 10.37 13/473 8400
/proc/meminfo: memFree=21554748/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=331772 CPUtime=203.34 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 89100 0 0 0 20313 21 0 0 25 0 4 0 669941297 339734528 39195 33554432000 4194304 4625164 140736943350208 18446744073709551615 4512046 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/8390/statm: 82943 39195 473 106 0 78446 0
[pid=8390/tid=8391] ppid=8388 vsize=331772 CPUtime=51.05 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 22358 0 0 0 5099 6 0 0 25 0 4 0 669941297 339734528 39195 33554432000 4194304 4625164 140736943350208 18446744073709551615 4339559 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=331772 CPUtime=49.7 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 27650 0 0 0 4962 8 0 0 25 0 4 0 669941297 339734528 39195 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=331772 CPUtime=51.18 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 31236 0 0 0 5113 5 0 0 25 0 4 0 669941297 339734528 39195 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 203.34
Current children cumulated vsize (KiB) 331772

[startup+162.301 s]

################
# More data... #
################

[pid=8390] ppid=8388 vsize=728656 CPUtime=1809.18 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 862487 0 0 0 180763 155 0 0 25 0 4 0 669941297 746143744 128857 33554432000 4194304 4625164 140736943350208 18446744073709551615 4512869 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/8390/statm: 182164 128857 1549 106 0 177667 0
[pid=8390/tid=8391] ppid=8388 vsize=728656 CPUtime=379.33 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 57964 0 0 0 37912 21 0 0 25 0 4 0 669941297 746143744 128857 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=728656 CPUtime=389.27 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 67210 0 0 0 38905 22 0 0 25 0 4 0 669941297 746143744 128857 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=728656 CPUtime=520.91 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 700157 0 0 0 51985 106 0 0 25 0 4 0 669941297 746143744 128857 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1809.18
Current children cumulated vsize (KiB) 728656

[startup+822.301 s]
/proc/loadavg: 12.07 12.13 11.32 13/473 8419
/proc/meminfo: memFree=20553020/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=728656 CPUtime=1958.25 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 865087 0 0 0 195668 157 0 0 25 0 4 0 669941297 746143744 131373 33554432000 4194304 4625164 140736943350208 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/8390/statm: 182164 131373 1486 106 0 177667 0
[pid=8390/tid=8391] ppid=8388 vsize=728656 CPUtime=409.35 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 59140 0 0 0 40912 23 0 0 25 0 4 0 669941297 746143744 131373 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=728656 CPUtime=419.29 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 67934 0 0 0 41906 23 0 0 25 0 4 0 669941297 746143744 131373 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=728656 CPUtime=550.91 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 700556 0 0 0 54985 106 0 0 25 0 4 0 669941297 746143744 131373 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1958.25
Current children cumulated vsize (KiB) 728656

[startup+882.3 s]
/proc/loadavg: 12.12 12.14 11.37 12/473 8421
/proc/meminfo: memFree=20501776/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=728656 CPUtime=2107.3 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 867362 0 0 0 210571 159 0 0 25 0 4 0 669941297 746143744 133588 33554432000 4194304 4625164 140736943350208 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/8390/statm: 182164 133588 1449 106 0 177667 0
[pid=8390/tid=8391] ppid=8388 vsize=728656 CPUtime=439.34 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 60126 0 0 0 43910 24 0 0 25 0 4 0 669941297 746143744 133588 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=728656 CPUtime=449.27 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 68906 0 0 0 44904 23 0 0 25 0 4 0 669941297 746143744 133588 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=728656 CPUtime=580.9 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 700758 0 0 0 57984 106 0 0 25 0 4 0 669941297 746143744 133588 33554432000 4194304 4625164 140736943350208 18446744073709551615 4547335 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2107.3
Current children cumulated vsize (KiB) 728656

[startup+942.318 s]
/proc/loadavg: 12.14 12.14 11.41 13/473 8422
/proc/meminfo: memFree=20405396/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=799932 CPUtime=2256.4 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 896939 0 0 0 225477 163 0 0 25 0 4 0 669941297 819130368 150207 33554432000 4194304 4625164 140736943350208 18446744073709551615 4512029 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/8390/statm: 199983 150207 1442 106 0 195486 0
[pid=8390/tid=8391] ppid=8388 vsize=799932 CPUtime=469.36 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 80860 0 0 0 46910 26 0 0 25 0 4 0 669941297 819130368 150207 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=799932 CPUtime=479.3 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 69734 0 0 0 47906 24 0 0 25 0 4 0 669941297 819130368 150207 33554432000 4194304 4625164 140736943350208 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=799932 CPUtime=610.9 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 700876 0 0 0 60984 106 0 0 25 0 4 0 669941297 819130368 150207 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351827 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2256.4
Current children cumulated vsize (KiB) 799932

[startup+1002.3 s]
/proc/loadavg: 12.21 12.16 11.46 13/473 8424
/proc/meminfo: memFree=20351048/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=799932 CPUtime=2405.42 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 906702 0 0 0 240377 165 0 0 25 0 4 0 669941297 819130368 154786 33554432000 4194304 4625164 140736943350208 18446744073709551615 4512854 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/8390/statm: 199983 154786 1443 106 0 195486 0
[pid=8390/tid=8391] ppid=8388 vsize=799932 CPUtime=499.35 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 81876 0 0 0 49908 27 0 0 25 0 4 0 669941297 819130368 154786 33554432000 4194304 4625164 140736943350208 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=799932 CPUtime=509.29 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 71974 0 0 0 50904 25 0 0 25 0 4 0 669941297 819130368 154786 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=799932 CPUtime=640.89 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 706941 0 0 0 63982 107 0 0 25 0 4 0 669941297 819130368 154786 33554432000 4194304 4625164 140736943350208 18446744073709551615 4335165 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2405.42
Current children cumulated vsize (KiB) 799932

[startup+1062.3 s]
/proc/loadavg: 12.20 12.16 11.50 13/473 8426
/proc/meminfo: memFree=20272032/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=799932 CPUtime=2554.49 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 937667 0 0 0 255280 169 0 0 25 0 4 0 669941297 819130368 156318 33554432000 4194304 4625164 140736943350208 18446744073709551615 4280391 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/8390/statm: 199983 156318 1442 106 0 195486 0
[pid=8390/tid=8391] ppid=8388 vsize=799932 CPUtime=529.36 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 82145 0 0 0 52909 27 0 0 25 0 4 0 669941297 819130368 156318 33554432000 4194304 4625164 140736943350208 18446744073709551615 4339643 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=799932 CPUtime=539.06 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 72291 0 0 0 53881 25 0 0 25 0 4 0 669941297 819130368 156318 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=799932 CPUtime=681.97 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 737211 0 0 0 68086 111 0 0 25 0 4 0 669941297 819130368 156318 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2554.49
Current children cumulated vsize (KiB) 799932

[startup+1122.3 s]
/proc/loadavg: 12.30 12.19 11.56 13/473 8427
/proc/meminfo: memFree=20223400/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=836068 CPUtime=2703.56 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 1129160 0 0 0 270160 196 0 0 25 0 4 0 669941297 856133632 159295 33554432000 4194304 4625164 140736943350208 18446744073709551615 4329151 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/8390/statm: 209017 159295 1442 106 0 204520 0
[pid=8390/tid=8391] ppid=8388 vsize=836068 CPUtime=559.36 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 82900 0 0 0 55908 28 0 0 25 0 4 0 669941297 856133632 159295 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=836068 CPUtime=568.61 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 91917 0 0 0 56833 28 0 0 25 0 4 0 669941297 856133632 159295 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=836068 CPUtime=741.95 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 908145 0 0 0 74061 134 0 0 25 0 4 0 669941297 856133632 159295 33554432000 4194304 4625164 140736943350208 18446744073709551615 4352002 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2703.56
Current children cumulated vsize (KiB) 836068

[startup+1182.3 s]
/proc/loadavg: 12.15 12.17 11.58 13/473 8429
/proc/meminfo: memFree=20170048/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=836068 CPUtime=2852.61 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 1393269 0 0 0 285028 233 0 0 25 0 4 0 669941297 856133632 162704 33554432000 4194304 4625164 140736943350208 18446744073709551615 4329151 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/8390/statm: 209017 162704 1529 106 0 204520 0
[pid=8390/tid=8391] ppid=8388 vsize=836068 CPUtime=589.37 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 83672 0 0 0 58909 28 0 0 25 0 4 0 669941297 856133632 162704 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=836068 CPUtime=598.12 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 92271 0 0 0 59784 28 0 0 25 0 4 0 669941297 856133632 162704 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=836068 CPUtime=801.94 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 1170757 0 0 0 80023 171 0 0 25 0 4 0 669941297 856133632 162704 33554432000 4194304 4625164 140736943350208 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2852.61
Current children cumulated vsize (KiB) 836068

[startup+1242.3 s]
/proc/loadavg: 12.16 12.16 11.62 13/473 8430
/proc/meminfo: memFree=20125884/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=836068 CPUtime=3025.8 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 1714183 0 0 0 302301 279 0 0 25 0 4 0 669941297 856133632 165221 33554432000 4194304 4625164 140736943350208 18446744073709551615 4519319 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/8390/statm: 209017 165242 1475 106 0 204520 0
[pid=8390/tid=8391] ppid=8388 vsize=836068 CPUtime=619.38 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 84731 0 0 0 61909 29 0 0 25 0 4 0 669941297 856133632 165252 33554432000 4194304 4625164 140736943350208 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=836068 CPUtime=651.39 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 93466 0 0 0 65111 28 0 0 25 0 4 0 669941297 856133632 165264 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=836068 CPUtime=861.93 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 1488769 0 0 0 85978 215 0 0 25 0 4 0 669941297 856133632 165275 33554432000 4194304 4625164 140736943350208 18446744073709551615 228244046273 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3025.8
Current children cumulated vsize (KiB) 836068



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 12.16 12.16 11.65 9/468 8433
/proc/meminfo: memFree=20990416/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=836068 CPUtime=3164.25 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 1880531 0 0 0 316123 302 0 0 25 0 4 0 669941297 856133632 167271 33554432000 4194304 4625164 140736943350208 18446744073709551615 4511967 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/8390/statm: 209017 167271 1442 106 0 204520 0
[pid=8390/tid=8391] ppid=8388 vsize=836068 CPUtime=647.69 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 85606 0 0 0 64740 29 0 0 25 0 4 0 669941297 856133632 167278 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=836068 CPUtime=686.99 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 93766 0 0 0 68671 28 0 0 25 0 4 0 669941297 856133632 167278 33554432000 4194304 4625164 140736943350208 18446744073709551615 4352010 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=836068 CPUtime=896.64 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 1653648 0 0 0 89427 237 0 0 25 0 4 0 669941297 856133632 167278 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3164.25
Current children cumulated vsize (KiB) 836068

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1300 s]
/proc/loadavg: 12.16 12.16 11.65 9/468 8433
/proc/meminfo: memFree=20990416/32951124 swapFree=64205392/67111528
[pid=8390] ppid=8388 vsize=836068 CPUtime=3164.25 cores=0,2,4,6
/proc/8390/stat : 8390 (SArTagnan) R 8388 8390 8244 0 -1 4202496 1880531 0 0 0 316123 302 0 0 25 0 4 0 669941297 856133632 167271 33554432000 4194304 4625164 140736943350208 18446744073709551615 4511967 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/8390/statm: 209017 167271 1442 106 0 204520 0
[pid=8390/tid=8391] ppid=8388 vsize=836068 CPUtime=647.69 cores=0,2,4,6
/proc/8390/task/8391/stat : 8391 (SArTagnan) R 8388 8390 8244 0 -1 4202560 85606 0 0 0 64740 29 0 0 25 0 4 0 669941297 856133632 167278 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8392] ppid=8388 vsize=836068 CPUtime=686.99 cores=0,2,4,6
/proc/8390/task/8392/stat : 8392 (SArTagnan) R 8388 8390 8244 0 -1 4202560 93766 0 0 0 68671 28 0 0 25 0 4 0 669941297 856133632 167278 33554432000 4194304 4625164 140736943350208 18446744073709551615 4352010 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=8390/tid=8393] ppid=8388 vsize=836068 CPUtime=896.64 cores=0,2,4,6
/proc/8390/task/8393/stat : 8393 (SArTagnan) R 8388 8390 8244 0 -1 4202560 1653648 0 0 0 89427 237 0 0 25 0 4 0 669941297 856133632 167278 33554432000 4194304 4625164 140736943350208 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3164.25
Current children cumulated vsize (KiB) 836068

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1300.06
CPU time (s): 3164.33
CPU user time (s): 3161.28
CPU system time (s): 3.05453
CPU usage (%): 243.4
Max. virtual memory (cumulated for all children) (KiB): 836068

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3161.28
system time used= 3.05454
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1880541
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 666
involuntary context switches= 38137

runsolver used 5.70813 second user time and 14.4018 second system time

The end

Launcher Data 

Begin job on node136 at 2011-05-17 00:03:53
IDJOB=3385934
IDBENCH=87455
IDSOLVER=1871
FILE ID=node136/3385934-1305583433
RUNJOBID= node136-1305582130-8262
PBS_JOBID= 13350293
Free space on /tmp= 72952 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT02/handmade/pehoushek/plainoldcnf/5cnf_4300_4300_110.shuffled.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385934-1305583433/watcher-3385934-1305583433 -o /tmp/evaluation-result-3385934-1305583433/solver-3385934-1305583433 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3385934-1305583433.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 34f8b747bfe8e7b589b8832e56ae8363
RANDOM SEED=1666624095

node136.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21952452 kB
Buffers:         50936 kB
Cached:         675776 kB
SwapCached:      24924 kB
Active:        4251616 kB
Inactive:      6531208 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21952452 kB
SwapTotal:    67111528 kB
SwapFree:     64205392 kB
Dirty:             784 kB
Writeback:           0 kB
AnonPages:    10036812 kB
Mapped:          34888 kB
Slab:            87168 kB
PageTables:      62800 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 135630908 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72952 MiB
End job on node136 at 2011-05-17 00:25:35