Trace number 3386119

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 4800.23 1205.05

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n160-s40283905-sat.cnf
MD5SUM8e3c134eb643f2d9a45c884774c054a8
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.19497
Satisfiable
(Un)Satisfiability was proved
Number of variables160
Number of clauses384
Sum of the clauses size960
Maximum clause length5
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2320
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 564
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386119-1305580328/watcher-3386119-1305580328 -o /tmp/evaluation-result-3386119-1305580328/solver-3386119-1305580328 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3386119-1305580328.cnf -threads=4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.44 5.26 6.83 2/474 10972
/proc/meminfo: memFree=14781340/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=49212 CPUtime=0.01 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 443 0 0 0 1 0 0 0 18 0 4 0 669629451 50393088 380 33554432000 4194304 4625164 140733451895344 18446744073709551615 47017691473842 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 12303 380 301 106 0 7806 0
[pid=10972/tid=10973] ppid=10969 vsize=49212 CPUtime=0 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 0 0 0 0 0 0 0 0 19 0 4 0 669629451 50393088 380 33554432000 4194304 4625164 140733451895344 18446744073709551615 254500748273 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=49212 CPUtime=0.01 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 1 0 0 0 1 0 0 0 19 0 4 0 669629451 50393088 380 33554432000 4194304 4625164 140733451895344 18446744073709551615 47017691473842 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=49212 CPUtime=0.01 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 1 0 0 0 1 0 0 0 19 0 4 0 669629451 50393088 380 33554432000 4194304 4625164 140733451895344 18446744073709551615 47017691473842 0 0 4096 2 0 0 0 -1 2 0 0 0

[startup+0.12164 s]
/proc/loadavg: 3.44 5.26 6.83 2/474 10972
/proc/meminfo: memFree=14781340/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=247836 CPUtime=0.45 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 1860 0 0 0 45 0 0 0 19 0 4 0 669629451 253784064 1325 33554432000 4194304 4625164 140733451895344 18446744073709551615 254500382686 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 61959 1325 345 106 0 57462 0
[pid=10972/tid=10973] ppid=10969 vsize=247836 CPUtime=0.1 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 396 0 0 0 10 0 0 0 20 0 4 0 669629451 253784064 1325 33554432000 4194304 4625164 140733451895344 18446744073709551615 4339536 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=247836 CPUtime=0.11 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 408 0 0 0 11 0 0 0 20 0 4 0 669629451 253784064 1325 33554432000 4194304 4625164 140733451895344 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=247836 CPUtime=0.11 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 346 0 0 0 11 0 0 0 20 0 4 0 669629451 253784064 1325 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 0.45
Current children cumulated vsize (KiB) 247836

[startup+0.221621 s]
/proc/loadavg: 3.44 5.26 6.83 2/474 10972
/proc/meminfo: memFree=14781340/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=248772 CPUtime=0.85 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 2794 0 0 0 85 0 0 0 19 0 4 0 669629451 254742528 1912 33554432000 4194304 4625164 140733451895344 18446744073709551615 4356399 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 62193 1912 345 106 0 57696 0
[pid=10972/tid=10973] ppid=10969 vsize=248772 CPUtime=0.2 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 705 0 0 0 20 0 0 0 20 0 4 0 669629451 254742528 1912 33554432000 4194304 4625164 140733451895344 18446744073709551615 4353584 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=248772 CPUtime=0.21 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 685 0 0 0 21 0 0 0 20 0 4 0 669629451 254742528 1912 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=248772 CPUtime=0.21 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 575 0 0 0 21 0 0 0 20 0 4 0 669629451 254742528 1912 33554432000 4194304 4625164 140733451895344 18446744073709551615 4335407 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 0.85
Current children cumulated vsize (KiB) 248772

[startup+0.321602 s]
/proc/loadavg: 3.44 5.26 6.83 2/474 10972
/proc/meminfo: memFree=14781340/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=250336 CPUtime=1.24 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 4006 0 0 0 124 0 0 0 19 0 4 0 669629451 256344064 2420 33554432000 4194304 4625164 140733451895344 18446744073709551615 4426786 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 62584 2420 345 106 0 58087 0
[pid=10972/tid=10973] ppid=10969 vsize=250336 CPUtime=0.3 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 1077 0 0 0 30 0 0 0 20 0 4 0 669629451 256344064 2420 33554432000 4194304 4625164 140733451895344 18446744073709551615 4339643 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=250336 CPUtime=0.31 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 1011 0 0 0 31 0 0 0 20 0 4 0 669629451 256344064 2420 33554432000 4194304 4625164 140733451895344 18446744073709551615 254500352709 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=250336 CPUtime=0.31 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 988 0 0 0 31 0 0 0 20 0 4 0 669629451 256344064 2420 33554432000 4194304 4625164 140733451895344 18446744073709551615 254516675611 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1.24
Current children cumulated vsize (KiB) 250336

[startup+0.700528 s]
/proc/loadavg: 3.44 5.26 6.83 2/474 10972
/proc/meminfo: memFree=14781340/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=252568 CPUtime=2.76 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 7412 0 0 0 276 0 0 0 19 0 4 0 669629451 258629632 3650 33554432000 4194304 4625164 140733451895344 18446744073709551615 4512156 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 63142 3650 346 106 0 58645 0
[pid=10972/tid=10973] ppid=10969 vsize=252568 CPUtime=0.68 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 1743 0 0 0 68 0 0 0 21 0 4 0 669629451 258629632 3650 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=252568 CPUtime=0.69 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 1762 0 0 0 69 0 0 0 21 0 4 0 669629451 258629632 3650 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=252568 CPUtime=0.68 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 2780 0 0 0 68 0 0 0 21 0 4 0 669629451 258629632 3650 33554432000 4194304 4625164 140733451895344 18446744073709551615 4552336 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2.76
Current children cumulated vsize (KiB) 252568

[startup+1.50037 s]
/proc/loadavg: 3.44 5.26 6.83 9/485 10984
/proc/meminfo: memFree=14739364/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=259740 CPUtime=5.94 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 16614 0 0 0 593 1 0 0 21 0 4 0 669629451 265973760 5733 33554432000 4194304 4625164 140733451895344 18446744073709551615 254500350824 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 64935 5733 346 106 0 60438 0
[pid=10972/tid=10973] ppid=10969 vsize=259740 CPUtime=1.45 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 3598 0 0 0 145 0 0 0 23 0 4 0 669629451 265973760 5733 33554432000 4194304 4625164 140733451895344 18446744073709551615 4288690 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=259740 CPUtime=1.49 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 2540 0 0 0 149 0 0 0 23 0 4 0 669629451 265973760 5733 33554432000 4194304 4625164 140733451895344 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=259740 CPUtime=1.48 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 8836 0 0 0 148 0 0 0 23 0 4 0 669629451 265973760 5733 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351798 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 5.94
Current children cumulated vsize (KiB) 259740

[startup+3.10105 s]
/proc/loadavg: 3.44 5.26 6.83 9/485 10984
/proc/meminfo: memFree=14715804/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=263992 CPUtime=12.32 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 33481 0 0 0 1229 3 0 0 25 0 4 0 669629451 270327808 8451 33554432000 4194304 4625164 140733451895344 18446744073709551615 4511973 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 65998 8451 350 106 0 61501 0
[pid=10972/tid=10973] ppid=10969 vsize=263992 CPUtime=3.04 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 4421 0 0 0 304 0 0 0 25 0 4 0 669629451 270327808 8451 33554432000 4194304 4625164 140733451895344 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=263992 CPUtime=3.09 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 3874 0 0 0 309 0 0 0 25 0 4 0 669629451 270327808 8451 33554432000 4194304 4625164 140733451895344 18446744073709551615 4464301 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=263992 CPUtime=3.08 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 22957 0 0 0 306 2 0 0 25 0 4 0 669629451 270327808 8451 33554432000 4194304 4625164 140733451895344 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 12.32
Current children cumulated vsize (KiB) 263992

[startup+6.30042 s]
/proc/loadavg: 3.81 5.30 6.83 9/485 10984
/proc/meminfo: memFree=14673392/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=275176 CPUtime=25.07 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 69410 0 0 0 2497 10 0 0 25 0 4 0 669629451 281780224 12967 33554432000 4194304 4625164 140733451895344 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 68794 12967 367 106 0 64297 0
[pid=10972/tid=10973] ppid=10969 vsize=275176 CPUtime=6.18 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 8835 0 0 0 617 1 0 0 25 0 4 0 669629451 281780224 12967 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=275176 CPUtime=6.28 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 7151 0 0 0 628 0 0 0 25 0 4 0 669629451 281780224 12967 33554432000 4194304 4625164 140733451895344 18446744073709551615 4352112 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=275176 CPUtime=6.29 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 50215 0 0 0 622 7 0 0 25 0 4 0 669629451 281780224 12967 33554432000 4194304 4625164 140733451895344 18446744073709551615 4546721 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 25.07
Current children cumulated vsize (KiB) 275176

[startup+12.7011 s]
/proc/loadavg: 4.14 5.35 6.84 9/485 10984
/proc/meminfo: memFree=14610900/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=289108 CPUtime=50.56 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 101072 0 0 0 5042 14 0 0 25 0 4 0 669629451 296046592 18346 33554432000 4194304 4625164 140733451895344 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 72277 18346 390 106 0 67780 0
[pid=10972/tid=10973] ppid=10969 vsize=289108 CPUtime=12.47 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 13180 0 0 0 1245 2 0 0 25 0 4 0 669629451 296046592 18346 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351925 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=289108 CPUtime=12.68 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 10467 0 0 0 1268 0 0 0 25 0 4 0 669629451 296046592 18346 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351844 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=289108 CPUtime=12.69 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 72804 0 0 0 1259 10 0 0 25 0 4 0 669629451 296046592 18346 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 50.56
Current children cumulated vsize (KiB) 289108

[startup+25.5006 s]
/proc/loadavg: 5.06 5.49 6.86 9/485 10984
/proc/meminfo: memFree=14533028/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=297300 CPUtime=101.55 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 168536 0 0 0 10130 25 0 0 25 0 4 0 669629451 304435200 25513 33554432000 4194304 4625164 140733451895344 18446744073709551615 4512750 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 74325 25513 393 106 0 69828 0
[pid=10972/tid=10973] ppid=10969 vsize=297300 CPUtime=25.06 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 19902 0 0 0 2502 4 0 0 25 0 4 0 669629451 304435200 25513 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=297300 CPUtime=25.49 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 11239 0 0 0 2548 1 0 0 25 0 4 0 669629451 304435200 25513 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=297300 CPUtime=25.48 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 132383 0 0 0 2530 18 0 0 25 0 4 0 669629451 304435200 25513 33554432000 4194304 4625164 140733451895344 18446744073709551615 4579367 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 101.55
Current children cumulated vsize (KiB) 297300

[startup+51.1005 s]
/proc/loadavg: 6.06 5.69 6.89 9/485 10985
/proc/meminfo: memFree=14416332/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=330460 CPUtime=203.53 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 231776 0 0 0 20320 33 0 0 25 0 4 0 669629451 338391040 38588 33554432000 4194304 4625164 140733451895344 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 82615 38588 386 106 0 78118 0
[pid=10972/tid=10973] ppid=10969 vsize=330460 CPUtime=50.24 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 28292 0 0 0 5019 5 0 0 25 0 4 0 669629451 338391040 38588 33554432000 4194304 4625164 140733451895344 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=330460 CPUtime=51.08 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 17255 0 0 0 5106 2 0 0 25 0 4 0 669629451 338391040 38588 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351989 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=330460 CPUtime=51.08 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 174659 0 0 0 5085 23 0 0 25 0 4 0 669629451 338391040 38588 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351884 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 203.53
Current children cumulated vsize (KiB) 330460

[startup+102.3 s]
/proc/loadavg: 7.16 6.05 6.95 9/485 10986
/proc/meminfo: memFree=14229788/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=435120 CPUtime=407.52 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 302674 0 0 0 40708 44 0 0 25 0 4 0 669629451 445562880 61999 33554432000 4194304 4625164 140733451895344 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 108780 61999 1031 106 0 104283 0
[pid=10972/tid=10973] ppid=10969 vsize=435120 CPUtime=100.59 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 41357 0 0 0 10053 6 0 0 25 0 4 0 669629451 445562880 61999 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=435120 CPUtime=102.3 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 25469 0 0 0 10225 5 0 0 25 0 4 0 669629451 445562880 61999 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=435120 CPUtime=102.31 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 213333 0 0 0 10202 29 0 0 25 0 4 0 669629451 445562880 61999 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 407.52
Current children cumulated vsize (KiB) 435120

[startup+162.301 s]

################
# More data... #
################

[pid=10972] ppid=10969 vsize=731268 CPUtime=2797.62 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 1099155 0 0 0 279581 181 0 0 25 0 4 0 669629451 748818432 138119 33554432000 4194304 4625164 140733451895344 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 182817 138119 427 106 0 178320 0
[pid=10972/tid=10973] ppid=10969 vsize=731268 CPUtime=690.66 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 103705 0 0 0 69042 24 0 0 25 0 4 0 669629451 748818432 138119 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=731268 CPUtime=702.31 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 55157 0 0 0 70218 13 0 0 25 0 4 0 669629451 748818432 138119 33554432000 4194304 4625164 140733451895344 18446744073709551615 4339628 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=731268 CPUtime=702.3 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 601089 0 0 0 70146 84 0 0 25 0 4 0 669629451 748818432 138119 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2797.62
Current children cumulated vsize (KiB) 731268

[startup+762.3 s]
/proc/loadavg: 8.12 7.89 7.53 9/485 11004
/proc/meminfo: memFree=13207348/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=744724 CPUtime=3036.65 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 1170520 0 0 0 303472 193 0 0 25 0 4 0 669629451 762597376 142833 33554432000 4194304 4625164 140733451895344 18446744073709551615 4512156 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 186181 142833 427 106 0 181684 0
[pid=10972/tid=10973] ppid=10969 vsize=744724 CPUtime=749.66 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 104870 0 0 0 74941 25 0 0 25 0 4 0 669629451 762597376 142833 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=744724 CPUtime=762.32 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 55381 0 0 0 76219 13 0 0 25 0 4 0 669629451 762597376 142833 33554432000 4194304 4625164 140733451895344 18446744073709551615 4352735 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=744724 CPUtime=762.32 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 603760 0 0 0 76147 85 0 0 25 0 4 0 669629451 762597376 142833 33554432000 4194304 4625164 140733451895344 18446744073709551615 4352765 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3036.65
Current children cumulated vsize (KiB) 744724

[startup+822.301 s]
/proc/loadavg: 8.20 7.95 7.57 9/485 11006
/proc/meminfo: memFree=13139852/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=813136 CPUtime=3275.64 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 1174103 0 0 0 327369 195 0 0 25 0 4 0 669629451 832651264 146320 33554432000 4194304 4625164 140733451895344 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 203284 146320 427 106 0 198787 0
[pid=10972/tid=10973] ppid=10969 vsize=813136 CPUtime=808.67 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 105617 0 0 0 80841 26 0 0 25 0 4 0 669629451 832651264 146320 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=813136 CPUtime=822.3 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 55433 0 0 0 82217 13 0 0 25 0 4 0 669629451 832651264 146320 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=813136 CPUtime=822.3 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 605210 0 0 0 82145 85 0 0 25 0 4 0 669629451 832651264 146320 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3275.64
Current children cumulated vsize (KiB) 813136

[startup+882.319 s]
/proc/loadavg: 8.17 7.99 7.61 9/485 11007
/proc/meminfo: memFree=13092064/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=838556 CPUtime=3514.71 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 1189806 0 0 0 351273 198 0 0 25 0 4 0 669629451 858681344 151520 33554432000 4194304 4625164 140733451895344 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 209639 151520 427 106 0 205142 0
[pid=10972/tid=10973] ppid=10969 vsize=838556 CPUtime=867.69 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 107089 0 0 0 86741 28 0 0 25 0 4 0 669629451 858681344 151520 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=838556 CPUtime=882.32 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 55696 0 0 0 88219 13 0 0 25 0 4 0 669629451 858681344 151520 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=838556 CPUtime=882.31 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 606664 0 0 0 88146 85 0 0 25 0 4 0 669629451 858681344 151520 33554432000 4194304 4625164 140733451895344 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3514.71
Current children cumulated vsize (KiB) 838556

[startup+942.301 s]
/proc/loadavg: 8.10 8.00 7.64 9/485 11009
/proc/meminfo: memFree=13014780/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=889412 CPUtime=3753.66 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 1383479 0 0 0 375138 228 0 0 25 0 4 0 669629451 910757888 162592 33554432000 4194304 4625164 140733451895344 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 222353 162592 459 106 0 217856 0
[pid=10972/tid=10973] ppid=10969 vsize=889412 CPUtime=926.69 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 114667 0 0 0 92639 30 0 0 25 0 4 0 669629451 910757888 162592 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=889412 CPUtime=942.31 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 56331 0 0 0 94217 14 0 0 25 0 4 0 669629451 910757888 162592 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=889412 CPUtime=942.3 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 791287 0 0 0 94118 112 0 0 25 0 4 0 669629451 910757888 162594 33554432000 4194304 4625164 140733451895344 18446744073709551615 4579598 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3753.66
Current children cumulated vsize (KiB) 889412

[startup+1002.3 s]
/proc/loadavg: 8.13 8.03 7.67 9/485 11010
/proc/meminfo: memFree=12956960/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=970156 CPUtime=3992.66 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 1829919 0 0 0 398973 293 0 0 25 0 4 0 669629451 993439744 168761 33554432000 4194304 4625164 140733451895344 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 242539 168761 460 106 0 238042 0
[pid=10972/tid=10973] ppid=10969 vsize=970156 CPUtime=985.69 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 147915 0 0 0 98535 34 0 0 25 0 4 0 669629451 993439744 168761 33554432000 4194304 4625164 140733451895344 18446744073709551615 4352005 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=970156 CPUtime=1002.31 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 56385 0 0 0 100217 14 0 0 25 0 4 0 669629451 993439744 168761 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351906 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=970156 CPUtime=1002.3 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 1192591 0 0 0 100059 171 0 0 25 0 4 0 669629451 993439744 168761 33554432000 4194304 4625164 140733451895344 18446744073709551615 4579396 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3992.66
Current children cumulated vsize (KiB) 970156

[startup+1062.3 s]
/proc/loadavg: 8.14 8.05 7.69 9/485 11012
/proc/meminfo: memFree=12921712/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=970156 CPUtime=4231.68 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 2104254 0 0 0 422833 335 0 0 25 0 4 0 669629451 993439744 169420 33554432000 4194304 4625164 140733451895344 18446744073709551615 4511870 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 242539 169420 388 106 0 238042 0
[pid=10972/tid=10973] ppid=10969 vsize=970156 CPUtime=1044.7 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 148226 0 0 0 104436 34 0 0 25 0 4 0 669629451 993439744 169420 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=970156 CPUtime=1062.31 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 56458 0 0 0 106217 14 0 0 25 0 4 0 669629451 993439744 169420 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351909 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=970156 CPUtime=1062.3 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 1466150 0 0 0 106017 213 0 0 25 0 4 0 669629451 993439744 169420 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351878 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4231.68
Current children cumulated vsize (KiB) 970156

[startup+1122.3 s]
/proc/loadavg: 8.13 8.07 7.72 9/485 11014
/proc/meminfo: memFree=12858420/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=970156 CPUtime=4470.68 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 2122735 0 0 0 446730 338 0 0 25 0 4 0 669629451 993439744 172188 33554432000 4194304 4625164 140733451895344 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 242539 172188 388 106 0 238042 0
[pid=10972/tid=10973] ppid=10969 vsize=970156 CPUtime=1103.7 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 149489 0 0 0 110335 35 0 0 25 0 4 0 669629451 993439744 172188 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=970156 CPUtime=1122.32 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 56467 0 0 0 112218 14 0 0 25 0 4 0 669629451 993439744 172188 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=970156 CPUtime=1122.3 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 1482899 0 0 0 112015 215 0 0 25 0 4 0 669629451 993439744 172188 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4470.68
Current children cumulated vsize (KiB) 970156

[startup+1182.3 s]
/proc/loadavg: 8.12 8.08 7.75 9/485 11015
/proc/meminfo: memFree=12821800/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=970156 CPUtime=4709.71 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 2224206 0 0 0 470617 354 0 0 25 0 4 0 669629451 993439744 175899 33554432000 4194304 4625164 140733451895344 18446744073709551615 4519319 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 242539 175899 388 106 0 238042 0
[pid=10972/tid=10973] ppid=10969 vsize=970156 CPUtime=1162.71 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 152559 0 0 0 116236 35 0 0 25 0 4 0 669629451 993439744 175899 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=970156 CPUtime=1182.32 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 56467 0 0 0 118217 15 0 0 25 0 4 0 669629451 993439744 175899 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=970156 CPUtime=1182.3 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 1580342 0 0 0 118001 229 0 0 25 0 4 0 669629451 993439744 175899 33554432000 4194304 4625164 140733451895344 18446744073709551615 4335407 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4709.71
Current children cumulated vsize (KiB) 970156



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1205 s]
/proc/loadavg: 8.20 8.11 7.76 5/481 11017
/proc/meminfo: memFree=14066776/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=970156 CPUtime=4800.15 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 2272914 0 0 0 479654 361 0 0 25 0 4 0 669629451 993439744 176622 33554432000 4194304 4625164 140733451895344 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 242539 176622 388 106 0 238042 0
[pid=10972/tid=10973] ppid=10969 vsize=970156 CPUtime=1185.03 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 153077 0 0 0 118467 36 0 0 25 0 4 0 669629451 993439744 176622 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=970156 CPUtime=1205.04 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 56467 0 0 0 120489 15 0 0 25 0 4 0 669629451 993439744 176622 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=970156 CPUtime=1205.02 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 1628312 0 0 0 120266 236 0 0 25 0 4 0 669629451 993439744 176622 33554432000 4194304 4625164 140733451895344 18446744073709551615 4547283 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.15
Current children cumulated vsize (KiB) 970156

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1205 s]
/proc/loadavg: 8.20 8.11 7.76 5/481 11017
/proc/meminfo: memFree=14066776/32950928 swapFree=45358800/67111528
[pid=10972] ppid=10969 vsize=970156 CPUtime=4800.15 cores=0,2,4,6
/proc/10972/stat : 10972 (SArTagnan) R 10969 10972 10402 0 -1 4202496 2272914 0 0 0 479654 361 0 0 25 0 4 0 669629451 993439744 176622 33554432000 4194304 4625164 140733451895344 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/10972/statm: 242539 176622 388 106 0 238042 0
[pid=10972/tid=10973] ppid=10969 vsize=970156 CPUtime=1185.03 cores=0,2,4,6
/proc/10972/task/10973/stat : 10973 (SArTagnan) R 10969 10972 10402 0 -1 4202560 153077 0 0 0 118467 36 0 0 25 0 4 0 669629451 993439744 176622 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=10972/tid=10974] ppid=10969 vsize=970156 CPUtime=1205.04 cores=0,2,4,6
/proc/10972/task/10974/stat : 10974 (SArTagnan) R 10969 10972 10402 0 -1 4202560 56467 0 0 0 120489 15 0 0 25 0 4 0 669629451 993439744 176622 33554432000 4194304 4625164 140733451895344 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=10972/tid=10975] ppid=10969 vsize=970156 CPUtime=1205.02 cores=0,2,4,6
/proc/10972/task/10975/stat : 10975 (SArTagnan) R 10969 10972 10402 0 -1 4202560 1628312 0 0 0 120266 236 0 0 25 0 4 0 669629451 993439744 176622 33554432000 4194304 4625164 140733451895344 18446744073709551615 4547283 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.15
Current children cumulated vsize (KiB) 970156

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1205.05
CPU time (s): 4800.23
CPU user time (s): 4796.59
CPU system time (s): 3.64645
CPU usage (%): 398.343
Max. virtual memory (cumulated for all children) (KiB): 1087540

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.59
system time used= 3.64644
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2272914
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 367
involuntary context switches= 26462

runsolver used 5.32019 second user time and 14.4118 second system time

The end

Launcher Data 

Begin job on node149 at 2011-05-16 23:12:08
IDJOB=3386119
IDBENCH=83138
IDSOLVER=1871
FILE ID=node149/3386119-1305580328
RUNJOBID= node149-1305580327-10957
PBS_JOBID= 13350208
Free space on /tmp= 73536 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n160-s40283905-sat.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386119-1305580328/watcher-3386119-1305580328 -o /tmp/evaluation-result-3386119-1305580328/solver-3386119-1305580328 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3386119-1305580328.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 8e3c134eb643f2d9a45c884774c054a8
RANDOM SEED=1829406007

node149.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      14781644 kB
Buffers:         87204 kB
Cached:         560836 kB
SwapCached:      36208 kB
Active:        8731112 kB
Inactive:      9161108 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      14781644 kB
SwapTotal:    67111528 kB
SwapFree:     45358800 kB
Dirty:             528 kB
Writeback:           0 kB
AnonPages:    17220824 kB
Mapped:          26508 kB
Slab:            97384 kB
PageTables:     112236 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 156256064 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73532 MiB
End job on node149 at 2011-05-16 23:32:15