Trace number 3438691

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 40000.7 5010.82

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n260-s62321009-sat.cnf
MD5SUMf2b430bdd9bc1c472e9e437567706bb7
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark17.0544
Satisfiable
(Un)Satisfiability was proved
Number of variables260
Number of clauses624
Sum of the clauses size1560
Maximum clause length5
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2520
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 5104
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3438691-1306921897/watcher-3438691-1306921897 -o /tmp/evaluation-result-3438691-1306921897/solver-3438691-1306921897 -C 40000 -W 5100 -M 31000 SArTagnan HOME/instance-3438691-1306921897.cnf -threads=8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.16 8.07 8.01 1/481 22764
/proc/meminfo: memFree=21940180/32951124 swapFree=46575304/67111528
[pid=22764] ppid=22762 vsize=90456 CPUtime=0.02 cores=0-7
/proc/22764/stat : 22764 (SArTagnan) R 22762 22764 22496 0 -1 4202496 503 0 0 0 2 0 0 0 25 0 8 0 536380916 92626944 439 33554432000 4194304 4625164 140736764711776 18446744073709551615 47571637471148 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22764/statm: 22614 439 300 106 0 18117 0
[pid=22764/tid=22765] ppid=22762 vsize=90456 CPUtime=0.01 cores=0-7
/proc/22764/task/22765/stat : 22765 (SArTagnan) R 22762 22764 22496 0 -1 4202560 1 0 0 0 1 0 0 0 25 0 8 0 536380916 92626944 439 33554432000 4194304 4625164 140736764711776 18446744073709551615 47571637471154 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22764/tid=22766] ppid=22762 vsize=90456 CPUtime=0 cores=0-7
/proc/22764/task/22766/stat : 22766 (SArTagnan) R 22762 22764 22496 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 8 0 536380916 92626944 439 33554432000 4194304 4625164 140736764711776 18446744073709551615 268660718577 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22764/tid=22767] ppid=22762 vsize=90456 CPUtime=0 cores=0-7
/proc/22764/task/22767/stat : 22767 (SArTagnan) R 22762 22764 22496 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 8 0 536380916 92626944 439 33554432000 4194304 4625164 140736764711776 18446744073709551615 268660718577 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22764/tid=22768] ppid=22762 vsize=90456 CPUtime=0 cores=0-7
/proc/22764/task/22768/stat : 22768 (SArTagnan) R 22762 22764 22496 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 8 0 536380916 92626944 439 33554432000 4194304 4625164 140736764711776 18446744073709551615 47571637471154 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22769] ppid=22762 vsize=90456 CPUtime=0 cores=0-7
/proc/22764/task/22769/stat : 22769 (SArTagnan) R 22762 22764 22496 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 8 0 536380916 92626944 439 33554432000 4194304 4625164 140736764711776 18446744073709551615 47571637471154 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22770] ppid=22762 vsize=90456 CPUtime=0.01 cores=0-7
/proc/22764/task/22770/stat : 22770 (SArTagnan) R 22762 22764 22496 0 -1 4202560 0 0 0 0 1 0 0 0 25 0 8 0 536380916 92626944 439 33554432000 4194304 4625164 140736764711776 18446744073709551615 47571637471148 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22764/tid=22771] ppid=22762 vsize=90456 CPUtime=0 cores=0-7
/proc/22764/task/22771/stat : 22771 (SArTagnan) R 22762 22764 22496 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 8 0 536380916 92626944 439 33554432000 4194304 4625164 140736764711776 18446744073709551615 47571637471154 0 0 4096 2 0 0 0 -1 0 0 0 0

[startup+0.0623059 s]
/proc/loadavg: 8.16 8.07 8.01 1/481 22764
/proc/meminfo: memFree=21940180/32951124 swapFree=46575304/67111528
[pid=22764] ppid=22762 vsize=354032 CPUtime=0.34 cores=0-7
/proc/22764/stat : 22764 (SArTagnan) R 22762 22764 22496 0 -1 4202496 1547 0 0 0 34 0 0 0 25 0 8 0 536380916 362528768 1270 33554432000 4194304 4625164 140736764711776 18446744073709551615 4511874 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22764/statm: 88508 1270 352 106 0 84011 0
[pid=22764/tid=22765] ppid=22762 vsize=354032 CPUtime=0.05 cores=0-7
/proc/22764/task/22765/stat : 22765 (SArTagnan) R 22762 22764 22496 0 -1 4202560 106 0 0 0 5 0 0 0 25 0 8 0 536380916 362528768 1270 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351878 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22764/tid=22766] ppid=22762 vsize=354032 CPUtime=0.03 cores=0-7
/proc/22764/task/22766/stat : 22766 (SArTagnan) R 22762 22764 22496 0 -1 4202560 95 0 0 0 3 0 0 0 18 0 8 0 536380916 362528768 1278 33554432000 4194304 4625164 140736764711776 18446744073709551615 4464281 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22764/tid=22767] ppid=22762 vsize=354032 CPUtime=0.03 cores=0-7
/proc/22764/task/22767/stat : 22767 (SArTagnan) R 22762 22764 22496 0 -1 4202560 190 0 0 0 3 0 0 0 18 0 8 0 536380916 362528768 1278 33554432000 4194304 4625164 140736764711776 18446744073709551615 4353037 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22764/tid=22768] ppid=22762 vsize=354032 CPUtime=0.01 cores=0-7
/proc/22764/task/22768/stat : 22768 (SArTagnan) R 22762 22764 22496 0 -1 4202560 15 0 0 0 1 0 0 0 25 0 8 0 536380916 362528768 1278 33554432000 4194304 4625164 140736764711776 18446744073709551615 268660767214 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22769] ppid=22762 vsize=354032 CPUtime=0.04 cores=0-7
/proc/22764/task/22769/stat : 22769 (SArTagnan) R 22762 22764 22496 0 -1 4202560 142 0 0 0 4 0 0 0 25 0 8 0 536380916 362528768 1278 33554432000 4194304 4625164 140736764711776 18446744073709551615 4279928 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22770] ppid=22762 vsize=354032 CPUtime=0.05 cores=0-7
/proc/22764/task/22770/stat : 22770 (SArTagnan) R 22762 22764 22496 0 -1 4202560 143 0 0 0 5 0 0 0 25 0 8 0 536380916 362528768 1279 33554432000 4194304 4625164 140736764711776 18446744073709551615 4462593 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22764/tid=22771] ppid=22762 vsize=354032 CPUtime=0.03 cores=0-7
/proc/22764/task/22771/stat : 22771 (SArTagnan) R 22762 22764 22496 0 -1 4202560 99 0 0 0 3 0 0 0 18 0 8 0 536380916 362528768 1279 33554432000 4194304 4625164 140736764711776 18446744073709551615 268660323013 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 0.34
Current children cumulated vsize (KiB) 354032

[startup+0.101265 s]
/proc/loadavg: 8.16 8.07 8.01 1/481 22764
/proc/meminfo: memFree=21940180/32951124 swapFree=46575304/67111528
[pid=22764] ppid=22762 vsize=355024 CPUtime=0.63 cores=0-7
/proc/22764/stat : 22764 (SArTagnan) R 22762 22764 22496 0 -1 4202496 2455 0 0 0 63 0 0 0 25 0 8 0 536380916 363544576 1819 33554432000 4194304 4625164 140736764711776 18446744073709551615 4356845 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22764/statm: 88756 1819 352 106 0 84259 0
[pid=22764/tid=22765] ppid=22762 vsize=355024 CPUtime=0.09 cores=0-7
/proc/22764/task/22765/stat : 22765 (SArTagnan) R 22762 22764 22496 0 -1 4202560 243 0 0 0 9 0 0 0 25 0 8 0 536380916 363544576 1819 33554432000 4194304 4625164 140736764711776 18446744073709551615 4353109 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22764/tid=22766] ppid=22762 vsize=355024 CPUtime=0.07 cores=0-7
/proc/22764/task/22766/stat : 22766 (SArTagnan) R 22762 22764 22496 0 -1 4202560 197 0 0 0 7 0 0 0 18 0 8 0 536380916 363544576 1820 33554432000 4194304 4625164 140736764711776 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22764/tid=22767] ppid=22762 vsize=355024 CPUtime=0.07 cores=0-7
/proc/22764/task/22767/stat : 22767 (SArTagnan) R 22762 22764 22496 0 -1 4202560 442 0 0 0 7 0 0 0 18 0 8 0 536380916 363544576 1820 33554432000 4194304 4625164 140736764711776 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22764/tid=22768] ppid=22762 vsize=355024 CPUtime=0.03 cores=0-7
/proc/22764/task/22768/stat : 22768 (SArTagnan) R 22762 22764 22496 0 -1 4202560 68 0 0 0 3 0 0 0 25 0 8 0 536380916 363544576 1820 33554432000 4194304 4625164 140736764711776 18446744073709551615 4352280 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=22764/tid=22769] ppid=22762 vsize=355024 CPUtime=0.08 cores=0-7
/proc/22764/task/22769/stat : 22769 (SArTagnan) R 22762 22764 22496 0 -1 4202560 244 0 0 0 8 0 0 0 25 0 8 0 536380916 363544576 1820 33554432000 4194304 4625164 140736764711776 18446744073709551615 4349024 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22770] ppid=22762 vsize=355024 CPUtime=0.09 cores=0-7
/proc/22764/task/22770/stat : 22770 (SArTagnan) R 22762 22764 22496 0 -1 4202560 245 0 0 0 9 0 0 0 25 0 8 0 536380916 363544576 1820 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22764/tid=22771] ppid=22762 vsize=355024 CPUtime=0.07 cores=0-7
/proc/22764/task/22771/stat : 22771 (SArTagnan) R 22762 22764 22496 0 -1 4202560 235 0 0 0 7 0 0 0 18 0 8 0 536380916 363544576 1821 33554432000 4194304 4625164 140736764711776 18446744073709551615 4352934 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 0.63
Current children cumulated vsize (KiB) 355024

[startup+0.301232 s]
/proc/loadavg: 8.16 8.07 8.01 1/481 22764
/proc/meminfo: memFree=21940180/32951124 swapFree=46575304/67111528
[pid=22764] ppid=22762 vsize=360592 CPUtime=2.23 cores=0-7
/proc/22764/stat : 22764 (SArTagnan) R 22762 22764 22496 0 -1 4202496 6759 0 0 0 223 0 0 0 25 0 8 0 536380916 369246208 4407 33554432000 4194304 4625164 140736764711776 18446744073709551615 4280397 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22764/statm: 90148 4407 352 106 0 85651 0
[pid=22764/tid=22765] ppid=22762 vsize=360592 CPUtime=0.29 cores=0-7
/proc/22764/task/22765/stat : 22765 (SArTagnan) R 22762 22764 22496 0 -1 4202560 618 0 0 0 29 0 0 0 25 0 8 0 536380916 369246208 4407 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22764/tid=22766] ppid=22762 vsize=360592 CPUtime=0.27 cores=0-7
/proc/22764/task/22766/stat : 22766 (SArTagnan) R 22762 22764 22496 0 -1 4202560 550 0 0 0 27 0 0 0 18 0 8 0 536380916 369246208 4407 33554432000 4194304 4625164 140736764711776 18446744073709551615 4462528 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22764/tid=22767] ppid=22762 vsize=360592 CPUtime=0.27 cores=0-7
/proc/22764/task/22767/stat : 22767 (SArTagnan) R 22762 22764 22496 0 -1 4202560 1931 0 0 0 27 0 0 0 18 0 8 0 536380916 369246208 4407 33554432000 4194304 4625164 140736764711776 18446744073709551615 4567096 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22764/tid=22768] ppid=22762 vsize=360592 CPUtime=0.23 cores=0-7
/proc/22764/task/22768/stat : 22768 (SArTagnan) R 22762 22764 22496 0 -1 4202560 556 0 0 0 23 0 0 0 25 0 8 0 536380916 369246208 4407 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=22764/tid=22769] ppid=22762 vsize=360592 CPUtime=0.28 cores=0-7
/proc/22764/task/22769/stat : 22769 (SArTagnan) R 22762 22764 22496 0 -1 4202560 659 0 0 0 28 0 0 0 25 0 8 0 536380916 369246208 4407 33554432000 4194304 4625164 140736764711776 18446744073709551615 4348870 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22770] ppid=22762 vsize=360592 CPUtime=0.29 cores=0-7
/proc/22764/task/22770/stat : 22770 (SArTagnan) R 22762 22764 22496 0 -1 4202560 803 0 0 0 29 0 0 0 25 0 8 0 536380916 369246208 4407 33554432000 4194304 4625164 140736764711776 18446744073709551615 4352047 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22764/tid=22771] ppid=22762 vsize=360592 CPUtime=0.27 cores=0-7
/proc/22764/task/22771/stat : 22771 (SArTagnan) R 22762 22764 22496 0 -1 4202560 610 0 0 0 27 0 0 0 18 0 8 0 536380916 369246208 4407 33554432000 4194304 4625164 140736764711776 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2.23
Current children cumulated vsize (KiB) 360592

[startup+0.701164 s]
/proc/loadavg: 8.16 8.07 8.01 1/481 22764
/proc/meminfo: memFree=21940180/32951124 swapFree=46575304/67111528
[pid=22764] ppid=22762 vsize=368892 CPUtime=5.42 cores=0-7
/proc/22764/stat : 22764 (SArTagnan) R 22762 22764 22496 0 -1 4202496 14353 0 0 0 541 1 0 0 25 0 8 0 536380916 377745408 8528 33554432000 4194304 4625164 140736764711776 18446744073709551615 4519349 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22764/statm: 92223 8528 352 106 0 87726 0
[pid=22764/tid=22765] ppid=22762 vsize=368892 CPUtime=0.69 cores=0-7
/proc/22764/task/22765/stat : 22765 (SArTagnan) R 22762 22764 22496 0 -1 4202560 1025 0 0 0 69 0 0 0 25 0 8 0 536380916 377745408 8528 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22764/tid=22766] ppid=22762 vsize=368892 CPUtime=0.67 cores=0-7
/proc/22764/task/22766/stat : 22766 (SArTagnan) R 22762 22764 22496 0 -1 4202560 1264 0 0 0 67 0 0 0 18 0 8 0 536380916 377745408 8528 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351774 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22764/tid=22767] ppid=22762 vsize=368892 CPUtime=0.66 cores=0-7
/proc/22764/task/22767/stat : 22767 (SArTagnan) R 22762 22764 22496 0 -1 4202560 4895 0 0 0 66 0 0 0 18 0 8 0 536380916 377745408 8528 33554432000 4194304 4625164 140736764711776 18446744073709551615 4579325 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22764/tid=22768] ppid=22762 vsize=368892 CPUtime=0.63 cores=0-7
/proc/22764/task/22768/stat : 22768 (SArTagnan) R 22762 22764 22496 0 -1 4202560 1259 0 0 0 63 0 0 0 25 0 8 0 536380916 377745408 8528 33554432000 4194304 4625164 140736764711776 18446744073709551615 4465382 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=22764/tid=22769] ppid=22762 vsize=368892 CPUtime=0.68 cores=0-7
/proc/22764/task/22769/stat : 22769 (SArTagnan) R 22762 22764 22496 0 -1 4202560 1408 0 0 0 68 0 0 0 25 0 8 0 536380916 377745408 8528 33554432000 4194304 4625164 140736764711776 18446744073709551615 4343631 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22770] ppid=22762 vsize=368892 CPUtime=0.69 cores=0-7
/proc/22764/task/22770/stat : 22770 (SArTagnan) R 22762 22764 22496 0 -1 4202560 1763 0 0 0 69 0 0 0 25 0 8 0 536380916 377745408 8528 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22764/tid=22771] ppid=22762 vsize=368892 CPUtime=0.67 cores=0-7
/proc/22764/task/22771/stat : 22771 (SArTagnan) R 22762 22764 22496 0 -1 4202560 1299 0 0 0 67 0 0 0 18 0 8 0 536380916 377745408 8534 33554432000 4194304 4625164 140736764711776 18446744073709551615 4352310 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 5.42
Current children cumulated vsize (KiB) 368892

[startup+1.50102 s]
/proc/loadavg: 8.15 8.07 8.01 9/489 22772
/proc/meminfo: memFree=21898880/32951124 swapFree=46575304/67111528
[pid=22764] ppid=22762 vsize=380020 CPUtime=11.81 cores=0-7
/proc/22764/stat : 22764 (SArTagnan) R 22762 22764 22496 0 -1 4202496 27158 0 0 0 1178 3 0 0 25 0 8 0 536380916 389140480 14112 33554432000 4194304 4625164 140736764711776 18446744073709551615 4356539 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22764/statm: 95005 14112 353 106 0 90508 0
[pid=22764/tid=22765] ppid=22762 vsize=380020 CPUtime=1.49 cores=0-7
/proc/22764/task/22765/stat : 22765 (SArTagnan) R 22762 22764 22496 0 -1 4202560 2173 0 0 0 149 0 0 0 25 0 8 0 536380916 389140480 14112 33554432000 4194304 4625164 140736764711776 18446744073709551615 4352280 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22764/tid=22766] ppid=22762 vsize=380020 CPUtime=1.46 cores=0-7
/proc/22764/task/22766/stat : 22766 (SArTagnan) R 22762 22764 22496 0 -1 4202560 2086 0 0 0 146 0 0 0 20 0 8 0 536380916 389140480 14112 33554432000 4194304 4625164 140736764711776 18446744073709551615 4278541 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22764/tid=22767] ppid=22762 vsize=380020 CPUtime=1.47 cores=0-7
/proc/22764/task/22767/stat : 22767 (SArTagnan) R 22762 22764 22496 0 -1 4202560 10973 0 0 0 146 1 0 0 19 0 8 0 536380916 389140480 14112 33554432000 4194304 4625164 140736764711776 18446744073709551615 4552464 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22764/tid=22768] ppid=22762 vsize=380020 CPUtime=1.41 cores=0-7
/proc/22764/task/22768/stat : 22768 (SArTagnan) R 22762 22764 22496 0 -1 4202560 2127 0 0 0 141 0 0 0 25 0 8 0 536380916 389140480 14112 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=22764/tid=22769] ppid=22762 vsize=380020 CPUtime=1.48 cores=0-7
/proc/22764/task/22769/stat : 22769 (SArTagnan) R 22762 22764 22496 0 -1 4202560 2771 0 0 0 148 0 0 0 25 0 8 0 536380916 389140480 14112 33554432000 4194304 4625164 140736764711776 18446744073709551615 4350077 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22770] ppid=22762 vsize=380020 CPUtime=1.49 cores=0-7
/proc/22764/task/22770/stat : 22770 (SArTagnan) R 22762 22764 22496 0 -1 4202560 2830 0 0 0 149 0 0 0 25 0 8 0 536380916 389140480 14112 33554432000 4194304 4625164 140736764711776 18446744073709551615 4339600 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22764/tid=22771] ppid=22762 vsize=380020 CPUtime=1.47 cores=0-7
/proc/22764/task/22771/stat : 22771 (SArTagnan) R 22762 22764 22496 0 -1 4202560 2157 0 0 0 147 0 0 0 19 0 8 0 536380916 389140480 14112 33554432000 4194304 4625164 140736764711776 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 11.81
Current children cumulated vsize (KiB) 380020

[startup+3.10074 s]
/proc/loadavg: 8.15 8.07 8.01 9/489 22772
/proc/meminfo: memFree=21868252/32951124 swapFree=46575304/67111528
[pid=22764] ppid=22762 vsize=399320 CPUtime=24.59 cores=0-7
/proc/22764/stat : 22764 (SArTagnan) R 22762 22764 22496 0 -1 4202496 49273 0 0 0 2453 6 0 0 25 0 8 0 536380916 408903680 23519 33554432000 4194304 4625164 140736764711776 18446744073709551615 4519319 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22764/statm: 99830 23519 355 106 0 95333 0
[pid=22764/tid=22765] ppid=22762 vsize=399320 CPUtime=3.09 cores=0-7
/proc/22764/task/22765/stat : 22765 (SArTagnan) R 22762 22764 22496 0 -1 4202560 3444 0 0 0 309 0 0 0 25 0 8 0 536380916 408903680 23519 33554432000 4194304 4625164 140736764711776 18446744073709551615 4339591 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22764/tid=22766] ppid=22762 vsize=399320 CPUtime=3.06 cores=0-7
/proc/22764/task/22766/stat : 22766 (SArTagnan) R 22762 22764 22496 0 -1 4202560 4335 0 0 0 306 0 0 0 25 0 8 0 536380916 408903680 23520 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22764/tid=22767] ppid=22762 vsize=399320 CPUtime=3.06 cores=0-7
/proc/22764/task/22767/stat : 22767 (SArTagnan) R 22762 22764 22496 0 -1 4202560 21438 0 0 0 304 2 0 0 25 0 8 0 536380916 408903680 23520 33554432000 4194304 4625164 140736764711776 18446744073709551615 4552423 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22764/tid=22768] ppid=22762 vsize=399320 CPUtime=2.99 cores=0-7
/proc/22764/task/22768/stat : 22768 (SArTagnan) R 22762 22764 22496 0 -1 4202560 4379 0 0 0 299 0 0 0 25 0 8 0 536380916 408903680 23520 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=22764/tid=22769] ppid=22762 vsize=399320 CPUtime=3.07 cores=0-7
/proc/22764/task/22769/stat : 22769 (SArTagnan) R 22762 22764 22496 0 -1 4202560 4478 0 0 0 307 0 0 0 25 0 8 0 536380916 408903680 23520 33554432000 4194304 4625164 140736764711776 18446744073709551615 4349768 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22770] ppid=22762 vsize=399320 CPUtime=3.08 cores=0-7
/proc/22764/task/22770/stat : 22770 (SArTagnan) R 22762 22764 22496 0 -1 4202560 4598 0 0 0 308 0 0 0 25 0 8 0 536380916 408903680 23520 33554432000 4194304 4625164 140736764711776 18446744073709551615 4352035 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22764/tid=22771] ppid=22762 vsize=399320 CPUtime=3.07 cores=0-7
/proc/22764/task/22771/stat : 22771 (SArTagnan) R 22762 22764 22496 0 -1 4202560 3628 0 0 0 307 0 0 0 25 0 8 0 536380916 408903680 23520 33554432000 4194304 4625164 140736764711776 18446744073709551615 4339639 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 24.59
Current children cumulated vsize (KiB) 399320

[startup+6.30118 s]
/proc/loadavg: 8.15 8.07 8.01 9/489 22772
/proc/meminfo: memFree=21808364/32951124 swapFree=46575304/67111528
[pid=22764] ppid=22762 vsize=439840 CPUtime=50.14 cores=0-7
/proc/22764/stat : 22764 (SArTagnan) R 22762 22764 22496 0 -1 4202496 94074 0 0 0 5001 13 0 0 25 0 8 0 536380916 450396160 37577 33554432000 4194304 4625164 140736764711776 18446744073709551615 4512122 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22764/statm: 109960 37577 381 106 0 105463 0
[pid=22764/tid=22765] ppid=22762 vsize=439840 CPUtime=6.29 cores=0-7
/proc/22764/task/22765/stat : 22765 (SArTagnan) R 22762 22764 22496 0 -1 4202560 7238 0 0 0 628 1 0 0 25 0 8 0 536380916 450396160 37577 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22764/tid=22766] ppid=22762 vsize=439840 CPUtime=6.26 cores=0-7
/proc/22764/task/22766/stat : 22766 (SArTagnan) R 22762 22764 22496 0 -1 4202560 6836 0 0 0 626 0 0 0 25 0 8 0 536380916 450396160 37577 33554432000 4194304 4625164 140736764711776 18446744073709551615 4288940 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22764/tid=22767] ppid=22762 vsize=439840 CPUtime=6.27 cores=0-7
/proc/22764/task/22767/stat : 22767 (SArTagnan) R 22762 22764 22496 0 -1 4202560 42892 0 0 0 622 5 0 0 25 0 8 0 536380916 450396160 37577 33554432000 4194304 4625164 140736764711776 18446744073709551615 4546944 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22764/tid=22768] ppid=22762 vsize=439840 CPUtime=6.14 cores=0-7
/proc/22764/task/22768/stat : 22768 (SArTagnan) R 22762 22764 22496 0 -1 4202560 7013 0 0 0 613 1 0 0 25 0 8 0 536380916 450396160 37577 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=22764/tid=22769] ppid=22762 vsize=439840 CPUtime=6.27 cores=0-7
/proc/22764/task/22769/stat : 22769 (SArTagnan) R 22762 22764 22496 0 -1 4202560 9298 0 0 0 627 0 0 0 25 0 8 0 536380916 450396160 37577 33554432000 4194304 4625164 140736764711776 18446744073709551615 4348903 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22770] ppid=22762 vsize=439840 CPUtime=6.29 cores=0-7
/proc/22764/task/22770/stat : 22770 (SArTagnan) R 22762 22764 22496 0 -1 4202560 8950 0 0 0 628 1 0 0 25 0 8 0 536380916 450396160 37577 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22764/tid=22771] ppid=22762 vsize=439840 CPUtime=6.26 cores=0-7
/proc/22764/task/22771/stat : 22771 (SArTagnan) R 22762 22764 22496 0 -1 4202560 7645 0 0 0 626 0 0 0 25 0 8 0 536380916 450396160 37577 33554432000 4194304 4625164 140736764711776 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 1 0 0 0

################
# More data... #
################

[pid=22764/tid=22767] ppid=22762 vsize=4865600 CPUtime=4661.59 cores=0-7
/proc/22764/task/22767/stat : 22767 (SArTagnan) R 22762 22764 22496 0 -1 4202560 15600707 0 0 0 463622 2537 0 0 25 0 8 0 536380916 4982374400 1053148 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22764/tid=22768] ppid=22762 vsize=4865600 CPUtime=4585.63 cores=0-7
/proc/22764/task/22768/stat : 22768 (SArTagnan) R 22762 22764 22496 0 -1 4202560 202694 0 0 0 458519 44 0 0 25 0 8 0 536380916 4982374400 1053148 33554432000 4194304 4625164 140736764711776 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=22764/tid=22769] ppid=22762 vsize=4865600 CPUtime=4662.38 cores=0-7
/proc/22764/task/22769/stat : 22769 (SArTagnan) R 22762 22764 22496 0 -1 4202560 256363 0 0 0 466189 49 0 0 25 0 8 0 536380916 4982374400 1053148 33554432000 4194304 4625164 140736764711776 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22770] ppid=22762 vsize=4865600 CPUtime=4662.39 cores=0-7
/proc/22764/task/22770/stat : 22770 (SArTagnan) R 22762 22764 22496 0 -1 4202560 267936 0 0 0 466177 62 0 0 25 0 8 0 536380916 4982374400 1053148 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22764/tid=22771] ppid=22762 vsize=4865600 CPUtime=4661.86 cores=0-7
/proc/22764/task/22771/stat : 22771 (SArTagnan) R 22762 22764 22496 0 -1 4202560 155207 0 0 0 466148 38 0 0 25 0 8 0 536380916 4982374400 1053148 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37219.7
Current children cumulated vsize (KiB) 4865600

[startup+4722.32 s]
/proc/loadavg: 8.04 8.05 8.01 9/489 22931
/proc/meminfo: memFree=17708712/32951124 swapFree=46575304/67111528
[pid=22764] ppid=22762 vsize=5030876 CPUtime=37698.8 cores=0-7
/proc/22764/stat : 22764 (SArTagnan) R 22762 22764 22496 0 -1 4202496 19338819 0 0 0 3766584 3297 0 0 25 0 8 0 536380916 5151617024 1055869 33554432000 4194304 4625164 140736764711776 18446744073709551615 4512818 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22764/statm: 1257719 1055869 355 106 0 1253222 0
[pid=22764/tid=22765] ppid=22762 vsize=5030876 CPUtime=4721.01 cores=0-7
/proc/22764/task/22765/stat : 22765 (SArTagnan) R 22762 22764 22496 0 -1 4202560 203624 0 0 0 472060 41 0 0 25 0 8 0 536380916 5151617024 1055869 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22764/tid=22766] ppid=22762 vsize=5030876 CPUtime=4722.39 cores=0-7
/proc/22764/task/22766/stat : 22766 (SArTagnan) R 22762 22764 22496 0 -1 4202560 202131 0 0 0 472186 53 0 0 25 0 8 0 536380916 5151617024 1055869 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22764/tid=22767] ppid=22762 vsize=5030876 CPUtime=4721.62 cores=0-7
/proc/22764/task/22767/stat : 22767 (SArTagnan) R 22762 22764 22496 0 -1 4202560 15757221 0 0 0 469596 2566 0 0 25 0 8 0 536380916 5151617024 1055869 33554432000 4194304 4625164 140736764711776 18446744073709551615 4352280 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22764/tid=22768] ppid=22762 vsize=5030876 CPUtime=4644.66 cores=0-7
/proc/22764/task/22768/stat : 22768 (SArTagnan) R 22762 22764 22496 0 -1 4202560 202709 0 0 0 464422 44 0 0 25 0 8 0 536380916 5151617024 1055869 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=22764/tid=22769] ppid=22762 vsize=5030876 CPUtime=4722.4 cores=0-7
/proc/22764/task/22769/stat : 22769 (SArTagnan) R 22762 22764 22496 0 -1 4202560 256379 0 0 0 472191 49 0 0 25 0 8 0 536380916 5151617024 1055869 33554432000 4194304 4625164 140736764711776 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22770] ppid=22762 vsize=5030876 CPUtime=4722.41 cores=0-7
/proc/22764/task/22770/stat : 22770 (SArTagnan) R 22762 22764 22496 0 -1 4202560 351041 0 0 0 472168 73 0 0 25 0 8 0 536380916 5151617024 1055869 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22764/tid=22771] ppid=22762 vsize=5030876 CPUtime=4721.88 cores=0-7
/proc/22764/task/22771/stat : 22771 (SArTagnan) R 22762 22764 22496 0 -1 4202560 155224 0 0 0 472150 38 0 0 25 0 8 0 536380916 5151617024 1055869 33554432000 4194304 4625164 140736764711776 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37698.8
Current children cumulated vsize (KiB) 5030876

[startup+4782.3 s]
/proc/loadavg: 8.08 8.06 8.01 9/489 22932
/proc/meminfo: memFree=17685940/32951124 swapFree=46575304/67111528
[pid=22764] ppid=22762 vsize=5030876 CPUtime=38177.7 cores=0-7
/proc/22764/stat : 22764 (SArTagnan) R 22762 22764 22496 0 -1 4202496 19534038 0 0 0 3814432 3335 0 0 25 0 8 0 536380916 5151617024 1058560 33554432000 4194304 4625164 140736764711776 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22764/statm: 1257719 1058560 355 106 0 1253222 0
[pid=22764/tid=22765] ppid=22762 vsize=5030876 CPUtime=4780.97 cores=0-7
/proc/22764/task/22765/stat : 22765 (SArTagnan) R 22762 22764 22496 0 -1 4202560 203642 0 0 0 478056 41 0 0 25 0 8 0 536380916 5151617024 1058560 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22764/tid=22766] ppid=22762 vsize=5030876 CPUtime=4782.37 cores=0-7
/proc/22764/task/22766/stat : 22766 (SArTagnan) R 22762 22764 22496 0 -1 4202560 202140 0 0 0 478184 53 0 0 25 0 8 0 536380916 5151617024 1058560 33554432000 4194304 4625164 140736764711776 18446744073709551615 4330810 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22764/tid=22767] ppid=22762 vsize=5030876 CPUtime=4781.6 cores=0-7
/proc/22764/task/22767/stat : 22767 (SArTagnan) R 22762 22764 22496 0 -1 4202560 15867444 0 0 0 475572 2588 0 0 25 0 8 0 536380916 5151617024 1058560 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22764/tid=22768] ppid=22762 vsize=5030876 CPUtime=4703.67 cores=0-7
/proc/22764/task/22768/stat : 22768 (SArTagnan) R 22762 22764 22496 0 -1 4202560 202725 0 0 0 470323 44 0 0 25 0 8 0 536380916 5151617024 1058560 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=22764/tid=22769] ppid=22762 vsize=5030876 CPUtime=4782.38 cores=0-7
/proc/22764/task/22769/stat : 22769 (SArTagnan) R 22762 22764 22496 0 -1 4202560 256397 0 0 0 478189 49 0 0 25 0 8 0 536380916 5151617024 1058560 33554432000 4194304 4625164 140736764711776 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22770] ppid=22762 vsize=5030876 CPUtime=4782.39 cores=0-7
/proc/22764/task/22770/stat : 22770 (SArTagnan) R 22762 22764 22496 0 -1 4202560 353214 0 0 0 478165 74 0 0 25 0 8 0 536380916 5151617024 1058560 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22764/tid=22771] ppid=22762 vsize=5030876 CPUtime=4781.86 cores=0-7
/proc/22764/task/22771/stat : 22771 (SArTagnan) R 22762 22764 22496 0 -1 4202560 155241 0 0 0 478148 38 0 0 25 0 8 0 536380916 5151617024 1058560 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38177.7
Current children cumulated vsize (KiB) 5030876

[startup+4842.3 s]
/proc/loadavg: 8.09 8.06 8.01 9/489 22934
/proc/meminfo: memFree=17671756/32951124 swapFree=46575304/67111528
[pid=22764] ppid=22762 vsize=5096412 CPUtime=38656.7 cores=0-7
/proc/22764/stat : 22764 (SArTagnan) R 22762 22764 22496 0 -1 4202496 19753302 0 0 0 3862291 3376 0 0 25 0 8 0 536380916 5218725888 1063989 33554432000 4194304 4625164 140736764711776 18446744073709551615 4512156 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22764/statm: 1274103 1063989 411 106 0 1269606 0
[pid=22764/tid=22765] ppid=22762 vsize=5096412 CPUtime=4840.96 cores=0-7
/proc/22764/task/22765/stat : 22765 (SArTagnan) R 22762 22764 22496 0 -1 4202560 203661 0 0 0 484055 41 0 0 25 0 8 0 536380916 5218725888 1063994 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22764/tid=22766] ppid=22762 vsize=5096412 CPUtime=4842.37 cores=0-7
/proc/22764/task/22766/stat : 22766 (SArTagnan) R 22762 22764 22496 0 -1 4202560 202154 0 0 0 484184 53 0 0 25 0 8 0 536380916 5218725888 1063998 33554432000 4194304 4625164 140736764711776 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22764/tid=22767] ppid=22762 vsize=5096412 CPUtime=4841.59 cores=0-7
/proc/22764/task/22767/stat : 22767 (SArTagnan) R 22762 22764 22496 0 -1 4202560 15999680 0 0 0 481546 2613 0 0 25 0 8 0 536380916 5218725888 1063998 33554432000 4194304 4625164 140736764711776 18446744073709551615 4579598 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22764/tid=22768] ppid=22762 vsize=5096412 CPUtime=4762.68 cores=0-7
/proc/22764/task/22768/stat : 22768 (SArTagnan) R 22762 22764 22496 0 -1 4202560 202741 0 0 0 476224 44 0 0 25 0 8 0 536380916 5218725888 1063998 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=22764/tid=22769] ppid=22762 vsize=5096412 CPUtime=4842.38 cores=0-7
/proc/22764/task/22769/stat : 22769 (SArTagnan) R 22762 22764 22496 0 -1 4202560 256414 0 0 0 484189 49 0 0 25 0 8 0 536380916 5218725888 1063998 33554432000 4194304 4625164 140736764711776 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22770] ppid=22762 vsize=5096412 CPUtime=4842.4 cores=0-7
/proc/22764/task/22770/stat : 22770 (SArTagnan) R 22762 22764 22496 0 -1 4202560 355199 0 0 0 484165 75 0 0 25 0 8 0 536380916 5218725888 1063998 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22764/tid=22771] ppid=22762 vsize=5096412 CPUtime=4841.86 cores=0-7
/proc/22764/task/22771/stat : 22771 (SArTagnan) R 22762 22764 22496 0 -1 4202560 155258 0 0 0 484148 38 0 0 25 0 8 0 536380916 5218725888 1063998 33554432000 4194304 4625164 140736764711776 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38656.7
Current children cumulated vsize (KiB) 5096412

[startup+4902.3 s]
/proc/loadavg: 8.09 8.06 8.01 9/489 22935
/proc/meminfo: memFree=17661276/32951124 swapFree=46575304/67111528
[pid=22764] ppid=22762 vsize=5096412 CPUtime=39135.7 cores=0-7
/proc/22764/stat : 22764 (SArTagnan) R 22762 22764 22496 0 -1 4202496 19971223 0 0 0 3910152 3417 0 0 25 0 8 0 536380916 5218725888 1068628 33554432000 4194304 4625164 140736764711776 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22764/statm: 1274103 1068628 587 106 0 1269606 0
[pid=22764/tid=22765] ppid=22762 vsize=5096412 CPUtime=4900.94 cores=0-7
/proc/22764/task/22765/stat : 22765 (SArTagnan) R 22762 22764 22496 0 -1 4202560 203677 0 0 0 490053 41 0 0 25 0 8 0 536380916 5218725888 1068628 33554432000 4194304 4625164 140736764711776 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22764/tid=22766] ppid=22762 vsize=5096412 CPUtime=4902.37 cores=0-7
/proc/22764/task/22766/stat : 22766 (SArTagnan) R 22762 22764 22496 0 -1 4202560 202164 0 0 0 490184 53 0 0 25 0 8 0 536380916 5218725888 1068628 33554432000 4194304 4625164 140736764711776 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22764/tid=22767] ppid=22762 vsize=5096412 CPUtime=4901.6 cores=0-7
/proc/22764/task/22767/stat : 22767 (SArTagnan) R 22762 22764 22496 0 -1 4202560 16134267 0 0 0 487521 2639 0 0 25 0 8 0 536380916 5218725888 1068628 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22764/tid=22768] ppid=22762 vsize=5096412 CPUtime=4821.71 cores=0-7
/proc/22764/task/22768/stat : 22768 (SArTagnan) R 22762 22764 22496 0 -1 4202560 202757 0 0 0 482127 44 0 0 25 0 8 0 536380916 5218725888 1068628 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=22764/tid=22769] ppid=22762 vsize=5096412 CPUtime=4902.38 cores=0-7
/proc/22764/task/22769/stat : 22769 (SArTagnan) R 22762 22764 22496 0 -1 4202560 256430 0 0 0 490189 49 0 0 25 0 8 0 536380916 5218725888 1068628 33554432000 4194304 4625164 140736764711776 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22770] ppid=22762 vsize=5096412 CPUtime=4902.39 cores=0-7
/proc/22764/task/22770/stat : 22770 (SArTagnan) R 22762 22764 22496 0 -1 4202560 356745 0 0 0 490164 75 0 0 25 0 8 0 536380916 5218725888 1068628 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351801 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22764/tid=22771] ppid=22762 vsize=5096412 CPUtime=4901.86 cores=0-7
/proc/22764/task/22771/stat : 22771 (SArTagnan) R 22762 22764 22496 0 -1 4202560 155275 0 0 0 490148 38 0 0 25 0 8 0 536380916 5218725888 1068628 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39135.7
Current children cumulated vsize (KiB) 5096412

[startup+4962.3 s]
/proc/loadavg: 8.08 8.06 8.01 9/489 22937
/proc/meminfo: memFree=17657896/32951124 swapFree=46575304/67111528
[pid=22764] ppid=22762 vsize=5096412 CPUtime=39614.7 cores=0-7
/proc/22764/stat : 22764 (SArTagnan) R 22762 22764 22496 0 -1 4202496 20233572 0 0 0 3958001 3467 0 0 25 0 8 0 536380916 5218725888 1067877 33554432000 4194304 4625164 140736764711776 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22764/statm: 1274103 1067878 367 106 0 1269606 0
[pid=22764/tid=22765] ppid=22762 vsize=5096412 CPUtime=4960.92 cores=0-7
/proc/22764/task/22765/stat : 22765 (SArTagnan) R 22762 22764 22496 0 -1 4202560 203693 0 0 0 496051 41 0 0 25 0 8 0 536380916 5218725888 1067878 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22764/tid=22766] ppid=22762 vsize=5096412 CPUtime=4962.38 cores=0-7
/proc/22764/task/22766/stat : 22766 (SArTagnan) R 22762 22764 22496 0 -1 4202560 202180 0 0 0 496184 54 0 0 25 0 8 0 536380916 5218725888 1067878 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22764/tid=22767] ppid=22762 vsize=5096412 CPUtime=4961.6 cores=0-7
/proc/22764/task/22767/stat : 22767 (SArTagnan) R 22762 22764 22496 0 -1 4202560 16306794 0 0 0 493491 2669 0 0 25 0 8 0 536380916 5218725888 1067878 33554432000 4194304 4625164 140736764711776 18446744073709551615 4579396 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22764/tid=22768] ppid=22762 vsize=5096412 CPUtime=4880.71 cores=0-7
/proc/22764/task/22768/stat : 22768 (SArTagnan) R 22762 22764 22496 0 -1 4202560 202773 0 0 0 488027 44 0 0 25 0 8 0 536380916 5218725888 1067878 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=22764/tid=22769] ppid=22762 vsize=5096412 CPUtime=4962.38 cores=0-7
/proc/22764/task/22769/stat : 22769 (SArTagnan) R 22762 22764 22496 0 -1 4202560 256448 0 0 0 496189 49 0 0 25 0 8 0 536380916 5218725888 1067878 33554432000 4194304 4625164 140736764711776 18446744073709551615 4329721 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22770] ppid=22762 vsize=5096412 CPUtime=4962.4 cores=0-7
/proc/22764/task/22770/stat : 22770 (SArTagnan) R 22762 22764 22496 0 -1 4202560 357816 0 0 0 496164 76 0 0 25 0 8 0 536380916 5218725888 1067878 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22764/tid=22771] ppid=22762 vsize=5096412 CPUtime=4961.86 cores=0-7
/proc/22764/task/22771/stat : 22771 (SArTagnan) R 22762 22764 22496 0 -1 4202560 155292 0 0 0 496148 38 0 0 25 0 8 0 536380916 5218725888 1067878 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331184 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39614.7
Current children cumulated vsize (KiB) 5096412



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.6 s]
/proc/loadavg: 8.09 8.07 8.01 9/489 22938
/proc/meminfo: memFree=17648160/32951124 swapFree=46575304/67111528
[pid=22764] ppid=22762 vsize=5096412 CPUtime=40000.4 cores=0-7
/proc/22764/stat : 22764 (SArTagnan) R 22762 22764 22496 0 -1 4202496 20455055 0 0 0 3996528 3507 0 0 25 0 8 0 536380916 5218725888 1071252 33554432000 4194304 4625164 140736764711776 18446744073709551615 4519319 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22764/statm: 1274103 1071252 475 106 0 1269606 0
[pid=22764/tid=22765] ppid=22762 vsize=5096412 CPUtime=5009.22 cores=0-7
/proc/22764/task/22765/stat : 22765 (SArTagnan) R 22762 22764 22496 0 -1 4202560 203705 0 0 0 500881 41 0 0 25 0 8 0 536380916 5218725888 1071252 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22764/tid=22766] ppid=22762 vsize=5096412 CPUtime=5010.7 cores=0-7
/proc/22764/task/22766/stat : 22766 (SArTagnan) R 22762 22764 22496 0 -1 4202560 202190 0 0 0 501016 54 0 0 25 0 8 0 536380916 5218725888 1071252 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22764/tid=22767] ppid=22762 vsize=5096412 CPUtime=5009.92 cores=0-7
/proc/22764/task/22767/stat : 22767 (SArTagnan) R 22762 22764 22496 0 -1 4202560 16461649 0 0 0 498296 2696 0 0 25 0 8 0 536380916 5218725888 1071252 33554432000 4194304 4625164 140736764711776 18446744073709551615 4334485 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22764/tid=22768] ppid=22762 vsize=5096412 CPUtime=4928.22 cores=0-7
/proc/22764/task/22768/stat : 22768 (SArTagnan) R 22762 22764 22496 0 -1 4202560 202786 0 0 0 492778 44 0 0 25 0 8 0 536380916 5218725888 1071252 33554432000 4194304 4625164 140736764711776 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=22764/tid=22769] ppid=22762 vsize=5096412 CPUtime=5010.7 cores=0-7
/proc/22764/task/22769/stat : 22769 (SArTagnan) R 22762 22764 22496 0 -1 4202560 256460 0 0 0 501021 49 0 0 25 0 8 0 536380916 5218725888 1071252 33554432000 4194304 4625164 140736764711776 18446744073709551615 4329735 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22764/tid=22770] ppid=22762 vsize=5096412 CPUtime=5010.72 cores=0-7
/proc/22764/task/22770/stat : 22770 (SArTagnan) R 22762 22764 22496 0 -1 4202560 358139 0 0 0 500996 76 0 0 25 0 8 0 536380916 5218725888 1071252 33554432000 4194304 4625164 140736764711776 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22764/tid=22771] ppid=22762 vsize=5096412 CPUtime=5010.18 cores=0-7
/proc/22764/task/22771/stat : 22771 (SArTagnan) R 22762 22764 22496 0 -1 4202560 155304 0 0 0 500980 38 0 0 25 0 8 0 536380916 5218725888 1071252 33554432000 4194304 4625164 140736764711776 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 5096412

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.8 s]
/proc/loadavg: 8.09 8.07 8.01 9/489 22938
/proc/meminfo: memFree=17648160/32951124 swapFree=46575304/67111528
[pid=22764] ppid=22762 vsize=0 CPUtime=40000.6 cores=0-7
/proc/22764/stat : 22764 (SArTagnan) Z 22762 22764 22496 0 -1 4203548 20455087 0 0 0 3996538 3525 0 0 25 0 2 0 536380916 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22764/statm: 0 0 0 0 0 0 0
[pid=22764/tid=22768] ppid=22762 vsize=0 CPUtime=4928.4 cores=0-7
/proc/22764/task/22768/stat : 22768 (SArTagnan) R 22762 22764 22496 0 -1 4203588 202786 0 0 0 492778 62 0 0 25 0 2 0 536380916 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5010.82
CPU time (s): 40000.7
CPU user time (s): 39965.4
CPU system time (s): 35.2756
CPU usage (%): 798.285
Max. virtual memory (cumulated for all children) (KiB): 5420804

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39965.4
system time used= 35.2756
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 20455087
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 542
involuntary context switches= 170605

runsolver used 22.5436 second user time and 59.414 second system time

The end

Launcher Data 

Begin job on node107 at 2011-06-01 11:51:37
IDJOB=3438691
IDBENCH=83153
IDSOLVER=1936
FILE ID=node107/3438691-1306921897
RUNJOBID= node107-1306916883-22510
PBS_JOBID= 13488518
Free space on /tmp= 73848 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n260-s62321009-sat.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3438691-1306921897/watcher-3438691-1306921897 -o /tmp/evaluation-result-3438691-1306921897/solver-3438691-1306921897 -C 40000 -W 5100 -M 31000  SArTagnan HOME/instance-3438691-1306921897.cnf -threads=8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= f2b430bdd9bc1c472e9e437567706bb7
RANDOM SEED=1659530363

node107.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5359.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21940832 kB
Buffers:        232124 kB
Cached:        1733000 kB
SwapCached:      22420 kB
Active:        8922436 kB
Inactive:      1743640 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21940832 kB
SwapTotal:    67111528 kB
SwapFree:     46575304 kB
Dirty:             392 kB
Writeback:           0 kB
AnonPages:     8696008 kB
Mapped:          31516 kB
Slab:           176764 kB
PageTables:      94424 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 145076588 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73848 MiB
End job on node107 at 2011-06-01 13:15:09