Trace number 3439636

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 40000.4 5009.72

General information on the benchmark

NameSAT02/handmade/pehoushek/plainoldcnf/
5cnf_3900_3900_220.shuffled.cnf
MD5SUMf0b7e0675915478b15cab3695fd0fd62
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.033994
Satisfiable
(Un)Satisfiability was proved
Number of variables220
Number of clauses3432
Sum of the clauses size17000
Maximum clause length5
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 32
Number of clauses of size 4156
Number of clauses of size 53274
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3439636-1306965641/watcher-3439636-1306965641 -o /tmp/evaluation-result-3439636-1306965641/solver-3439636-1306965641 -C 40000 -W 5100 -M 31000 SArTagnan HOME/instance-3439636-1306965641.cnf -threads=8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.55 7.94 7.95 2/436 3817
/proc/meminfo: memFree=30216940/32950928 swapFree=52246700/67111528
[pid=3817] ppid=3815 vsize=92044 CPUtime=0 cores=0-7
/proc/3817/stat : 3817 (SArTagnan) R 3815 3817 3607 0 -1 4202496 841 0 0 0 0 0 0 0 25 0 8 0 886794815 94253056 767 33554432000 4194304 4625164 140736242539712 18446744073709551615 231895546865 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/3817/statm: 23011 767 301 106 0 18514 0
[pid=3817/tid=3818] ppid=3815 vsize=289376 CPUtime=0.01 cores=0-7
/proc/3817/task/3818/stat : 3818 (SArTagnan) R 3815 3817 3607 0 -1 4202560 72 0 0 0 1 0 0 0 18 0 8 0 886794815 296321024 1087 33554432000 4194304 4625164 140736242539712 18446744073709551615 4340391 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=3817/tid=3819] ppid=3815 vsize=289376 CPUtime=0 cores=0-7
/proc/3817/task/3819/stat : 3819 (SArTagnan) R 3815 3817 3607 0 -1 4202560 59 0 0 0 0 0 0 0 25 0 8 0 886794815 296321024 1087 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=3817/tid=3820] ppid=3815 vsize=289376 CPUtime=0 cores=0-7
/proc/3817/task/3820/stat : 3820 (SArTagnan) R 3815 3817 3607 0 -1 4202560 54 0 0 0 0 0 0 0 18 0 8 0 886794815 296321024 1087 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=3817/tid=3821] ppid=3815 vsize=289376 CPUtime=0 cores=0-7
/proc/3817/task/3821/stat : 3821 (SArTagnan) R 3815 3817 3607 0 -1 4202560 58 0 0 0 0 0 0 0 18 0 8 0 886794815 296321024 1087 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=3817/tid=3822] ppid=3815 vsize=289376 CPUtime=0 cores=0-7
/proc/3817/task/3822/stat : 3822 (SArTagnan) R 3815 3817 3607 0 -1 4202560 71 0 0 0 0 0 0 0 18 0 8 0 886794815 296321024 1087 33554432000 4194304 4625164 140736242539712 18446744073709551615 231895151649 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3817/tid=3823] ppid=3815 vsize=289376 CPUtime=0 cores=0-7
/proc/3817/task/3823/stat : 3823 (SArTagnan) R 3815 3817 3607 0 -1 4202560 33 0 0 0 0 0 0 0 25 0 8 0 886794815 296321024 1087 33554432000 4194304 4625164 140736242539712 18446744073709551615 231895187211 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=3817/tid=3824] ppid=3815 vsize=289376 CPUtime=0 cores=0-7
/proc/3817/task/3824/stat : 3824 (SArTagnan) R 3815 3817 3607 0 -1 4202560 25 0 0 0 0 0 0 0 25 0 8 0 886794815 296321024 1089 33554432000 4194304 4625164 140736242539712 18446744073709551615 4353037 0 0 4096 2 0 0 0 -1 6 0 0 0

[startup+0.120954 s]
/proc/loadavg: 7.55 7.94 7.95 2/436 3817
/proc/meminfo: memFree=30216940/32950928 swapFree=52246700/67111528
[pid=3817] ppid=3815 vsize=291968 CPUtime=0.86 cores=0-7
/proc/3817/stat : 3817 (SArTagnan) R 3815 3817 3607 0 -1 4202496 3384 0 0 0 86 0 0 0 18 0 8 0 886794815 298975232 2445 33554432000 4194304 4625164 140736242539712 18446744073709551615 4519365 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/3817/statm: 72992 2445 348 106 0 68495 0
[pid=3817/tid=3818] ppid=3815 vsize=291968 CPUtime=0.11 cores=0-7
/proc/3817/task/3818/stat : 3818 (SArTagnan) R 3815 3817 3607 0 -1 4202560 266 0 0 0 11 0 0 0 18 0 8 0 886794815 298975232 2445 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=3817/tid=3819] ppid=3815 vsize=291968 CPUtime=0.1 cores=0-7
/proc/3817/task/3819/stat : 3819 (SArTagnan) R 3815 3817 3607 0 -1 4202560 252 0 0 0 10 0 0 0 25 0 8 0 886794815 298975232 2445 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352035 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=3817/tid=3820] ppid=3815 vsize=291968 CPUtime=0.1 cores=0-7
/proc/3817/task/3820/stat : 3820 (SArTagnan) R 3815 3817 3607 0 -1 4202560 796 0 0 0 10 0 0 0 18 0 8 0 886794815 298975232 2445 33554432000 4194304 4625164 140736242539712 18446744073709551615 4559931 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=3817/tid=3821] ppid=3815 vsize=291968 CPUtime=0.1 cores=0-7
/proc/3817/task/3821/stat : 3821 (SArTagnan) R 3815 3817 3607 0 -1 4202560 281 0 0 0 10 0 0 0 18 0 8 0 886794815 298975232 2445 33554432000 4194304 4625164 140736242539712 18446744073709551615 4465353 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=3817/tid=3822] ppid=3815 vsize=291968 CPUtime=0.11 cores=0-7
/proc/3817/task/3822/stat : 3822 (SArTagnan) R 3815 3817 3607 0 -1 4202560 268 0 0 0 11 0 0 0 18 0 8 0 886794815 298975232 2445 33554432000 4194304 4625164 140736242539712 18446744073709551615 4349075 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3817/tid=3823] ppid=3815 vsize=291968 CPUtime=0.1 cores=0-7
/proc/3817/task/3823/stat : 3823 (SArTagnan) R 3815 3817 3607 0 -1 4202560 246 0 0 0 10 0 0 0 25 0 8 0 886794815 298975232 2445 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=3817/tid=3824] ppid=3815 vsize=291968 CPUtime=0.1 cores=0-7
/proc/3817/task/3824/stat : 3824 (SArTagnan) R 3815 3817 3607 0 -1 4202560 224 0 0 0 10 0 0 0 25 0 8 0 886794815 298975232 2445 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352010 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 0.86
Current children cumulated vsize (KiB) 291968

[startup+0.20094 s]
/proc/loadavg: 7.55 7.94 7.95 2/436 3817
/proc/meminfo: memFree=30216940/32950928 swapFree=52246700/67111528
[pid=3817] ppid=3815 vsize=293416 CPUtime=1.51 cores=0-7
/proc/3817/stat : 3817 (SArTagnan) R 3815 3817 3607 0 -1 4202496 4691 0 0 0 150 1 0 0 18 0 8 0 886794815 300457984 3446 33554432000 4194304 4625164 140736242539712 18446744073709551615 4356329 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/3817/statm: 73354 3446 348 106 0 68857 0
[pid=3817/tid=3818] ppid=3815 vsize=293416 CPUtime=0.19 cores=0-7
/proc/3817/task/3818/stat : 3818 (SArTagnan) R 3815 3817 3607 0 -1 4202560 330 0 0 0 19 0 0 0 18 0 8 0 886794815 300457984 3446 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352299 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=3817/tid=3819] ppid=3815 vsize=293416 CPUtime=0.18 cores=0-7
/proc/3817/task/3819/stat : 3819 (SArTagnan) R 3815 3817 3607 0 -1 4202560 312 0 0 0 18 0 0 0 25 0 8 0 886794815 300457984 3446 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351901 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=3817/tid=3820] ppid=3815 vsize=293416 CPUtime=0.18 cores=0-7
/proc/3817/task/3820/stat : 3820 (SArTagnan) R 3815 3817 3607 0 -1 4202560 1329 0 0 0 18 0 0 0 18 0 8 0 886794815 300457984 3446 33554432000 4194304 4625164 140736242539712 18446744073709551615 231895150003 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=3817/tid=3821] ppid=3815 vsize=293416 CPUtime=0.18 cores=0-7
/proc/3817/task/3821/stat : 3821 (SArTagnan) R 3815 3817 3607 0 -1 4202560 350 0 0 0 18 0 0 0 18 0 8 0 886794815 300457984 3447 33554432000 4194304 4625164 140736242539712 18446744073709551615 4464688 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=3817/tid=3822] ppid=3815 vsize=293416 CPUtime=0.19 cores=0-7
/proc/3817/task/3822/stat : 3822 (SArTagnan) R 3815 3817 3607 0 -1 4202560 332 0 0 0 19 0 0 0 18 0 8 0 886794815 300457984 3447 33554432000 4194304 4625164 140736242539712 18446744073709551615 4349112 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3817/tid=3823] ppid=3815 vsize=293416 CPUtime=0.18 cores=0-7
/proc/3817/task/3823/stat : 3823 (SArTagnan) R 3815 3817 3607 0 -1 4202560 420 0 0 0 18 0 0 0 25 0 8 0 886794815 300457984 3447 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352522 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=3817/tid=3824] ppid=3815 vsize=293416 CPUtime=0.18 cores=0-7
/proc/3817/task/3824/stat : 3824 (SArTagnan) R 3815 3817 3607 0 -1 4202560 399 0 0 0 18 0 0 0 25 0 8 0 886794815 300457984 3447 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 1.51
Current children cumulated vsize (KiB) 293416

[startup+0.300922 s]
/proc/loadavg: 7.55 7.94 7.95 2/436 3817
/proc/meminfo: memFree=30216940/32950928 swapFree=52246700/67111528
[pid=3817] ppid=3815 vsize=295604 CPUtime=2.3 cores=0-7
/proc/3817/stat : 3817 (SArTagnan) R 3815 3817 3607 0 -1 4202496 6547 0 0 0 229 1 0 0 18 0 8 0 886794815 302698496 4565 33554432000 4194304 4625164 140736242539712 18446744073709551615 4356870 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/3817/statm: 73901 4565 348 106 0 69404 0
[pid=3817/tid=3818] ppid=3815 vsize=295604 CPUtime=0.29 cores=0-7
/proc/3817/task/3818/stat : 3818 (SArTagnan) R 3815 3817 3607 0 -1 4202560 516 0 0 0 29 0 0 0 18 0 8 0 886794815 302698496 4565 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=3817/tid=3819] ppid=3815 vsize=295604 CPUtime=0.28 cores=0-7
/proc/3817/task/3819/stat : 3819 (SArTagnan) R 3815 3817 3607 0 -1 4202560 474 0 0 0 28 0 0 0 25 0 8 0 886794815 302698496 4565 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=3817/tid=3820] ppid=3815 vsize=295604 CPUtime=0.27 cores=0-7
/proc/3817/task/3820/stat : 3820 (SArTagnan) R 3815 3817 3607 0 -1 4202560 2158 0 0 0 27 0 0 0 18 0 8 0 886794815 302698496 4565 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351996 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=3817/tid=3821] ppid=3815 vsize=295604 CPUtime=0.28 cores=0-7
/proc/3817/task/3821/stat : 3821 (SArTagnan) R 3815 3817 3607 0 -1 4202560 534 0 0 0 28 0 0 0 18 0 8 0 886794815 302698496 4565 33554432000 4194304 4625164 140736242539712 18446744073709551615 231895151649 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=3817/tid=3822] ppid=3815 vsize=295604 CPUtime=0.29 cores=0-7
/proc/3817/task/3822/stat : 3822 (SArTagnan) R 3815 3817 3607 0 -1 4202560 505 0 0 0 29 0 0 0 18 0 8 0 886794815 302698496 4565 33554432000 4194304 4625164 140736242539712 18446744073709551615 4348903 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3817/tid=3823] ppid=3815 vsize=295604 CPUtime=0.27 cores=0-7
/proc/3817/task/3823/stat : 3823 (SArTagnan) R 3815 3817 3607 0 -1 4202560 475 0 0 0 27 0 0 0 25 0 8 0 886794815 302698496 4565 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351989 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=3817/tid=3824] ppid=3815 vsize=295604 CPUtime=0.28 cores=0-7
/proc/3817/task/3824/stat : 3824 (SArTagnan) R 3815 3817 3607 0 -1 4202560 467 0 0 0 28 0 0 0 25 0 8 0 886794815 302698496 4565 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352305 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2.3
Current children cumulated vsize (KiB) 295604

[startup+0.70085 s]
/proc/loadavg: 7.55 7.94 7.95 2/436 3817
/proc/meminfo: memFree=30216940/32950928 swapFree=52246700/67111528
[pid=3817] ppid=3815 vsize=304048 CPUtime=5.5 cores=0-7
/proc/3817/stat : 3817 (SArTagnan) R 3815 3817 3607 0 -1 4202496 13475 0 0 0 548 2 0 0 18 0 8 0 886794815 311345152 8361 33554432000 4194304 4625164 140736242539712 18446744073709551615 231895595551 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/3817/statm: 76012 8361 348 106 0 71515 0
[pid=3817/tid=3818] ppid=3815 vsize=304048 CPUtime=0.69 cores=0-7
/proc/3817/task/3818/stat : 3818 (SArTagnan) R 3815 3817 3607 0 -1 4202560 1205 0 0 0 69 0 0 0 18 0 8 0 886794815 311345152 8361 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352133 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=3817/tid=3819] ppid=3815 vsize=304048 CPUtime=0.68 cores=0-7
/proc/3817/task/3819/stat : 3819 (SArTagnan) R 3815 3817 3607 0 -1 4202560 884 0 0 0 68 0 0 0 25 0 8 0 886794815 311345152 8361 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=3817/tid=3820] ppid=3815 vsize=304048 CPUtime=0.67 cores=0-7
/proc/3817/task/3820/stat : 3820 (SArTagnan) R 3815 3817 3607 0 -1 4202560 4808 0 0 0 67 0 0 0 18 0 8 0 886794815 311345152 8361 33554432000 4194304 4625164 140736242539712 18446744073709551615 231911474203 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=3817/tid=3821] ppid=3815 vsize=304048 CPUtime=0.68 cores=0-7
/proc/3817/task/3821/stat : 3821 (SArTagnan) R 3815 3817 3607 0 -1 4202560 1205 0 0 0 68 0 0 0 18 0 8 0 886794815 311345152 8361 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351853 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=3817/tid=3822] ppid=3815 vsize=304048 CPUtime=0.68 cores=0-7
/proc/3817/task/3822/stat : 3822 (SArTagnan) R 3815 3817 3607 0 -1 4202560 1174 0 0 0 68 0 0 0 18 0 8 0 886794815 311345152 8362 33554432000 4194304 4625164 140736242539712 18446744073709551615 4349019 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3817/tid=3823] ppid=3815 vsize=304048 CPUtime=0.67 cores=0-7
/proc/3817/task/3823/stat : 3823 (SArTagnan) R 3815 3817 3607 0 -1 4202560 1174 0 0 0 67 0 0 0 25 0 8 0 886794815 311345152 8362 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352512 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=3817/tid=3824] ppid=3815 vsize=304048 CPUtime=0.68 cores=0-7
/proc/3817/task/3824/stat : 3824 (SArTagnan) R 3815 3817 3607 0 -1 4202560 1170 0 0 0 68 0 0 0 25 0 8 0 886794815 311345152 8367 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352072 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 5.5
Current children cumulated vsize (KiB) 304048

[startup+1.5007 s]
/proc/loadavg: 7.55 7.94 7.95 9/444 3825
/proc/meminfo: memFree=30170260/32950928 swapFree=52246700/67111528
[pid=3817] ppid=3815 vsize=318928 CPUtime=11.89 cores=0-7
/proc/3817/stat : 3817 (SArTagnan) R 3815 3817 3607 0 -1 4202496 27100 0 0 0 1185 4 0 0 20 0 8 0 886794815 326582272 14651 33554432000 4194304 4625164 140736242539712 18446744073709551615 4396715 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/3817/statm: 79732 14652 348 106 0 75235 0
[pid=3817/tid=3818] ppid=3815 vsize=318928 CPUtime=1.49 cores=0-7
/proc/3817/task/3818/stat : 3818 (SArTagnan) R 3815 3817 3607 0 -1 4202560 1977 0 0 0 149 0 0 0 20 0 8 0 886794815 326582272 14652 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=3817/tid=3819] ppid=3815 vsize=318928 CPUtime=1.48 cores=0-7
/proc/3817/task/3819/stat : 3819 (SArTagnan) R 3815 3817 3607 0 -1 4202560 2006 0 0 0 148 0 0 0 25 0 8 0 886794815 326582272 14652 33554432000 4194304 4625164 140736242539712 18446744073709551615 4340391 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=3817/tid=3820] ppid=3815 vsize=318928 CPUtime=1.47 cores=0-7
/proc/3817/task/3820/stat : 3820 (SArTagnan) R 3815 3817 3607 0 -1 4202560 11843 0 0 0 146 1 0 0 20 0 8 0 886794815 326582272 14652 33554432000 4194304 4625164 140736242539712 18446744073709551615 4559864 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=3817/tid=3821] ppid=3815 vsize=318928 CPUtime=1.48 cores=0-7
/proc/3817/task/3821/stat : 3821 (SArTagnan) R 3815 3817 3607 0 -1 4202560 1986 0 0 0 148 0 0 0 20 0 8 0 886794815 326582272 14652 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352731 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=3817/tid=3822] ppid=3815 vsize=318928 CPUtime=1.48 cores=0-7
/proc/3817/task/3822/stat : 3822 (SArTagnan) R 3815 3817 3607 0 -1 4202560 1904 0 0 0 148 0 0 0 19 0 8 0 886794815 326582272 14652 33554432000 4194304 4625164 140736242539712 18446744073709551615 4348918 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3817/tid=3823] ppid=3815 vsize=318928 CPUtime=1.46 cores=0-7
/proc/3817/task/3823/stat : 3823 (SArTagnan) R 3815 3817 3607 0 -1 4202560 2466 0 0 0 146 0 0 0 25 0 8 0 886794815 326582272 14652 33554432000 4194304 4625164 140736242539712 18446744073709551615 4279920 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=3817/tid=3824] ppid=3815 vsize=318928 CPUtime=1.48 cores=0-7
/proc/3817/task/3824/stat : 3824 (SArTagnan) R 3815 3817 3607 0 -1 4202560 1994 0 0 0 148 0 0 0 25 0 8 0 886794815 326582272 14652 33554432000 4194304 4625164 140736242539712 18446744073709551615 4462528 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.89
Current children cumulated vsize (KiB) 318928

[startup+3.1004 s]
/proc/loadavg: 7.55 7.94 7.95 9/444 3825
/proc/meminfo: memFree=30140624/32950928 swapFree=52246700/67111528
[pid=3817] ppid=3815 vsize=336308 CPUtime=24.67 cores=0-7
/proc/3817/stat : 3817 (SArTagnan) R 3815 3817 3607 0 -1 4202496 46703 0 0 0 2460 7 0 0 25 0 8 0 886794815 344379392 23801 33554432000 4194304 4625164 140736242539712 18446744073709551615 4512763 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/3817/statm: 84077 23801 348 106 0 79580 0
[pid=3817/tid=3818] ppid=3815 vsize=336308 CPUtime=3.09 cores=0-7
/proc/3817/task/3818/stat : 3818 (SArTagnan) R 3815 3817 3607 0 -1 4202560 3064 0 0 0 309 0 0 0 25 0 8 0 886794815 344379392 23801 33554432000 4194304 4625164 140736242539712 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=3817/tid=3819] ppid=3815 vsize=336308 CPUtime=3.08 cores=0-7
/proc/3817/task/3819/stat : 3819 (SArTagnan) R 3815 3817 3607 0 -1 4202560 3266 0 0 0 308 0 0 0 25 0 8 0 886794815 344379392 23801 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352285 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=3817/tid=3820] ppid=3815 vsize=336308 CPUtime=3.08 cores=0-7
/proc/3817/task/3820/stat : 3820 (SArTagnan) R 3815 3817 3607 0 -1 4202560 23026 0 0 0 305 3 0 0 25 0 8 0 886794815 344379392 23801 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352776 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=3817/tid=3821] ppid=3815 vsize=336308 CPUtime=3.08 cores=0-7
/proc/3817/task/3821/stat : 3821 (SArTagnan) R 3815 3817 3607 0 -1 4202560 3011 0 0 0 308 0 0 0 25 0 8 0 886794815 344379392 23801 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=3817/tid=3822] ppid=3815 vsize=336308 CPUtime=3.08 cores=0-7
/proc/3817/task/3822/stat : 3822 (SArTagnan) R 3815 3817 3607 0 -1 4202560 3197 0 0 0 308 0 0 0 25 0 8 0 886794815 344379392 23801 33554432000 4194304 4625164 140736242539712 18446744073709551615 4349019 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3817/tid=3823] ppid=3815 vsize=336308 CPUtime=3.04 cores=0-7
/proc/3817/task/3823/stat : 3823 (SArTagnan) R 3815 3817 3607 0 -1 4202560 3987 0 0 0 304 0 0 0 25 0 8 0 886794815 344379392 23801 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352728 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=3817/tid=3824] ppid=3815 vsize=336308 CPUtime=3.08 cores=0-7
/proc/3817/task/3824/stat : 3824 (SArTagnan) R 3815 3817 3607 0 -1 4202560 2957 0 0 0 308 0 0 0 25 0 8 0 886794815 344379392 23801 33554432000 4194304 4625164 140736242539712 18446744073709551615 4464320 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.67
Current children cumulated vsize (KiB) 336308

[startup+6.3008 s]
/proc/loadavg: 7.59 7.94 7.95 9/444 3825
/proc/meminfo: memFree=30081732/32950928 swapFree=52246700/67111528
[pid=3817] ppid=3815 vsize=360968 CPUtime=50.22 cores=0-7
/proc/3817/stat : 3817 (SArTagnan) R 3815 3817 3607 0 -1 4202496 79404 0 0 0 5009 13 0 0 25 0 8 0 886794815 369631232 36453 33554432000 4194304 4625164 140736242539712 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/3817/statm: 90242 36453 348 106 0 85745 0
[pid=3817/tid=3818] ppid=3815 vsize=360968 CPUtime=6.28 cores=0-7
/proc/3817/task/3818/stat : 3818 (SArTagnan) R 3815 3817 3607 0 -1 4202560 4861 0 0 0 628 0 0 0 25 0 8 0 886794815 369631232 36453 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351818 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=3817/tid=3819] ppid=3815 vsize=360968 CPUtime=6.28 cores=0-7
/proc/3817/task/3819/stat : 3819 (SArTagnan) R 3815 3817 3607 0 -1 4202560 4762 0 0 0 628 0 0 0 25 0 8 0 886794815 369631232 36453 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=3817/tid=3820] ppid=3815 vsize=360968 CPUtime=6.28 cores=0-7
/proc/3817/task/3820/stat : 3820 (SArTagnan) R 3815 3817 3607 0 -1 4202560 41970 0 0 0 621 7 0 0 25 0 8 0 886794815 369631232 36453 33554432000 4194304 4625164 140736242539712 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=3817/tid=3821] ppid=3815 vsize=360968 CPUtime=6.28 cores=0-7
/proc/3817/task/3821/stat : 3821 (SArTagnan) R 3815 3817 3607 0 -1 4202560 4662 0 0 0 628 0 0 0 25 0 8 0 886794815 369631232 36453 33554432000 4194304 4625164 140736242539712 18446744073709551615 4352035 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=3817/tid=3822] ppid=3815 vsize=360968 CPUtime=6.29 cores=0-7
/proc/3817/task/3822/stat : 3822 (SArTagnan) R 3815 3817 3607 0 -1 4202560 6124 0 0 0 628 1 0 0 25 0 8 0 886794815 369631232 36454 33554432000 4194304 4625164 140736242539712 18446744073709551615 4348844 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3817/tid=3823] ppid=3815 vsize=360968 CPUtime=6.19 cores=0-7
/proc/3817/task/3823/stat : 3823 (SArTagnan) R 3815 3817 3607 0 -1 4202560 6388 0 0 0 619 0 0 0 25 0 8 0 886794815 369631232 36455 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351901 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=3817/tid=3824] ppid=3815 vsize=360968 CPUtime=6.28 cores=0-7
/proc/3817/task/3824/stat : 3824 (SArTagnan) R 3815 3817 3607 0 -1 4202560 4556 0 0 0 628 0 0 0 25 0 8 0 886794815 369631232 36455 33554432000 4194304 4625164 140736242539712 18446744073709551615 4340391 0 0 4096 2 0 0 0 -1 6 0 0 0

################
# More data... #
################

[pid=3817/tid=3820] ppid=3815 vsize=4710136 CPUtime=4602.49 cores=0-7
/proc/3817/task/3820/stat : 3820 (SArTagnan) R 3815 3817 3607 0 -1 4202560 13425360 0 0 0 457871 2378 0 0 25 0 8 0 886794815 4823179264 957534 33554432000 4194304 4625164 140736242539712 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=3817/tid=3821] ppid=3815 vsize=4710136 CPUtime=4662.43 cores=0-7
/proc/3817/task/3821/stat : 3821 (SArTagnan) R 3815 3817 3607 0 -1 4202560 132186 0 0 0 466213 30 0 0 25 0 8 0 886794815 4823179264 957534 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=3817/tid=3822] ppid=3815 vsize=4710136 CPUtime=4661.33 cores=0-7
/proc/3817/task/3822/stat : 3822 (SArTagnan) R 3815 3817 3607 0 -1 4202560 162924 0 0 0 466101 32 0 0 25 0 8 0 886794815 4823179264 957534 33554432000 4194304 4625164 140736242539712 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3817/tid=3823] ppid=3815 vsize=4710136 CPUtime=4661.45 cores=0-7
/proc/3817/task/3823/stat : 3823 (SArTagnan) R 3815 3817 3607 0 -1 4202560 328691 0 0 0 466069 76 0 0 25 0 8 0 886794815 4823179264 957534 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=3817/tid=3824] ppid=3815 vsize=4710136 CPUtime=4661.59 cores=0-7
/proc/3817/task/3824/stat : 3824 (SArTagnan) R 3815 3817 3607 0 -1 4202560 129853 0 0 0 466134 25 0 0 25 0 8 0 886794815 4823179264 957534 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37227.6
Current children cumulated vsize (KiB) 4710136

[startup+4722.3 s]
/proc/loadavg: 8.02 8.05 8.00 9/444 3973
/proc/meminfo: memFree=26367620/32950928 swapFree=52246700/67111528
[pid=3817] ppid=3815 vsize=4710136 CPUtime=37706.8 cores=0-7
/proc/3817/stat : 3817 (SArTagnan) R 3815 3817 3607 0 -1 4202496 20443792 0 0 0 3766970 3706 0 0 25 0 8 0 886794815 4823179264 960766 33554432000 4194304 4625164 140736242539712 18446744073709551615 4392981 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/3817/statm: 1177534 960766 548 106 0 1173037 0
[pid=3817/tid=3818] ppid=3815 vsize=4710136 CPUtime=4722.46 cores=0-7
/proc/3817/task/3818/stat : 3818 (SArTagnan) R 3815 3817 3607 0 -1 4202560 130943 0 0 0 472220 26 0 0 25 0 8 0 886794815 4823179264 960766 33554432000 4194304 4625164 140736242539712 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=3817/tid=3819] ppid=3815 vsize=4710136 CPUtime=4713.55 cores=0-7
/proc/3817/task/3819/stat : 3819 (SArTagnan) R 3815 3817 3607 0 -1 4202560 165582 0 0 0 471323 32 0 0 25 0 8 0 886794815 4823179264 960766 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=3817/tid=3820] ppid=3815 vsize=4710136 CPUtime=4661.59 cores=0-7
/proc/3817/task/3820/stat : 3820 (SArTagnan) R 3815 3817 3607 0 -1 4202560 13655936 0 0 0 463735 2424 0 0 25 0 8 0 886794815 4823179264 960766 33554432000 4194304 4625164 140736242539712 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=3817/tid=3821] ppid=3815 vsize=4710136 CPUtime=4722.45 cores=0-7
/proc/3817/task/3821/stat : 3821 (SArTagnan) R 3815 3817 3607 0 -1 4202560 132200 0 0 0 472215 30 0 0 25 0 8 0 886794815 4823179264 960766 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=3817/tid=3822] ppid=3815 vsize=4710136 CPUtime=4721.33 cores=0-7
/proc/3817/task/3822/stat : 3822 (SArTagnan) R 3815 3817 3607 0 -1 4202560 162937 0 0 0 472101 32 0 0 25 0 8 0 886794815 4823179264 960766 33554432000 4194304 4625164 140736242539712 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3817/tid=3823] ppid=3815 vsize=4710136 CPUtime=4721.47 cores=0-7
/proc/3817/task/3823/stat : 3823 (SArTagnan) R 3815 3817 3607 0 -1 4202560 329255 0 0 0 472071 76 0 0 25 0 8 0 886794815 4823179264 960766 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=3817/tid=3824] ppid=3815 vsize=4710136 CPUtime=4721.61 cores=0-7
/proc/3817/task/3824/stat : 3824 (SArTagnan) R 3815 3817 3607 0 -1 4202560 129868 0 0 0 472136 25 0 0 25 0 8 0 886794815 4823179264 960766 33554432000 4194304 4625164 140736242539712 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37706.8
Current children cumulated vsize (KiB) 4710136

[startup+4782.3 s]
/proc/loadavg: 8.01 8.04 8.00 9/444 3974
/proc/meminfo: memFree=26375380/32950928 swapFree=52246700/67111528
[pid=3817] ppid=3815 vsize=4710720 CPUtime=38185.8 cores=0-7
/proc/3817/stat : 3817 (SArTagnan) R 3815 3817 3607 0 -1 4202496 20694262 0 0 0 3814829 3752 0 0 25 0 8 0 886794815 4823777280 959865 33554432000 4194304 4625164 140736242539712 18446744073709551615 4512156 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/3817/statm: 1177680 959865 348 106 0 1173183 0
[pid=3817/tid=3818] ppid=3815 vsize=4710720 CPUtime=4782.45 cores=0-7
/proc/3817/task/3818/stat : 3818 (SArTagnan) R 3815 3817 3607 0 -1 4202560 130960 0 0 0 478219 26 0 0 25 0 8 0 886794815 4823777280 959865 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=3817/tid=3819] ppid=3815 vsize=4710720 CPUtime=4773.54 cores=0-7
/proc/3817/task/3819/stat : 3819 (SArTagnan) R 3815 3817 3607 0 -1 4202560 165595 0 0 0 477322 32 0 0 25 0 8 0 886794815 4823777280 959865 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=3817/tid=3820] ppid=3815 vsize=4710720 CPUtime=4720.67 cores=0-7
/proc/3817/task/3820/stat : 3820 (SArTagnan) R 3815 3817 3607 0 -1 4202560 13904079 0 0 0 469599 2468 0 0 25 0 8 0 886794815 4823777280 959865 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=3817/tid=3821] ppid=3815 vsize=4710720 CPUtime=4782.44 cores=0-7
/proc/3817/task/3821/stat : 3821 (SArTagnan) R 3815 3817 3607 0 -1 4202560 132216 0 0 0 478214 30 0 0 25 0 8 0 886794815 4823777280 959865 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=3817/tid=3822] ppid=3815 vsize=4710720 CPUtime=4781.3 cores=0-7
/proc/3817/task/3822/stat : 3822 (SArTagnan) R 3815 3817 3607 0 -1 4202560 162951 0 0 0 478098 32 0 0 25 0 8 0 886794815 4823777280 959865 33554432000 4194304 4625164 140736242539712 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3817/tid=3823] ppid=3815 vsize=4710720 CPUtime=4781.46 cores=0-7
/proc/3817/task/3823/stat : 3823 (SArTagnan) R 3815 3817 3607 0 -1 4202560 329970 0 0 0 478069 77 0 0 25 0 8 0 886794815 4823777280 959865 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=3817/tid=3824] ppid=3815 vsize=4710720 CPUtime=4781.6 cores=0-7
/proc/3817/task/3824/stat : 3824 (SArTagnan) R 3815 3817 3607 0 -1 4202560 129905 0 0 0 478135 25 0 0 25 0 8 0 886794815 4823777280 959865 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38185.8
Current children cumulated vsize (KiB) 4710720

[startup+4842.32 s]
/proc/loadavg: 8.04 8.04 8.00 9/444 3976
/proc/meminfo: memFree=26364672/32950928 swapFree=52246700/67111528
[pid=3817] ppid=3815 vsize=4710720 CPUtime=38665 cores=0-7
/proc/3817/stat : 3817 (SArTagnan) R 3815 3817 3607 0 -1 4202496 20965696 0 0 0 3862698 3805 0 0 25 0 8 0 886794815 4823777280 961332 33554432000 4194304 4625164 140736242539712 18446744073709551615 4329151 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/3817/statm: 1177680 961332 348 106 0 1173183 0
[pid=3817/tid=3818] ppid=3815 vsize=4710720 CPUtime=4842.47 cores=0-7
/proc/3817/task/3818/stat : 3818 (SArTagnan) R 3815 3817 3607 0 -1 4202560 130974 0 0 0 484221 26 0 0 25 0 8 0 886794815 4823777280 961332 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=3817/tid=3819] ppid=3815 vsize=4710720 CPUtime=4833.56 cores=0-7
/proc/3817/task/3819/stat : 3819 (SArTagnan) R 3815 3817 3607 0 -1 4202560 165606 0 0 0 483324 32 0 0 25 0 8 0 886794815 4823777280 961332 33554432000 4194304 4625164 140736242539712 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=3817/tid=3820] ppid=3815 vsize=4710720 CPUtime=4779.78 cores=0-7
/proc/3817/task/3820/stat : 3820 (SArTagnan) R 3815 3817 3607 0 -1 4202560 14174107 0 0 0 475457 2521 0 0 25 0 8 0 886794815 4823777280 961332 33554432000 4194304 4625164 140736242539712 18446744073709551615 4552315 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=3817/tid=3821] ppid=3815 vsize=4710720 CPUtime=4842.46 cores=0-7
/proc/3817/task/3821/stat : 3821 (SArTagnan) R 3815 3817 3607 0 -1 4202560 132232 0 0 0 484216 30 0 0 25 0 8 0 886794815 4823777280 961332 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=3817/tid=3822] ppid=3815 vsize=4710720 CPUtime=4841.31 cores=0-7
/proc/3817/task/3822/stat : 3822 (SArTagnan) R 3815 3817 3607 0 -1 4202560 162965 0 0 0 484099 32 0 0 25 0 8 0 886794815 4823777280 961332 33554432000 4194304 4625164 140736242539712 18446744073709551615 4329721 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3817/tid=3823] ppid=3815 vsize=4710720 CPUtime=4841.48 cores=0-7
/proc/3817/task/3823/stat : 3823 (SArTagnan) R 3815 3817 3607 0 -1 4202560 330709 0 0 0 484071 77 0 0 25 0 8 0 886794815 4823777280 961332 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=3817/tid=3824] ppid=3815 vsize=4710720 CPUtime=4841.61 cores=0-7
/proc/3817/task/3824/stat : 3824 (SArTagnan) R 3815 3817 3607 0 -1 4202560 129919 0 0 0 484136 25 0 0 25 0 8 0 886794815 4823777280 961332 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38665
Current children cumulated vsize (KiB) 4710720

[startup+4902.3 s]
/proc/loadavg: 8.05 8.05 8.00 9/444 3978
/proc/meminfo: memFree=26349556/32950928 swapFree=52246700/67111528
[pid=3817] ppid=3815 vsize=4710720 CPUtime=39144 cores=0-7
/proc/3817/stat : 3817 (SArTagnan) R 3815 3817 3607 0 -1 4202496 21253683 0 0 0 3910537 3861 0 0 25 0 8 0 886794815 4823777280 961104 33554432000 4194304 4625164 140736242539712 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/3817/statm: 1177680 961104 348 106 0 1173183 0
[pid=3817/tid=3818] ppid=3815 vsize=4710720 CPUtime=4902.45 cores=0-7
/proc/3817/task/3818/stat : 3818 (SArTagnan) R 3815 3817 3607 0 -1 4202560 130989 0 0 0 490219 26 0 0 25 0 8 0 886794815 4823777280 961104 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=3817/tid=3819] ppid=3815 vsize=4710720 CPUtime=4893.54 cores=0-7
/proc/3817/task/3819/stat : 3819 (SArTagnan) R 3815 3817 3607 0 -1 4202560 165616 0 0 0 489322 32 0 0 25 0 8 0 886794815 4823777280 961104 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331104 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=3817/tid=3820] ppid=3815 vsize=4710720 CPUtime=4838.86 cores=0-7
/proc/3817/task/3820/stat : 3820 (SArTagnan) R 3815 3817 3607 0 -1 4202560 14389574 0 0 0 481325 2561 0 0 25 0 8 0 886794815 4823777280 961104 33554432000 4194304 4625164 140736242539712 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=3817/tid=3821] ppid=3815 vsize=4710720 CPUtime=4902.44 cores=0-7
/proc/3817/task/3821/stat : 3821 (SArTagnan) R 3815 3817 3607 0 -1 4202560 132273 0 0 0 490214 30 0 0 25 0 8 0 886794815 4823777280 961104 33554432000 4194304 4625164 140736242539712 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=3817/tid=3822] ppid=3815 vsize=4710720 CPUtime=4901.27 cores=0-7
/proc/3817/task/3822/stat : 3822 (SArTagnan) R 3815 3817 3607 0 -1 4202560 162979 0 0 0 490095 32 0 0 25 0 8 0 886794815 4823777280 961104 33554432000 4194304 4625164 140736242539712 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3817/tid=3823] ppid=3815 vsize=4710720 CPUtime=4901.46 cores=0-7
/proc/3817/task/3823/stat : 3823 (SArTagnan) R 3815 3817 3607 0 -1 4202560 331161 0 0 0 490069 77 0 0 25 0 8 0 886794815 4823777280 961104 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=3817/tid=3824] ppid=3815 vsize=4710720 CPUtime=4901.6 cores=0-7
/proc/3817/task/3824/stat : 3824 (SArTagnan) R 3815 3817 3607 0 -1 4202560 129934 0 0 0 490135 25 0 0 25 0 8 0 886794815 4823777280 961104 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39144
Current children cumulated vsize (KiB) 4710720

[startup+4962.3 s]
/proc/loadavg: 8.10 8.05 8.01 9/444 3979
/proc/meminfo: memFree=26349372/32950928 swapFree=52246700/67111528
[pid=3817] ppid=3815 vsize=4710720 CPUtime=39623.1 cores=0-7
/proc/3817/stat : 3817 (SArTagnan) R 3815 3817 3607 0 -1 4202496 21522787 0 0 0 3958396 3911 0 0 25 0 8 0 886794815 4823777280 965211 33554432000 4194304 4625164 140736242539712 18446744073709551615 4393124 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/3817/statm: 1177680 965211 460 106 0 1173183 0
[pid=3817/tid=3818] ppid=3815 vsize=4710720 CPUtime=4962.45 cores=0-7
/proc/3817/task/3818/stat : 3818 (SArTagnan) R 3815 3817 3607 0 -1 4202560 131003 0 0 0 496219 26 0 0 25 0 8 0 886794815 4823777280 965211 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=3817/tid=3819] ppid=3815 vsize=4710720 CPUtime=4953.54 cores=0-7
/proc/3817/task/3819/stat : 3819 (SArTagnan) R 3815 3817 3607 0 -1 4202560 165629 0 0 0 495322 32 0 0 25 0 8 0 886794815 4823777280 965211 33554432000 4194304 4625164 140736242539712 18446744073709551615 4330791 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=3817/tid=3820] ppid=3815 vsize=4710720 CPUtime=4897.94 cores=0-7
/proc/3817/task/3820/stat : 3820 (SArTagnan) R 3815 3817 3607 0 -1 4202560 14632023 0 0 0 487191 2603 0 0 25 0 8 0 886794815 4823777280 965211 33554432000 4194304 4625164 140736242539712 18446744073709551615 4339643 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=3817/tid=3821] ppid=3815 vsize=4710720 CPUtime=4962.45 cores=0-7
/proc/3817/task/3821/stat : 3821 (SArTagnan) R 3815 3817 3607 0 -1 4202560 132288 0 0 0 496214 31 0 0 25 0 8 0 886794815 4823777280 965211 33554432000 4194304 4625164 140736242539712 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=3817/tid=3822] ppid=3815 vsize=4710720 CPUtime=4961.26 cores=0-7
/proc/3817/task/3822/stat : 3822 (SArTagnan) R 3815 3817 3607 0 -1 4202560 162992 0 0 0 496094 32 0 0 25 0 8 0 886794815 4823777280 965211 33554432000 4194304 4625164 140736242539712 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3817/tid=3823] ppid=3815 vsize=4710720 CPUtime=4961.47 cores=0-7
/proc/3817/task/3823/stat : 3823 (SArTagnan) R 3815 3817 3607 0 -1 4202560 331290 0 0 0 496069 78 0 0 25 0 8 0 886794815 4823777280 965211 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=3817/tid=3824] ppid=3815 vsize=4710720 CPUtime=4961.6 cores=0-7
/proc/3817/task/3824/stat : 3824 (SArTagnan) R 3815 3817 3607 0 -1 4202560 129951 0 0 0 496135 25 0 0 25 0 8 0 886794815 4823777280 965211 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39623.1
Current children cumulated vsize (KiB) 4710720



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5009.52 s]
/proc/loadavg: 8.04 8.04 8.00 9/444 3981
/proc/meminfo: memFree=26354676/32950928 swapFree=52246700/67111528
[pid=3817] ppid=3815 vsize=4710720 CPUtime=40000.1 cores=0-7
/proc/3817/stat : 3817 (SArTagnan) R 3815 3817 3607 0 -1 4202496 21725127 0 0 0 3996064 3949 0 0 25 0 8 0 886794815 4823777280 963705 33554432000 4194304 4625164 140736242539712 18446744073709551615 4329131 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/3817/statm: 1177680 963705 378 106 0 1173183 0
[pid=3817/tid=3818] ppid=3815 vsize=4710720 CPUtime=5009.69 cores=0-7
/proc/3817/task/3818/stat : 3818 (SArTagnan) R 3815 3817 3607 0 -1 4202560 131016 0 0 0 500943 26 0 0 25 0 8 0 886794815 4823777280 963705 33554432000 4194304 4625164 140736242539712 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=3817/tid=3819] ppid=3815 vsize=4710720 CPUtime=5000.78 cores=0-7
/proc/3817/task/3819/stat : 3819 (SArTagnan) R 3815 3817 3607 0 -1 4202560 165642 0 0 0 500046 32 0 0 25 0 8 0 886794815 4823777280 963705 33554432000 4194304 4625164 140736242539712 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=3817/tid=3820] ppid=3815 vsize=4710720 CPUtime=4944.44 cores=0-7
/proc/3817/task/3820/stat : 3820 (SArTagnan) R 3815 3817 3607 0 -1 4202560 14833906 0 0 0 491803 2641 0 0 25 0 8 0 886794815 4823777280 963705 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=3817/tid=3821] ppid=3815 vsize=4710720 CPUtime=5009.69 cores=0-7
/proc/3817/task/3821/stat : 3821 (SArTagnan) R 3815 3817 3607 0 -1 4202560 132300 0 0 0 500938 31 0 0 25 0 8 0 886794815 4823777280 963705 33554432000 4194304 4625164 140736242539712 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=3817/tid=3822] ppid=3815 vsize=4710720 CPUtime=5008.49 cores=0-7
/proc/3817/task/3822/stat : 3822 (SArTagnan) R 3815 3817 3607 0 -1 4202560 163003 0 0 0 500817 32 0 0 25 0 8 0 886794815 4823777280 963705 33554432000 4194304 4625164 140736242539712 18446744073709551615 4460412 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3817/tid=3823] ppid=3815 vsize=4710720 CPUtime=5008.7 cores=0-7
/proc/3817/task/3823/stat : 3823 (SArTagnan) R 3815 3817 3607 0 -1 4202560 331449 0 0 0 500792 78 0 0 25 0 8 0 886794815 4823777280 963705 33554432000 4194304 4625164 140736242539712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=3817/tid=3824] ppid=3815 vsize=4710720 CPUtime=5008.84 cores=0-7
/proc/3817/task/3824/stat : 3824 (SArTagnan) R 3815 3817 3607 0 -1 4202560 129964 0 0 0 500859 25 0 0 25 0 8 0 886794815 4823777280 963705 33554432000 4194304 4625164 140736242539712 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 4710720

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5009.7 s]
/proc/loadavg: 8.04 8.04 8.00 9/444 3981
/proc/meminfo: memFree=26354676/32950928 swapFree=52246700/67111528
[pid=3817] ppid=3815 vsize=0 CPUtime=40000.4 cores=0-7
/proc/3817/stat : 3817 (SArTagnan) Z 3815 3817 3607 0 -1 4203548 21725127 0 0 0 3996076 3965 0 0 25 0 2 0 886794815 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/3817/statm: 0 0 0 0 0 0 0
[pid=3817/tid=3820] ppid=3815 vsize=0 CPUtime=4944.61 cores=0-7
/proc/3817/task/3820/stat : 3820 (SArTagnan) R 3815 3817 3607 0 -1 4203588 14833906 0 0 0 491804 2657 0 0 25 0 2 0 886794815 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5009.72
CPU time (s): 40000.4
CPU user time (s): 39960.8
CPU system time (s): 39.676
CPU usage (%): 798.456
Max. virtual memory (cumulated for all children) (KiB): 5010196

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39960.8
system time used= 39.676
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 21725127
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 736
involuntary context switches= 176817

runsolver used 19.998 second user time and 55.0526 second system time

The end

Launcher Data 

Begin job on node114 at 2011-06-02 00:00:41
IDJOB=3439636
IDBENCH=87453
IDSOLVER=1936
FILE ID=node114/3439636-1306965641
RUNJOBID= node114-1306960629-3621
PBS_JOBID= 13497639
Free space on /tmp= 73648 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT02/handmade/pehoushek/plainoldcnf/5cnf_3900_3900_220.shuffled.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3439636-1306965641/watcher-3439636-1306965641 -o /tmp/evaluation-result-3439636-1306965641/solver-3439636-1306965641 -C 40000 -W 5100 -M 31000  SArTagnan HOME/instance-3439636-1306965641.cnf -threads=8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= f0b7e0675915478b15cab3695fd0fd62
RANDOM SEED=538612366

node114.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30217204 kB
Buffers:        193092 kB
Cached:         237032 kB
SwapCached:      10660 kB
Active:        2175224 kB
Inactive:       290480 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30217204 kB
SwapTotal:    67111528 kB
SwapFree:     52246700 kB
Dirty:             408 kB
Writeback:           0 kB
AnonPages:     2032916 kB
Mapped:          20824 kB
Slab:           135192 kB
PageTables:      61972 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 108155560 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73644 MiB
End job on node114 at 2011-06-02 01:24:13