Trace number 3439066

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 40001.1 5010.69

General information on the benchmark

Namecrafted/kullmann/GreenTao/
GreenTaok1k2/GreenTao_2-4-4_511.cnf
MD5SUM766a362c83d920a8f8dd49df62ed42c9
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark1.27781
Satisfiable
(Un)Satisfiability was proved
Number of variables509
Number of clauses4478
Sum of the clauses size17912
Maximum clause length4
Minimum clause length4
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 44478
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3439066-1306926108/watcher-3439066-1306926108 -o /tmp/evaluation-result-3439066-1306926108/solver-3439066-1306926108 -C 40000 -W 5100 -M 31000 SArTagnan HOME/instance-3439066-1306926108.cnf -threads=8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.47 7.93 7.97 1/466 11724
/proc/meminfo: memFree=21033128/32950928 swapFree=61858088/67111528
[pid=11724] ppid=11722 vsize=92140 CPUtime=0.01 cores=0-7
/proc/11724/stat : 11724 (SArTagnan) R 11722 11724 11509 0 -1 4202496 929 0 0 0 1 0 0 0 18 0 8 0 804210428 94351360 865 33554432000 4194304 4625164 140733311487760 18446744073709551615 46997576439730 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/11724/statm: 23035 865 301 106 0 18538 0
[pid=11724/tid=11725] ppid=11722 vsize=92140 CPUtime=0 cores=0-7
/proc/11724/task/11725/stat : 11725 (SArTagnan) R 11722 11724 11509 0 -1 4202560 0 0 0 0 0 0 0 0 19 0 8 0 804210429 94351360 865 33554432000 4194304 4625164 140733311487760 18446744073709551615 274474023921 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=11724/tid=11726] ppid=11722 vsize=92140 CPUtime=0 cores=0-7
/proc/11724/task/11726/stat : 11726 (SArTagnan) R 11722 11724 11509 0 -1 4202560 0 0 0 0 0 0 0 0 19 0 8 0 804210429 94351360 865 33554432000 4194304 4625164 140733311487760 18446744073709551615 274474023921 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=11724/tid=11727] ppid=11722 vsize=92140 CPUtime=0 cores=0-7
/proc/11724/task/11727/stat : 11727 (SArTagnan) R 11722 11724 11509 0 -1 4202560 0 0 0 0 0 0 0 0 19 0 8 0 804210429 94351360 865 33554432000 4194304 4625164 140733311487760 18446744073709551615 46997576439720 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=11724/tid=11728] ppid=11722 vsize=92140 CPUtime=0.01 cores=0-7
/proc/11724/task/11728/stat : 11728 (SArTagnan) R 11722 11724 11509 0 -1 4202560 0 0 0 0 1 0 0 0 19 0 8 0 804210429 94351360 865 33554432000 4194304 4625164 140733311487760 18446744073709551615 46997576439730 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=11724/tid=11729] ppid=11722 vsize=92140 CPUtime=0.01 cores=0-7
/proc/11724/task/11729/stat : 11729 (SArTagnan) R 11722 11724 11509 0 -1 4202560 0 0 0 0 1 0 0 0 19 0 8 0 804210429 94351360 865 33554432000 4194304 4625164 140733311487760 18446744073709551615 46997576439730 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=11724/tid=11730] ppid=11722 vsize=92140 CPUtime=0.01 cores=0-7
/proc/11724/task/11730/stat : 11730 (SArTagnan) R 11722 11724 11509 0 -1 4202560 1 0 0 0 1 0 0 0 20 0 8 0 804210429 94351360 865 33554432000 4194304 4625164 140733311487760 18446744073709551615 46997576439730 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=11724/tid=11731] ppid=11722 vsize=92140 CPUtime=0.01 cores=0-7
/proc/11724/task/11731/stat : 11731 (SArTagnan) R 11722 11724 11509 0 -1 4202560 0 0 0 0 1 0 0 0 20 0 8 0 804210429 94351360 865 33554432000 4194304 4625164 140733311487760 18446744073709551615 46997576439728 0 0 4096 2 0 0 0 -1 3 0 0 0

[startup+0.0184791 s]
/proc/loadavg: 7.47 7.93 7.97 1/466 11724
/proc/meminfo: memFree=21033128/32950928 swapFree=61858088/67111528
[pid=11724] ppid=11722 vsize=92140 CPUtime=0.07 cores=0-7
/proc/11724/stat : 11724 (SArTagnan) R 11722 11724 11509 0 -1 4202496 929 0 0 0 7 0 0 0 18 0 8 0 804210428 94351360 865 33554432000 4194304 4625164 140733311487760 18446744073709551615 46997576439730 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/11724/statm: 23035 865 301 106 0 18538 0
[pid=11724/tid=11725] ppid=11722 vsize=92140 CPUtime=0 cores=0-7
/proc/11724/task/11725/stat : 11725 (SArTagnan) R 11722 11724 11509 0 -1 4202560 0 0 0 0 0 0 0 0 19 0 8 0 804210429 94351360 865 33554432000 4194304 4625164 140733311487760 18446744073709551615 274474023921 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=11724/tid=11726] ppid=11722 vsize=92140 CPUtime=0 cores=0-7
/proc/11724/task/11726/stat : 11726 (SArTagnan) R 11722 11724 11509 0 -1 4202560 0 0 0 0 0 0 0 0 19 0 8 0 804210429 94351360 865 33554432000 4194304 4625164 140733311487760 18446744073709551615 274474023921 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=11724/tid=11727] ppid=11722 vsize=92140 CPUtime=0 cores=0-7
/proc/11724/task/11727/stat : 11727 (SArTagnan) R 11722 11724 11509 0 -1 4202560 0 0 0 0 0 0 0 0 19 0 8 0 804210429 94351360 865 33554432000 4194304 4625164 140733311487760 18446744073709551615 46997576439730 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=11724/tid=11728] ppid=11722 vsize=92140 CPUtime=0.01 cores=0-7
/proc/11724/task/11728/stat : 11728 (SArTagnan) R 11722 11724 11509 0 -1 4202560 0 0 0 0 1 0 0 0 19 0 8 0 804210429 94351360 865 33554432000 4194304 4625164 140733311487760 18446744073709551615 46997576439730 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=11724/tid=11729] ppid=11722 vsize=92140 CPUtime=0.01 cores=0-7
/proc/11724/task/11729/stat : 11729 (SArTagnan) R 11722 11724 11509 0 -1 4202560 0 0 0 0 1 0 0 0 19 0 8 0 804210429 94351360 865 33554432000 4194304 4625164 140733311487760 18446744073709551615 46997576439730 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=11724/tid=11730] ppid=11722 vsize=92140 CPUtime=0.01 cores=0-7
/proc/11724/task/11730/stat : 11730 (SArTagnan) R 11722 11724 11509 0 -1 4202560 1 0 0 0 1 0 0 0 20 0 8 0 804210429 94351360 865 33554432000 4194304 4625164 140733311487760 18446744073709551615 46997576439734 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=11724/tid=11731] ppid=11722 vsize=92140 CPUtime=0.01 cores=0-7
/proc/11724/task/11731/stat : 11731 (SArTagnan) R 11722 11724 11509 0 -1 4202560 0 0 0 0 1 0 0 0 20 0 8 0 804210429 94351360 865 33554432000 4194304 4625164 140733311487760 18446744073709551615 46997576439728 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 92140

[startup+0.174396 s]
/proc/loadavg: 7.47 7.93 7.97 1/466 11724
/proc/meminfo: memFree=21033128/32950928 swapFree=61858088/67111528
[pid=11724] ppid=11722 vsize=490172 CPUtime=1.15 cores=0-7
/proc/11724/stat : 11724 (SArTagnan) R 11722 11724 11509 0 -1 4202496 4812 0 0 0 115 0 0 0 20 0 8 0 804210428 501936128 2987 33554432000 4194304 4625164 140733311487760 18446744073709551615 4385085 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/11724/statm: 122543 2987 351 106 0 118046 0
[pid=11724/tid=11725] ppid=11722 vsize=490172 CPUtime=0.13 cores=0-7
/proc/11724/task/11725/stat : 11725 (SArTagnan) R 11722 11724 11509 0 -1 4202560 364 0 0 0 13 0 0 0 20 0 8 0 804210429 501936128 2988 33554432000 4194304 4625164 140733311487760 18446744073709551615 274473628705 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=11724/tid=11726] ppid=11722 vsize=490172 CPUtime=0.13 cores=0-7
/proc/11724/task/11726/stat : 11726 (SArTagnan) R 11722 11724 11509 0 -1 4202560 412 0 0 0 13 0 0 0 20 0 8 0 804210429 501936128 2988 33554432000 4194304 4625164 140733311487760 18446744073709551615 4352035 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=11724/tid=11727] ppid=11722 vsize=490172 CPUtime=0.14 cores=0-7
/proc/11724/task/11727/stat : 11727 (SArTagnan) R 11722 11724 11509 0 -1 4202560 973 0 0 0 14 0 0 0 20 0 8 0 804210429 501936128 2988 33554432000 4194304 4625164 140733311487760 18446744073709551615 4562308 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=11724/tid=11728] ppid=11722 vsize=490172 CPUtime=0.15 cores=0-7
/proc/11724/task/11728/stat : 11728 (SArTagnan) R 11722 11724 11509 0 -1 4202560 442 0 0 0 15 0 0 0 20 0 8 0 804210429 501936128 2989 33554432000 4194304 4625164 140733311487760 18446744073709551615 4352112 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=11724/tid=11729] ppid=11722 vsize=490172 CPUtime=0.14 cores=0-7
/proc/11724/task/11729/stat : 11729 (SArTagnan) R 11722 11724 11509 0 -1 4202560 422 0 0 0 14 0 0 0 20 0 8 0 804210429 501936128 2989 33554432000 4194304 4625164 140733311487760 18446744073709551615 4349087 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=11724/tid=11730] ppid=11722 vsize=490172 CPUtime=0.15 cores=0-7
/proc/11724/task/11730/stat : 11730 (SArTagnan) R 11722 11724 11509 0 -1 4202560 466 0 0 0 15 0 0 0 21 0 8 0 804210429 501936128 2990 33554432000 4194304 4625164 140733311487760 18446744073709551615 4340391 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=11724/tid=11731] ppid=11722 vsize=490172 CPUtime=0.15 cores=0-7
/proc/11724/task/11731/stat : 11731 (SArTagnan) R 11722 11724 11509 0 -1 4202560 400 0 0 0 15 0 0 0 21 0 8 0 804210429 501936128 2991 33554432000 4194304 4625164 140733311487760 18446744073709551615 4467253 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1.15
Current children cumulated vsize (KiB) 490172

[startup+0.314596 s]
/proc/loadavg: 7.47 7.93 7.97 1/466 11724
/proc/meminfo: memFree=21033128/32950928 swapFree=61858088/67111528
[pid=11724] ppid=11722 vsize=491952 CPUtime=2.75 cores=0-7
/proc/11724/stat : 11724 (SArTagnan) D 11722 11724 11509 0 -1 4202496 8511 0 0 0 274 1 0 0 20 0 8 0 804210428 503758848 4784 33554432000 4194304 4625164 140733311487760 18446744073709551615 274474010807 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/11724/statm: 122988 4784 351 106 0 118491 0
[pid=11724/tid=11725] ppid=11722 vsize=491952 CPUtime=0.33 cores=0-7
/proc/11724/task/11725/stat : 11725 (SArTagnan) R 11722 11724 11509 0 -1 4202560 720 0 0 0 33 0 0 0 20 0 8 0 804210429 503758848 4785 33554432000 4194304 4625164 140733311487760 18446744073709551615 274474010807 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=11724/tid=11726] ppid=11722 vsize=491952 CPUtime=0.33 cores=0-7
/proc/11724/task/11726/stat : 11726 (SArTagnan) R 11722 11724 11509 0 -1 4202560 687 0 0 0 33 0 0 0 20 0 8 0 804210429 503758848 4786 33554432000 4194304 4625164 140733311487760 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=11724/tid=11727] ppid=11722 vsize=491952 CPUtime=0.34 cores=0-7
/proc/11724/task/11727/stat : 11727 (SArTagnan) R 11722 11724 11509 0 -1 4202560 2126 0 0 0 34 0 0 0 20 0 8 0 804210429 503758848 4788 33554432000 4194304 4625164 140733311487760 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=11724/tid=11728] ppid=11722 vsize=491952 CPUtime=0.35 cores=0-7
/proc/11724/task/11728/stat : 11728 (SArTagnan) R 11722 11724 11509 0 -1 4202560 831 0 0 0 35 0 0 0 20 0 8 0 804210429 503758848 4788 33554432000 4194304 4625164 140733311487760 18446744073709551615 4340391 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=11724/tid=11729] ppid=11722 vsize=491952 CPUtime=0.34 cores=0-7
/proc/11724/task/11729/stat : 11729 (SArTagnan) R 11722 11724 11509 0 -1 4202560 778 0 0 0 34 0 0 0 20 0 8 0 804210429 503758848 4788 33554432000 4194304 4625164 140733311487760 18446744073709551615 4349444 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=11724/tid=11730] ppid=11722 vsize=491952 CPUtime=0.34 cores=0-7
/proc/11724/task/11730/stat : 11730 (SArTagnan) R 11722 11724 11509 0 -1 4202560 1078 0 0 0 34 0 0 0 21 0 8 0 804210429 503758848 4789 33554432000 4194304 4625164 140733311487760 18446744073709551615 4352285 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=11724/tid=11731] ppid=11722 vsize=491952 CPUtime=0.35 cores=0-7
/proc/11724/task/11731/stat : 11731 (SArTagnan) R 11722 11724 11509 0 -1 4202560 689 0 0 0 35 0 0 0 21 0 8 0 804210429 503758848 4789 33554432000 4194304 4625164 140733311487760 18446744073709551615 4375331 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2.75
Current children cumulated vsize (KiB) 491952

[startup+0.700306 s]
/proc/loadavg: 7.47 7.93 7.97 1/466 11724
/proc/meminfo: memFree=21033128/32950928 swapFree=61858088/67111528
[pid=11724] ppid=11722 vsize=496292 CPUtime=5.35 cores=0-7
/proc/11724/stat : 11724 (SArTagnan) R 11722 11724 11509 0 -1 4202496 14224 0 0 0 533 2 0 0 21 0 8 0 804210428 508203008 7542 33554432000 4194304 4625164 140733311487760 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/11724/statm: 124073 7542 351 106 0 119576 0
[pid=11724/tid=11725] ppid=11722 vsize=496292 CPUtime=0.66 cores=0-7
/proc/11724/task/11725/stat : 11725 (SArTagnan) R 11722 11724 11509 0 -1 4202560 1241 0 0 0 66 0 0 0 20 0 8 0 804210429 508203008 7542 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351881 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=11724/tid=11726] ppid=11722 vsize=496292 CPUtime=0.65 cores=0-7
/proc/11724/task/11726/stat : 11726 (SArTagnan) R 11722 11724 11509 0 -1 4202560 1559 0 0 0 65 0 0 0 21 0 8 0 804210429 508203008 7542 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=11724/tid=11727] ppid=11722 vsize=496292 CPUtime=0.66 cores=0-7
/proc/11724/task/11727/stat : 11727 (SArTagnan) R 11722 11724 11509 0 -1 4202560 3880 0 0 0 66 0 0 0 21 0 8 0 804210429 508203008 7542 33554432000 4194304 4625164 140733311487760 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=11724/tid=11728] ppid=11722 vsize=496292 CPUtime=0.67 cores=0-7
/proc/11724/task/11728/stat : 11728 (SArTagnan) R 11722 11724 11509 0 -1 4202560 1358 0 0 0 67 0 0 0 20 0 8 0 804210429 508203008 7543 33554432000 4194304 4625164 140733311487760 18446744073709551615 4340391 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=11724/tid=11729] ppid=11722 vsize=496292 CPUtime=0.66 cores=0-7
/proc/11724/task/11729/stat : 11729 (SArTagnan) R 11722 11724 11509 0 -1 4202560 1233 0 0 0 66 0 0 0 20 0 8 0 804210429 508203008 7543 33554432000 4194304 4625164 140733311487760 18446744073709551615 4291425 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=11724/tid=11730] ppid=11722 vsize=496292 CPUtime=0.67 cores=0-7
/proc/11724/task/11730/stat : 11730 (SArTagnan) R 11722 11724 11509 0 -1 4202560 1696 0 0 0 67 0 0 0 22 0 8 0 804210429 508203008 7544 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=11724/tid=11731] ppid=11722 vsize=496292 CPUtime=0.67 cores=0-7
/proc/11724/task/11731/stat : 11731 (SArTagnan) R 11722 11724 11509 0 -1 4202560 1215 0 0 0 67 0 0 0 22 0 8 0 804210429 508203008 7544 33554432000 4194304 4625164 140733311487760 18446744073709551615 274473628705 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 5.35
Current children cumulated vsize (KiB) 496292

[startup+1.50117 s]
/proc/loadavg: 7.47 7.93 7.97 9/474 11732
/proc/meminfo: memFree=20993596/32950928 swapFree=61858088/67111528
[pid=11724] ppid=11722 vsize=503160 CPUtime=11.74 cores=0-7
/proc/11724/stat : 11724 (SArTagnan) R 11722 11724 11509 0 -1 4202496 26844 0 0 0 1170 4 0 0 23 0 8 0 804210428 515235840 12964 33554432000 4194304 4625164 140733311487760 18446744073709551615 4512193 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/11724/statm: 125790 12964 351 106 0 121293 0
[pid=11724/tid=11725] ppid=11722 vsize=503160 CPUtime=1.46 cores=0-7
/proc/11724/task/11725/stat : 11725 (SArTagnan) R 11722 11724 11509 0 -1 4202560 2551 0 0 0 146 0 0 0 23 0 8 0 804210429 515235840 12964 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=11724/tid=11726] ppid=11722 vsize=503160 CPUtime=1.45 cores=0-7
/proc/11724/task/11726/stat : 11726 (SArTagnan) R 11722 11724 11509 0 -1 4202560 2620 0 0 0 145 0 0 0 23 0 8 0 804210429 515235840 12964 33554432000 4194304 4625164 140733311487760 18446744073709551615 4352098 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=11724/tid=11727] ppid=11722 vsize=503160 CPUtime=1.46 cores=0-7
/proc/11724/task/11727/stat : 11727 (SArTagnan) R 11722 11724 11509 0 -1 4202560 8534 0 0 0 145 1 0 0 23 0 8 0 804210429 515235840 12964 33554432000 4194304 4625164 140733311487760 18446744073709551615 4552464 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=11724/tid=11728] ppid=11722 vsize=503160 CPUtime=1.47 cores=0-7
/proc/11724/task/11728/stat : 11728 (SArTagnan) R 11722 11724 11509 0 -1 4202560 2222 0 0 0 147 0 0 0 23 0 8 0 804210429 515235840 12964 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=11724/tid=11729] ppid=11722 vsize=503160 CPUtime=1.46 cores=0-7
/proc/11724/task/11729/stat : 11729 (SArTagnan) R 11722 11724 11509 0 -1 4202560 2643 0 0 0 146 0 0 0 23 0 8 0 804210429 515235840 12964 33554432000 4194304 4625164 140733311487760 18446744073709551615 4348864 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=11724/tid=11730] ppid=11722 vsize=503160 CPUtime=1.47 cores=0-7
/proc/11724/task/11730/stat : 11730 (SArTagnan) R 11722 11724 11509 0 -1 4202560 3520 0 0 0 147 0 0 0 25 0 8 0 804210429 515235840 12964 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351909 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=11724/tid=11731] ppid=11722 vsize=503160 CPUtime=1.47 cores=0-7
/proc/11724/task/11731/stat : 11731 (SArTagnan) R 11722 11724 11509 0 -1 4202560 2061 0 0 0 147 0 0 0 25 0 8 0 804210429 515235840 12965 33554432000 4194304 4625164 140733311487760 18446744073709551615 4352047 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 11.74
Current children cumulated vsize (KiB) 503160

[startup+3.10089 s]
/proc/loadavg: 7.47 7.93 7.97 9/474 11732
/proc/meminfo: memFree=20965572/32950928 swapFree=61858088/67111528
[pid=11724] ppid=11722 vsize=518448 CPUtime=24.53 cores=0-7
/proc/11724/stat : 11724 (SArTagnan) R 11722 11724 11509 0 -1 4202496 44957 0 0 0 2445 8 0 0 25 0 8 0 804210428 530890752 22989 33554432000 4194304 4625164 140733311487760 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/11724/statm: 129612 22989 473 106 0 125115 0
[pid=11724/tid=11725] ppid=11722 vsize=518448 CPUtime=3.06 cores=0-7
/proc/11724/task/11725/stat : 11725 (SArTagnan) R 11722 11724 11509 0 -1 4202560 4071 0 0 0 306 0 0 0 25 0 8 0 804210429 530890752 22989 33554432000 4194304 4625164 140733311487760 18446744073709551615 4352208 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=11724/tid=11726] ppid=11722 vsize=518448 CPUtime=3.05 cores=0-7
/proc/11724/task/11726/stat : 11726 (SArTagnan) R 11722 11724 11509 0 -1 4202560 3819 0 0 0 305 0 0 0 25 0 8 0 804210429 530890752 22989 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351792 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=11724/tid=11727] ppid=11722 vsize=518448 CPUtime=3.06 cores=0-7
/proc/11724/task/11727/stat : 11727 (SArTagnan) R 11722 11724 11509 0 -1 4202560 15644 0 0 0 304 2 0 0 25 0 8 0 804210429 530890752 22989 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=11724/tid=11728] ppid=11722 vsize=518448 CPUtime=3.08 cores=0-7
/proc/11724/task/11728/stat : 11728 (SArTagnan) R 11722 11724 11509 0 -1 4202560 4265 0 0 0 307 1 0 0 25 0 8 0 804210429 530890752 22990 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=11724/tid=11729] ppid=11722 vsize=518448 CPUtime=3.06 cores=0-7
/proc/11724/task/11729/stat : 11729 (SArTagnan) R 11722 11724 11509 0 -1 4202560 3964 0 0 0 306 0 0 0 25 0 8 0 804210429 530890752 23002 33554432000 4194304 4625164 140733311487760 18446744073709551615 4349019 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=11724/tid=11730] ppid=11722 vsize=518448 CPUtime=3.07 cores=0-7
/proc/11724/task/11730/stat : 11730 (SArTagnan) R 11722 11724 11509 0 -1 4202560 5499 0 0 0 307 0 0 0 25 0 8 0 804210429 530890752 23012 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=11724/tid=11731] ppid=11722 vsize=518448 CPUtime=3.07 cores=0-7
/proc/11724/task/11731/stat : 11731 (SArTagnan) R 11722 11724 11509 0 -1 4202560 4051 0 0 0 307 0 0 0 25 0 8 0 804210429 530890752 23013 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 24.53
Current children cumulated vsize (KiB) 518448

[startup+6.30032 s]
/proc/loadavg: 7.51 7.93 7.97 9/474 11733
/proc/meminfo: memFree=20906800/32950928 swapFree=61858088/67111528
[pid=11724] ppid=11722 vsize=528940 CPUtime=50.07 cores=0-7
/proc/11724/stat : 11724 (SArTagnan) R 11722 11724 11509 0 -1 4202496 71816 0 0 0 4995 12 0 0 25 0 8 0 804210428 541634560 34236 33554432000 4194304 4625164 140733311487760 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/11724/statm: 132235 34236 576 106 0 127738 0
[pid=11724/tid=11725] ppid=11722 vsize=528940 CPUtime=6.25 cores=0-7
/proc/11724/task/11725/stat : 11725 (SArTagnan) R 11722 11724 11509 0 -1 4202560 6294 0 0 0 625 0 0 0 25 0 8 0 804210429 541634560 34236 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=11724/tid=11726] ppid=11722 vsize=528940 CPUtime=6.25 cores=0-7
/proc/11724/task/11726/stat : 11726 (SArTagnan) R 11722 11724 11509 0 -1 4202560 5938 0 0 0 624 1 0 0 25 0 8 0 804210429 541634560 34236 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=11724/tid=11727] ppid=11722 vsize=528940 CPUtime=6.26 cores=0-7
/proc/11724/task/11727/stat : 11727 (SArTagnan) R 11722 11724 11509 0 -1 4202560 28467 0 0 0 623 3 0 0 25 0 8 0 804210429 541634560 34236 33554432000 4194304 4625164 140733311487760 18446744073709551615 4339616 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=11724/tid=11728] ppid=11722 vsize=528940 CPUtime=6.27 cores=0-7
/proc/11724/task/11728/stat : 11728 (SArTagnan) R 11722 11724 11509 0 -1 4202560 6043 0 0 0 626 1 0 0 25 0 8 0 804210429 541634560 34236 33554432000 4194304 4625164 140733311487760 18446744073709551615 4352170 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=11724/tid=11729] ppid=11722 vsize=528940 CPUtime=6.27 cores=0-7
/proc/11724/task/11729/stat : 11729 (SArTagnan) R 11722 11724 11509 0 -1 4202560 6107 0 0 0 626 1 0 0 25 0 8 0 804210429 541634560 34236 33554432000 4194304 4625164 140733311487760 18446744073709551615 4349320 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=11724/tid=11730] ppid=11722 vsize=528940 CPUtime=6.27 cores=0-7
/proc/11724/task/11730/stat : 11730 (SArTagnan) R 11722 11724 11509 0 -1 4202560 8290 0 0 0 626 1 0 0 25 0 8 0 804210429 541634560 34236 33554432000 4194304 4625164 140733311487760 18446744073709551615 4339660 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=11724/tid=11731] ppid=11722 vsize=528940 CPUtime=6.27 cores=0-7
/proc/11724/task/11731/stat : 11731 (SArTagnan) R 11722 11724 11509 0 -1 4202560 5802 0 0 0 627 0 0 0 25 0 8 0 804210429 541634560 34236 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351818 0 0 4096 2 0 0 0 -1 3 0 0 0

################
# More data... #
################

[pid=11724/tid=11726] ppid=11722 vsize=5593084 CPUtime=4662.33 cores=0-7
/proc/11724/task/11726/stat : 11726 (SArTagnan) R 11722 11724 11509 0 -1 4202560 260600 0 0 0 466140 93 0 0 25 0 8 0 804210429 5727318016 1157438 33554432000 4194304 4625164 140733311487760 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=11724/tid=11727] ppid=11722 vsize=5593084 CPUtime=4660.76 cores=0-7
/proc/11724/task/11727/stat : 11727 (SArTagnan) R 11722 11724 11509 0 -1 4202560 22864316 0 0 0 462285 3791 0 0 25 0 8 0 804210429 5727318016 1157438 33554432000 4194304 4625164 140733311487760 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=11724/tid=11728] ppid=11722 vsize=5593084 CPUtime=4662.37 cores=0-7
/proc/11724/task/11728/stat : 11728 (SArTagnan) R 11722 11724 11509 0 -1 4202560 220512 0 0 0 466152 85 0 0 25 0 8 0 804210429 5727318016 1157438 33554432000 4194304 4625164 140733311487760 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=11724/tid=11729] ppid=11722 vsize=5593084 CPUtime=4661.01 cores=0-7
/proc/11724/task/11729/stat : 11729 (SArTagnan) R 11722 11724 11509 0 -1 4202560 205175 0 0 0 466036 65 0 0 25 0 8 0 804210429 5727318016 1157438 33554432000 4194304 4625164 140733311487760 18446744073709551615 4460412 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=11724/tid=11730] ppid=11722 vsize=5593084 CPUtime=4662.33 cores=0-7
/proc/11724/task/11730/stat : 11730 (SArTagnan) R 11722 11724 11509 0 -1 4202560 498112 0 0 0 466067 166 0 0 25 0 8 0 804210429 5727318016 1157438 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=11724/tid=11731] ppid=11722 vsize=5593084 CPUtime=4662.23 cores=0-7
/proc/11724/task/11731/stat : 11731 (SArTagnan) R 11722 11724 11509 0 -1 4202560 254184 0 0 0 466141 82 0 0 25 0 8 0 804210429 5727318016 1157438 33554432000 4194304 4625164 140733311487760 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37221.4
Current children cumulated vsize (KiB) 5593084

[startup+4722.32 s]
/proc/loadavg: 8.16 8.08 8.01 9/474 11873
/proc/meminfo: memFree=16394548/32950928 swapFree=61858088/67111528
[pid=11724] ppid=11722 vsize=5593084 CPUtime=37700.6 cores=0-7
/proc/11724/stat : 11724 (SArTagnan) R 11722 11724 11509 0 -1 4202496 25979575 0 0 0 3765375 4684 0 0 25 0 8 0 804210428 5727318016 1159266 33554432000 4194304 4625164 140733311487760 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/11724/statm: 1398271 1159266 456 106 0 1393774 0
[pid=11724/tid=11725] ppid=11722 vsize=5593084 CPUtime=4722.35 cores=0-7
/proc/11724/task/11725/stat : 11725 (SArTagnan) R 11722 11724 11509 0 -1 4202560 223318 0 0 0 472155 80 0 0 25 0 8 0 804210429 5727318016 1159266 33554432000 4194304 4625164 140733311487760 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=11724/tid=11726] ppid=11722 vsize=5593084 CPUtime=4722.34 cores=0-7
/proc/11724/task/11726/stat : 11726 (SArTagnan) R 11722 11724 11509 0 -1 4202560 260757 0 0 0 472141 93 0 0 25 0 8 0 804210429 5727318016 1159266 33554432000 4194304 4625164 140733311487760 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=11724/tid=11727] ppid=11722 vsize=5593084 CPUtime=4720.77 cores=0-7
/proc/11724/task/11727/stat : 11727 (SArTagnan) R 11722 11724 11509 0 -1 4202560 23196207 0 0 0 468222 3855 0 0 25 0 8 0 804210429 5727318016 1159266 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=11724/tid=11728] ppid=11722 vsize=5593084 CPUtime=4722.39 cores=0-7
/proc/11724/task/11728/stat : 11728 (SArTagnan) R 11722 11724 11509 0 -1 4202560 220534 0 0 0 472154 85 0 0 25 0 8 0 804210429 5727318016 1159266 33554432000 4194304 4625164 140733311487760 18446744073709551615 4331194 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=11724/tid=11729] ppid=11722 vsize=5593084 CPUtime=4721.03 cores=0-7
/proc/11724/task/11729/stat : 11729 (SArTagnan) R 11722 11724 11509 0 -1 4202560 205199 0 0 0 472038 65 0 0 25 0 8 0 804210429 5727318016 1159266 33554432000 4194304 4625164 140733311487760 18446744073709551615 4329984 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=11724/tid=11730] ppid=11722 vsize=5593084 CPUtime=4722.35 cores=0-7
/proc/11724/task/11730/stat : 11730 (SArTagnan) R 11722 11724 11509 0 -1 4202560 498406 0 0 0 472068 167 0 0 25 0 8 0 804210429 5727318016 1159266 33554432000 4194304 4625164 140733311487760 18446744073709551615 4352047 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=11724/tid=11731] ppid=11722 vsize=5593084 CPUtime=4722.25 cores=0-7
/proc/11724/task/11731/stat : 11731 (SArTagnan) R 11722 11724 11509 0 -1 4202560 254207 0 0 0 472143 82 0 0 25 0 8 0 804210429 5727318016 1159266 33554432000 4194304 4625164 140733311487760 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37700.6
Current children cumulated vsize (KiB) 5593084

[startup+4782.3 s]
/proc/loadavg: 8.06 8.06 8.00 10/474 11875
/proc/meminfo: memFree=16380736/32950928 swapFree=61858088/67111528
[pid=11724] ppid=11722 vsize=5593084 CPUtime=38179.5 cores=0-7
/proc/11724/stat : 11724 (SArTagnan) R 11722 11724 11509 0 -1 4202496 26343248 0 0 0 3813192 4755 0 0 25 0 8 0 804210428 5727318016 1159757 33554432000 4194304 4625164 140733311487760 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/11724/statm: 1398271 1159757 364 106 0 1393774 0
[pid=11724/tid=11725] ppid=11722 vsize=5593084 CPUtime=4782.34 cores=0-7
/proc/11724/task/11725/stat : 11725 (SArTagnan) R 11722 11724 11509 0 -1 4202560 223339 0 0 0 478154 80 0 0 25 0 8 0 804210429 5727318016 1159757 33554432000 4194304 4625164 140733311487760 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=11724/tid=11726] ppid=11722 vsize=5593084 CPUtime=4782.33 cores=0-7
/proc/11724/task/11726/stat : 11726 (SArTagnan) R 11722 11724 11509 0 -1 4202560 260775 0 0 0 478140 93 0 0 25 0 8 0 804210429 5727318016 1159758 33554432000 4194304 4625164 140733311487760 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=11724/tid=11727] ppid=11722 vsize=5593084 CPUtime=4780.74 cores=0-7
/proc/11724/task/11727/stat : 11727 (SArTagnan) R 11722 11724 11509 0 -1 4202560 23538711 0 0 0 474154 3920 0 0 25 0 8 0 804210429 5727318016 1159758 33554432000 4194304 4625164 140733311487760 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=11724/tid=11728] ppid=11722 vsize=5593084 CPUtime=4782.37 cores=0-7
/proc/11724/task/11728/stat : 11728 (SArTagnan) R 11722 11724 11509 0 -1 4202560 220556 0 0 0 478152 85 0 0 25 0 8 0 804210429 5727318016 1159760 33554432000 4194304 4625164 140733311487760 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=11724/tid=11729] ppid=11722 vsize=5593084 CPUtime=4781.02 cores=0-7
/proc/11724/task/11729/stat : 11729 (SArTagnan) R 11722 11724 11509 0 -1 4202560 205218 0 0 0 478037 65 0 0 25 0 8 0 804210429 5727318016 1159760 33554432000 4194304 4625164 140733311487760 18446744073709551615 4329721 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=11724/tid=11730] ppid=11722 vsize=5593084 CPUtime=4782.33 cores=0-7
/proc/11724/task/11730/stat : 11730 (SArTagnan) R 11722 11724 11509 0 -1 4202560 499661 0 0 0 478064 169 0 0 25 0 8 0 804210429 5727318016 1159760 33554432000 4194304 4625164 140733311487760 18446744073709551615 4352285 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=11724/tid=11731] ppid=11722 vsize=5593084 CPUtime=4782.23 cores=0-7
/proc/11724/task/11731/stat : 11731 (SArTagnan) R 11722 11724 11509 0 -1 4202560 254224 0 0 0 478141 82 0 0 25 0 8 0 804210429 5727318016 1159760 33554432000 4194304 4625164 140733311487760 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38179.5
Current children cumulated vsize (KiB) 5593084

[startup+4842.3 s]
/proc/loadavg: 8.13 8.08 8.01 9/474 11876
/proc/meminfo: memFree=16354652/32950928 swapFree=61858088/67111528
[pid=11724] ppid=11722 vsize=5593084 CPUtime=38658.5 cores=0-7
/proc/11724/stat : 11724 (SArTagnan) R 11722 11724 11509 0 -1 4202496 26659658 0 0 0 3861028 4818 0 0 25 0 8 0 804210428 5727318016 1167695 33554432000 4194304 4625164 140733311487760 18446744073709551615 4512382 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/11724/statm: 1398271 1167695 500 106 0 1393774 0
[pid=11724/tid=11725] ppid=11722 vsize=5593084 CPUtime=4842.33 cores=0-7
/proc/11724/task/11725/stat : 11725 (SArTagnan) R 11722 11724 11509 0 -1 4202560 223358 0 0 0 484153 80 0 0 25 0 8 0 804210429 5727318016 1167695 33554432000 4194304 4625164 140733311487760 18446744073709551615 4488054 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=11724/tid=11726] ppid=11722 vsize=5593084 CPUtime=4842.33 cores=0-7
/proc/11724/task/11726/stat : 11726 (SArTagnan) R 11722 11724 11509 0 -1 4202560 260790 0 0 0 484140 93 0 0 25 0 8 0 804210429 5727318016 1167695 33554432000 4194304 4625164 140733311487760 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=11724/tid=11727] ppid=11722 vsize=5593084 CPUtime=4840.72 cores=0-7
/proc/11724/task/11727/stat : 11727 (SArTagnan) R 11722 11724 11509 0 -1 4202560 23833857 0 0 0 480097 3975 0 0 25 0 8 0 804210429 5727318016 1167695 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=11724/tid=11728] ppid=11722 vsize=5593084 CPUtime=4842.37 cores=0-7
/proc/11724/task/11728/stat : 11728 (SArTagnan) R 11722 11724 11509 0 -1 4202560 220577 0 0 0 484152 85 0 0 25 0 8 0 804210429 5727318016 1167695 33554432000 4194304 4625164 140733311487760 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=11724/tid=11729] ppid=11722 vsize=5593084 CPUtime=4841.02 cores=0-7
/proc/11724/task/11729/stat : 11729 (SArTagnan) R 11722 11724 11509 0 -1 4202560 205235 0 0 0 484037 65 0 0 25 0 8 0 804210429 5727318016 1167695 33554432000 4194304 4625164 140733311487760 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=11724/tid=11730] ppid=11722 vsize=5593084 CPUtime=4842.33 cores=0-7
/proc/11724/task/11730/stat : 11730 (SArTagnan) R 11722 11724 11509 0 -1 4202560 501142 0 0 0 484063 170 0 0 25 0 8 0 804210429 5727318016 1167695 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=11724/tid=11731] ppid=11722 vsize=5593084 CPUtime=4842.23 cores=0-7
/proc/11724/task/11731/stat : 11731 (SArTagnan) R 11722 11724 11509 0 -1 4202560 254241 0 0 0 484141 82 0 0 25 0 8 0 804210429 5727318016 1167695 33554432000 4194304 4625164 140733311487760 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38658.5
Current children cumulated vsize (KiB) 5593084

[startup+4902.3 s]
/proc/loadavg: 8.12 8.08 8.01 9/474 11878
/proc/meminfo: memFree=16342088/32950928 swapFree=61858088/67111528
[pid=11724] ppid=11722 vsize=5593084 CPUtime=39137.5 cores=0-7
/proc/11724/stat : 11724 (SArTagnan) R 11722 11724 11509 0 -1 4202496 26924286 0 0 0 3908875 4873 0 0 25 0 8 0 804210428 5727318016 1170778 33554432000 4194304 4625164 140733311487760 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/11724/statm: 1398271 1170778 424 106 0 1393774 0
[pid=11724/tid=11725] ppid=11722 vsize=5593084 CPUtime=4902.34 cores=0-7
/proc/11724/task/11725/stat : 11725 (SArTagnan) R 11722 11724 11509 0 -1 4202560 223381 0 0 0 490154 80 0 0 25 0 8 0 804210429 5727318016 1170778 33554432000 4194304 4625164 140733311487760 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=11724/tid=11726] ppid=11722 vsize=5593084 CPUtime=4902.34 cores=0-7
/proc/11724/task/11726/stat : 11726 (SArTagnan) R 11722 11724 11509 0 -1 4202560 260809 0 0 0 490140 94 0 0 25 0 8 0 804210429 5727318016 1170778 33554432000 4194304 4625164 140733311487760 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=11724/tid=11727] ppid=11722 vsize=5593084 CPUtime=4900.7 cores=0-7
/proc/11724/task/11727/stat : 11727 (SArTagnan) R 11722 11724 11509 0 -1 4202560 24074682 0 0 0 486047 4023 0 0 25 0 8 0 804210429 5727318016 1170778 33554432000 4194304 4625164 140733311487760 18446744073709551615 4546798 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=11724/tid=11728] ppid=11722 vsize=5593084 CPUtime=4902.38 cores=0-7
/proc/11724/task/11728/stat : 11728 (SArTagnan) R 11722 11724 11509 0 -1 4202560 220601 0 0 0 490153 85 0 0 25 0 8 0 804210429 5727318016 1170778 33554432000 4194304 4625164 140733311487760 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=11724/tid=11729] ppid=11722 vsize=5593084 CPUtime=4901.02 cores=0-7
/proc/11724/task/11729/stat : 11729 (SArTagnan) R 11722 11724 11509 0 -1 4202560 205262 0 0 0 490037 65 0 0 25 0 8 0 804210429 5727318016 1170778 33554432000 4194304 4625164 140733311487760 18446744073709551615 4329721 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=11724/tid=11730] ppid=11722 vsize=5593084 CPUtime=4902.33 cores=0-7
/proc/11724/task/11730/stat : 11730 (SArTagnan) R 11722 11724 11509 0 -1 4202560 504685 0 0 0 490062 171 0 0 25 0 8 0 804210429 5727318016 1170778 33554432000 4194304 4625164 140733311487760 18446744073709551615 4352760 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=11724/tid=11731] ppid=11722 vsize=5593084 CPUtime=4902.23 cores=0-7
/proc/11724/task/11731/stat : 11731 (SArTagnan) R 11722 11724 11509 0 -1 4202560 254261 0 0 0 490141 82 0 0 25 0 8 0 804210429 5727318016 1170778 33554432000 4194304 4625164 140733311487760 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39137.5
Current children cumulated vsize (KiB) 5593084

[startup+4962.3 s]
/proc/loadavg: 8.11 8.07 8.01 9/474 11880
/proc/meminfo: memFree=16325168/32950928 swapFree=61858088/67111528
[pid=11724] ppid=11722 vsize=5593084 CPUtime=39616.5 cores=0-7
/proc/11724/stat : 11724 (SArTagnan) R 11722 11724 11509 0 -1 4202496 27206548 0 0 0 3956713 4934 0 0 25 0 8 0 804210428 5727318016 1173430 33554432000 4194304 4625164 140733311487760 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/11724/statm: 1398271 1173430 351 106 0 1393774 0
[pid=11724/tid=11725] ppid=11722 vsize=5593084 CPUtime=4962.34 cores=0-7
/proc/11724/task/11725/stat : 11725 (SArTagnan) R 11722 11724 11509 0 -1 4202560 223404 0 0 0 496154 80 0 0 25 0 8 0 804210429 5727318016 1173430 33554432000 4194304 4625164 140733311487760 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=11724/tid=11726] ppid=11722 vsize=5593084 CPUtime=4962.34 cores=0-7
/proc/11724/task/11726/stat : 11726 (SArTagnan) R 11722 11724 11509 0 -1 4202560 260829 0 0 0 496140 94 0 0 25 0 8 0 804210429 5727318016 1173430 33554432000 4194304 4625164 140733311487760 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=11724/tid=11727] ppid=11722 vsize=5593084 CPUtime=4960.7 cores=0-7
/proc/11724/task/11727/stat : 11727 (SArTagnan) R 11722 11724 11509 0 -1 4202560 24335929 0 0 0 491993 4077 0 0 25 0 8 0 804210429 5727318016 1173430 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=11724/tid=11728] ppid=11722 vsize=5593084 CPUtime=4962.37 cores=0-7
/proc/11724/task/11728/stat : 11728 (SArTagnan) R 11722 11724 11509 0 -1 4202560 220623 0 0 0 496152 85 0 0 25 0 8 0 804210429 5727318016 1173430 33554432000 4194304 4625164 140733311487760 18446744073709551615 4488056 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=11724/tid=11729] ppid=11722 vsize=5593084 CPUtime=4961.02 cores=0-7
/proc/11724/task/11729/stat : 11729 (SArTagnan) R 11722 11724 11509 0 -1 4202560 205282 0 0 0 496037 65 0 0 25 0 8 0 804210429 5727318016 1173430 33554432000 4194304 4625164 140733311487760 18446744073709551615 4329721 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=11724/tid=11730] ppid=11722 vsize=5593084 CPUtime=4962.33 cores=0-7
/proc/11724/task/11730/stat : 11730 (SArTagnan) R 11722 11724 11509 0 -1 4202560 507419 0 0 0 496060 173 0 0 25 0 8 0 804210429 5727318016 1173430 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=11724/tid=11731] ppid=11722 vsize=5593084 CPUtime=4962.23 cores=0-7
/proc/11724/task/11731/stat : 11731 (SArTagnan) R 11722 11724 11509 0 -1 4202560 254282 0 0 0 496141 82 0 0 25 0 8 0 804210429 5727318016 1173430 33554432000 4194304 4625164 140733311487760 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39616.5
Current children cumulated vsize (KiB) 5593084



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.4 s]
/proc/loadavg: 8.12 8.08 8.01 10/474 11881
/proc/meminfo: memFree=16311112/32950928 swapFree=61858088/67111528
[pid=11724] ppid=11722 vsize=5593084 CPUtime=40000.7 cores=0-7
/proc/11724/stat : 11724 (SArTagnan) R 11722 11724 11509 0 -1 4202496 27452106 0 0 0 3995085 4984 0 0 25 0 8 0 804210428 5727318016 1178344 33554432000 4194304 4625164 140733311487760 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/11724/statm: 1398271 1178344 351 106 0 1393774 0
[pid=11724/tid=11725] ppid=11722 vsize=5593084 CPUtime=5010.47 cores=0-7
/proc/11724/task/11725/stat : 11725 (SArTagnan) R 11722 11724 11509 0 -1 4202560 223424 0 0 0 500967 80 0 0 25 0 8 0 804210429 5727318016 1176952 33554432000 4194304 4625164 140733311487760 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=11724/tid=11726] ppid=11722 vsize=5593084 CPUtime=5010.48 cores=0-7
/proc/11724/task/11726/stat : 11726 (SArTagnan) R 11722 11724 11509 0 -1 4202560 260838 0 0 0 500954 94 0 0 25 0 8 0 804210429 5727318016 1176952 33554432000 4194304 4625164 140733311487760 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=11724/tid=11727] ppid=11722 vsize=5593084 CPUtime=5008.82 cores=0-7
/proc/11724/task/11727/stat : 11727 (SArTagnan) R 11722 11724 11509 0 -1 4202560 24559541 0 0 0 496761 4121 0 0 25 0 8 0 804210429 5727318016 1176952 33554432000 4194304 4625164 140733311487760 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=11724/tid=11728] ppid=11722 vsize=5593084 CPUtime=5010.51 cores=0-7
/proc/11724/task/11728/stat : 11728 (SArTagnan) R 11722 11724 11509 0 -1 4202560 220643 0 0 0 500966 85 0 0 25 0 8 0 804210429 5727318016 1176952 33554432000 4194304 4625164 140733311487760 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=11724/tid=11729] ppid=11722 vsize=5593084 CPUtime=5009.16 cores=0-7
/proc/11724/task/11729/stat : 11729 (SArTagnan) R 11722 11724 11509 0 -1 4202560 205302 0 0 0 500851 65 0 0 25 0 8 0 804210429 5727318016 1176952 33554432000 4194304 4625164 140733311487760 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=11724/tid=11730] ppid=11722 vsize=5593084 CPUtime=5010.47 cores=0-7
/proc/11724/task/11730/stat : 11730 (SArTagnan) R 11722 11724 11509 0 -1 4202560 511665 0 0 0 500872 175 0 0 25 0 8 0 804210429 5727318016 1176952 33554432000 4194304 4625164 140733311487760 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=11724/tid=11731] ppid=11722 vsize=5593084 CPUtime=5010.36 cores=0-7
/proc/11724/task/11731/stat : 11731 (SArTagnan) R 11722 11724 11509 0 -1 4202560 254299 0 0 0 500954 82 0 0 25 0 8 0 804210429 5727318016 1176952 33554432000 4194304 4625164 140733311487760 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 5593084

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.6 s]
/proc/loadavg: 8.12 8.08 8.01 10/474 11881
/proc/meminfo: memFree=16311112/32950928 swapFree=61858088/67111528
[pid=11724] ppid=11722 vsize=0 CPUtime=40000.9 cores=0-7
/proc/11724/stat : 11724 (SArTagnan) R 11722 11724 11509 0 -1 4203524 27452106 0 0 0 3995096 4999 0 0 25 0 1 0 804210428 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/11724/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5010.69
CPU time (s): 40001.1
CPU user time (s): 39951
CPU system time (s): 50.0904
CPU usage (%): 798.314
Max. virtual memory (cumulated for all children) (KiB): 5911396

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39951
system time used= 50.0904
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 27452106
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2902
involuntary context switches= 169082

runsolver used 20.4179 second user time and 59.6829 second system time

The end

Launcher Data 

Begin job on node125 at 2011-06-01 13:01:48
IDJOB=3439066
IDBENCH=83250
IDSOLVER=1936
FILE ID=node125/3439066-1306926108
RUNJOBID= node125-1306921095-11523
PBS_JOBID= 13488549
Free space on /tmp= 73804 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT11/crafted/kullmann/GreenTao/GreenTaok1k2/GreenTao_2-4-4_511.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3439066-1306926108/watcher-3439066-1306926108 -o /tmp/evaluation-result-3439066-1306926108/solver-3439066-1306926108 -C 40000 -W 5100 -M 31000  SArTagnan HOME/instance-3439066-1306926108.cnf -threads=8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 766a362c83d920a8f8dd49df62ed42c9
RANDOM SEED=471018734

node125.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5266.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      21033284 kB
Buffers:        297788 kB
Cached:        2917192 kB
SwapCached:       3584 kB
Active:        8581448 kB
Inactive:      3002184 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      21033284 kB
SwapTotal:    67111528 kB
SwapFree:     61858088 kB
Dirty:             400 kB
Writeback:           0 kB
AnonPages:     8367180 kB
Mapped:          33696 kB
Slab:           201552 kB
PageTables:      61308 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 130003456 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73804 MiB
End job on node125 at 2011-06-01 14:25:20