Trace number 3385987

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 4800.46 1204.26

General information on the benchmark

NameSAT04/handmade/connamacher/connm-d0.02/
connm-ue-csp-sat-n800-d0.02-s111845174.used-as.sat04-958.cnf
MD5SUM7bb85ebdeae46262f72a4c54d4e16683
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables772
Number of clauses9601
Sum of the clauses size28996
Maximum clause length4
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 39408
Number of clauses of size 4193
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385987-1305582762/watcher-3385987-1305582762 -o /tmp/evaluation-result-3385987-1305582762/solver-3385987-1305582762 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3385987-1305582762.cnf -threads=4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.09 8.11 7.95 5/401 2301
/proc/meminfo: memFree=12606860/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=19340 CPUtime=0 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 631 0 0 0 0 0 0 0 25 0 1 0 669875872 19804160 536 33554432000 4194304 4625164 140734696737200 18446744073709551615 4339447 0 0 4096 2 0 0 0 17 3 0 0 0
/proc/2301/statm: 4835 537 296 106 0 338 0
[pid=2301/tid=2302] ppid=2299 vsize=51732 CPUtime=0 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 4 0 669875872 52973568 980 33554432000 4194304 4625164 140734696737200 18446744073709551615 208080288753 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=51732 CPUtime=0 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 4 0 669875872 52973568 980 33554432000 4194304 4625164 140734696737200 18446744073709551615 47558634801074 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=51732 CPUtime=0 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 4 0 669875872 52973568 980 33554432000 4194304 4625164 140734696737200 18446744073709551615 47558634801078 0 0 4096 2 0 0 0 -1 5 0 0 0

[startup+0.163915 s]
/proc/loadavg: 8.09 8.11 7.95 5/401 2301
/proc/meminfo: memFree=12606860/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=185112 CPUtime=0.61 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 2813 0 0 0 61 0 0 0 25 0 4 0 669875872 189554688 1940 33554432000 4194304 4625164 140734696737200 18446744073709551615 4512046 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 46278 1940 342 106 0 41781 0
[pid=2301/tid=2302] ppid=2299 vsize=185112 CPUtime=0.14 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 319 0 0 0 14 0 0 0 25 0 4 0 669875872 189554688 1940 33554432000 4194304 4625164 140734696737200 18446744073709551615 4352653 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=185112 CPUtime=0.15 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 308 0 0 0 15 0 0 0 25 0 4 0 669875872 189554688 1940 33554432000 4194304 4625164 140734696737200 18446744073709551615 4352072 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=185112 CPUtime=0.15 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 665 0 0 0 15 0 0 0 25 0 4 0 669875872 189554688 1940 33554432000 4194304 4625164 140734696737200 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 0.61
Current children cumulated vsize (KiB) 185112

[startup+0.200908 s]
/proc/loadavg: 8.09 8.11 7.95 5/401 2301
/proc/meminfo: memFree=12606860/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=186008 CPUtime=0.76 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 3130 0 0 0 76 0 0 0 25 0 4 0 669875872 190472192 2121 33554432000 4194304 4625164 140734696737200 18446744073709551615 4396709 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 46502 2121 342 106 0 42005 0
[pid=2301/tid=2302] ppid=2299 vsize=186008 CPUtime=0.18 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 335 0 0 0 18 0 0 0 25 0 4 0 669875872 190472192 2121 33554432000 4194304 4625164 140734696737200 18446744073709551615 4288793 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=186008 CPUtime=0.19 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 321 0 0 0 19 0 0 0 25 0 4 0 669875872 190472192 2121 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=186008 CPUtime=0.19 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 891 0 0 0 19 0 0 0 25 0 4 0 669875872 190472192 2121 33554432000 4194304 4625164 140734696737200 18446744073709551615 4339062 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 0.76
Current children cumulated vsize (KiB) 186008

[startup+0.30089 s]
/proc/loadavg: 8.09 8.11 7.95 5/401 2301
/proc/meminfo: memFree=12606860/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=186524 CPUtime=1.16 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 3687 0 0 0 116 0 0 0 25 0 4 0 669875872 191000576 2423 33554432000 4194304 4625164 140734696737200 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 46631 2423 342 106 0 42134 0
[pid=2301/tid=2302] ppid=2299 vsize=186524 CPUtime=0.28 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 536 0 0 0 28 0 0 0 25 0 4 0 669875872 191000576 2423 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351839 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=186524 CPUtime=0.29 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 355 0 0 0 29 0 0 0 25 0 4 0 669875872 191000576 2423 33554432000 4194304 4625164 140734696737200 18446744073709551615 4352629 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=186524 CPUtime=0.29 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 1186 0 0 0 29 0 0 0 25 0 4 0 669875872 191000576 2423 33554432000 4194304 4625164 140734696737200 18446744073709551615 4548821 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1.16
Current children cumulated vsize (KiB) 186524

[startup+0.700813 s]
/proc/loadavg: 8.09 8.11 7.95 5/401 2301
/proc/meminfo: memFree=12606860/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=188676 CPUtime=2.75 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 6815 0 0 0 275 0 0 0 25 0 4 0 669875872 193204224 3691 33554432000 4194304 4625164 140734696737200 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 47169 3691 345 106 0 42672 0
[pid=2301/tid=2302] ppid=2299 vsize=188676 CPUtime=0.68 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 986 0 0 0 68 0 0 0 25 0 4 0 669875872 193204224 3691 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=188676 CPUtime=0.68 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 1029 0 0 0 68 0 0 0 25 0 4 0 669875872 193204224 3691 33554432000 4194304 4625164 140734696737200 18446744073709551615 4352035 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=188676 CPUtime=0.68 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 2850 0 0 0 68 0 0 0 25 0 4 0 669875872 193204224 3691 33554432000 4194304 4625164 140734696737200 18446744073709551615 4547248 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2.75
Current children cumulated vsize (KiB) 188676

[startup+1.50066 s]
/proc/loadavg: 8.09 8.11 7.95 9/405 2305
/proc/meminfo: memFree=12582564/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=192492 CPUtime=5.94 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 11381 0 0 0 593 1 0 0 25 0 4 0 669875872 197111808 5598 33554432000 4194304 4625164 140734696737200 18446744073709551615 4512818 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 48123 5598 345 106 0 43626 0
[pid=2301/tid=2302] ppid=2299 vsize=192492 CPUtime=1.46 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 1552 0 0 0 146 0 0 0 25 0 4 0 669875872 197111808 5598 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351814 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=192492 CPUtime=1.48 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 1572 0 0 0 148 0 0 0 25 0 4 0 669875872 197111808 5598 33554432000 4194304 4625164 140734696737200 18446744073709551615 4352760 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=192492 CPUtime=1.48 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 5949 0 0 0 148 0 0 0 25 0 4 0 669875872 197111808 5598 33554432000 4194304 4625164 140734696737200 18446744073709551615 4579595 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 5.94
Current children cumulated vsize (KiB) 192492

[startup+3.10036 s]
/proc/loadavg: 8.09 8.11 7.95 9/405 2305
/proc/meminfo: memFree=12566816/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=197084 CPUtime=12.33 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 20436 0 0 0 1231 2 0 0 25 0 4 0 669875872 201814016 8365 33554432000 4194304 4625164 140734696737200 18446744073709551615 4512066 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 49271 8365 346 106 0 44774 0
[pid=2301/tid=2302] ppid=2299 vsize=197084 CPUtime=3.05 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 2510 0 0 0 305 0 0 0 25 0 4 0 669875872 201814016 8365 33554432000 4194304 4625164 140734696737200 18446744073709551615 4352072 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=197084 CPUtime=3.08 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 2480 0 0 0 308 0 0 0 25 0 4 0 669875872 201814016 8365 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=197084 CPUtime=3.08 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 12701 0 0 0 307 1 0 0 25 0 4 0 669875872 201814016 8365 33554432000 4194304 4625164 140734696737200 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 12.33
Current children cumulated vsize (KiB) 197084

[startup+6.30075 s]
/proc/loadavg: 8.08 8.11 7.95 9/405 2305
/proc/meminfo: memFree=12538644/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=208196 CPUtime=25.09 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 35426 0 0 0 2506 3 0 0 25 0 4 0 669875872 213192704 11993 33554432000 4194304 4625164 140734696737200 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 52049 11993 346 106 0 47552 0
[pid=2301/tid=2302] ppid=2299 vsize=208196 CPUtime=6.21 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 3774 0 0 0 621 0 0 0 25 0 4 0 669875872 213192704 11993 33554432000 4194304 4625164 140734696737200 18446744073709551615 4339591 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=208196 CPUtime=6.28 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 3753 0 0 0 628 0 0 0 25 0 4 0 669875872 213192704 11993 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=208196 CPUtime=6.28 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 24310 0 0 0 626 2 0 0 25 0 4 0 669875872 213192704 11993 33554432000 4194304 4625164 140734696737200 18446744073709551615 4552432 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 25.09
Current children cumulated vsize (KiB) 208196

[startup+12.7005 s]
/proc/loadavg: 8.08 8.11 7.95 9/405 2305
/proc/meminfo: memFree=12502248/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=218876 CPUtime=50.61 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 65182 0 0 0 5053 8 0 0 25 0 4 0 669875872 224129024 16664 33554432000 4194304 4625164 140734696737200 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 54719 16664 346 106 0 50222 0
[pid=2301/tid=2302] ppid=2299 vsize=218876 CPUtime=12.52 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 5929 0 0 0 1252 0 0 0 25 0 4 0 669875872 224129024 16664 33554432000 4194304 4625164 140734696737200 18446744073709551615 4339536 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=218876 CPUtime=12.68 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 5879 0 0 0 1268 0 0 0 25 0 4 0 669875872 224129024 16664 33554432000 4194304 4625164 140734696737200 18446744073709551615 4339581 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=218876 CPUtime=12.69 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 48663 0 0 0 1263 6 0 0 25 0 4 0 669875872 224129024 16664 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351786 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 50.61
Current children cumulated vsize (KiB) 218876

[startup+25.5011 s]
/proc/loadavg: 8.22 8.14 7.96 9/405 2306
/proc/meminfo: memFree=12456724/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=232292 CPUtime=101.62 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 131623 0 0 0 10142 20 0 0 25 0 4 0 669875872 237867008 23691 33554432000 4194304 4625164 140734696737200 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 58073 23691 347 106 0 53576 0
[pid=2301/tid=2302] ppid=2299 vsize=232292 CPUtime=25.16 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 8777 0 0 0 2515 1 0 0 25 0 4 0 669875872 237867008 23691 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=232292 CPUtime=25.47 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 8768 0 0 0 2546 1 0 0 25 0 4 0 669875872 237867008 23691 33554432000 4194304 4625164 140734696737200 18446744073709551615 4352930 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=232292 CPUtime=25.48 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 107311 0 0 0 2532 16 0 0 25 0 4 0 669875872 237867008 23691 33554432000 4194304 4625164 140734696737200 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 101.62
Current children cumulated vsize (KiB) 232292

[startup+51.1012 s]
/proc/loadavg: 8.14 8.13 7.96 9/405 2307
/proc/meminfo: memFree=12377100/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=320152 CPUtime=203.71 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 203697 0 0 0 20337 34 0 0 25 0 4 0 669875872 327835648 35318 33554432000 4194304 4625164 140734696737200 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 80038 35318 395 106 0 75541 0
[pid=2301/tid=2302] ppid=2299 vsize=320152 CPUtime=50.43 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 11032 0 0 0 5041 2 0 0 25 0 4 0 669875872 327835648 35318 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=320152 CPUtime=51.08 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 13622 0 0 0 5106 2 0 0 25 0 4 0 669875872 327835648 35318 33554432000 4194304 4625164 140734696737200 18446744073709551615 4353004 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=320152 CPUtime=51.08 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 167214 0 0 0 5081 27 0 0 25 0 4 0 669875872 327835648 35318 33554432000 4194304 4625164 140734696737200 18446744073709551615 4352112 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 203.71
Current children cumulated vsize (KiB) 320152

[startup+102.3 s]
/proc/loadavg: 8.24 8.15 7.97 9/405 2308
/proc/meminfo: memFree=12259140/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=341240 CPUtime=407.88 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 373709 0 0 0 40723 65 0 0 25 0 4 0 669875872 349429760 53629 33554432000 4194304 4625164 140734696737200 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 85310 53629 603 106 0 80813 0
[pid=2301/tid=2302] ppid=2299 vsize=341240 CPUtime=100.99 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 15334 0 0 0 10096 3 0 0 25 0 4 0 669875872 349429760 53654 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351792 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=341240 CPUtime=102.27 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 21267 0 0 0 10222 5 0 0 25 0 4 0 669875872 349429760 53654 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=341240 CPUtime=102.3 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 311553 0 0 0 10179 51 0 0 25 0 4 0 669875872 349429760 53654 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 407.88
Current children cumulated vsize (KiB) 341240

[startup+162.301 s]

################
# More data... #
################

[pid=2301] ppid=2299 vsize=884552 CPUtime=2799.45 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 3282308 0 0 0 279378 567 0 0 25 0 4 0 669875872 905781248 165508 33554432000 4194304 4625164 140734696737200 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 221138 165508 592 106 0 216641 0
[pid=2301/tid=2302] ppid=2299 vsize=884552 CPUtime=693.32 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 39042 0 0 0 69319 13 0 0 25 0 4 0 669875872 905781248 165508 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351792 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=884552 CPUtime=702.14 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 71425 0 0 0 70195 19 0 0 25 0 4 0 669875872 905781248 165508 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=884552 CPUtime=701.65 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 3028156 0 0 0 69657 508 0 0 25 0 4 0 669875872 905781248 165508 33554432000 4194304 4625164 140734696737200 18446744073709551615 4334499 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2799.45
Current children cumulated vsize (KiB) 884552

[startup+762.301 s]
/proc/loadavg: 8.32 8.21 8.06 9/405 2398
/proc/meminfo: memFree=11092208/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=885576 CPUtime=3038.5 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 3644255 0 0 0 303223 627 0 0 25 0 4 0 669875872 906829824 174490 33554432000 4194304 4625164 140734696737200 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 221394 174490 599 106 0 216897 0
[pid=2301/tid=2302] ppid=2299 vsize=885576 CPUtime=752.57 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 42003 0 0 0 75243 14 0 0 25 0 4 0 669875872 906829824 174491 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=885576 CPUtime=762.15 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 72898 0 0 0 76195 20 0 0 25 0 4 0 669875872 906829824 174491 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=885576 CPUtime=761.48 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 3383656 0 0 0 75581 567 0 0 25 0 4 0 669875872 906829824 174491 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3038.5
Current children cumulated vsize (KiB) 885576

[startup+822.3 s]
/proc/loadavg: 8.15 8.18 8.06 9/405 2400
/proc/meminfo: memFree=11050748/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=886208 CPUtime=3277.71 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 3993449 0 0 0 327089 682 0 0 25 0 4 0 669875872 907476992 179027 33554432000 4194304 4625164 140734696737200 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 221552 179027 596 106 0 217055 0
[pid=2301/tid=2302] ppid=2299 vsize=886208 CPUtime=811.79 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 42175 0 0 0 81165 14 0 0 25 0 4 0 669875872 907476992 179027 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=886208 CPUtime=822.12 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 73772 0 0 0 82192 20 0 0 25 0 4 0 669875872 907476992 179027 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=886208 CPUtime=821.46 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 3729821 0 0 0 81525 621 0 0 25 0 4 0 669875872 907476992 179027 33554432000 4194304 4625164 140734696737200 18446744073709551615 4335179 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3277.71
Current children cumulated vsize (KiB) 886208

[startup+882.316 s]
/proc/loadavg: 8.05 8.15 8.05 9/405 2402
/proc/meminfo: memFree=10999232/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=1063852 CPUtime=3516.98 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 4378151 0 0 0 350954 744 0 0 25 0 4 0 669875872 1089384448 191375 33554432000 4194304 4625164 140734696737200 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 265963 191375 595 106 0 261466 0
[pid=2301/tid=2302] ppid=2299 vsize=1063852 CPUtime=871.03 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 42321 0 0 0 87089 14 0 0 25 0 4 0 669875872 1089384448 191375 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=1063852 CPUtime=882.12 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 75474 0 0 0 88192 20 0 0 25 0 4 0 669875872 1089384448 191375 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351827 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=1063852 CPUtime=881.47 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 4060061 0 0 0 87472 675 0 0 25 0 4 0 669875872 1089384448 191375 33554432000 4194304 4625164 140734696737200 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3516.98
Current children cumulated vsize (KiB) 1063852

[startup+942.3 s]
/proc/loadavg: 8.08 8.13 8.05 9/405 2403
/proc/meminfo: memFree=10966568/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=1063852 CPUtime=3756.15 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 4708580 0 0 0 374815 800 0 0 25 0 4 0 669875872 1089384448 194163 33554432000 4194304 4625164 140734696737200 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 265963 194163 571 106 0 261466 0
[pid=2301/tid=2302] ppid=2299 vsize=1063852 CPUtime=930.25 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 42507 0 0 0 93011 14 0 0 25 0 4 0 669875872 1089384448 194165 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=1063852 CPUtime=942.1 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 75741 0 0 0 94189 21 0 0 25 0 4 0 669875872 1089384448 194165 33554432000 4194304 4625164 140734696737200 18446744073709551615 4339591 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=1063852 CPUtime=941.45 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 4389091 0 0 0 93415 730 0 0 25 0 4 0 669875872 1089384448 194165 33554432000 4194304 4625164 140734696737200 18446744073709551615 4579598 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3756.15
Current children cumulated vsize (KiB) 1063852

[startup+1002.3 s]
/proc/loadavg: 8.09 8.12 8.05 9/405 2405
/proc/meminfo: memFree=10918032/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=1065852 CPUtime=3995.37 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 5056259 0 0 0 398681 856 0 0 25 0 4 0 669875872 1091432448 200692 33554432000 4194304 4625164 140734696737200 18446744073709551615 4511950 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 266463 200692 490 106 0 261966 0
[pid=2301/tid=2302] ppid=2299 vsize=1065852 CPUtime=989.47 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 43714 0 0 0 98933 14 0 0 25 0 4 0 669875872 1091432448 200692 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351792 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=1065852 CPUtime=1002.09 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 76517 0 0 0 100188 21 0 0 25 0 4 0 669875872 1091432448 200692 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=1065852 CPUtime=1001.45 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 4732157 0 0 0 99360 785 0 0 25 0 4 0 669875872 1091432448 200692 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3995.37
Current children cumulated vsize (KiB) 1065852

[startup+1062.3 s]
/proc/loadavg: 8.09 8.11 8.05 9/405 2406
/proc/meminfo: memFree=10879896/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=1066960 CPUtime=4234.59 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 5398220 0 0 0 422550 909 0 0 25 0 4 0 669875872 1092567040 205135 33554432000 4194304 4625164 140734696737200 18446744073709551615 4512190 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 266740 205135 352 106 0 262243 0
[pid=2301/tid=2302] ppid=2299 vsize=1066960 CPUtime=1048.71 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 44435 0 0 0 104855 16 0 0 25 0 4 0 669875872 1092567040 205135 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351786 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=1066960 CPUtime=1062.07 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 76775 0 0 0 106186 21 0 0 25 0 4 0 669875872 1092567040 205135 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=1066960 CPUtime=1061.45 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 5071545 0 0 0 105309 836 0 0 25 0 4 0 669875872 1092567040 205135 33554432000 4194304 4625164 140734696737200 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4234.59
Current children cumulated vsize (KiB) 1066960

[startup+1122.3 s]
/proc/loadavg: 8.08 8.11 8.05 9/405 2408
/proc/meminfo: memFree=10803580/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=1067700 CPUtime=4473.81 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 5733845 0 0 0 446415 966 0 0 25 0 4 0 669875872 1093324800 211137 33554432000 4194304 4625164 140734696737200 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 266925 211139 352 106 0 262428 0
[pid=2301/tid=2302] ppid=2299 vsize=1067700 CPUtime=1107.94 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 45160 0 0 0 110777 17 0 0 25 0 4 0 669875872 1093324800 211139 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=1067700 CPUtime=1122.07 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 78235 0 0 0 112184 23 0 0 25 0 4 0 669875872 1093324800 211139 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=1067700 CPUtime=1121.46 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 5402968 0 0 0 111256 890 0 0 25 0 4 0 669875872 1093324800 211139 33554432000 4194304 4625164 140734696737200 18446744073709551615 4579598 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4473.81
Current children cumulated vsize (KiB) 1067700

[startup+1182.3 s]
/proc/loadavg: 8.08 8.10 8.05 9/405 2410
/proc/meminfo: memFree=10737676/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=1069712 CPUtime=4713.05 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 6070026 0 0 0 470283 1022 0 0 25 0 4 0 669875872 1095385088 218033 33554432000 4194304 4625164 140734696737200 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 267428 218033 348 106 0 262931 0
[pid=2301/tid=2302] ppid=2299 vsize=1069712 CPUtime=1167.18 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 45929 0 0 0 116700 18 0 0 25 0 4 0 669875872 1095385088 218033 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=1069712 CPUtime=1182.05 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 79219 0 0 0 118181 24 0 0 25 0 4 0 669875872 1095385088 218033 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=1069712 CPUtime=1181.46 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 5733761 0 0 0 117204 942 0 0 25 0 4 0 669875872 1095385088 218033 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4713.05
Current children cumulated vsize (KiB) 1069712



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1204.2 s]
/proc/loadavg: 8.13 8.11 8.05 9/405 2410
/proc/meminfo: memFree=10711012/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=1070240 CPUtime=4800.37 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 6190698 0 0 0 478997 1040 0 0 25 0 4 0 669875872 1095925760 220434 33554432000 4194304 4625164 140734696737200 18446744073709551615 4511953 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 267560 220434 350 106 0 263063 0
[pid=2301/tid=2302] ppid=2299 vsize=1070240 CPUtime=1188.79 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 46127 0 0 0 118861 18 0 0 25 0 4 0 669875872 1095925760 220438 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=1070240 CPUtime=1203.96 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 79510 0 0 0 120371 25 0 0 25 0 4 0 669875872 1095925760 220438 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=1070240 CPUtime=1203.37 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 5853151 0 0 0 119378 959 0 0 25 0 4 0 669875872 1095925760 220438 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.37
Current children cumulated vsize (KiB) 1070240

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1204.2 s]
/proc/loadavg: 8.13 8.11 8.05 9/405 2410
/proc/meminfo: memFree=10711012/32950928 swapFree=57376972/67111528
[pid=2301] ppid=2299 vsize=1070240 CPUtime=4800.37 cores=1,3,5,7
/proc/2301/stat : 2301 (SArTagnan) R 2299 2301 1658 0 -1 4202496 6190698 0 0 0 478997 1040 0 0 25 0 4 0 669875872 1095925760 220434 33554432000 4194304 4625164 140734696737200 18446744073709551615 4511953 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/2301/statm: 267560 220434 350 106 0 263063 0
[pid=2301/tid=2302] ppid=2299 vsize=1070240 CPUtime=1188.79 cores=1,3,5,7
/proc/2301/task/2302/stat : 2302 (SArTagnan) R 2299 2301 1658 0 -1 4202560 46127 0 0 0 118861 18 0 0 25 0 4 0 669875872 1095925760 220438 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=2301/tid=2303] ppid=2299 vsize=1070240 CPUtime=1203.96 cores=1,3,5,7
/proc/2301/task/2303/stat : 2303 (SArTagnan) R 2299 2301 1658 0 -1 4202560 79510 0 0 0 120371 25 0 0 25 0 4 0 669875872 1095925760 220438 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=2301/tid=2304] ppid=2299 vsize=1070240 CPUtime=1203.37 cores=1,3,5,7
/proc/2301/task/2304/stat : 2304 (SArTagnan) R 2299 2301 1658 0 -1 4202560 5853151 0 0 0 119378 959 0 0 25 0 4 0 669875872 1095925760 220438 33554432000 4194304 4625164 140734696737200 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.37
Current children cumulated vsize (KiB) 1070240

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1204.26
CPU time (s): 4800.46
CPU user time (s): 4790.02
CPU system time (s): 10.4434
CPU usage (%): 398.624
Max. virtual memory (cumulated for all children) (KiB): 1153988

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4790.02
system time used= 10.4434
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6190702
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 756
involuntary context switches= 32085

runsolver used 4.23536 second user time and 11.2113 second system time

The end

Launcher Data 

Begin job on node127 at 2011-05-16 23:52:42
IDJOB=3385987
IDBENCH=84608
IDSOLVER=1871
FILE ID=node127/3385987-1305582762
RUNJOBID= node127-1305581554-2114
PBS_JOBID= 13350216
Free space on /tmp= 73384 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT04/handmade/connamacher/connm-d0.02/connm-ue-csp-sat-n800-d0.02-s111845174.used-as.sat04-958.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385987-1305582762/watcher-3385987-1305582762 -o /tmp/evaluation-result-3385987-1305582762/solver-3385987-1305582762 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3385987-1305582762.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 7bb85ebdeae46262f72a4c54d4e16683
RANDOM SEED=490019913

node127.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.60
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      12607140 kB
Buffers:         84248 kB
Cached:         469816 kB
SwapCached:     117904 kB
Active:        5570636 kB
Inactive:     14553248 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      12607140 kB
SwapTotal:    67111528 kB
SwapFree:     57376972 kB
Dirty:             964 kB
Writeback:           0 kB
AnonPages:    19457928 kB
Mapped:          27124 kB
Slab:            72488 kB
PageTables:      81288 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 84686952 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73364 MiB
End job on node127 at 2011-05-17 00:12:48