Trace number 3385986

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 4091.28 1300.06

General information on the benchmark

NameSAT03/handmade/stanion/hwb/
hwb-n28-02-S818962541.shuffled-as.sat03-1628.cnf
MD5SUMd705043763df769aab1e385bbbc2b11e
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables184
Number of clauses896
Sum of the clauses size3042
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 214
Number of clauses of size 3514
Number of clauses of size 4368
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385986-1305582448/watcher-3385986-1305582448 -o /tmp/evaluation-result-3385986-1305582448/solver-3385986-1305582448 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3385986-1305582448.cnf -threads=4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.75 6.31 7.82 4/384 4352
/proc/meminfo: memFree=21065844/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=49364 CPUtime=0 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 484 0 0 0 0 0 0 0 24 0 4 0 669842731 50548736 421 33554432000 4194304 4625164 140736206272960 18446744073709551615 47050915719080 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/4352/statm: 12341 421 300 106 0 7844 0
[pid=4352/tid=4355] ppid=4350 vsize=49364 CPUtime=0 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 4 0 669842731 50548736 421 33554432000 4194304 4625164 140736206272960 18446744073709551615 228122770417 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=49364 CPUtime=0 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 4 0 669842731 50548736 421 33554432000 4194304 4625164 140736206272960 18446744073709551615 228134511670 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=49364 CPUtime=0 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 4 0 669842731 50548736 421 33554432000 4194304 4625164 140736206272960 18446744073709551615 47050915719090 0 0 4096 2 0 0 0 -1 4 0 0 0

[startup+0.0902979 s]
/proc/loadavg: 5.75 6.31 7.82 4/384 4352
/proc/meminfo: memFree=21065844/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=181496 CPUtime=0.24 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 1238 0 0 0 24 0 0 0 25 0 4 0 669842731 185851904 1025 33554432000 4194304 4625164 140736206272960 18446744073709551615 228122819103 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/4352/statm: 45374 1025 345 106 0 40877 0
[pid=4352/tid=4355] ppid=4350 vsize=181496 CPUtime=0.04 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 166 0 0 0 4 0 0 0 25 0 4 0 669842731 185851904 1025 33554432000 4194304 4625164 140736206272960 18446744073709551615 4462417 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=181496 CPUtime=0.07 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 227 0 0 0 7 0 0 0 25 0 4 0 669842731 185851904 1025 33554432000 4194304 4625164 140736206272960 18446744073709551615 228122819054 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=181496 CPUtime=0.08 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 185 0 0 0 8 0 0 0 25 0 4 0 669842731 185851904 1025 33554432000 4194304 4625164 140736206272960 18446744073709551615 4579399 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 0.24
Current children cumulated vsize (KiB) 181496

[startup+0.139114 s]
/proc/loadavg: 5.75 6.31 7.82 4/384 4352
/proc/meminfo: memFree=21065844/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=181840 CPUtime=0.39 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 1405 0 0 0 39 0 0 0 25 0 4 0 669842731 186204160 1180 33554432000 4194304 4625164 140736206272960 18446744073709551615 4519403 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/4352/statm: 45460 1180 345 106 0 40963 0
[pid=4352/tid=4355] ppid=4350 vsize=181840 CPUtime=0.04 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 166 0 0 0 4 0 0 0 25 0 4 0 669842731 186204160 1180 33554432000 4194304 4625164 140736206272960 18446744073709551615 4462417 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=181840 CPUtime=0.12 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 266 0 0 0 12 0 0 0 25 0 4 0 669842731 186204160 1180 33554432000 4194304 4625164 140736206272960 18446744073709551615 4465732 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=181840 CPUtime=0.13 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 237 0 0 0 13 0 0 0 25 0 4 0 669842731 186204160 1180 33554432000 4194304 4625164 140736206272960 18446744073709551615 4352053 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 0.39
Current children cumulated vsize (KiB) 181840

[startup+0.301085 s]
/proc/loadavg: 5.75 6.31 7.82 4/384 4352
/proc/meminfo: memFree=21065844/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=183132 CPUtime=0.87 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 2249 0 0 0 87 0 0 0 25 0 4 0 669842731 187527168 1612 33554432000 4194304 4625164 140736206272960 18446744073709551615 4329153 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/4352/statm: 45783 1612 345 106 0 41286 0
[pid=4352/tid=4355] ppid=4350 vsize=183132 CPUtime=0.08 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 249 0 0 0 8 0 0 0 25 0 4 0 669842731 187527168 1612 33554432000 4194304 4625164 140736206272960 18446744073709551615 4352208 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=183132 CPUtime=0.28 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 605 0 0 0 28 0 0 0 25 0 4 0 669842731 187527168 1612 33554432000 4194304 4625164 140736206272960 18446744073709551615 4352612 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=183132 CPUtime=0.29 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 549 0 0 0 29 0 0 0 25 0 4 0 669842731 187527168 1612 33554432000 4194304 4625164 140736206272960 18446744073709551615 4552377 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 0.87
Current children cumulated vsize (KiB) 183132

[startup+0.701013 s]
/proc/loadavg: 5.75 6.31 7.82 4/384 4352
/proc/meminfo: memFree=21065844/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=186064 CPUtime=2.07 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 4234 0 0 0 207 0 0 0 25 0 4 0 669842731 190529536 2800 33554432000 4194304 4625164 140736206272960 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/4352/statm: 46516 2800 348 106 0 42019 0
[pid=4352/tid=4355] ppid=4350 vsize=186064 CPUtime=0.28 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 536 0 0 0 28 0 0 0 25 0 4 0 669842731 190529536 2800 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=186064 CPUtime=0.68 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 1467 0 0 0 68 0 0 0 25 0 4 0 669842731 190529536 2800 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=186064 CPUtime=0.69 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 1185 0 0 0 69 0 0 0 25 0 4 0 669842731 190529536 2800 33554432000 4194304 4625164 140736206272960 18446744073709551615 4552432 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2.07
Current children cumulated vsize (KiB) 186064

[startup+1.50087 s]
/proc/loadavg: 6.09 6.37 7.83 11/396 4367
/proc/meminfo: memFree=21036628/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=189816 CPUtime=4.46 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 7085 0 0 0 445 1 0 0 25 0 4 0 669842731 194371584 4552 33554432000 4194304 4625164 140736206272960 18446744073709551615 4519559 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/4352/statm: 47454 4552 347 106 0 42957 0
[pid=4352/tid=4355] ppid=4350 vsize=189816 CPUtime=0.68 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 1240 0 0 0 68 0 0 0 25 0 4 0 669842731 194371584 4552 33554432000 4194304 4625164 140736206272960 18446744073709551615 228122819054 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=189816 CPUtime=1.46 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 2429 0 0 0 146 0 0 0 25 0 4 0 669842731 194371584 4552 33554432000 4194304 4625164 140736206272960 18446744073709551615 4352112 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=189816 CPUtime=1.31 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 1906 0 0 0 131 0 0 0 25 0 4 0 669842731 194371584 4552 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351894 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4.46
Current children cumulated vsize (KiB) 189816

[startup+3.10058 s]
/proc/loadavg: 6.09 6.37 7.83 11/396 4367
/proc/meminfo: memFree=21017284/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=196756 CPUtime=9.24 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 10685 0 0 0 922 2 0 0 25 0 4 0 669842731 201478144 6457 33554432000 4194304 4625164 140736206272960 18446744073709551615 4512974 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/4352/statm: 49189 6457 349 106 0 44692 0
[pid=4352/tid=4355] ppid=4350 vsize=196756 CPUtime=1.44 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 1995 0 0 0 144 0 0 0 25 0 4 0 669842731 201478144 6457 33554432000 4194304 4625164 140736206272960 18446744073709551615 4465280 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=196756 CPUtime=3.05 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 3368 0 0 0 305 0 0 0 25 0 4 0 669842731 201478144 6457 33554432000 4194304 4625164 140736206272960 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=196756 CPUtime=2.15 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 3186 0 0 0 215 0 0 0 25 0 4 0 669842731 201478144 6457 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351958 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 9.24
Current children cumulated vsize (KiB) 196756

[startup+6.30101 s]
/proc/loadavg: 6.09 6.37 7.83 11/396 4367
/proc/meminfo: memFree=20981408/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=198300 CPUtime=18.81 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 20820 0 0 0 1877 4 0 0 25 0 4 0 669842731 203059200 9767 33554432000 4194304 4625164 140736206272960 18446744073709551615 4511874 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/4352/statm: 49575 9767 348 106 0 45078 0
[pid=4352/tid=4355] ppid=4350 vsize=198300 CPUtime=2.89 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 3177 0 0 0 289 0 0 0 25 0 4 0 669842731 203059200 9767 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=198300 CPUtime=6.2 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 5323 0 0 0 620 0 0 0 25 0 4 0 669842731 203059200 9767 33554432000 4194304 4625164 140736206272960 18446744073709551615 4339591 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=198300 CPUtime=3.9 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 9269 0 0 0 389 1 0 0 25 0 4 0 669842731 203059200 9767 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 18.81
Current children cumulated vsize (KiB) 198300

[startup+12.7009 s]
/proc/loadavg: 6.69 6.49 7.86 11/396 4368
/proc/meminfo: memFree=20930472/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=222036 CPUtime=37.91 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 31928 0 0 0 3786 5 0 0 25 0 4 0 669842731 227364864 16953 33554432000 4194304 4625164 140736206272960 18446744073709551615 4396715 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/4352/statm: 55509 16953 503 106 0 51012 0
[pid=4352/tid=4355] ppid=4350 vsize=222036 CPUtime=6.08 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 6287 0 0 0 608 0 0 0 25 0 4 0 669842731 227364864 16953 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=222036 CPUtime=12.52 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 10699 0 0 0 1251 1 0 0 25 0 4 0 669842731 227364864 16953 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=222036 CPUtime=7.1 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 10664 0 0 0 708 2 0 0 25 0 4 0 669842731 227364864 16953 33554432000 4194304 4625164 140736206272960 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37.91
Current children cumulated vsize (KiB) 222036

[startup+25.5005 s]
/proc/loadavg: 7.20 6.60 7.88 11/396 4368
/proc/meminfo: memFree=20870200/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=231936 CPUtime=76.15 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 39437 0 0 0 7607 8 0 0 25 0 4 0 669842731 237502464 22731 33554432000 4194304 4625164 140736206272960 18446744073709551615 4512312 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/4352/statm: 57984 22731 380 106 0 53487 0
[pid=4352/tid=4355] ppid=4350 vsize=231936 CPUtime=13.13 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 6947 0 0 0 1313 0 0 0 25 0 4 0 669842731 237502464 22731 33554432000 4194304 4625164 140736206272960 18446744073709551615 4352728 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=231936 CPUtime=24.46 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 12985 0 0 0 2444 2 0 0 25 0 4 0 669842731 237502464 22731 33554432000 4194304 4625164 140736206272960 18446744073709551615 4353328 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=231936 CPUtime=13.54 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 14203 0 0 0 1351 3 0 0 25 0 4 0 669842731 237502464 22731 33554432000 4194304 4625164 140736206272960 18446744073709551615 4352569 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 76.15
Current children cumulated vsize (KiB) 231936

[startup+51.1009 s]
/proc/loadavg: 8.15 6.88 7.93 11/396 4369
/proc/meminfo: memFree=20761540/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=282920 CPUtime=152.63 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 62187 0 0 0 15251 12 0 0 25 0 4 0 669842731 289710080 37012 33554432000 4194304 4625164 140736206272960 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/4352/statm: 70730 37012 603 106 0 66233 0
[pid=4352/tid=4355] ppid=4350 vsize=282920 CPUtime=25.93 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 14692 0 0 0 2592 1 0 0 25 0 4 0 669842731 289710080 37012 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351801 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=282920 CPUtime=49.74 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 18721 0 0 0 4970 4 0 0 25 0 4 0 669842731 289710080 37012 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=282920 CPUtime=26.35 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 20172 0 0 0 2631 4 0 0 25 0 4 0 669842731 289710080 37012 33554432000 4194304 4625164 140736206272960 18446744073709551615 4335407 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 152.63
Current children cumulated vsize (KiB) 282920

[startup+102.315 s]
/proc/loadavg: 9.30 7.41 8.06 11/396 4370
/proc/meminfo: memFree=20629680/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=324424 CPUtime=305.61 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 151354 0 0 0 30531 30 0 0 25 0 4 0 669842731 332210176 50448 33554432000 4194304 4625164 140736206272960 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/4352/statm: 81106 50448 463 106 0 76609 0
[pid=4352/tid=4355] ppid=4350 vsize=324424 CPUtime=51.6 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 15884 0 0 0 5158 2 0 0 25 0 4 0 669842731 332210176 50448 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=324424 CPUtime=100.29 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 38663 0 0 0 10022 7 0 0 25 0 4 0 669842731 332210176 50448 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=324424 CPUtime=51.9 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 84955 0 0 0 5173 17 0 0 25 0 4 0 669842731 332210176 50448 33554432000 4194304 4625164 140736206272960 18446744073709551615 4335407 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 305.61
Current children cumulated vsize (KiB) 324424

[startup+162.301 s]

################
# More data... #
################

[pid=4352] ppid=4350 vsize=737844 CPUtime=2097.9 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 1289769 0 0 0 209569 221 0 0 25 0 4 0 669842731 755552256 143397 33554432000 4194304 4625164 140736206272960 18446744073709551615 4519381 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/4352/statm: 184461 143397 1134 106 0 179964 0
[pid=4352/tid=4355] ppid=4350 vsize=737844 CPUtime=389.01 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 53438 0 0 0 38893 8 0 0 25 0 4 0 669842731 755552256 143397 33554432000 4194304 4625164 140736206272960 18446744073709551615 4352728 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=737844 CPUtime=636.12 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 98243 0 0 0 63585 27 0 0 25 0 4 0 669842731 755552256 143397 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=737844 CPUtime=370.93 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 1068597 0 0 0 36922 171 0 0 25 0 4 0 669842731 755552256 143397 33554432000 4194304 4625164 140736206272960 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2097.9
Current children cumulated vsize (KiB) 737844

[startup+762.315 s]
/proc/loadavg: 10.10 9.84 9.10 11/396 4388
/proc/meminfo: memFree=19719780/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=803380 CPUtime=2279.77 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 1446698 0 0 0 227730 247 0 0 25 0 4 0 669842731 822661120 147660 33554432000 4194304 4625164 140736206272960 18446744073709551615 4512017 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/4352/statm: 200845 147660 1084 106 0 196348 0
[pid=4352/tid=4355] ppid=4350 vsize=803380 CPUtime=449.03 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 56087 0 0 0 44893 10 0 0 25 0 4 0 669842731 822661120 147660 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351989 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=803380 CPUtime=692.95 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 99156 0 0 0 69268 27 0 0 25 0 4 0 669842731 822661120 147660 33554432000 4194304 4625164 140736206272960 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=803380 CPUtime=400.94 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 1209673 0 0 0 39902 192 0 0 25 0 4 0 669842731 822661120 147660 33554432000 4194304 4625164 140736206272960 18446744073709551615 4547598 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2279.77
Current children cumulated vsize (KiB) 803380

[startup+822.301 s]
/proc/loadavg: 10.07 9.88 9.16 11/396 4398
/proc/meminfo: memFree=19650472/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=843584 CPUtime=2458.95 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 1495312 0 0 0 245638 257 0 0 25 0 4 0 669842731 863830016 160541 33554432000 4194304 4625164 140736206272960 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/4352/statm: 210896 160541 1276 106 0 206399 0
[pid=4352/tid=4355] ppid=4350 vsize=843584 CPUtime=509.03 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 59630 0 0 0 50890 13 0 0 25 0 4 0 669842731 863830016 160541 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=843584 CPUtime=752.15 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 100604 0 0 0 75187 28 0 0 25 0 4 0 669842731 863830016 160541 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=843584 CPUtime=430.94 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 1253060 0 0 0 42895 199 0 0 25 0 4 0 669842731 863830016 160541 33554432000 4194304 4625164 140736206272960 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2458.95
Current children cumulated vsize (KiB) 843584

[startup+882.301 s]
/proc/loadavg: 10.09 9.92 9.22 11/396 4400
/proc/meminfo: memFree=19557172/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=929376 CPUtime=2638.18 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 1531654 0 0 0 263555 263 0 0 25 0 4 0 669842731 951681024 176029 33554432000 4194304 4625164 140736206272960 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/4352/statm: 232344 176029 1261 106 0 227847 0
[pid=4352/tid=4355] ppid=4350 vsize=929376 CPUtime=569.03 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 84019 0 0 0 56886 17 0 0 25 0 4 0 669842731 951681024 176029 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=929376 CPUtime=811.38 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 101547 0 0 0 81110 28 0 0 25 0 4 0 669842731 951681024 176029 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=929376 CPUtime=460.94 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 1253140 0 0 0 45895 199 0 0 25 0 4 0 669842731 951681024 176029 33554432000 4194304 4625164 140736206272960 18446744073709551615 4352731 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2638.18
Current children cumulated vsize (KiB) 929376

[startup+942.3 s]
/proc/loadavg: 10.37 10.03 9.30 11/396 4401
/proc/meminfo: memFree=19518688/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=994912 CPUtime=2844.63 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 1563911 0 0 0 284194 269 0 0 25 0 4 0 669842731 1018789888 179030 33554432000 4194304 4625164 140736206272960 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4352/statm: 248728 179030 692 106 0 244231 0
[pid=4352/tid=4355] ppid=4350 vsize=994912 CPUtime=625.85 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 84532 0 0 0 62568 17 0 0 25 0 4 0 669842731 1018789888 179030 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=994912 CPUtime=870.62 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 101886 0 0 0 87034 28 0 0 25 0 4 0 669842731 1018789888 179030 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=994912 CPUtime=518.17 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 1283693 0 0 0 51613 204 0 0 25 0 4 0 669842731 1018789888 179030 33554432000 4194304 4625164 140736206272960 18446744073709551615 4552315 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2844.63
Current children cumulated vsize (KiB) 994912

[startup+1002.3 s]
/proc/loadavg: 10.21 10.04 9.34 11/396 4403
/proc/meminfo: memFree=19469540/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=1016220 CPUtime=3053.85 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 1690027 0 0 0 305095 290 0 0 25 0 4 0 669842731 1040609280 184016 33554432000 4194304 4625164 140736206272960 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4352/statm: 254055 184016 692 106 0 249558 0
[pid=4352/tid=4355] ppid=4350 vsize=1016220 CPUtime=655.84 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 84976 0 0 0 65567 17 0 0 25 0 4 0 669842731 1040609280 184016 33554432000 4194304 4625164 140736206272960 18446744073709551615 4352208 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=1016220 CPUtime=929.86 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 104040 0 0 0 92956 30 0 0 25 0 4 0 669842731 1040609280 184016 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=1016220 CPUtime=578.17 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 1386455 0 0 0 57597 220 0 0 25 0 4 0 669842731 1040609280 184016 33554432000 4194304 4625164 140736206272960 18446744073709551615 4552452 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3053.85
Current children cumulated vsize (KiB) 1016220

[startup+1062.3 s]
/proc/loadavg: 10.34 10.10 9.41 11/396 4408
/proc/meminfo: memFree=19440432/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=1016220 CPUtime=3262.25 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 1938047 0 0 0 325900 325 0 0 25 0 4 0 669842731 1040609280 186788 33554432000 4194304 4625164 140736206272960 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4352/statm: 254055 186788 724 106 0 249558 0
[pid=4352/tid=4355] ppid=4350 vsize=1016220 CPUtime=685.83 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 85121 0 0 0 68566 17 0 0 25 0 4 0 669842731 1040609280 186788 33554432000 4194304 4625164 140736206272960 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=1016220 CPUtime=989.11 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 104466 0 0 0 98881 30 0 0 25 0 4 0 669842731 1040609280 186788 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=1016220 CPUtime=638.17 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 1633360 0 0 0 63562 255 0 0 25 0 4 0 669842731 1040609280 186788 33554432000 4194304 4625164 140736206272960 18446744073709551615 4339646 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3262.25
Current children cumulated vsize (KiB) 1016220

[startup+1122.3 s]
/proc/loadavg: 10.20 10.10 9.45 11/396 4409
/proc/meminfo: memFree=19407536/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=1016220 CPUtime=3471.49 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 2037280 0 0 0 346809 340 0 0 25 0 4 0 669842731 1040609280 187561 33554432000 4194304 4625164 140736206272960 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4352/statm: 254055 187561 639 106 0 249558 0
[pid=4352/tid=4355] ppid=4350 vsize=1016220 CPUtime=715.85 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 85377 0 0 0 71567 18 0 0 25 0 4 0 669842731 1040609280 187561 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=1016220 CPUtime=1048.34 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 104689 0 0 0 104804 30 0 0 25 0 4 0 669842731 1040609280 187561 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=1016220 CPUtime=698.17 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 1730851 0 0 0 69549 268 0 0 25 0 4 0 669842731 1040609280 187561 33554432000 4194304 4625164 140736206272960 18446744073709551615 4552380 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3471.49
Current children cumulated vsize (KiB) 1016220

[startup+1182.3 s]
/proc/loadavg: 10.14 10.09 9.48 11/396 4411
/proc/meminfo: memFree=19379064/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=1016220 CPUtime=3680.74 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 2167316 0 0 0 367714 360 0 0 25 0 4 0 669842731 1040609280 188540 33554432000 4194304 4625164 140736206272960 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4352/statm: 254055 188540 539 106 0 249558 0
[pid=4352/tid=4355] ppid=4350 vsize=1016220 CPUtime=745.85 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 85538 0 0 0 74567 18 0 0 25 0 4 0 669842731 1040609280 188540 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=1016220 CPUtime=1107.58 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 104988 0 0 0 110728 30 0 0 25 0 4 0 669842731 1040609280 188540 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351801 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=1016220 CPUtime=758.17 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 1859833 0 0 0 75529 288 0 0 25 0 4 0 669842731 1040609280 188540 33554432000 4194304 4625164 140736206272960 18446744073709551615 228122411335 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3680.74
Current children cumulated vsize (KiB) 1016220

[startup+1242.3 s]
/proc/loadavg: 10.17 10.10 9.52 11/396 4412
/proc/meminfo: memFree=19353080/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=1016220 CPUtime=3889.96 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 2259281 0 0 0 388621 375 0 0 25 0 4 0 669842731 1040609280 190776 33554432000 4194304 4625164 140736206272960 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4352/statm: 254055 190776 458 106 0 249558 0
[pid=4352/tid=4355] ppid=4350 vsize=1016220 CPUtime=775.86 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 85675 0 0 0 77568 18 0 0 25 0 4 0 669842731 1040609280 190776 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=1016220 CPUtime=1166.81 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 105253 0 0 0 116651 30 0 0 25 0 4 0 669842731 1040609280 190776 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351795 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=1016220 CPUtime=818.18 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 1935231 0 0 0 81519 299 0 0 25 0 4 0 669842731 1040609280 190776 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3889.96
Current children cumulated vsize (KiB) 1016220



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 10.19 10.12 9.56 11/396 4414
/proc/meminfo: memFree=19275396/32951124 swapFree=49826708/67111528
[pid=4352] ppid=4350 vsize=1127804 CPUtime=4091.21 cores=0,2,4,6
/proc/4352/stat : 4352 (SArTagnan) R 4350 4352 3578 0 -1 4202496 2504744 0 0 0 408709 412 0 0 25 0 4 0 669842731 1154871296 205243 33554432000 4194304 4625164 140736206272960 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4352/statm: 281951 205243 446 106 0 277454 0
[pid=4352/tid=4355] ppid=4350 vsize=1127804 CPUtime=804.71 cores=0,2,4,6
/proc/4352/task/4355/stat : 4355 (SArTagnan) R 4350 4352 3578 0 -1 4202560 85807 0 0 0 80453 18 0 0 25 0 4 0 669842731 1154871296 205243 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=4352/tid=4356] ppid=4350 vsize=1127804 CPUtime=1223.79 cores=0,2,4,6
/proc/4352/task/4356/stat : 4356 (SArTagnan) R 4350 4352 3578 0 -1 4202560 148231 0 0 0 122343 36 0 0 25 0 4 0 669842731 1154871296 205243 33554432000 4194304 4625164 140736206272960 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=4352/tid=4357] ppid=4350 vsize=1127804 CPUtime=875.89 cores=0,2,4,6
/proc/4352/task/4357/stat : 4357 (SArTagnan) R 4350 4352 3578 0 -1 4202560 2136659 0 0 0 87260 329 0 0 25 0 4 0 669842731 1154871296 205243 33554432000 4194304 4625164 140736206272960 18446744073709551615 4552315 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4091.21
Current children cumulated vsize (KiB) 1127804

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1300.06
CPU time (s): 4091.28
CPU user time (s): 4087.12
CPU system time (s): 4.15637
CPU usage (%): 314.7
Max. virtual memory (cumulated for all children) (KiB): 1249268

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4087.12
system time used= 4.15637
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2504744
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 129
involuntary context switches= 32036

runsolver used 4.65629 second user time and 11.9142 second system time

The end

Launcher Data 

Begin job on node138 at 2011-05-16 23:47:28
IDJOB=3385986
IDBENCH=84018
IDSOLVER=1871
FILE ID=node138/3385986-1305582448
RUNJOBID= node138-1305582447-4340
PBS_JOBID= 13350210
Free space on /tmp= 73440 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT03/handmade/stanion/hwb/hwb-n28-02-S818962541.shuffled-as.sat03-1628.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385986-1305582448/watcher-3385986-1305582448 -o /tmp/evaluation-result-3385986-1305582448/solver-3385986-1305582448 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3385986-1305582448.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= d705043763df769aab1e385bbbc2b11e
RANDOM SEED=1624740309

node138.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21066124 kB
Buffers:          5304 kB
Cached:          92612 kB
SwapCached:      70804 kB
Active:        1647308 kB
Inactive:     10013144 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21066124 kB
SwapTotal:    67111528 kB
SwapFree:     49826708 kB
Dirty:             424 kB
Writeback:           0 kB
AnonPages:    11501412 kB
Mapped:          11520 kB
Slab:            72920 kB
PageTables:      85660 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 96023560 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73440 MiB
End job on node138 at 2011-05-17 00:09:10