Trace number 3273782

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-03-24? (TO) 4800.3 1201.76

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n220-s24359701-sat.cnf
MD5SUMceff918324677440f95867755c4e7133
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark5.76412
Satisfiable
(Un)Satisfiability was proved
Number of variables220
Number of clauses528
Sum of the clauses size1320
Maximum clause length5
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2440
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 588
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3273782-1304085985/watcher-3273782-1304085985 -o /tmp/evaluation-result-3273782-1304085985/solver-3273782-1304085985 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3273782-1304085985.cnf -threads=4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.16 7.88 7.95 3/176 27396
/proc/meminfo: memFree=17465136/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=49208 CPUtime=0 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 450 0 0 0 0 0 0 0 24 0 4 0 598829323 50388992 386 33554432000 4194304 4623224 140737479367776 18446744073709551615 47055137068978 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 12302 386 301 105 0 7806 0
[pid=27394/tid=27397] ppid=27391 vsize=49208 CPUtime=0 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 4 0 598829323 50388992 386 33554432000 4194304 4623224 140737479367776 18446744073709551615 215988648945 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=49208 CPUtime=0 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 4 0 598829323 50388992 386 33554432000 4194304 4623224 140737479367776 18446744073709551615 47055137068978 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=49208 CPUtime=0 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 4 0 598829323 50388992 386 33554432000 4194304 4623224 140737479367776 18446744073709551615 47055137068978 0 0 4096 2 0 0 0 -1 0 0 0 0

[startup+0.0582309 s]
/proc/loadavg: 7.16 7.88 7.95 3/176 27396
/proc/meminfo: memFree=17465136/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=246948 CPUtime=0.21 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 1202 0 0 0 21 0 0 0 25 0 4 0 598829323 252874752 1011 33554432000 4194304 4623224 140737479367776 18446744073709551615 215988241681 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 61737 1011 343 105 0 57241 0
[pid=27394/tid=27397] ppid=27391 vsize=246948 CPUtime=0.04 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 192 0 0 0 4 0 0 0 25 0 4 0 598829323 252874752 1011 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=246948 CPUtime=0.05 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 144 0 0 0 5 0 0 0 25 0 4 0 598829323 252874752 1011 33554432000 4194304 4623224 140737479367776 18446744073709551615 4461896 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=246948 CPUtime=0.05 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 200 0 0 0 5 0 0 0 25 0 4 0 598829323 252874752 1011 33554432000 4194304 4623224 140737479367776 18446744073709551615 4550432 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 246948

[startup+0.158214 s]
/proc/loadavg: 7.16 7.88 7.95 3/176 27396
/proc/meminfo: memFree=17465136/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=248228 CPUtime=0.61 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 2283 0 0 0 61 0 0 0 25 0 4 0 598829323 254185472 1570 33554432000 4194304 4623224 140737479367776 18446744073709551615 4394961 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 62057 1570 343 105 0 57561 0
[pid=27394/tid=27397] ppid=27391 vsize=248228 CPUtime=0.14 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 531 0 0 0 14 0 0 0 25 0 4 0 598829323 254185472 1570 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376612 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=248228 CPUtime=0.15 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 428 0 0 0 15 0 0 0 25 0 4 0 598829323 254185472 1570 33554432000 4194304 4623224 140737479367776 18446744073709551615 4461225 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=248228 CPUtime=0.15 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 554 0 0 0 15 0 0 0 25 0 4 0 598829323 254185472 1570 33554432000 4194304 4623224 140737479367776 18446744073709551615 4545317 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 0.61
Current children cumulated vsize (KiB) 248228

[startup+0.300191 s]
/proc/loadavg: 7.16 7.88 7.95 3/176 27396
/proc/meminfo: memFree=17465136/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=250060 CPUtime=1.17 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 3850 0 0 0 117 0 0 0 25 0 4 0 598829323 256061440 2384 33554432000 4194304 4623224 140737479367776 18446744073709551615 4510939 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 62515 2384 344 105 0 58019 0
[pid=27394/tid=27397] ppid=27391 vsize=250060 CPUtime=0.28 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 899 0 0 0 28 0 0 0 25 0 4 0 598829323 256061440 2384 33554432000 4194304 4623224 140737479367776 18446744073709551615 4461036 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=250060 CPUtime=0.29 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 965 0 0 0 29 0 0 0 25 0 4 0 598829323 256061440 2384 33554432000 4194304 4623224 140737479367776 18446744073709551615 4463644 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=250060 CPUtime=0.29 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 1055 0 0 0 29 0 0 0 25 0 4 0 598829323 256061440 2384 33554432000 4194304 4623224 140737479367776 18446744073709551615 4577681 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1.17
Current children cumulated vsize (KiB) 250060

[startup+0.701123 s]
/proc/loadavg: 7.16 7.88 7.95 3/176 27396
/proc/meminfo: memFree=17465136/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=253620 CPUtime=2.77 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 7950 0 0 0 277 0 0 0 25 0 4 0 598829323 259706880 4006 33554432000 4194304 4623224 140737479367776 18446744073709551615 4510090 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 63405 4006 344 105 0 58909 0
[pid=27394/tid=27397] ppid=27391 vsize=253620 CPUtime=0.69 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 1895 0 0 0 69 0 0 0 25 0 4 0 598829323 259706880 4006 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376933 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=253620 CPUtime=0.69 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 1848 0 0 0 69 0 0 0 25 0 4 0 598829323 259706880 4006 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=253620 CPUtime=0.69 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 3089 0 0 0 69 0 0 0 25 0 4 0 598829323 259706880 4006 33554432000 4194304 4623224 140737479367776 18446744073709551615 4550506 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2.77
Current children cumulated vsize (KiB) 253620

[startup+1.50098 s]
/proc/loadavg: 7.16 7.88 7.95 10/184 27404
/proc/meminfo: memFree=17423940/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=258336 CPUtime=5.97 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 17153 0 0 0 596 1 0 0 25 0 4 0 598829323 264536064 6291 33554432000 4194304 4623224 140737479367776 18446744073709551615 4511029 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 64584 6291 344 105 0 60088 0
[pid=27394/tid=27397] ppid=27391 vsize=258336 CPUtime=1.48 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 3151 0 0 0 148 0 0 0 25 0 4 0 598829323 264536064 6291 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376672 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=258336 CPUtime=1.49 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 2898 0 0 0 149 0 0 0 25 0 4 0 598829323 264536064 6223 33554432000 4194304 4623224 140737479367776 18446744073709551615 4364212 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=258336 CPUtime=1.49 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 9596 0 0 0 148 1 0 0 25 0 4 0 598829323 264536064 6223 33554432000 4194304 4623224 140737479367776 18446744073709551615 216004576283 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.97
Current children cumulated vsize (KiB) 258336

[startup+3.1007 s]
/proc/loadavg: 7.16 7.88 7.95 10/184 27404
/proc/meminfo: memFree=17398520/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=268472 CPUtime=12.37 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 37046 0 0 0 1233 4 0 0 25 0 4 0 598829323 274915328 10073 33554432000 4194304 4623224 140737479367776 18446744073709551615 4510119 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 67118 10073 345 105 0 62622 0
[pid=27394/tid=27397] ppid=27391 vsize=268472 CPUtime=3.07 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 4981 0 0 0 307 0 0 0 25 0 4 0 598829323 274915328 10073 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376646 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=268472 CPUtime=3.08 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 5792 0 0 0 308 0 0 0 25 0 4 0 598829323 274915328 10073 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376823 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=268472 CPUtime=3.09 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 24127 0 0 0 307 2 0 0 25 0 4 0 598829323 274915328 10073 33554432000 4194304 4623224 140737479367776 18446744073709551615 4550556 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 12.37
Current children cumulated vsize (KiB) 268472

[startup+6.30115 s]
/proc/loadavg: 7.22 7.89 7.95 10/184 27404
/proc/meminfo: memFree=17346196/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=279864 CPUtime=25.14 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 73885 0 0 0 2506 8 0 0 25 0 4 0 598829323 286580736 14875 33554432000 4194304 4623224 140737479367776 18446744073709551615 215988252069 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 69966 14875 356 105 0 65470 0
[pid=27394/tid=27397] ppid=27391 vsize=279864 CPUtime=6.26 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 8954 0 0 0 625 1 0 0 25 0 4 0 598829323 286580736 14875 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376832 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=279864 CPUtime=6.29 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 8858 0 0 0 628 1 0 0 25 0 4 0 598829323 286580736 14875 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376672 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=279864 CPUtime=6.28 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 52839 0 0 0 623 5 0 0 25 0 4 0 598829323 286580736 14875 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376895 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 25.14
Current children cumulated vsize (KiB) 279864

[startup+12.701 s]
/proc/loadavg: 7.29 7.89 7.95 10/184 27405
/proc/meminfo: memFree=17283468/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=282292 CPUtime=50.7 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 127044 0 0 0 5051 19 0 0 25 0 4 0 598829323 289067008 20101 33554432000 4194304 4623224 140737479367776 18446744073709551615 4510127 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 70573 20101 375 105 0 66077 0
[pid=27394/tid=27397] ppid=27391 vsize=282292 CPUtime=12.61 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 10811 0 0 0 1259 2 0 0 25 0 4 0 598829323 289067008 20101 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=282292 CPUtime=12.69 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 12422 0 0 0 1267 2 0 0 25 0 4 0 598829323 289067008 20101 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376672 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=282292 CPUtime=12.69 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 99577 0 0 0 1255 14 0 0 25 0 4 0 598829323 289067008 20101 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 50.7
Current children cumulated vsize (KiB) 282292

[startup+25.5008 s]
/proc/loadavg: 7.59 7.92 7.96 10/184 27405
/proc/meminfo: memFree=17184032/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=312412 CPUtime=101.83 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 172362 0 0 0 10156 27 0 0 25 0 4 0 598829323 319909888 29804 33554432000 4194304 4623224 140737479367776 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 78103 29804 391 105 0 73607 0
[pid=27394/tid=27397] ppid=27391 vsize=312412 CPUtime=25.35 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 20135 0 0 0 2531 4 0 0 25 0 4 0 598829323 319909888 29804 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=312412 CPUtime=25.49 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 18812 0 0 0 2546 3 0 0 25 0 4 0 598829323 319909888 29804 33554432000 4194304 4623224 140737479367776 18446744073709551615 4366416 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=312412 CPUtime=25.48 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 127062 0 0 0 2530 18 0 0 25 0 4 0 598829323 319909888 29804 33554432000 4194304 4623224 140737479367776 18446744073709551615 4550411 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 101.83
Current children cumulated vsize (KiB) 312412

[startup+51.1003 s]
/proc/loadavg: 7.73 7.93 7.96 10/184 27406
/proc/meminfo: memFree=17063240/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=348360 CPUtime=204.09 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 266942 0 0 0 20368 41 0 0 25 0 4 0 598829323 356720640 43344 33554432000 4194304 4623224 140737479367776 18446744073709551615 4510923 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 87090 43344 466 105 0 82594 0
[pid=27394/tid=27397] ppid=27391 vsize=348360 CPUtime=50.81 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 22772 0 0 0 5076 5 0 0 25 0 4 0 598829323 356720640 43344 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376869 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=348360 CPUtime=51.08 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 26992 0 0 0 5104 4 0 0 25 0 4 0 598829323 356720640 43344 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376846 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=348360 CPUtime=51.08 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 202697 0 0 0 5080 28 0 0 25 0 4 0 598829323 356720640 43344 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 204.09
Current children cumulated vsize (KiB) 348360

[startup+102.306 s]
/proc/loadavg: 7.99 7.97 7.97 10/184 27407
/proc/meminfo: memFree=16849696/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=458372 CPUtime=408.64 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 586937 0 0 0 40773 91 0 0 25 0 4 0 598829323 469372928 71958 33554432000 4194304 4623224 140737479367776 18446744073709551615 4517563 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 114593 71958 854 105 0 110097 0
[pid=27394/tid=27397] ppid=27391 vsize=458372 CPUtime=101.73 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 44766 0 0 0 10163 10 0 0 25 0 4 0 598829323 469372928 71958 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=458372 CPUtime=102.29 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 39844 0 0 0 10222 7 0 0 25 0 4 0 598829323 469372928 71958 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=458372 CPUtime=102.29 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 480036 0 0 0 10160 69 0 0 25 0 4 0 598829323 469372928 71958 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 408.64
Current children cumulated vsize (KiB) 458372

[startup+162.301 s]

################
# More data... #
################

[pid=27394] ppid=27391 vsize=997016 CPUtime=2805.33 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 1191924 0 0 0 280313 220 0 0 25 0 4 0 598829323 1020944384 188456 33554432000 4194304 4623224 140737479367776 18446744073709551615 4510316 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 249254 188456 1210 105 0 244758 0
[pid=27394/tid=27397] ppid=27391 vsize=997016 CPUtime=698.47 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 113665 0 0 0 69801 46 0 0 25 0 4 0 598829323 1020944384 188456 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=997016 CPUtime=702.28 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 129512 0 0 0 70196 32 0 0 25 0 4 0 598829323 1020944384 188456 33554432000 4194304 4623224 140737479367776 18446744073709551615 4377576 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=997016 CPUtime=702.3 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 887716 0 0 0 70099 131 0 0 25 0 4 0 598829323 1020944384 188456 33554432000 4194304 4623224 140737479367776 18446744073709551615 4364911 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2805.33
Current children cumulated vsize (KiB) 997016

[startup+762.307 s]
/proc/loadavg: 8.20 8.09 8.02 10/184 27425
/proc/meminfo: memFree=15825508/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=997016 CPUtime=3045.02 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 1197242 0 0 0 304277 225 0 0 25 0 4 0 598829323 1020944384 193350 33554432000 4194304 4623224 140737479367776 18446744073709551615 4510105 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 249254 193350 842 105 0 244758 0
[pid=27394/tid=27397] ppid=27391 vsize=997016 CPUtime=758.15 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 115692 0 0 0 75767 48 0 0 25 0 4 0 598829323 1020944384 193350 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376869 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=997016 CPUtime=762.29 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 131436 0 0 0 76194 35 0 0 25 0 4 0 598829323 1020944384 193350 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=997016 CPUtime=762.31 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 888464 0 0 0 76099 132 0 0 25 0 4 0 598829323 1020944384 193350 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376837 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3045.02
Current children cumulated vsize (KiB) 997016

[startup+822.3 s]
/proc/loadavg: 8.13 8.09 8.02 10/184 27426
/proc/meminfo: memFree=15759268/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=1126932 CPUtime=3284.67 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 1212568 0 0 0 328236 231 0 0 25 0 4 0 598829323 1153978368 208377 33554432000 4194304 4623224 140737479367776 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 281733 208377 599 105 0 277237 0
[pid=27394/tid=27397] ppid=27391 vsize=1126932 CPUtime=817.83 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 117679 0 0 0 81732 51 0 0 25 0 4 0 598829323 1153978368 208377 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376640 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=1126932 CPUtime=822.29 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 133123 0 0 0 82192 37 0 0 25 0 4 0 598829323 1153978368 208377 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=1126932 CPUtime=822.3 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 888902 0 0 0 82098 132 0 0 25 0 4 0 598829323 1153978368 208377 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3284.67
Current children cumulated vsize (KiB) 1126932

[startup+882.301 s]
/proc/loadavg: 8.28 8.13 8.04 10/184 27428
/proc/meminfo: memFree=15655316/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=1174936 CPUtime=3524.34 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 1223744 0 0 0 352198 236 0 0 25 0 4 0 598829323 1203134464 219471 33554432000 4194304 4623224 140737479367776 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 293734 219471 584 105 0 289238 0
[pid=27394/tid=27397] ppid=27391 vsize=1174936 CPUtime=877.5 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 120527 0 0 0 87697 53 0 0 25 0 4 0 598829323 1203134464 219471 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=1174936 CPUtime=882.29 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 140578 0 0 0 88190 39 0 0 25 0 4 0 598829323 1203134464 219471 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376652 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=1174936 CPUtime=882.3 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 889252 0 0 0 88098 132 0 0 25 0 4 0 598829323 1203134464 219471 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3524.34
Current children cumulated vsize (KiB) 1174936

[startup+942.3 s]
/proc/loadavg: 8.21 8.14 8.04 10/184 27429
/proc/meminfo: memFree=15597732/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=1256652 CPUtime=3764.02 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 1246374 0 0 0 376162 240 0 0 25 0 4 0 598829323 1286811648 230471 33554432000 4194304 4623224 140737479367776 18446744073709551615 4510090 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 314163 230471 559 105 0 309667 0
[pid=27394/tid=27397] ppid=27391 vsize=1256652 CPUtime=937.17 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 127891 0 0 0 93662 55 0 0 25 0 4 0 598829323 1286811648 230471 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=1256652 CPUtime=942.29 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 142116 0 0 0 94189 40 0 0 25 0 4 0 598829323 1286811648 230471 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=1256652 CPUtime=942.31 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 902221 0 0 0 94097 134 0 0 25 0 4 0 598829323 1286811648 230471 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3764.02
Current children cumulated vsize (KiB) 1256652

[startup+1002.3 s]
/proc/loadavg: 8.18 8.14 8.04 10/184 27431
/proc/meminfo: memFree=15524040/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=1256840 CPUtime=4003.69 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 1249869 0 0 0 400128 241 0 0 25 0 4 0 598829323 1287004160 233885 33554432000 4194304 4623224 140737479367776 18446744073709551615 4517494 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 314210 233885 559 105 0 309714 0
[pid=27394/tid=27397] ppid=27391 vsize=1256840 CPUtime=996.84 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 128315 0 0 0 99629 55 0 0 25 0 4 0 598829323 1287004160 233885 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=1256840 CPUtime=1002.29 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 143624 0 0 0 100189 40 0 0 25 0 4 0 598829323 1287004160 233885 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=1256840 CPUtime=1002.31 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 902853 0 0 0 100097 134 0 0 25 0 4 0 598829323 1287004160 233885 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4003.69
Current children cumulated vsize (KiB) 1256840

[startup+1062.3 s]
/proc/loadavg: 8.16 8.14 8.05 10/184 27433
/proc/meminfo: memFree=15475772/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=1259928 CPUtime=4243.36 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 1252850 0 0 0 424094 242 0 0 25 0 4 0 598829323 1290166272 236782 33554432000 4194304 4623224 140737479367776 18446744073709551615 4517479 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 314982 236782 559 105 0 310486 0
[pid=27394/tid=27397] ppid=27391 vsize=1259928 CPUtime=1056.53 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 128900 0 0 0 105597 56 0 0 25 0 4 0 598829323 1290166272 236782 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376672 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=1259928 CPUtime=1062.29 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 144104 0 0 0 106189 40 0 0 25 0 4 0 598829323 1290166272 236782 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376675 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=1259928 CPUtime=1062.31 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 903778 0 0 0 106097 134 0 0 25 0 4 0 598829323 1290166272 236782 33554432000 4194304 4623224 140737479367776 18446744073709551615 4550465 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4243.36
Current children cumulated vsize (KiB) 1259928

[startup+1122.3 s]
/proc/loadavg: 8.15 8.14 8.05 10/184 27434
/proc/meminfo: memFree=15421932/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=1260824 CPUtime=4483.04 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 1254974 0 0 0 448061 243 0 0 25 0 4 0 598829323 1291083776 238842 33554432000 4194304 4623224 140737479367776 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 315206 238842 559 105 0 310710 0
[pid=27394/tid=27397] ppid=27391 vsize=1260824 CPUtime=1116.2 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 129639 0 0 0 111564 56 0 0 25 0 4 0 598829323 1291083776 238842 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=1260824 CPUtime=1122.29 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 144791 0 0 0 112189 40 0 0 25 0 4 0 598829323 1291083776 238842 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376625 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=1260824 CPUtime=1122.31 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 904061 0 0 0 112097 134 0 0 25 0 4 0 598829323 1291083776 238842 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4483.04
Current children cumulated vsize (KiB) 1260824

[startup+1182.3 s]
/proc/loadavg: 8.13 8.14 8.06 10/184 27436
/proc/meminfo: memFree=15392880/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=1263484 CPUtime=4722.71 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 1257276 0 0 0 472028 243 0 0 25 0 4 0 598829323 1293807616 241022 33554432000 4194304 4623224 140737479367776 18446744073709551615 4510090 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 315871 241022 460 105 0 311375 0
[pid=27394/tid=27397] ppid=27391 vsize=1263484 CPUtime=1175.88 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 130224 0 0 0 117532 56 0 0 25 0 4 0 598829323 1293807616 241022 33554432000 4194304 4623224 140737479367776 18446744073709551615 4366416 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=1263484 CPUtime=1182.3 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 145201 0 0 0 118189 41 0 0 25 0 4 0 598829323 1293807616 241022 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=1263484 CPUtime=1182.31 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 905263 0 0 0 118097 134 0 0 25 0 4 0 598829323 1293807616 241022 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4722.71
Current children cumulated vsize (KiB) 1263484



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 8.17 8.15 8.06 10/184 27436
/proc/meminfo: memFree=15388268/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=1263484 CPUtime=4800.22 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 1257678 0 0 0 479779 243 0 0 25 0 4 0 598829323 1293807616 241416 33554432000 4194304 4623224 140737479367776 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 315871 241416 460 105 0 311375 0
[pid=27394/tid=27397] ppid=27391 vsize=1263484 CPUtime=1195.17 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 130396 0 0 0 119461 56 0 0 25 0 4 0 598829323 1293807616 241416 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=1263484 CPUtime=1201.7 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 145325 0 0 0 120129 41 0 0 25 0 4 0 598829323 1293807616 241416 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=1263484 CPUtime=1201.71 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 905353 0 0 0 120037 134 0 0 25 0 4 0 598829323 1293807616 241416 33554432000 4194304 4623224 140737479367776 18446744073709551615 4550465 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.22
Current children cumulated vsize (KiB) 1263484

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 8.17 8.15 8.06 10/184 27436
/proc/meminfo: memFree=15388268/32950940 swapFree=67111528/67111528
[pid=27394] ppid=27391 vsize=1263484 CPUtime=4800.22 cores=0,2,4,6
/proc/27394/stat : 27394 (SArTagnan) R 27391 27394 26674 0 -1 4202496 1257678 0 0 0 479779 243 0 0 25 0 4 0 598829323 1293807616 241416 33554432000 4194304 4623224 140737479367776 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/27394/statm: 315871 241416 460 105 0 311375 0
[pid=27394/tid=27397] ppid=27391 vsize=1263484 CPUtime=1195.17 cores=0,2,4,6
/proc/27394/task/27397/stat : 27397 (SArTagnan) R 27391 27394 26674 0 -1 4202560 130396 0 0 0 119461 56 0 0 25 0 4 0 598829323 1293807616 241416 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=27394/tid=27398] ppid=27391 vsize=1263484 CPUtime=1201.7 cores=0,2,4,6
/proc/27394/task/27398/stat : 27398 (SArTagnan) R 27391 27394 26674 0 -1 4202560 145325 0 0 0 120129 41 0 0 25 0 4 0 598829323 1293807616 241416 33554432000 4194304 4623224 140737479367776 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=27394/tid=27399] ppid=27391 vsize=1263484 CPUtime=1201.71 cores=0,2,4,6
/proc/27394/task/27399/stat : 27399 (SArTagnan) R 27391 27394 26674 0 -1 4202560 905353 0 0 0 120037 134 0 0 25 0 4 0 598829323 1293807616 241416 33554432000 4194304 4623224 140737479367776 18446744073709551615 4550465 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.22
Current children cumulated vsize (KiB) 1263484

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.76
CPU time (s): 4800.3
CPU user time (s): 4797.81
CPU system time (s): 2.48362
CPU usage (%): 399.439
Max. virtual memory (cumulated for all children) (KiB): 1263484

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.81
system time used= 2.48362
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1257678
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 307
involuntary context switches= 49485

runsolver used 1.78773 second user time and 4.76628 second system time

The end

Launcher Data 

Begin job on node117 at 2011-04-29 16:06:26
IDJOB=3273782
IDBENCH=83146
IDSOLVER=1656
FILE ID=node117/3273782-1304085985
RUNJOBID= node117-1304085985-27377
PBS_JOBID= 13170523
Free space on /tmp= 71680 MiB

SOLVER NAME= SArTagnan 2011-03-24
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n220-s24359701-sat.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3273782-1304085985/watcher-3273782-1304085985 -o /tmp/evaluation-result-3273782-1304085985/solver-3273782-1304085985 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3273782-1304085985.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= ceff918324677440f95867755c4e7133
RANDOM SEED=1366619540

node117.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5267.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950940 kB
MemFree:      17465448 kB
Buffers:       2379592 kB
Cached:       12460216 kB
SwapCached:          0 kB
Active:        8138616 kB
Inactive:      6765588 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950940 kB
LowFree:      17465448 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:             896 kB
Writeback:           0 kB
AnonPages:       63488 kB
Mapped:          14496 kB
Slab:           518556 kB
PageTables:       4300 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586996 kB
Committed_AS:   187428 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71680 MiB
End job on node117 at 2011-04-29 16:26:29