Trace number 3280665

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-03-24? (TO) 4800.44 1202.48

General information on the benchmark

Nameindustrial/palacios/hard/
sortnet-8-ipc5-h19-sat.cnf
MD5SUM409f4401e9e8b60ee665ac5567a13e41
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark1110.09
Satisfiable
(Un)Satisfiability was proved
Number of variables359341
Number of clauses1254773
Sum of the clauses size3046189
Maximum clause length57
Minimum clause length1
Number of clauses of size 1256
Number of clauses of size 2896240
Number of clauses of size 3346435
Number of clauses of size 4830
Number of clauses of size 5571
Number of clauses of size over 510441

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3280665-1304038012/watcher-3280665-1304038012 -o /tmp/evaluation-result-3280665-1304038012/solver-3280665-1304038012 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3280665-1304038012.cnf -threads=4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.49 8.05 8.02 3/178 16946
/proc/meminfo: memFree=25433312/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=24196 CPUtime=0 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 1253 0 0 0 0 0 0 0 23 0 1 0 515399446 24776704 1190 33554432000 4194304 4623224 140735602964224 18446744073709551615 4315013 0 0 4096 2 0 0 0 17 3 0 0 0
/proc/16946/statm: 6049 1192 290 105 0 1553 0

[startup+0.0759969 s]
/proc/loadavg: 7.49 8.05 8.02 3/178 16946
/proc/meminfo: memFree=25433312/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=82960 CPUtime=0.07 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 9514 0 0 0 6 1 0 0 23 0 1 0 515399446 84951040 8180 33554432000 4194304 4623224 140735602964224 18446744073709551615 4366247 0 0 4096 2 0 0 0 17 3 0 0 0
/proc/16946/statm: 20740 8181 301 105 0 16244 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 82960

[startup+0.100992 s]
/proc/loadavg: 7.49 8.05 8.02 3/178 16946
/proc/meminfo: memFree=25433312/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=85232 CPUtime=0.09 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 10995 0 0 0 8 1 0 0 23 0 1 0 515399446 87277568 8787 33554432000 4194304 4623224 140735602964224 18446744073709551615 232335553221 0 0 4096 2 0 0 0 17 3 0 0 0
/proc/16946/statm: 21308 8787 301 105 0 16812 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 85232

[startup+0.300958 s]
/proc/loadavg: 7.49 8.05 8.02 3/178 16946
/proc/meminfo: memFree=25433312/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=103964 CPUtime=0.29 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 22108 0 0 0 26 3 0 0 25 0 1 0 515399446 106459136 13672 33554432000 4194304 4623224 140735602964224 18446744073709551615 4366247 0 0 4096 2 0 0 0 17 3 0 0 0
/proc/16946/statm: 25991 13672 301 105 0 21495 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 103964

[startup+0.700843 s]
/proc/loadavg: 7.49 8.05 8.02 3/178 16946
/proc/meminfo: memFree=25433312/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=396896 CPUtime=0.88 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 76107 0 0 0 79 9 0 0 25 0 4 0 515399446 406421504 59357 33554432000 4194304 4623224 140735602964224 18446744073709551615 4497487 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 99224 59357 327 105 0 94728 0
Current children cumulated CPU time (s) 0.88
Current children cumulated vsize (KiB) 396896

[startup+1.5007 s]
/proc/loadavg: 7.49 8.05 8.02 6/185 16956
/proc/meminfo: memFree=25065024/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=580592 CPUtime=4.07 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 110306 0 0 0 393 14 0 0 25 0 4 0 515399446 594526208 72508 33554432000 4194304 4623224 140735602964224 18446744073709551615 4501791 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 145148 72508 334 105 0 140652 0
[pid=16946/tid=16954] ppid=16944 vsize=580592 CPUtime=0.85 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 11479 0 0 0 84 1 0 0 25 0 4 0 515399509 594526208 72508 33554432000 4194304 4623224 140735602964224 18446744073709551615 4363752 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=580592 CPUtime=0.85 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 11151 0 0 0 84 1 0 0 18 0 4 0 515399509 594526208 72508 33554432000 4194304 4623224 140735602964224 18446744073709551615 4377034 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=580592 CPUtime=0.86 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 10687 0 0 0 85 1 0 0 25 0 4 0 515399509 594526208 72508 33554432000 4194304 4623224 140735602964224 18446744073709551615 4467877 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4.07
Current children cumulated vsize (KiB) 580592

[startup+3.10042 s]
/proc/loadavg: 7.49 8.05 8.02 10/188 16959
/proc/meminfo: memFree=24747188/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=580592 CPUtime=10.47 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 114314 0 0 0 1032 15 0 0 25 0 4 0 515399446 594526208 76470 33554432000 4194304 4623224 140735602964224 18446744073709551615 4412513 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 145148 76470 335 105 0 140652 0
[pid=16946/tid=16954] ppid=16944 vsize=580592 CPUtime=2.45 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 12574 0 0 0 244 1 0 0 25 0 4 0 515399509 594526208 76470 33554432000 4194304 4623224 140735602964224 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=580592 CPUtime=2.44 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 12362 0 0 0 243 1 0 0 21 0 4 0 515399509 594526208 76470 33554432000 4194304 4623224 140735602964224 18446744073709551615 4364232 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=580592 CPUtime=2.45 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 11414 0 0 0 244 1 0 0 25 0 4 0 515399509 594526208 76470 33554432000 4194304 4623224 140735602964224 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 10.47
Current children cumulated vsize (KiB) 580592

[startup+6.30085 s]
/proc/loadavg: 7.61 8.06 8.03 9/188 16959
/proc/meminfo: memFree=24707016/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=613604 CPUtime=23.25 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 135380 0 0 0 2306 19 0 0 25 0 4 0 515399446 628330496 80601 33554432000 4194304 4623224 140735602964224 18446744073709551615 4412446 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 153401 80601 335 105 0 148905 0
[pid=16946/tid=16954] ppid=16944 vsize=613604 CPUtime=5.65 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 17727 0 0 0 563 2 0 0 25 0 4 0 515399509 628330496 80601 33554432000 4194304 4623224 140735602964224 18446744073709551615 4289917 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=613604 CPUtime=5.62 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 17940 0 0 0 560 2 0 0 25 0 4 0 515399509 628330496 80601 33554432000 4194304 4623224 140735602964224 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=613604 CPUtime=5.65 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 16540 0 0 0 563 2 0 0 25 0 4 0 515399509 628330496 80601 33554432000 4194304 4623224 140735602964224 18446744073709551615 4364001 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 23.25
Current children cumulated vsize (KiB) 613604

[startup+12.7007 s]
/proc/loadavg: 7.64 8.06 8.03 9/188 16960
/proc/meminfo: memFree=24666148/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=663168 CPUtime=48.81 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 168757 0 0 0 4858 23 0 0 25 0 4 0 515399446 679084032 87964 33554432000 4194304 4623224 140735602964224 18446744073709551615 4412444 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 165792 87964 335 105 0 161296 0
[pid=16946/tid=16954] ppid=16944 vsize=663168 CPUtime=12.06 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 26407 0 0 0 1203 3 0 0 25 0 4 0 515399509 679084032 87964 33554432000 4194304 4623224 140735602964224 18446744073709551615 4289853 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=663168 CPUtime=11.99 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 27104 0 0 0 1195 4 0 0 25 0 4 0 515399509 679084032 87964 33554432000 4194304 4623224 140735602964224 18446744073709551615 4330743 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=663168 CPUtime=12.05 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 24318 0 0 0 1202 3 0 0 25 0 4 0 515399509 679084032 87964 33554432000 4194304 4623224 140735602964224 18446744073709551615 4289735 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 48.81
Current children cumulated vsize (KiB) 663168

[startup+25.5004 s]
/proc/loadavg: 7.72 8.06 8.03 10/188 16960
/proc/meminfo: memFree=24596080/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=650864 CPUtime=99.94 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 185780 0 0 0 9966 28 0 0 25 0 4 0 515399446 666484736 97317 33554432000 4194304 4623224 140735602964224 18446744073709551615 4278624 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 162716 97317 340 105 0 158220 0
[pid=16946/tid=16954] ppid=16944 vsize=650864 CPUtime=24.85 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 28212 0 0 0 2482 3 0 0 25 0 4 0 515399509 666484736 97317 33554432000 4194304 4623224 140735602964224 18446744073709551615 4376832 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=650864 CPUtime=24.72 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 33825 0 0 0 2466 6 0 0 25 0 4 0 515399509 666484736 97317 33554432000 4194304 4623224 140735602964224 18446744073709551615 4289942 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=650864 CPUtime=24.86 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 26083 0 0 0 2482 4 0 0 25 0 4 0 515399509 666484736 97317 33554432000 4194304 4623224 140735602964224 18446744073709551615 4289757 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 99.94
Current children cumulated vsize (KiB) 650864

[startup+51.1009 s]
/proc/loadavg: 8.01 8.10 8.04 9/188 16961
/proc/meminfo: memFree=24543392/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=725148 CPUtime=202.19 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 241175 0 0 0 20182 37 0 0 25 0 4 0 515399446 742551552 104642 33554432000 4194304 4623224 140735602964224 18446744073709551615 4412446 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 181287 104642 342 105 0 176791 0
[pid=16946/tid=16954] ppid=16944 vsize=725148 CPUtime=50.45 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 39250 0 0 0 5040 5 0 0 25 0 4 0 515399509 742551552 104642 33554432000 4194304 4623224 140735602964224 18446744073709551615 4377038 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=725148 CPUtime=50.18 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 56491 0 0 0 5008 10 0 0 25 0 4 0 515399509 742551552 104642 33554432000 4194304 4623224 140735602964224 18446744073709551615 4462431 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=725148 CPUtime=50.44 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 36929 0 0 0 5039 5 0 0 25 0 4 0 515399509 742551552 104642 33554432000 4194304 4623224 140735602964224 18446744073709551615 4545376 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 202.19
Current children cumulated vsize (KiB) 725148

[startup+102.307 s]
/proc/loadavg: 8.08 8.11 8.05 9/188 16962
/proc/meminfo: memFree=24461436/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=728720 CPUtime=406.74 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 278133 0 0 0 40628 46 0 0 25 0 4 0 515399446 746209280 118202 33554432000 4194304 4623224 140735602964224 18446744073709551615 4507139 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 182180 118202 342 105 0 177684 0
[pid=16946/tid=16954] ppid=16944 vsize=728720 CPUtime=101.66 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 50215 0 0 0 10158 8 0 0 25 0 4 0 515399509 746209280 118202 33554432000 4194304 4623224 140735602964224 18446744073709551615 4289875 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=728720 CPUtime=101.1 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 75024 0 0 0 10096 14 0 0 25 0 4 0 515399509 746209280 118202 33554432000 4194304 4623224 140735602964224 18446744073709551615 4317194 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=728720 CPUtime=101.64 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 39551 0 0 0 10159 5 0 0 25 0 4 0 515399509 746209280 118202 33554432000 4194304 4623224 140735602964224 18446744073709551615 4289723 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 406.74
Current children cumulated vsize (KiB) 728720

[startup+162.301 s]
/proc/loadavg: 8.26 8.15 8.06 9/188 16964
/proc/meminfo: memFree=24336808/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=754224 CPUtime=646.4 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 344992 0 0 0 64580 60 0 0 25 0 4 0 515399446 772325376 139222 33554432000 4194304 4623224 140735602964224 18446744073709551615 4507171 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 188556 139222 342 105 0 184060 0
[pid=16946/tid=16954] ppid=16944 vsize=754224 CPUtime=161.66 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 66623 0 0 0 16154 12 0 0 25 0 4 0 515399509 772325376 139222 33554432000 4194304 4623224 140735602964224 18446744073709551615 4289723 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=754224 CPUtime=160.77 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 120089 0 0 0 16053 24 0 0 25 0 4 0 515399509 772325376 139222 33554432000 4194304 4623224 140735602964224 18446744073709551615 4317176 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=754224 CPUtime=161.64 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 42197 0 0 0 16158 6 0 0 25 0 4 0 515399509 772325376 139222 33554432000 4194304 4623224 140735602964224 18446744073709551615 4330875 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 646.4
Current children cumulated vsize (KiB) 754224

[startup+222.301 s]
/proc/loadavg: 8.27 8.18 8.08 9/188 16965
/proc/meminfo: memFree=24207712/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=863552 CPUtime=886.07 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 424274 0 0 0 88534 73 0 0 25 0 4 0 515399446 884277248 157298 33554432000 4194304 4623224 140735602964224 18446744073709551615 4510034 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 215888 157298 343 105 0 211392 0
[pid=16946/tid=16954] ppid=16944 vsize=863552 CPUtime=221.65 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 89952 0 0 0 22150 15 0 0 25 0 4 0 515399509 884277248 157298 33554432000 4194304 4623224 140735602964224 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=863552 CPUtime=220.45 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 166157 0 0 0 22013 32 0 0 25 0 4 0 515399509 884277248 157298 33554432000 4194304 4623224 140735602964224 18446744073709551615 4317194 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=863552 CPUtime=221.63 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 51289 0 0 0 22156 7 0 0 25 0 4 0 515399509 884277248 157298 33554432000 4194304 4623224 140735602964224 18446744073709551615 4363755 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 886.07
Current children cumulated vsize (KiB) 863552

[startup+282.301 s]

################
# More data... #
################

[pid=16946] ppid=16944 vsize=1539236 CPUtime=2563.83 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 973283 0 0 0 256196 187 0 0 25 0 4 0 515399446 1576177664 292029 33554432000 4194304 4623224 140735602964224 18446744073709551615 4507139 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 384809 292029 564 105 0 380313 0
[pid=16946/tid=16954] ppid=16944 vsize=1539236 CPUtime=641.67 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 212603 0 0 0 64128 39 0 0 25 0 4 0 515399509 1576177664 292029 33554432000 4194304 4623224 140735602964224 18446744073709551615 4330746 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=1539236 CPUtime=638.17 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 519109 0 0 0 63709 108 0 0 25 0 4 0 515399509 1576177664 292029 33554432000 4194304 4623224 140735602964224 18446744073709551615 4317194 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=1539236 CPUtime=641.64 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 101084 0 0 0 64148 16 0 0 25 0 4 0 515399509 1576177664 292029 33554432000 4194304 4623224 140735602964224 18446744073709551615 4545527 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2563.83
Current children cumulated vsize (KiB) 1539236

[startup+702.3 s]
/proc/loadavg: 8.02 8.09 8.08 9/188 16978
/proc/meminfo: memFree=22874732/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=1645392 CPUtime=2803.47 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 1096374 0 0 0 280139 208 0 0 25 0 4 0 515399446 1684881408 310751 33554432000 4194304 4623224 140735602964224 18446744073709551615 4507171 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 411348 310751 644 105 0 406852 0
[pid=16946/tid=16954] ppid=16944 vsize=1645392 CPUtime=701.67 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 254484 0 0 0 70122 45 0 0 25 0 4 0 515399509 1684881408 310751 33554432000 4194304 4623224 140735602964224 18446744073709551615 4376749 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=1645392 CPUtime=697.84 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 592738 0 0 0 69661 123 0 0 25 0 4 0 515399509 1684881408 310751 33554432000 4194304 4623224 140735602964224 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=1645392 CPUtime=701.64 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 103506 0 0 0 70147 17 0 0 25 0 4 0 515399509 1684881408 310751 33554432000 4194304 4623224 140735602964224 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2803.47
Current children cumulated vsize (KiB) 1645392

[startup+762.307 s]
/proc/loadavg: 8.10 8.10 8.08 9/188 16980
/proc/meminfo: memFree=22733484/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=1718660 CPUtime=3043.17 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 1175276 0 0 0 304093 224 0 0 25 0 4 0 515399446 1759907840 326107 33554432000 4194304 4623224 140735602964224 18446744073709551615 4510034 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 429665 326107 644 105 0 425169 0
[pid=16946/tid=16954] ppid=16944 vsize=1718660 CPUtime=761.67 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 264802 0 0 0 76120 47 0 0 25 0 4 0 515399509 1759907840 326107 33554432000 4194304 4623224 140735602964224 18446744073709551615 4289853 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=1718660 CPUtime=757.51 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 658035 0 0 0 75615 136 0 0 25 0 4 0 515399509 1759907840 326107 33554432000 4194304 4623224 140735602964224 18446744073709551615 4316616 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=1718660 CPUtime=761.64 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 106627 0 0 0 76147 17 0 0 25 0 4 0 515399509 1759907840 326107 33554432000 4194304 4623224 140735602964224 18446744073709551615 4545376 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3043.17
Current children cumulated vsize (KiB) 1718660

[startup+822.301 s]
/proc/loadavg: 8.15 8.13 8.09 9/188 16981
/proc/meminfo: memFree=22660064/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=1722844 CPUtime=3282.84 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 1235154 0 0 0 328047 237 0 0 25 0 4 0 515399446 1764192256 340014 33554432000 4194304 4623224 140735602964224 18446744073709551615 4510181 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 430711 340014 644 105 0 426215 0
[pid=16946/tid=16954] ppid=16944 vsize=1722844 CPUtime=821.67 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 274284 0 0 0 82118 49 0 0 25 0 4 0 515399509 1764192256 340014 33554432000 4194304 4623224 140735602964224 18446744073709551615 4357641 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=1722844 CPUtime=817.19 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 705845 0 0 0 81573 146 0 0 25 0 4 0 515399509 1764192256 340014 33554432000 4194304 4623224 140735602964224 18446744073709551615 4317194 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=1722844 CPUtime=821.63 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 108559 0 0 0 82146 17 0 0 25 0 4 0 515399509 1764192256 340014 33554432000 4194304 4623224 140735602964224 18446744073709551615 4552768 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3282.84
Current children cumulated vsize (KiB) 1722844

[startup+882.3 s]
/proc/loadavg: 8.09 8.11 8.09 9/188 16992
/proc/meminfo: memFree=22586104/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=1766640 CPUtime=3522.5 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 1293610 0 0 0 352003 247 0 0 25 0 4 0 515399446 1809039360 349910 33554432000 4194304 4623224 140735602964224 18446744073709551615 4507318 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 441660 349910 644 105 0 437164 0
[pid=16946/tid=16954] ppid=16944 vsize=1766640 CPUtime=881.67 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 274798 0 0 0 88117 50 0 0 25 0 4 0 515399509 1809039360 349910 33554432000 4194304 4623224 140735602964224 18446744073709551615 4357622 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=1766640 CPUtime=876.86 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 743276 0 0 0 87533 153 0 0 25 0 4 0 515399509 1809039360 349910 33554432000 4194304 4623224 140735602964224 18446744073709551615 4289678 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=1766640 CPUtime=881.63 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 128763 0 0 0 88143 20 0 0 25 0 4 0 515399509 1809039360 349910 33554432000 4194304 4623224 140735602964224 18446744073709551615 4545353 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3522.5
Current children cumulated vsize (KiB) 1766640

[startup+942.301 s]
/proc/loadavg: 8.07 8.10 8.09 9/188 16993
/proc/meminfo: memFree=22521992/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=1835128 CPUtime=3762.18 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 1330217 0 0 0 375962 256 0 0 25 0 4 0 515399446 1879171072 359498 33554432000 4194304 4623224 140735602964224 18446744073709551615 4510896 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 458782 359498 644 105 0 454286 0
[pid=16946/tid=16954] ppid=16944 vsize=1835128 CPUtime=941.67 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 275360 0 0 0 94117 50 0 0 25 0 4 0 515399509 1879171072 359498 33554432000 4194304 4623224 140735602964224 18446744073709551615 4357901 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=1835128 CPUtime=936.53 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 777260 0 0 0 93492 161 0 0 25 0 4 0 515399509 1879171072 359498 33554432000 4194304 4623224 140735602964224 18446744073709551615 4317176 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=1835128 CPUtime=941.63 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 130427 0 0 0 94142 21 0 0 25 0 4 0 515399509 1879171072 359498 33554432000 4194304 4623224 140735602964224 18446744073709551615 4363752 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3762.18
Current children cumulated vsize (KiB) 1835128

[startup+1002.3 s]
/proc/loadavg: 8.14 8.11 8.09 9/188 16995
/proc/meminfo: memFree=22463576/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=1842572 CPUtime=4001.86 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 1372411 0 0 0 399920 266 0 0 25 0 4 0 515399446 1886793728 369199 33554432000 4194304 4623224 140735602964224 18446744073709551615 4509953 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 460643 369199 644 105 0 456147 0
[pid=16946/tid=16954] ppid=16944 vsize=1842572 CPUtime=1001.67 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 275841 0 0 0 100117 50 0 0 25 0 4 0 515399509 1886793728 369199 33554432000 4194304 4623224 140735602964224 18446744073709551615 4357901 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=1842572 CPUtime=996.2 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 816179 0 0 0 99450 170 0 0 25 0 4 0 515399509 1886793728 369199 33554432000 4194304 4623224 140735602964224 18446744073709551615 4462587 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=1842572 CPUtime=1001.63 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 132884 0 0 0 100142 21 0 0 25 0 4 0 515399509 1886793728 369199 33554432000 4194304 4623224 140735602964224 18446744073709551615 4363755 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4001.86
Current children cumulated vsize (KiB) 1842572

[startup+1062.3 s]
/proc/loadavg: 8.29 8.15 8.10 10/189 17000
/proc/meminfo: memFree=22398720/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=1851548 CPUtime=4241.35 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 1428577 0 0 0 423859 276 0 0 25 0 4 0 515399446 1895985152 375227 33554432000 4194304 4623224 140735602964224 18446744073709551615 4507318 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 462887 375227 644 105 0 458391 0
[pid=16946/tid=16954] ppid=16944 vsize=1851548 CPUtime=1061.67 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 276080 0 0 0 106117 50 0 0 25 0 4 0 515399509 1895985152 375227 33554432000 4194304 4623224 140735602964224 18446744073709551615 4357390 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=1851548 CPUtime=1055.7 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 869062 0 0 0 105391 179 0 0 25 0 4 0 515399509 1895985152 375227 33554432000 4194304 4623224 140735602964224 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=1851548 CPUtime=1061.63 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 135794 0 0 0 106141 22 0 0 25 0 4 0 515399509 1895985152 375227 33554432000 4194304 4623224 140735602964224 18446744073709551615 4289723 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4241.35
Current children cumulated vsize (KiB) 1851548

[startup+1122.3 s]
/proc/loadavg: 8.31 8.19 8.12 9/188 17001
/proc/meminfo: memFree=22360924/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=1854072 CPUtime=4480.36 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 1471275 0 0 0 447750 286 0 0 25 0 4 0 515399446 1898569728 379410 33554432000 4194304 4623224 140735602964224 18446744073709551615 4507139 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 463518 379410 644 105 0 459022 0
[pid=16946/tid=16954] ppid=16944 vsize=1854072 CPUtime=1121.67 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 276559 0 0 0 112117 50 0 0 25 0 4 0 515399509 1898569728 379410 33554432000 4194304 4623224 140735602964224 18446744073709551615 4357901 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=1854072 CPUtime=1114.71 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 909446 0 0 0 111282 189 0 0 25 0 4 0 515399509 1898569728 379410 33554432000 4194304 4623224 140735602964224 18446744073709551615 4317180 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=1854072 CPUtime=1121.63 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 137192 0 0 0 112141 22 0 0 25 0 4 0 515399509 1898569728 379410 33554432000 4194304 4623224 140735602964224 18446744073709551615 4363755 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4480.36
Current children cumulated vsize (KiB) 1854072

[startup+1182.3 s]
/proc/loadavg: 8.17 8.17 8.11 9/188 17003
/proc/meminfo: memFree=22299288/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=1925948 CPUtime=4720.04 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 1515120 0 0 0 471710 294 0 0 25 0 4 0 515399446 1972170752 385867 33554432000 4194304 4623224 140735602964224 18446744073709551615 4509955 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 481487 385867 644 105 0 476991 0
[pid=16946/tid=16954] ppid=16944 vsize=1925948 CPUtime=1181.68 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 277179 0 0 0 118118 50 0 0 25 0 4 0 515399509 1972170752 385867 33554432000 4194304 4623224 140735602964224 18446744073709551615 4357984 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=1925948 CPUtime=1174.38 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 950007 0 0 0 117242 196 0 0 25 0 4 0 515399509 1972170752 385867 33554432000 4194304 4623224 140735602964224 18446744073709551615 4317194 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=1925948 CPUtime=1181.63 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 139515 0 0 0 118141 22 0 0 25 0 4 0 515399509 1972170752 385867 33554432000 4194304 4623224 140735602964224 18446744073709551615 4364354 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4720.04
Current children cumulated vsize (KiB) 1925948



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.4 s]
/proc/loadavg: 8.26 8.19 8.12 10/188 17003
/proc/meminfo: memFree=22253400/32951124 swapFree=67111524/67111528
[pid=16946] ppid=16944 vsize=1960164 CPUtime=4800.34 cores=1,3,5,7
/proc/16946/stat : 16946 (SArTagnan) R 16944 16946 16820 0 -1 4202496 1543075 0 0 0 479734 300 0 0 25 0 4 0 515399446 2007207936 392945 33554432000 4194304 4623224 140735602964224 18446744073709551615 4497487 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/16946/statm: 490041 392945 423 105 0 485545 0
[pid=16946/tid=16954] ppid=16944 vsize=1960164 CPUtime=1201.78 cores=1,3,5,7
/proc/16946/task/16954/stat : 16954 (SArTagnan) R 16944 16946 16820 0 -1 4202560 277280 0 0 0 120128 50 0 0 25 0 4 0 515399509 2007207936 392946 33554432000 4194304 4623224 140735602964224 18446744073709551615 4357901 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=16946/tid=16955] ppid=16944 vsize=1960164 CPUtime=1194.37 cores=1,3,5,7
/proc/16946/task/16955/stat : 16955 (SArTagnan) R 16944 16946 16820 0 -1 4202560 967180 0 0 0 119237 200 0 0 25 0 4 0 515399509 2007207936 392946 33554432000 4194304 4623224 140735602964224 18446744073709551615 4289917 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=16946/tid=16956] ppid=16944 vsize=1960164 CPUtime=1201.74 cores=1,3,5,7
/proc/16946/task/16956/stat : 16956 (SArTagnan) R 16944 16946 16820 0 -1 4202560 140060 0 0 0 120152 22 0 0 25 0 4 0 515399509 2007207936 392946 33554432000 4194304 4623224 140735602964224 18446744073709551615 4364107 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.34
Current children cumulated vsize (KiB) 1960164

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.48
CPU time (s): 4800.44
CPU user time (s): 4797.37
CPU system time (s): 3.07053
CPU usage (%): 399.212
Max. virtual memory (cumulated for all children) (KiB): 1992928

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.37
system time used= 3.07053
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1543083
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 173
involuntary context switches= 32565

runsolver used 1.78773 second user time and 4.78227 second system time

The end

Launcher Data 

Begin job on node132 at 2011-04-29 02:46:53
IDJOB=3280665
IDBENCH=20410
IDSOLVER=1656
FILE ID=node132/3280665-1304038012
RUNJOBID= node132-1304036807-16839
PBS_JOBID= 13170365
Free space on /tmp= 71836 MiB

SOLVER NAME= SArTagnan 2011-03-24
BENCH NAME= SAT07/industrial/palacios/hard/sortnet-8-ipc5-h19-sat.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3280665-1304038012/watcher-3280665-1304038012 -o /tmp/evaluation-result-3280665-1304038012/solver-3280665-1304038012 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3280665-1304038012.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 409f4401e9e8b60ee665ac5567a13e41
RANDOM SEED=1854936353

node132.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25433592 kB
Buffers:        739016 kB
Cached:        6220548 kB
SwapCached:          4 kB
Active:        1589832 kB
Inactive:      5438100 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25433592 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           40012 kB
Writeback:           0 kB
AnonPages:       68036 kB
Mapped:          14320 kB
Slab:           426144 kB
PageTables:       4404 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   192644 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71824 MiB
End job on node132 at 2011-04-29 03:06:57