Trace number 3438586

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 40000.8 5004.06

General information on the benchmark

NameCRAFTED/rbsat/crafted/
forced/rbsat-v2640c305320gyes3.cnf
MD5SUM3b033385d50481e5bd2fe507f2941f4f
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2640
Number of clauses305320
Sum of the clauses size613120
Maximum clause length33
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2305240
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 580

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3438586-1307660303/watcher-3438586-1307660303 -o /tmp/evaluation-result-3438586-1307660303/solver-3438586-1307660303 -C 40000 -W 5100 -M 31000 SArTagnan HOME/instance-3438586-1307660303.cnf -threads=8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.00 8.03 8.00 2/170 12581
/proc/meminfo: memFree=32574588/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=26252 CPUtime=0 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 568 0 0 0 0 0 0 0 25 0 1 0 877630798 26882048 506 33554432000 4194304 4625164 140734578062048 18446744073709551615 4339447 0 0 4096 2 0 0 0 17 4 0 0 0
/proc/12581/statm: 6563 485 296 106 0 2066 0

[startup+0.045169 s]
/proc/loadavg: 8.00 8.03 8.00 2/170 12581
/proc/meminfo: memFree=32574588/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=29064 CPUtime=0.04 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 2560 0 0 0 4 0 0 0 25 0 1 0 877630798 29761536 1007 33554432000 4194304 4625164 140734578062048 18446744073709551615 4585971 0 0 4096 2 0 0 0 17 4 0 0 0
/proc/12581/statm: 7266 1007 296 106 0 2769 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 29064

[startup+0.105839 s]
/proc/loadavg: 8.00 8.03 8.00 2/170 12581
/proc/meminfo: memFree=32574588/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=267376 CPUtime=0.19 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 10557 0 0 0 19 0 0 0 25 0 8 0 877630798 273793024 8212 33554432000 4194304 4625164 140734578062048 18446744073709551615 4503594 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/12581/statm: 66844 8213 337 106 0 62347 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 267376

[startup+1.12635 s]
/proc/loadavg: 8.00 8.03 8.00 2/170 12581
/proc/meminfo: memFree=32574588/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=490312 CPUtime=8.16 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 36879 0 0 0 811 5 0 0 25 0 8 0 877630798 502079488 27799 33554432000 4194304 4625164 140734578062048 18446744073709551615 4518211 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/12581/statm: 122578 27804 350 106 0 118081 0
Current children cumulated CPU time (s) 8.16
Current children cumulated vsize (KiB) 490312

[startup+1.25091 s]
/proc/loadavg: 8.00 8.03 8.00 2/170 12581
/proc/meminfo: memFree=32574588/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=498880 CPUtime=9.09 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 39136 0 0 0 904 5 0 0 25 0 8 0 877630798 510853120 28806 33554432000 4194304 4625164 140734578062048 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/12581/statm: 124720 28806 350 106 0 120223 0
Current children cumulated CPU time (s) 9.09
Current children cumulated vsize (KiB) 498880

[startup+1.55086 s]
/proc/loadavg: 8.00 8.03 8.00 2/170 12581
/proc/meminfo: memFree=32574588/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=498880 CPUtime=11.49 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 43604 0 0 0 1143 6 0 0 25 0 8 0 877630798 510853120 33114 33554432000 4194304 4625164 140734578062048 18446744073709551615 4511874 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/12581/statm: 124720 33115 350 106 0 120223 0
Current children cumulated CPU time (s) 11.49
Current children cumulated vsize (KiB) 498880

[startup+3.10057 s]
/proc/loadavg: 7.44 7.91 7.96 9/178 12589
/proc/meminfo: memFree=32402312/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=507424 CPUtime=23.87 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 64887 0 0 0 2377 10 0 0 25 0 8 0 877630798 519602176 45822 33554432000 4194304 4625164 140734578062048 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/12581/statm: 126856 45822 354 106 0 122359 0
[pid=12581/tid=12583] ppid=12579 vsize=507424 CPUtime=3 cores=0-7
/proc/12581/task/12583/stat : 12583 (SArTagnan) R 12579 12581 12360 0 -1 4202560 5087 0 0 0 300 0 0 0 23 0 8 0 877630807 519602176 45822 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12581/tid=12584] ppid=12579 vsize=507424 CPUtime=2.99 cores=0-7
/proc/12581/task/12584/stat : 12584 (SArTagnan) R 12579 12581 12360 0 -1 4202560 4454 0 0 0 299 0 0 0 24 0 8 0 877630807 519602176 45822 33554432000 4194304 4625164 140734578062048 18446744073709551615 4465246 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=12581/tid=12585] ppid=12579 vsize=507424 CPUtime=2.73 cores=0-7
/proc/12581/task/12585/stat : 12585 (SArTagnan) R 12579 12581 12360 0 -1 4202560 8558 0 0 0 272 1 0 0 25 0 8 0 877630807 519602176 45822 33554432000 4194304 4625164 140734578062048 18446744073709551615 230808862883 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=12581/tid=12586] ppid=12579 vsize=507424 CPUtime=3.01 cores=0-7
/proc/12581/task/12586/stat : 12586 (SArTagnan) R 12579 12581 12360 0 -1 4202560 5750 0 0 0 300 1 0 0 25 0 8 0 877630807 519602176 45822 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=12581/tid=12587] ppid=12579 vsize=507424 CPUtime=3 cores=0-7
/proc/12581/task/12587/stat : 12587 (SArTagnan) R 12579 12581 12360 0 -1 4202560 4245 0 0 0 300 0 0 0 25 0 8 0 877630807 519602176 45822 33554432000 4194304 4625164 140734578062048 18446744073709551615 4318472 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12581/tid=12588] ppid=12579 vsize=507424 CPUtime=3 cores=0-7
/proc/12581/task/12588/stat : 12588 (SArTagnan) R 12579 12581 12360 0 -1 4202560 6113 0 0 0 299 1 0 0 24 0 8 0 877630807 519602176 45822 33554432000 4194304 4625164 140734578062048 18446744073709551615 230825149467 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12581/tid=12589] ppid=12579 vsize=507424 CPUtime=2.98 cores=0-7
/proc/12581/task/12589/stat : 12589 (SArTagnan) R 12579 12581 12360 0 -1 4202560 4675 0 0 0 298 0 0 0 24 0 8 0 877630807 519602176 45822 33554432000 4194304 4625164 140734578062048 18446744073709551615 230808833026 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 23.87
Current children cumulated vsize (KiB) 507424

[startup+6.30699 s]
/proc/loadavg: 7.49 7.91 7.96 9/178 12589
/proc/meminfo: memFree=32315208/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=533180 CPUtime=49.51 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 92815 0 0 0 4936 15 0 0 25 0 8 0 877630798 545976320 65626 33554432000 4194304 4625164 140734578062048 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/12581/statm: 133295 65626 364 106 0 128798 0
[pid=12581/tid=12583] ppid=12579 vsize=533180 CPUtime=6.21 cores=0-7
/proc/12581/task/12583/stat : 12583 (SArTagnan) R 12579 12581 12360 0 -1 4202560 7289 0 0 0 620 1 0 0 25 0 8 0 877630807 545976320 65626 33554432000 4194304 4625164 140734578062048 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12581/tid=12584] ppid=12579 vsize=533180 CPUtime=6.2 cores=0-7
/proc/12581/task/12584/stat : 12584 (SArTagnan) R 12579 12581 12360 0 -1 4202560 6425 0 0 0 619 1 0 0 25 0 8 0 877630807 545976320 65626 33554432000 4194304 4625164 140734578062048 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=12581/tid=12585] ppid=12579 vsize=533180 CPUtime=5.94 cores=0-7
/proc/12581/task/12585/stat : 12585 (SArTagnan) R 12579 12581 12360 0 -1 4202560 13109 0 0 0 592 2 0 0 25 0 8 0 877630807 545976320 65626 33554432000 4194304 4625164 140734578062048 18446744073709551615 4334485 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=12581/tid=12586] ppid=12579 vsize=533180 CPUtime=6.21 cores=0-7
/proc/12581/task/12586/stat : 12586 (SArTagnan) R 12579 12581 12360 0 -1 4202560 8369 0 0 0 620 1 0 0 25 0 8 0 877630807 545976320 65626 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=12581/tid=12587] ppid=12579 vsize=533180 CPUtime=6.21 cores=0-7
/proc/12581/task/12587/stat : 12587 (SArTagnan) R 12579 12581 12360 0 -1 4202560 6957 0 0 0 620 1 0 0 25 0 8 0 877630807 545976320 65626 33554432000 4194304 4625164 140734578062048 18446744073709551615 4348903 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12581/tid=12588] ppid=12579 vsize=533180 CPUtime=6.21 cores=0-7
/proc/12581/task/12588/stat : 12588 (SArTagnan) R 12579 12581 12360 0 -1 4202560 9593 0 0 0 620 1 0 0 25 0 8 0 877630807 545976320 65626 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12581/tid=12589] ppid=12579 vsize=533180 CPUtime=6.16 cores=0-7
/proc/12581/task/12589/stat : 12589 (SArTagnan) R 12579 12581 12360 0 -1 4202560 6947 0 0 0 616 0 0 0 18 0 8 0 877630807 545976320 65626 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 49.51
Current children cumulated vsize (KiB) 533180

[startup+12.7008 s]
/proc/loadavg: 7.53 7.91 7.96 9/178 12589
/proc/meminfo: memFree=32241452/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=629948 CPUtime=100.62 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 145272 0 0 0 10041 21 0 0 25 0 8 0 877630798 645066752 89803 33554432000 4194304 4625164 140734578062048 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/12581/statm: 157487 89803 417 106 0 152990 0
[pid=12581/tid=12583] ppid=12579 vsize=629948 CPUtime=12.6 cores=0-7
/proc/12581/task/12583/stat : 12583 (SArTagnan) R 12579 12581 12360 0 -1 4202560 11353 0 0 0 1259 1 0 0 25 0 8 0 877630807 645066752 89804 33554432000 4194304 4625164 140734578062048 18446744073709551615 4352072 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12581/tid=12584] ppid=12579 vsize=629948 CPUtime=12.59 cores=0-7
/proc/12581/task/12584/stat : 12584 (SArTagnan) R 12579 12581 12360 0 -1 4202560 9803 0 0 0 1258 1 0 0 25 0 8 0 877630807 645066752 89804 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=12581/tid=12585] ppid=12579 vsize=629948 CPUtime=12.34 cores=0-7
/proc/12581/task/12585/stat : 12585 (SArTagnan) R 12579 12581 12360 0 -1 4202560 34847 0 0 0 1230 4 0 0 25 0 8 0 877630807 645066752 89804 33554432000 4194304 4625164 140734578062048 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=12581/tid=12586] ppid=12579 vsize=629948 CPUtime=12.61 cores=0-7
/proc/12581/task/12586/stat : 12586 (SArTagnan) R 12579 12581 12360 0 -1 4202560 13314 0 0 0 1259 2 0 0 25 0 8 0 877630807 645066752 89805 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=12581/tid=12587] ppid=12579 vsize=629948 CPUtime=12.6 cores=0-7
/proc/12581/task/12587/stat : 12587 (SArTagnan) R 12579 12581 12360 0 -1 4202560 11204 0 0 0 1259 1 0 0 25 0 8 0 877630807 645066752 89805 33554432000 4194304 4625164 140734578062048 18446744073709551615 4349019 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12581/tid=12588] ppid=12579 vsize=629948 CPUtime=12.6 cores=0-7
/proc/12581/task/12588/stat : 12588 (SArTagnan) R 12579 12581 12360 0 -1 4202560 10684 0 0 0 1259 1 0 0 25 0 8 0 877630807 645066752 89806 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12581/tid=12589] ppid=12579 vsize=629948 CPUtime=12.52 cores=0-7
/proc/12581/task/12589/stat : 12589 (SArTagnan) R 12579 12581 12360 0 -1 4202560 7527 0 0 0 1252 0 0 0 25 0 8 0 877630807 645066752 89806 33554432000 4194304 4625164 140734578062048 18446744073709551615 4352280 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 100.62
Current children cumulated vsize (KiB) 629948

[startup+25.5005 s]
/proc/loadavg: 7.60 7.92 7.96 9/178 12590
/proc/meminfo: memFree=32091552/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=740224 CPUtime=202.95 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 232219 0 0 0 20262 33 0 0 25 0 8 0 877630798 757989376 122641 33554432000 4194304 4625164 140734578062048 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/12581/statm: 185056 122641 430 106 0 180559 0
[pid=12581/tid=12583] ppid=12579 vsize=740224 CPUtime=25.41 cores=0-7
/proc/12581/task/12583/stat : 12583 (SArTagnan) R 12579 12581 12360 0 -1 4202560 15699 0 0 0 2539 2 0 0 25 0 8 0 877630807 757989376 122641 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351975 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12581/tid=12584] ppid=12579 vsize=740224 CPUtime=25.39 cores=0-7
/proc/12581/task/12584/stat : 12584 (SArTagnan) R 12579 12581 12360 0 -1 4202560 13997 0 0 0 2538 1 0 0 25 0 8 0 877630807 757989376 122641 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=12581/tid=12585] ppid=12579 vsize=740224 CPUtime=25.14 cores=0-7
/proc/12581/task/12585/stat : 12585 (SArTagnan) R 12579 12581 12360 0 -1 4202560 85025 0 0 0 2504 10 0 0 25 0 8 0 877630807 757989376 122641 33554432000 4194304 4625164 140734578062048 18446744073709551615 4547221 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=12581/tid=12586] ppid=12579 vsize=740224 CPUtime=25.4 cores=0-7
/proc/12581/task/12586/stat : 12586 (SArTagnan) R 12579 12581 12360 0 -1 4202560 14943 0 0 0 2538 2 0 0 25 0 8 0 877630807 757989376 122641 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=12581/tid=12587] ppid=12579 vsize=740224 CPUtime=25.4 cores=0-7
/proc/12581/task/12587/stat : 12587 (SArTagnan) R 12579 12581 12360 0 -1 4202560 12581 0 0 0 2538 2 0 0 25 0 8 0 877630807 757989376 122641 33554432000 4194304 4625164 140734578062048 18446744073709551615 4348903 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12581/tid=12588] ppid=12579 vsize=740224 CPUtime=25.4 cores=0-7
/proc/12581/task/12588/stat : 12588 (SArTagnan) R 12579 12581 12360 0 -1 4202560 15696 0 0 0 2538 2 0 0 25 0 8 0 877630807 757989376 122641 33554432000 4194304 4625164 140734578062048 18446744073709551615 4339569 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12581/tid=12589] ppid=12579 vsize=740224 CPUtime=25.25 cores=0-7
/proc/12581/task/12589/stat : 12589 (SArTagnan) R 12579 12581 12360 0 -1 4202560 10252 0 0 0 2524 1 0 0 25 0 8 0 877630807 757989376 122641 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 202.95
Current children cumulated vsize (KiB) 740224

[startup+51.1008 s]
/proc/loadavg: 7.76 7.92 7.96 9/178 12590
/proc/meminfo: memFree=31909208/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=975360 CPUtime=407.61 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 404094 0 0 0 40702 59 0 0 25 0 8 0 877630798 998768640 167982 33554432000 4194304 4625164 140734578062048 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/12581/statm: 243840 167982 565 106 0 239343 0
[pid=12581/tid=12583] ppid=12579 vsize=975360 CPUtime=51.01 cores=0-7
/proc/12581/task/12583/stat : 12583 (SArTagnan) R 12579 12581 12360 0 -1 4202560 21986 0 0 0 5098 3 0 0 25 0 8 0 877630807 998768640 167982 33554432000 4194304 4625164 140734578062048 18446744073709551615 4352029 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12581/tid=12584] ppid=12579 vsize=975360 CPUtime=50.99 cores=0-7
/proc/12581/task/12584/stat : 12584 (SArTagnan) R 12579 12581 12360 0 -1 4202560 15312 0 0 0 5097 2 0 0 25 0 8 0 877630807 998768640 167982 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=12581/tid=12585] ppid=12579 vsize=975360 CPUtime=50.74 cores=0-7
/proc/12581/task/12585/stat : 12585 (SArTagnan) R 12579 12581 12360 0 -1 4202560 197938 0 0 0 5048 26 0 0 25 0 8 0 877630807 998768640 167982 33554432000 4194304 4625164 140734578062048 18446744073709551615 230808826913 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=12581/tid=12586] ppid=12579 vsize=975360 CPUtime=51 cores=0-7
/proc/12581/task/12586/stat : 12586 (SArTagnan) R 12579 12581 12360 0 -1 4202560 23097 0 0 0 5097 3 0 0 25 0 8 0 877630807 998768640 167982 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=12581/tid=12587] ppid=12579 vsize=975360 CPUtime=51.01 cores=0-7
/proc/12581/task/12587/stat : 12587 (SArTagnan) R 12579 12581 12360 0 -1 4202560 20647 0 0 0 5097 4 0 0 25 0 8 0 877630807 998768640 167982 33554432000 4194304 4625164 140734578062048 18446744073709551615 4349325 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12581/tid=12588] ppid=12579 vsize=975360 CPUtime=51 cores=0-7
/proc/12581/task/12588/stat : 12588 (SArTagnan) R 12579 12581 12360 0 -1 4202560 24659 0 0 0 5096 4 0 0 25 0 8 0 877630807 998768640 167982 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12581/tid=12589] ppid=12579 vsize=975360 CPUtime=50.71 cores=0-7
/proc/12581/task/12589/stat : 12589 (SArTagnan) R 12579 12581 12360 0 -1 4202560 11101 0 0 0 5070 1 0 0 25 0 8 0 877630807 998768640 167982 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 407.61
Current children cumulated vsize (KiB) 975360

[startup+102.3 s]
/proc/loadavg: 7.89 7.93 7.96 9/178 12592
/proc/meminfo: memFree=31712264/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=1313568 CPUtime=816.92 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 792714 0 0 0 81573 119 0 0 25 0 8 0 877630798 1345093632 217890 33554432000 4194304 4625164 140734578062048 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/12581/statm: 328392 217890 426 106 0 323895 0
[pid=12581/tid=12583] ppid=12579 vsize=1313568 CPUtime=102.21 cores=0-7
/proc/12581/task/12583/stat : 12583 (SArTagnan) R 12579 12581 12360 0 -1 4202560 31127 0 0 0 10216 5 0 0 25 0 8 0 877630807 1345093632 217890 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12581/tid=12584] ppid=12579 vsize=1313568 CPUtime=102.18 cores=0-7
/proc/12581/task/12584/stat : 12584 (SArTagnan) R 12579 12581 12360 0 -1 4202560 22860 0 0 0 10214 4 0 0 25 0 8 0 877630807 1345093632 217890 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=12581/tid=12585] ppid=12579 vsize=1313568 CPUtime=101.94 cores=0-7
/proc/12581/task/12585/stat : 12585 (SArTagnan) R 12579 12581 12360 0 -1 4202560 517032 0 0 0 10124 70 0 0 25 0 8 0 877630807 1345093632 217890 33554432000 4194304 4625164 140734578062048 18446744073709551615 4352296 0 0 4096 2 0 0 0 -1 2 0 0 0

################
# More data... #
################

[pid=12581/tid=12585] ppid=12579 vsize=6737124 CPUtime=4661.63 cores=0-7
/proc/12581/task/12585/stat : 12585 (SArTagnan) R 12579 12581 12360 0 -1 4202560 30265013 0 0 0 461468 4695 0 0 25 0 8 0 877630807 6898814976 1424036 33554432000 4194304 4625164 140734578062048 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=12581/tid=12586] ppid=12579 vsize=6737124 CPUtime=4661.64 cores=0-7
/proc/12581/task/12586/stat : 12586 (SArTagnan) R 12579 12581 12360 0 -1 4202560 133505 0 0 0 466138 26 0 0 25 0 8 0 877630807 6898814976 1424036 33554432000 4194304 4625164 140734578062048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=12581/tid=12587] ppid=12579 vsize=6737124 CPUtime=4662.32 cores=0-7
/proc/12581/task/12587/stat : 12587 (SArTagnan) R 12579 12581 12360 0 -1 4202560 129851 0 0 0 466197 35 0 0 25 0 8 0 877630807 6898814976 1424036 33554432000 4194304 4625164 140734578062048 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12581/tid=12588] ppid=12579 vsize=6737124 CPUtime=4662.32 cores=0-7
/proc/12581/task/12588/stat : 12588 (SArTagnan) R 12579 12581 12360 0 -1 4202560 169640 0 0 0 466192 40 0 0 25 0 8 0 877630807 6898814976 1424037 33554432000 4194304 4625164 140734578062048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12581/tid=12589] ppid=12579 vsize=6737124 CPUtime=4636.87 cores=0-7
/proc/12581/task/12589/stat : 12589 (SArTagnan) R 12579 12581 12360 0 -1 4202560 87126 0 0 0 463632 55 0 0 25 0 8 0 877630807 6898814976 1424037 33554432000 4194304 4625164 140734578062048 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 37270.4
Current children cumulated vsize (KiB) 6737124

[startup+4722.3 s]
/proc/loadavg: 8.02 8.04 8.00 9/178 12739
/proc/meminfo: memFree=26829476/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=6744028 CPUtime=37750.1 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 34252631 0 0 0 3769374 5639 0 0 25 0 8 0 877630798 6905884672 1434882 33554432000 4194304 4625164 140734578062048 18446744073709551615 4393110 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12581/statm: 1686007 1434882 349 106 0 1681510 0
[pid=12581/tid=12583] ppid=12579 vsize=6744028 CPUtime=4722.15 cores=0-7
/proc/12581/task/12583/stat : 12583 (SArTagnan) R 12579 12581 12360 0 -1 4202560 131287 0 0 0 472188 27 0 0 25 0 8 0 877630807 6905884672 1434882 33554432000 4194304 4625164 140734578062048 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=12581/tid=12584] ppid=12579 vsize=6744028 CPUtime=4721.17 cores=0-7
/proc/12581/task/12584/stat : 12584 (SArTagnan) R 12579 12581 12360 0 -1 4202560 309155 0 0 0 471859 258 0 0 25 0 8 0 877630807 6905884672 1434882 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=12581/tid=12585] ppid=12579 vsize=6744028 CPUtime=4721.63 cores=0-7
/proc/12581/task/12585/stat : 12585 (SArTagnan) R 12579 12581 12360 0 -1 4202560 30743974 0 0 0 467392 4771 0 0 25 0 8 0 877630807 6905884672 1434882 33554432000 4194304 4625164 140734578062048 18446744073709551615 4560031 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=12581/tid=12586] ppid=12579 vsize=6744028 CPUtime=4721.64 cores=0-7
/proc/12581/task/12586/stat : 12586 (SArTagnan) R 12579 12581 12360 0 -1 4202560 133699 0 0 0 472138 26 0 0 25 0 8 0 877630807 6905884672 1434882 33554432000 4194304 4625164 140734578062048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=12581/tid=12587] ppid=12579 vsize=6744028 CPUtime=4722.32 cores=0-7
/proc/12581/task/12587/stat : 12587 (SArTagnan) R 12579 12581 12360 0 -1 4202560 130060 0 0 0 472197 35 0 0 25 0 8 0 877630807 6905884672 1434882 33554432000 4194304 4625164 140734578062048 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12581/tid=12588] ppid=12579 vsize=6744028 CPUtime=4722.32 cores=0-7
/proc/12581/task/12588/stat : 12588 (SArTagnan) R 12579 12581 12360 0 -1 4202560 169735 0 0 0 472192 40 0 0 25 0 8 0 877630807 6905884672 1434882 33554432000 4194304 4625164 140734578062048 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12581/tid=12589] ppid=12579 vsize=6744028 CPUtime=4696.55 cores=0-7
/proc/12581/task/12589/stat : 12589 (SArTagnan) R 12579 12581 12360 0 -1 4202560 88147 0 0 0 469599 56 0 0 25 0 8 0 877630807 6905884672 1434882 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 37750.1
Current children cumulated vsize (KiB) 6744028

[startup+4782.3 s]
/proc/loadavg: 8.00 8.03 8.00 9/178 12741
/proc/meminfo: memFree=26755276/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=6853660 CPUtime=38229.8 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 34710714 0 0 0 3817257 5723 0 0 25 0 8 0 877630798 7018147840 1453111 33554432000 4194304 4625164 140734578062048 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12581/statm: 1713415 1453111 349 106 0 1708918 0
[pid=12581/tid=12583] ppid=12579 vsize=6853660 CPUtime=4782.14 cores=0-7
/proc/12581/task/12583/stat : 12583 (SArTagnan) R 12579 12581 12360 0 -1 4202560 131452 0 0 0 478187 27 0 0 25 0 8 0 877630807 7018147840 1453111 33554432000 4194304 4625164 140734578062048 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=12581/tid=12584] ppid=12579 vsize=6853660 CPUtime=4781.16 cores=0-7
/proc/12581/task/12584/stat : 12584 (SArTagnan) R 12579 12581 12360 0 -1 4202560 314303 0 0 0 477852 264 0 0 25 0 8 0 877630807 7018147840 1453111 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=12581/tid=12585] ppid=12579 vsize=6853660 CPUtime=4781.63 cores=0-7
/proc/12581/task/12585/stat : 12585 (SArTagnan) R 12579 12581 12360 0 -1 4202560 31177457 0 0 0 473323 4840 0 0 25 0 8 0 877630807 7018147840 1453111 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=12581/tid=12586] ppid=12579 vsize=6853660 CPUtime=4781.65 cores=0-7
/proc/12581/task/12586/stat : 12586 (SArTagnan) R 12579 12581 12360 0 -1 4202560 133923 0 0 0 478138 27 0 0 25 0 8 0 877630807 7018147840 1453111 33554432000 4194304 4625164 140734578062048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=12581/tid=12587] ppid=12579 vsize=6853660 CPUtime=4782.32 cores=0-7
/proc/12581/task/12587/stat : 12587 (SArTagnan) R 12579 12581 12360 0 -1 4202560 130228 0 0 0 478197 35 0 0 25 0 8 0 877630807 7018147840 1453111 33554432000 4194304 4625164 140734578062048 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12581/tid=12588] ppid=12579 vsize=6853660 CPUtime=4782.33 cores=0-7
/proc/12581/task/12588/stat : 12588 (SArTagnan) R 12579 12581 12360 0 -1 4202560 170000 0 0 0 478192 41 0 0 25 0 8 0 877630807 7018147840 1453111 33554432000 4194304 4625164 140734578062048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12581/tid=12589] ppid=12579 vsize=6853660 CPUtime=4756.22 cores=0-7
/proc/12581/task/12589/stat : 12589 (SArTagnan) R 12579 12581 12360 0 -1 4202560 90198 0 0 0 475562 60 0 0 25 0 8 0 877630807 7018147840 1453111 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38229.8
Current children cumulated vsize (KiB) 6853660

[startup+4842.3 s]
/proc/loadavg: 8.00 8.02 8.00 9/178 12742
/proc/meminfo: memFree=26707628/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=6853824 CPUtime=38709.5 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 35124341 0 0 0 3865155 5793 0 0 25 0 8 0 877630798 7018315776 1464175 33554432000 4194304 4625164 140734578062048 18446744073709551615 4393080 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12581/statm: 1713456 1464175 349 106 0 1708959 0
[pid=12581/tid=12583] ppid=12579 vsize=6853824 CPUtime=4842.14 cores=0-7
/proc/12581/task/12583/stat : 12583 (SArTagnan) R 12579 12581 12360 0 -1 4202560 131620 0 0 0 484187 27 0 0 25 0 8 0 877630807 7018315776 1464175 33554432000 4194304 4625164 140734578062048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=12581/tid=12584] ppid=12579 vsize=6853824 CPUtime=4841.17 cores=0-7
/proc/12581/task/12584/stat : 12584 (SArTagnan) R 12579 12581 12360 0 -1 4202560 316161 0 0 0 483849 268 0 0 25 0 8 0 877630807 7018315776 1464175 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=12581/tid=12585] ppid=12579 vsize=6853824 CPUtime=4841.63 cores=0-7
/proc/12581/task/12585/stat : 12585 (SArTagnan) R 12579 12581 12360 0 -1 4202560 31572736 0 0 0 479261 4902 0 0 25 0 8 0 877630807 7018315776 1464175 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=12581/tid=12586] ppid=12579 vsize=6853824 CPUtime=4841.65 cores=0-7
/proc/12581/task/12586/stat : 12586 (SArTagnan) R 12579 12581 12360 0 -1 4202560 134136 0 0 0 484138 27 0 0 25 0 8 0 877630807 7018315776 1464175 33554432000 4194304 4625164 140734578062048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=12581/tid=12587] ppid=12579 vsize=6853824 CPUtime=4842.32 cores=0-7
/proc/12581/task/12587/stat : 12587 (SArTagnan) R 12579 12581 12360 0 -1 4202560 130354 0 0 0 484197 35 0 0 25 0 8 0 877630807 7018315776 1464175 33554432000 4194304 4625164 140734578062048 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12581/tid=12588] ppid=12579 vsize=6853824 CPUtime=4842.33 cores=0-7
/proc/12581/task/12588/stat : 12588 (SArTagnan) R 12579 12581 12360 0 -1 4202560 170144 0 0 0 484192 41 0 0 25 0 8 0 877630807 7018315776 1464175 33554432000 4194304 4625164 140734578062048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12581/tid=12589] ppid=12579 vsize=6853824 CPUtime=4815.9 cores=0-7
/proc/12581/task/12589/stat : 12589 (SArTagnan) R 12579 12581 12360 0 -1 4202560 92069 0 0 0 481528 62 0 0 25 0 8 0 877630807 7018315776 1464175 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38709.5
Current children cumulated vsize (KiB) 6853824

[startup+4902.3 s]
/proc/loadavg: 8.00 8.02 8.00 9/178 12744
/proc/meminfo: memFree=26391080/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=8003944 CPUtime=39189 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 36196056 0 0 0 3912951 5953 0 0 25 0 8 0 877630798 8196038656 1543449 33554432000 4194304 4625164 140734578062048 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/12581/statm: 2000986 1543450 349 106 0 1996489 0
[pid=12581/tid=12583] ppid=12579 vsize=8003944 CPUtime=4902.14 cores=0-7
/proc/12581/task/12583/stat : 12583 (SArTagnan) R 12579 12581 12360 0 -1 4202560 131805 0 0 0 490187 27 0 0 25 0 8 0 877630807 8196038656 1543451 33554432000 4194304 4625164 140734578062048 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=12581/tid=12584] ppid=12579 vsize=8003944 CPUtime=4901.1 cores=0-7
/proc/12581/task/12584/stat : 12584 (SArTagnan) R 12579 12581 12360 0 -1 4202560 320072 0 0 0 489839 271 0 0 25 0 8 0 877630807 8196038656 1543452 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=12581/tid=12585] ppid=12579 vsize=8003944 CPUtime=4901.55 cores=0-7
/proc/12581/task/12585/stat : 12585 (SArTagnan) R 12579 12581 12360 0 -1 4202560 31993836 0 0 0 485178 4977 0 0 25 0 8 0 877630807 8196038656 1543452 33554432000 4194304 4625164 140734578062048 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12581/tid=12586] ppid=12579 vsize=8003944 CPUtime=4901.66 cores=0-7
/proc/12581/task/12586/stat : 12586 (SArTagnan) R 12579 12581 12360 0 -1 4202560 169427 0 0 0 490134 32 0 0 25 0 8 0 877630807 8196038656 1543452 33554432000 4194304 4625164 140734578062048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=12581/tid=12587] ppid=12579 vsize=8003944 CPUtime=4902.33 cores=0-7
/proc/12581/task/12587/stat : 12587 (SArTagnan) R 12579 12581 12360 0 -1 4202560 130557 0 0 0 490198 35 0 0 25 0 8 0 877630807 8196038656 1543452 33554432000 4194304 4625164 140734578062048 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12581/tid=12588] ppid=12579 vsize=8003944 CPUtime=4902.33 cores=0-7
/proc/12581/task/12588/stat : 12588 (SArTagnan) R 12579 12581 12360 0 -1 4202560 170275 0 0 0 490192 41 0 0 25 0 8 0 877630807 8196038656 1543452 33554432000 4194304 4625164 140734578062048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12581/tid=12589] ppid=12579 vsize=8003944 CPUtime=4875.57 cores=0-7
/proc/12581/task/12589/stat : 12589 (SArTagnan) R 12579 12581 12360 0 -1 4202560 92235 0 0 0 487495 62 0 0 25 0 8 0 877630807 8196038656 1543453 33554432000 4194304 4625164 140734578062048 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39189
Current children cumulated vsize (KiB) 8003944

[startup+4962.3 s]
/proc/loadavg: 8.00 8.01 8.00 9/178 12745
/proc/meminfo: memFree=26347276/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=8003944 CPUtime=39668.7 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 36606288 0 0 0 3960833 6034 0 0 25 0 8 0 877630798 8196038656 1554596 33554432000 4194304 4625164 140734578062048 18446744073709551615 4393074 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/12581/statm: 2000986 1554596 349 106 0 1996489 0
[pid=12581/tid=12583] ppid=12579 vsize=8003944 CPUtime=4962.13 cores=0-7
/proc/12581/task/12583/stat : 12583 (SArTagnan) R 12579 12581 12360 0 -1 4202560 131990 0 0 0 496186 27 0 0 25 0 8 0 877630807 8196038656 1554596 33554432000 4194304 4625164 140734578062048 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=12581/tid=12584] ppid=12579 vsize=8003944 CPUtime=4961.09 cores=0-7
/proc/12581/task/12584/stat : 12584 (SArTagnan) R 12579 12581 12360 0 -1 4202560 325524 0 0 0 495832 277 0 0 25 0 8 0 877630807 8196038656 1554596 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=12581/tid=12585] ppid=12579 vsize=8003944 CPUtime=4961.55 cores=0-7
/proc/12581/task/12585/stat : 12585 (SArTagnan) R 12579 12581 12360 0 -1 4202560 32378096 0 0 0 491108 5047 0 0 25 0 8 0 877630807 8196038656 1554596 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12581/tid=12586] ppid=12579 vsize=8003944 CPUtime=4961.65 cores=0-7
/proc/12581/task/12586/stat : 12586 (SArTagnan) R 12579 12581 12360 0 -1 4202560 169557 0 0 0 496133 32 0 0 25 0 8 0 877630807 8196038656 1554596 33554432000 4194304 4625164 140734578062048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=12581/tid=12587] ppid=12579 vsize=8003944 CPUtime=4962.32 cores=0-7
/proc/12581/task/12587/stat : 12587 (SArTagnan) R 12579 12581 12360 0 -1 4202560 130732 0 0 0 496197 35 0 0 25 0 8 0 877630807 8196038656 1554596 33554432000 4194304 4625164 140734578062048 18446744073709551615 4460412 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12581/tid=12588] ppid=12579 vsize=8003944 CPUtime=4962.32 cores=0-7
/proc/12581/task/12588/stat : 12588 (SArTagnan) R 12579 12581 12360 0 -1 4202560 170707 0 0 0 496191 41 0 0 25 0 8 0 877630807 8196038656 1554596 33554432000 4194304 4625164 140734578062048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12581/tid=12589] ppid=12579 vsize=8003944 CPUtime=4935.24 cores=0-7
/proc/12581/task/12589/stat : 12589 (SArTagnan) R 12579 12581 12360 0 -1 4202560 93455 0 0 0 493461 63 0 0 25 0 8 0 877630807 8196038656 1554596 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39668.7
Current children cumulated vsize (KiB) 8003944



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5003.8 s]
/proc/loadavg: 8.00 8.01 8.00 9/178 12746
/proc/meminfo: memFree=26310412/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=8003944 CPUtime=40000.5 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) R 12579 12581 12360 0 -1 4202496 36920572 0 0 0 3993948 6101 0 0 25 0 8 0 877630798 8196038656 1563342 33554432000 4194304 4625164 140734578062048 18446744073709551615 4511906 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/12581/statm: 2000986 1563342 349 106 0 1996489 0
[pid=12581/tid=12583] ppid=12579 vsize=8003944 CPUtime=5003.64 cores=0-7
/proc/12581/task/12583/stat : 12583 (SArTagnan) R 12579 12581 12360 0 -1 4202560 132251 0 0 0 500337 27 0 0 25 0 8 0 877630807 8196038656 1563342 33554432000 4194304 4625164 140734578062048 18446744073709551615 4488051 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=12581/tid=12584] ppid=12579 vsize=8003944 CPUtime=5002.6 cores=0-7
/proc/12581/task/12584/stat : 12584 (SArTagnan) R 12579 12581 12360 0 -1 4202560 331726 0 0 0 499973 287 0 0 25 0 8 0 877630807 8196038656 1563342 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=12581/tid=12585] ppid=12579 vsize=8003944 CPUtime=5003.05 cores=0-7
/proc/12581/task/12585/stat : 12585 (SArTagnan) R 12579 12581 12360 0 -1 4202560 32667795 0 0 0 495204 5101 0 0 25 0 8 0 877630807 8196038656 1563342 33554432000 4194304 4625164 140734578062048 18446744073709551615 4335407 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12581/tid=12586] ppid=12579 vsize=8003944 CPUtime=5003.16 cores=0-7
/proc/12581/task/12586/stat : 12586 (SArTagnan) R 12579 12581 12360 0 -1 4202560 169732 0 0 0 500284 32 0 0 25 0 8 0 877630807 8196038656 1563342 33554432000 4194304 4625164 140734578062048 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=12581/tid=12587] ppid=12579 vsize=8003944 CPUtime=5003.83 cores=0-7
/proc/12581/task/12587/stat : 12587 (SArTagnan) R 12579 12581 12360 0 -1 4202560 130950 0 0 0 500348 35 0 0 25 0 8 0 877630807 8196038656 1563342 33554432000 4194304 4625164 140734578062048 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=12581/tid=12588] ppid=12579 vsize=8003944 CPUtime=5003.83 cores=0-7
/proc/12581/task/12588/stat : 12588 (SArTagnan) R 12579 12581 12360 0 -1 4202560 170788 0 0 0 500342 41 0 0 25 0 8 0 877630807 8196038656 1563342 33554432000 4194304 4625164 140734578062048 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12581/tid=12589] ppid=12579 vsize=8003944 CPUtime=4976.52 cores=0-7
/proc/12581/task/12589/stat : 12589 (SArTagnan) R 12579 12581 12360 0 -1 4202560 93486 0 0 0 497589 63 0 0 25 0 8 0 877630807 8196038656 1563342 33554432000 4194304 4625164 140734578062048 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 8003944

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5004 s]
/proc/loadavg: 8.00 8.01 8.00 9/178 12746
/proc/meminfo: memFree=26310412/32951124 swapFree=67052924/67111528
[pid=12581] ppid=12579 vsize=0 CPUtime=40000.7 cores=0-7
/proc/12581/stat : 12581 (SArTagnan) Z 12579 12581 12360 0 -1 4203548 36920573 0 0 0 3993953 6120 0 0 25 0 2 0 877630798 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/12581/statm: 0 0 0 0 0 0 0
[pid=12581/tid=12589] ppid=12579 vsize=0 CPUtime=4976.71 cores=0-7
/proc/12581/task/12589/stat : 12589 (SArTagnan) R 12579 12581 12360 0 -1 4203588 93486 0 0 0 497589 82 0 0 25 0 2 0 877630807 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5004.06
CPU time (s): 40000.8
CPU user time (s): 39939.5
CPU system time (s): 61.2647
CPU usage (%): 799.367
Max. virtual memory (cumulated for all children) (KiB): 10166948

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39939.5
system time used= 61.2647
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 36920573
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 11294
involuntary context switches= 166091

runsolver used 7.05393 second user time and 20.2439 second system time

The end

Launcher Data 

Begin job on node104 at 2011-06-10 00:58:24
IDJOB=3438586
IDBENCH=71603
IDSOLVER=1936
FILE ID=node104/3438586-1307660303
RUNJOBID= node104-1307655297-12374
PBS_JOBID= 13549422
Free space on /tmp= 74180 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/forced/rbsat-v2640c305320gyes3.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3438586-1307660303/watcher-3438586-1307660303 -o /tmp/evaluation-result-3438586-1307660303/solver-3438586-1307660303 -C 40000 -W 5100 -M 31000  SArTagnan HOME/instance-3438586-1307660303.cnf -threads=8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 3b033385d50481e5bd2fe507f2941f4f
RANDOM SEED=1585455779

node104.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5293.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      32575264 kB
Buffers:         48656 kB
Cached:         169676 kB
SwapCached:      10312 kB
Active:          94144 kB
Inactive:       141272 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      32575264 kB
SwapTotal:    67111528 kB
SwapFree:     67052924 kB
Dirty:            4304 kB
Writeback:           0 kB
AnonPages:       15244 kB
Mapped:           8384 kB
Slab:            73516 kB
PageTables:       4252 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   189144 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74180 MiB
End job on node104 at 2011-06-10 02:21:49