Trace number 3322780

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 35803.5 5100.99

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m47-q841.5-n45-p15-s16393788.cnf
MD5SUM683c9f75b69575e6451b8563d12f176b
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.196969
Satisfiable
(Un)Satisfiability was proved
Number of variables1981
Number of clauses333838
Sum of the clauses size669567
Maximum clause length47
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2333793
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...395309/instance-3322780-1304395309.cnf
1.89/1.96	c Solving...
35801.99/5100.09	
35801.99/5100.09	*** INTERRUPTED! ***
35801.99/5100.09	*** Shutting down threads...
35803.39/5100.98	s UNKNOWN
35803.39/5100.98	
35803.39/5100.98	c INTERRUPTED   : 1
35803.39/5100.98	c Models        : 0+    
35803.39/5100.98	c Time          : 5100.107s (Solving: 5098.14s 1st Model: 0.00s Unsat: 0.00s)
35803.39/5100.98	c CPU Time      : 35802.600s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3322780-1304395309/watcher-3322780-1304395309 -o /tmp/evaluation-result-3322780-1304395309/solver-3322780-1304395309 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322780-1304395309.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 9.10 9.02 9.09 3/281 22197
/proc/meminfo: memFree=23368828/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=23548 CPUtime=0.01 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 1360 0 0 0 1 0 0 0 25 0 1 0 551129056 24113152 1104 33554432000 4194304 4964704 140734522692080 18446744073709551615 232335553221 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/22197/statm: 5887 1105 386 189 0 761 0

[startup+0.084952 s]
/proc/loadavg: 9.10 9.02 9.09 3/281 22197
/proc/meminfo: memFree=23368828/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=37280 CPUtime=0.07 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 6172 0 0 0 7 0 0 0 25 0 1 0 551129056 38174720 4311 33554432000 4194304 4964704 140734522692080 18446744073709551615 232335553221 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/22197/statm: 9320 4312 397 189 0 4194 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 37280

[startup+0.100949 s]
/proc/loadavg: 9.10 9.02 9.09 3/281 22197
/proc/meminfo: memFree=23368828/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=41020 CPUtime=0.09 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 7029 0 0 0 9 0 0 0 25 0 1 0 551129056 42004480 5168 33554432000 4194304 4964704 140734522692080 18446744073709551615 232335553221 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/22197/statm: 10255 5171 397 189 0 5129 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 41020

[startup+0.300917 s]
/proc/loadavg: 9.10 9.02 9.09 3/281 22197
/proc/meminfo: memFree=23368828/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=43532 CPUtime=0.29 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 7578 0 0 0 29 0 0 0 25 0 1 0 551129056 44576768 5717 33554432000 4194304 4964704 140734522692080 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/22197/statm: 10883 5717 399 189 0 5757 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 43532

[startup+0.700842 s]
/proc/loadavg: 9.10 9.02 9.09 3/281 22197
/proc/meminfo: memFree=23368828/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=43532 CPUtime=0.69 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 7578 0 0 0 69 0 0 0 25 0 1 0 551129056 44576768 5717 33554432000 4194304 4964704 140734522692080 18446744073709551615 4607575 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/22197/statm: 10883 5717 399 189 0 5757 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 43532

[startup+1.5007 s]
/proc/loadavg: 9.10 9.02 9.09 3/282 22198
/proc/meminfo: memFree=23348484/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=43532 CPUtime=1.49 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 7579 0 0 0 149 0 0 0 25 0 1 0 551129056 44576768 5718 33554432000 4194304 4964704 140734522692080 18446744073709551615 4600353 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/22197/statm: 10883 5718 400 189 0 5757 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 43532

[startup+3.10037 s]
/proc/loadavg: 9.10 9.02 9.09 3/289 22205
/proc/meminfo: memFree=23346492/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=401044 CPUtime=9 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 20852 0 0 0 892 8 0 0 19 0 8 0 551129056 410669056 18306 33554432000 4194304 4964704 140734522692080 18446744073709551615 4321002 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/22197/statm: 100261 18306 464 189 0 94610 0
[pid=22197/tid=22199] ppid=22195 vsize=401044 CPUtime=0.5 cores=0-7
/proc/22197/task/22199/stat : 22199 (claspmt) R 22195 22197 22123 0 -1 4202560 1248 0 0 0 50 0 0 0 18 0 8 0 551129253 410669056 18306 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=22197/tid=22200] ppid=22195 vsize=401044 CPUtime=0.81 cores=0-7
/proc/22197/task/22200/stat : 22200 (claspmt) R 22195 22197 22123 0 -1 4202560 1320 0 0 0 80 1 0 0 19 0 8 0 551129253 410669056 18306 33554432000 4194304 4964704 140734522692080 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=22197/tid=22201] ppid=22195 vsize=401044 CPUtime=0.63 cores=0-7
/proc/22197/task/22201/stat : 22201 (claspmt) R 22195 22197 22123 0 -1 4202560 1281 0 0 0 63 0 0 0 18 0 8 0 551129253 410669056 18306 33554432000 4194304 4964704 140734522692080 18446744073709551615 232335553569 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=22197/tid=22202] ppid=22195 vsize=401044 CPUtime=0.98 cores=0-7
/proc/22197/task/22202/stat : 22202 (claspmt) R 22195 22197 22123 0 -1 4202560 2346 0 0 0 97 1 0 0 17 0 8 0 551129253 410669056 18306 33554432000 4194304 4964704 140734522692080 18446744073709551615 4362375 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=22197/tid=22203] ppid=22195 vsize=401044 CPUtime=1.11 cores=0-7
/proc/22197/task/22203/stat : 22203 (claspmt) R 22195 22197 22123 0 -1 4202560 541 0 0 0 111 0 0 0 19 0 8 0 551129253 410669056 18307 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=22197/tid=22204] ppid=22195 vsize=401044 CPUtime=0.97 cores=0-7
/proc/22197/task/22204/stat : 22204 (claspmt) R 22195 22197 22123 0 -1 4202560 2521 0 0 0 96 1 0 0 17 0 8 0 551129253 410669056 18307 33554432000 4194304 4964704 140734522692080 18446744073709551615 4332507 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=22197/tid=22205] ppid=22195 vsize=401044 CPUtime=0.97 cores=0-7
/proc/22197/task/22205/stat : 22205 (claspmt) R 22195 22197 22123 0 -1 4202560 1981 0 0 0 97 0 0 0 17 0 8 0 551129253 410669056 18307 33554432000 4194304 4964704 140734522692080 18446744073709551615 232335553569 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 9
Current children cumulated vsize (KiB) 401044

[startup+6.30081 s]
/proc/loadavg: 9.09 9.02 9.09 10/289 22205
/proc/meminfo: memFree=23219556/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=544032 CPUtime=31.26 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 47973 0 0 0 3106 20 0 0 25 0 8 0 551129056 557088768 45427 33554432000 4194304 4964704 140734522692080 18446744073709551615 232335997422 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/22197/statm: 136008 45428 464 189 0 130357 0
[pid=22197/tid=22199] ppid=22195 vsize=544032 CPUtime=3.23 cores=0-7
/proc/22197/task/22199/stat : 22199 (claspmt) R 22195 22197 22123 0 -1 4202560 3985 0 0 0 321 2 0 0 25 0 8 0 551129253 557088768 45428 33554432000 4194304 4964704 140734522692080 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=22197/tid=22200] ppid=22195 vsize=544032 CPUtime=3.97 cores=0-7
/proc/22197/task/22200/stat : 22200 (claspmt) R 22195 22197 22123 0 -1 4202560 3793 0 0 0 395 2 0 0 25 0 8 0 551129253 557088768 45428 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=22197/tid=22201] ppid=22195 vsize=544032 CPUtime=3.28 cores=0-7
/proc/22197/task/22201/stat : 22201 (claspmt) R 22195 22197 22123 0 -1 4202560 4605 0 0 0 326 2 0 0 25 0 8 0 551129253 557088768 45428 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=22197/tid=22202] ppid=22195 vsize=544032 CPUtime=4.16 cores=0-7
/proc/22197/task/22202/stat : 22202 (claspmt) R 22195 22197 22123 0 -1 4202560 6776 0 0 0 413 3 0 0 25 0 8 0 551129253 557088768 45428 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=22197/tid=22203] ppid=22195 vsize=544032 CPUtime=3.15 cores=0-7
/proc/22197/task/22203/stat : 22203 (claspmt) R 22195 22197 22123 0 -1 4202560 2900 0 0 0 314 1 0 0 25 0 8 0 551129253 557088768 45428 33554432000 4194304 4964704 140734522692080 18446744073709551615 232335553569 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=22197/tid=22204] ppid=22195 vsize=544032 CPUtime=4.04 cores=0-7
/proc/22197/task/22204/stat : 22204 (claspmt) R 22195 22197 22123 0 -1 4202560 6442 0 0 0 402 2 0 0 24 0 8 0 551129253 557088768 45428 33554432000 4194304 4964704 140734522692080 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=22197/tid=22205] ppid=22195 vsize=544032 CPUtime=4.16 cores=0-7
/proc/22197/task/22205/stat : 22205 (claspmt) R 22195 22197 22123 0 -1 4202560 6208 0 0 0 413 3 0 0 25 0 8 0 551129253 557088768 45428 33554432000 4194304 4964704 140734522692080 18446744073709551615 4660077 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 31.26
Current children cumulated vsize (KiB) 544032

[startup+12.7007 s]
/proc/loadavg: 9.16 9.04 9.10 10/289 22205
/proc/meminfo: memFree=23058348/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=564784 CPUtime=75.85 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 83591 0 0 0 7547 38 0 0 25 0 8 0 551129056 578338816 81045 33554432000 4194304 4964704 140734522692080 18446744073709551615 4362287 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/22197/statm: 141196 81045 466 189 0 135545 0
[pid=22197/tid=22199] ppid=22195 vsize=564784 CPUtime=9.07 cores=0-7
/proc/22197/task/22199/stat : 22199 (claspmt) R 22195 22197 22123 0 -1 4202560 9686 0 0 0 903 4 0 0 25 0 8 0 551129253 578338816 81045 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=22197/tid=22200] ppid=22195 vsize=564784 CPUtime=10.2 cores=0-7
/proc/22197/task/22200/stat : 22200 (claspmt) R 22195 22197 22123 0 -1 4202560 5781 0 0 0 1017 3 0 0 19 0 8 0 551129253 578338816 81045 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=22197/tid=22201] ppid=22195 vsize=564784 CPUtime=9.49 cores=0-7
/proc/22197/task/22201/stat : 22201 (claspmt) R 22195 22197 22123 0 -1 4202560 10261 0 0 0 944 5 0 0 25 0 8 0 551129253 578338816 81045 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=22197/tid=22202] ppid=22195 vsize=564784 CPUtime=10.12 cores=0-7
/proc/22197/task/22202/stat : 22202 (claspmt) R 22195 22197 22123 0 -1 4202560 11215 0 0 0 1007 5 0 0 20 0 8 0 551129253 578338816 81045 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=22197/tid=22203] ppid=22195 vsize=564784 CPUtime=5.6 cores=0-7
/proc/22197/task/22203/stat : 22203 (claspmt) R 22195 22197 22123 0 -1 4202560 5309 0 0 0 559 1 0 0 25 0 8 0 551129253 578338816 81045 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320963 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=22197/tid=22204] ppid=22195 vsize=564784 CPUtime=9.83 cores=0-7
/proc/22197/task/22204/stat : 22204 (claspmt) R 22195 22197 22123 0 -1 4202560 10011 0 0 0 979 4 0 0 21 0 8 0 551129253 578338816 81047 33554432000 4194304 4964704 140734522692080 18446744073709551615 232335553569 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=22197/tid=22205] ppid=22195 vsize=564784 CPUtime=10.37 cores=0-7
/proc/22197/task/22205/stat : 22205 (claspmt) R 22195 22197 22123 0 -1 4202560 13312 0 0 0 1030 7 0 0 23 0 8 0 551129253 578338816 81047 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 75.85
Current children cumulated vsize (KiB) 564784

[startup+25.5004 s]
/proc/loadavg: 9.20 9.05 9.10 10/289 22206
/proc/meminfo: memFree=22839496/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=797740 CPUtime=164.6 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 135843 0 0 0 16391 69 0 0 25 0 8 0 551129056 816885760 133297 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/22197/statm: 199435 133297 466 189 0 193784 0
[pid=22197/tid=22199] ppid=22195 vsize=797740 CPUtime=17.89 cores=0-7
/proc/22197/task/22199/stat : 22199 (claspmt) R 22195 22197 22123 0 -1 4202560 18702 0 0 0 1779 10 0 0 25 0 8 0 551129253 816885760 133297 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=22197/tid=22200] ppid=22195 vsize=797740 CPUtime=22.21 cores=0-7
/proc/22197/task/22200/stat : 22200 (claspmt) R 22195 22197 22123 0 -1 4202560 7283 0 0 0 2217 4 0 0 25 0 8 0 551129253 816885760 133297 33554432000 4194304 4964704 140734522692080 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=22197/tid=22201] ppid=22195 vsize=797740 CPUtime=19.19 cores=0-7
/proc/22197/task/22201/stat : 22201 (claspmt) R 22195 22197 22123 0 -1 4202560 13733 0 0 0 1912 7 0 0 25 0 8 0 551129253 816885760 133297 33554432000 4194304 4964704 140734522692080 18446744073709551615 4465565 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=22197/tid=22202] ppid=22195 vsize=797740 CPUtime=22.53 cores=0-7
/proc/22197/task/22202/stat : 22202 (claspmt) R 22195 22197 22123 0 -1 4202560 19058 0 0 0 2245 8 0 0 25 0 8 0 551129253 816885760 133297 33554432000 4194304 4964704 140734522692080 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=22197/tid=22203] ppid=22195 vsize=797740 CPUtime=17.36 cores=0-7
/proc/22197/task/22203/stat : 22203 (claspmt) R 22195 22197 22123 0 -1 4202560 13120 0 0 0 1730 6 0 0 25 0 8 0 551129253 816885760 133297 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670184 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=22197/tid=22204] ppid=22195 vsize=797740 CPUtime=20.69 cores=0-7
/proc/22197/task/22204/stat : 22204 (claspmt) R 22195 22197 22123 0 -1 4202560 12198 0 0 0 2064 5 0 0 25 0 8 0 551129253 816885760 133297 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=22197/tid=22205] ppid=22195 vsize=797740 CPUtime=22.87 cores=0-7
/proc/22197/task/22205/stat : 22205 (claspmt) R 22195 22197 22123 0 -1 4202560 26900 0 0 0 2271 16 0 0 21 0 8 0 551129253 816885760 133297 33554432000 4194304 4964704 140734522692080 18446744073709551615 4653154 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 164.6
Current children cumulated vsize (KiB) 797740

[startup+51.1009 s]
/proc/loadavg: 9.20 9.06 9.10 10/289 22206
/proc/meminfo: memFree=22665972/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=1066048 CPUtime=343.3 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 178932 0 0 0 34230 100 0 0 25 0 8 0 551129056 1091633152 176381 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/22197/statm: 266512 176381 466 189 0 260861 0
[pid=22197/tid=22199] ppid=22195 vsize=1066048 CPUtime=36.53 cores=0-7
/proc/22197/task/22199/stat : 22199 (claspmt) R 22195 22197 22123 0 -1 4202560 24790 0 0 0 3638 15 0 0 25 0 8 0 551129253 1091633152 176381 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=22197/tid=22200] ppid=22195 vsize=1066048 CPUtime=47.73 cores=0-7
/proc/22197/task/22200/stat : 22200 (claspmt) R 22195 22197 22123 0 -1 4202560 9378 0 0 0 4768 5 0 0 25 0 8 0 551129253 1091633152 176381 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=22197/tid=22201] ppid=22195 vsize=1066048 CPUtime=38.41 cores=0-7
/proc/22197/task/22201/stat : 22201 (claspmt) R 22195 22197 22123 0 -1 4202560 17471 0 0 0 3831 10 0 0 25 0 8 0 551129253 1091633152 176381 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=22197/tid=22202] ppid=22195 vsize=1066048 CPUtime=46.09 cores=0-7
/proc/22197/task/22202/stat : 22202 (claspmt) R 22195 22197 22123 0 -1 4202560 23813 0 0 0 4598 11 0 0 25 0 8 0 551129253 1091633152 176381 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=22197/tid=22203] ppid=22195 vsize=1066048 CPUtime=41.67 cores=0-7
/proc/22197/task/22203/stat : 22203 (claspmt) R 22195 22197 22123 0 -1 4202560 18046 0 0 0 4159 8 0 0 25 0 8 0 551129253 1091633152 176381 33554432000 4194304 4964704 140734522692080 18446744073709551615 4668931 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=22197/tid=22204] ppid=22195 vsize=1066048 CPUtime=44.59 cores=0-7
/proc/22197/task/22204/stat : 22204 (claspmt) R 22195 22197 22123 0 -1 4202560 16603 0 0 0 4451 8 0 0 25 0 8 0 551129253 1091633152 176381 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=22197/tid=22205] ppid=22195 vsize=1066048 CPUtime=46.2 cores=0-7
/proc/22197/task/22205/stat : 22205 (claspmt) R 22195 22197 22123 0 -1 4202560 33831 0 0 0 4598 22 0 0 25 0 8 0 551129253 1091633152 176381 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670277 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 343.3
Current children cumulated vsize (KiB) 1066048

[startup+102.311 s]
/proc/loadavg: 9.14 9.06 9.09 10/289 22208
/proc/meminfo: memFree=22378040/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=1240844 CPUtime=700.93 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 250763 0 0 0 69943 150 0 0 25 0 8 0 551129056 1270624256 248167 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670288 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/22197/statm: 310211 248167 466 189 0 304560 0
[pid=22197/tid=22199] ppid=22195 vsize=1240844 CPUtime=74.39 cores=0-7
/proc/22197/task/22199/stat : 22199 (claspmt) R 22195 22197 22123 0 -1 4202560 43745 0 0 0 7409 30 0 0 25 0 8 0 551129253 1270624256 248167 33554432000 4194304 4964704 140734522692080 18446744073709551615 4321082 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=22197/tid=22200] ppid=22195 vsize=1240844 CPUtime=98.93 cores=0-7
/proc/22197/task/22200/stat : 22200 (claspmt) R 22195 22197 22123 0 -1 4202560 13557 0 0 0 9887 6 0 0 25 0 8 0 551129253 1270624256 248167 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=22197/tid=22201] ppid=22195 vsize=1240844 CPUtime=86.32 cores=0-7
/proc/22197/task/22201/stat : 22201 (claspmt) R 22195 22197 22123 0 -1 4202560 21695 0 0 0 8618 14 0 0 25 0 8 0 551129253 1270624256 248167 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0

################
# More data... #
################

[pid=22197] ppid=22195 vsize=5479828 CPUtime=34000.6 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 1305124 0 0 0 3398684 1379 0 0 25 0 8 0 551129056 5611343872 1288011 33554432000 4194304 4964704 140734522692080 18446744073709551615 4653336 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/22197/statm: 1369957 1288011 466 189 0 1364306 0
[pid=22197/tid=22199] ppid=22195 vsize=5479828 CPUtime=4394.21 cores=0-7
/proc/22197/task/22199/stat : 22199 (claspmt) R 22195 22197 22123 0 -1 4202560 143337 0 0 0 439278 143 0 0 25 0 8 0 551129253 5611343872 1288011 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=22197/tid=22200] ppid=22195 vsize=5479828 CPUtime=4754.3 cores=0-7
/proc/22197/task/22200/stat : 22200 (claspmt) R 22195 22197 22123 0 -1 4202560 81900 0 0 0 475393 37 0 0 25 0 8 0 551129253 5611343872 1288011 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=22197/tid=22201] ppid=22195 vsize=5479828 CPUtime=4247.31 cores=0-7
/proc/22197/task/22201/stat : 22201 (claspmt) R 22195 22197 22123 0 -1 4202560 159974 0 0 0 424526 205 0 0 25 0 8 0 551129253 5611343872 1288011 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=22197/tid=22202] ppid=22195 vsize=5479828 CPUtime=4398.19 cores=0-7
/proc/22197/task/22202/stat : 22202 (claspmt) R 22195 22197 22123 0 -1 4202560 91278 0 0 0 439735 84 0 0 25 0 8 0 551129253 5611343872 1288011 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=22197/tid=22203] ppid=22195 vsize=5479828 CPUtime=4409.12 cores=0-7
/proc/22197/task/22203/stat : 22203 (claspmt) R 22195 22197 22123 0 -1 4202560 116791 0 0 0 440809 103 0 0 25 0 8 0 551129253 5611343872 1288011 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=22197/tid=22204] ppid=22195 vsize=5479828 CPUtime=4263.67 cores=0-7
/proc/22197/task/22204/stat : 22204 (claspmt) R 22195 22197 22123 0 -1 4202560 156436 0 0 0 426170 197 0 0 25 0 8 0 551129253 5611343872 1288011 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=22197/tid=22205] ppid=22195 vsize=5479828 CPUtime=3852.02 cores=0-7
/proc/22197/task/22205/stat : 22205 (claspmt) R 22195 22197 22123 0 -1 4202560 256521 0 0 0 384901 301 0 0 25 0 8 0 551129253 5611343872 1288011 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 34000.6
Current children cumulated vsize (KiB) 5479828

[startup+4902.3 s]
/proc/loadavg: 9.10 9.12 9.11 10/289 22351
/proc/meminfo: memFree=18208084/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=5479828 CPUtime=34420.1 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 1305233 0 0 0 3440630 1379 0 0 25 0 8 0 551129056 5611343872 1288120 33554432000 4194304 4964704 140734522692080 18446744073709551615 4321076 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/22197/statm: 1369957 1288120 466 189 0 1364306 0
[pid=22197/tid=22199] ppid=22195 vsize=5479828 CPUtime=4454.21 cores=0-7
/proc/22197/task/22199/stat : 22199 (claspmt) R 22195 22197 22123 0 -1 4202560 143347 0 0 0 445278 143 0 0 25 0 8 0 551129253 5611343872 1288120 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=22197/tid=22200] ppid=22195 vsize=5479828 CPUtime=4814.3 cores=0-7
/proc/22197/task/22200/stat : 22200 (claspmt) R 22195 22197 22123 0 -1 4202560 81910 0 0 0 481393 37 0 0 25 0 8 0 551129253 5611343872 1288120 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=22197/tid=22201] ppid=22195 vsize=5479828 CPUtime=4307.3 cores=0-7
/proc/22197/task/22201/stat : 22201 (claspmt) R 22195 22197 22123 0 -1 4202560 160014 0 0 0 430525 205 0 0 25 0 8 0 551129253 5611343872 1288120 33554432000 4194304 4964704 140734522692080 18446744073709551615 232335589998 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=22197/tid=22202] ppid=22195 vsize=5479828 CPUtime=4457.65 cores=0-7
/proc/22197/task/22202/stat : 22202 (claspmt) R 22195 22197 22123 0 -1 4202560 91305 0 0 0 445681 84 0 0 25 0 8 0 551129253 5611343872 1288120 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=22197/tid=22203] ppid=22195 vsize=5479828 CPUtime=4461.39 cores=0-7
/proc/22197/task/22203/stat : 22203 (claspmt) R 22195 22197 22123 0 -1 4202560 116803 0 0 0 446036 103 0 0 25 0 8 0 551129253 5611343872 1288120 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=22197/tid=22204] ppid=22195 vsize=5479828 CPUtime=4323.68 cores=0-7
/proc/22197/task/22204/stat : 22204 (claspmt) R 22195 22197 22123 0 -1 4202560 156440 0 0 0 432171 197 0 0 25 0 8 0 551129253 5611343872 1288120 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=22197/tid=22205] ppid=22195 vsize=5479828 CPUtime=3881.65 cores=0-7
/proc/22197/task/22205/stat : 22205 (claspmt) R 22195 22197 22123 0 -1 4202560 256521 0 0 0 387864 301 0 0 25 0 8 0 551129253 5611343872 1288120 33554432000 4194304 4964704 140734522692080 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 34420.1
Current children cumulated vsize (KiB) 5479828

[startup+4962.3 s]
/proc/loadavg: 9.07 9.11 9.10 10/289 22352
/proc/meminfo: memFree=18207808/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=5479828 CPUtime=34839.5 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 1305411 0 0 0 3482574 1380 0 0 25 0 8 0 551129056 5611343872 1288298 33554432000 4194304 4964704 140734522692080 18446744073709551615 4342157 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/22197/statm: 1369957 1288298 466 189 0 1364306 0
[pid=22197/tid=22199] ppid=22195 vsize=5479828 CPUtime=4514.2 cores=0-7
/proc/22197/task/22199/stat : 22199 (claspmt) R 22195 22197 22123 0 -1 4202560 143354 0 0 0 451277 143 0 0 25 0 8 0 551129253 5611343872 1288298 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=22197/tid=22200] ppid=22195 vsize=5479828 CPUtime=4874.3 cores=0-7
/proc/22197/task/22200/stat : 22200 (claspmt) R 22195 22197 22123 0 -1 4202560 81925 0 0 0 487393 37 0 0 25 0 8 0 551129253 5611343872 1288298 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=22197/tid=22201] ppid=22195 vsize=5479828 CPUtime=4367.3 cores=0-7
/proc/22197/task/22201/stat : 22201 (claspmt) R 22195 22197 22123 0 -1 4202560 160014 0 0 0 436524 206 0 0 25 0 8 0 551129253 5611343872 1288298 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=22197/tid=22202] ppid=22195 vsize=5479828 CPUtime=4517.12 cores=0-7
/proc/22197/task/22202/stat : 22202 (claspmt) R 22195 22197 22123 0 -1 4202560 91336 0 0 0 451627 85 0 0 25 0 8 0 551129253 5611343872 1288298 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670193 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=22197/tid=22203] ppid=22195 vsize=5479828 CPUtime=4517.89 cores=0-7
/proc/22197/task/22203/stat : 22203 (claspmt) R 22195 22197 22123 0 -1 4202560 116806 0 0 0 451686 103 0 0 25 0 8 0 551129253 5611343872 1288298 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=22197/tid=22204] ppid=22195 vsize=5479828 CPUtime=4383.68 cores=0-7
/proc/22197/task/22204/stat : 22204 (claspmt) R 22195 22197 22123 0 -1 4202560 156555 0 0 0 438171 197 0 0 25 0 8 0 551129253 5611343872 1288298 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=22197/tid=22205] ppid=22195 vsize=5479828 CPUtime=3911.84 cores=0-7
/proc/22197/task/22205/stat : 22205 (claspmt) R 22195 22197 22123 0 -1 4202560 256527 0 0 0 390883 301 0 0 25 0 8 0 551129253 5611343872 1288298 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 34839.5
Current children cumulated vsize (KiB) 5479828

[startup+5022.3 s]
/proc/loadavg: 9.09 9.10 9.10 10/289 22354
/proc/meminfo: memFree=18196992/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=5479828 CPUtime=35258.9 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 1308040 0 0 0 3524509 1385 0 0 25 0 8 0 551129056 5611343872 1290927 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/22197/statm: 1369957 1290927 466 189 0 1364306 0
[pid=22197/tid=22199] ppid=22195 vsize=5479828 CPUtime=4574.2 cores=0-7
/proc/22197/task/22199/stat : 22199 (claspmt) R 22195 22197 22123 0 -1 4202560 143747 0 0 0 457276 144 0 0 25 0 8 0 551129253 5611343872 1290927 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=22197/tid=22200] ppid=22195 vsize=5479828 CPUtime=4934.3 cores=0-7
/proc/22197/task/22200/stat : 22200 (claspmt) R 22195 22197 22123 0 -1 4202560 81948 0 0 0 493393 37 0 0 25 0 8 0 551129253 5611343872 1290927 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=22197/tid=22201] ppid=22195 vsize=5479828 CPUtime=4427.3 cores=0-7
/proc/22197/task/22201/stat : 22201 (claspmt) R 22195 22197 22123 0 -1 4202560 160016 0 0 0 442524 206 0 0 25 0 8 0 551129253 5611343872 1290927 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=22197/tid=22202] ppid=22195 vsize=5479828 CPUtime=4576.53 cores=0-7
/proc/22197/task/22202/stat : 22202 (claspmt) R 22195 22197 22123 0 -1 4202560 91356 0 0 0 457568 85 0 0 25 0 8 0 551129253 5611343872 1290927 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=22197/tid=22203] ppid=22195 vsize=5479828 CPUtime=4551.57 cores=0-7
/proc/22197/task/22203/stat : 22203 (claspmt) R 22195 22197 22123 0 -1 4202560 116806 0 0 0 455054 103 0 0 25 0 8 0 551129253 5611343872 1290927 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=22197/tid=22204] ppid=22195 vsize=5479828 CPUtime=4416.42 cores=0-7
/proc/22197/task/22204/stat : 22204 (claspmt) R 22195 22197 22123 0 -1 4202560 156559 0 0 0 441445 197 0 0 25 0 8 0 551129253 5611343872 1290927 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=22197/tid=22205] ppid=22195 vsize=5479828 CPUtime=3969.53 cores=0-7
/proc/22197/task/22205/stat : 22205 (claspmt) R 22195 22197 22123 0 -1 4202560 256536 0 0 0 396651 302 0 0 25 0 8 0 551129253 5611343872 1290927 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320916 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 35258.9
Current children cumulated vsize (KiB) 5479828

[startup+5082.3 s]
/proc/loadavg: 9.12 9.11 9.10 10/289 22356
/proc/meminfo: memFree=18193252/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=5479828 CPUtime=35678.4 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 1308925 0 0 0 3566452 1386 0 0 25 0 8 0 551129056 5611343872 1291812 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320966 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/22197/statm: 1369957 1291812 466 189 0 1364306 0
[pid=22197/tid=22199] ppid=22195 vsize=5479828 CPUtime=4634.17 cores=0-7
/proc/22197/task/22199/stat : 22199 (claspmt) R 22195 22197 22123 0 -1 4202560 144573 0 0 0 463273 144 0 0 25 0 8 0 551129253 5611343872 1291812 33554432000 4194304 4964704 140734522692080 18446744073709551615 232335590088 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=22197/tid=22200] ppid=22195 vsize=5479828 CPUtime=4994.31 cores=0-7
/proc/22197/task/22200/stat : 22200 (claspmt) R 22195 22197 22123 0 -1 4202560 81965 0 0 0 499394 37 0 0 25 0 8 0 551129253 5611343872 1291812 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=22197/tid=22201] ppid=22195 vsize=5479828 CPUtime=4487.3 cores=0-7
/proc/22197/task/22201/stat : 22201 (claspmt) R 22195 22197 22123 0 -1 4202560 160020 0 0 0 448524 206 0 0 25 0 8 0 551129253 5611343872 1291812 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=22197/tid=22202] ppid=22195 vsize=5479828 CPUtime=4635.98 cores=0-7
/proc/22197/task/22202/stat : 22202 (claspmt) R 22195 22197 22123 0 -1 4202560 91365 0 0 0 463513 85 0 0 25 0 8 0 551129253 5611343872 1291812 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=22197/tid=22203] ppid=22195 vsize=5479828 CPUtime=4611.57 cores=0-7
/proc/22197/task/22203/stat : 22203 (claspmt) R 22195 22197 22123 0 -1 4202560 116808 0 0 0 461054 103 0 0 25 0 8 0 551129253 5611343872 1291812 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=22197/tid=22204] ppid=22195 vsize=5479828 CPUtime=4463.42 cores=0-7
/proc/22197/task/22204/stat : 22204 (claspmt) R 22195 22197 22123 0 -1 4202560 156586 0 0 0 446145 197 0 0 25 0 8 0 551129253 5611343872 1291812 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=22197/tid=22205] ppid=22195 vsize=5479828 CPUtime=4012.53 cores=0-7
/proc/22197/task/22205/stat : 22205 (claspmt) R 22195 22197 22123 0 -1 4202560 256536 0 0 0 400951 302 0 0 25 0 8 0 551129253 5611343872 1291812 33554432000 4194304 4964704 140734522692080 18446744073709551615 232335552648 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 35678.4
Current children cumulated vsize (KiB) 5479828



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 9.09 9.10 9.09 10/289 22356
/proc/meminfo: memFree=18193612/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=5479828 CPUtime=35802 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 1309195 0 0 0 3578812 1387 0 0 25 0 8 0 551129056 5611343872 1291543 33554432000 4194304 4964704 140734522692080 18446744073709551615 4627204 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/22197/statm: 1369957 1291543 466 189 0 1364306 0
[pid=22197/tid=22199] ppid=22195 vsize=5479828 CPUtime=4651.7 cores=0-7
/proc/22197/task/22199/stat : 22199 (claspmt) R 22195 22197 22123 0 -1 4202560 144573 0 0 0 465025 145 0 0 25 0 8 0 551129253 5611343872 1291543 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=22197/tid=22200] ppid=22195 vsize=5479828 CPUtime=5012.02 cores=0-7
/proc/22197/task/22200/stat : 22200 (claspmt) R 22195 22197 22123 0 -1 4202560 82219 0 0 0 501164 38 0 0 25 0 8 0 551129253 5611343872 1291543 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670416 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=22197/tid=22201] ppid=22195 vsize=5479828 CPUtime=4505.01 cores=0-7
/proc/22197/task/22201/stat : 22201 (claspmt) R 22195 22197 22123 0 -1 4202560 160020 0 0 0 450295 206 0 0 25 0 8 0 551129253 5611343872 1291543 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=22197/tid=22202] ppid=22195 vsize=5479828 CPUtime=4653.52 cores=0-7
/proc/22197/task/22202/stat : 22202 (claspmt) R 22195 22197 22123 0 -1 4202560 91375 0 0 0 465267 85 0 0 25 0 8 0 551129253 5611343872 1291543 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=22197/tid=22203] ppid=22195 vsize=5479828 CPUtime=4629.28 cores=0-7
/proc/22197/task/22203/stat : 22203 (claspmt) R 22195 22197 22123 0 -1 4202560 116808 0 0 0 462825 103 0 0 25 0 8 0 551129253 5611343872 1291543 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=22197/tid=22204] ppid=22195 vsize=5479828 CPUtime=4481.13 cores=0-7
/proc/22197/task/22204/stat : 22204 (claspmt) R 22195 22197 22123 0 -1 4202560 156591 0 0 0 447916 197 0 0 25 0 8 0 551129253 5611343872 1291543 33554432000 4194304 4964704 140734522692080 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=22197/tid=22205] ppid=22195 vsize=5479828 CPUtime=4021.45 cores=0-7
/proc/22197/task/22205/stat : 22205 (claspmt) R 22195 22197 22123 0 -1 4202560 256536 0 0 0 401843 302 0 0 25 0 8 0 551129253 5611343872 1291543 33554432000 4194304 4964704 140734522692080 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 35802
Current children cumulated vsize (KiB) 5479828

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5100.6 s]
/proc/loadavg: 9.09 9.10 9.09 3/283 22357
/proc/meminfo: memFree=18204200/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=5467276 CPUtime=35803.1 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 1309205 0 0 0 3578923 1387 0 0 25 0 1 0 551129056 5598490624 1288629 33554432000 4194304 4964704 140734522692080 18446744073709551615 232335549428 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/22197/statm: 1366819 1288629 476 189 0 1361168 0
Current children cumulated CPU time (s) 35803.1
Current children cumulated vsize (KiB) 5467276

[startup+5100.8 s]
/proc/loadavg: 9.09 9.10 9.09 3/283 22357
/proc/meminfo: memFree=18204200/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=4729972 CPUtime=35803.3 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 1309205 0 0 0 3578939 1390 0 0 25 0 1 0 551129056 4843491328 1099736 33554432000 4194304 4964704 140734522692080 18446744073709551615 232335935847 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/22197/statm: 1182493 1099736 476 189 0 1176842 0
Current children cumulated CPU time (s) 35803.3
Current children cumulated vsize (KiB) 4729972

[startup+5100.9 s]
/proc/loadavg: 9.09 9.10 9.09 3/283 22357
/proc/meminfo: memFree=18204200/32951124 swapFree=66863592/67111528
[pid=22197] ppid=22195 vsize=2310244 CPUtime=35803.4 cores=0-7
/proc/22197/stat : 22197 (claspmt) R 22195 22197 22123 0 -1 4202496 1309205 0 0 0 3578939 1400 0 0 25 0 1 0 551129056 2365689856 495295 33554432000 4194304 4964704 140734522692080 18446744073709551615 232335935623 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/22197/statm: 577561 494783 476 189 0 571910 0
Current children cumulated CPU time (s) 35803.4
Current children cumulated vsize (KiB) 2310244

Child status: 0
Real time (s): 5100.99
CPU time (s): 35803.5
CPU user time (s): 35789.4
CPU system time (s): 14.0969
CPU usage (%): 701.893
Max. virtual memory (cumulated for all children) (KiB): 5479828

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 35789.4
system time used= 14.0969
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1309206
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 74373
involuntary context switches= 447437

runsolver used 12.5591 second user time and 33.8279 second system time

The end

Launcher Data 

Begin job on node132 at 2011-05-03 06:01:49
IDJOB=3322780
IDBENCH=88221
IDSOLVER=1711
FILE ID=node132/3322780-1304395309
RUNJOBID= node132-1304395124-22137
PBS_JOBID= 13181864
Free space on /tmp= 73444 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m47-q841.5-n45-p15-s16393788.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322780-1304395309/watcher-3322780-1304395309 -o /tmp/evaluation-result-3322780-1304395309/solver-3322780-1304395309 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322780-1304395309.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 683c9f75b69575e6451b8563d12f176b
RANDOM SEED=900427737

node132.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23370092 kB
Buffers:        462612 kB
Cached:        3022600 kB
SwapCached:       1520 kB
Active:        5767088 kB
Inactive:      3393100 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23370092 kB
SwapTotal:    67111528 kB
SwapFree:     66863592 kB
Dirty:            5068 kB
Writeback:           0 kB
AnonPages:     5674364 kB
Mapped:          23816 kB
Slab:           333840 kB
PageTables:      24952 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 35030116 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73440 MiB
End job on node132 at 2011-05-03 07:26:51