Trace number 3381381

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40001.8 5023.8

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m62-q1327.5-n60-p15-s1351253.cnf
MD5SUMb23c52475e3d098cb9d5a92c025d6982
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.465928
Satisfiable
(Un)Satisfiability was proved
Number of variables3420
Number of clauses1120760
Sum of the clauses size2244820
Maximum clause length62
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21120700
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 560

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...514478/instance-3381381-1305514478.cnf
15.59/15.67	c Solving...
40000.54/5022.63	
40000.54/5022.63	*** INTERRUPTED! ***
40000.54/5022.63	*** Shutting down threads...
40001.69/5023.80	s UNKNOWN
40001.69/5023.80	
40001.69/5023.80	c INTERRUPTED   : 1
40001.69/5023.80	c Models        : 0+    
40001.69/5023.80	c Time          : 5022.640s (Solving: 5006.97s 1st Model: 0.00s Unsat: 0.00s)
40001.69/5023.80	c CPU Time      : 40000.610s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3381381-1305514478/watcher-3381381-1305514478 -o /tmp/evaluation-result-3381381-1305514478/solver-3381381-1305514478 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381381-1305514478.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.08 3.05 5.65 2/365 10139
/proc/meminfo: memFree=23588936/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=23312 CPUtime=0.01 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 1292 0 0 0 1 0 0 0 23 0 1 0 663048245 23871488 1025 33554432000 4194304 4964800 140735784806224 18446744073709551615 244446606021 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/10139/statm: 5828 1026 386 189 0 702 0

[startup+0.0799959 s]
/proc/loadavg: 2.08 3.05 5.65 2/365 10139
/proc/meminfo: memFree=23588936/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=36376 CPUtime=0.07 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 5974 0 0 0 7 0 0 0 23 0 1 0 663048245 37249024 4120 33554432000 4194304 4964800 140735784806224 18446744073709551615 244446606021 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/10139/statm: 9127 4122 386 189 0 4001 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 36376

[startup+0.100979 s]
/proc/loadavg: 2.08 3.05 5.65 2/365 10139
/proc/meminfo: memFree=23588936/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=39544 CPUtime=0.09 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 6960 0 0 0 9 0 0 0 23 0 1 0 663048245 40493056 5106 33554432000 4194304 4964800 140735784806224 18446744073709551615 244446945024 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/10139/statm: 9886 5109 386 189 0 4760 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 39544

[startup+0.300943 s]
/proc/loadavg: 2.08 3.05 5.65 2/365 10139
/proc/meminfo: memFree=23588936/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=81688 CPUtime=0.29 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 20826 0 0 0 28 1 0 0 25 0 1 0 663048245 83648512 14608 33554432000 4194304 4964800 140735784806224 18446744073709551615 4625038 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/10139/statm: 20422 14609 397 189 0 15296 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 81688

[startup+0.700867 s]
/proc/loadavg: 2.08 3.05 5.65 2/365 10139
/proc/meminfo: memFree=23588936/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=96224 CPUtime=0.69 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 24164 0 0 0 68 1 0 0 25 0 1 0 663048245 98533376 17946 33554432000 4194304 4964800 140735784806224 18446744073709551615 4607513 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/10139/statm: 24056 17946 399 189 0 18930 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 96224

[startup+1.50072 s]
/proc/loadavg: 2.08 3.05 5.65 2/366 10140
/proc/meminfo: memFree=23518372/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=96224 CPUtime=1.49 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 24164 0 0 0 148 1 0 0 25 0 1 0 663048245 98533376 17946 33554432000 4194304 4964800 140735784806224 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/10139/statm: 24056 17946 399 189 0 18930 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 96224

[startup+3.10042 s]
/proc/loadavg: 2.08 3.05 5.65 2/366 10140
/proc/meminfo: memFree=23517876/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=96224 CPUtime=3.09 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 24164 0 0 0 308 1 0 0 25 0 1 0 663048245 98533376 17946 33554432000 4194304 4964800 140735784806224 18446744073709551615 4607513 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/10139/statm: 24056 17946 399 189 0 18930 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 96224

[startup+6.30083 s]
/proc/loadavg: 1.99 3.02 5.62 2/366 10140
/proc/meminfo: memFree=23518620/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=96224 CPUtime=6.29 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 24164 0 0 0 628 1 0 0 25 0 1 0 663048245 98533376 17946 33554432000 4194304 4964800 140735784806224 18446744073709551615 4607536 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/10139/statm: 24056 17946 399 189 0 18930 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 96224

[startup+12.7006 s]
/proc/loadavg: 1.91 2.98 5.60 2/366 10140
/proc/meminfo: memFree=23516768/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=99948 CPUtime=12.69 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 25098 0 0 0 1267 2 0 0 25 0 1 0 663048245 102346752 18880 33554432000 4194304 4964800 140735784806224 18446744073709551615 4600421 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/10139/statm: 24987 18880 400 189 0 19861 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 99948

[startup+25.5012 s]
/proc/loadavg: 2.79 3.12 5.60 9/373 10147
/proc/meminfo: memFree=23329772/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=515552 CPUtime=86.11 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 76306 0 0 0 8575 36 0 0 25 0 8 0 663048245 527925248 67871 33554432000 4194304 4964800 140735784806224 18446744073709551615 244446605633 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/10139/statm: 128888 67871 466 189 0 123237 0
[pid=10139/tid=10141] ppid=10137 vsize=515552 CPUtime=8.56 cores=0-7
/proc/10139/task/10141/stat : 10141 (claspmt) R 10137 10139 10046 0 -1 4202560 4771 0 0 0 853 3 0 0 25 0 8 0 663049813 527925248 67871 33554432000 4194304 4964800 140735784806224 18446744073709551615 4627263 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=10139/tid=10142] ppid=10137 vsize=515552 CPUtime=8.68 cores=0-7
/proc/10139/task/10142/stat : 10142 (claspmt) R 10137 10139 10046 0 -1 4202560 4683 0 0 0 866 2 0 0 25 0 8 0 663049813 527925248 67871 33554432000 4194304 4964800 140735784806224 18446744073709551615 244446606021 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10139/tid=10143] ppid=10137 vsize=515552 CPUtime=8.67 cores=0-7
/proc/10139/task/10143/stat : 10143 (claspmt) R 10137 10139 10046 0 -1 4202560 6073 0 0 0 863 4 0 0 25 0 8 0 663049813 527925248 67872 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10139/tid=10144] ppid=10137 vsize=515552 CPUtime=8.66 cores=0-7
/proc/10139/task/10144/stat : 10144 (claspmt) R 10137 10139 10046 0 -1 4202560 6588 0 0 0 862 4 0 0 25 0 8 0 663049813 527925248 67872 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670371 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=10139/tid=10145] ppid=10137 vsize=515552 CPUtime=9.78 cores=0-7
/proc/10139/task/10145/stat : 10145 (claspmt) R 10137 10139 10046 0 -1 4202560 6184 0 0 0 975 3 0 0 25 0 8 0 663049813 527925248 67873 33554432000 4194304 4964800 140735784806224 18446744073709551615 244446606369 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=10139/tid=10146] ppid=10137 vsize=515552 CPUtime=8.67 cores=0-7
/proc/10139/task/10146/stat : 10146 (claspmt) R 10137 10139 10046 0 -1 4202560 6767 0 0 0 862 5 0 0 25 0 8 0 663049813 527925248 67873 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10139/tid=10147] ppid=10137 vsize=515552 CPUtime=8.68 cores=0-7
/proc/10139/task/10147/stat : 10147 (claspmt) R 10137 10139 10046 0 -1 4202560 8106 0 0 0 863 5 0 0 25 0 8 0 663049813 527925248 67873 33554432000 4194304 4964800 140735784806224 18446744073709551615 4627923 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 86.11
Current children cumulated vsize (KiB) 515552

[startup+51.1004 s]
/proc/loadavg: 4.56 3.51 5.66 9/373 10148
/proc/meminfo: memFree=23013828/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=939632 CPUtime=290.57 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 154576 0 0 0 28957 100 0 0 25 0 8 0 663048245 962183168 146141 33554432000 4194304 4964800 140735784806224 18446744073709551615 4660419 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/10139/statm: 234908 146141 466 189 0 229257 0
[pid=10139/tid=10141] ppid=10137 vsize=939632 CPUtime=33.86 cores=0-7
/proc/10139/task/10141/stat : 10141 (claspmt) R 10137 10139 10046 0 -1 4202560 11791 0 0 0 3377 9 0 0 25 0 8 0 663049813 962183168 146141 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=10139/tid=10142] ppid=10137 vsize=939632 CPUtime=34.29 cores=0-7
/proc/10139/task/10142/stat : 10142 (claspmt) R 10137 10139 10046 0 -1 4202560 9669 0 0 0 3424 5 0 0 25 0 8 0 663049813 962183168 146141 33554432000 4194304 4964800 140735784806224 18446744073709551615 4321076 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10139/tid=10143] ppid=10137 vsize=939632 CPUtime=34.26 cores=0-7
/proc/10139/task/10143/stat : 10143 (claspmt) R 10137 10139 10046 0 -1 4202560 16798 0 0 0 3412 14 0 0 25 0 8 0 663049813 962183168 146141 33554432000 4194304 4964800 140735784806224 18446744073709551615 4627335 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10139/tid=10144] ppid=10137 vsize=939632 CPUtime=34.25 cores=0-7
/proc/10139/task/10144/stat : 10144 (claspmt) R 10137 10139 10046 0 -1 4202560 14834 0 0 0 3414 11 0 0 25 0 8 0 663049813 962183168 146141 33554432000 4194304 4964800 140735784806224 18446744073709551615 4627540 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=10139/tid=10145] ppid=10137 vsize=939632 CPUtime=35.37 cores=0-7
/proc/10139/task/10145/stat : 10145 (claspmt) R 10137 10139 10046 0 -1 4202560 19497 0 0 0 3524 13 0 0 25 0 8 0 663049813 962183168 146142 33554432000 4194304 4964800 140735784806224 18446744073709551615 244446606369 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=10139/tid=10146] ppid=10137 vsize=939632 CPUtime=34.27 cores=0-7
/proc/10139/task/10146/stat : 10146 (claspmt) R 10137 10139 10046 0 -1 4202560 13533 0 0 0 3415 12 0 0 25 0 8 0 663049813 962183168 146142 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10139/tid=10147] ppid=10137 vsize=939632 CPUtime=34.27 cores=0-7
/proc/10139/task/10147/stat : 10147 (claspmt) R 10137 10139 10046 0 -1 4202560 21215 0 0 0 3413 14 0 0 25 0 8 0 663049813 962183168 146142 33554432000 4194304 4964800 140735784806224 18446744073709551615 4627293 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 290.57
Current children cumulated vsize (KiB) 939632

[startup+102.314 s]
/proc/loadavg: 6.56 4.21 5.79 9/373 10149
/proc/meminfo: memFree=22591472/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=1263684 CPUtime=699.66 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 259138 0 0 0 69793 173 0 0 25 0 8 0 663048245 1294012416 250648 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670499 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/10139/statm: 315921 250648 466 189 0 310270 0
[pid=10139/tid=10141] ppid=10137 vsize=1263684 CPUtime=84.46 cores=0-7
/proc/10139/task/10141/stat : 10141 (claspmt) R 10137 10139 10046 0 -1 4202560 16543 0 0 0 8433 13 0 0 25 0 8 0 663049813 1294012416 250648 33554432000 4194304 4964800 140735784806224 18446744073709551615 4684040 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=10139/tid=10142] ppid=10137 vsize=1263684 CPUtime=85.5 cores=0-7
/proc/10139/task/10142/stat : 10142 (claspmt) R 10137 10139 10046 0 -1 4202560 13850 0 0 0 8543 7 0 0 25 0 8 0 663049813 1294012416 250648 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10139/tid=10143] ppid=10137 vsize=1263684 CPUtime=85.48 cores=0-7
/proc/10139/task/10143/stat : 10143 (claspmt) R 10137 10139 10046 0 -1 4202560 20744 0 0 0 8531 17 0 0 25 0 8 0 663049813 1294012416 250648 33554432000 4194304 4964800 140735784806224 18446744073709551615 4320984 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10139/tid=10144] ppid=10137 vsize=1263684 CPUtime=85.46 cores=0-7
/proc/10139/task/10144/stat : 10144 (claspmt) R 10137 10139 10046 0 -1 4202560 24151 0 0 0 8526 20 0 0 25 0 8 0 663049813 1294012416 250648 33554432000 4194304 4964800 140735784806224 18446744073709551615 4627281 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=10139/tid=10145] ppid=10137 vsize=1263684 CPUtime=86.59 cores=0-7
/proc/10139/task/10145/stat : 10145 (claspmt) R 10137 10139 10046 0 -1 4202560 27791 0 0 0 8640 19 0 0 25 0 8 0 663049813 1294012416 250648 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=10139/tid=10146] ppid=10137 vsize=1263684 CPUtime=85.47 cores=0-7
/proc/10139/task/10146/stat : 10146 (claspmt) R 10137 10139 10046 0 -1 4202560 20965 0 0 0 8530 17 0 0 25 0 8 0 663049813 1294012416 250648 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10139/tid=10147] ppid=10137 vsize=1263684 CPUtime=85.48 cores=0-7
/proc/10139/task/10147/stat : 10147 (claspmt) R 10137 10139 10046 0 -1 4202560 53854 0 0 0 8511 37 0 0 25 0 8 0 663049813 1294012416 250648 33554432000 4194304 4964800 140735784806224 18446744073709551615 4660168 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 699.66
Current children cumulated vsize (KiB) 1263684

[startup+162.301 s]
/proc/loadavg: 7.47 4.90 5.92 9/373 10151
/proc/meminfo: memFree=22323732/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=1429280 CPUtime=1178.84 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 326045 0 0 0 117651 233 0 0 25 0 8 0 663048245 1463582720 317531 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670538 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/10139/statm: 357320 317531 466 189 0 351669 0
[pid=10139/tid=10141] ppid=10137 vsize=1429280 CPUtime=143.74 cores=0-7
/proc/10139/task/10141/stat : 10141 (claspmt) R 10137 10139 10046 0 -1 4202560 22089 0 0 0 14354 20 0 0 25 0 8 0 663049813 1463582720 317531 33554432000 4194304 4964800 140735784806224 18446744073709551615 4627218 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=10139/tid=10142] ppid=10137 vsize=1429280 CPUtime=145.49 cores=0-7
/proc/10139/task/10142/stat : 10142 (claspmt) R 10137 10139 10046 0 -1 4202560 15524 0 0 0 14541 8 0 0 25 0 8 0 663049813 1463582720 317531 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10139/tid=10143] ppid=10137 vsize=1429280 CPUtime=145.47 cores=0-7
/proc/10139/task/10143/stat : 10143 (claspmt) R 10137 10139 10046 0 -1 4202560 24433 0 0 0 14527 20 0 0 25 0 8 0 663049813 1463582720 317531 33554432000 4194304 4964800 140735784806224 18446744073709551615 4627818 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10139/tid=10144] ppid=10137 vsize=1429280 CPUtime=145.43 cores=0-7
/proc/10139/task/10144/stat : 10144 (claspmt) R 10137 10139 10046 0 -1 4202560 28223 0 0 0 14519 24 0 0 25 0 8 0 663049813 1463582720 317531 33554432000 4194304 4964800 140735784806224 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=10139/tid=10145] ppid=10137 vsize=1429280 CPUtime=146.58 cores=0-7
/proc/10139/task/10145/stat : 10145 (claspmt) R 10137 10139 10046 0 -1 4202560 32707 0 0 0 14636 22 0 0 25 0 8 0 663049813 1463582720 317531 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=10139/tid=10146] ppid=10137 vsize=1429280 CPUtime=145.47 cores=0-7
/proc/10139/task/10146/stat : 10146 (claspmt) R 10137 10139 10046 0 -1 4202560 22156 0 0 0 14528 19 0 0 25 0 8 0 663049813 1463582720 317531 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10139/tid=10147] ppid=10137 vsize=1429280 CPUtime=145.46 cores=0-7
/proc/10139/task/10147/stat : 10147 (claspmt) R 10137 10139 10046 0 -1 4202560 71921 0 0 0 14497 49 0 0 25 0 8 0 663049813 1463582720 317531 33554432000 4194304 4964800 140735784806224 18446744073709551615 4320963 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1178.84
Current children cumulated vsize (KiB) 1429280

[startup+222.301 s]
/proc/loadavg: 7.85 5.48 6.05 9/373 10152
/proc/meminfo: memFree=22036260/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=1869736 CPUtime=1658.12 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 398072 0 0 0 165517 295 0 0 25 0 8 0 663048245 1914609664 389558 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/10139/statm: 467434 389558 466 189 0 461783 0
[pid=10139/tid=10141] ppid=10137 vsize=1869736 CPUtime=203.02 cores=0-7
/proc/10139/task/10141/stat : 10141 (claspmt) R 10137 10139 10046 0 -1 4202560 29547 0 0 0 20276 26 0 0 25 0 8 0 663049813 1914609664 389558 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0

################
# More data... #
################

[pid=10139] ppid=10137 vsize=5645968 CPUtime=38080.9 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 1339186 0 0 0 3806674 1411 0 0 25 0 8 0 663048245 5781471232 1325834 33554432000 4194304 4964800 140735784806224 18446744073709551615 4332545 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/10139/statm: 1411492 1325834 466 189 0 1405841 0
[pid=10139/tid=10141] ppid=10137 vsize=5645968 CPUtime=4749.03 cores=0-7
/proc/10139/task/10141/stat : 10141 (claspmt) R 10137 10139 10046 0 -1 4202560 93393 0 0 0 474797 106 0 0 25 0 8 0 663049813 5781471232 1325834 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=10139/tid=10142] ppid=10137 vsize=5645968 CPUtime=4765.34 cores=0-7
/proc/10139/task/10142/stat : 10142 (claspmt) R 10137 10139 10046 0 -1 4202560 85540 0 0 0 476494 40 0 0 25 0 8 0 663049813 5781471232 1325834 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10139/tid=10143] ppid=10137 vsize=5645968 CPUtime=4765.43 cores=0-7
/proc/10139/task/10143/stat : 10143 (claspmt) R 10137 10139 10046 0 -1 4202560 79364 0 0 0 476467 76 0 0 25 0 8 0 663049813 5781471232 1325834 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10139/tid=10144] ppid=10137 vsize=5645968 CPUtime=4763.88 cores=0-7
/proc/10139/task/10144/stat : 10144 (claspmt) R 10137 10139 10046 0 -1 4202560 78261 0 0 0 476319 69 0 0 25 0 8 0 663049813 5781471232 1325834 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=10139/tid=10145] ppid=10137 vsize=5645968 CPUtime=4759.56 cores=0-7
/proc/10139/task/10145/stat : 10145 (claspmt) R 10137 10139 10046 0 -1 4202560 95039 0 0 0 475860 96 0 0 25 0 8 0 663049813 5781471232 1325834 33554432000 4194304 4964800 140735784806224 18446744073709551615 4627293 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10139/tid=10146] ppid=10137 vsize=5645968 CPUtime=4764.49 cores=0-7
/proc/10139/task/10146/stat : 10146 (claspmt) R 10137 10139 10046 0 -1 4202560 62478 0 0 0 476384 65 0 0 25 0 8 0 663049813 5781471232 1325834 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10139/tid=10147] ppid=10137 vsize=5645968 CPUtime=4757.98 cores=0-7
/proc/10139/task/10147/stat : 10147 (claspmt) R 10137 10139 10046 0 -1 4202560 432975 0 0 0 475286 512 0 0 25 0 8 0 663049813 5781471232 1325834 33554432000 4194304 4964800 140735784806224 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38080.8
Current children cumulated vsize (KiB) 5645968

[startup+4842.3 s]
/proc/loadavg: 8.09 8.06 8.01 9/373 10300
/proc/meminfo: memFree=18260148/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=5645968 CPUtime=38560.1 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 1344264 0 0 0 3854593 1419 0 0 25 0 8 0 663048245 5781471232 1330912 33554432000 4194304 4964800 140735784806224 18446744073709551615 4653382 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/10139/statm: 1411492 1330912 466 189 0 1405841 0
[pid=10139/tid=10141] ppid=10137 vsize=5645968 CPUtime=4808.31 cores=0-7
/proc/10139/task/10141/stat : 10141 (claspmt) R 10137 10139 10046 0 -1 4202560 93413 0 0 0 480725 106 0 0 25 0 8 0 663049813 5781471232 1330912 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=10139/tid=10142] ppid=10137 vsize=5645968 CPUtime=4825.34 cores=0-7
/proc/10139/task/10142/stat : 10142 (claspmt) R 10137 10139 10046 0 -1 4202560 85609 0 0 0 482494 40 0 0 25 0 8 0 663049813 5781471232 1330912 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10139/tid=10143] ppid=10137 vsize=5645968 CPUtime=4825.43 cores=0-7
/proc/10139/task/10143/stat : 10143 (claspmt) R 10137 10139 10046 0 -1 4202560 79364 0 0 0 482467 76 0 0 25 0 8 0 663049813 5781471232 1330912 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10139/tid=10144] ppid=10137 vsize=5645968 CPUtime=4823.87 cores=0-7
/proc/10139/task/10144/stat : 10144 (claspmt) R 10137 10139 10046 0 -1 4202560 78266 0 0 0 482318 69 0 0 25 0 8 0 663049813 5781471232 1330912 33554432000 4194304 4964800 140735784806224 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=10139/tid=10145] ppid=10137 vsize=5645968 CPUtime=4819.56 cores=0-7
/proc/10139/task/10145/stat : 10145 (claspmt) R 10137 10139 10046 0 -1 4202560 95075 0 0 0 481860 96 0 0 25 0 8 0 663049813 5781471232 1330912 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10139/tid=10146] ppid=10137 vsize=5645968 CPUtime=4824.49 cores=0-7
/proc/10139/task/10146/stat : 10146 (claspmt) R 10137 10139 10046 0 -1 4202560 62501 0 0 0 482384 65 0 0 25 0 8 0 663049813 5781471232 1330912 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10139/tid=10147] ppid=10137 vsize=5645968 CPUtime=4817.98 cores=0-7
/proc/10139/task/10147/stat : 10147 (claspmt) R 10137 10139 10046 0 -1 4202560 432975 0 0 0 481286 512 0 0 25 0 8 0 663049813 5781471232 1330912 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38560.1
Current children cumulated vsize (KiB) 5645968

[startup+4902.3 s]
/proc/loadavg: 8.03 8.05 8.00 9/373 10301
/proc/meminfo: memFree=18256776/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=5645968 CPUtime=39039.4 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 1345035 0 0 0 3902522 1420 0 0 25 0 8 0 663048245 5781471232 1331683 33554432000 4194304 4964800 140735784806224 18446744073709551615 4320981 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/10139/statm: 1411492 1331683 466 189 0 1405841 0
[pid=10139/tid=10141] ppid=10137 vsize=5645968 CPUtime=4867.61 cores=0-7
/proc/10139/task/10141/stat : 10141 (claspmt) R 10137 10139 10046 0 -1 4202560 93437 0 0 0 486655 106 0 0 25 0 8 0 663049813 5781471232 1331683 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670321 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=10139/tid=10142] ppid=10137 vsize=5645968 CPUtime=4885.34 cores=0-7
/proc/10139/task/10142/stat : 10142 (claspmt) R 10137 10139 10046 0 -1 4202560 86077 0 0 0 488494 40 0 0 25 0 8 0 663049813 5781471232 1331683 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670340 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10139/tid=10143] ppid=10137 vsize=5645968 CPUtime=4885.43 cores=0-7
/proc/10139/task/10143/stat : 10143 (claspmt) R 10137 10139 10046 0 -1 4202560 79611 0 0 0 488467 76 0 0 25 0 8 0 663049813 5781471232 1331683 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10139/tid=10144] ppid=10137 vsize=5645968 CPUtime=4883.87 cores=0-7
/proc/10139/task/10144/stat : 10144 (claspmt) R 10137 10139 10046 0 -1 4202560 78266 0 0 0 488318 69 0 0 25 0 8 0 663049813 5781471232 1331683 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=10139/tid=10145] ppid=10137 vsize=5645968 CPUtime=4879.56 cores=0-7
/proc/10139/task/10145/stat : 10145 (claspmt) R 10137 10139 10046 0 -1 4202560 95103 0 0 0 487860 96 0 0 25 0 8 0 663049813 5781471232 1331683 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10139/tid=10146] ppid=10137 vsize=5645968 CPUtime=4884.49 cores=0-7
/proc/10139/task/10146/stat : 10146 (claspmt) R 10137 10139 10046 0 -1 4202560 62501 0 0 0 488384 65 0 0 25 0 8 0 663049813 5781471232 1331683 33554432000 4194304 4964800 140735784806224 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10139/tid=10147] ppid=10137 vsize=5645968 CPUtime=4877.98 cores=0-7
/proc/10139/task/10147/stat : 10147 (claspmt) R 10137 10139 10046 0 -1 4202560 432975 0 0 0 487286 512 0 0 25 0 8 0 663049813 5781471232 1331683 33554432000 4194304 4964800 140735784806224 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39039.4
Current children cumulated vsize (KiB) 5645968

[startup+4962.3 s]
/proc/loadavg: 8.08 8.05 8.00 9/373 10303
/proc/meminfo: memFree=18240356/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=5711504 CPUtime=39518.7 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 1349303 0 0 0 3950442 1427 0 0 25 0 8 0 663048245 5848580096 1335951 33554432000 4194304 4964800 140735784806224 18446744073709551615 4321102 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/10139/statm: 1427876 1335951 466 189 0 1422225 0
[pid=10139/tid=10141] ppid=10137 vsize=5711504 CPUtime=4926.89 cores=0-7
/proc/10139/task/10141/stat : 10141 (claspmt) R 10137 10139 10046 0 -1 4202560 93448 0 0 0 492583 106 0 0 25 0 8 0 663049813 5848580096 1335951 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670378 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=10139/tid=10142] ppid=10137 vsize=5711504 CPUtime=4945.34 cores=0-7
/proc/10139/task/10142/stat : 10142 (claspmt) R 10137 10139 10046 0 -1 4202560 86185 0 0 0 494494 40 0 0 25 0 8 0 663049813 5848580096 1335951 33554432000 4194304 4964800 140735784806224 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10139/tid=10143] ppid=10137 vsize=5711504 CPUtime=4945.44 cores=0-7
/proc/10139/task/10143/stat : 10143 (claspmt) R 10137 10139 10046 0 -1 4202560 79611 0 0 0 494467 77 0 0 25 0 8 0 663049813 5848580096 1335951 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10139/tid=10144] ppid=10137 vsize=5711504 CPUtime=4943.85 cores=0-7
/proc/10139/task/10144/stat : 10144 (claspmt) R 10137 10139 10046 0 -1 4202560 78266 0 0 0 494316 69 0 0 25 0 8 0 663049813 5848580096 1335951 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=10139/tid=10145] ppid=10137 vsize=5711504 CPUtime=4939.56 cores=0-7
/proc/10139/task/10145/stat : 10145 (claspmt) R 10137 10139 10046 0 -1 4202560 95127 0 0 0 493860 96 0 0 25 0 8 0 663049813 5848580096 1335951 33554432000 4194304 4964800 140735784806224 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10139/tid=10146] ppid=10137 vsize=5711504 CPUtime=4944.49 cores=0-7
/proc/10139/task/10146/stat : 10146 (claspmt) R 10137 10139 10046 0 -1 4202560 62502 0 0 0 494384 65 0 0 25 0 8 0 663049813 5848580096 1335951 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10139/tid=10147] ppid=10137 vsize=5711504 CPUtime=4937.97 cores=0-7
/proc/10139/task/10147/stat : 10147 (claspmt) R 10137 10139 10046 0 -1 4202560 432977 0 0 0 493285 512 0 0 25 0 8 0 663049813 5848580096 1335951 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39518.7
Current children cumulated vsize (KiB) 5711504

[startup+5022.3 s]
/proc/loadavg: 8.09 8.06 8.00 9/373 10304
/proc/meminfo: memFree=18226936/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=5711504 CPUtime=39997.9 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 1352797 0 0 0 3998361 1434 0 0 25 0 8 0 663048245 5848580096 1339445 33554432000 4194304 4964800 140735784806224 18446744073709551615 4627252 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/10139/statm: 1427876 1339445 466 189 0 1422225 0
[pid=10139/tid=10141] ppid=10137 vsize=5711504 CPUtime=4986.15 cores=0-7
/proc/10139/task/10141/stat : 10141 (claspmt) R 10137 10139 10046 0 -1 4202560 93466 0 0 0 498509 106 0 0 25 0 8 0 663049813 5848580096 1339445 33554432000 4194304 4964800 140735784806224 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=10139/tid=10142] ppid=10137 vsize=5711504 CPUtime=5005.34 cores=0-7
/proc/10139/task/10142/stat : 10142 (claspmt) R 10137 10139 10046 0 -1 4202560 86272 0 0 0 500494 40 0 0 25 0 8 0 663049813 5848580096 1339445 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10139/tid=10143] ppid=10137 vsize=5711504 CPUtime=5005.45 cores=0-7
/proc/10139/task/10143/stat : 10143 (claspmt) R 10137 10139 10046 0 -1 4202560 79631 0 0 0 500468 77 0 0 25 0 8 0 663049813 5848580096 1339445 33554432000 4194304 4964800 140735784806224 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10139/tid=10144] ppid=10137 vsize=5711504 CPUtime=5003.84 cores=0-7
/proc/10139/task/10144/stat : 10144 (claspmt) R 10137 10139 10046 0 -1 4202560 78266 0 0 0 500315 69 0 0 25 0 8 0 663049813 5848580096 1339445 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=10139/tid=10145] ppid=10137 vsize=5711504 CPUtime=4999.57 cores=0-7
/proc/10139/task/10145/stat : 10145 (claspmt) R 10137 10139 10046 0 -1 4202560 95155 0 0 0 499861 96 0 0 25 0 8 0 663049813 5848580096 1339445 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10139/tid=10146] ppid=10137 vsize=5711504 CPUtime=5004.49 cores=0-7
/proc/10139/task/10146/stat : 10146 (claspmt) R 10137 10139 10046 0 -1 4202560 62504 0 0 0 500384 65 0 0 25 0 8 0 663049813 5848580096 1339445 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10139/tid=10147] ppid=10137 vsize=5711504 CPUtime=4997.97 cores=0-7
/proc/10139/task/10147/stat : 10147 (claspmt) R 10137 10139 10046 0 -1 4202560 433612 0 0 0 499283 514 0 0 25 0 8 0 663049813 5848580096 1339445 33554432000 4194304 4964800 140735784806224 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39997.9
Current children cumulated vsize (KiB) 5711504



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5022.61 s]
/proc/loadavg: 8.09 8.06 8.00 9/373 10304
/proc/meminfo: memFree=18225572/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=5711504 CPUtime=40000.5 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 1352933 0 0 0 3998620 1434 0 0 25 0 8 0 663048245 5848580096 1339581 33554432000 4194304 4964800 140735784806224 18446744073709551615 4320905 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/10139/statm: 1427876 1339581 466 189 0 1422225 0
[pid=10139/tid=10141] ppid=10137 vsize=5711504 CPUtime=4986.45 cores=0-7
/proc/10139/task/10141/stat : 10141 (claspmt) R 10137 10139 10046 0 -1 4202560 93468 0 0 0 498539 106 0 0 25 0 8 0 663049813 5848580096 1339581 33554432000 4194304 4964800 140735784806224 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=10139/tid=10142] ppid=10137 vsize=5711504 CPUtime=5005.67 cores=0-7
/proc/10139/task/10142/stat : 10142 (claspmt) R 10137 10139 10046 0 -1 4202560 86272 0 0 0 500527 40 0 0 25 0 8 0 663049813 5848580096 1339581 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10139/tid=10143] ppid=10137 vsize=5711504 CPUtime=5005.77 cores=0-7
/proc/10139/task/10143/stat : 10143 (claspmt) R 10137 10139 10046 0 -1 4202560 79631 0 0 0 500500 77 0 0 25 0 8 0 663049813 5848580096 1339581 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10139/tid=10144] ppid=10137 vsize=5711504 CPUtime=5004.17 cores=0-7
/proc/10139/task/10144/stat : 10144 (claspmt) R 10137 10139 10046 0 -1 4202560 78266 0 0 0 500348 69 0 0 25 0 8 0 663049813 5848580096 1339581 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=10139/tid=10145] ppid=10137 vsize=5711504 CPUtime=4999.89 cores=0-7
/proc/10139/task/10145/stat : 10145 (claspmt) R 10137 10139 10046 0 -1 4202560 95155 0 0 0 499893 96 0 0 25 0 8 0 663049813 5848580096 1339581 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10139/tid=10146] ppid=10137 vsize=5711504 CPUtime=5004.82 cores=0-7
/proc/10139/task/10146/stat : 10146 (claspmt) R 10137 10139 10046 0 -1 4202560 62504 0 0 0 500417 65 0 0 25 0 8 0 663049813 5848580096 1339581 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10139/tid=10147] ppid=10137 vsize=5711504 CPUtime=4998.3 cores=0-7
/proc/10139/task/10147/stat : 10147 (claspmt) R 10137 10139 10046 0 -1 4202560 433746 0 0 0 499316 514 0 0 25 0 8 0 663049813 5848580096 1339581 33554432000 4194304 4964800 140735784806224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 5711504

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5023 s]
/proc/loadavg: 8.09 8.06 8.00 9/373 10304
/proc/meminfo: memFree=18225572/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=5711076 CPUtime=40001 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 1352946 0 0 0 3998663 1434 0 0 25 0 1 0 663048245 5848141824 1338749 33554432000 4194304 4964800 140735784806224 18446744073709551615 244446602218 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/10139/statm: 1427769 1338749 476 189 0 1422118 0
Current children cumulated CPU time (s) 40001
Current children cumulated vsize (KiB) 5711076

[startup+5023.4 s]
/proc/loadavg: 8.09 8.06 8.00 9/373 10304
/proc/meminfo: memFree=18225572/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=5711076 CPUtime=40001.4 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202496 1352946 0 0 0 3998703 1434 0 0 25 0 1 0 663048245 5848141824 1338697 33554432000 4194304 4964800 140735784806224 18446744073709551615 244446601787 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/10139/statm: 1427769 1338697 476 189 0 1422118 0
Current children cumulated CPU time (s) 40001.4
Current children cumulated vsize (KiB) 5711076

[startup+5023.8 s]
/proc/loadavg: 7.52 7.94 7.97 2/367 10306
/proc/meminfo: memFree=18275096/32951124 swapFree=44485772/67111528
[pid=10139] ppid=10137 vsize=0 CPUtime=40001.8 cores=0-7
/proc/10139/stat : 10139 (claspmt) R 10137 10139 10046 0 -1 4202500 1352947 0 0 0 3998723 1454 0 0 25 0 1 0 663048245 0 0 33554432000 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/10139/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 40001.8
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 5023.8
CPU time (s): 40001.8
CPU user time (s): 39987.2
CPU system time (s): 14.5498
CPU usage (%): 796.245
Max. virtual memory (cumulated for all children) (KiB): 5711504

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39987.2
system time used= 14.5498
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1352947
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 57618
involuntary context switches= 136124

runsolver used 16.7964 second user time and 42.8865 second system time

The end

Launcher Data 

Begin job on node105 at 2011-05-16 04:54:38
IDJOB=3381381
IDBENCH=88217
IDSOLVER=1865
FILE ID=node105/3381381-1305514478
RUNJOBID= node105-1305514081-10060
PBS_JOBID= 13325556
Free space on /tmp= 73608 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m62-q1327.5-n60-p15-s1351253.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3381381-1305514478/watcher-3381381-1305514478 -o /tmp/evaluation-result-3381381-1305514478/solver-3381381-1305514478 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381381-1305514478.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= b23c52475e3d098cb9d5a92c025d6982
RANDOM SEED=837512242

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23589208 kB
Buffers:        317816 kB
Cached:        6924552 kB
SwapCached:      39648 kB
Active:        2106624 kB
Inactive:      6969392 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23589208 kB
SwapTotal:    67111528 kB
SwapFree:     44485772 kB
Dirty:           15564 kB
Writeback:           0 kB
AnonPages:     1802332 kB
Mapped:          15068 kB
Slab:           142792 kB
PageTables:      72060 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 91323632 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73608 MiB
End job on node105 at 2011-05-16 06:18:23