Trace number 3321672

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40001.4 5008.35

General information on the benchmark

Nameapplication/jarvisalo/HWMCC10-combination-timeframe-expansion/
hwmcc10-timeframe-expansion-k45-pdtvisns3p02-tseitin.cnf
MD5SUMa0ec1baa890da62d0c6df7f1e62c6af3
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark2133.18
Satisfiable
(Un)Satisfiability was proved
Number of variables163622
Number of clauses488120
Sum of the clauses size1138944
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2325412
Number of clauses of size 3162706
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...366470/instance-3321672-1304366470.cnf
1.29/1.38	c Solving...
40000.49/5007.51	
40000.49/5007.51	*** INTERRUPTED! ***
40000.49/5007.51	*** Shutting down threads...
40001.34/5008.34	s UNKNOWN
40001.34/5008.34	
40001.34/5008.34	c INTERRUPTED   : 1
40001.34/5008.34	c Models        : 0+    
40001.34/5008.34	c Time          : 5007.519s (Solving: 5006.14s 1st Model: 0.00s Unsat: 0.00s)
40001.34/5008.34	c CPU Time      : 40000.550s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321672-1304366470/watcher-3321672-1304366470 -o /tmp/evaluation-result-3321672-1304366470/solver-3321672-1304366470 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321672-1304366470.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.00 0.00 0.00 1/278 30773
/proc/meminfo: memFree=8312392/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=41652 CPUtime=0 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 2945 0 0 0 0 0 0 0 22 0 1 0 280838330 42651648 2874 33554432000 4194304 4964704 140736031753408 18446744073709551615 4700332 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/30773/statm: 10413 2882 376 189 0 5287 0

[startup+0.0922241 s]
/proc/loadavg: 0.00 0.00 0.00 1/278 30773
/proc/meminfo: memFree=8312392/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=64060 CPUtime=0.08 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 13407 0 0 0 6 2 0 0 22 0 1 0 280838330 65597440 10908 33554432000 4194304 4964704 140736031753408 18446744073709551615 268660323013 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/30773/statm: 16015 10909 386 189 0 10889 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 64060

[startup+0.100219 s]
/proc/loadavg: 0.00 0.00 0.00 1/278 30773
/proc/meminfo: memFree=8312392/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=65116 CPUtime=0.09 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 13740 0 0 0 7 2 0 0 22 0 1 0 280838330 66678784 11241 33554432000 4194304 4964704 140736031753408 18446744073709551615 268660323013 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/30773/statm: 16279 11242 386 189 0 11153 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 65116

[startup+0.300185 s]
/proc/loadavg: 0.00 0.00 0.00 1/278 30773
/proc/meminfo: memFree=8312392/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=101952 CPUtime=0.28 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 25478 0 0 0 25 3 0 0 23 0 1 0 280838330 104398848 19822 33554432000 4194304 4964704 140736031753408 18446744073709551615 4607472 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/30773/statm: 25488 19822 403 189 0 20362 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 101952

[startup+0.701116 s]
/proc/loadavg: 0.00 0.00 0.00 1/278 30773
/proc/meminfo: memFree=8312392/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=111984 CPUtime=0.69 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 28010 0 0 0 65 4 0 0 25 0 1 0 280838330 114671616 22354 33554432000 4194304 4964704 140736031753408 18446744073709551615 4619881 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/30773/statm: 27996 22354 404 189 0 22870 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 111984

[startup+1.50094 s]
/proc/loadavg: 0.00 0.00 0.00 2/279 30774
/proc/meminfo: memFree=8219156/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=671604 CPUtime=1.69 cores=0-7
/proc/30773/stat : 30773 (claspmt) S 30771 30773 30747 0 -1 4202496 72612 0 0 0 146 23 0 0 25 0 8 0 280838330 687722496 55168 33554432000 4194304 4964704 140736031753408 18446744073709551615 268672478105 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/30773/statm: 167901 55171 437 189 0 162250 0
Current children cumulated CPU time (s) 1.69
Current children cumulated vsize (KiB) 671604

[startup+3.10066 s]
/proc/loadavg: 0.64 0.13 0.04 9/286 30781
/proc/meminfo: memFree=8087288/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=671604 CPUtime=12.26 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 74626 0 0 0 1201 25 0 0 25 0 8 0 280838330 687722496 57182 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670100 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/30773/statm: 167901 57182 464 189 0 162250 0
[pid=30773/tid=30775] ppid=30771 vsize=671604 CPUtime=1.33 cores=0-7
/proc/30773/task/30775/stat : 30775 (claspmt) R 30771 30773 30747 0 -1 4202560 5308 0 0 0 131 2 0 0 17 0 8 0 280838469 687722496 57182 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670100 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30773/tid=30776] ppid=30771 vsize=671604 CPUtime=1.31 cores=0-7
/proc/30773/task/30776/stat : 30776 (claspmt) R 30771 30773 30747 0 -1 4202560 7144 0 0 0 128 3 0 0 17 0 8 0 280838469 687722496 57182 33554432000 4194304 4964704 140736031753408 18446744073709551615 4342144 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30773/tid=30777] ppid=30771 vsize=671604 CPUtime=1.34 cores=0-7
/proc/30773/task/30777/stat : 30777 (claspmt) R 30771 30773 30747 0 -1 4202560 6753 0 0 0 131 3 0 0 17 0 8 0 280838469 687722496 57182 33554432000 4194304 4964704 140736031753408 18446744073709551615 4669051 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30773/tid=30778] ppid=30771 vsize=671604 CPUtime=1.3 cores=0-7
/proc/30773/task/30778/stat : 30778 (claspmt) R 30771 30773 30747 0 -1 4202560 5376 0 0 0 128 2 0 0 17 0 8 0 280838469 687722496 57182 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30773/tid=30779] ppid=30771 vsize=671604 CPUtime=1.63 cores=0-7
/proc/30773/task/30779/stat : 30779 (claspmt) R 30771 30773 30747 0 -1 4202560 6228 0 0 0 161 2 0 0 20 0 8 0 280838469 687722496 57182 33554432000 4194304 4964704 140736031753408 18446744073709551615 4668464 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30773/tid=30780] ppid=30771 vsize=671604 CPUtime=1.32 cores=0-7
/proc/30773/task/30780/stat : 30780 (claspmt) R 30771 30773 30747 0 -1 4202560 6736 0 0 0 129 3 0 0 17 0 8 0 280838469 687722496 57182 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30773/tid=30781] ppid=30771 vsize=671604 CPUtime=1.34 cores=0-7
/proc/30773/task/30781/stat : 30781 (claspmt) R 30771 30773 30747 0 -1 4202560 6339 0 0 0 132 2 0 0 17 0 8 0 280838469 687722496 57182 33554432000 4194304 4964704 140736031753408 18446744073709551615 268660329376 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 12.26
Current children cumulated vsize (KiB) 671604

[startup+6.3011 s]
/proc/loadavg: 0.64 0.13 0.04 9/286 30781
/proc/meminfo: memFree=8081620/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=672044 CPUtime=37.84 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 75916 0 0 0 3757 27 0 0 25 0 8 0 280838330 688173056 58472 33554432000 4194304 4964704 140736031753408 18446744073709551615 4668693 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/30773/statm: 168011 58472 464 189 0 162360 0
[pid=30773/tid=30775] ppid=30771 vsize=672044 CPUtime=4.53 cores=0-7
/proc/30773/task/30775/stat : 30775 (claspmt) R 30771 30773 30747 0 -1 4202560 5449 0 0 0 451 2 0 0 24 0 8 0 280838469 688173056 58472 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30773/tid=30776] ppid=30771 vsize=672044 CPUtime=4.51 cores=0-7
/proc/30773/task/30776/stat : 30776 (claspmt) R 30771 30773 30747 0 -1 4202560 7339 0 0 0 448 3 0 0 23 0 8 0 280838469 688173056 58472 33554432000 4194304 4964704 140736031753408 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30773/tid=30777] ppid=30771 vsize=672044 CPUtime=4.54 cores=0-7
/proc/30773/task/30777/stat : 30777 (claspmt) R 30771 30773 30747 0 -1 4202560 7012 0 0 0 451 3 0 0 23 0 8 0 280838469 688173056 58472 33554432000 4194304 4964704 140736031753408 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30773/tid=30778] ppid=30771 vsize=672044 CPUtime=4.51 cores=0-7
/proc/30773/task/30778/stat : 30778 (claspmt) R 30771 30773 30747 0 -1 4202560 5568 0 0 0 448 3 0 0 23 0 8 0 280838469 688173056 58472 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30773/tid=30779] ppid=30771 vsize=672044 CPUtime=4.8 cores=0-7
/proc/30773/task/30779/stat : 30779 (claspmt) R 30771 30773 30747 0 -1 4202560 6230 0 0 0 478 2 0 0 25 0 8 0 280838469 688173056 58472 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30773/tid=30780] ppid=30771 vsize=672044 CPUtime=4.52 cores=0-7
/proc/30773/task/30780/stat : 30780 (claspmt) R 30771 30773 30747 0 -1 4202560 7020 0 0 0 449 3 0 0 23 0 8 0 280838469 688173056 58472 33554432000 4194304 4964704 140736031753408 18446744073709551615 4668907 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30773/tid=30781] ppid=30771 vsize=672044 CPUtime=4.54 cores=0-7
/proc/30773/task/30781/stat : 30781 (claspmt) R 30771 30773 30747 0 -1 4202560 6429 0 0 0 452 2 0 0 23 0 8 0 280838469 688173056 58472 33554432000 4194304 4964704 140736031753408 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37.84
Current children cumulated vsize (KiB) 672044

[startup+12.701 s]
/proc/loadavg: 1.77 0.39 0.13 9/286 30782
/proc/meminfo: memFree=8068624/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=672976 CPUtime=88.97 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 79180 0 0 0 8867 30 0 0 25 0 8 0 280838330 689127424 61736 33554432000 4194304 4964704 140736031753408 18446744073709551615 4668629 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/30773/statm: 168244 61736 464 189 0 162593 0
[pid=30773/tid=30775] ppid=30771 vsize=672976 CPUtime=10.93 cores=0-7
/proc/30773/task/30775/stat : 30775 (claspmt) R 30771 30773 30747 0 -1 4202560 5669 0 0 0 1090 3 0 0 25 0 8 0 280838469 689127424 61736 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670100 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30773/tid=30776] ppid=30771 vsize=672976 CPUtime=10.92 cores=0-7
/proc/30773/task/30776/stat : 30776 (claspmt) R 30771 30773 30747 0 -1 4202560 8110 0 0 0 1088 4 0 0 25 0 8 0 280838469 689127424 61736 33554432000 4194304 4964704 140736031753408 18446744073709551615 4668618 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30773/tid=30777] ppid=30771 vsize=672976 CPUtime=10.93 cores=0-7
/proc/30773/task/30777/stat : 30777 (claspmt) R 30771 30773 30747 0 -1 4202560 7693 0 0 0 1090 3 0 0 25 0 8 0 280838469 689127424 61736 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30773/tid=30778] ppid=30771 vsize=672976 CPUtime=10.91 cores=0-7
/proc/30773/task/30778/stat : 30778 (claspmt) R 30771 30773 30747 0 -1 4202560 6166 0 0 0 1088 3 0 0 25 0 8 0 280838469 689127424 61736 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30773/tid=30779] ppid=30771 vsize=672976 CPUtime=11.14 cores=0-7
/proc/30773/task/30779/stat : 30779 (claspmt) R 30771 30773 30747 0 -1 4202560 6268 0 0 0 1112 2 0 0 25 0 8 0 280838469 689127424 61736 33554432000 4194304 4964704 140736031753408 18446744073709551615 4342199 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30773/tid=30780] ppid=30771 vsize=672976 CPUtime=10.92 cores=0-7
/proc/30773/task/30780/stat : 30780 (claspmt) R 30771 30773 30747 0 -1 4202560 7441 0 0 0 1088 4 0 0 25 0 8 0 280838469 689127424 61736 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30773/tid=30781] ppid=30771 vsize=672976 CPUtime=10.94 cores=0-7
/proc/30773/task/30781/stat : 30781 (claspmt) R 30771 30773 30747 0 -1 4202560 6798 0 0 0 1091 3 0 0 25 0 8 0 280838469 689127424 61736 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 88.97
Current children cumulated vsize (KiB) 672976

[startup+25.5007 s]
/proc/loadavg: 2.73 0.64 0.21 9/286 30782
/proc/meminfo: memFree=8040640/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=674248 CPUtime=191.25 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 86493 0 0 0 19085 40 0 0 25 0 8 0 280838330 690429952 69049 33554432000 4194304 4964704 140736031753408 18446744073709551615 4668635 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/30773/statm: 168562 69049 464 189 0 162911 0
[pid=30773/tid=30775] ppid=30771 vsize=674248 CPUtime=23.73 cores=0-7
/proc/30773/task/30775/stat : 30775 (claspmt) R 30771 30773 30747 0 -1 4202560 5990 0 0 0 2370 3 0 0 25 0 8 0 280838469 690429952 69049 33554432000 4194304 4964704 140736031753408 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30773/tid=30776] ppid=30771 vsize=674248 CPUtime=23.71 cores=0-7
/proc/30773/task/30776/stat : 30776 (claspmt) R 30771 30773 30747 0 -1 4202560 10321 0 0 0 2364 7 0 0 25 0 8 0 280838469 690429952 69049 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670100 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30773/tid=30777] ppid=30771 vsize=674248 CPUtime=23.74 cores=0-7
/proc/30773/task/30777/stat : 30777 (claspmt) R 30771 30773 30747 0 -1 4202560 9389 0 0 0 2368 6 0 0 25 0 8 0 280838469 690429952 69049 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30773/tid=30778] ppid=30771 vsize=674248 CPUtime=23.71 cores=0-7
/proc/30773/task/30778/stat : 30778 (claspmt) R 30771 30773 30747 0 -1 4202560 7135 0 0 0 2367 4 0 0 25 0 8 0 280838469 690429952 69049 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30773/tid=30779] ppid=30771 vsize=674248 CPUtime=23.82 cores=0-7
/proc/30773/task/30779/stat : 30779 (claspmt) R 30771 30773 30747 0 -1 4202560 6346 0 0 0 2380 2 0 0 25 0 8 0 280838469 690429952 69049 33554432000 4194304 4964704 140736031753408 18446744073709551615 4669472 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30773/tid=30780] ppid=30771 vsize=674248 CPUtime=23.72 cores=0-7
/proc/30773/task/30780/stat : 30780 (claspmt) R 30771 30773 30747 0 -1 4202560 8483 0 0 0 2367 5 0 0 25 0 8 0 280838469 690429952 69049 33554432000 4194304 4964704 140736031753408 18446744073709551615 4669391 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30773/tid=30781] ppid=30771 vsize=674248 CPUtime=23.74 cores=0-7
/proc/30773/task/30781/stat : 30781 (claspmt) R 30771 30773 30747 0 -1 4202560 7733 0 0 0 2370 4 0 0 25 0 8 0 280838469 690429952 69049 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 191.25
Current children cumulated vsize (KiB) 674248

[startup+51.1012 s]
/proc/loadavg: 4.53 1.23 0.42 9/286 30783
/proc/meminfo: memFree=8000988/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=679156 CPUtime=395.81 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 96116 0 0 0 39528 53 0 0 25 0 8 0 280838330 695455744 78672 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/30773/statm: 169789 78672 466 189 0 164138 0
[pid=30773/tid=30775] ppid=30771 vsize=679156 CPUtime=49.33 cores=0-7
/proc/30773/task/30775/stat : 30775 (claspmt) R 30771 30773 30747 0 -1 4202560 6080 0 0 0 4930 3 0 0 25 0 8 0 280838469 695455744 78672 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30773/tid=30776] ppid=30771 vsize=679156 CPUtime=49.3 cores=0-7
/proc/30773/task/30776/stat : 30776 (claspmt) R 30771 30773 30747 0 -1 4202560 12627 0 0 0 4919 11 0 0 25 0 8 0 280838469 695455744 78672 33554432000 4194304 4964704 140736031753408 18446744073709551615 4668632 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30773/tid=30777] ppid=30771 vsize=679156 CPUtime=49.33 cores=0-7
/proc/30773/task/30777/stat : 30777 (claspmt) R 30771 30773 30747 0 -1 4202560 10337 0 0 0 4927 6 0 0 25 0 8 0 280838469 695455744 78672 33554432000 4194304 4964704 140736031753408 18446744073709551615 4668609 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30773/tid=30778] ppid=30771 vsize=679156 CPUtime=49.31 cores=0-7
/proc/30773/task/30778/stat : 30778 (claspmt) R 30771 30773 30747 0 -1 4202560 8640 0 0 0 4925 6 0 0 25 0 8 0 280838469 695455744 78672 33554432000 4194304 4964704 140736031753408 18446744073709551615 4668632 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30773/tid=30779] ppid=30771 vsize=679156 CPUtime=49.19 cores=0-7
/proc/30773/task/30779/stat : 30779 (claspmt) R 30771 30773 30747 0 -1 4202560 7123 0 0 0 4916 3 0 0 25 0 8 0 280838469 695455744 78672 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30773/tid=30780] ppid=30771 vsize=679156 CPUtime=49.32 cores=0-7
/proc/30773/task/30780/stat : 30780 (claspmt) R 30771 30773 30747 0 -1 4202560 9608 0 0 0 4926 6 0 0 25 0 8 0 280838469 695455744 78672 33554432000 4194304 4964704 140736031753408 18446744073709551615 4321044 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30773/tid=30781] ppid=30771 vsize=679156 CPUtime=49.34 cores=0-7
/proc/30773/task/30781/stat : 30781 (claspmt) R 30771 30773 30747 0 -1 4202560 9954 0 0 0 4926 8 0 0 25 0 8 0 280838469 695455744 78672 33554432000 4194304 4964704 140736031753408 18446744073709551615 4669309 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 395.81
Current children cumulated vsize (KiB) 679156

[startup+102.311 s]
/proc/loadavg: 6.61 2.37 0.85 9/286 30784
/proc/meminfo: memFree=7909208/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=690308 CPUtime=805 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 119077 0 0 0 80417 83 0 0 25 0 8 0 280838330 706875392 101545 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670215 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/30773/statm: 172577 101545 466 189 0 166926 0
[pid=30773/tid=30775] ppid=30771 vsize=690308 CPUtime=100.54 cores=0-7
/proc/30773/task/30775/stat : 30775 (claspmt) R 30771 30773 30747 0 -1 4202560 6588 0 0 0 10050 4 0 0 25 0 8 0 280838469 706875392 101545 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30773/tid=30776] ppid=30771 vsize=690308 CPUtime=100.51 cores=0-7
/proc/30773/task/30776/stat : 30776 (claspmt) R 30771 30773 30747 0 -1 4202560 15052 0 0 0 10036 15 0 0 25 0 8 0 280838469 706875392 101545 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670184 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30773/tid=30777] ppid=30771 vsize=690308 CPUtime=100.54 cores=0-7
/proc/30773/task/30777/stat : 30777 (claspmt) R 30771 30773 30747 0 -1 4202560 13343 0 0 0 10044 10 0 0 25 0 8 0 280838469 706875392 101545 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670357 0 0 4096 18946 0 0 0 -1 0 0 0 0

################
# More data... #
################

[pid=30773] ppid=30771 vsize=2745672 CPUtime=37721.5 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 632172 0 0 0 3771337 816 0 0 25 0 8 0 280838330 2811568128 613789 33554432000 4194304 4964704 140736031753408 18446744073709551615 4669270 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/30773/statm: 686418 613789 466 189 0 680767 0
[pid=30773/tid=30775] ppid=30771 vsize=2745672 CPUtime=4686.68 cores=0-7
/proc/30773/task/30775/stat : 30775 (claspmt) R 30771 30773 30747 0 -1 4202560 21718 0 0 0 468638 30 0 0 25 0 8 0 280838469 2811568128 613789 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30773/tid=30776] ppid=30771 vsize=2745672 CPUtime=4719.89 cores=0-7
/proc/30773/task/30776/stat : 30776 (claspmt) R 30771 30773 30747 0 -1 4202560 76107 0 0 0 471898 91 0 0 25 0 8 0 280838469 2811568128 613789 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30773/tid=30777] ppid=30771 vsize=2745672 CPUtime=4720.29 cores=0-7
/proc/30773/task/30777/stat : 30777 (claspmt) R 30771 30773 30747 0 -1 4202560 34129 0 0 0 471982 47 0 0 25 0 8 0 280838469 2811568128 613789 33554432000 4194304 4964704 140736031753408 18446744073709551615 4669477 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30773/tid=30778] ppid=30771 vsize=2745672 CPUtime=4719.87 cores=0-7
/proc/30773/task/30778/stat : 30778 (claspmt) R 30771 30773 30747 0 -1 4202560 106389 0 0 0 471846 141 0 0 25 0 8 0 280838469 2811568128 613789 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30773/tid=30779] ppid=30771 vsize=2745672 CPUtime=4711.7 cores=0-7
/proc/30773/task/30779/stat : 30779 (claspmt) R 30771 30773 30747 0 -1 4202560 55520 0 0 0 471097 73 0 0 25 0 8 0 280838469 2811568128 613789 33554432000 4194304 4964704 140736031753408 18446744073709551615 4669420 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30773/tid=30780] ppid=30771 vsize=2745672 CPUtime=4720.55 cores=0-7
/proc/30773/task/30780/stat : 30780 (claspmt) R 30771 30773 30747 0 -1 4202560 41869 0 0 0 472004 51 0 0 25 0 8 0 280838469 2811568128 613789 33554432000 4194304 4964704 140736031753408 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30773/tid=30781] ppid=30771 vsize=2745672 CPUtime=4720.62 cores=0-7
/proc/30773/task/30781/stat : 30781 (claspmt) R 30771 30773 30747 0 -1 4202560 244500 0 0 0 471719 343 0 0 25 0 8 0 280838469 2811568128 613789 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37721.5
Current children cumulated vsize (KiB) 2745672

[startup+4782.3 s]
/proc/loadavg: 8.00 8.00 7.93 9/286 30921
/proc/meminfo: memFree=5835516/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=2754912 CPUtime=38200.9 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 636941 0 0 0 3819267 823 0 0 25 0 8 0 280838330 2821029888 618558 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670100 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/30773/statm: 688728 618558 466 189 0 683077 0
[pid=30773/tid=30775] ppid=30771 vsize=2754912 CPUtime=4746.13 cores=0-7
/proc/30773/task/30775/stat : 30775 (claspmt) R 30771 30773 30747 0 -1 4202560 22007 0 0 0 474583 30 0 0 25 0 8 0 280838469 2821029888 618558 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30773/tid=30776] ppid=30771 vsize=2754912 CPUtime=4779.87 cores=0-7
/proc/30773/task/30776/stat : 30776 (claspmt) R 30771 30773 30747 0 -1 4202560 76108 0 0 0 477896 91 0 0 25 0 8 0 280838469 2821029888 618558 33554432000 4194304 4964704 140736031753408 18446744073709551615 4653154 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30773/tid=30777] ppid=30771 vsize=2754912 CPUtime=4780.27 cores=0-7
/proc/30773/task/30777/stat : 30777 (claspmt) R 30771 30773 30747 0 -1 4202560 36451 0 0 0 477977 50 0 0 25 0 8 0 280838469 2821029888 618558 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30773/tid=30778] ppid=30771 vsize=2754912 CPUtime=4779.86 cores=0-7
/proc/30773/task/30778/stat : 30778 (claspmt) R 30771 30773 30747 0 -1 4202560 106390 0 0 0 477845 141 0 0 25 0 8 0 280838469 2821029888 618558 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30773/tid=30779] ppid=30771 vsize=2754912 CPUtime=4771.69 cores=0-7
/proc/30773/task/30779/stat : 30779 (claspmt) R 30771 30773 30747 0 -1 4202560 57675 0 0 0 477094 75 0 0 25 0 8 0 280838469 2821029888 618558 33554432000 4194304 4964704 140736031753408 18446744073709551615 4668632 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30773/tid=30780] ppid=30771 vsize=2754912 CPUtime=4780.54 cores=0-7
/proc/30773/task/30780/stat : 30780 (claspmt) R 30771 30773 30747 0 -1 4202560 41869 0 0 0 478003 51 0 0 25 0 8 0 280838469 2821029888 618558 33554432000 4194304 4964704 140736031753408 18446744073709551615 4669467 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30773/tid=30781] ppid=30771 vsize=2754912 CPUtime=4780.61 cores=0-7
/proc/30773/task/30781/stat : 30781 (claspmt) R 30771 30773 30747 0 -1 4202560 244501 0 0 0 477718 343 0 0 25 0 8 0 280838469 2821029888 618558 33554432000 4194304 4964704 140736031753408 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38200.9
Current children cumulated vsize (KiB) 2754912

[startup+4842.3 s]
/proc/loadavg: 8.11 8.03 7.94 9/286 30922
/proc/meminfo: memFree=5823188/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=2765780 CPUtime=38680.3 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 640015 0 0 0 3867207 826 0 0 25 0 8 0 280838330 2832158720 621335 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670219 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/30773/statm: 691445 621335 466 189 0 685794 0
[pid=30773/tid=30775] ppid=30771 vsize=2765780 CPUtime=4805.56 cores=0-7
/proc/30773/task/30775/stat : 30775 (claspmt) R 30771 30773 30747 0 -1 4202560 22372 0 0 0 480525 31 0 0 25 0 8 0 280838469 2832158720 621335 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670215 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30773/tid=30776] ppid=30771 vsize=2765780 CPUtime=4839.87 cores=0-7
/proc/30773/task/30776/stat : 30776 (claspmt) R 30771 30773 30747 0 -1 4202560 76109 0 0 0 483896 91 0 0 25 0 8 0 280838469 2832158720 621335 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670458 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30773/tid=30777] ppid=30771 vsize=2765780 CPUtime=4840.27 cores=0-7
/proc/30773/task/30777/stat : 30777 (claspmt) R 30771 30773 30747 0 -1 4202560 36456 0 0 0 483977 50 0 0 25 0 8 0 280838469 2832158720 621335 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30773/tid=30778] ppid=30771 vsize=2765780 CPUtime=4839.86 cores=0-7
/proc/30773/task/30778/stat : 30778 (claspmt) R 30771 30773 30747 0 -1 4202560 106390 0 0 0 483845 141 0 0 25 0 8 0 280838469 2832158720 621335 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30773/tid=30779] ppid=30771 vsize=2765780 CPUtime=4831.69 cores=0-7
/proc/30773/task/30779/stat : 30779 (claspmt) R 30771 30773 30747 0 -1 4202560 60330 0 0 0 483091 78 0 0 25 0 8 0 280838469 2832158720 621335 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30773/tid=30780] ppid=30771 vsize=2765780 CPUtime=4840.54 cores=0-7
/proc/30773/task/30780/stat : 30780 (claspmt) R 30771 30773 30747 0 -1 4202560 41916 0 0 0 484003 51 0 0 25 0 8 0 280838469 2832158720 621335 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670263 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30773/tid=30781] ppid=30771 vsize=2765780 CPUtime=4840.61 cores=0-7
/proc/30773/task/30781/stat : 30781 (claspmt) R 30771 30773 30747 0 -1 4202560 244501 0 0 0 483718 343 0 0 25 0 8 0 280838469 2832158720 621335 33554432000 4194304 4964704 140736031753408 18446744073709551615 4668676 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38680.3
Current children cumulated vsize (KiB) 2765780

[startup+4902.3 s]
/proc/loadavg: 8.04 8.02 7.94 9/286 30924
/proc/meminfo: memFree=5786468/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=2802940 CPUtime=39159.8 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 649308 0 0 0 3915146 833 0 0 25 0 8 0 280838330 2870210560 630628 33554432000 4194304 4964704 140736031753408 18446744073709551615 4668904 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/30773/statm: 700735 630628 466 189 0 695084 0
[pid=30773/tid=30775] ppid=30771 vsize=2802940 CPUtime=4865.02 cores=0-7
/proc/30773/task/30775/stat : 30775 (claspmt) R 30771 30773 30747 0 -1 4202560 22372 0 0 0 486471 31 0 0 25 0 8 0 280838469 2870210560 630628 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30773/tid=30776] ppid=30771 vsize=2802940 CPUtime=4899.86 cores=0-7
/proc/30773/task/30776/stat : 30776 (claspmt) R 30771 30773 30747 0 -1 4202560 76146 0 0 0 489895 91 0 0 25 0 8 0 280838469 2870210560 630628 33554432000 4194304 4964704 140736031753408 18446744073709551615 4320911 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30773/tid=30777] ppid=30771 vsize=2802940 CPUtime=4900.27 cores=0-7
/proc/30773/task/30777/stat : 30777 (claspmt) R 30771 30773 30747 0 -1 4202560 36479 0 0 0 489977 50 0 0 25 0 8 0 280838469 2870210560 630628 33554432000 4194304 4964704 140736031753408 18446744073709551615 4361321 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30773/tid=30778] ppid=30771 vsize=2802940 CPUtime=4899.86 cores=0-7
/proc/30773/task/30778/stat : 30778 (claspmt) R 30771 30773 30747 0 -1 4202560 106391 0 0 0 489845 141 0 0 25 0 8 0 280838469 2870210560 630628 33554432000 4194304 4964704 140736031753408 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30773/tid=30779] ppid=30771 vsize=2802940 CPUtime=4891.69 cores=0-7
/proc/30773/task/30779/stat : 30779 (claspmt) R 30771 30773 30747 0 -1 4202560 60330 0 0 0 489091 78 0 0 25 0 8 0 280838469 2870210560 630628 33554432000 4194304 4964704 140736031753408 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30773/tid=30780] ppid=30771 vsize=2802940 CPUtime=4900.54 cores=0-7
/proc/30773/task/30780/stat : 30780 (claspmt) R 30771 30773 30747 0 -1 4202560 41976 0 0 0 490003 51 0 0 25 0 8 0 280838469 2870210560 630628 33554432000 4194304 4964704 140736031753408 18446744073709551615 268660359794 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30773/tid=30781] ppid=30771 vsize=2802940 CPUtime=4900.61 cores=0-7
/proc/30773/task/30781/stat : 30781 (claspmt) R 30771 30773 30747 0 -1 4202560 253673 0 0 0 489711 350 0 0 25 0 8 0 280838469 2870210560 630628 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39159.8
Current children cumulated vsize (KiB) 2802940

[startup+4962.3 s]
/proc/loadavg: 8.12 8.04 7.95 9/286 30926
/proc/meminfo: memFree=5732844/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=2913884 CPUtime=39639.2 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 662767 0 0 0 3963078 845 0 0 25 0 8 0 280838330 2983817216 644087 33554432000 4194304 4964704 140736031753408 18446744073709551615 4668693 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/30773/statm: 728471 644087 466 189 0 722820 0
[pid=30773/tid=30775] ppid=30771 vsize=2913884 CPUtime=4924.47 cores=0-7
/proc/30773/task/30775/stat : 30775 (claspmt) R 30771 30773 30747 0 -1 4202560 22375 0 0 0 492416 31 0 0 25 0 8 0 280838469 2983817216 644087 33554432000 4194304 4964704 140736031753408 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30773/tid=30776] ppid=30771 vsize=2913884 CPUtime=4959.84 cores=0-7
/proc/30773/task/30776/stat : 30776 (claspmt) R 30771 30773 30747 0 -1 4202560 78263 0 0 0 495891 93 0 0 25 0 8 0 280838469 2983817216 644087 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30773/tid=30777] ppid=30771 vsize=2913884 CPUtime=4960.27 cores=0-7
/proc/30773/task/30777/stat : 30777 (claspmt) R 30771 30773 30747 0 -1 4202560 36479 0 0 0 495977 50 0 0 25 0 8 0 280838469 2983817216 644087 33554432000 4194304 4964704 140736031753408 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30773/tid=30778] ppid=30771 vsize=2913884 CPUtime=4959.86 cores=0-7
/proc/30773/task/30778/stat : 30778 (claspmt) R 30771 30773 30747 0 -1 4202560 106391 0 0 0 495845 141 0 0 25 0 8 0 280838469 2983817216 644087 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30773/tid=30779] ppid=30771 vsize=2913884 CPUtime=4951.7 cores=0-7
/proc/30773/task/30779/stat : 30779 (claspmt) R 30771 30773 30747 0 -1 4202560 60330 0 0 0 495092 78 0 0 25 0 8 0 280838469 2983817216 644087 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670184 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30773/tid=30780] ppid=30771 vsize=2913884 CPUtime=4960.54 cores=0-7
/proc/30773/task/30780/stat : 30780 (claspmt) R 30771 30773 30747 0 -1 4202560 41979 0 0 0 496003 51 0 0 25 0 8 0 280838469 2983817216 644087 33554432000 4194304 4964704 140736031753408 18446744073709551615 4378499 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30773/tid=30781] ppid=30771 vsize=2913884 CPUtime=4960.61 cores=0-7
/proc/30773/task/30781/stat : 30781 (claspmt) R 30771 30773 30747 0 -1 4202560 265009 0 0 0 495702 359 0 0 25 0 8 0 280838469 2983817216 644087 33554432000 4194304 4964704 140736031753408 18446744073709551615 4669192 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39639.2
Current children cumulated vsize (KiB) 2913884



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5007.5 s]
/proc/loadavg: 8.06 8.04 7.95 9/286 30927
/proc/meminfo: memFree=5675280/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=2920068 CPUtime=40000.5 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 677017 0 0 0 3999186 863 0 0 25 0 8 0 280838330 2990149632 658337 33554432000 4194304 4964704 140736031753408 18446744073709551615 4669230 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/30773/statm: 730017 658337 466 189 0 724366 0
[pid=30773/tid=30775] ppid=30771 vsize=2920068 CPUtime=4969.26 cores=0-7
/proc/30773/task/30775/stat : 30775 (claspmt) R 30771 30773 30747 0 -1 4202560 22375 0 0 0 496895 31 0 0 25 0 8 0 280838469 2990149632 658337 33554432000 4194304 4964704 140736031753408 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30773/tid=30776] ppid=30771 vsize=2920068 CPUtime=5005.06 cores=0-7
/proc/30773/task/30776/stat : 30776 (claspmt) R 30771 30773 30747 0 -1 4202560 79715 0 0 0 500410 96 0 0 25 0 8 0 280838469 2990149632 658337 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30773/tid=30777] ppid=30771 vsize=2920068 CPUtime=5005.48 cores=0-7
/proc/30773/task/30777/stat : 30777 (claspmt) R 30771 30773 30747 0 -1 4202560 36479 0 0 0 500498 50 0 0 25 0 8 0 280838469 2990149632 658337 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30773/tid=30778] ppid=30771 vsize=2920068 CPUtime=5005.08 cores=0-7
/proc/30773/task/30778/stat : 30778 (claspmt) R 30771 30773 30747 0 -1 4202560 111493 0 0 0 500359 149 0 0 25 0 8 0 280838469 2990149632 658337 33554432000 4194304 4964704 140736031753408 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30773/tid=30779] ppid=30771 vsize=2920068 CPUtime=4996.91 cores=0-7
/proc/30773/task/30779/stat : 30779 (claspmt) R 30771 30773 30747 0 -1 4202560 60330 0 0 0 499613 78 0 0 25 0 8 0 280838469 2990149632 658337 33554432000 4194304 4964704 140736031753408 18446744073709551615 4669429 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30773/tid=30780] ppid=30771 vsize=2920068 CPUtime=5005.75 cores=0-7
/proc/30773/task/30780/stat : 30780 (claspmt) R 30771 30773 30747 0 -1 4202560 41979 0 0 0 500524 51 0 0 25 0 8 0 280838469 2990149632 658337 33554432000 4194304 4964704 140736031753408 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30773/tid=30781] ppid=30771 vsize=2920068 CPUtime=5005.83 cores=0-7
/proc/30773/task/30781/stat : 30781 (claspmt) R 30771 30773 30747 0 -1 4202560 272662 0 0 0 500215 368 0 0 25 0 8 0 280838469 2990149632 658337 33554432000 4194304 4964704 140736031753408 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 2920068

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5007.8 s]
/proc/loadavg: 8.06 8.04 7.95 9/286 30927
/proc/meminfo: memFree=5675280/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=2884108 CPUtime=40000.8 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 677029 0 0 0 3999219 863 0 0 25 0 1 0 280838330 2953326592 643252 33554432000 4194304 4964704 140736031753408 18446744073709551615 268660311169 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/30773/statm: 721027 643252 476 189 0 715376 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 2884108

[startup+5008.2 s]
/proc/loadavg: 8.06 8.04 7.95 2/280 30928
/proc/meminfo: memFree=5890716/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=2744332 CPUtime=40001.2 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 677029 0 0 0 3999260 864 0 0 25 0 1 0 280838330 2810195968 602511 33554432000 4194304 4964704 140736031753408 18446744073709551615 268660705639 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/30773/statm: 686083 602271 476 189 0 680432 0
Current children cumulated CPU time (s) 40001.2
Current children cumulated vsize (KiB) 2744332

[startup+5008.31 s]
/proc/loadavg: 8.06 8.04 7.95 2/280 30928
/proc/meminfo: memFree=5890716/32951124 swapFree=67111524/67111528
[pid=30773] ppid=30771 vsize=1110108 CPUtime=40001.3 cores=0-7
/proc/30773/stat : 30773 (claspmt) R 30771 30773 30747 0 -1 4202496 677029 0 0 0 3999263 871 0 0 25 0 1 0 280838330 1136750592 201789 33554432000 4194304 4964704 140736031753408 18446744073709551615 268660705639 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/30773/statm: 277527 201383 476 189 0 271876 0
Current children cumulated CPU time (s) 40001.3
Current children cumulated vsize (KiB) 1110108

Child status: 0
Real time (s): 5008.35
CPU time (s): 40001.4
CPU user time (s): 39992.6
CPU system time (s): 8.75267
CPU usage (%): 798.693
Max. virtual memory (cumulated for all children) (KiB): 2920068

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39992.6
system time used= 8.75267
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 677030
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 28506
involuntary context switches= 189528

runsolver used 13.036 second user time and 32.5081 second system time

The end

Launcher Data 

Begin job on node107 at 2011-05-02 22:01:11
IDJOB=3321672
IDBENCH=82468
IDSOLVER=1711
FILE ID=node107/3321672-1304366470
RUNJOBID= node107-1304366468-30761
PBS_JOBID= 13181763
Free space on /tmp= 72916 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT11/application/jarvisalo/HWMCC10-combination-timeframe-expansion/hwmcc10-timeframe-expansion-k45-pdtvisns3p02-tseitin.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321672-1304366470/watcher-3321672-1304366470 -o /tmp/evaluation-result-3321672-1304366470/solver-3321672-1304366470 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321672-1304366470.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= a0ec1baa890da62d0c6df7f1e62c6af3
RANDOM SEED=113861162

node107.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5359.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:       8312688 kB
Buffers:        307036 kB
Cached:        3150292 kB
SwapCached:          4 kB
Active:       20973224 kB
Inactive:      3276008 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:       8312688 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:            9488 kB
Writeback:           0 kB
AnonPages:    20791832 kB
Mapped:          24960 kB
Slab:           265996 kB
PageTables:      58100 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 54922448 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72916 MiB
End job on node107 at 2011-05-02 23:24:40