Trace number 3322787

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40001.2 5013.82

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m52-q918.75-n50-p15-s98191766.cnf
MD5SUM71638ad618721ad48c6bd2d48b828020
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.32195
Satisfiable
(Un)Satisfiability was proved
Number of variables2377
Number of clauses566749
Sum of the clauses size1135775
Maximum clause length52
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2566699
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 550

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...395557/instance-3322787-1304395557.cnf
5.19/5.23	c Solving...
40000.13/5012.82	
40000.13/5012.82	*** INTERRUPTED! ***
40000.13/5012.82	*** Shutting down threads...
40001.19/5013.81	s UNKNOWN
40001.19/5013.81	
40001.19/5013.81	c INTERRUPTED   : 1
40001.19/5013.81	c Models        : 0+    
40001.19/5013.81	c Time          : 5012.832s (Solving: 5007.60s 1st Model: 0.00s Unsat: 0.00s)
40001.19/5013.81	c CPU Time      : 40000.200s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3322787-1304395557/watcher-3322787-1304395557 -o /tmp/evaluation-result-3322787-1304395557/solver-3322787-1304395557 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322787-1304395557.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.50 2.30 2.23 2/351 30897
/proc/meminfo: memFree=4131644/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=23428 CPUtime=0.01 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 1185 0 0 0 1 0 0 0 23 0 1 0 629786423 23990272 1006 33554432000 4194304 4964704 140736224426272 18446744073709551615 251367243984 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/30897/statm: 5857 1020 386 189 0 731 0

[startup+0.064491 s]
/proc/loadavg: 5.50 2.30 2.23 2/351 30897
/proc/meminfo: memFree=4131644/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=32972 CPUtime=0.06 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 4670 0 0 0 6 0 0 0 23 0 1 0 629786423 33763328 3431 33554432000 4194304 4964704 140736224426272 18446744073709551615 4599599 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/30897/statm: 8243 3433 386 189 0 3117 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 32972

[startup+0.100475 s]
/proc/loadavg: 5.50 2.30 2.23 2/351 30897
/proc/meminfo: memFree=4131644/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=39468 CPUtime=0.09 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 6945 0 0 0 9 0 0 0 23 0 1 0 629786423 40415232 5098 33554432000 4194304 4964704 140736224426272 18446744073709551615 251367207621 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/30897/statm: 9867 5100 386 189 0 4741 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 39468

[startup+0.300446 s]
/proc/loadavg: 5.50 2.30 2.23 2/351 30897
/proc/meminfo: memFree=4131644/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=56928 CPUtime=0.28 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 11842 0 0 0 28 0 0 0 25 0 1 0 629786423 58294272 9061 33554432000 4194304 4964704 140736224426272 18446744073709551615 4607644 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/30897/statm: 14232 9061 399 189 0 9106 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 56928

[startup+0.700372 s]
/proc/loadavg: 5.50 2.30 2.23 2/351 30897
/proc/meminfo: memFree=4131644/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=56928 CPUtime=0.68 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 11842 0 0 0 68 0 0 0 25 0 1 0 629786423 58294272 9061 33554432000 4194304 4964704 140736224426272 18446744073709551615 4607585 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/30897/statm: 14232 9061 399 189 0 9106 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 56928

[startup+1.50022 s]
/proc/loadavg: 5.14 2.28 2.23 2/352 30898
/proc/meminfo: memFree=4096668/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=56928 CPUtime=1.48 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 11842 0 0 0 148 0 0 0 25 0 1 0 629786423 58294272 9061 33554432000 4194304 4964704 140736224426272 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/30897/statm: 14232 9061 399 189 0 9106 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 56928

[startup+3.10092 s]
/proc/loadavg: 5.14 2.28 2.23 2/352 30898
/proc/meminfo: memFree=4096060/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=56928 CPUtime=3.09 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 11843 0 0 0 309 0 0 0 25 0 1 0 629786423 58294272 9062 33554432000 4194304 4964704 140736224426272 18446744073709551615 4600389 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/30897/statm: 14232 9062 400 189 0 9106 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 56928

[startup+6.30125 s]
/proc/loadavg: 4.81 2.25 2.22 2/359 30906
/proc/meminfo: memFree=4092308/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=415380 CPUtime=10.51 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 21527 0 0 0 1039 12 0 0 16 0 8 0 629786423 425349120 17601 33554432000 4194304 4964704 140736224426272 18446744073709551615 4627338 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/30897/statm: 103845 17601 464 189 0 98194 0
[pid=30897/tid=30900] ppid=30895 vsize=415380 CPUtime=0.63 cores=0-7
/proc/30897/task/30900/stat : 30900 (claspmt) R 30895 30897 30664 0 -1 4202560 1187 0 0 0 63 0 0 0 16 0 8 0 629786947 425349120 17601 33554432000 4194304 4964704 140736224426272 18446744073709551615 4362287 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30897/tid=30901] ppid=30895 vsize=415380 CPUtime=0.62 cores=0-7
/proc/30897/task/30901/stat : 30901 (claspmt) R 30895 30897 30664 0 -1 4202560 1215 0 0 0 62 0 0 0 16 0 8 0 629786947 425349120 17601 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670269 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30897/tid=30902] ppid=30895 vsize=415380 CPUtime=0.59 cores=0-7
/proc/30897/task/30902/stat : 30902 (claspmt) R 30895 30897 30664 0 -1 4202560 1441 0 0 0 56 3 0 0 16 0 8 0 629786947 425349120 17602 33554432000 4194304 4964704 140736224426272 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30897/tid=30903] ppid=30895 vsize=415380 CPUtime=0.55 cores=0-7
/proc/30897/task/30903/stat : 30903 (claspmt) R 30895 30897 30664 0 -1 4202560 1003 0 0 0 55 0 0 0 16 0 8 0 629786947 425349120 17602 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30897/tid=30904] ppid=30895 vsize=415380 CPUtime=1.05 cores=0-7
/proc/30897/task/30904/stat : 30904 (claspmt) R 30895 30897 30664 0 -1 4202560 187 0 0 0 105 0 0 0 19 0 8 0 629786947 425349120 17602 33554432000 4194304 4964704 140736224426272 18446744073709551615 4399339 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30897/tid=30905] ppid=30895 vsize=415380 CPUtime=0.57 cores=0-7
/proc/30897/task/30905/stat : 30905 (claspmt) R 30895 30897 30664 0 -1 4202560 1311 0 0 0 55 2 0 0 16 0 8 0 629786947 425349120 17602 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30897/tid=30906] ppid=30895 vsize=415380 CPUtime=0.59 cores=0-7
/proc/30897/task/30906/stat : 30906 (claspmt) R 30895 30897 30664 0 -1 4202560 1278 0 0 0 59 0 0 0 16 0 8 0 629786947 425349120 17602 33554432000 4194304 4964704 140736224426272 18446744073709551615 4627248 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 10.51
Current children cumulated vsize (KiB) 415380

[startup+12.7011 s]
/proc/loadavg: 5.06 2.35 2.25 9/359 30906
/proc/meminfo: memFree=3936976/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=429524 CPUtime=61.62 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 57477 0 0 0 6128 34 0 0 25 0 8 0 629786423 439832576 53551 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/30897/statm: 107381 53552 466 189 0 101730 0
[pid=30897/tid=30900] ppid=30895 vsize=429524 CPUtime=6.95 cores=0-7
/proc/30897/task/30900/stat : 30900 (claspmt) R 30895 30897 30664 0 -1 4202560 4568 0 0 0 693 2 0 0 25 0 8 0 629786947 439832576 53552 33554432000 4194304 4964704 140736224426272 18446744073709551615 4627347 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30897/tid=30901] ppid=30895 vsize=429524 CPUtime=7.02 cores=0-7
/proc/30897/task/30901/stat : 30901 (claspmt) R 30895 30897 30664 0 -1 4202560 3836 0 0 0 700 2 0 0 25 0 8 0 629786947 439832576 53552 33554432000 4194304 4964704 140736224426272 18446744073709551615 4668820 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30897/tid=30902] ppid=30895 vsize=429524 CPUtime=6.98 cores=0-7
/proc/30897/task/30902/stat : 30902 (claspmt) R 30895 30897 30664 0 -1 4202560 6390 0 0 0 693 5 0 0 25 0 8 0 629786947 439832576 53552 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30897/tid=30903] ppid=30895 vsize=429524 CPUtime=6.95 cores=0-7
/proc/30897/task/30903/stat : 30903 (claspmt) R 30895 30897 30664 0 -1 4202560 6480 0 0 0 692 3 0 0 25 0 8 0 629786947 439832576 53552 33554432000 4194304 4964704 140736224426272 18446744073709551615 251367207969 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30897/tid=30904] ppid=30895 vsize=429524 CPUtime=7.45 cores=0-7
/proc/30897/task/30904/stat : 30904 (claspmt) R 30895 30897 30664 0 -1 4202560 5489 0 0 0 742 3 0 0 25 0 8 0 629786947 439832576 53552 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670250 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30897/tid=30905] ppid=30895 vsize=429524 CPUtime=6.97 cores=0-7
/proc/30897/task/30905/stat : 30905 (claspmt) R 30895 30897 30664 0 -1 4202560 4590 0 0 0 693 4 0 0 25 0 8 0 629786947 439832576 53552 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670241 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30897/tid=30906] ppid=30895 vsize=429524 CPUtime=6.99 cores=0-7
/proc/30897/task/30906/stat : 30906 (claspmt) R 30895 30897 30664 0 -1 4202560 8617 0 0 0 694 5 0 0 25 0 8 0 629786947 439832576 53552 33554432000 4194304 4964704 140736224426272 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 61.62
Current children cumulated vsize (KiB) 429524

[startup+25.5007 s]
/proc/loadavg: 5.71 2.62 2.34 9/359 30906
/proc/meminfo: memFree=3751112/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=523928 CPUtime=163.85 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 100180 0 0 0 16324 61 0 0 25 0 8 0 629786423 536502272 96254 33554432000 4194304 4964704 140736224426272 18446744073709551615 4332387 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/30897/statm: 130982 96254 466 189 0 125331 0
[pid=30897/tid=30900] ppid=30895 vsize=523928 CPUtime=19.59 cores=0-7
/proc/30897/task/30900/stat : 30900 (claspmt) R 30895 30897 30664 0 -1 4202560 9503 0 0 0 1953 6 0 0 25 0 8 0 629786947 536502272 96254 33554432000 4194304 4964704 140736224426272 18446744073709551615 4658370 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30897/tid=30901] ppid=30895 vsize=523928 CPUtime=19.82 cores=0-7
/proc/30897/task/30901/stat : 30901 (claspmt) R 30895 30897 30664 0 -1 4202560 6339 0 0 0 1979 3 0 0 25 0 8 0 629786947 536502272 96254 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30897/tid=30902] ppid=30895 vsize=523928 CPUtime=19.79 cores=0-7
/proc/30897/task/30902/stat : 30902 (claspmt) R 30895 30897 30664 0 -1 4202560 10665 0 0 0 1971 8 0 0 25 0 8 0 629786947 536502272 96254 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30897/tid=30903] ppid=30895 vsize=523928 CPUtime=19.75 cores=0-7
/proc/30897/task/30903/stat : 30903 (claspmt) R 30895 30897 30664 0 -1 4202560 11578 0 0 0 1969 6 0 0 25 0 8 0 629786947 536502272 96254 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30897/tid=30904] ppid=30895 vsize=523928 CPUtime=20.24 cores=0-7
/proc/30897/task/30904/stat : 30904 (claspmt) R 30895 30897 30664 0 -1 4202560 11774 0 0 0 2018 6 0 0 25 0 8 0 629786947 536502272 96254 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30897/tid=30905] ppid=30895 vsize=523928 CPUtime=19.77 cores=0-7
/proc/30897/task/30905/stat : 30905 (claspmt) R 30895 30897 30664 0 -1 4202560 9643 0 0 0 1970 7 0 0 25 0 8 0 629786947 536502272 96254 33554432000 4194304 4964704 140736224426272 18446744073709551615 4321010 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30897/tid=30906] ppid=30895 vsize=523928 CPUtime=19.79 cores=0-7
/proc/30897/task/30906/stat : 30906 (claspmt) R 30895 30897 30664 0 -1 4202560 17589 0 0 0 1967 12 0 0 25 0 8 0 629786947 536502272 96254 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670298 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 163.85
Current children cumulated vsize (KiB) 523928

[startup+51.101 s]
/proc/loadavg: 6.62 3.09 2.50 9/359 30907
/proc/meminfo: memFree=3529656/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=828872 CPUtime=368.34 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 155986 0 0 0 36734 100 0 0 25 0 8 0 629786423 848764928 152060 33554432000 4194304 4964704 140736224426272 18446744073709551615 4362289 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/30897/statm: 207218 152060 466 189 0 201567 0
[pid=30897/tid=30900] ppid=30895 vsize=828872 CPUtime=44.89 cores=0-7
/proc/30897/task/30900/stat : 30900 (claspmt) R 30895 30897 30664 0 -1 4202560 15923 0 0 0 4476 13 0 0 25 0 8 0 629786947 848764928 152060 33554432000 4194304 4964704 140736224426272 18446744073709551615 4669494 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30897/tid=30901] ppid=30895 vsize=828872 CPUtime=45.42 cores=0-7
/proc/30897/task/30901/stat : 30901 (claspmt) R 30895 30897 30664 0 -1 4202560 9972 0 0 0 4537 5 0 0 25 0 8 0 629786947 848764928 152060 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30897/tid=30902] ppid=30895 vsize=828872 CPUtime=45.37 cores=0-7
/proc/30897/task/30902/stat : 30902 (claspmt) R 30895 30897 30664 0 -1 4202560 16607 0 0 0 4526 11 0 0 25 0 8 0 629786947 848764928 152060 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30897/tid=30903] ppid=30895 vsize=828872 CPUtime=45.35 cores=0-7
/proc/30897/task/30903/stat : 30903 (claspmt) R 30895 30897 30664 0 -1 4202560 17206 0 0 0 4524 11 0 0 25 0 8 0 629786947 848764928 152060 33554432000 4194304 4964704 140736224426272 18446744073709551615 4321020 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30897/tid=30904] ppid=30895 vsize=828872 CPUtime=45.84 cores=0-7
/proc/30897/task/30904/stat : 30904 (claspmt) R 30895 30897 30664 0 -1 4202560 19143 0 0 0 4573 11 0 0 25 0 8 0 629786947 848764928 152060 33554432000 4194304 4964704 140736224426272 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30897/tid=30905] ppid=30895 vsize=828872 CPUtime=45.36 cores=0-7
/proc/30897/task/30905/stat : 30905 (claspmt) R 30895 30897 30664 0 -1 4202560 12862 0 0 0 4528 8 0 0 25 0 8 0 629786947 848764928 152060 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30897/tid=30906] ppid=30895 vsize=828872 CPUtime=45.39 cores=0-7
/proc/30897/task/30906/stat : 30906 (claspmt) R 30895 30897 30664 0 -1 4202560 27930 0 0 0 4521 18 0 0 25 0 8 0 629786947 848764928 152060 33554432000 4194304 4964704 140736224426272 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 368.34
Current children cumulated vsize (KiB) 828872

[startup+102.313 s]
/proc/loadavg: 7.40 3.84 2.79 9/359 30908
/proc/meminfo: memFree=3246412/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=1137296 CPUtime=777.4 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 225483 0 0 0 77589 151 0 0 25 0 8 0 629786423 1164591104 221557 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/30897/statm: 284324 221557 466 189 0 278673 0
[pid=30897/tid=30900] ppid=30895 vsize=1137296 CPUtime=95.49 cores=0-7
/proc/30897/task/30900/stat : 30900 (claspmt) R 30895 30897 30664 0 -1 4202560 25300 0 0 0 9528 21 0 0 25 0 8 0 629786947 1164591104 221557 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30897/tid=30901] ppid=30895 vsize=1137296 CPUtime=96.63 cores=0-7
/proc/30897/task/30901/stat : 30901 (claspmt) R 30895 30897 30664 0 -1 4202560 14277 0 0 0 9657 6 0 0 25 0 8 0 629786947 1164591104 221557 33554432000 4194304 4964704 140736224426272 18446744073709551615 4668816 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30897/tid=30902] ppid=30895 vsize=1137296 CPUtime=96.57 cores=0-7
/proc/30897/task/30902/stat : 30902 (claspmt) R 30895 30897 30664 0 -1 4202560 20781 0 0 0 9643 14 0 0 25 0 8 0 629786947 1164591104 221557 33554432000 4194304 4964704 140736224426272 18446744073709551615 4660132 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30897/tid=30903] ppid=30895 vsize=1137296 CPUtime=96.56 cores=0-7
/proc/30897/task/30903/stat : 30903 (claspmt) R 30895 30897 30664 0 -1 4202560 25431 0 0 0 9639 17 0 0 25 0 8 0 629786947 1164591104 221557 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670286 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30897/tid=30904] ppid=30895 vsize=1137296 CPUtime=97.06 cores=0-7
/proc/30897/task/30904/stat : 30904 (claspmt) R 30895 30897 30664 0 -1 4202560 24908 0 0 0 9691 15 0 0 25 0 8 0 629786947 1164591104 221557 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30897/tid=30905] ppid=30895 vsize=1137296 CPUtime=96.58 cores=0-7
/proc/30897/task/30905/stat : 30905 (claspmt) R 30895 30897 30664 0 -1 4202560 16680 0 0 0 9648 10 0 0 25 0 8 0 629786947 1164591104 221557 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670454 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30897/tid=30906] ppid=30895 vsize=1137296 CPUtime=96.6 cores=0-7
/proc/30897/task/30906/stat : 30906 (claspmt) R 30895 30897 30664 0 -1 4202560 46779 0 0 0 9626 34 0 0 25 0 8 0 629786947 1164591104 221557 33554432000 4194304 4964704 140736224426272 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 777.4
Current children cumulated vsize (KiB) 1137296

[startup+162.3 s]
/proc/loadavg: 7.78 4.60 3.11 9/359 30910
/proc/meminfo: memFree=3106628/32950928 swapFree=62390276/67111528

################
# More data... #
################

/proc/30897/task/30905/stat : 30905 (claspmt) R 30895 30897 30664 0 -1 4202560 60341 0 0 0 465578 55 0 0 25 0 8 0 629786947 3794362368 866829 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670241 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30897/tid=30906] ppid=30895 vsize=3705432 CPUtime=4628.15 cores=0-7
/proc/30897/task/30906/stat : 30906 (claspmt) R 30895 30897 30664 0 -1 4202560 185362 0 0 0 462580 235 0 0 25 0 8 0 629786947 3794362368 866829 33554432000 4194304 4964704 140736224426272 18446744073709551615 4365674 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 37200.2
Current children cumulated vsize (KiB) 3705432

[startup+4722.31 s]
/proc/loadavg: 8.08 8.06 7.99 9/359 31043
/proc/meminfo: memFree=650668/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=3708312 CPUtime=37679.6 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 876589 0 0 0 3767055 907 0 0 25 0 8 0 629786423 3797311488 868367 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/30897/statm: 927078 868367 466 189 0 921427 0
[pid=30897/tid=30900] ppid=30895 vsize=3708312 CPUtime=4688.62 cores=0-7
/proc/30897/task/30900/stat : 30900 (claspmt) R 30895 30897 30664 0 -1 4202560 69380 0 0 0 468788 74 0 0 25 0 8 0 629786947 3797311488 868367 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30897/tid=30901] ppid=30895 vsize=3708312 CPUtime=4716.69 cores=0-7
/proc/30897/task/30901/stat : 30901 (claspmt) R 30895 30897 30664 0 -1 4202560 87802 0 0 0 471630 39 0 0 25 0 8 0 629786947 3797311488 868367 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30897/tid=30902] ppid=30895 vsize=3708312 CPUtime=4715.4 cores=0-7
/proc/30897/task/30902/stat : 30902 (claspmt) R 30895 30897 30664 0 -1 4202560 62413 0 0 0 471486 54 0 0 25 0 8 0 629786947 3797311488 868367 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30897/tid=30903] ppid=30895 vsize=3708312 CPUtime=4716.5 cores=0-7
/proc/30897/task/30903/stat : 30903 (claspmt) R 30895 30897 30664 0 -1 4202560 70941 0 0 0 471581 69 0 0 25 0 8 0 629786947 3797311488 868367 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30897/tid=30904] ppid=30895 vsize=3708312 CPUtime=4716.93 cores=0-7
/proc/30897/task/30904/stat : 30904 (claspmt) R 30895 30897 30664 0 -1 4202560 73596 0 0 0 471621 72 0 0 25 0 8 0 629786947 3797311488 868367 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30897/tid=30905] ppid=30895 vsize=3708312 CPUtime=4716.34 cores=0-7
/proc/30897/task/30905/stat : 30905 (claspmt) R 30895 30897 30664 0 -1 4202560 60366 0 0 0 471579 55 0 0 25 0 8 0 629786947 3797311488 868367 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30897/tid=30906] ppid=30895 vsize=3708312 CPUtime=4687.44 cores=0-7
/proc/30897/task/30906/stat : 30906 (claspmt) R 30895 30897 30664 0 -1 4202560 185371 0 0 0 468509 235 0 0 22 0 8 0 629786947 3797311488 868367 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 37679.6
Current children cumulated vsize (KiB) 3708312

[startup+4782.3 s]
/proc/loadavg: 8.07 8.06 8.00 9/359 31045
/proc/meminfo: memFree=634552/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=3708816 CPUtime=38158.8 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 880982 0 0 0 3814969 911 0 0 25 0 8 0 629786423 3797827584 872760 33554432000 4194304 4964704 140736224426272 18446744073709551615 4332519 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/30897/statm: 927204 872760 466 189 0 921553 0
[pid=30897/tid=30900] ppid=30895 vsize=3708816 CPUtime=4748.61 cores=0-7
/proc/30897/task/30900/stat : 30900 (claspmt) R 30895 30897 30664 0 -1 4202560 69450 0 0 0 474787 74 0 0 25 0 8 0 629786947 3797827584 872760 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30897/tid=30901] ppid=30895 vsize=3708816 CPUtime=4776.67 cores=0-7
/proc/30897/task/30901/stat : 30901 (claspmt) R 30895 30897 30664 0 -1 4202560 89362 0 0 0 477628 39 0 0 25 0 8 0 629786947 3797827584 872760 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670291 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30897/tid=30902] ppid=30895 vsize=3708816 CPUtime=4775.38 cores=0-7
/proc/30897/task/30902/stat : 30902 (claspmt) R 30895 30897 30664 0 -1 4202560 62413 0 0 0 477484 54 0 0 25 0 8 0 629786947 3797827584 872760 33554432000 4194304 4964704 140736224426272 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30897/tid=30903] ppid=30895 vsize=3708816 CPUtime=4776.49 cores=0-7
/proc/30897/task/30903/stat : 30903 (claspmt) R 30895 30897 30664 0 -1 4202560 71178 0 0 0 477580 69 0 0 25 0 8 0 629786947 3797827584 872760 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30897/tid=30904] ppid=30895 vsize=3708816 CPUtime=4776.92 cores=0-7
/proc/30897/task/30904/stat : 30904 (claspmt) R 30895 30897 30664 0 -1 4202560 73596 0 0 0 477620 72 0 0 25 0 8 0 629786947 3797827584 872760 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30897/tid=30905] ppid=30895 vsize=3708816 CPUtime=4776.33 cores=0-7
/proc/30897/task/30905/stat : 30905 (claspmt) R 30895 30897 30664 0 -1 4202560 60425 0 0 0 477578 55 0 0 25 0 8 0 629786947 3797827584 872760 33554432000 4194304 4964704 140736224426272 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30897/tid=30906] ppid=30895 vsize=3708816 CPUtime=4746.72 cores=0-7
/proc/30897/task/30906/stat : 30906 (claspmt) R 30895 30897 30664 0 -1 4202560 185372 0 0 0 474437 235 0 0 25 0 8 0 629786947 3797827584 872760 33554432000 4194304 4964704 140736224426272 18446744073709551615 4320961 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38158.8
Current children cumulated vsize (KiB) 3708816

[startup+4842.3 s]
/proc/loadavg: 8.07 8.06 8.00 9/359 31046
/proc/meminfo: memFree=609152/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=3708816 CPUtime=38638.1 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 886909 0 0 0 3862889 920 0 0 25 0 8 0 629786423 3797827584 878687 33554432000 4194304 4964704 140736224426272 18446744073709551615 4320944 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/30897/statm: 927204 878687 466 189 0 921553 0
[pid=30897/tid=30900] ppid=30895 vsize=3708816 CPUtime=4808.61 cores=0-7
/proc/30897/task/30900/stat : 30900 (claspmt) R 30895 30897 30664 0 -1 4202560 69769 0 0 0 480786 75 0 0 25 0 8 0 629786947 3797827584 878687 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30897/tid=30901] ppid=30895 vsize=3708816 CPUtime=4836.68 cores=0-7
/proc/30897/task/30901/stat : 30901 (claspmt) R 30895 30897 30664 0 -1 4202560 90016 0 0 0 483628 40 0 0 25 0 8 0 629786947 3797827584 878687 33554432000 4194304 4964704 140736224426272 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30897/tid=30902] ppid=30895 vsize=3708816 CPUtime=4835.37 cores=0-7
/proc/30897/task/30902/stat : 30902 (claspmt) R 30895 30897 30664 0 -1 4202560 62443 0 0 0 483483 54 0 0 25 0 8 0 629786947 3797827584 878687 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30897/tid=30903] ppid=30895 vsize=3708816 CPUtime=4836.49 cores=0-7
/proc/30897/task/30903/stat : 30903 (claspmt) R 30895 30897 30664 0 -1 4202560 71220 0 0 0 483580 69 0 0 25 0 8 0 629786947 3797827584 878687 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30897/tid=30904] ppid=30895 vsize=3708816 CPUtime=4836.92 cores=0-7
/proc/30897/task/30904/stat : 30904 (claspmt) R 30895 30897 30664 0 -1 4202560 73650 0 0 0 483620 72 0 0 25 0 8 0 629786947 3797827584 878687 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670454 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30897/tid=30905] ppid=30895 vsize=3708816 CPUtime=4836.33 cores=0-7
/proc/30897/task/30905/stat : 30905 (claspmt) R 30895 30897 30664 0 -1 4202560 60449 0 0 0 483578 55 0 0 25 0 8 0 629786947 3797827584 878687 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30897/tid=30906] ppid=30895 vsize=3708816 CPUtime=4806.01 cores=0-7
/proc/30897/task/30906/stat : 30906 (claspmt) R 30895 30897 30664 0 -1 4202560 185372 0 0 0 480366 235 0 0 25 0 8 0 629786947 3797827584 878687 33554432000 4194304 4964704 140736224426272 18446744073709551615 4627252 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38638.1
Current children cumulated vsize (KiB) 3708816

[startup+4902.3 s]
/proc/loadavg: 8.06 8.06 8.00 9/359 31048
/proc/meminfo: memFree=601912/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=3711112 CPUtime=39117.4 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 888684 0 0 0 3910816 922 0 0 25 0 8 0 629786423 3800178688 880462 33554432000 4194304 4964704 140736224426272 18446744073709551615 4332525 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/30897/statm: 927778 880462 466 189 0 922127 0
[pid=30897/tid=30900] ppid=30895 vsize=3711112 CPUtime=4868.62 cores=0-7
/proc/30897/task/30900/stat : 30900 (claspmt) R 30895 30897 30664 0 -1 4202560 69912 0 0 0 486786 76 0 0 25 0 8 0 629786947 3800178688 880462 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30897/tid=30901] ppid=30895 vsize=3711112 CPUtime=4896.68 cores=0-7
/proc/30897/task/30901/stat : 30901 (claspmt) R 30895 30897 30664 0 -1 4202560 90623 0 0 0 489628 40 0 0 25 0 8 0 629786947 3800178688 880462 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30897/tid=30902] ppid=30895 vsize=3711112 CPUtime=4895.36 cores=0-7
/proc/30897/task/30902/stat : 30902 (claspmt) R 30895 30897 30664 0 -1 4202560 62443 0 0 0 489482 54 0 0 25 0 8 0 629786947 3800178688 880462 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30897/tid=30903] ppid=30895 vsize=3711112 CPUtime=4896.5 cores=0-7
/proc/30897/task/30903/stat : 30903 (claspmt) R 30895 30897 30664 0 -1 4202560 71610 0 0 0 489580 70 0 0 25 0 8 0 629786947 3800178688 880462 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30897/tid=30904] ppid=30895 vsize=3711112 CPUtime=4896.93 cores=0-7
/proc/30897/task/30904/stat : 30904 (claspmt) R 30895 30897 30664 0 -1 4202560 74181 0 0 0 489620 73 0 0 25 0 8 0 629786947 3800178688 880462 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30897/tid=30905] ppid=30895 vsize=3711112 CPUtime=4896.34 cores=0-7
/proc/30897/task/30905/stat : 30905 (claspmt) R 30895 30897 30664 0 -1 4202560 60545 0 0 0 489578 56 0 0 25 0 8 0 629786947 3800178688 880462 33554432000 4194304 4964704 140736224426272 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30897/tid=30906] ppid=30895 vsize=3711112 CPUtime=4865.3 cores=0-7
/proc/30897/task/30906/stat : 30906 (claspmt) R 30895 30897 30664 0 -1 4202560 185377 0 0 0 486295 235 0 0 25 0 8 0 629786947 3800178688 880462 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39117.4
Current children cumulated vsize (KiB) 3711112

[startup+4962.3 s]
/proc/loadavg: 8.06 8.06 8.00 9/359 31050
/proc/meminfo: memFree=587652/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=3720112 CPUtime=39596.7 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 892286 0 0 0 3958743 924 0 0 25 0 8 0 629786423 3809394688 883990 33554432000 4194304 4964704 140736224426272 18446744073709551615 4320890 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/30897/statm: 930028 883990 466 189 0 924377 0
[pid=30897/tid=30900] ppid=30895 vsize=3720112 CPUtime=4928.62 cores=0-7
/proc/30897/task/30900/stat : 30900 (claspmt) R 30895 30897 30664 0 -1 4202560 71518 0 0 0 492785 77 0 0 25 0 8 0 629786947 3809394688 883990 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30897/tid=30901] ppid=30895 vsize=3720112 CPUtime=4956.68 cores=0-7
/proc/30897/task/30901/stat : 30901 (claspmt) R 30895 30897 30664 0 -1 4202560 91474 0 0 0 495628 40 0 0 25 0 8 0 629786947 3809394688 883990 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30897/tid=30902] ppid=30895 vsize=3720112 CPUtime=4955.35 cores=0-7
/proc/30897/task/30902/stat : 30902 (claspmt) R 30895 30897 30664 0 -1 4202560 62544 0 0 0 495481 54 0 0 25 0 8 0 629786947 3809394688 883990 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30897/tid=30903] ppid=30895 vsize=3720112 CPUtime=4956.5 cores=0-7
/proc/30897/task/30903/stat : 30903 (claspmt) R 30895 30897 30664 0 -1 4202560 72356 0 0 0 495580 70 0 0 25 0 8 0 629786947 3809394688 883990 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30897/tid=30904] ppid=30895 vsize=3720112 CPUtime=4956.93 cores=0-7
/proc/30897/task/30904/stat : 30904 (claspmt) R 30895 30897 30664 0 -1 4202560 74476 0 0 0 495620 73 0 0 25 0 8 0 629786947 3809394688 883990 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30897/tid=30905] ppid=30895 vsize=3720112 CPUtime=4956.35 cores=0-7
/proc/30897/task/30905/stat : 30905 (claspmt) R 30895 30897 30664 0 -1 4202560 60545 0 0 0 495579 56 0 0 25 0 8 0 629786947 3809394688 883990 33554432000 4194304 4964704 140736224426272 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30897/tid=30906] ppid=30895 vsize=3720112 CPUtime=4924.59 cores=0-7
/proc/30897/task/30906/stat : 30906 (claspmt) R 30895 30897 30664 0 -1 4202560 185377 0 0 0 492224 235 0 0 25 0 8 0 629786947 3809394688 883990 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39596.7
Current children cumulated vsize (KiB) 3720112



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5012.8 s]
/proc/loadavg: 8.02 8.05 8.00 9/359 31051
/proc/meminfo: memFree=563896/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=3732964 CPUtime=40000.1 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 898266 0 0 0 3999080 933 0 0 25 0 8 0 629786423 3822555136 889895 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/30897/statm: 933241 889895 466 189 0 927590 0
[pid=30897/tid=30900] ppid=30895 vsize=3732964 CPUtime=4979.13 cores=0-7
/proc/30897/task/30900/stat : 30900 (claspmt) R 30895 30897 30664 0 -1 4202560 71634 0 0 0 497836 77 0 0 25 0 8 0 629786947 3822555136 889895 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30897/tid=30901] ppid=30895 vsize=3732964 CPUtime=5007.19 cores=0-7
/proc/30897/task/30901/stat : 30901 (claspmt) R 30895 30897 30664 0 -1 4202560 91849 0 0 0 500679 40 0 0 25 0 8 0 629786947 3822555136 889895 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30897/tid=30902] ppid=30895 vsize=3732964 CPUtime=5005.85 cores=0-7
/proc/30897/task/30902/stat : 30902 (claspmt) R 30895 30897 30664 0 -1 4202560 62780 0 0 0 500531 54 0 0 25 0 8 0 629786947 3822555136 889895 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30897/tid=30903] ppid=30895 vsize=3732964 CPUtime=5007.01 cores=0-7
/proc/30897/task/30903/stat : 30903 (claspmt) R 30895 30897 30664 0 -1 4202560 73057 0 0 0 500630 71 0 0 25 0 8 0 629786947 3822555136 889895 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670243 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30897/tid=30904] ppid=30895 vsize=3732964 CPUtime=5007.45 cores=0-7
/proc/30897/task/30904/stat : 30904 (claspmt) R 30895 30897 30664 0 -1 4202560 75491 0 0 0 500671 74 0 0 25 0 8 0 629786947 3822555136 889895 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30897/tid=30905] ppid=30895 vsize=3732964 CPUtime=5006.82 cores=0-7
/proc/30897/task/30905/stat : 30905 (claspmt) R 30895 30897 30664 0 -1 4202560 60546 0 0 0 500626 56 0 0 25 0 8 0 629786947 3822555136 889895 33554432000 4194304 4964704 140736224426272 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30897/tid=30906] ppid=30895 vsize=3732964 CPUtime=4974.5 cores=0-7
/proc/30897/task/30906/stat : 30906 (claspmt) R 30895 30897 30664 0 -1 4202560 187321 0 0 0 497210 240 0 0 25 0 8 0 629786947 3822555136 889895 33554432000 4194304 4964704 140736224426272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 3732964

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5013.4 s]
/proc/loadavg: 8.02 8.05 8.00 9/359 31051
/proc/meminfo: memFree=563896/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=3732664 CPUtime=40000.8 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 898276 0 0 0 3999144 933 0 0 25 0 1 0 629786423 3822247936 889427 33554432000 4194304 4964704 140736224426272 18446744073709551615 4313376 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/30897/statm: 933166 889427 476 189 0 927515 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 3732664

[startup+5013.81 s]
/proc/loadavg: 8.02 8.05 8.00 2/353 31052
/proc/meminfo: memFree=1199940/32950928 swapFree=62390276/67111528
[pid=30897] ppid=30895 vsize=415168 CPUtime=40001.2 cores=0-7
/proc/30897/stat : 30897 (claspmt) R 30895 30897 30664 0 -1 4202496 898276 0 0 0 3999172 947 0 0 25 0 1 0 629786423 425132032 22742 33554432000 4194304 4964704 140736224426272 18446744073709551615 251367590247 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/30897/statm: 103792 22267 476 189 0 98141 0
Current children cumulated CPU time (s) 40001.2
Current children cumulated vsize (KiB) 415168

Child status: 0
Real time (s): 5013.82
CPU time (s): 40001.2
CPU user time (s): 39991.7
CPU system time (s): 9.47456
CPU usage (%): 797.819
Max. virtual memory (cumulated for all children) (KiB): 3732964

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39991.7
system time used= 9.47456
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 898277
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 85428
involuntary context switches= 185561

runsolver used 15.9386 second user time and 43.3174 second system time

The end

Launcher Data 

Begin job on node113 at 2011-05-03 06:05:58
IDJOB=3322787
IDBENCH=88228
IDSOLVER=1711
FILE ID=node113/3322787-1304395557
RUNJOBID= node113-1304393293-30678
PBS_JOBID= 13181859
Free space on /tmp= 73512 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m52-q918.75-n50-p15-s98191766.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322787-1304395557/watcher-3322787-1304395557 -o /tmp/evaluation-result-3322787-1304395557/solver-3322787-1304395557 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322787-1304395557.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 71638ad618721ad48c6bd2d48b828020
RANDOM SEED=909120974

node113.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:       4131916 kB
Buffers:         64932 kB
Cached:         477788 kB
SwapCached:      53944 kB
Active:       17924136 kB
Inactive:     10645324 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:       4131916 kB
SwapTotal:    67111528 kB
SwapFree:     62390276 kB
Dirty:            8128 kB
Writeback:           0 kB
AnonPages:    27983468 kB
Mapped:          19560 kB
Slab:            93172 kB
PageTables:      91176 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 87039192 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73512 MiB
End job on node113 at 2011-05-03 07:29:32