Trace number 3381038

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40001.3 5010.95

General information on the benchmark

Namecrafted/skvortsov/automata-synchronization/
crn_40_1520_u.cnf
MD5SUMac02a3a5c06227748ecf75dd451ebb7e
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables63840
Number of clauses123900
Sum of the clauses size369320
Maximum clause length3
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 22380
Number of clauses of size 3121520
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...500947/instance-3381038-1305500947.cnf
0.59/0.61	c Solving...
40000.25/5010.02	
40000.25/5010.02	*** INTERRUPTED! ***
40000.25/5010.02	*** Shutting down threads...
40001.10/5010.90	s UNKNOWN
40001.10/5010.90	
40001.10/5010.90	c INTERRUPTED   : 1
40001.10/5010.90	c Models        : 0+    
40001.10/5010.90	c Time          : 5010.022s (Solving: 5009.41s 1st Model: 0.00s Unsat: 0.00s)
40001.10/5010.90	c CPU Time      : 40000.330s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3381038-1305500947/watcher-3381038-1305500947 -o /tmp/evaluation-result-3381038-1305500947/solver-3381038-1305500947 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381038-1305500947.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.61 7.90 7.96 1/412 25989
/proc/meminfo: memFree=18390716/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=32796 CPUtime=0 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 3476 0 0 0 0 0 0 0 25 0 1 0 740325203 33583104 3107 33554432000 4194304 4964800 140733252717808 18446744073709551615 4679151 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/25989/statm: 8199 3111 377 189 0 3073 0

[startup+0.0281999 s]
/proc/loadavg: 7.61 7.90 7.96 1/412 25989
/proc/meminfo: memFree=18390716/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=36036 CPUtime=0.02 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 4861 0 0 0 2 0 0 0 25 0 1 0 740325203 36900864 4121 33554432000 4194304 4964800 140733252717808 18446744073709551615 252457726661 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/25989/statm: 9009 4122 386 189 0 3883 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 36036

[startup+0.100187 s]
/proc/loadavg: 7.61 7.90 7.96 1/412 25989
/proc/meminfo: memFree=18390716/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=49568 CPUtime=0.08 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 9069 0 0 0 8 0 0 0 25 0 1 0 740325203 50757632 7404 33554432000 4194304 4964800 140733252717808 18446744073709551615 4618469 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/25989/statm: 12392 7404 400 189 0 7266 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 49568

[startup+0.30015 s]
/proc/loadavg: 7.61 7.90 7.96 1/412 25989
/proc/meminfo: memFree=18390716/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=54584 CPUtime=0.29 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 10354 0 0 0 28 1 0 0 25 0 1 0 740325203 55894016 8689 33554432000 4194304 4964800 140733252717808 18446744073709551615 4600392 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/25989/statm: 13646 8689 401 189 0 8520 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 54584

[startup+0.701113 s]
/proc/loadavg: 7.61 7.90 7.96 1/412 25989
/proc/meminfo: memFree=18390716/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=518676 CPUtime=0.78 cores=0-7
/proc/25989/stat : 25989 (claspmt) S 25987 25989 25928 0 -1 4202496 34412 0 0 0 66 12 0 0 25 0 8 0 740325203 531124224 28197 33554432000 4194304 4964800 140733252717808 18446744073709551615 252469881753 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/25989/statm: 129669 28210 433 189 0 124018 0
Current children cumulated CPU time (s) 0.78
Current children cumulated vsize (KiB) 518676

[startup+1.50089 s]
/proc/loadavg: 7.61 7.90 7.96 10/420 25997
/proc/meminfo: memFree=18250292/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=522024 CPUtime=6.74 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 44022 0 0 0 657 17 0 0 25 0 8 0 740325203 534552576 37741 33554432000 4194304 4964800 140733252717808 18446744073709551615 4371527 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25989/statm: 130506 37741 464 189 0 124855 0
[pid=25989/tid=25991] ppid=25987 vsize=522024 CPUtime=0.78 cores=0-7
/proc/25989/task/25991/stat : 25991 (claspmt) R 25987 25989 25928 0 -1 4202560 3590 0 0 0 76 2 0 0 18 0 8 0 740325265 534552576 37741 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25989/tid=25992] ppid=25987 vsize=522024 CPUtime=0.76 cores=0-7
/proc/25989/task/25992/stat : 25992 (claspmt) R 25987 25989 25928 0 -1 4202560 4787 0 0 0 74 2 0 0 18 0 8 0 740325265 534552576 37741 33554432000 4194304 4964800 140733252717808 18446744073709551615 4362982 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25989/tid=25993] ppid=25987 vsize=522024 CPUtime=0.76 cores=0-7
/proc/25989/task/25993/stat : 25993 (claspmt) R 25987 25989 25928 0 -1 4202560 5148 0 0 0 75 1 0 0 18 0 8 0 740325265 534552576 37741 33554432000 4194304 4964800 140733252717808 18446744073709551615 4660016 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25989/tid=25994] ppid=25987 vsize=522024 CPUtime=0.75 cores=0-7
/proc/25989/task/25994/stat : 25994 (claspmt) R 25987 25989 25928 0 -1 4202560 3870 0 0 0 74 1 0 0 18 0 8 0 740325265 534552576 37741 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25989/tid=25995] ppid=25987 vsize=522024 CPUtime=0.75 cores=0-7
/proc/25989/task/25995/stat : 25995 (claspmt) R 25987 25989 25928 0 -1 4202560 4424 0 0 0 73 2 0 0 18 0 8 0 740325265 534552576 37741 33554432000 4194304 4964800 140733252717808 18446744073709551615 4399202 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25989/tid=25996] ppid=25987 vsize=522024 CPUtime=0.78 cores=0-7
/proc/25989/task/25996/stat : 25996 (claspmt) R 25987 25989 25928 0 -1 4202560 4522 0 0 0 76 2 0 0 18 0 8 0 740325265 534552576 37741 33554432000 4194304 4964800 140733252717808 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25989/tid=25997] ppid=25987 vsize=522024 CPUtime=0.76 cores=0-7
/proc/25989/task/25997/stat : 25997 (claspmt) R 25987 25989 25928 0 -1 4202560 4405 0 0 0 74 2 0 0 18 0 8 0 740325265 534552576 37741 33554432000 4194304 4964800 140733252717808 18446744073709551615 4669509 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 6.74
Current children cumulated vsize (KiB) 522024

[startup+3.1006 s]
/proc/loadavg: 7.61 7.90 7.96 9/420 25997
/proc/meminfo: memFree=18226732/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=524852 CPUtime=19.51 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 50992 0 0 0 1931 20 0 0 25 0 8 0 740325203 537448448 44711 33554432000 4194304 4964800 140733252717808 18446744073709551615 4370295 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25989/statm: 131213 44711 464 189 0 125562 0
[pid=25989/tid=25991] ppid=25987 vsize=524852 CPUtime=2.38 cores=0-7
/proc/25989/task/25991/stat : 25991 (claspmt) R 25987 25989 25928 0 -1 4202560 3840 0 0 0 236 2 0 0 21 0 8 0 740325265 537448448 44711 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25989/tid=25992] ppid=25987 vsize=524852 CPUtime=2.36 cores=0-7
/proc/25989/task/25992/stat : 25992 (claspmt) R 25987 25989 25928 0 -1 4202560 6116 0 0 0 234 2 0 0 21 0 8 0 740325265 537448448 44711 33554432000 4194304 4964800 140733252717808 18446744073709551615 4342174 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25989/tid=25993] ppid=25987 vsize=524852 CPUtime=2.36 cores=0-7
/proc/25989/task/25993/stat : 25993 (claspmt) R 25987 25989 25928 0 -1 4202560 6347 0 0 0 234 2 0 0 21 0 8 0 740325265 537448448 44711 33554432000 4194304 4964800 140733252717808 18446744073709551615 4659670 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25989/tid=25994] ppid=25987 vsize=524852 CPUtime=2.35 cores=0-7
/proc/25989/task/25994/stat : 25994 (claspmt) R 25987 25989 25928 0 -1 4202560 4703 0 0 0 234 1 0 0 21 0 8 0 740325265 537448448 44711 33554432000 4194304 4964800 140733252717808 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25989/tid=25995] ppid=25987 vsize=524852 CPUtime=2.34 cores=0-7
/proc/25989/task/25995/stat : 25995 (claspmt) R 25987 25989 25928 0 -1 4202560 4630 0 0 0 232 2 0 0 22 0 8 0 740325265 537448448 44711 33554432000 4194304 4964800 140733252717808 18446744073709551615 4370236 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25989/tid=25996] ppid=25987 vsize=524852 CPUtime=2.38 cores=0-7
/proc/25989/task/25996/stat : 25996 (claspmt) R 25987 25989 25928 0 -1 4202560 6144 0 0 0 235 3 0 0 21 0 8 0 740325265 537448448 44711 33554432000 4194304 4964800 140733252717808 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25989/tid=25997] ppid=25987 vsize=524852 CPUtime=2.36 cores=0-7
/proc/25989/task/25997/stat : 25997 (claspmt) R 25987 25989 25928 0 -1 4202560 5544 0 0 0 233 3 0 0 21 0 8 0 740325265 537448448 44711 33554432000 4194304 4964800 140733252717808 18446744073709551615 4660303 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 19.51
Current children cumulated vsize (KiB) 524852

[startup+6.301 s]
/proc/loadavg: 7.65 7.90 7.96 9/420 25997
/proc/meminfo: memFree=18187100/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=528380 CPUtime=45.06 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 57916 0 0 0 4482 24 0 0 25 0 8 0 740325203 541061120 51605 33554432000 4194304 4964800 140733252717808 18446744073709551615 4658566 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25989/statm: 132095 51605 464 189 0 126444 0
[pid=25989/tid=25991] ppid=25987 vsize=528380 CPUtime=5.57 cores=0-7
/proc/25989/task/25991/stat : 25991 (claspmt) R 25987 25989 25928 0 -1 4202560 4522 0 0 0 555 2 0 0 25 0 8 0 740325265 541061120 51605 33554432000 4194304 4964800 140733252717808 18446744073709551615 4660155 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25989/tid=25992] ppid=25987 vsize=528380 CPUtime=5.55 cores=0-7
/proc/25989/task/25992/stat : 25992 (claspmt) R 25987 25989 25928 0 -1 4202560 7514 0 0 0 553 2 0 0 25 0 8 0 740325265 541061120 51605 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25989/tid=25993] ppid=25987 vsize=528380 CPUtime=5.57 cores=0-7
/proc/25989/task/25993/stat : 25993 (claspmt) R 25987 25989 25928 0 -1 4202560 7221 0 0 0 554 3 0 0 25 0 8 0 740325265 541061120 51605 33554432000 4194304 4964800 140733252717808 18446744073709551615 4668607 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25989/tid=25994] ppid=25987 vsize=528380 CPUtime=5.56 cores=0-7
/proc/25989/task/25994/stat : 25994 (claspmt) R 25987 25989 25928 0 -1 4202560 5013 0 0 0 554 2 0 0 25 0 8 0 740325265 541061120 51605 33554432000 4194304 4964800 140733252717808 18446744073709551615 4370264 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25989/tid=25995] ppid=25987 vsize=528380 CPUtime=5.5 cores=0-7
/proc/25989/task/25995/stat : 25995 (claspmt) R 25987 25989 25928 0 -1 4202560 5369 0 0 0 547 3 0 0 25 0 8 0 740325265 541061120 51605 33554432000 4194304 4964800 140733252717808 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25989/tid=25996] ppid=25987 vsize=528380 CPUtime=5.58 cores=0-7
/proc/25989/task/25996/stat : 25996 (claspmt) R 25987 25989 25928 0 -1 4202560 7415 0 0 0 554 4 0 0 25 0 8 0 740325265 541061120 51605 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25989/tid=25997] ppid=25987 vsize=528380 CPUtime=5.56 cores=0-7
/proc/25989/task/25997/stat : 25997 (claspmt) R 25987 25989 25928 0 -1 4202560 6773 0 0 0 553 3 0 0 25 0 8 0 740325265 541061120 51605 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 45.06
Current children cumulated vsize (KiB) 528380

[startup+12.7008 s]
/proc/loadavg: 7.67 7.91 7.96 9/420 25997
/proc/meminfo: memFree=18139892/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=534500 CPUtime=96.16 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 71142 0 0 0 9585 31 0 0 25 0 8 0 740325203 547328000 64799 33554432000 4194304 4964800 140733252717808 18446744073709551615 4370236 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25989/statm: 133625 64799 466 189 0 127974 0
[pid=25989/tid=25991] ppid=25987 vsize=534500 CPUtime=11.97 cores=0-7
/proc/25989/task/25991/stat : 25991 (claspmt) R 25987 25989 25928 0 -1 4202560 5118 0 0 0 1195 2 0 0 25 0 8 0 740325265 547328000 64799 33554432000 4194304 4964800 140733252717808 18446744073709551615 4370264 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25989/tid=25992] ppid=25987 vsize=534500 CPUtime=11.95 cores=0-7
/proc/25989/task/25992/stat : 25992 (claspmt) R 25987 25989 25928 0 -1 4202560 9887 0 0 0 1191 4 0 0 25 0 8 0 740325265 547328000 64799 33554432000 4194304 4964800 140733252717808 18446744073709551615 4321192 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25989/tid=25993] ppid=25987 vsize=534500 CPUtime=11.96 cores=0-7
/proc/25989/task/25993/stat : 25993 (claspmt) R 25987 25989 25928 0 -1 4202560 9547 0 0 0 1192 4 0 0 25 0 8 0 740325265 547328000 64799 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25989/tid=25994] ppid=25987 vsize=534500 CPUtime=11.95 cores=0-7
/proc/25989/task/25994/stat : 25994 (claspmt) R 25987 25989 25928 0 -1 4202560 5950 0 0 0 1193 2 0 0 25 0 8 0 740325265 547328000 64799 33554432000 4194304 4964800 140733252717808 18446744073709551615 4372782 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25989/tid=25995] ppid=25987 vsize=534500 CPUtime=11.8 cores=0-7
/proc/25989/task/25995/stat : 25995 (claspmt) R 25987 25989 25928 0 -1 4202560 6428 0 0 0 1177 3 0 0 25 0 8 0 740325265 547328000 64800 33554432000 4194304 4964800 140733252717808 18446744073709551615 4370266 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25989/tid=25996] ppid=25987 vsize=534500 CPUtime=11.98 cores=0-7
/proc/25989/task/25996/stat : 25996 (claspmt) R 25987 25989 25928 0 -1 4202560 9683 0 0 0 1193 5 0 0 18 0 8 0 740325265 547328000 64800 33554432000 4194304 4964800 140733252717808 18446744073709551615 4319632 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25989/tid=25997] ppid=25987 vsize=534500 CPUtime=11.96 cores=0-7
/proc/25989/task/25997/stat : 25997 (claspmt) R 25987 25989 25928 0 -1 4202560 9491 0 0 0 1192 4 0 0 25 0 8 0 740325265 547328000 64800 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 96.16
Current children cumulated vsize (KiB) 534500

[startup+25.5004 s]
/proc/loadavg: 7.75 7.91 7.96 9/420 25997
/proc/meminfo: memFree=18055316/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=551484 CPUtime=198.34 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 91604 0 0 0 19791 43 0 0 25 0 8 0 740325203 564719616 85261 33554432000 4194304 4964800 140733252717808 18446744073709551615 4371537 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25989/statm: 137871 85261 466 189 0 132220 0
[pid=25989/tid=25991] ppid=25987 vsize=551484 CPUtime=24.76 cores=0-7
/proc/25989/task/25991/stat : 25991 (claspmt) R 25987 25989 25928 0 -1 4202560 6191 0 0 0 2473 3 0 0 25 0 8 0 740325265 564719616 85261 33554432000 4194304 4964800 140733252717808 18446744073709551615 4669574 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25989/tid=25992] ppid=25987 vsize=551484 CPUtime=24.74 cores=0-7
/proc/25989/task/25992/stat : 25992 (claspmt) R 25987 25989 25928 0 -1 4202560 14198 0 0 0 2469 5 0 0 25 0 8 0 740325265 564719616 85261 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25989/tid=25993] ppid=25987 vsize=551484 CPUtime=24.75 cores=0-7
/proc/25989/task/25993/stat : 25993 (claspmt) R 25987 25989 25928 0 -1 4202560 12110 0 0 0 2470 5 0 0 25 0 8 0 740325265 564719616 85261 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25989/tid=25994] ppid=25987 vsize=551484 CPUtime=24.74 cores=0-7
/proc/25989/task/25994/stat : 25994 (claspmt) R 25987 25989 25928 0 -1 4202560 7914 0 0 0 2470 4 0 0 25 0 8 0 740325265 564719616 85261 33554432000 4194304 4964800 140733252717808 18446744073709551615 4370264 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25989/tid=25995] ppid=25987 vsize=551484 CPUtime=24.42 cores=0-7
/proc/25989/task/25995/stat : 25995 (claspmt) R 25987 25989 25928 0 -1 4202560 7963 0 0 0 2438 4 0 0 25 0 8 0 740325265 564719616 85261 33554432000 4194304 4964800 140733252717808 18446744073709551615 4370236 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25989/tid=25996] ppid=25987 vsize=551484 CPUtime=24.77 cores=0-7
/proc/25989/task/25996/stat : 25996 (claspmt) R 25987 25989 25928 0 -1 4202560 13682 0 0 0 2470 7 0 0 25 0 8 0 740325265 564719616 85261 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25989/tid=25997] ppid=25987 vsize=551484 CPUtime=24.75 cores=0-7
/proc/25989/task/25997/stat : 25997 (claspmt) R 25987 25989 25928 0 -1 4202560 13131 0 0 0 2468 7 0 0 25 0 8 0 740325265 564719616 85261 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670392 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 198.34
Current children cumulated vsize (KiB) 551484

[startup+51.1007 s]
/proc/loadavg: 7.89 7.93 7.96 9/420 25998
/proc/meminfo: memFree=17953360/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=563116 CPUtime=402.7 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 116813 0 0 0 40213 57 0 0 25 0 8 0 740325203 576630784 110256 33554432000 4194304 4964800 140733252717808 18446744073709551615 4370229 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25989/statm: 140779 110256 466 189 0 135128 0
[pid=25989/tid=25991] ppid=25987 vsize=563116 CPUtime=50.34 cores=0-7
/proc/25989/task/25991/stat : 25991 (claspmt) R 25987 25989 25928 0 -1 4202560 7845 0 0 0 5030 4 0 0 25 0 8 0 740325265 576630784 110256 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670437 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25989/tid=25992] ppid=25987 vsize=563116 CPUtime=50.34 cores=0-7
/proc/25989/task/25992/stat : 25992 (claspmt) R 25987 25989 25928 0 -1 4202560 17823 0 0 0 5026 8 0 0 25 0 8 0 740325265 576630784 110256 33554432000 4194304 4964800 140733252717808 18446744073709551615 4653692 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25989/tid=25993] ppid=25987 vsize=563116 CPUtime=50.35 cores=0-7
/proc/25989/task/25993/stat : 25993 (claspmt) R 25987 25989 25928 0 -1 4202560 16405 0 0 0 5028 7 0 0 25 0 8 0 740325265 576630784 110256 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670330 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25989/tid=25994] ppid=25987 vsize=563116 CPUtime=50.33 cores=0-7
/proc/25989/task/25994/stat : 25994 (claspmt) R 25987 25989 25928 0 -1 4202560 10044 0 0 0 5027 6 0 0 25 0 8 0 740325265 576630784 110256 33554432000 4194304 4964800 140733252717808 18446744073709551615 4370264 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25989/tid=25995] ppid=25987 vsize=563116 CPUtime=49.67 cores=0-7
/proc/25989/task/25995/stat : 25995 (claspmt) R 25987 25989 25928 0 -1 4202560 10504 0 0 0 4960 7 0 0 25 0 8 0 740325265 576630784 110256 33554432000 4194304 4964800 140733252717808 18446744073709551615 4658122 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25989/tid=25996] ppid=25987 vsize=563116 CPUtime=50.36 cores=0-7
/proc/25989/task/25996/stat : 25996 (claspmt) R 25987 25989 25928 0 -1 4202560 17161 0 0 0 5027 9 0 0 25 0 8 0 740325265 576630784 110256 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25989/tid=25997] ppid=25987 vsize=563116 CPUtime=50.34 cores=0-7
/proc/25989/task/25997/stat : 25997 (claspmt) R 25987 25989 25928 0 -1 4202560 18226 0 0 0 5025 9 0 0 25 0 8 0 740325265 576630784 110256 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 402.7

################
# More data... #
################

[pid=25989/tid=25994] ppid=25987 vsize=4672648 CPUtime=4718.19 cores=0-7
/proc/25989/task/25994/stat : 25994 (claspmt) R 25987 25989 25928 0 -1 4202560 103825 0 0 0 471746 73 0 0 25 0 8 0 740325265 4784791552 1069159 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25989/tid=25995] ppid=25987 vsize=4672648 CPUtime=4717.48 cores=0-7
/proc/25989/task/25995/stat : 25995 (claspmt) R 25987 25989 25928 0 -1 4202560 118422 0 0 0 471675 73 0 0 25 0 8 0 740325265 4784791552 1069159 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670264 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25989/tid=25996] ppid=25987 vsize=4672648 CPUtime=4715.45 cores=0-7
/proc/25989/task/25996/stat : 25996 (claspmt) R 25987 25989 25928 0 -1 4202560 151664 0 0 0 471436 109 0 0 25 0 8 0 740325265 4784791552 1069159 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25989/tid=25997] ppid=25987 vsize=4672648 CPUtime=4721.12 cores=0-7
/proc/25989/task/25997/stat : 25997 (claspmt) R 25987 25989 25928 0 -1 4202560 198984 0 0 0 471971 141 0 0 25 0 8 0 740325265 4784791552 1069159 33554432000 4194304 4964800 140733252717808 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37702.7
Current children cumulated vsize (KiB) 4672648

[startup+4782.3 s]
/proc/loadavg: 8.05 8.08 8.02 9/420 26136
/proc/meminfo: memFree=14090976/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=4677836 CPUtime=38181.8 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 1109302 0 0 0 3817391 784 0 0 25 0 8 0 740325203 4790104064 1073320 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/25989/statm: 1169459 1073320 466 189 0 1163808 0
[pid=25989/tid=25991] ppid=25987 vsize=4677836 CPUtime=4744.84 cores=0-7
/proc/25989/task/25991/stat : 25991 (claspmt) R 25987 25989 25928 0 -1 4202560 85529 0 0 0 474414 70 0 0 25 0 8 0 740325265 4790104064 1073320 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25989/tid=25992] ppid=25987 vsize=4677836 CPUtime=4771.99 cores=0-7
/proc/25989/task/25992/stat : 25992 (claspmt) R 25987 25989 25928 0 -1 4202560 160413 0 0 0 477107 92 0 0 25 0 8 0 740325265 4790104064 1073320 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25989/tid=25993] ppid=25987 vsize=4677836 CPUtime=4776.44 cores=0-7
/proc/25989/task/25993/stat : 25993 (claspmt) R 25987 25989 25928 0 -1 4202560 167079 0 0 0 477535 109 0 0 25 0 8 0 740325265 4790104064 1073320 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25989/tid=25994] ppid=25987 vsize=4677836 CPUtime=4778.17 cores=0-7
/proc/25989/task/25994/stat : 25994 (claspmt) R 25987 25989 25928 0 -1 4202560 105300 0 0 0 477744 73 0 0 25 0 8 0 740325265 4790104064 1073320 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25989/tid=25995] ppid=25987 vsize=4677836 CPUtime=4777.46 cores=0-7
/proc/25989/task/25995/stat : 25995 (claspmt) R 25987 25989 25928 0 -1 4202560 118730 0 0 0 477672 74 0 0 25 0 8 0 740325265 4790104064 1073320 33554432000 4194304 4964800 140733252717808 18446744073709551615 4370264 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25989/tid=25996] ppid=25987 vsize=4677836 CPUtime=4775.44 cores=0-7
/proc/25989/task/25996/stat : 25996 (claspmt) R 25987 25989 25928 0 -1 4202560 152731 0 0 0 477434 110 0 0 25 0 8 0 740325265 4790104064 1073320 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25989/tid=25997] ppid=25987 vsize=4677836 CPUtime=4781.1 cores=0-7
/proc/25989/task/25997/stat : 25997 (claspmt) R 25987 25989 25928 0 -1 4202560 199612 0 0 0 477969 141 0 0 25 0 8 0 740325265 4790104064 1073320 33554432000 4194304 4964800 140733252717808 18446744073709551615 4320916 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38181.8
Current children cumulated vsize (KiB) 4677836

[startup+4842.3 s]
/proc/loadavg: 8.02 8.06 8.01 9/420 26138
/proc/meminfo: memFree=14074720/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=4681932 CPUtime=38660.9 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 1113620 0 0 0 3865300 789 0 0 25 0 8 0 740325203 4794298368 1077383 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/25989/statm: 1170483 1077383 466 189 0 1164832 0
[pid=25989/tid=25991] ppid=25987 vsize=4681932 CPUtime=4804 cores=0-7
/proc/25989/task/25991/stat : 25991 (claspmt) R 25987 25989 25928 0 -1 4202560 86212 0 0 0 480330 70 0 0 25 0 8 0 740325265 4794298368 1077383 33554432000 4194304 4964800 140733252717808 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25989/tid=25992] ppid=25987 vsize=4681932 CPUtime=4831.99 cores=0-7
/proc/25989/task/25992/stat : 25992 (claspmt) R 25987 25989 25928 0 -1 4202560 160488 0 0 0 483107 92 0 0 25 0 8 0 740325265 4794298368 1077383 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25989/tid=25993] ppid=25987 vsize=4681932 CPUtime=4836.43 cores=0-7
/proc/25989/task/25993/stat : 25993 (claspmt) R 25987 25989 25928 0 -1 4202560 167748 0 0 0 483534 109 0 0 25 0 8 0 740325265 4794298368 1077383 33554432000 4194304 4964800 140733252717808 18446744073709551615 4660157 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25989/tid=25994] ppid=25987 vsize=4681932 CPUtime=4838.17 cores=0-7
/proc/25989/task/25994/stat : 25994 (claspmt) R 25987 25989 25928 0 -1 4202560 105832 0 0 0 483743 74 0 0 25 0 8 0 740325265 4794298368 1077383 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25989/tid=25995] ppid=25987 vsize=4681932 CPUtime=4837.45 cores=0-7
/proc/25989/task/25995/stat : 25995 (claspmt) R 25987 25989 25928 0 -1 4202560 118760 0 0 0 483671 74 0 0 25 0 8 0 740325265 4794298368 1077383 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25989/tid=25996] ppid=25987 vsize=4681932 CPUtime=4835.44 cores=0-7
/proc/25989/task/25996/stat : 25996 (claspmt) R 25987 25989 25928 0 -1 4202560 154231 0 0 0 483432 112 0 0 25 0 8 0 740325265 4794298368 1077383 33554432000 4194304 4964800 140733252717808 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25989/tid=25997] ppid=25987 vsize=4681932 CPUtime=4841.1 cores=0-7
/proc/25989/task/25997/stat : 25997 (claspmt) R 25987 25989 25928 0 -1 4202560 200439 0 0 0 483968 142 0 0 25 0 8 0 740325265 4794298368 1077383 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38660.9
Current children cumulated vsize (KiB) 4681932

[startup+4902.3 s]
/proc/loadavg: 8.04 8.06 8.01 9/420 26139
/proc/meminfo: memFree=14068252/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=4681932 CPUtime=39140.1 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 1115421 0 0 0 3913214 791 0 0 25 0 8 0 740325203 4794298368 1078814 33554432000 4194304 4964800 140733252717808 18446744073709551615 4669350 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/25989/statm: 1170483 1078814 466 189 0 1164832 0
[pid=25989/tid=25991] ppid=25987 vsize=4681932 CPUtime=4863.17 cores=0-7
/proc/25989/task/25991/stat : 25991 (claspmt) R 25987 25989 25928 0 -1 4202560 86511 0 0 0 486247 70 0 0 25 0 8 0 740325265 4794298368 1078814 33554432000 4194304 4964800 140733252717808 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25989/tid=25992] ppid=25987 vsize=4681932 CPUtime=4891.98 cores=0-7
/proc/25989/task/25992/stat : 25992 (claspmt) R 25987 25989 25928 0 -1 4202560 161014 0 0 0 489106 92 0 0 25 0 8 0 740325265 4794298368 1078814 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25989/tid=25993] ppid=25987 vsize=4681932 CPUtime=4896.44 cores=0-7
/proc/25989/task/25993/stat : 25993 (claspmt) R 25987 25989 25928 0 -1 4202560 168255 0 0 0 489534 110 0 0 25 0 8 0 740325265 4794298368 1078814 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25989/tid=25994] ppid=25987 vsize=4681932 CPUtime=4898.17 cores=0-7
/proc/25989/task/25994/stat : 25994 (claspmt) R 25987 25989 25928 0 -1 4202560 106042 0 0 0 489743 74 0 0 25 0 8 0 740325265 4794298368 1078814 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25989/tid=25995] ppid=25987 vsize=4681932 CPUtime=4897.43 cores=0-7
/proc/25989/task/25995/stat : 25995 (claspmt) R 25987 25989 25928 0 -1 4202560 118764 0 0 0 489669 74 0 0 25 0 8 0 740325265 4794298368 1078814 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25989/tid=25996] ppid=25987 vsize=4681932 CPUtime=4895.44 cores=0-7
/proc/25989/task/25996/stat : 25996 (claspmt) R 25987 25989 25928 0 -1 4202560 154250 0 0 0 489432 112 0 0 25 0 8 0 740325265 4794298368 1078814 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25989/tid=25997] ppid=25987 vsize=4681932 CPUtime=4901.1 cores=0-7
/proc/25989/task/25997/stat : 25997 (claspmt) R 25987 25989 25928 0 -1 4202560 200578 0 0 0 489968 142 0 0 25 0 8 0 740325265 4794298368 1078814 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39140.1
Current children cumulated vsize (KiB) 4681932

[startup+4962.3 s]
/proc/loadavg: 8.05 8.06 8.01 9/420 26141
/proc/meminfo: memFree=14059424/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=4684528 CPUtime=39619.2 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 1117955 0 0 0 3961129 792 0 0 25 0 8 0 740325203 4796956672 1081106 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/25989/statm: 1171132 1081106 466 189 0 1165481 0
[pid=25989/tid=25991] ppid=25987 vsize=4684528 CPUtime=4922.35 cores=0-7
/proc/25989/task/25991/stat : 25991 (claspmt) R 25987 25989 25928 0 -1 4202560 86789 0 0 0 492164 71 0 0 25 0 8 0 740325265 4796956672 1081106 33554432000 4194304 4964800 140733252717808 18446744073709551615 4370229 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25989/tid=25992] ppid=25987 vsize=4684528 CPUtime=4951.99 cores=0-7
/proc/25989/task/25992/stat : 25992 (claspmt) R 25987 25989 25928 0 -1 4202560 161348 0 0 0 495107 92 0 0 25 0 8 0 740325265 4796956672 1081106 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25989/tid=25993] ppid=25987 vsize=4684528 CPUtime=4956.43 cores=0-7
/proc/25989/task/25993/stat : 25993 (claspmt) R 25987 25989 25928 0 -1 4202560 168531 0 0 0 495533 110 0 0 25 0 8 0 740325265 4796956672 1081106 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25989/tid=25994] ppid=25987 vsize=4684528 CPUtime=4958.17 cores=0-7
/proc/25989/task/25994/stat : 25994 (claspmt) R 25987 25989 25928 0 -1 4202560 106431 0 0 0 495743 74 0 0 25 0 8 0 740325265 4796956672 1081106 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25989/tid=25995] ppid=25987 vsize=4684528 CPUtime=4957.42 cores=0-7
/proc/25989/task/25995/stat : 25995 (claspmt) R 25987 25989 25928 0 -1 4202560 118905 0 0 0 495668 74 0 0 25 0 8 0 740325265 4796956672 1081106 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25989/tid=25996] ppid=25987 vsize=4684528 CPUtime=4955.43 cores=0-7
/proc/25989/task/25996/stat : 25996 (claspmt) R 25987 25989 25928 0 -1 4202560 155077 0 0 0 495431 112 0 0 25 0 8 0 740325265 4796956672 1081106 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25989/tid=25997] ppid=25987 vsize=4684528 CPUtime=4961.1 cores=0-7
/proc/25989/task/25997/stat : 25997 (claspmt) R 25987 25989 25928 0 -1 4202560 200650 0 0 0 495968 142 0 0 25 0 8 0 740325265 4796956672 1081106 33554432000 4194304 4964800 140733252717808 18446744073709551615 4366624 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39619.2
Current children cumulated vsize (KiB) 4684528



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010 s]
/proc/loadavg: 8.06 8.06 8.01 9/420 26142
/proc/meminfo: memFree=14050112/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=4693268 CPUtime=40000.2 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 1120732 0 0 0 3999231 794 0 0 25 0 8 0 740325203 4805906432 1083394 33554432000 4194304 4964800 140733252717808 18446744073709551615 4668737 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/25989/statm: 1173317 1083394 466 189 0 1167666 0
[pid=25989/tid=25991] ppid=25987 vsize=4693268 CPUtime=4969.4 cores=0-7
/proc/25989/task/25991/stat : 25991 (claspmt) R 25987 25989 25928 0 -1 4202560 87717 0 0 0 496869 71 0 0 25 0 8 0 740325265 4805906432 1083394 33554432000 4194304 4964800 140733252717808 18446744073709551615 4321010 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25989/tid=25992] ppid=25987 vsize=4693268 CPUtime=4999.7 cores=0-7
/proc/25989/task/25992/stat : 25992 (claspmt) R 25987 25989 25928 0 -1 4202560 161642 0 0 0 499878 92 0 0 25 0 8 0 740325265 4805906432 1083394 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25989/tid=25993] ppid=25987 vsize=4693268 CPUtime=5004.15 cores=0-7
/proc/25989/task/25993/stat : 25993 (claspmt) R 25987 25989 25928 0 -1 4202560 169096 0 0 0 500305 110 0 0 25 0 8 0 740325265 4805906432 1083394 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25989/tid=25994] ppid=25987 vsize=4693268 CPUtime=5005.89 cores=0-7
/proc/25989/task/25994/stat : 25994 (claspmt) R 25987 25989 25928 0 -1 4202560 106942 0 0 0 500514 75 0 0 25 0 8 0 740325265 4805906432 1083394 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25989/tid=25995] ppid=25987 vsize=4693268 CPUtime=5005.13 cores=0-7
/proc/25989/task/25995/stat : 25995 (claspmt) R 25987 25989 25928 0 -1 4202560 119036 0 0 0 500439 74 0 0 25 0 8 0 740325265 4805906432 1083394 33554432000 4194304 4964800 140733252717808 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25989/tid=25996] ppid=25987 vsize=4693268 CPUtime=5003.14 cores=0-7
/proc/25989/task/25996/stat : 25996 (claspmt) R 25987 25989 25928 0 -1 4202560 155377 0 0 0 500202 112 0 0 25 0 8 0 740325265 4805906432 1083394 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25989/tid=25997] ppid=25987 vsize=4693268 CPUtime=5008.82 cores=0-7
/proc/25989/task/25997/stat : 25997 (claspmt) R 25987 25989 25928 0 -1 4202560 200675 0 0 0 500740 142 0 0 25 0 8 0 740325265 4805906432 1083394 33554432000 4194304 4964800 140733252717808 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40000.2
Current children cumulated vsize (KiB) 4693268

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.2 s]
/proc/loadavg: 8.06 8.06 8.01 9/420 26142
/proc/meminfo: memFree=14050112/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=4679292 CPUtime=40000.5 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 1120743 0 0 0 3999256 794 0 0 25 0 1 0 740325203 4791595008 1079382 33554432000 4194304 4964800 140733252717808 18446744073709551615 252457720955 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/25989/statm: 1169823 1079382 476 189 0 1164172 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 4679292

[startup+5010.6 s]
/proc/loadavg: 8.06 8.06 8.01 2/414 26143
/proc/meminfo: memFree=14071488/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=4663008 CPUtime=40000.9 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 1120743 0 0 0 3999296 794 0 0 25 0 1 0 740325203 4774920192 1074607 33554432000 4194304 4964800 140733252717808 18446744073709551615 252457721035 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/25989/statm: 1165752 1074607 476 189 0 1160101 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 4663008

[startup+5010.8 s]
/proc/loadavg: 8.06 8.06 8.01 2/414 26143
/proc/meminfo: memFree=14071488/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=3273464 CPUtime=40001.1 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202496 1120743 0 0 0 3999310 800 0 0 25 0 1 0 740325203 3352027136 720125 33554432000 4194304 4964800 140733252717808 18446744073709551615 252458109287 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/25989/statm: 818366 720125 476 189 0 812715 0
Current children cumulated CPU time (s) 40001.1
Current children cumulated vsize (KiB) 3273464

[startup+5010.9 s]
/proc/loadavg: 8.06 8.06 8.01 2/414 26143
/proc/meminfo: memFree=14071488/32951124 swapFree=57414320/67111528
[pid=25989] ppid=25987 vsize=0 CPUtime=40001.2 cores=0-7
/proc/25989/stat : 25989 (claspmt) R 25987 25989 25928 0 -1 4202500 1120744 0 0 0 3999310 810 0 0 25 0 1 0 740325203 0 0 33554432000 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/25989/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 40001.2
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 5010.95
CPU time (s): 40001.3
CPU user time (s): 39993.1
CPU system time (s): 8.14876
CPU usage (%): 798.277
Max. virtual memory (cumulated for all children) (KiB): 4693268

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39993.1
system time used= 8.14876
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1120744
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 172405
involuntary context switches= 131752

runsolver used 18.7002 second user time and 49.6505 second system time

The end

Launcher Data 

Begin job on node111 at 2011-05-16 01:09:07
IDJOB=3381038
IDBENCH=83198
IDSOLVER=1865
FILE ID=node111/3381038-1305500947
RUNJOBID= node111-1305500920-25942
PBS_JOBID= 13325463
Free space on /tmp= 73628 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/crafted/skvortsov/automata-synchronization/crn_40_1520_u.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3381038-1305500947/watcher-3381038-1305500947 -o /tmp/evaluation-result-3381038-1305500947/solver-3381038-1305500947 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381038-1305500947.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= ac02a3a5c06227748ecf75dd451ebb7e
RANDOM SEED=860268428

node111.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5428.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      18390624 kB
Buffers:        303512 kB
Cached:        6041160 kB
SwapCached:      19184 kB
Active:        6945148 kB
Inactive:      7353288 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      18390624 kB
SwapTotal:    67111528 kB
SwapFree:     57414320 kB
Dirty:            3836 kB
Writeback:           4 kB
AnonPages:     7944584 kB
Mapped:          24956 kB
Slab:           131992 kB
PageTables:      59956 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 124886132 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73628 MiB
End job on node111 at 2011-05-16 02:32:39