Trace number 3321495

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40001.4 5006.61

General information on the benchmark

NameCRAFTED/rbsat/random/
forced/rbsat-v2640c305320gyes5.cnf
MD5SUMf892a2bda6fe6aec6f2fcb4e1c541df8
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2640
Number of clauses305320
Sum of the clauses size613120
Maximum clause length33
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2305240
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 580

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...343802/instance-3321495-1304343802.cnf
1.69/1.71	c Solving...
40000.35/5005.61	
40000.35/5005.61	*** INTERRUPTED! ***
40000.35/5005.61	*** Shutting down threads...
40001.39/5006.60	s UNKNOWN
40001.39/5006.60	
40001.39/5006.60	c INTERRUPTED   : 1
40001.39/5006.60	c Models        : 0+    
40001.39/5006.60	c Time          : 5005.622s (Solving: 5003.91s 1st Model: 0.00s Unsat: 0.00s)
40001.39/5006.60	c CPU Time      : 40000.410s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321495-1304343802/watcher-3321495-1304343802 -o /tmp/evaluation-result-3321495-1304343802/solver-3321495-1304343802 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321495-1304343802.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.44 7.83 7.38 2/175 27503
/proc/meminfo: memFree=28302312/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=21872 CPUtime=0 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 759 0 0 0 0 0 0 0 22 0 1 0 545980964 22396928 669 33554432000 4194304 4964704 140736615323392 18446744073709551615 222921776896 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/27503/statm: 5468 669 386 189 0 342 0

[startup+0.0933511 s]
/proc/loadavg: 7.44 7.83 7.38 2/175 27503
/proc/meminfo: memFree=28302312/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=37468 CPUtime=0.08 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 5839 0 0 0 8 0 0 0 22 0 1 0 545980964 38367232 4575 33554432000 4194304 4964704 140736615323392 18446744073709551615 222921437893 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/27503/statm: 9367 4578 397 189 0 4241 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 37468

[startup+0.100355 s]
/proc/loadavg: 7.44 7.83 7.38 2/175 27503
/proc/meminfo: memFree=28302312/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=39028 CPUtime=0.09 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 6177 0 0 0 9 0 0 0 22 0 1 0 545980964 39964672 4913 33554432000 4194304 4964704 140736615323392 18446744073709551615 4618467 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/27503/statm: 9757 4916 397 189 0 4631 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 39028

[startup+0.300314 s]
/proc/loadavg: 7.44 7.83 7.38 2/175 27503
/proc/meminfo: memFree=28302312/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=39820 CPUtime=0.29 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 6439 0 0 0 29 0 0 0 23 0 1 0 545980964 40775680 5175 33554432000 4194304 4964704 140736615323392 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/27503/statm: 9955 5175 399 189 0 4829 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 39820

[startup+0.700238 s]
/proc/loadavg: 7.44 7.83 7.38 2/175 27503
/proc/meminfo: memFree=28302312/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=39820 CPUtime=0.69 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 6439 0 0 0 69 0 0 0 25 0 1 0 545980964 40775680 5175 33554432000 4194304 4964704 140736615323392 18446744073709551615 4607644 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/27503/statm: 9955 5175 399 189 0 4829 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 39820

[startup+1.50109 s]
/proc/loadavg: 7.44 7.83 7.38 2/176 27504
/proc/meminfo: memFree=28282592/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=43120 CPUtime=1.49 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 7253 0 0 0 149 0 0 0 25 0 1 0 545980964 44154880 5989 33554432000 4194304 4964704 140736615323392 18446744073709551615 4600360 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/27503/statm: 10780 5989 400 189 0 5654 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 43120

[startup+3.10077 s]
/proc/loadavg: 7.44 7.83 7.38 10/183 27511
/proc/meminfo: memFree=28266168/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=401300 CPUtime=11.06 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 19824 0 0 0 1097 9 0 0 19 0 8 0 545980964 410931200 17920 33554432000 4194304 4964704 140736615323392 18446744073709551615 4627248 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/27503/statm: 100325 17920 464 189 0 94674 0
[pid=27503/tid=27505] ppid=27501 vsize=401300 CPUtime=1.14 cores=0-7
/proc/27503/task/27505/stat : 27505 (claspmt) R 27501 27503 27252 0 -1 4202560 1110 0 0 0 114 0 0 0 17 0 8 0 545981135 410931200 17920 33554432000 4194304 4964704 140736615323392 18446744073709551615 4654520 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=27503/tid=27506] ppid=27501 vsize=401300 CPUtime=1.15 cores=0-7
/proc/27503/task/27506/stat : 27506 (claspmt) R 27501 27503 27252 0 -1 4202560 1781 0 0 0 114 1 0 0 17 0 8 0 545981135 410931200 17920 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=27503/tid=27507] ppid=27501 vsize=401300 CPUtime=1.13 cores=0-7
/proc/27503/task/27507/stat : 27507 (claspmt) R 27501 27503 27252 0 -1 4202560 1477 0 0 0 113 0 0 0 19 0 8 0 545981135 410931200 17920 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=27503/tid=27508] ppid=27501 vsize=401300 CPUtime=1.14 cores=0-7
/proc/27503/task/27508/stat : 27508 (claspmt) R 27501 27503 27252 0 -1 4202560 1530 0 0 0 114 0 0 0 17 0 8 0 545981135 410931200 17920 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27503/tid=27509] ppid=27501 vsize=401300 CPUtime=1.37 cores=0-7
/proc/27503/task/27509/stat : 27509 (claspmt) R 27501 27503 27252 0 -1 4202560 1053 0 0 0 137 0 0 0 19 0 8 0 545981135 410931200 17920 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670450 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=27503/tid=27510] ppid=27501 vsize=401300 CPUtime=1.14 cores=0-7
/proc/27503/task/27510/stat : 27510 (claspmt) R 27501 27503 27252 0 -1 4202560 1775 0 0 0 113 1 0 0 17 0 8 0 545981135 410931200 17920 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27503/tid=27511] ppid=27501 vsize=401300 CPUtime=1.14 cores=0-7
/proc/27503/task/27511/stat : 27511 (claspmt) R 27501 27503 27252 0 -1 4202560 2470 0 0 0 114 0 0 0 17 0 8 0 545981135 410931200 17920 33554432000 4194304 4964704 140736615323392 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 11.06
Current children cumulated vsize (KiB) 401300

[startup+6.30118 s]
/proc/loadavg: 7.48 7.83 7.38 9/183 27511
/proc/meminfo: memFree=28186572/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=409368 CPUtime=36.64 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 35335 0 0 0 3646 18 0 0 25 0 8 0 545980964 419192832 33431 33554432000 4194304 4964704 140736615323392 18446744073709551615 4362264 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/27503/statm: 102342 33431 464 189 0 96691 0
[pid=27503/tid=27505] ppid=27501 vsize=409368 CPUtime=4.35 cores=0-7
/proc/27503/task/27505/stat : 27505 (claspmt) R 27501 27503 27252 0 -1 4202560 2728 0 0 0 433 2 0 0 23 0 8 0 545981135 419192832 33431 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=27503/tid=27506] ppid=27501 vsize=409368 CPUtime=4.35 cores=0-7
/proc/27503/task/27506/stat : 27506 (claspmt) R 27501 27503 27252 0 -1 4202560 4087 0 0 0 433 2 0 0 23 0 8 0 545981135 419192832 33431 33554432000 4194304 4964704 140736615323392 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=27503/tid=27507] ppid=27501 vsize=409368 CPUtime=4.31 cores=0-7
/proc/27503/task/27507/stat : 27507 (claspmt) R 27501 27503 27252 0 -1 4202560 3181 0 0 0 429 2 0 0 25 0 8 0 545981135 419192832 33431 33554432000 4194304 4964704 140736615323392 18446744073709551615 4342174 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=27503/tid=27508] ppid=27501 vsize=409368 CPUtime=4.35 cores=0-7
/proc/27503/task/27508/stat : 27508 (claspmt) R 27501 27503 27252 0 -1 4202560 3464 0 0 0 433 2 0 0 23 0 8 0 545981135 419192832 33431 33554432000 4194304 4964704 140736615323392 18446744073709551615 222921438241 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27503/tid=27509] ppid=27501 vsize=409368 CPUtime=4.57 cores=0-7
/proc/27503/task/27509/stat : 27509 (claspmt) R 27501 27503 27252 0 -1 4202560 1338 0 0 0 457 0 0 0 25 0 8 0 545981135 419192832 33431 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670458 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=27503/tid=27510] ppid=27501 vsize=409368 CPUtime=4.35 cores=0-7
/proc/27503/task/27510/stat : 27510 (claspmt) R 27501 27503 27252 0 -1 4202560 3347 0 0 0 432 3 0 0 23 0 8 0 545981135 419192832 33431 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670281 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27503/tid=27511] ppid=27501 vsize=409368 CPUtime=4.35 cores=0-7
/proc/27503/task/27511/stat : 27511 (claspmt) R 27501 27503 27252 0 -1 4202560 7122 0 0 0 433 2 0 0 23 0 8 0 545981135 419192832 33432 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670560 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 36.64
Current children cumulated vsize (KiB) 409368

[startup+12.701 s]
/proc/loadavg: 7.53 7.83 7.38 9/183 27512
/proc/meminfo: memFree=28076664/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=428868 CPUtime=87.8 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 62376 0 0 0 8748 32 0 0 25 0 8 0 545980964 439160832 60472 33554432000 4194304 4964704 140736615323392 18446744073709551615 4320944 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/27503/statm: 107217 60472 466 189 0 101566 0
[pid=27503/tid=27505] ppid=27501 vsize=428868 CPUtime=10.74 cores=0-7
/proc/27503/task/27505/stat : 27505 (claspmt) R 27501 27503 27252 0 -1 4202560 6198 0 0 0 1070 4 0 0 25 0 8 0 545981135 439160832 60472 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670215 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=27503/tid=27506] ppid=27501 vsize=428868 CPUtime=10.75 cores=0-7
/proc/27503/task/27506/stat : 27506 (claspmt) R 27501 27503 27252 0 -1 4202560 7342 0 0 0 1071 4 0 0 25 0 8 0 545981135 439160832 60473 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=27503/tid=27507] ppid=27501 vsize=428868 CPUtime=10.67 cores=0-7
/proc/27503/task/27507/stat : 27507 (claspmt) R 27501 27503 27252 0 -1 4202560 5854 0 0 0 1064 3 0 0 25 0 8 0 545981135 439160832 60473 33554432000 4194304 4964704 140736615323392 18446744073709551615 4660050 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=27503/tid=27508] ppid=27501 vsize=428868 CPUtime=10.74 cores=0-7
/proc/27503/task/27508/stat : 27508 (claspmt) R 27501 27503 27252 0 -1 4202560 6276 0 0 0 1071 3 0 0 25 0 8 0 545981135 439160832 60473 33554432000 4194304 4964704 140736615323392 18446744073709551615 4668907 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27503/tid=27509] ppid=27501 vsize=428868 CPUtime=10.97 cores=0-7
/proc/27503/task/27509/stat : 27509 (claspmt) R 27501 27503 27252 0 -1 4202560 3695 0 0 0 1095 2 0 0 25 0 8 0 545981135 439160832 60473 33554432000 4194304 4964704 140736615323392 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=27503/tid=27510] ppid=27501 vsize=428868 CPUtime=10.74 cores=0-7
/proc/27503/task/27510/stat : 27510 (claspmt) R 27501 27503 27252 0 -1 4202560 5165 0 0 0 1071 3 0 0 25 0 8 0 545981135 439160832 60473 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670440 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27503/tid=27511] ppid=27501 vsize=428868 CPUtime=10.75 cores=0-7
/proc/27503/task/27511/stat : 27511 (claspmt) R 27501 27503 27252 0 -1 4202560 15501 0 0 0 1069 6 0 0 25 0 8 0 545981135 439160832 60473 33554432000 4194304 4964704 140736615323392 18446744073709551615 222921438241 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 87.8
Current children cumulated vsize (KiB) 428868

[startup+25.5007 s]
/proc/loadavg: 7.63 7.84 7.39 9/183 27512
/proc/meminfo: memFree=27970072/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=444104 CPUtime=190.11 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 86190 0 0 0 18965 46 0 0 25 0 8 0 545980964 454762496 84286 33554432000 4194304 4964704 140736615323392 18446744073709551615 4627248 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/27503/statm: 111026 84286 466 189 0 105375 0
[pid=27503/tid=27505] ppid=27501 vsize=444104 CPUtime=23.51 cores=0-7
/proc/27503/task/27505/stat : 27505 (claspmt) R 27501 27503 27252 0 -1 4202560 8856 0 0 0 2345 6 0 0 25 0 8 0 545981135 454762496 84286 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=27503/tid=27506] ppid=27501 vsize=444104 CPUtime=23.55 cores=0-7
/proc/27503/task/27506/stat : 27506 (claspmt) R 27501 27503 27252 0 -1 4202560 10062 0 0 0 2349 6 0 0 25 0 8 0 545981135 454762496 84286 33554432000 4194304 4964704 140736615323392 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=27503/tid=27507] ppid=27501 vsize=444104 CPUtime=23.43 cores=0-7
/proc/27503/task/27507/stat : 27507 (claspmt) R 27501 27503 27252 0 -1 4202560 7861 0 0 0 2338 5 0 0 21 0 8 0 545981135 454762496 84286 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=27503/tid=27508] ppid=27501 vsize=444104 CPUtime=23.53 cores=0-7
/proc/27503/task/27508/stat : 27508 (claspmt) R 27501 27503 27252 0 -1 4202560 8519 0 0 0 2349 4 0 0 25 0 8 0 545981135 454762496 84286 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670277 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27503/tid=27509] ppid=27501 vsize=444104 CPUtime=23.77 cores=0-7
/proc/27503/task/27509/stat : 27509 (claspmt) R 27501 27503 27252 0 -1 4202560 5582 0 0 0 2374 3 0 0 25 0 8 0 545981135 454762496 84286 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=27503/tid=27510] ppid=27501 vsize=444104 CPUtime=23.54 cores=0-7
/proc/27503/task/27510/stat : 27510 (claspmt) R 27501 27503 27252 0 -1 4202560 6428 0 0 0 2350 4 0 0 25 0 8 0 545981135 454762496 84286 33554432000 4194304 4964704 140736615323392 18446744073709551615 4342232 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27503/tid=27511] ppid=27501 vsize=444104 CPUtime=23.54 cores=0-7
/proc/27503/task/27511/stat : 27511 (claspmt) R 27501 27503 27252 0 -1 4202560 24482 0 0 0 2344 10 0 0 25 0 8 0 545981135 454762496 84286 33554432000 4194304 4964704 140736615323392 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 190.11
Current children cumulated vsize (KiB) 444104

[startup+51.101 s]
/proc/loadavg: 7.75 7.85 7.41 9/183 27513
/proc/meminfo: memFree=27912944/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=583000 CPUtime=394.73 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 100960 0 0 0 39417 56 0 0 25 0 8 0 545980964 596992000 99051 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/27503/statm: 145750 99051 466 189 0 140099 0
[pid=27503/tid=27505] ppid=27501 vsize=583000 CPUtime=48.95 cores=0-7
/proc/27503/task/27505/stat : 27505 (claspmt) R 27501 27503 27252 0 -1 4202560 11250 0 0 0 4888 7 0 0 25 0 8 0 545981135 596992000 99051 33554432000 4194304 4964704 140736615323392 18446744073709551615 4362295 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=27503/tid=27506] ppid=27501 vsize=583000 CPUtime=49.15 cores=0-7
/proc/27503/task/27506/stat : 27506 (claspmt) R 27501 27503 27252 0 -1 4202560 11591 0 0 0 4908 7 0 0 25 0 8 0 545981135 596992000 99051 33554432000 4194304 4964704 140736615323392 18446744073709551615 4668964 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=27503/tid=27507] ppid=27501 vsize=583000 CPUtime=49.03 cores=0-7
/proc/27503/task/27507/stat : 27507 (claspmt) R 27501 27503 27252 0 -1 4202560 8782 0 0 0 4897 6 0 0 25 0 8 0 545981135 596992000 99051 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=27503/tid=27508] ppid=27501 vsize=583000 CPUtime=49.13 cores=0-7
/proc/27503/task/27508/stat : 27508 (claspmt) R 27501 27503 27252 0 -1 4202560 9779 0 0 0 4907 6 0 0 25 0 8 0 545981135 596992000 99051 33554432000 4194304 4964704 140736615323392 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27503/tid=27509] ppid=27501 vsize=583000 CPUtime=49.37 cores=0-7
/proc/27503/task/27509/stat : 27509 (claspmt) R 27501 27503 27252 0 -1 4202560 7032 0 0 0 4933 4 0 0 25 0 8 0 545981135 596992000 99051 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=27503/tid=27510] ppid=27501 vsize=583000 CPUtime=49.13 cores=0-7
/proc/27503/task/27510/stat : 27510 (claspmt) R 27501 27503 27252 0 -1 4202560 7704 0 0 0 4909 4 0 0 25 0 8 0 545981135 596992000 99051 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27503/tid=27511] ppid=27501 vsize=583000 CPUtime=49.14 cores=0-7
/proc/27503/task/27511/stat : 27511 (claspmt) R 27501 27503 27252 0 -1 4202560 29030 0 0 0 4901 13 0 0 25 0 8 0 545981135 596992000 99051 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 394.73
Current children cumulated vsize (KiB) 583000

[startup+102.307 s]
/proc/loadavg: 7.94 7.89 7.44 9/183 27514
/proc/meminfo: memFree=27819424/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=798152 CPUtime=804.04 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 123785 0 0 0 80328 76 0 0 25 0 8 0 545980964 817307648 121835 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/27503/statm: 199538 121835 466 189 0 193887 0
[pid=27503/tid=27505] ppid=27501 vsize=798152 CPUtime=99.87 cores=0-7
/proc/27503/task/27505/stat : 27505 (claspmt) R 27501 27503 27252 0 -1 4202560 14398 0 0 0 9976 11 0 0 25 0 8 0 545981135 817307648 121835 33554432000 4194304 4964704 140736615323392 18446744073709551615 4668937 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=27503/tid=27506] ppid=27501 vsize=798152 CPUtime=100.35 cores=0-7
/proc/27503/task/27506/stat : 27506 (claspmt) R 27501 27503 27252 0 -1 4202560 13592 0 0 0 10027 8 0 0 25 0 8 0 545981135 817307648 121835 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=27503/tid=27507] ppid=27501 vsize=798152 CPUtime=100.23 cores=0-7
/proc/27503/task/27507/stat : 27507 (claspmt) R 27501 27503 27252 0 -1 4202560 10481 0 0 0 10016 7 0 0 25 0 8 0 545981135 817307648 121835 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0

################
# More data... #
################

/proc/27503/task/27510/stat : 27510 (claspmt) R 27501 27503 27252 0 -1 4202560 30596 0 0 0 465984 38 0 0 25 0 8 0 545981135 2748596224 597431 33554432000 4194304 4964704 140736615323392 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27503/tid=27511] ppid=27501 vsize=2684176 CPUtime=4660.01 cores=0-7
/proc/27503/task/27511/stat : 27511 (claspmt) R 27501 27503 27252 0 -1 4202560 289265 0 0 0 465665 336 0 0 25 0 8 0 545981135 2748596224 597431 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 37256.7
Current children cumulated vsize (KiB) 2684176

[startup+4722.31 s]
/proc/loadavg: 8.00 8.03 8.00 9/183 27658
/proc/meminfo: memFree=25863268/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=2684176 CPUtime=37736.4 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 620283 0 0 0 3772924 715 0 0 25 0 8 0 545980964 2748596224 609507 33554432000 4194304 4964704 140736615323392 18446744073709551615 4321102 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/27503/statm: 671044 609507 466 189 0 665393 0
[pid=27503/tid=27505] ppid=27501 vsize=2684176 CPUtime=4693.97 cores=0-7
/proc/27503/task/27505/stat : 27505 (claspmt) R 27501 27503 27252 0 -1 4202560 63877 0 0 0 469324 73 0 0 25 0 8 0 545981135 2748596224 609507 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=27503/tid=27506] ppid=27501 vsize=2684176 CPUtime=4720.33 cores=0-7
/proc/27503/task/27506/stat : 27506 (claspmt) R 27501 27503 27252 0 -1 4202560 43542 0 0 0 471994 39 0 0 25 0 8 0 545981135 2748596224 609507 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=27503/tid=27507] ppid=27501 vsize=2684176 CPUtime=4719.7 cores=0-7
/proc/27503/task/27507/stat : 27507 (claspmt) R 27501 27503 27252 0 -1 4202560 35207 0 0 0 471923 47 0 0 25 0 8 0 545981135 2748596224 609507 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=27503/tid=27508] ppid=27501 vsize=2684176 CPUtime=4719.84 cores=0-7
/proc/27503/task/27508/stat : 27508 (claspmt) R 27501 27503 27252 0 -1 4202560 39809 0 0 0 471941 43 0 0 25 0 8 0 545981135 2748596224 609507 33554432000 4194304 4964704 140736615323392 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27503/tid=27509] ppid=27501 vsize=2684176 CPUtime=4720.5 cores=0-7
/proc/27503/task/27509/stat : 27509 (claspmt) R 27501 27503 27252 0 -1 4202560 33914 0 0 0 472016 34 0 0 25 0 8 0 545981135 2748596224 609507 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=27503/tid=27510] ppid=27501 vsize=2684176 CPUtime=4720.23 cores=0-7
/proc/27503/task/27510/stat : 27510 (claspmt) R 27501 27503 27252 0 -1 4202560 30640 0 0 0 471985 38 0 0 25 0 8 0 545981135 2748596224 609507 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670357 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27503/tid=27511] ppid=27501 vsize=2684176 CPUtime=4720.02 cores=0-7
/proc/27503/task/27511/stat : 27511 (claspmt) R 27501 27503 27252 0 -1 4202560 298079 0 0 0 471650 352 0 0 25 0 8 0 545981135 2748596224 609507 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 37736.4
Current children cumulated vsize (KiB) 2684176

[startup+4782.3 s]
/proc/loadavg: 8.07 8.04 8.00 9/183 27660
/proc/meminfo: memFree=25849984/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=2684736 CPUtime=38216 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 623486 0 0 0 3820876 722 0 0 25 0 8 0 545980964 2749169664 612710 33554432000 4194304 4964704 140736615323392 18446744073709551615 4627281 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/27503/statm: 671184 612710 466 189 0 665533 0
[pid=27503/tid=27505] ppid=27501 vsize=2684736 CPUtime=4753.61 cores=0-7
/proc/27503/task/27505/stat : 27505 (claspmt) R 27501 27503 27252 0 -1 4202560 64127 0 0 0 475286 75 0 0 25 0 8 0 545981135 2749169664 612710 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=27503/tid=27506] ppid=27501 vsize=2684736 CPUtime=4780.32 cores=0-7
/proc/27503/task/27506/stat : 27506 (claspmt) R 27501 27503 27252 0 -1 4202560 43963 0 0 0 477993 39 0 0 25 0 8 0 545981135 2749169664 612710 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=27503/tid=27507] ppid=27501 vsize=2684736 CPUtime=4779.7 cores=0-7
/proc/27503/task/27507/stat : 27507 (claspmt) R 27501 27503 27252 0 -1 4202560 35534 0 0 0 477922 48 0 0 25 0 8 0 545981135 2749169664 612710 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=27503/tid=27508] ppid=27501 vsize=2684736 CPUtime=4779.83 cores=0-7
/proc/27503/task/27508/stat : 27508 (claspmt) R 27501 27503 27252 0 -1 4202560 39816 0 0 0 477940 43 0 0 25 0 8 0 545981135 2749169664 612710 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27503/tid=27509] ppid=27501 vsize=2684736 CPUtime=4780.49 cores=0-7
/proc/27503/task/27509/stat : 27509 (claspmt) R 27501 27503 27252 0 -1 4202560 33941 0 0 0 478015 34 0 0 25 0 8 0 545981135 2749169664 612710 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=27503/tid=27510] ppid=27501 vsize=2684736 CPUtime=4780.22 cores=0-7
/proc/27503/task/27510/stat : 27510 (claspmt) R 27501 27503 27252 0 -1 4202560 30704 0 0 0 477984 38 0 0 25 0 8 0 545981135 2749169664 612710 33554432000 4194304 4964704 140736615323392 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27503/tid=27511] ppid=27501 vsize=2684736 CPUtime=4780.02 cores=0-7
/proc/27503/task/27511/stat : 27511 (claspmt) R 27501 27503 27252 0 -1 4202560 299397 0 0 0 477648 354 0 0 25 0 8 0 545981135 2749169664 612710 33554432000 4194304 4964704 140736615323392 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38216
Current children cumulated vsize (KiB) 2684736

[startup+4842.3 s]
/proc/loadavg: 8.25 8.07 8.02 10/184 27664
/proc/meminfo: memFree=25832080/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=2685244 CPUtime=38695.4 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 623610 0 0 0 3868812 723 0 0 25 0 8 0 545980964 2749689856 612834 33554432000 4194304 4964704 140736615323392 18446744073709551615 4342213 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/27503/statm: 671311 612834 466 189 0 665660 0
[pid=27503/tid=27505] ppid=27501 vsize=2685244 CPUtime=4812.99 cores=0-7
/proc/27503/task/27505/stat : 27505 (claspmt) R 27501 27503 27252 0 -1 4202560 64129 0 0 0 481224 75 0 0 25 0 8 0 545981135 2749689856 612834 33554432000 4194304 4964704 140736615323392 18446744073709551615 4359851 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=27503/tid=27506] ppid=27501 vsize=2685244 CPUtime=4840.32 cores=0-7
/proc/27503/task/27506/stat : 27506 (claspmt) R 27501 27503 27252 0 -1 4202560 43968 0 0 0 483993 39 0 0 25 0 8 0 545981135 2749689856 612834 33554432000 4194304 4964704 140736615323392 18446744073709551615 4683983 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=27503/tid=27507] ppid=27501 vsize=2685244 CPUtime=4839.68 cores=0-7
/proc/27503/task/27507/stat : 27507 (claspmt) R 27501 27503 27252 0 -1 4202560 35592 0 0 0 483920 48 0 0 25 0 8 0 545981135 2749689856 612834 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=27503/tid=27508] ppid=27501 vsize=2685244 CPUtime=4839.82 cores=0-7
/proc/27503/task/27508/stat : 27508 (claspmt) R 27501 27503 27252 0 -1 4202560 39831 0 0 0 483939 43 0 0 25 0 8 0 545981135 2749689856 612834 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27503/tid=27509] ppid=27501 vsize=2685244 CPUtime=4840.49 cores=0-7
/proc/27503/task/27509/stat : 27509 (claspmt) R 27501 27503 27252 0 -1 4202560 33941 0 0 0 484015 34 0 0 25 0 8 0 545981135 2749689856 612834 33554432000 4194304 4964704 140736615323392 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=27503/tid=27510] ppid=27501 vsize=2685244 CPUtime=4840.22 cores=0-7
/proc/27503/task/27510/stat : 27510 (claspmt) R 27501 27503 27252 0 -1 4202560 30721 0 0 0 483984 38 0 0 25 0 8 0 545981135 2749689856 612834 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27503/tid=27511] ppid=27501 vsize=2685244 CPUtime=4840.01 cores=0-7
/proc/27503/task/27511/stat : 27511 (claspmt) R 27501 27503 27252 0 -1 4202560 299413 0 0 0 483647 354 0 0 25 0 8 0 545981135 2749689856 612834 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38695.4
Current children cumulated vsize (KiB) 2685244

[startup+4902.3 s]
/proc/loadavg: 8.22 8.10 8.03 9/183 27666
/proc/meminfo: memFree=25846588/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=2687100 CPUtime=39174.5 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 624121 0 0 0 3916723 724 0 0 25 0 8 0 545980964 2751590400 613217 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/27503/statm: 671775 613217 466 189 0 666124 0
[pid=27503/tid=27505] ppid=27501 vsize=2687100 CPUtime=4872.12 cores=0-7
/proc/27503/task/27505/stat : 27505 (claspmt) R 27501 27503 27252 0 -1 4202560 64129 0 0 0 487137 75 0 0 25 0 8 0 545981135 2751590400 613217 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670560 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=27503/tid=27506] ppid=27501 vsize=2687100 CPUtime=4900.32 cores=0-7
/proc/27503/task/27506/stat : 27506 (claspmt) R 27501 27503 27252 0 -1 4202560 44391 0 0 0 489993 39 0 0 25 0 8 0 545981135 2751590400 613217 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=27503/tid=27507] ppid=27501 vsize=2687100 CPUtime=4899.68 cores=0-7
/proc/27503/task/27507/stat : 27507 (claspmt) R 27501 27503 27252 0 -1 4202560 35637 0 0 0 489920 48 0 0 25 0 8 0 545981135 2751590400 613217 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=27503/tid=27508] ppid=27501 vsize=2687100 CPUtime=4899.82 cores=0-7
/proc/27503/task/27508/stat : 27508 (claspmt) R 27501 27503 27252 0 -1 4202560 39853 0 0 0 489939 43 0 0 25 0 8 0 545981135 2751590400 613217 33554432000 4194304 4964704 140736615323392 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27503/tid=27509] ppid=27501 vsize=2687100 CPUtime=4900.5 cores=0-7
/proc/27503/task/27509/stat : 27509 (claspmt) R 27501 27503 27252 0 -1 4202560 33941 0 0 0 490016 34 0 0 25 0 8 0 545981135 2751590400 613217 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=27503/tid=27510] ppid=27501 vsize=2687100 CPUtime=4900.22 cores=0-7
/proc/27503/task/27510/stat : 27510 (claspmt) R 27501 27503 27252 0 -1 4202560 30721 0 0 0 489983 39 0 0 25 0 8 0 545981135 2751590400 613217 33554432000 4194304 4964704 140736615323392 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27503/tid=27511] ppid=27501 vsize=2687100 CPUtime=4900.01 cores=0-7
/proc/27503/task/27511/stat : 27511 (claspmt) R 27501 27503 27252 0 -1 4202560 299413 0 0 0 489647 354 0 0 25 0 8 0 545981135 2751590400 613217 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670458 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39174.5
Current children cumulated vsize (KiB) 2687100

[startup+4962.3 s]
/proc/loadavg: 8.14 8.10 8.02 9/183 27668
/proc/meminfo: memFree=25834032/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=2752976 CPUtime=39654.1 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 627551 0 0 0 3964684 731 0 0 25 0 8 0 545980964 2819047424 616647 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/27503/statm: 688244 616647 466 189 0 682593 0
[pid=27503/tid=27505] ppid=27501 vsize=2752976 CPUtime=4931.79 cores=0-7
/proc/27503/task/27505/stat : 27505 (claspmt) R 27501 27503 27252 0 -1 4202560 64138 0 0 0 493104 75 0 0 25 0 8 0 545981135 2819047424 616647 33554432000 4194304 4964704 140736615323392 18446744073709551615 4321076 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=27503/tid=27506] ppid=27501 vsize=2752976 CPUtime=4960.33 cores=0-7
/proc/27503/task/27506/stat : 27506 (claspmt) R 27501 27503 27252 0 -1 4202560 44585 0 0 0 495993 40 0 0 25 0 8 0 545981135 2819047424 616647 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=27503/tid=27507] ppid=27501 vsize=2752976 CPUtime=4959.68 cores=0-7
/proc/27503/task/27507/stat : 27507 (claspmt) R 27501 27503 27252 0 -1 4202560 35677 0 0 0 495920 48 0 0 25 0 8 0 545981135 2819047424 616647 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=27503/tid=27508] ppid=27501 vsize=2752976 CPUtime=4959.81 cores=0-7
/proc/27503/task/27508/stat : 27508 (claspmt) R 27501 27503 27252 0 -1 4202560 40084 0 0 0 495938 43 0 0 25 0 8 0 545981135 2819047424 616647 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27503/tid=27509] ppid=27501 vsize=2752976 CPUtime=4960.49 cores=0-7
/proc/27503/task/27509/stat : 27509 (claspmt) R 27501 27503 27252 0 -1 4202560 34410 0 0 0 496015 34 0 0 25 0 8 0 545981135 2819047424 616647 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=27503/tid=27510] ppid=27501 vsize=2752976 CPUtime=4960.22 cores=0-7
/proc/27503/task/27510/stat : 27510 (claspmt) R 27501 27503 27252 0 -1 4202560 30927 0 0 0 495982 40 0 0 25 0 8 0 545981135 2819047424 616647 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27503/tid=27511] ppid=27501 vsize=2752976 CPUtime=4960.01 cores=0-7
/proc/27503/task/27511/stat : 27511 (claspmt) R 27501 27503 27252 0 -1 4202560 299413 0 0 0 495647 354 0 0 25 0 8 0 545981135 2819047424 616647 33554432000 4194304 4964704 140736615323392 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39654.1
Current children cumulated vsize (KiB) 2752976



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5005.6 s]
/proc/loadavg: 8.13 8.10 8.03 9/183 27669
/proc/meminfo: memFree=25826688/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=2753448 CPUtime=40000.4 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 629396 0 0 0 3999302 733 0 0 25 0 8 0 545980964 2819530752 618492 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/27503/statm: 688362 618492 466 189 0 682711 0
[pid=27503/tid=27505] ppid=27501 vsize=2753448 CPUtime=4974.86 cores=0-7
/proc/27503/task/27505/stat : 27505 (claspmt) R 27501 27503 27252 0 -1 4202560 64213 0 0 0 497411 75 0 0 25 0 8 0 545981135 2819530752 618492 33554432000 4194304 4964704 140736615323392 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=27503/tid=27506] ppid=27501 vsize=2753448 CPUtime=5003.64 cores=0-7
/proc/27503/task/27506/stat : 27506 (claspmt) R 27501 27503 27252 0 -1 4202560 44693 0 0 0 500324 40 0 0 25 0 8 0 545981135 2819530752 618492 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=27503/tid=27507] ppid=27501 vsize=2753448 CPUtime=5002.98 cores=0-7
/proc/27503/task/27507/stat : 27507 (claspmt) R 27501 27503 27252 0 -1 4202560 35677 0 0 0 500250 48 0 0 25 0 8 0 545981135 2819530752 618492 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=27503/tid=27508] ppid=27501 vsize=2753448 CPUtime=5003.11 cores=0-7
/proc/27503/task/27508/stat : 27508 (claspmt) R 27501 27503 27252 0 -1 4202560 40377 0 0 0 500267 44 0 0 25 0 8 0 545981135 2819530752 618492 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27503/tid=27509] ppid=27501 vsize=2753448 CPUtime=5003.8 cores=0-7
/proc/27503/task/27509/stat : 27509 (claspmt) R 27501 27503 27252 0 -1 4202560 34537 0 0 0 500346 34 0 0 25 0 8 0 545981135 2819530752 618492 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670193 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=27503/tid=27510] ppid=27501 vsize=2753448 CPUtime=5003.53 cores=0-7
/proc/27503/task/27510/stat : 27510 (claspmt) R 27501 27503 27252 0 -1 4202560 31310 0 0 0 500313 40 0 0 25 0 8 0 545981135 2819530752 618492 33554432000 4194304 4964704 140736615323392 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27503/tid=27511] ppid=27501 vsize=2753448 CPUtime=5003.31 cores=0-7
/proc/27503/task/27511/stat : 27511 (claspmt) R 27501 27503 27252 0 -1 4202560 299414 0 0 0 499977 354 0 0 25 0 8 0 545981135 2819530752 618492 33554432000 4194304 4964704 140736615323392 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.3
Current children cumulated vsize (KiB) 2753448

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5006.2 s]
/proc/loadavg: 8.13 8.10 8.03 2/177 27670
/proc/meminfo: memFree=25835168/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=2744912 CPUtime=40001 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 629406 0 0 0 3999366 733 0 0 25 0 1 0 545980964 2810789888 616224 33554432000 4194304 4964704 140736615323392 18446744073709551615 222921433659 0 0 4096 18946 0 0 0 17 3 0 0 0
/proc/27503/statm: 686228 616224 476 189 0 680577 0
Current children cumulated CPU time (s) 40001
Current children cumulated vsize (KiB) 2744912

[startup+5006.6 s]
/proc/loadavg: 8.13 8.10 8.03 2/177 27670
/proc/meminfo: memFree=25835168/32951124 swapFree=67110188/67111528
[pid=27503] ppid=27501 vsize=400968 CPUtime=40001.4 cores=0-7
/proc/27503/stat : 27503 (claspmt) R 27501 27503 27252 0 -1 4202496 629406 0 0 0 3999396 743 0 0 25 0 1 0 545980964 410591232 23319 33554432000 4194304 4964704 140736615323392 18446744073709551615 222921820519 0 0 4096 18946 0 0 0 17 3 0 0 0
/proc/27503/statm: 100242 23319 476 189 0 94591 0
Current children cumulated CPU time (s) 40001.4
Current children cumulated vsize (KiB) 400968

Child status: 0
Real time (s): 5006.61
CPU time (s): 40001.4
CPU user time (s): 39994
CPU system time (s): 7.43887
CPU usage (%): 798.972
Max. virtual memory (cumulated for all children) (KiB): 2753448

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39994
system time used= 7.43887
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 629407
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 83206
involuntary context switches= 189799

runsolver used 7.51886 second user time and 19.802 second system time

The end

Launcher Data 

Begin job on node109 at 2011-05-02 15:43:22
IDJOB=3321495
IDBENCH=71408
IDSOLVER=1711
FILE ID=node109/3321495-1304343802
RUNJOBID= node109-1304340984-27266
PBS_JOBID= 13178057
Free space on /tmp= 73264 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT09/CRAFTED/rbsat/random/forced/rbsat-v2640c305320gyes5.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321495-1304343802/watcher-3321495-1304343802 -o /tmp/evaluation-result-3321495-1304343802/solver-3321495-1304343802 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321495-1304343802.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= f892a2bda6fe6aec6f2fcb4e1c541df8
RANDOM SEED=381540432

node109.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      28302592 kB
Buffers:        851536 kB
Cached:        2737052 kB
SwapCached:        712 kB
Active:        1006888 kB
Inactive:      3243284 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      28302592 kB
SwapTotal:    67111528 kB
SwapFree:     67110188 kB
Dirty:            4708 kB
Writeback:           0 kB
AnonPages:      660948 kB
Mapped:          16276 kB
Slab:           331068 kB
PageTables:       5988 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   882932 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73264 MiB
End job on node109 at 2011-05-02 17:06:50