Trace number 3380582

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40000.5 5007.94

General information on the benchmark

NameAPPLICATIONS/bitverif/countbitsrotate/
countbitsrotate032.cnf
MD5SUM8d76aa747359ea8ccf15726792949452
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables8527
Number of clauses25484
Sum of the clauses size59460
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 216988
Number of clauses of size 38494
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...481852/instance-3380582-1305481852.cnf
0.00/0.04	c Solving...
40000.28/5007.81	
40000.28/5007.81	*** INTERRUPTED! ***
40000.28/5007.81	*** Shutting down threads...
40000.48/5007.93	s UNKNOWN
40000.48/5007.93	
40000.48/5007.93	c INTERRUPTED   : 1
40000.48/5007.93	c Models        : 0+    
40000.48/5007.93	c Time          : 5007.818s (Solving: 5007.78s 1st Model: 0.00s Unsat: 0.00s)
40000.48/5007.93	c CPU Time      : 40000.380s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3380582-1305481852/watcher-3380582-1305481852 -o /tmp/evaluation-result-3380582-1305481852/solver-3380582-1305481852 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380582-1305481852.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.23 6.23 3.37 1/354 4509
/proc/meminfo: memFree=14630028/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=22780 CPUtime=0 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 1041 0 0 0 0 0 0 0 25 0 1 0 640601062 23326720 905 33554432000 4194304 4964800 140737191817504 18446744073709551615 4599599 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/4509/statm: 5695 905 386 189 0 569 0

[startup+0.857782 s]
/proc/loadavg: 7.23 6.23 3.37 1/354 4509
/proc/meminfo: memFree=14630028/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=389776 CPUtime=6.28 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 6534 0 0 0 627 1 0 0 21 0 8 0 640601062 399130624 5680 33554432000 4194304 4964800 140737191817504 18446744073709551615 4321102 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 97444 5680 464 189 0 91793 0
Current children cumulated CPU time (s) 6.28
Current children cumulated vsize (KiB) 389776

[startup+0.957765 s]
/proc/loadavg: 7.23 6.23 3.37 1/354 4509
/proc/meminfo: memFree=14630028/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=389908 CPUtime=7.09 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 6768 0 0 0 707 2 0 0 21 0 8 0 640601062 399265792 5914 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670299 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 97477 5914 464 189 0 91826 0
Current children cumulated CPU time (s) 7.09
Current children cumulated vsize (KiB) 389908

[startup+1.05775 s]
/proc/loadavg: 7.23 6.23 3.37 1/354 4509
/proc/meminfo: memFree=14630028/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=390040 CPUtime=7.88 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 7015 0 0 0 786 2 0 0 21 0 8 0 640601062 399400960 6161 33554432000 4194304 4964800 140737191817504 18446744073709551615 4660564 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 97510 6162 464 189 0 91859 0
Current children cumulated CPU time (s) 7.88
Current children cumulated vsize (KiB) 390040

[startup+1.1226 s]
/proc/loadavg: 7.23 6.23 3.37 1/354 4509
/proc/meminfo: memFree=14630028/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=390040 CPUtime=8.4 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 7183 0 0 0 838 2 0 0 21 0 8 0 640601062 399400960 6329 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668987 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 97510 6329 464 189 0 91859 0
Current children cumulated CPU time (s) 8.4
Current children cumulated vsize (KiB) 390040

[startup+1.50067 s]
/proc/loadavg: 7.23 6.23 3.37 1/354 4509
/proc/meminfo: memFree=14630028/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=390456 CPUtime=11.43 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 8143 0 0 0 1140 3 0 0 22 0 8 0 640601062 399826944 7289 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670129 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 97614 7289 464 189 0 91963 0
Current children cumulated CPU time (s) 11.43
Current children cumulated vsize (KiB) 390456

[startup+3.10038 s]
/proc/loadavg: 7.23 6.23 3.37 9/362 4517
/proc/meminfo: memFree=14597912/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=392084 CPUtime=24.19 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 9309 0 0 0 2416 3 0 0 25 0 8 0 640601062 401494016 8455 33554432000 4194304 4964800 140737191817504 18446744073709551615 4660435 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 98021 8455 464 189 0 92370 0
[pid=4509/tid=4511] ppid=4507 vsize=392084 CPUtime=3.03 cores=0-7
/proc/4509/task/4511/stat : 4511 (claspmt) R 4507 4509 4178 0 -1 4202560 650 0 0 0 303 0 0 0 25 0 8 0 640601067 401494016 8455 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670239 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4509/tid=4512] ppid=4507 vsize=392084 CPUtime=3.03 cores=0-7
/proc/4509/task/4512/stat : 4512 (claspmt) R 4507 4509 4178 0 -1 4202560 693 0 0 0 303 0 0 0 21 0 8 0 640601067 401494016 8455 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4509/tid=4513] ppid=4507 vsize=392084 CPUtime=3.03 cores=0-7
/proc/4509/task/4513/stat : 4513 (claspmt) R 4507 4509 4178 0 -1 4202560 630 0 0 0 303 0 0 0 25 0 8 0 640601067 401494016 8455 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668969 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4509/tid=4514] ppid=4507 vsize=392084 CPUtime=3.03 cores=0-7
/proc/4509/task/4514/stat : 4514 (claspmt) R 4507 4509 4178 0 -1 4202560 750 0 0 0 303 0 0 0 25 0 8 0 640601067 401494016 8455 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4509/tid=4515] ppid=4507 vsize=392084 CPUtime=3.01 cores=0-7
/proc/4509/task/4515/stat : 4515 (claspmt) R 4507 4509 4178 0 -1 4202560 998 0 0 0 301 0 0 0 25 0 8 0 640601067 401494016 8455 33554432000 4194304 4964800 140737191817504 18446744073709551615 4669280 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4509/tid=4516] ppid=4507 vsize=392084 CPUtime=2.92 cores=0-7
/proc/4509/task/4516/stat : 4516 (claspmt) R 4507 4509 4178 0 -1 4202560 719 0 0 0 292 0 0 0 25 0 8 0 640601067 401494016 8455 33554432000 4194304 4964800 140737191817504 18446744073709551615 4660500 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4509/tid=4517] ppid=4507 vsize=392084 CPUtime=3.02 cores=0-7
/proc/4509/task/4517/stat : 4517 (claspmt) R 4507 4509 4178 0 -1 4202560 2399 0 0 0 301 1 0 0 24 0 8 0 640601067 401494016 8455 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 24.19
Current children cumulated vsize (KiB) 392084

[startup+6.31381 s]
/proc/loadavg: 7.29 6.26 3.39 9/362 4518
/proc/meminfo: memFree=14587136/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=397496 CPUtime=49.87 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 12018 0 0 0 4982 5 0 0 25 0 8 0 640601062 407035904 11164 33554432000 4194304 4964800 140737191817504 18446744073709551615 4660155 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 99374 11164 464 189 0 93723 0
[pid=4509/tid=4511] ppid=4507 vsize=397496 CPUtime=6.24 cores=0-7
/proc/4509/task/4511/stat : 4511 (claspmt) R 4507 4509 4178 0 -1 4202560 751 0 0 0 624 0 0 0 25 0 8 0 640601067 407035904 11164 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4509/tid=4512] ppid=4507 vsize=397496 CPUtime=6.25 cores=0-7
/proc/4509/task/4512/stat : 4512 (claspmt) R 4507 4509 4178 0 -1 4202560 804 0 0 0 625 0 0 0 25 0 8 0 640601067 407035904 11164 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4509/tid=4513] ppid=4507 vsize=397496 CPUtime=6.24 cores=0-7
/proc/4509/task/4513/stat : 4513 (claspmt) R 4507 4509 4178 0 -1 4202560 677 0 0 0 624 0 0 0 25 0 8 0 640601067 407035904 11164 33554432000 4194304 4964800 140737191817504 18446744073709551615 4361295 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4509/tid=4514] ppid=4507 vsize=397496 CPUtime=6.24 cores=0-7
/proc/4509/task/4514/stat : 4514 (claspmt) R 4507 4509 4178 0 -1 4202560 834 0 0 0 624 0 0 0 25 0 8 0 640601067 407035904 11164 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668737 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4509/tid=4515] ppid=4507 vsize=397496 CPUtime=6.18 cores=0-7
/proc/4509/task/4515/stat : 4515 (claspmt) R 4507 4509 4178 0 -1 4202560 1111 0 0 0 618 0 0 0 25 0 8 0 640601067 407035904 11164 33554432000 4194304 4964800 140737191817504 18446744073709551615 4669131 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4509/tid=4516] ppid=4507 vsize=397496 CPUtime=6.13 cores=0-7
/proc/4509/task/4516/stat : 4516 (claspmt) R 4507 4509 4178 0 -1 4202560 746 0 0 0 613 0 0 0 25 0 8 0 640601067 407035904 11164 33554432000 4194304 4964800 140737191817504 18446744073709551615 4342213 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4509/tid=4517] ppid=4507 vsize=397496 CPUtime=6.24 cores=0-7
/proc/4509/task/4517/stat : 4517 (claspmt) R 4507 4509 4178 0 -1 4202560 4605 0 0 0 622 2 0 0 25 0 8 0 640601067 407035904 11164 33554432000 4194304 4964800 140737191817504 18446744073709551615 4660384 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 49.87
Current children cumulated vsize (KiB) 397496

[startup+12.7007 s]
/proc/loadavg: 7.35 6.29 3.41 9/362 4518
/proc/meminfo: memFree=14575516/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=397496 CPUtime=100.88 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 15041 0 0 0 10081 7 0 0 25 0 8 0 640601062 407035904 14187 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668568 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 99374 14187 464 189 0 93723 0
[pid=4509/tid=4511] ppid=4507 vsize=397496 CPUtime=12.62 cores=0-7
/proc/4509/task/4511/stat : 4511 (claspmt) R 4507 4509 4178 0 -1 4202560 788 0 0 0 1262 0 0 0 25 0 8 0 640601067 407035904 14187 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670340 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4509/tid=4512] ppid=4507 vsize=397496 CPUtime=12.63 cores=0-7
/proc/4509/task/4512/stat : 4512 (claspmt) R 4507 4509 4178 0 -1 4202560 897 0 0 0 1263 0 0 0 25 0 8 0 640601067 407035904 14187 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4509/tid=4513] ppid=4507 vsize=397496 CPUtime=12.63 cores=0-7
/proc/4509/task/4513/stat : 4513 (claspmt) R 4507 4509 4178 0 -1 4202560 688 0 0 0 1263 0 0 0 25 0 8 0 640601067 407035904 14187 33554432000 4194304 4964800 140737191817504 18446744073709551615 4321082 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4509/tid=4514] ppid=4507 vsize=397496 CPUtime=12.63 cores=0-7
/proc/4509/task/4514/stat : 4514 (claspmt) R 4507 4509 4178 0 -1 4202560 941 0 0 0 1263 0 0 0 25 0 8 0 640601067 407035904 14187 33554432000 4194304 4964800 140737191817504 18446744073709551615 4338784 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4509/tid=4515] ppid=4507 vsize=397496 CPUtime=12.51 cores=0-7
/proc/4509/task/4515/stat : 4515 (claspmt) R 4507 4509 4178 0 -1 4202560 1310 0 0 0 1251 0 0 0 25 0 8 0 640601067 407035904 14187 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668748 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4509/tid=4516] ppid=4507 vsize=397496 CPUtime=12.52 cores=0-7
/proc/4509/task/4516/stat : 4516 (claspmt) R 4507 4509 4178 0 -1 4202560 774 0 0 0 1252 0 0 0 25 0 8 0 640601067 407035904 14187 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668698 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4509/tid=4517] ppid=4507 vsize=397496 CPUtime=12.61 cores=0-7
/proc/4509/task/4517/stat : 4517 (claspmt) R 4507 4509 4178 0 -1 4202560 7074 0 0 0 1258 3 0 0 25 0 8 0 640601067 407035904 14187 33554432000 4194304 4964800 140737191817504 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 100.88
Current children cumulated vsize (KiB) 397496

[startup+25.5004 s]
/proc/loadavg: 7.49 6.38 3.49 9/362 4518
/proc/meminfo: memFree=14562476/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=397496 CPUtime=203.13 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 18745 0 0 0 20303 10 0 0 25 0 8 0 640601062 407035904 17891 33554432000 4194304 4964800 140737191817504 18446744073709551615 4660498 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 99374 17891 466 189 0 93723 0
[pid=4509/tid=4511] ppid=4507 vsize=397496 CPUtime=25.42 cores=0-7
/proc/4509/task/4511/stat : 4511 (claspmt) R 4507 4509 4178 0 -1 4202560 823 0 0 0 2542 0 0 0 25 0 8 0 640601067 407035904 17891 33554432000 4194304 4964800 140737191817504 18446744073709551615 4660435 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4509/tid=4512] ppid=4507 vsize=397496 CPUtime=25.27 cores=0-7
/proc/4509/task/4512/stat : 4512 (claspmt) R 4507 4509 4178 0 -1 4202560 978 0 0 0 2527 0 0 0 25 0 8 0 640601067 407035904 17891 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668846 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4509/tid=4513] ppid=4507 vsize=397496 CPUtime=25.43 cores=0-7
/proc/4509/task/4513/stat : 4513 (claspmt) R 4507 4509 4178 0 -1 4202560 766 0 0 0 2543 0 0 0 25 0 8 0 640601067 407035904 17891 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4509/tid=4514] ppid=4507 vsize=397496 CPUtime=25.43 cores=0-7
/proc/4509/task/4514/stat : 4514 (claspmt) R 4507 4509 4178 0 -1 4202560 1001 0 0 0 2543 0 0 0 25 0 8 0 640601067 407035904 17891 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668814 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4509/tid=4515] ppid=4507 vsize=397496 CPUtime=25.31 cores=0-7
/proc/4509/task/4515/stat : 4515 (claspmt) R 4507 4509 4178 0 -1 4202560 1446 0 0 0 2530 1 0 0 25 0 8 0 640601067 407035904 17891 33554432000 4194304 4964800 140737191817504 18446744073709551615 4324747 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4509/tid=4516] ppid=4507 vsize=397496 CPUtime=25.32 cores=0-7
/proc/4509/task/4516/stat : 4516 (claspmt) R 4507 4509 4178 0 -1 4202560 821 0 0 0 2532 0 0 0 25 0 8 0 640601067 407035904 17891 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4509/tid=4517] ppid=4507 vsize=397496 CPUtime=25.42 cores=0-7
/proc/4509/task/4517/stat : 4517 (claspmt) R 4507 4509 4178 0 -1 4202560 10205 0 0 0 2536 6 0 0 25 0 8 0 640601067 407035904 17891 33554432000 4194304 4964800 140737191817504 18446744073709551615 4321076 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 203.13
Current children cumulated vsize (KiB) 397496

[startup+51.1008 s]
/proc/loadavg: 7.74 6.52 3.61 9/362 4519
/proc/meminfo: memFree=14541300/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=403248 CPUtime=407.62 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 24230 0 0 0 40746 16 0 0 25 0 8 0 640601062 412925952 23352 33554432000 4194304 4964800 140737191817504 18446744073709551615 4660390 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 100812 23352 466 189 0 95161 0
[pid=4509/tid=4511] ppid=4507 vsize=403248 CPUtime=51.02 cores=0-7
/proc/4509/task/4511/stat : 4511 (claspmt) R 4507 4509 4178 0 -1 4202560 947 0 0 0 5102 0 0 0 25 0 8 0 640601067 412925952 23352 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4509/tid=4512] ppid=4507 vsize=403248 CPUtime=50.57 cores=0-7
/proc/4509/task/4512/stat : 4512 (claspmt) R 4507 4509 4178 0 -1 4202560 1105 0 0 0 5057 0 0 0 25 0 8 0 640601067 412925952 23352 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4509/tid=4513] ppid=4507 vsize=403248 CPUtime=51.03 cores=0-7
/proc/4509/task/4513/stat : 4513 (claspmt) R 4507 4509 4178 0 -1 4202560 840 0 0 0 5103 0 0 0 25 0 8 0 640601067 412925952 23352 33554432000 4194304 4964800 140737191817504 18446744073709551615 4631088 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4509/tid=4514] ppid=4507 vsize=403248 CPUtime=51.02 cores=0-7
/proc/4509/task/4514/stat : 4514 (claspmt) R 4507 4509 4178 0 -1 4202560 1379 0 0 0 5102 0 0 0 25 0 8 0 640601067 412925952 23352 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4509/tid=4515] ppid=4507 vsize=403248 CPUtime=50.91 cores=0-7
/proc/4509/task/4515/stat : 4515 (claspmt) R 4507 4509 4178 0 -1 4202560 1555 0 0 0 5090 1 0 0 25 0 8 0 640601067 412925952 23352 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668689 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4509/tid=4516] ppid=4507 vsize=403248 CPUtime=50.92 cores=0-7
/proc/4509/task/4516/stat : 4516 (claspmt) R 4507 4509 4178 0 -1 4202560 843 0 0 0 5092 0 0 0 25 0 8 0 640601067 412925952 23352 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668829 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4509/tid=4517] ppid=4507 vsize=403248 CPUtime=51.02 cores=0-7
/proc/4509/task/4517/stat : 4517 (claspmt) R 4507 4509 4178 0 -1 4202560 14650 0 0 0 5091 11 0 0 25 0 8 0 640601067 412925952 23352 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 407.62
Current children cumulated vsize (KiB) 403248

[startup+102.301 s]
/proc/loadavg: 7.88 6.75 3.84 9/362 4520
/proc/meminfo: memFree=14504436/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=419568 CPUtime=816.6 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 32877 0 0 0 81636 24 0 0 25 0 8 0 640601062 429637632 31878 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 104892 31878 466 189 0 99241 0
[pid=4509/tid=4511] ppid=4507 vsize=419568 CPUtime=102.22 cores=0-7
/proc/4509/task/4511/stat : 4511 (claspmt) R 4507 4509 4178 0 -1 4202560 1343 0 0 0 10221 1 0 0 25 0 8 0 640601067 429637632 31878 33554432000 4194304 4964800 140737191817504 18446744073709551615 4653536 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4509/tid=4512] ppid=4507 vsize=419568 CPUtime=101.17 cores=0-7
/proc/4509/task/4512/stat : 4512 (claspmt) R 4507 4509 4178 0 -1 4202560 1433 0 0 0 10116 1 0 0 25 0 8 0 640601067 429637632 31878 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4509/tid=4513] ppid=4507 vsize=419568 CPUtime=102.23 cores=0-7
/proc/4509/task/4513/stat : 4513 (claspmt) R 4507 4509 4178 0 -1 4202560 899 0 0 0 10223 0 0 0 25 0 8 0 640601067 429637632 31878 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0

################
# More data... #
################

[pid=4509/tid=4512] ppid=4507 vsize=796460 CPUtime=4660.45 cores=0-7
/proc/4509/task/4512/stat : 4512 (claspmt) R 4507 4509 4178 0 -1 4202560 3465 0 0 0 466037 8 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4378951 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4509/tid=4513] ppid=4507 vsize=796460 CPUtime=4620.3 cores=0-7
/proc/4509/task/4513/stat : 4513 (claspmt) R 4507 4509 4178 0 -1 4202560 1854 0 0 0 462022 8 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4509/tid=4514] ppid=4507 vsize=796460 CPUtime=4661.32 cores=0-7
/proc/4509/task/4514/stat : 4514 (claspmt) R 4507 4509 4178 0 -1 4202560 8031 0 0 0 466119 13 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670144 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4509/tid=4515] ppid=4507 vsize=796460 CPUtime=4654.08 cores=0-7
/proc/4509/task/4515/stat : 4515 (claspmt) R 4507 4509 4178 0 -1 4202560 6884 0 0 0 465395 13 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4509/tid=4516] ppid=4507 vsize=796460 CPUtime=4662.04 cores=0-7
/proc/4509/task/4516/stat : 4516 (claspmt) R 4507 4509 4178 0 -1 4202560 1752 0 0 0 466197 7 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4332327 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4509/tid=4517] ppid=4507 vsize=796460 CPUtime=4658.04 cores=0-7
/proc/4509/task/4517/stat : 4517 (claspmt) R 4507 4509 4178 0 -1 4202560 90747 0 0 0 465667 137 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670349 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37240.4
Current children cumulated vsize (KiB) 796460

[startup+4722.3 s]
/proc/loadavg: 8.02 8.05 8.00 9/362 4664
/proc/meminfo: memFree=14167940/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=796460 CPUtime=37719.7 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 120234 0 0 0 3771756 214 0 0 25 0 8 0 640601062 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670180 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 199115 115370 466 189 0 193464 0
[pid=4509/tid=4511] ppid=4507 vsize=796460 CPUtime=4722.05 cores=0-7
/proc/4509/task/4511/stat : 4511 (claspmt) R 4507 4509 4178 0 -1 4202560 3435 0 0 0 472193 12 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668698 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4509/tid=4512] ppid=4507 vsize=796460 CPUtime=4720.45 cores=0-7
/proc/4509/task/4512/stat : 4512 (claspmt) R 4507 4509 4178 0 -1 4202560 3465 0 0 0 472037 8 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4509/tid=4513] ppid=4507 vsize=796460 CPUtime=4680.3 cores=0-7
/proc/4509/task/4513/stat : 4513 (claspmt) R 4507 4509 4178 0 -1 4202560 1854 0 0 0 468022 8 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668636 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4509/tid=4514] ppid=4507 vsize=796460 CPUtime=4721.31 cores=0-7
/proc/4509/task/4514/stat : 4514 (claspmt) R 4507 4509 4178 0 -1 4202560 8031 0 0 0 472117 14 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4509/tid=4515] ppid=4507 vsize=796460 CPUtime=4713.36 cores=0-7
/proc/4509/task/4515/stat : 4515 (claspmt) R 4507 4509 4178 0 -1 4202560 6884 0 0 0 471323 13 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4509/tid=4516] ppid=4507 vsize=796460 CPUtime=4722.04 cores=0-7
/proc/4509/task/4516/stat : 4516 (claspmt) R 4507 4509 4178 0 -1 4202560 1752 0 0 0 472197 7 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4509/tid=4517] ppid=4507 vsize=796460 CPUtime=4718.03 cores=0-7
/proc/4509/task/4517/stat : 4517 (claspmt) R 4507 4509 4178 0 -1 4202560 90747 0 0 0 471666 137 0 0 18 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668766 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37719.7
Current children cumulated vsize (KiB) 796460

[startup+4782.3 s]
/proc/loadavg: 8.06 8.05 8.00 9/362 4666
/proc/meminfo: memFree=14167896/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=796460 CPUtime=38199 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 120234 0 0 0 3819683 214 0 0 25 0 8 0 640601062 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4660114 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 199115 115370 466 189 0 193464 0
[pid=4509/tid=4511] ppid=4507 vsize=796460 CPUtime=4782.05 cores=0-7
/proc/4509/task/4511/stat : 4511 (claspmt) R 4507 4509 4178 0 -1 4202560 3435 0 0 0 478192 13 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4509/tid=4512] ppid=4507 vsize=796460 CPUtime=4780.45 cores=0-7
/proc/4509/task/4512/stat : 4512 (claspmt) R 4507 4509 4178 0 -1 4202560 3465 0 0 0 478037 8 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4509/tid=4513] ppid=4507 vsize=796460 CPUtime=4740.3 cores=0-7
/proc/4509/task/4513/stat : 4513 (claspmt) R 4507 4509 4178 0 -1 4202560 1854 0 0 0 474022 8 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4509/tid=4514] ppid=4507 vsize=796460 CPUtime=4781.3 cores=0-7
/proc/4509/task/4514/stat : 4514 (claspmt) R 4507 4509 4178 0 -1 4202560 8031 0 0 0 478116 14 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4658177 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4509/tid=4515] ppid=4507 vsize=796460 CPUtime=4772.66 cores=0-7
/proc/4509/task/4515/stat : 4515 (claspmt) R 4507 4509 4178 0 -1 4202560 6884 0 0 0 477253 13 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4509/tid=4516] ppid=4507 vsize=796460 CPUtime=4782.04 cores=0-7
/proc/4509/task/4516/stat : 4516 (claspmt) R 4507 4509 4178 0 -1 4202560 1752 0 0 0 478197 7 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4509/tid=4517] ppid=4507 vsize=796460 CPUtime=4778.03 cores=0-7
/proc/4509/task/4517/stat : 4517 (claspmt) R 4507 4509 4178 0 -1 4202560 90747 0 0 0 477666 137 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38199
Current children cumulated vsize (KiB) 796460

[startup+4842.3 s]
/proc/loadavg: 8.02 8.04 8.00 9/362 4667
/proc/meminfo: memFree=14167968/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=796460 CPUtime=38678.2 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 120234 0 0 0 3867609 215 0 0 25 0 8 0 640601062 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4362240 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 199115 115370 466 189 0 193464 0
[pid=4509/tid=4511] ppid=4507 vsize=796460 CPUtime=4842.05 cores=0-7
/proc/4509/task/4511/stat : 4511 (claspmt) R 4507 4509 4178 0 -1 4202560 3435 0 0 0 484192 13 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4509/tid=4512] ppid=4507 vsize=796460 CPUtime=4840.45 cores=0-7
/proc/4509/task/4512/stat : 4512 (claspmt) R 4507 4509 4178 0 -1 4202560 3465 0 0 0 484037 8 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670381 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4509/tid=4513] ppid=4507 vsize=796460 CPUtime=4800.3 cores=0-7
/proc/4509/task/4513/stat : 4513 (claspmt) R 4507 4509 4178 0 -1 4202560 1854 0 0 0 480022 8 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4509/tid=4514] ppid=4507 vsize=796460 CPUtime=4841.29 cores=0-7
/proc/4509/task/4514/stat : 4514 (claspmt) R 4507 4509 4178 0 -1 4202560 8031 0 0 0 484115 14 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4509/tid=4515] ppid=4507 vsize=796460 CPUtime=4831.95 cores=0-7
/proc/4509/task/4515/stat : 4515 (claspmt) R 4507 4509 4178 0 -1 4202560 6884 0 0 0 483182 13 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4509/tid=4516] ppid=4507 vsize=796460 CPUtime=4842.04 cores=0-7
/proc/4509/task/4516/stat : 4516 (claspmt) R 4507 4509 4178 0 -1 4202560 1752 0 0 0 484197 7 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4509/tid=4517] ppid=4507 vsize=796460 CPUtime=4838.03 cores=0-7
/proc/4509/task/4517/stat : 4517 (claspmt) R 4507 4509 4178 0 -1 4202560 90747 0 0 0 483666 137 0 0 25 0 8 0 640601067 815575040 115370 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38678.2
Current children cumulated vsize (KiB) 796460

[startup+4902.3 s]
/proc/loadavg: 8.05 8.05 8.00 9/362 4669
/proc/meminfo: memFree=14167932/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=796460 CPUtime=39157.6 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 120235 0 0 0 3915540 216 0 0 25 0 8 0 640601062 815575040 115371 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668698 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 199115 115371 466 189 0 193464 0
[pid=4509/tid=4511] ppid=4507 vsize=796460 CPUtime=4902.06 cores=0-7
/proc/4509/task/4511/stat : 4511 (claspmt) R 4507 4509 4178 0 -1 4202560 3435 0 0 0 490193 13 0 0 25 0 8 0 640601067 815575040 115371 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4509/tid=4512] ppid=4507 vsize=796460 CPUtime=4900.47 cores=0-7
/proc/4509/task/4512/stat : 4512 (claspmt) R 4507 4509 4178 0 -1 4202560 3465 0 0 0 490039 8 0 0 25 0 8 0 640601067 815575040 115371 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4509/tid=4513] ppid=4507 vsize=796460 CPUtime=4860.31 cores=0-7
/proc/4509/task/4513/stat : 4513 (claspmt) R 4507 4509 4178 0 -1 4202560 1854 0 0 0 486023 8 0 0 25 0 8 0 640601067 815575040 115371 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4509/tid=4514] ppid=4507 vsize=796460 CPUtime=4901.29 cores=0-7
/proc/4509/task/4514/stat : 4514 (claspmt) R 4507 4509 4178 0 -1 4202560 8031 0 0 0 490115 14 0 0 25 0 8 0 640601067 815575040 115371 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4509/tid=4515] ppid=4507 vsize=796460 CPUtime=4891.24 cores=0-7
/proc/4509/task/4515/stat : 4515 (claspmt) R 4507 4509 4178 0 -1 4202560 6884 0 0 0 489111 13 0 0 25 0 8 0 640601067 815575040 115371 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4509/tid=4516] ppid=4507 vsize=796460 CPUtime=4902.05 cores=0-7
/proc/4509/task/4516/stat : 4516 (claspmt) R 4507 4509 4178 0 -1 4202560 1752 0 0 0 490198 7 0 0 25 0 8 0 640601067 815575040 115371 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4509/tid=4517] ppid=4507 vsize=796460 CPUtime=4898.04 cores=0-7
/proc/4509/task/4517/stat : 4517 (claspmt) R 4507 4509 4178 0 -1 4202560 90748 0 0 0 489667 137 0 0 25 0 8 0 640601067 815575040 115371 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670335 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39157.6
Current children cumulated vsize (KiB) 796460

[startup+4962.3 s]
/proc/loadavg: 8.06 8.05 8.00 9/362 4671
/proc/meminfo: memFree=14168124/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=796460 CPUtime=39636.8 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 120237 0 0 0 3963462 216 0 0 25 0 8 0 640601062 815575040 115373 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670299 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 199115 115373 466 189 0 193464 0
[pid=4509/tid=4511] ppid=4507 vsize=796460 CPUtime=4962.04 cores=0-7
/proc/4509/task/4511/stat : 4511 (claspmt) R 4507 4509 4178 0 -1 4202560 3435 0 0 0 496191 13 0 0 25 0 8 0 640601067 815575040 115373 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670295 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4509/tid=4512] ppid=4507 vsize=796460 CPUtime=4960.45 cores=0-7
/proc/4509/task/4512/stat : 4512 (claspmt) R 4507 4509 4178 0 -1 4202560 3465 0 0 0 496037 8 0 0 25 0 8 0 640601067 815575040 115373 33554432000 4194304 4964800 140737191817504 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4509/tid=4513] ppid=4507 vsize=796460 CPUtime=4920.29 cores=0-7
/proc/4509/task/4513/stat : 4513 (claspmt) R 4507 4509 4178 0 -1 4202560 1854 0 0 0 492021 8 0 0 25 0 8 0 640601067 815575040 115373 33554432000 4194304 4964800 140737191817504 18446744073709551615 4660408 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4509/tid=4514] ppid=4507 vsize=796460 CPUtime=4961.26 cores=0-7
/proc/4509/task/4514/stat : 4514 (claspmt) R 4507 4509 4178 0 -1 4202560 8031 0 0 0 496112 14 0 0 25 0 8 0 640601067 815575040 115373 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4509/tid=4515] ppid=4507 vsize=796460 CPUtime=4950.53 cores=0-7
/proc/4509/task/4515/stat : 4515 (claspmt) R 4507 4509 4178 0 -1 4202560 6884 0 0 0 495040 13 0 0 25 0 8 0 640601067 815575040 115373 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4509/tid=4516] ppid=4507 vsize=796460 CPUtime=4962.04 cores=0-7
/proc/4509/task/4516/stat : 4516 (claspmt) R 4507 4509 4178 0 -1 4202560 1753 0 0 0 496197 7 0 0 25 0 8 0 640601067 815575040 115373 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4509/tid=4517] ppid=4507 vsize=796460 CPUtime=4958.02 cores=0-7
/proc/4509/task/4517/stat : 4517 (claspmt) R 4507 4509 4178 0 -1 4202560 90749 0 0 0 495665 137 0 0 25 0 8 0 640601067 815575040 115373 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39636.8
Current children cumulated vsize (KiB) 796460



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5007.8 s]
/proc/loadavg: 8.07 8.06 8.00 9/362 4672
/proc/meminfo: memFree=14167980/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=796460 CPUtime=40000.3 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 120238 0 0 0 3999811 217 0 0 25 0 8 0 640601062 815575040 115374 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4509/statm: 199115 115374 466 189 0 193464 0
[pid=4509/tid=4511] ppid=4507 vsize=796460 CPUtime=5007.55 cores=0-7
/proc/4509/task/4511/stat : 4511 (claspmt) R 4507 4509 4178 0 -1 4202560 3435 0 0 0 500742 13 0 0 25 0 8 0 640601067 815575040 115374 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668698 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4509/tid=4512] ppid=4507 vsize=796460 CPUtime=5005.96 cores=0-7
/proc/4509/task/4512/stat : 4512 (claspmt) R 4507 4509 4178 0 -1 4202560 3465 0 0 0 500588 8 0 0 25 0 8 0 640601067 815575040 115374 33554432000 4194304 4964800 140737191817504 18446744073709551615 4361247 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4509/tid=4513] ppid=4507 vsize=796460 CPUtime=4965.8 cores=0-7
/proc/4509/task/4513/stat : 4513 (claspmt) R 4507 4509 4178 0 -1 4202560 1854 0 0 0 496572 8 0 0 25 0 8 0 640601067 815575040 115374 33554432000 4194304 4964800 140737191817504 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4509/tid=4514] ppid=4507 vsize=796460 CPUtime=5006.77 cores=0-7
/proc/4509/task/4514/stat : 4514 (claspmt) R 4507 4509 4178 0 -1 4202560 8031 0 0 0 500663 14 0 0 25 0 8 0 640601067 815575040 115374 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4509/tid=4515] ppid=4507 vsize=796460 CPUtime=4995.49 cores=0-7
/proc/4509/task/4515/stat : 4515 (claspmt) R 4507 4509 4178 0 -1 4202560 6884 0 0 0 499536 13 0 0 25 0 8 0 640601067 815575040 115374 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4509/tid=4516] ppid=4507 vsize=796460 CPUtime=5007.55 cores=0-7
/proc/4509/task/4516/stat : 4516 (claspmt) R 4507 4509 4178 0 -1 4202560 1753 0 0 0 500748 7 0 0 25 0 8 0 640601067 815575040 115374 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4509/tid=4517] ppid=4507 vsize=796460 CPUtime=5003.54 cores=0-7
/proc/4509/task/4517/stat : 4517 (claspmt) R 4507 4509 4178 0 -1 4202560 90750 0 0 0 500217 137 0 0 25 0 8 0 640601067 815575040 115374 33554432000 4194304 4964800 140737191817504 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 40000.3
Current children cumulated vsize (KiB) 796460

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5007.92 s]
/proc/loadavg: 8.07 8.06 8.00 2/356 4673
/proc/meminfo: memFree=14367668/32951124 swapFree=67111348/67111528
[pid=4509] ppid=4507 vsize=445660 CPUtime=40000.5 cores=0-7
/proc/4509/stat : 4509 (claspmt) R 4507 4509 4178 0 -1 4202496 120248 0 0 0 3999829 219 0 0 25 0 1 0 640601062 456355840 41551 33554432000 4194304 4964800 140737191817504 18446744073709551615 233510323306 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/4509/statm: 111415 41551 476 189 0 105764 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 445660

Child status: 0
Real time (s): 5007.94
CPU time (s): 40000.5
CPU user time (s): 39998.3
CPU system time (s): 2.20466
CPU usage (%): 798.742
Max. virtual memory (cumulated for all children) (KiB): 796460

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39998.3
system time used= 2.20466
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 120249
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 66496
involuntary context switches= 193469

runsolver used 16.7315 second user time and 42.4565 second system time

The end

Launcher Data 

Begin job on node145 at 2011-05-15 19:50:52
IDJOB=3380582
IDBENCH=70775
IDSOLVER=1865
FILE ID=node145/3380582-1305481852
RUNJOBID= node145-1305481200-4192
PBS_JOBID= 13325335
Free space on /tmp= 73668 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT09/APPLICATIONS/bitverif/countbitsrotate/countbitsrotate032.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3380582-1305481852/watcher-3380582-1305481852 -o /tmp/evaluation-result-3380582-1305481852/solver-3380582-1305481852 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380582-1305481852.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 8d76aa747359ea8ccf15726792949452
RANDOM SEED=1829980503

node145.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      14630308 kB
Buffers:        324852 kB
Cached:        8558080 kB
SwapCached:          0 kB
Active:       11146188 kB
Inactive:      6807280 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      14630308 kB
SwapTotal:    67111528 kB
SwapFree:     67111348 kB
Dirty:            1200 kB
Writeback:           0 kB
AnonPages:     9070824 kB
Mapped:          28420 kB
Slab:           258764 kB
PageTables:      41952 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 83664728 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73668 MiB
End job on node145 at 2011-05-15 21:14:22