Trace number 3321484

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40001.6 5006.31

General information on the benchmark

NameCRAFTED/rbsat/random/unforced/
rbsat-v2100c215164g4.cnf
MD5SUM399c54da7337981a88a374a67e87ba30
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2100
Number of clauses215164
Sum of the clauses size432288
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2215094
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 570

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...342792/instance-3321484-1304342792.cnf
0.99/1.01	c Solving...
40000.62/5005.41	
40000.62/5005.41	*** INTERRUPTED! ***
40000.62/5005.41	*** Shutting down threads...
40001.56/5006.30	s UNKNOWN
40001.56/5006.30	
40001.56/5006.30	c INTERRUPTED   : 1
40001.56/5006.30	c Models        : 0+    
40001.56/5006.30	c Time          : 5005.420s (Solving: 5004.41s 1st Model: 0.00s Unsat: 0.00s)
40001.56/5006.30	c CPU Time      : 40000.680s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321484-1304342792/watcher-3321484-1304342792 -o /tmp/evaluation-result-3321484-1304342792/solver-3321484-1304342792 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321484-1304342792.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.89 7.79 7.92 1/203 23864
/proc/meminfo: memFree=24959212/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=21908 CPUtime=0 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 795 0 0 0 0 0 0 0 23 0 1 0 545877966 22433792 708 33554432000 4194304 4964704 140734630464112 18446744073709551615 268272349893 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/23864/statm: 5477 709 386 189 0 351 0

[startup+0.0586161 s]
/proc/loadavg: 6.89 7.79 7.92 1/203 23864
/proc/meminfo: memFree=24959212/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=32260 CPUtime=0.04 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 4328 0 0 0 4 0 0 0 23 0 1 0 545877966 33034240 3073 33554432000 4194304 4964704 140734630464112 18446744073709551615 4618467 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/23864/statm: 8065 3073 397 189 0 2939 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 32260

[startup+0.100609 s]
/proc/loadavg: 6.89 7.79 7.92 1/203 23864
/proc/meminfo: memFree=24959212/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=36088 CPUtime=0.09 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 5220 0 0 0 9 0 0 0 23 0 1 0 545877966 36954112 3964 33554432000 4194304 4964704 140734630464112 18446744073709551615 4607641 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/23864/statm: 9022 3964 399 189 0 3896 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 36088

[startup+0.300573 s]
/proc/loadavg: 6.89 7.79 7.92 1/203 23864
/proc/meminfo: memFree=24959212/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=36088 CPUtime=0.29 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 5220 0 0 0 29 0 0 0 25 0 1 0 545877966 36954112 3964 33554432000 4194304 4964704 140734630464112 18446744073709551615 4607585 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/23864/statm: 9022 3964 399 189 0 3896 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 36088

[startup+0.700501 s]
/proc/loadavg: 6.89 7.79 7.92 1/203 23864
/proc/meminfo: memFree=24959212/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=36616 CPUtime=0.69 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 5380 0 0 0 69 0 0 0 25 0 1 0 545877966 37494784 4124 33554432000 4194304 4964704 140734630464112 18446744073709551615 4600360 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/23864/statm: 9154 4124 400 189 0 4028 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 36616

[startup+1.50036 s]
/proc/loadavg: 6.89 7.79 7.92 2/204 23865
/proc/meminfo: memFree=24943112/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=395564 CPUtime=3.7 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 11171 0 0 0 365 5 0 0 16 0 8 0 545877966 405057536 9472 33554432000 4194304 4964704 140734630464112 18446744073709551615 4627263 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/23864/statm: 98891 9480 464 189 0 93240 0
Current children cumulated CPU time (s) 3.7
Current children cumulated vsize (KiB) 395564

[startup+3.10108 s]
/proc/loadavg: 6.98 7.80 7.92 9/211 23872
/proc/meminfo: memFree=24905112/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=398052 CPUtime=16.49 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 19855 0 0 0 1640 9 0 0 18 0 8 0 545877966 407605248 18156 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/23864/statm: 99513 18156 464 189 0 93862 0
[pid=23864/tid=23866] ppid=23862 vsize=398052 CPUtime=1.92 cores=0-7
/proc/23864/task/23866/stat : 23866 (claspmt) R 23862 23864 23804 0 -1 4202560 1400 0 0 0 191 1 0 0 18 0 8 0 545878068 407605248 18156 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23864/tid=23867] ppid=23862 vsize=398052 CPUtime=1.92 cores=0-7
/proc/23864/task/23867/stat : 23867 (claspmt) R 23862 23864 23804 0 -1 4202560 1696 0 0 0 192 0 0 0 25 0 8 0 545878068 407605248 18156 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23864/tid=23868] ppid=23862 vsize=398052 CPUtime=1.89 cores=0-7
/proc/23864/task/23868/stat : 23868 (claspmt) R 23862 23864 23804 0 -1 4202560 1760 0 0 0 189 0 0 0 20 0 8 0 545878068 407605248 18156 33554432000 4194304 4964704 140734630464112 18446744073709551615 4631168 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23864/tid=23869] ppid=23862 vsize=398052 CPUtime=1.92 cores=0-7
/proc/23864/task/23869/stat : 23869 (claspmt) R 23862 23864 23804 0 -1 4202560 1795 0 0 0 192 0 0 0 18 0 8 0 545878068 407605248 18156 33554432000 4194304 4964704 140734630464112 18446744073709551615 4627338 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23864/tid=23870] ppid=23862 vsize=398052 CPUtime=2.02 cores=0-7
/proc/23864/task/23870/stat : 23870 (claspmt) R 23862 23864 23804 0 -1 4202560 810 0 0 0 201 1 0 0 21 0 8 0 545878068 407605248 18156 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23864/tid=23871] ppid=23862 vsize=398052 CPUtime=1.91 cores=0-7
/proc/23864/task/23871/stat : 23871 (claspmt) R 23862 23864 23804 0 -1 4202560 1739 0 0 0 190 1 0 0 18 0 8 0 545878068 407605248 18156 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670347 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23864/tid=23872] ppid=23862 vsize=398052 CPUtime=1.91 cores=0-7
/proc/23864/task/23872/stat : 23872 (claspmt) R 23862 23864 23804 0 -1 4202560 3362 0 0 0 190 1 0 0 18 0 8 0 545878068 407605248 18156 33554432000 4194304 4964704 140734630464112 18446744073709551615 4660115 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 16.49
Current children cumulated vsize (KiB) 398052

[startup+6.30051 s]
/proc/loadavg: 6.98 7.80 7.92 9/211 23872
/proc/meminfo: memFree=24847212/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=407684 CPUtime=42.06 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 33951 0 0 0 4189 17 0 0 25 0 8 0 545877966 417468416 32252 33554432000 4194304 4964704 140734630464112 18446744073709551615 4379965 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/23864/statm: 101921 32252 466 189 0 96270 0
[pid=23864/tid=23866] ppid=23862 vsize=407684 CPUtime=5.13 cores=0-7
/proc/23864/task/23866/stat : 23866 (claspmt) R 23862 23864 23804 0 -1 4202560 2875 0 0 0 511 2 0 0 25 0 8 0 545878068 417468416 32253 33554432000 4194304 4964704 140734630464112 18446744073709551615 4626894 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23864/tid=23867] ppid=23862 vsize=407684 CPUtime=5.12 cores=0-7
/proc/23864/task/23867/stat : 23867 (claspmt) R 23862 23864 23804 0 -1 4202560 3296 0 0 0 511 1 0 0 25 0 8 0 545878068 417468416 32253 33554432000 4194304 4964704 140734630464112 18446744073709551615 4627910 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23864/tid=23868] ppid=23862 vsize=407684 CPUtime=5.07 cores=0-7
/proc/23864/task/23868/stat : 23868 (claspmt) R 23862 23864 23804 0 -1 4202560 3362 0 0 0 506 1 0 0 25 0 8 0 545878068 417468416 32253 33554432000 4194304 4964704 140734630464112 18446744073709551615 268272350150 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23864/tid=23869] ppid=23862 vsize=407684 CPUtime=5.11 cores=0-7
/proc/23864/task/23869/stat : 23869 (claspmt) R 23862 23864 23804 0 -1 4202560 3723 0 0 0 510 1 0 0 25 0 8 0 545878068 417468416 32253 33554432000 4194304 4964704 140734630464112 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23864/tid=23870] ppid=23862 vsize=407684 CPUtime=5.22 cores=0-7
/proc/23864/task/23870/stat : 23870 (claspmt) R 23862 23864 23804 0 -1 4202560 1474 0 0 0 521 1 0 0 25 0 8 0 545878068 417468416 32253 33554432000 4194304 4964704 140734630464112 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23864/tid=23871] ppid=23862 vsize=407684 CPUtime=5.11 cores=0-7
/proc/23864/task/23871/stat : 23871 (claspmt) R 23862 23864 23804 0 -1 4202560 3079 0 0 0 510 1 0 0 25 0 8 0 545878068 417468416 32253 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23864/tid=23872] ppid=23862 vsize=407684 CPUtime=5.1 cores=0-7
/proc/23864/task/23872/stat : 23872 (claspmt) R 23862 23864 23804 0 -1 4202560 7529 0 0 0 507 3 0 0 25 0 8 0 545878068 417468416 32253 33554432000 4194304 4964704 140734630464112 18446744073709551615 268272350241 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 42.06
Current children cumulated vsize (KiB) 407684

[startup+12.7004 s]
/proc/loadavg: 7.14 7.80 7.92 9/211 23873
/proc/meminfo: memFree=24770256/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=416956 CPUtime=93.22 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 51251 0 0 0 9296 26 0 0 25 0 8 0 545877966 426962944 49552 33554432000 4194304 4964704 140734630464112 18446744073709551615 4321016 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/23864/statm: 104239 49552 466 189 0 98588 0
[pid=23864/tid=23866] ppid=23862 vsize=416956 CPUtime=11.52 cores=0-7
/proc/23864/task/23866/stat : 23866 (claspmt) R 23862 23864 23804 0 -1 4202560 4615 0 0 0 1149 3 0 0 25 0 8 0 545878068 426962944 49552 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670263 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23864/tid=23867] ppid=23862 vsize=416956 CPUtime=11.52 cores=0-7
/proc/23864/task/23867/stat : 23867 (claspmt) R 23862 23864 23804 0 -1 4202560 4990 0 0 0 1150 2 0 0 25 0 8 0 545878068 426962944 49552 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23864/tid=23868] ppid=23862 vsize=416956 CPUtime=11.43 cores=0-7
/proc/23864/task/23868/stat : 23868 (claspmt) R 23862 23864 23804 0 -1 4202560 4726 0 0 0 1141 2 0 0 25 0 8 0 545878068 426962944 49552 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23864/tid=23869] ppid=23862 vsize=416956 CPUtime=11.52 cores=0-7
/proc/23864/task/23869/stat : 23869 (claspmt) R 23862 23864 23804 0 -1 4202560 4892 0 0 0 1150 2 0 0 25 0 8 0 545878068 426962944 49552 33554432000 4194304 4964704 140734630464112 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23864/tid=23870] ppid=23862 vsize=416956 CPUtime=11.62 cores=0-7
/proc/23864/task/23870/stat : 23870 (claspmt) R 23862 23864 23804 0 -1 4202560 3219 0 0 0 1160 2 0 0 25 0 8 0 545878068 426962944 49552 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23864/tid=23871] ppid=23862 vsize=416956 CPUtime=11.51 cores=0-7
/proc/23864/task/23871/stat : 23871 (claspmt) R 23862 23864 23804 0 -1 4202560 4841 0 0 0 1148 3 0 0 25 0 8 0 545878068 426962944 49552 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23864/tid=23872] ppid=23862 vsize=416956 CPUtime=11.5 cores=0-7
/proc/23864/task/23872/stat : 23872 (claspmt) R 23862 23864 23804 0 -1 4202560 13483 0 0 0 1145 5 0 0 25 0 8 0 545878068 426962944 49552 33554432000 4194304 4964704 140734630464112 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 93.22
Current children cumulated vsize (KiB) 416956

[startup+25.5011 s]
/proc/loadavg: 7.27 7.81 7.92 9/211 23873
/proc/meminfo: memFree=24735324/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=423968 CPUtime=195.51 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 59708 0 0 0 19520 31 0 0 25 0 8 0 545877966 434143232 58009 33554432000 4194304 4964704 140734630464112 18446744073709551615 4321102 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/23864/statm: 105992 58009 466 189 0 100341 0
[pid=23864/tid=23866] ppid=23862 vsize=423968 CPUtime=24.32 cores=0-7
/proc/23864/task/23866/stat : 23866 (claspmt) R 23862 23864 23804 0 -1 4202560 5718 0 0 0 2428 4 0 0 25 0 8 0 545878068 434143232 58009 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670263 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23864/tid=23867] ppid=23862 vsize=423968 CPUtime=24.32 cores=0-7
/proc/23864/task/23867/stat : 23867 (claspmt) R 23862 23864 23804 0 -1 4202560 6509 0 0 0 2429 3 0 0 25 0 8 0 545878068 434143232 58009 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23864/tid=23868] ppid=23862 vsize=423968 CPUtime=24.14 cores=0-7
/proc/23864/task/23868/stat : 23868 (claspmt) R 23862 23864 23804 0 -1 4202560 5213 0 0 0 2412 2 0 0 25 0 8 0 545878068 434143232 58009 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23864/tid=23869] ppid=23862 vsize=423968 CPUtime=24.31 cores=0-7
/proc/23864/task/23869/stat : 23869 (claspmt) R 23862 23864 23804 0 -1 4202560 5863 0 0 0 2429 2 0 0 25 0 8 0 545878068 434143232 58009 33554432000 4194304 4964704 140734630464112 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23864/tid=23870] ppid=23862 vsize=423968 CPUtime=24.42 cores=0-7
/proc/23864/task/23870/stat : 23870 (claspmt) R 23862 23864 23804 0 -1 4202560 3926 0 0 0 2440 2 0 0 25 0 8 0 545878068 434143232 58009 33554432000 4194304 4964704 140734630464112 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23864/tid=23871] ppid=23862 vsize=423968 CPUtime=24.3 cores=0-7
/proc/23864/task/23871/stat : 23871 (claspmt) R 23862 23864 23804 0 -1 4202560 5199 0 0 0 2427 3 0 0 25 0 8 0 545878068 434143232 58010 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670263 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23864/tid=23872] ppid=23862 vsize=423968 CPUtime=24.3 cores=0-7
/proc/23864/task/23872/stat : 23872 (claspmt) R 23862 23864 23804 0 -1 4202560 16158 0 0 0 2423 7 0 0 25 0 8 0 545878068 434143232 58010 33554432000 4194304 4964704 140734630464112 18446744073709551615 268272350241 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 195.51
Current children cumulated vsize (KiB) 423968

[startup+51.1005 s]
/proc/loadavg: 7.52 7.82 7.92 9/211 23874
/proc/meminfo: memFree=24649288/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=440152 CPUtime=400.13 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 81200 0 0 0 39965 48 0 0 25 0 8 0 545877966 450715648 79466 33554432000 4194304 4964704 140734630464112 18446744073709551615 4320848 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/23864/statm: 110038 79466 466 189 0 104387 0
[pid=23864/tid=23866] ppid=23862 vsize=440152 CPUtime=49.91 cores=0-7
/proc/23864/task/23866/stat : 23866 (claspmt) R 23862 23864 23804 0 -1 4202560 8704 0 0 0 4985 6 0 0 25 0 8 0 545878068 450715648 79466 33554432000 4194304 4964704 140734630464112 18446744073709551615 4320984 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23864/tid=23867] ppid=23862 vsize=440152 CPUtime=49.92 cores=0-7
/proc/23864/task/23867/stat : 23867 (claspmt) R 23862 23864 23804 0 -1 4202560 9628 0 0 0 4988 4 0 0 25 0 8 0 545878068 450715648 79466 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670454 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23864/tid=23868] ppid=23862 vsize=440152 CPUtime=49.59 cores=0-7
/proc/23864/task/23868/stat : 23868 (claspmt) R 23862 23864 23804 0 -1 4202560 6010 0 0 0 4956 3 0 0 25 0 8 0 545878068 450715648 79466 33554432000 4194304 4964704 140734630464112 18446744073709551615 4627170 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23864/tid=23869] ppid=23862 vsize=440152 CPUtime=49.91 cores=0-7
/proc/23864/task/23869/stat : 23869 (claspmt) R 23862 23864 23804 0 -1 4202560 7449 0 0 0 4988 3 0 0 25 0 8 0 545878068 450715648 79466 33554432000 4194304 4964704 140734630464112 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23864/tid=23870] ppid=23862 vsize=440152 CPUtime=50.02 cores=0-7
/proc/23864/task/23870/stat : 23870 (claspmt) R 23862 23864 23804 0 -1 4202560 5486 0 0 0 4998 4 0 0 25 0 8 0 545878068 450715648 79466 33554432000 4194304 4964704 140734630464112 18446744073709551615 4669024 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23864/tid=23871] ppid=23862 vsize=440152 CPUtime=49.89 cores=0-7
/proc/23864/task/23871/stat : 23871 (claspmt) R 23862 23864 23804 0 -1 4202560 6759 0 0 0 4985 4 0 0 25 0 8 0 545878068 450715648 79466 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23864/tid=23872] ppid=23862 vsize=440152 CPUtime=49.9 cores=0-7
/proc/23864/task/23872/stat : 23872 (claspmt) R 23862 23864 23804 0 -1 4202560 23740 0 0 0 4976 14 0 0 25 0 8 0 545878068 450715648 79466 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670243 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 400.13
Current children cumulated vsize (KiB) 440152

[startup+102.307 s]
/proc/loadavg: 7.81 7.85 7.92 9/211 23875
/proc/meminfo: memFree=24524628/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=726932 CPUtime=809.4 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 111488 0 0 0 80863 77 0 0 25 0 8 0 545877966 744378368 109716 33554432000 4194304 4964704 140734630464112 18446744073709551615 4627204 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/23864/statm: 181733 109716 466 189 0 176082 0
[pid=23864/tid=23866] ppid=23862 vsize=726932 CPUtime=101.1 cores=0-7
/proc/23864/task/23866/stat : 23866 (claspmt) R 23862 23864 23804 0 -1 4202560 11353 0 0 0 10102 8 0 0 25 0 8 0 545878068 744378368 109716 33554432000 4194304 4964704 140734630464112 18446744073709551615 4668907 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23864/tid=23867] ppid=23862 vsize=726932 CPUtime=101.13 cores=0-7
/proc/23864/task/23867/stat : 23867 (claspmt) R 23862 23864 23804 0 -1 4202560 11272 0 0 0 10107 6 0 0 25 0 8 0 545878068 744378368 109716 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23864/tid=23868] ppid=23862 vsize=726932 CPUtime=100.45 cores=0-7
/proc/23864/task/23868/stat : 23868 (claspmt) R 23862 23864 23804 0 -1 4202560 7636 0 0 0 10040 5 0 0 25 0 8 0 545878068 744378368 109716 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0

################
# More data... #
################

[pid=23864] ppid=23862 vsize=2043280 CPUtime=37737.7 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 480475 0 0 0 3773189 582 0 0 25 0 8 0 545877966 2092318720 472012 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/23864/statm: 510820 472012 466 189 0 505169 0
[pid=23864/tid=23866] ppid=23862 vsize=2043280 CPUtime=4718.53 cores=0-7
/proc/23864/task/23866/stat : 23866 (claspmt) R 23862 23864 23804 0 -1 4202560 45322 0 0 0 471801 52 0 0 25 0 8 0 545878068 2092318720 472012 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23864/tid=23867] ppid=23862 vsize=2043280 CPUtime=4720.33 cores=0-7
/proc/23864/task/23867/stat : 23867 (claspmt) R 23862 23864 23804 0 -1 4202560 38927 0 0 0 471999 34 0 0 25 0 8 0 545878068 2092318720 472012 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670193 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23864/tid=23868] ppid=23862 vsize=2043280 CPUtime=4719.69 cores=0-7
/proc/23864/task/23868/stat : 23868 (claspmt) R 23862 23864 23804 0 -1 4202560 27032 0 0 0 471931 38 0 0 25 0 8 0 545878068 2092318720 472012 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=23864/tid=23869] ppid=23862 vsize=2043280 CPUtime=4721.08 cores=0-7
/proc/23864/task/23869/stat : 23869 (claspmt) R 23862 23864 23804 0 -1 4202560 28665 0 0 0 472077 31 0 0 25 0 8 0 545878068 2092318720 472012 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23864/tid=23870] ppid=23862 vsize=2043280 CPUtime=4721.2 cores=0-7
/proc/23864/task/23870/stat : 23870 (claspmt) R 23862 23864 23804 0 -1 4202560 27900 0 0 0 472088 32 0 0 25 0 8 0 545878068 2092318720 472012 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23864/tid=23871] ppid=23862 vsize=2043280 CPUtime=4720.88 cores=0-7
/proc/23864/task/23871/stat : 23871 (claspmt) R 23862 23864 23804 0 -1 4202560 23363 0 0 0 472060 28 0 0 25 0 8 0 545878068 2092318720 472012 33554432000 4194304 4964704 140734630464112 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23864/tid=23872] ppid=23862 vsize=2043280 CPUtime=4694.54 cores=0-7
/proc/23864/task/23872/stat : 23872 (claspmt) R 23862 23864 23804 0 -1 4202560 221559 0 0 0 469172 282 0 0 25 0 8 0 545878068 2092318720 472012 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37737.7
Current children cumulated vsize (KiB) 2043280

[startup+4782.3 s]
/proc/loadavg: 8.04 8.03 8.00 9/211 24012
/proc/meminfo: memFree=23033988/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=2044028 CPUtime=38217.2 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 489673 0 0 0 3821126 596 0 0 25 0 8 0 545877966 2093084672 481210 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670419 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/23864/statm: 511007 481210 466 189 0 505356 0
[pid=23864/tid=23866] ppid=23862 vsize=2044028 CPUtime=4778.11 cores=0-7
/proc/23864/task/23866/stat : 23866 (claspmt) R 23862 23864 23804 0 -1 4202560 46040 0 0 0 477758 53 0 0 25 0 8 0 545878068 2093084672 481210 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23864/tid=23867] ppid=23862 vsize=2044028 CPUtime=4780.33 cores=0-7
/proc/23864/task/23867/stat : 23867 (claspmt) R 23862 23864 23804 0 -1 4202560 39060 0 0 0 477998 35 0 0 25 0 8 0 545878068 2093084672 481210 33554432000 4194304 4964704 140734630464112 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23864/tid=23868] ppid=23862 vsize=2044028 CPUtime=4779.68 cores=0-7
/proc/23864/task/23868/stat : 23868 (claspmt) R 23862 23864 23804 0 -1 4202560 27119 0 0 0 477930 38 0 0 25 0 8 0 545878068 2093084672 481210 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=23864/tid=23869] ppid=23862 vsize=2044028 CPUtime=4781.08 cores=0-7
/proc/23864/task/23869/stat : 23869 (claspmt) R 23862 23864 23804 0 -1 4202560 28786 0 0 0 478077 31 0 0 25 0 8 0 545878068 2093084672 481210 33554432000 4194304 4964704 140734630464112 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23864/tid=23870] ppid=23862 vsize=2044028 CPUtime=4781.19 cores=0-7
/proc/23864/task/23870/stat : 23870 (claspmt) R 23862 23864 23804 0 -1 4202560 28711 0 0 0 478085 34 0 0 25 0 8 0 545878068 2093084672 481210 33554432000 4194304 4964704 140734630464112 18446744073709551615 4644844 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23864/tid=23871] ppid=23862 vsize=2044028 CPUtime=4780.87 cores=0-7
/proc/23864/task/23871/stat : 23871 (claspmt) R 23862 23864 23804 0 -1 4202560 23444 0 0 0 478059 28 0 0 25 0 8 0 545878068 2093084672 481210 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23864/tid=23872] ppid=23862 vsize=2044028 CPUtime=4754.53 cores=0-7
/proc/23864/task/23872/stat : 23872 (claspmt) R 23862 23864 23804 0 -1 4202560 226855 0 0 0 475164 289 0 0 25 0 8 0 545878068 2093084672 481210 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38217.2
Current children cumulated vsize (KiB) 2044028

[startup+4842.3 s]
/proc/loadavg: 8.05 8.04 8.00 9/211 24013
/proc/meminfo: memFree=23032344/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=2044660 CPUtime=38696.8 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 490117 0 0 0 3869084 597 0 0 25 0 8 0 545877966 2093731840 481654 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670298 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/23864/statm: 511165 481654 466 189 0 505514 0
[pid=23864/tid=23866] ppid=23862 vsize=2044660 CPUtime=4837.72 cores=0-7
/proc/23864/task/23866/stat : 23866 (claspmt) R 23862 23864 23804 0 -1 4202560 46042 0 0 0 483719 53 0 0 25 0 8 0 545878068 2093731840 481654 33554432000 4194304 4964704 140734630464112 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23864/tid=23867] ppid=23862 vsize=2044660 CPUtime=4840.33 cores=0-7
/proc/23864/task/23867/stat : 23867 (claspmt) R 23862 23864 23804 0 -1 4202560 39149 0 0 0 483998 35 0 0 25 0 8 0 545878068 2093731840 481654 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23864/tid=23868] ppid=23862 vsize=2044660 CPUtime=4839.67 cores=0-7
/proc/23864/task/23868/stat : 23868 (claspmt) R 23862 23864 23804 0 -1 4202560 27238 0 0 0 483929 38 0 0 25 0 8 0 545878068 2093731840 481654 33554432000 4194304 4964704 140734630464112 18446744073709551615 4378499 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=23864/tid=23869] ppid=23862 vsize=2044660 CPUtime=4841.08 cores=0-7
/proc/23864/task/23869/stat : 23869 (claspmt) R 23862 23864 23804 0 -1 4202560 28959 0 0 0 484077 31 0 0 25 0 8 0 545878068 2093731840 481654 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23864/tid=23870] ppid=23862 vsize=2044660 CPUtime=4841.17 cores=0-7
/proc/23864/task/23870/stat : 23870 (claspmt) R 23862 23864 23804 0 -1 4202560 28770 0 0 0 484083 34 0 0 25 0 8 0 545878068 2093731840 481654 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23864/tid=23871] ppid=23862 vsize=2044660 CPUtime=4840.87 cores=0-7
/proc/23864/task/23871/stat : 23871 (claspmt) R 23862 23864 23804 0 -1 4202560 23444 0 0 0 484059 28 0 0 25 0 8 0 545878068 2093731840 481654 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23864/tid=23872] ppid=23862 vsize=2044660 CPUtime=4814.53 cores=0-7
/proc/23864/task/23872/stat : 23872 (claspmt) R 23862 23864 23804 0 -1 4202560 226857 0 0 0 481163 290 0 0 25 0 8 0 545878068 2093731840 481654 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670243 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38696.8
Current children cumulated vsize (KiB) 2044660

[startup+4902.3 s]
/proc/loadavg: 8.13 8.06 8.01 9/211 24015
/proc/meminfo: memFree=23030828/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=2045300 CPUtime=39176.4 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 490426 0 0 0 3917047 598 0 0 25 0 8 0 545877966 2094387200 481948 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670419 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/23864/statm: 511325 481948 466 189 0 505674 0
[pid=23864/tid=23866] ppid=23862 vsize=2045300 CPUtime=4897.35 cores=0-7
/proc/23864/task/23866/stat : 23866 (claspmt) R 23862 23864 23804 0 -1 4202560 46049 0 0 0 489682 53 0 0 25 0 8 0 545878068 2094387200 481948 33554432000 4194304 4964704 140734630464112 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23864/tid=23867] ppid=23862 vsize=2045300 CPUtime=4900.34 cores=0-7
/proc/23864/task/23867/stat : 23867 (claspmt) R 23862 23864 23804 0 -1 4202560 39354 0 0 0 489999 35 0 0 25 0 8 0 545878068 2094387200 481948 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23864/tid=23868] ppid=23862 vsize=2045300 CPUtime=4899.67 cores=0-7
/proc/23864/task/23868/stat : 23868 (claspmt) R 23862 23864 23804 0 -1 4202560 27238 0 0 0 489929 38 0 0 25 0 8 0 545878068 2094387200 481948 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=23864/tid=23869] ppid=23862 vsize=2045300 CPUtime=4901.08 cores=0-7
/proc/23864/task/23869/stat : 23869 (claspmt) R 23862 23864 23804 0 -1 4202560 28964 0 0 0 490077 31 0 0 25 0 8 0 545878068 2094387200 481948 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23864/tid=23870] ppid=23862 vsize=2045300 CPUtime=4901.18 cores=0-7
/proc/23864/task/23870/stat : 23870 (claspmt) R 23862 23864 23804 0 -1 4202560 28775 0 0 0 490084 34 0 0 25 0 8 0 545878068 2094387200 481948 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23864/tid=23871] ppid=23862 vsize=2045300 CPUtime=4900.87 cores=0-7
/proc/23864/task/23871/stat : 23871 (claspmt) R 23862 23864 23804 0 -1 4202560 23530 0 0 0 490059 28 0 0 25 0 8 0 545878068 2094387200 481948 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23864/tid=23872] ppid=23862 vsize=2045300 CPUtime=4874.53 cores=0-7
/proc/23864/task/23872/stat : 23872 (claspmt) R 23862 23864 23804 0 -1 4202560 226857 0 0 0 487163 290 0 0 25 0 8 0 545878068 2094387200 481948 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39176.4
Current children cumulated vsize (KiB) 2045300

[startup+4962.3 s]
/proc/loadavg: 8.12 8.06 8.00 9/211 24017
/proc/meminfo: memFree=23028200/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=2047632 CPUtime=39656.1 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 491151 0 0 0 3965008 598 0 0 25 0 8 0 545877966 2096775168 482673 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670343 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/23864/statm: 511908 482673 466 189 0 506257 0
[pid=23864/tid=23866] ppid=23862 vsize=2047632 CPUtime=4956.96 cores=0-7
/proc/23864/task/23866/stat : 23866 (claspmt) R 23862 23864 23804 0 -1 4202560 46084 0 0 0 495643 53 0 0 25 0 8 0 545878068 2096775168 482673 33554432000 4194304 4964704 140734630464112 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23864/tid=23867] ppid=23862 vsize=2047632 CPUtime=4960.34 cores=0-7
/proc/23864/task/23867/stat : 23867 (claspmt) R 23862 23864 23804 0 -1 4202560 39609 0 0 0 495999 35 0 0 25 0 8 0 545878068 2096775168 482673 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23864/tid=23868] ppid=23862 vsize=2047632 CPUtime=4959.68 cores=0-7
/proc/23864/task/23868/stat : 23868 (claspmt) R 23862 23864 23804 0 -1 4202560 27285 0 0 0 495930 38 0 0 25 0 8 0 545878068 2096775168 482673 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=23864/tid=23869] ppid=23862 vsize=2047632 CPUtime=4961.08 cores=0-7
/proc/23864/task/23869/stat : 23869 (claspmt) R 23862 23864 23804 0 -1 4202560 29241 0 0 0 496077 31 0 0 25 0 8 0 545878068 2096775168 482673 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23864/tid=23870] ppid=23862 vsize=2047632 CPUtime=4961.18 cores=0-7
/proc/23864/task/23870/stat : 23870 (claspmt) R 23862 23864 23804 0 -1 4202560 28775 0 0 0 496084 34 0 0 25 0 8 0 545878068 2096775168 482673 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23864/tid=23871] ppid=23862 vsize=2047632 CPUtime=4960.87 cores=0-7
/proc/23864/task/23871/stat : 23871 (claspmt) R 23862 23864 23804 0 -1 4202560 23637 0 0 0 496059 28 0 0 25 0 8 0 545878068 2096775168 482673 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23864/tid=23872] ppid=23862 vsize=2047632 CPUtime=4934.53 cores=0-7
/proc/23864/task/23872/stat : 23872 (claspmt) R 23862 23864 23804 0 -1 4202560 226858 0 0 0 493163 290 0 0 25 0 8 0 545878068 2096775168 482673 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39656.1
Current children cumulated vsize (KiB) 2047632



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5005.4 s]
/proc/loadavg: 8.05 8.05 8.00 9/211 24018
/proc/meminfo: memFree=23027060/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=2048296 CPUtime=40000.6 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 491401 0 0 0 3999464 598 0 0 25 0 8 0 545877966 2097455104 482923 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/23864/statm: 512074 482923 466 189 0 506423 0
[pid=23864/tid=23866] ppid=23862 vsize=2048296 CPUtime=4999.79 cores=0-7
/proc/23864/task/23866/stat : 23866 (claspmt) R 23862 23864 23804 0 -1 4202560 46116 0 0 0 499926 53 0 0 25 0 8 0 545878068 2097455104 482923 33554432000 4194304 4964704 140734630464112 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23864/tid=23867] ppid=23862 vsize=2048296 CPUtime=5003.44 cores=0-7
/proc/23864/task/23867/stat : 23867 (claspmt) R 23862 23864 23804 0 -1 4202560 39712 0 0 0 500309 35 0 0 25 0 8 0 545878068 2097455104 482923 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23864/tid=23868] ppid=23862 vsize=2048296 CPUtime=5002.78 cores=0-7
/proc/23864/task/23868/stat : 23868 (claspmt) R 23862 23864 23804 0 -1 4202560 27288 0 0 0 500240 38 0 0 25 0 8 0 545878068 2097455104 482923 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670263 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=23864/tid=23869] ppid=23862 vsize=2048296 CPUtime=5004.19 cores=0-7
/proc/23864/task/23869/stat : 23869 (claspmt) R 23862 23864 23804 0 -1 4202560 29327 0 0 0 500388 31 0 0 25 0 8 0 545878068 2097455104 482923 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23864/tid=23870] ppid=23862 vsize=2048296 CPUtime=5004.29 cores=0-7
/proc/23864/task/23870/stat : 23870 (claspmt) R 23862 23864 23804 0 -1 4202560 28775 0 0 0 500395 34 0 0 25 0 8 0 545878068 2097455104 482923 33554432000 4194304 4964704 140734630464112 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23864/tid=23871] ppid=23862 vsize=2048296 CPUtime=5003.97 cores=0-7
/proc/23864/task/23871/stat : 23871 (claspmt) R 23862 23864 23804 0 -1 4202560 23663 0 0 0 500369 28 0 0 25 0 8 0 545878068 2097455104 482923 33554432000 4194304 4964704 140734630464112 18446744073709551615 4321105 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23864/tid=23872] ppid=23862 vsize=2048296 CPUtime=4977.63 cores=0-7
/proc/23864/task/23872/stat : 23872 (claspmt) R 23862 23864 23804 0 -1 4202560 226858 0 0 0 497473 290 0 0 25 0 8 0 545878068 2097455104 482923 33554432000 4194304 4964704 140734630464112 18446744073709551615 4670230 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 2048296

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5005.8 s]
/proc/loadavg: 8.05 8.05 8.00 9/211 24018
/proc/meminfo: memFree=23027060/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=2037088 CPUtime=40001.1 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 491411 0 0 0 3999508 598 0 0 25 0 1 0 545877966 2085978112 479769 33554432000 4194304 4964704 140734630464112 18446744073709551615 268272344267 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/23864/statm: 509272 479769 476 189 0 503621 0
Current children cumulated CPU time (s) 40001.1
Current children cumulated vsize (KiB) 2037088

[startup+5006.2 s]
/proc/loadavg: 8.05 8.05 8.00 2/205 24019
/proc/meminfo: memFree=23038392/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=2031616 CPUtime=40001.5 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 491411 0 0 0 3999548 598 0 0 25 0 1 0 545877966 2080374784 477728 33554432000 4194304 4964704 140734630464112 18446744073709551615 268272345659 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/23864/statm: 507904 477728 476 189 0 502253 0
Current children cumulated CPU time (s) 40001.5
Current children cumulated vsize (KiB) 2031616

[startup+5006.3 s]
/proc/loadavg: 8.05 8.05 8.00 2/205 24019
/proc/meminfo: memFree=23038392/32950928 swapFree=67111464/67111528
[pid=23864] ppid=23862 vsize=395300 CPUtime=40001.6 cores=0-7
/proc/23864/stat : 23864 (claspmt) R 23862 23864 23804 0 -1 4202496 491411 0 0 0 3999550 606 0 0 25 0 1 0 545877966 404787200 20639 33554432000 4194304 4964704 140734630464112 18446744073709551615 268272732519 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/23864/statm: 98825 20162 476 189 0 93174 0
Current children cumulated CPU time (s) 40001.6
Current children cumulated vsize (KiB) 395300

Child status: 0
Real time (s): 5006.31
CPU time (s): 40001.6
CPU user time (s): 39995.5
CPU system time (s): 6.07208
CPU usage (%): 799.024
Max. virtual memory (cumulated for all children) (KiB): 2048296

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39995.5
system time used= 6.07208
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 491412
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 74786
involuntary context switches= 189273

runsolver used 8.49971 second user time and 23.3664 second system time

The end

Launcher Data 

Begin job on node122 at 2011-05-02 15:26:32
IDJOB=3321484
IDBENCH=71308
IDSOLVER=1711
FILE ID=node122/3321484-1304342792
RUNJOBID= node122-1304342790-23818
PBS_JOBID= 13178068
Free space on /tmp= 73648 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT09/CRAFTED/rbsat/random/unforced/rbsat-v2100c215164g4.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321484-1304342792/watcher-3321484-1304342792 -o /tmp/evaluation-result-3321484-1304342792/solver-3321484-1304342792 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321484-1304342792.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 399c54da7337981a88a374a67e87ba30
RANDOM SEED=22611817

node122.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      24959484 kB
Buffers:        376292 kB
Cached:        1941812 kB
SwapCached:         64 kB
Active:        5758224 kB
Inactive:      1869272 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      24959484 kB
SwapTotal:    67111528 kB
SwapFree:     67111464 kB
Dirty:            4040 kB
Writeback:           0 kB
AnonPages:     5309452 kB
Mapped:          18732 kB
Slab:           284956 kB
PageTables:      15452 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:  5687228 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73648 MiB
End job on node122 at 2011-05-02 16:49:59