Trace number 3321515

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40000.5 5006.09

General information on the benchmark

NameCRAFTED/rbsat/crafted/forced/
rbsat-v945c61409gyes1.cnf
MD5SUM26b9098de258b0434de90e744962df46
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark10.76
Satisfiable
(Un)Satisfiability was proved
Number of variables945
Number of clauses61409
Sum of the clauses size123673
Maximum clause length21
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 261364
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...357777/instance-3321515-1304357777.cnf
0.09/0.18	c Solving...
40000.20/5005.81	
40000.20/5005.81	*** INTERRUPTED! ***
40000.20/5005.81	*** Shutting down threads...
40000.44/5006.09	s UNKNOWN
40000.44/5006.09	
40000.44/5006.09	c INTERRUPTED   : 1
40000.44/5006.09	c Models        : 0+    
40000.44/5006.09	c Time          : 5005.813s (Solving: 5005.64s 1st Model: 0.00s Unsat: 0.00s)
40000.44/5006.09	c CPU Time      : 40000.240s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321515-1304357777/watcher-3321515-1304357777 -o /tmp/evaluation-result-3321515-1304357777/solver-3321515-1304357777 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321515-1304357777.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.05 8.09 8.02 1/254 11827
/proc/meminfo: memFree=16997216/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=22148 CPUtime=0 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 858 0 0 0 0 0 0 0 23 0 1 0 547380066 22679552 744 33554432000 4194304 4964704 140734704709232 18446744073709551615 212435677893 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/11827/statm: 5537 746 386 189 0 411 0

[startup+0.074385 s]
/proc/loadavg: 8.05 8.09 8.02 1/254 11827
/proc/meminfo: memFree=16997216/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=24788 CPUtime=0.06 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 1687 0 0 0 6 0 0 0 23 0 1 0 547380066 25382912 1429 33554432000 4194304 4964704 140734704709232 18446744073709551615 4607641 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/11827/statm: 6197 1429 399 189 0 1071 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 24788

[startup+0.100378 s]
/proc/loadavg: 8.05 8.09 8.02 1/254 11827
/proc/meminfo: memFree=16997216/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=24920 CPUtime=0.09 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 1723 0 0 0 9 0 0 0 23 0 1 0 547380066 25518080 1465 33554432000 4194304 4964704 140734704709232 18446744073709551615 4600353 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/11827/statm: 6230 1465 400 189 0 1104 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 24920

[startup+0.301727 s]
/proc/loadavg: 8.05 8.09 8.02 1/254 11827
/proc/meminfo: memFree=16997216/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=319404 CPUtime=0.84 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 3842 0 0 0 82 2 0 0 18 0 8 0 547380066 327069696 3439 33554432000 4194304 4964704 140734704709232 18446744073709551615 4660059 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/11827/statm: 79851 3439 464 189 0 74200 0
Current children cumulated CPU time (s) 0.84
Current children cumulated vsize (KiB) 319404

[startup+0.700226 s]
/proc/loadavg: 8.05 8.09 8.02 1/254 11827
/proc/meminfo: memFree=16997216/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=322604 CPUtime=4.02 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 8265 0 0 0 399 3 0 0 18 0 8 0 547380066 330346496 7862 33554432000 4194304 4964704 140734704709232 18446744073709551615 4321097 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/11827/statm: 80651 7863 464 189 0 75000 0
Current children cumulated CPU time (s) 4.02
Current children cumulated vsize (KiB) 322604

[startup+1.50108 s]
/proc/loadavg: 8.04 8.09 8.02 9/262 11835
/proc/meminfo: memFree=16964796/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=325628 CPUtime=10.41 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 11616 0 0 0 1036 5 0 0 19 0 8 0 547380066 333443072 11213 33554432000 4194304 4964704 140734704709232 18446744073709551615 4320955 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/11827/statm: 81407 11213 464 189 0 75756 0
[pid=11827/tid=11829] ppid=11825 vsize=325628 CPUtime=1.27 cores=0-7
/proc/11827/task/11829/stat : 11829 (claspmt) R 11825 11827 10653 0 -1 4202560 1009 0 0 0 127 0 0 0 19 0 8 0 547380085 333443072 11213 33554432000 4194304 4964704 140734704709232 18446744073709551615 4627252 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11827/tid=11830] ppid=11825 vsize=325628 CPUtime=1.26 cores=0-7
/proc/11827/task/11830/stat : 11830 (claspmt) R 11825 11827 10653 0 -1 4202560 1188 0 0 0 126 0 0 0 19 0 8 0 547380085 333443072 11213 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670357 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11827/tid=11831] ppid=11825 vsize=325628 CPUtime=1.27 cores=0-7
/proc/11827/task/11831/stat : 11831 (claspmt) R 11825 11827 10653 0 -1 4202560 1145 0 0 0 127 0 0 0 19 0 8 0 547380085 333443072 11213 33554432000 4194304 4964704 140734704709232 18446744073709551615 4320849 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11827/tid=11832] ppid=11825 vsize=325628 CPUtime=1.26 cores=0-7
/proc/11827/task/11832/stat : 11832 (claspmt) R 11825 11827 10653 0 -1 4202560 1148 0 0 0 126 0 0 0 25 0 8 0 547380085 333443072 11213 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670263 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11827/tid=11833] ppid=11825 vsize=325628 CPUtime=1.31 cores=0-7
/proc/11827/task/11833/stat : 11833 (claspmt) R 11825 11827 10653 0 -1 4202560 659 0 0 0 131 0 0 0 19 0 8 0 547380085 333443072 11213 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670258 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11827/tid=11834] ppid=11825 vsize=325628 CPUtime=1.25 cores=0-7
/proc/11827/task/11834/stat : 11834 (claspmt) R 11825 11827 10653 0 -1 4202560 1413 0 0 0 125 0 0 0 19 0 8 0 547380085 333443072 11213 33554432000 4194304 4964704 140734704709232 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11827/tid=11835] ppid=11825 vsize=325628 CPUtime=1.26 cores=0-7
/proc/11827/task/11835/stat : 11835 (claspmt) R 11825 11827 10653 0 -1 4202560 1969 0 0 0 126 0 0 0 19 0 8 0 547380085 333443072 11213 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 10.41
Current children cumulated vsize (KiB) 325628

[startup+3.10077 s]
/proc/loadavg: 8.04 8.09 8.02 9/262 11835
/proc/meminfo: memFree=16952036/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=326496 CPUtime=23.2 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 13496 0 0 0 2314 6 0 0 25 0 8 0 547380066 334331904 13093 33554432000 4194304 4964704 140734704709232 18446744073709551615 4320976 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/11827/statm: 81624 13093 466 189 0 75973 0
[pid=11827/tid=11829] ppid=11825 vsize=326496 CPUtime=2.87 cores=0-7
/proc/11827/task/11829/stat : 11829 (claspmt) R 11825 11827 10653 0 -1 4202560 1229 0 0 0 287 0 0 0 22 0 8 0 547380085 334331904 13094 33554432000 4194304 4964704 140734704709232 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11827/tid=11830] ppid=11825 vsize=326496 CPUtime=2.86 cores=0-7
/proc/11827/task/11830/stat : 11830 (claspmt) R 11825 11827 10653 0 -1 4202560 1388 0 0 0 286 0 0 0 22 0 8 0 547380085 334331904 13094 33554432000 4194304 4964704 140734704709232 18446744073709551615 212435678241 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11827/tid=11831] ppid=11825 vsize=326496 CPUtime=2.87 cores=0-7
/proc/11827/task/11831/stat : 11831 (claspmt) R 11825 11827 10653 0 -1 4202560 1349 0 0 0 287 0 0 0 22 0 8 0 547380085 334331904 13094 33554432000 4194304 4964704 140734704709232 18446744073709551615 4677965 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11827/tid=11832] ppid=11825 vsize=326496 CPUtime=2.86 cores=0-7
/proc/11827/task/11832/stat : 11832 (claspmt) R 11825 11827 10653 0 -1 4202560 1284 0 0 0 286 0 0 0 25 0 8 0 547380085 334331904 13094 33554432000 4194304 4964704 140734704709232 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11827/tid=11833] ppid=11825 vsize=326496 CPUtime=2.91 cores=0-7
/proc/11827/task/11833/stat : 11833 (claspmt) R 11825 11827 10653 0 -1 4202560 907 0 0 0 291 0 0 0 24 0 8 0 547380085 334331904 13094 33554432000 4194304 4964704 140734704709232 18446744073709551615 4465641 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11827/tid=11834] ppid=11825 vsize=326496 CPUtime=2.85 cores=0-7
/proc/11827/task/11834/stat : 11834 (claspmt) R 11825 11827 10653 0 -1 4202560 1603 0 0 0 284 1 0 0 23 0 8 0 547380085 334331904 13094 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670241 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11827/tid=11835] ppid=11825 vsize=326496 CPUtime=2.87 cores=0-7
/proc/11827/task/11835/stat : 11835 (claspmt) R 11825 11827 10653 0 -1 4202560 2498 0 0 0 286 1 0 0 22 0 8 0 547380085 334331904 13094 33554432000 4194304 4964704 140734704709232 18446744073709551615 4658370 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 23.2
Current children cumulated vsize (KiB) 326496

[startup+6.30117 s]
/proc/loadavg: 8.04 8.09 8.02 9/262 11835
/proc/meminfo: memFree=16938396/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=329588 CPUtime=48.77 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 17151 0 0 0 4869 8 0 0 25 0 8 0 547380066 337498112 16748 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670450 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/11827/statm: 82397 16748 466 189 0 76746 0
[pid=11827/tid=11829] ppid=11825 vsize=329588 CPUtime=6.07 cores=0-7
/proc/11827/task/11829/stat : 11829 (claspmt) R 11825 11827 10653 0 -1 4202560 1673 0 0 0 606 1 0 0 25 0 8 0 547380085 337498112 16748 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670241 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11827/tid=11830] ppid=11825 vsize=329588 CPUtime=6.06 cores=0-7
/proc/11827/task/11830/stat : 11830 (claspmt) R 11825 11827 10653 0 -1 4202560 1970 0 0 0 606 0 0 0 25 0 8 0 547380085 337498112 16748 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670427 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11827/tid=11831] ppid=11825 vsize=329588 CPUtime=6.07 cores=0-7
/proc/11827/task/11831/stat : 11831 (claspmt) R 11825 11827 10653 0 -1 4202560 1652 0 0 0 607 0 0 0 25 0 8 0 547380085 337498112 16748 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11827/tid=11832] ppid=11825 vsize=329588 CPUtime=6.06 cores=0-7
/proc/11827/task/11832/stat : 11832 (claspmt) R 11825 11827 10653 0 -1 4202560 1665 0 0 0 606 0 0 0 25 0 8 0 547380085 337498112 16748 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11827/tid=11833] ppid=11825 vsize=329588 CPUtime=6.1 cores=0-7
/proc/11827/task/11833/stat : 11833 (claspmt) R 11825 11827 10653 0 -1 4202560 1268 0 0 0 610 0 0 0 25 0 8 0 547380085 337498112 16748 33554432000 4194304 4964704 140734704709232 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11827/tid=11834] ppid=11825 vsize=329588 CPUtime=6.02 cores=0-7
/proc/11827/task/11834/stat : 11834 (claspmt) R 11825 11827 10653 0 -1 4202560 2025 0 0 0 601 1 0 0 25 0 8 0 547380085 337498112 16748 33554432000 4194304 4964704 140734704709232 18446744073709551615 4321023 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11827/tid=11835] ppid=11825 vsize=329588 CPUtime=6.07 cores=0-7
/proc/11827/task/11835/stat : 11835 (claspmt) R 11825 11827 10653 0 -1 4202560 3312 0 0 0 606 1 0 0 21 0 8 0 547380085 337498112 16748 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 48.77
Current children cumulated vsize (KiB) 329588

[startup+12.701 s]
/proc/loadavg: 8.11 8.10 8.02 9/262 11835
/proc/meminfo: memFree=16908072/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=334996 CPUtime=99.91 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 24307 0 0 0 9979 12 0 0 25 0 8 0 547380066 343035904 23904 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670277 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/11827/statm: 83749 23904 466 189 0 78098 0
[pid=11827/tid=11829] ppid=11825 vsize=334996 CPUtime=12.46 cores=0-7
/proc/11827/task/11829/stat : 11829 (claspmt) R 11825 11827 10653 0 -1 4202560 2670 0 0 0 1245 1 0 0 25 0 8 0 547380085 343035904 23904 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11827/tid=11830] ppid=11825 vsize=334996 CPUtime=12.47 cores=0-7
/proc/11827/task/11830/stat : 11830 (claspmt) R 11825 11827 10653 0 -1 4202560 3086 0 0 0 1246 1 0 0 25 0 8 0 547380085 343035904 23904 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670466 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11827/tid=11831] ppid=11825 vsize=334996 CPUtime=12.47 cores=0-7
/proc/11827/task/11831/stat : 11831 (claspmt) R 11825 11827 10653 0 -1 4202560 2396 0 0 0 1246 1 0 0 25 0 8 0 547380085 343035904 23904 33554432000 4194304 4964704 140734704709232 18446744073709551615 4321272 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11827/tid=11832] ppid=11825 vsize=334996 CPUtime=12.46 cores=0-7
/proc/11827/task/11832/stat : 11832 (claspmt) R 11825 11827 10653 0 -1 4202560 2456 0 0 0 1245 1 0 0 25 0 8 0 547380085 343035904 23904 33554432000 4194304 4964704 140734704709232 18446744073709551615 4334304 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11827/tid=11833] ppid=11825 vsize=334996 CPUtime=12.51 cores=0-7
/proc/11827/task/11833/stat : 11833 (claspmt) R 11825 11827 10653 0 -1 4202560 1988 0 0 0 1250 1 0 0 25 0 8 0 547380085 343035904 23904 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11827/tid=11834] ppid=11825 vsize=334996 CPUtime=12.37 cores=0-7
/proc/11827/task/11834/stat : 11834 (claspmt) R 11825 11827 10653 0 -1 4202560 2633 0 0 0 1236 1 0 0 25 0 8 0 547380085 343035904 23904 33554432000 4194304 4964704 140734704709232 18446744073709551615 4631168 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11827/tid=11835] ppid=11825 vsize=334996 CPUtime=12.46 cores=0-7
/proc/11827/task/11835/stat : 11835 (claspmt) R 11825 11827 10653 0 -1 4202560 4866 0 0 0 1244 2 0 0 25 0 8 0 547380085 343035904 23904 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 99.91
Current children cumulated vsize (KiB) 334996

[startup+25.5005 s]
/proc/loadavg: 8.09 8.10 8.02 9/262 11836
/proc/meminfo: memFree=16882364/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=342220 CPUtime=202.17 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 31002 0 0 0 20200 17 0 0 25 0 8 0 547380066 350433280 30598 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670263 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/11827/statm: 85555 30598 466 189 0 79904 0
[pid=11827/tid=11829] ppid=11825 vsize=342220 CPUtime=25.26 cores=0-7
/proc/11827/task/11829/stat : 11829 (claspmt) R 11825 11827 10653 0 -1 4202560 3476 0 0 0 2524 2 0 0 25 0 8 0 547380085 350433280 30598 33554432000 4194304 4964704 140734704709232 18446744073709551615 4647543 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11827/tid=11830] ppid=11825 vsize=342220 CPUtime=25.26 cores=0-7
/proc/11827/task/11830/stat : 11830 (claspmt) R 11825 11827 10653 0 -1 4202560 4040 0 0 0 2524 2 0 0 25 0 8 0 547380085 350433280 30598 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11827/tid=11831] ppid=11825 vsize=342220 CPUtime=25.26 cores=0-7
/proc/11827/task/11831/stat : 11831 (claspmt) R 11825 11827 10653 0 -1 4202560 2970 0 0 0 2525 1 0 0 25 0 8 0 547380085 350433280 30598 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670440 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11827/tid=11832] ppid=11825 vsize=342220 CPUtime=25.26 cores=0-7
/proc/11827/task/11832/stat : 11832 (claspmt) R 11825 11827 10653 0 -1 4202560 3280 0 0 0 2525 1 0 0 25 0 8 0 547380085 350433280 30598 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11827/tid=11833] ppid=11825 vsize=342220 CPUtime=25.3 cores=0-7
/proc/11827/task/11833/stat : 11833 (claspmt) R 11825 11827 10653 0 -1 4202560 2765 0 0 0 2529 1 0 0 25 0 8 0 547380085 350433280 30598 33554432000 4194304 4964704 140734704709232 18446744073709551615 4658314 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11827/tid=11834] ppid=11825 vsize=342220 CPUtime=25.05 cores=0-7
/proc/11827/task/11834/stat : 11834 (claspmt) R 11825 11827 10653 0 -1 4202560 3257 0 0 0 2504 1 0 0 25 0 8 0 547380085 350433280 30598 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11827/tid=11835] ppid=11825 vsize=342220 CPUtime=25.27 cores=0-7
/proc/11827/task/11835/stat : 11835 (claspmt) R 11825 11827 10653 0 -1 4202560 6528 0 0 0 2523 4 0 0 25 0 8 0 547380085 350433280 30598 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 202.17
Current children cumulated vsize (KiB) 342220

[startup+51.1007 s]
/proc/loadavg: 8.06 8.09 8.02 9/262 11836
/proc/meminfo: memFree=16822924/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=356464 CPUtime=406.74 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 46531 0 0 0 40648 26 0 0 25 0 8 0 547380066 365019136 45471 33554432000 4194304 4964704 140734704709232 18446744073709551615 4320944 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/11827/statm: 89116 45471 466 189 0 83465 0
[pid=11827/tid=11829] ppid=11825 vsize=356464 CPUtime=50.85 cores=0-7
/proc/11827/task/11829/stat : 11829 (claspmt) R 11825 11827 10653 0 -1 4202560 5559 0 0 0 5082 3 0 0 25 0 8 0 547380085 365019136 45471 33554432000 4194304 4964704 140734704709232 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11827/tid=11830] ppid=11825 vsize=356464 CPUtime=50.86 cores=0-7
/proc/11827/task/11830/stat : 11830 (claspmt) R 11825 11827 10653 0 -1 4202560 6607 0 0 0 5084 2 0 0 25 0 8 0 547380085 365019136 45471 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11827/tid=11831] ppid=11825 vsize=356464 CPUtime=50.86 cores=0-7
/proc/11827/task/11831/stat : 11831 (claspmt) R 11825 11827 10653 0 -1 4202560 4394 0 0 0 5084 2 0 0 25 0 8 0 547380085 365019136 45471 33554432000 4194304 4964704 140734704709232 18446744073709551615 4321097 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11827/tid=11832] ppid=11825 vsize=356464 CPUtime=50.85 cores=0-7
/proc/11827/task/11832/stat : 11832 (claspmt) R 11825 11827 10653 0 -1 4202560 4896 0 0 0 5083 2 0 0 25 0 8 0 547380085 365019136 45471 33554432000 4194304 4964704 140734704709232 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11827/tid=11833] ppid=11825 vsize=356464 CPUtime=50.9 cores=0-7
/proc/11827/task/11833/stat : 11833 (claspmt) R 11825 11827 10653 0 -1 4202560 4211 0 0 0 5088 2 0 0 25 0 8 0 547380085 365019136 45471 33554432000 4194304 4964704 140734704709232 18446744073709551615 4627248 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11827/tid=11834] ppid=11825 vsize=356464 CPUtime=50.44 cores=0-7
/proc/11827/task/11834/stat : 11834 (claspmt) R 11825 11827 10653 0 -1 4202560 4596 0 0 0 5042 2 0 0 25 0 8 0 547380085 365019136 45471 33554432000 4194304 4964704 140734704709232 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11827/tid=11835] ppid=11825 vsize=356464 CPUtime=50.86 cores=0-7
/proc/11827/task/11835/stat : 11835 (claspmt) R 11825 11827 10653 0 -1 4202560 10295 0 0 0 5080 6 0 0 25 0 8 0 547380085 365019136 45471 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 406.74

################
# More data... #
################

[pid=11827/tid=11830] ppid=11825 vsize=1024552 CPUtime=4661.29 cores=0-7
/proc/11827/task/11830/stat : 11830 (claspmt) R 11825 11827 10653 0 -1 4202560 46868 0 0 0 466106 23 0 0 25 0 8 0 547380085 1049141248 216542 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11827/tid=11831] ppid=11825 vsize=1024552 CPUtime=4661.87 cores=0-7
/proc/11827/task/11831/stat : 11831 (claspmt) R 11825 11827 10653 0 -1 4202560 17171 0 0 0 466165 22 0 0 25 0 8 0 547380085 1049141248 216542 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11827/tid=11832] ppid=11825 vsize=1024552 CPUtime=4661.97 cores=0-7
/proc/11827/task/11832/stat : 11832 (claspmt) R 11825 11827 10653 0 -1 4202560 22826 0 0 0 466176 21 0 0 25 0 8 0 547380085 1049141248 216542 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11827/tid=11833] ppid=11825 vsize=1024552 CPUtime=4649.72 cores=0-7
/proc/11827/task/11833/stat : 11833 (claspmt) R 11825 11827 10653 0 -1 4202560 24752 0 0 0 464945 27 0 0 25 0 8 0 547380085 1049141248 216542 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11827/tid=11834] ppid=11825 vsize=1024552 CPUtime=4652.14 cores=0-7
/proc/11827/task/11834/stat : 11834 (claspmt) R 11825 11827 10653 0 -1 4202560 18394 0 0 0 465187 27 0 0 25 0 8 0 547380085 1049141248 216542 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11827/tid=11835] ppid=11825 vsize=1024552 CPUtime=4660.02 cores=0-7
/proc/11827/task/11835/stat : 11835 (claspmt) R 11825 11827 10653 0 -1 4202560 39806 0 0 0 465943 59 0 0 25 0 8 0 547380085 1049141248 216542 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37255.2
Current children cumulated vsize (KiB) 1024552

[startup+4722.31 s]
/proc/loadavg: 8.11 8.06 8.00 9/262 11974
/proc/meminfo: memFree=16130264/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=1024840 CPUtime=37734.7 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 231550 0 0 0 3773205 264 0 0 25 0 8 0 547380066 1049436160 217891 33554432000 4194304 4964704 140734704709232 18446744073709551615 4325296 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11827/statm: 256210 217891 466 189 0 250559 0
[pid=11827/tid=11829] ppid=11825 vsize=1024840 CPUtime=4714.79 cores=0-7
/proc/11827/task/11829/stat : 11829 (claspmt) R 11825 11827 10653 0 -1 4202560 39691 0 0 0 471437 42 0 0 25 0 8 0 547380085 1049436160 217891 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11827/tid=11830] ppid=11825 vsize=1024840 CPUtime=4721.3 cores=0-7
/proc/11827/task/11830/stat : 11830 (claspmt) R 11825 11827 10653 0 -1 4202560 47710 0 0 0 472106 24 0 0 25 0 8 0 547380085 1049436160 217891 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670271 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11827/tid=11831] ppid=11825 vsize=1024840 CPUtime=4721.89 cores=0-7
/proc/11827/task/11831/stat : 11831 (claspmt) R 11825 11827 10653 0 -1 4202560 17285 0 0 0 472166 23 0 0 25 0 8 0 547380085 1049436160 217891 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670193 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11827/tid=11832] ppid=11825 vsize=1024840 CPUtime=4721.98 cores=0-7
/proc/11827/task/11832/stat : 11832 (claspmt) R 11825 11827 10653 0 -1 4202560 22947 0 0 0 472177 21 0 0 25 0 8 0 547380085 1049436160 217891 33554432000 4194304 4964704 140734704709232 18446744073709551615 4321029 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11827/tid=11833] ppid=11825 vsize=1024840 CPUtime=4709.46 cores=0-7
/proc/11827/task/11833/stat : 11833 (claspmt) R 11825 11827 10653 0 -1 4202560 24792 0 0 0 470919 27 0 0 25 0 8 0 547380085 1049436160 217891 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11827/tid=11834] ppid=11825 vsize=1024840 CPUtime=4712.15 cores=0-7
/proc/11827/task/11834/stat : 11834 (claspmt) R 11825 11827 10653 0 -1 4202560 18459 0 0 0 471188 27 0 0 25 0 8 0 547380085 1049436160 217891 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670277 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11827/tid=11835] ppid=11825 vsize=1024840 CPUtime=4720.02 cores=0-7
/proc/11827/task/11835/stat : 11835 (claspmt) R 11825 11827 10653 0 -1 4202560 39840 0 0 0 471943 59 0 0 25 0 8 0 547380085 1049436160 217891 33554432000 4194304 4964704 140734704709232 18446744073709551615 4321093 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37734.7
Current children cumulated vsize (KiB) 1024840

[startup+4782.3 s]
/proc/loadavg: 8.10 8.06 8.00 9/262 11975
/proc/meminfo: memFree=16128504/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=1025344 CPUtime=38214.1 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 232100 0 0 0 3821144 266 0 0 25 0 8 0 547380066 1049952256 218332 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11827/statm: 256336 218332 466 189 0 250685 0
[pid=11827/tid=11829] ppid=11825 vsize=1025344 CPUtime=4774.28 cores=0-7
/proc/11827/task/11829/stat : 11829 (claspmt) R 11825 11827 10653 0 -1 4202560 39929 0 0 0 477386 42 0 0 25 0 8 0 547380085 1049952256 218332 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11827/tid=11830] ppid=11825 vsize=1025344 CPUtime=4781.28 cores=0-7
/proc/11827/task/11830/stat : 11830 (claspmt) R 11825 11827 10653 0 -1 4202560 47737 0 0 0 478104 24 0 0 25 0 8 0 547380085 1049952256 218332 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11827/tid=11831] ppid=11825 vsize=1025344 CPUtime=4781.87 cores=0-7
/proc/11827/task/11831/stat : 11831 (claspmt) R 11825 11827 10653 0 -1 4202560 17337 0 0 0 478164 23 0 0 25 0 8 0 547380085 1049952256 218332 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11827/tid=11832] ppid=11825 vsize=1025344 CPUtime=4781.97 cores=0-7
/proc/11827/task/11832/stat : 11832 (claspmt) R 11825 11827 10653 0 -1 4202560 22981 0 0 0 478176 21 0 0 25 0 8 0 547380085 1049952256 218332 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11827/tid=11833] ppid=11825 vsize=1025344 CPUtime=4769.45 cores=0-7
/proc/11827/task/11833/stat : 11833 (claspmt) R 11825 11827 10653 0 -1 4202560 24885 0 0 0 476918 27 0 0 25 0 8 0 547380085 1049952256 218332 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11827/tid=11834] ppid=11825 vsize=1025344 CPUtime=4772.13 cores=0-7
/proc/11827/task/11834/stat : 11834 (claspmt) R 11825 11827 10653 0 -1 4202560 18540 0 0 0 477186 27 0 0 25 0 8 0 547380085 1049952256 218332 33554432000 4194304 4964704 140734704709232 18446744073709551615 4627272 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11827/tid=11835] ppid=11825 vsize=1025344 CPUtime=4780.02 cores=0-7
/proc/11827/task/11835/stat : 11835 (claspmt) R 11825 11827 10653 0 -1 4202560 39853 0 0 0 477942 60 0 0 25 0 8 0 547380085 1049952256 218332 33554432000 4194304 4964704 140734704709232 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38214.1
Current children cumulated vsize (KiB) 1025344

[startup+4842.3 s]
/proc/loadavg: 8.04 8.05 8.00 9/262 11977
/proc/meminfo: memFree=16121912/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=1027756 CPUtime=38693.6 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 233955 0 0 0 3869089 269 0 0 25 0 8 0 547380066 1052422144 219893 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670266 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11827/statm: 256939 219893 466 189 0 251288 0
[pid=11827/tid=11829] ppid=11825 vsize=1027756 CPUtime=4833.77 cores=0-7
/proc/11827/task/11829/stat : 11829 (claspmt) R 11825 11827 10653 0 -1 4202560 40655 0 0 0 483334 43 0 0 25 0 8 0 547380085 1052422144 219893 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11827/tid=11830] ppid=11825 vsize=1027756 CPUtime=4841.27 cores=0-7
/proc/11827/task/11830/stat : 11830 (claspmt) R 11825 11827 10653 0 -1 4202560 48001 0 0 0 484103 24 0 0 25 0 8 0 547380085 1052422144 219893 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11827/tid=11831] ppid=11825 vsize=1027756 CPUtime=4841.87 cores=0-7
/proc/11827/task/11831/stat : 11831 (claspmt) R 11825 11827 10653 0 -1 4202560 17386 0 0 0 484164 23 0 0 25 0 8 0 547380085 1052422144 219893 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11827/tid=11832] ppid=11825 vsize=1027756 CPUtime=4841.97 cores=0-7
/proc/11827/task/11832/stat : 11832 (claspmt) R 11825 11827 10653 0 -1 4202560 23118 0 0 0 484176 21 0 0 25 0 8 0 547380085 1052422144 219893 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11827/tid=11833] ppid=11825 vsize=1027756 CPUtime=4829.45 cores=0-7
/proc/11827/task/11833/stat : 11833 (claspmt) R 11825 11827 10653 0 -1 4202560 24915 0 0 0 482918 27 0 0 25 0 8 0 547380085 1052422144 219893 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11827/tid=11834] ppid=11825 vsize=1027756 CPUtime=4832.13 cores=0-7
/proc/11827/task/11834/stat : 11834 (claspmt) R 11825 11827 10653 0 -1 4202560 18578 0 0 0 483186 27 0 0 25 0 8 0 547380085 1052422144 219893 33554432000 4194304 4964704 140734704709232 18446744073709551615 4321020 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11827/tid=11835] ppid=11825 vsize=1027756 CPUtime=4840.02 cores=0-7
/proc/11827/task/11835/stat : 11835 (claspmt) R 11825 11827 10653 0 -1 4202560 39965 0 0 0 483941 61 0 0 25 0 8 0 547380085 1052422144 219893 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38693.6
Current children cumulated vsize (KiB) 1027756

[startup+4902.3 s]
/proc/loadavg: 8.14 8.07 8.01 9/262 11979
/proc/meminfo: memFree=16120152/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=1028792 CPUtime=39173 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 234636 0 0 0 3917034 270 0 0 25 0 8 0 547380066 1053483008 220451 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11827/statm: 257198 220451 466 189 0 251547 0
[pid=11827/tid=11829] ppid=11825 vsize=1028792 CPUtime=4893.25 cores=0-7
/proc/11827/task/11829/stat : 11829 (claspmt) R 11825 11827 10653 0 -1 4202560 40969 0 0 0 489282 43 0 0 25 0 8 0 547380085 1053483008 220451 33554432000 4194304 4964704 140734704709232 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11827/tid=11830] ppid=11825 vsize=1028792 CPUtime=4901.25 cores=0-7
/proc/11827/task/11830/stat : 11830 (claspmt) R 11825 11827 10653 0 -1 4202560 48163 0 0 0 490101 24 0 0 25 0 8 0 547380085 1053483008 220451 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11827/tid=11831] ppid=11825 vsize=1028792 CPUtime=4901.86 cores=0-7
/proc/11827/task/11831/stat : 11831 (claspmt) R 11825 11827 10653 0 -1 4202560 17389 0 0 0 490163 23 0 0 25 0 8 0 547380085 1053483008 220451 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670241 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11827/tid=11832] ppid=11825 vsize=1028792 CPUtime=4901.97 cores=0-7
/proc/11827/task/11832/stat : 11832 (claspmt) R 11825 11827 10653 0 -1 4202560 23214 0 0 0 490176 21 0 0 25 0 8 0 547380085 1053483008 220451 33554432000 4194304 4964704 140734704709232 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11827/tid=11833] ppid=11825 vsize=1028792 CPUtime=4889.45 cores=0-7
/proc/11827/task/11833/stat : 11833 (claspmt) R 11825 11827 10653 0 -1 4202560 24990 0 0 0 488918 27 0 0 25 0 8 0 547380085 1053483008 220451 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11827/tid=11834] ppid=11825 vsize=1028792 CPUtime=4892.13 cores=0-7
/proc/11827/task/11834/stat : 11834 (claspmt) R 11825 11827 10653 0 -1 4202560 18606 0 0 0 489186 27 0 0 25 0 8 0 547380085 1053483008 220451 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11827/tid=11835] ppid=11825 vsize=1028792 CPUtime=4900.02 cores=0-7
/proc/11827/task/11835/stat : 11835 (claspmt) R 11825 11827 10653 0 -1 4202560 39968 0 0 0 489941 61 0 0 25 0 8 0 547380085 1053483008 220451 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39173
Current children cumulated vsize (KiB) 1028792

[startup+4962.3 s]
/proc/loadavg: 8.09 8.07 8.01 9/262 11980
/proc/meminfo: memFree=16116788/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=1028792 CPUtime=39652.5 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 235475 0 0 0 3964980 271 0 0 25 0 8 0 547380066 1053483008 221142 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11827/statm: 257198 221142 466 189 0 251547 0
[pid=11827/tid=11829] ppid=11825 vsize=1028792 CPUtime=4952.75 cores=0-7
/proc/11827/task/11829/stat : 11829 (claspmt) R 11825 11827 10653 0 -1 4202560 41110 0 0 0 495231 44 0 0 25 0 8 0 547380085 1053483008 221142 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11827/tid=11830] ppid=11825 vsize=1028792 CPUtime=4961.25 cores=0-7
/proc/11827/task/11830/stat : 11830 (claspmt) R 11825 11827 10653 0 -1 4202560 48577 0 0 0 496101 24 0 0 25 0 8 0 547380085 1053483008 221142 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11827/tid=11831] ppid=11825 vsize=1028792 CPUtime=4961.86 cores=0-7
/proc/11827/task/11831/stat : 11831 (claspmt) R 11825 11827 10653 0 -1 4202560 17406 0 0 0 496163 23 0 0 25 0 8 0 547380085 1053483008 221142 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11827/tid=11832] ppid=11825 vsize=1028792 CPUtime=4961.97 cores=0-7
/proc/11827/task/11832/stat : 11832 (claspmt) R 11825 11827 10653 0 -1 4202560 23262 0 0 0 496176 21 0 0 25 0 8 0 547380085 1053483008 221142 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11827/tid=11833] ppid=11825 vsize=1028792 CPUtime=4949.44 cores=0-7
/proc/11827/task/11833/stat : 11833 (claspmt) R 11825 11827 10653 0 -1 4202560 25100 0 0 0 494917 27 0 0 25 0 8 0 547380085 1053483008 221142 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11827/tid=11834] ppid=11825 vsize=1028792 CPUtime=4952.12 cores=0-7
/proc/11827/task/11834/stat : 11834 (claspmt) R 11825 11827 10653 0 -1 4202560 18681 0 0 0 495185 27 0 0 25 0 8 0 547380085 1053483008 221142 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670255 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11827/tid=11835] ppid=11825 vsize=1028792 CPUtime=4960.02 cores=0-7
/proc/11827/task/11835/stat : 11835 (claspmt) R 11825 11827 10653 0 -1 4202560 39994 0 0 0 495941 61 0 0 25 0 8 0 547380085 1053483008 221142 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39652.5
Current children cumulated vsize (KiB) 1028792



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5005.8 s]
/proc/loadavg: 8.16 8.09 8.01 9/262 11981
/proc/meminfo: memFree=16115652/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=1028792 CPUtime=40000.2 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 235888 0 0 0 3999747 273 0 0 25 0 8 0 547380066 1053483008 221446 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670193 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11827/statm: 257198 221446 466 189 0 251547 0
[pid=11827/tid=11829] ppid=11825 vsize=1028792 CPUtime=4995.89 cores=0-7
/proc/11827/task/11829/stat : 11829 (claspmt) R 11825 11827 10653 0 -1 4202560 41433 0 0 0 499545 44 0 0 25 0 8 0 547380085 1053483008 221446 33554432000 4194304 4964704 140734704709232 18446744073709551615 4660755 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11827/tid=11830] ppid=11825 vsize=1028792 CPUtime=5004.75 cores=0-7
/proc/11827/task/11830/stat : 11830 (claspmt) R 11825 11827 10653 0 -1 4202560 48586 0 0 0 500451 24 0 0 25 0 8 0 547380085 1053483008 221446 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11827/tid=11831] ppid=11825 vsize=1028792 CPUtime=5005.37 cores=0-7
/proc/11827/task/11831/stat : 11831 (claspmt) R 11825 11827 10653 0 -1 4202560 17435 0 0 0 500514 23 0 0 25 0 8 0 547380085 1053483008 221446 33554432000 4194304 4964704 140734704709232 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11827/tid=11832] ppid=11825 vsize=1028792 CPUtime=5005.47 cores=0-7
/proc/11827/task/11832/stat : 11832 (claspmt) R 11825 11827 10653 0 -1 4202560 23262 0 0 0 500526 21 0 0 25 0 8 0 547380085 1053483008 221446 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11827/tid=11833] ppid=11825 vsize=1028792 CPUtime=4992.94 cores=0-7
/proc/11827/task/11833/stat : 11833 (claspmt) R 11825 11827 10653 0 -1 4202560 25105 0 0 0 499267 27 0 0 25 0 8 0 547380085 1053483008 221446 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670269 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11827/tid=11834] ppid=11825 vsize=1028792 CPUtime=4995.64 cores=0-7
/proc/11827/task/11834/stat : 11834 (claspmt) R 11825 11827 10653 0 -1 4202560 18689 0 0 0 499536 28 0 0 25 0 8 0 547380085 1053483008 221446 33554432000 4194304 4964704 140734704709232 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11827/tid=11835] ppid=11825 vsize=1028792 CPUtime=5003.53 cores=0-7
/proc/11827/task/11835/stat : 11835 (claspmt) R 11825 11827 10653 0 -1 4202560 40010 0 0 0 500292 61 0 0 25 0 8 0 547380085 1053483008 221446 33554432000 4194304 4964704 140734704709232 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 40000.2
Current children cumulated vsize (KiB) 1028792

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5006 s]
/proc/loadavg: 8.16 8.09 8.01 2/256 11982
/proc/meminfo: memFree=16120040/32951124 swapFree=67111416/67111528
[pid=11827] ppid=11825 vsize=1023920 CPUtime=40000.4 cores=0-7
/proc/11827/stat : 11827 (claspmt) R 11825 11827 10653 0 -1 4202496 235898 0 0 0 3999771 273 0 0 25 0 1 0 547380066 1048494080 220051 33554432000 4194304 4964704 140734704709232 18446744073709551615 212435673659 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/11827/statm: 255980 220051 476 189 0 250329 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 1023920

Child status: 0
Real time (s): 5006.09
CPU time (s): 40000.5
CPU user time (s): 39997.8
CPU system time (s): 2.77458
CPU usage (%): 799.037
Max. virtual memory (cumulated for all children) (KiB): 1028792

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39997.8
system time used= 2.77458
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 235899
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 121683
involuntary context switches= 189108

runsolver used 11.0083 second user time and 29.9654 second system time

The end

Launcher Data 

Begin job on node102 at 2011-05-02 19:36:17
IDJOB=3321515
IDBENCH=71583
IDSOLVER=1711
FILE ID=node102/3321515-1304357777
RUNJOBID= node102-1304347539-10667
PBS_JOBID= 13178657
Free space on /tmp= 73440 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/forced/rbsat-v945c61409gyes1.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321515-1304357777/watcher-3321515-1304357777 -o /tmp/evaluation-result-3321515-1304357777/solver-3321515-1304357777 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321515-1304357777.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 26b9098de258b0434de90e744962df46
RANDOM SEED=1073921132

node102.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      16997496 kB
Buffers:        670264 kB
Cached:        1876052 kB
SwapCached:          0 kB
Active:       13400560 kB
Inactive:      2091360 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      16997496 kB
SwapTotal:    67111528 kB
SwapFree:     67111416 kB
Dirty:            1580 kB
Writeback:           4 kB
AnonPages:    12946500 kB
Mapped:          22092 kB
Slab:           357008 kB
PageTables:      38060 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 37664776 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73440 MiB
End job on node102 at 2011-05-02 20:59:45