Trace number 3373461

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 4800.41 1202.95

General information on the benchmark

Nameindustrial/IBM_FV_2002_13_rule_1/
SAT_dat.k20.cnf
MD5SUM2bee695ce4278b72ee9295631ad94252
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark180.219
Satisfiable
(Un)Satisfiability was proved
Number of variables31357
Number of clauses133420
Sum of the clauses size370492
Maximum clause length37
Minimum clause length1
Number of clauses of size 1278
Number of clauses of size 258046
Number of clauses of size 369900
Number of clauses of size 4166
Number of clauses of size 5305
Number of clauses of size over 54725

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...418199/instance-3373461-1305418199.cnf
0.19/0.29	c Solving...
4800.24/1202.81	
4800.24/1202.81	*** INTERRUPTED! ***
4800.24/1202.81	*** Shutting down threads...
4800.37/1202.95	s UNKNOWN
4800.37/1202.95	
4800.37/1202.95	c INTERRUPTED   : 1
4800.37/1202.95	c Models        : 0+    
4800.37/1202.95	c Time          : 1202.814s (Solving: 1202.53s 1st Model: 0.00s Unsat: 0.00s)
4800.37/1202.95	c CPU Time      : 4800.270s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3373461-1305418199/watcher-3373461-1305418199 -o /tmp/evaluation-result-3373461-1305418199/solver-3373461-1305418199 -C 4800 -W 1300 -M 15500 HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373461-1305418199.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.81 7.89 7.98 5/275 27632
/proc/meminfo: memFree=20507024/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=26760 CPUtime=0 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 1993 0 0 0 0 0 0 0 25 0 1 0 653419628 27402240 1766 33554432000 4194304 4964800 140734278257984 18446744073709551615 4679094 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/27632/statm: 6690 1770 376 189 0 1564 0

[startup+0.0975209 s]
/proc/loadavg: 7.81 7.89 7.98 5/275 27632
/proc/meminfo: memFree=20507024/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=39100 CPUtime=0.08 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 6025 0 0 0 8 0 0 0 25 0 1 0 653419628 40038400 4932 33554432000 4194304 4964800 140734278257984 18446744073709551615 4607585 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/27632/statm: 9775 4932 402 189 0 4649 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 39100

[startup+0.100518 s]
/proc/loadavg: 7.81 7.89 7.98 5/275 27632
/proc/meminfo: memFree=20507024/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=39100 CPUtime=0.09 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 6025 0 0 0 9 0 0 0 25 0 1 0 653419628 40038400 4932 33554432000 4194304 4964800 140734278257984 18446744073709551615 4607744 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/27632/statm: 9775 4932 402 189 0 4649 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 39100

[startup+0.300486 s]
/proc/loadavg: 7.81 7.89 7.98 5/275 27632
/proc/meminfo: memFree=20507024/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=196776 CPUtime=0.3 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) S 27630 27632 26831 0 -1 4202496 11057 0 0 0 29 1 0 0 25 0 4 0 653419628 201498624 8660 33554432000 4194304 4964800 140734278257984 18446744073709551615 237290695577 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/27632/statm: 49194 8672 433 189 0 43543 0
Current children cumulated CPU time (s) 0.3
Current children cumulated vsize (KiB) 196776

[startup+0.700418 s]
/proc/loadavg: 7.81 7.89 7.98 5/275 27632
/proc/meminfo: memFree=20507024/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=263612 CPUtime=1.87 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 13820 0 0 0 185 2 0 0 25 0 4 0 653419628 269938688 11420 33554432000 4194304 4964800 140734278257984 18446744073709551615 4668734 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 65903 11420 458 189 0 60252 0
Current children cumulated CPU time (s) 1.87
Current children cumulated vsize (KiB) 263612

[startup+1.50028 s]
/proc/loadavg: 7.81 7.89 7.98 9/279 27636
/proc/meminfo: memFree=20458888/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=263612 CPUtime=5.07 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 14872 0 0 0 504 3 0 0 25 0 4 0 653419628 269938688 12472 33554432000 4194304 4964800 140734278257984 18446744073709551615 4370224 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 65903 12473 458 189 0 60252 0
[pid=27632/tid=27634] ppid=27630 vsize=263612 CPUtime=1.19 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 2268 0 0 0 119 0 0 0 25 0 4 0 653419657 269938688 12473 33554432000 4194304 4964800 140734278257984 18446744073709551615 4370264 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=263612 CPUtime=1.19 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 2701 0 0 0 119 0 0 0 25 0 4 0 653419657 269938688 12473 33554432000 4194304 4964800 140734278257984 18446744073709551615 237278540833 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=263612 CPUtime=1.18 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 1855 0 0 0 118 0 0 0 25 0 4 0 653419657 269938688 12473 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 5.07
Current children cumulated vsize (KiB) 263612

[startup+3.10099 s]
/proc/loadavg: 7.81 7.89 7.98 9/279 27636
/proc/meminfo: memFree=20450212/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=265872 CPUtime=11.46 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 17915 0 0 0 1142 4 0 0 25 0 4 0 653419628 272252928 15515 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670313 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 66468 15515 458 189 0 60817 0
[pid=27632/tid=27634] ppid=27630 vsize=265872 CPUtime=2.79 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 2685 0 0 0 278 1 0 0 25 0 4 0 653419657 272252928 15515 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=265872 CPUtime=2.79 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 3410 0 0 0 279 0 0 0 25 0 4 0 653419657 272252928 15515 33554432000 4194304 4964800 140734278257984 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=265872 CPUtime=2.77 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 3256 0 0 0 277 0 0 0 25 0 4 0 653419657 272252928 15515 33554432000 4194304 4964800 140734278257984 18446744073709551615 4653307 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.46
Current children cumulated vsize (KiB) 265872

[startup+6.30042 s]
/proc/loadavg: 7.83 7.90 7.98 9/279 27636
/proc/meminfo: memFree=20420700/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=267200 CPUtime=24.23 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 20636 0 0 0 2417 6 0 0 25 0 4 0 653419628 273612800 18236 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670437 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 66800 18236 458 189 0 61149 0
[pid=27632/tid=27634] ppid=27630 vsize=267200 CPUtime=5.99 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 3017 0 0 0 598 1 0 0 25 0 4 0 653419657 273612800 18236 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=267200 CPUtime=5.99 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 4351 0 0 0 598 1 0 0 25 0 4 0 653419657 273612800 18236 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670335 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=267200 CPUtime=5.94 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 4425 0 0 0 593 1 0 0 25 0 4 0 653419657 273612800 18236 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.23
Current children cumulated vsize (KiB) 267200

[startup+12.7003 s]
/proc/loadavg: 7.84 7.90 7.98 9/279 27637
/proc/meminfo: memFree=20387664/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=272844 CPUtime=49.78 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 25129 0 0 0 4969 9 0 0 25 0 4 0 653419628 279392256 22729 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670299 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 68211 22729 460 189 0 62560 0
[pid=27632/tid=27634] ppid=27630 vsize=272844 CPUtime=12.39 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 3310 0 0 0 1238 1 0 0 25 0 4 0 653419657 279392256 22729 33554432000 4194304 4964800 140734278257984 18446744073709551615 4372015 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=272844 CPUtime=12.39 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 5409 0 0 0 1238 1 0 0 25 0 4 0 653419657 279392256 22729 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=272844 CPUtime=12.29 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 6766 0 0 0 1226 3 0 0 25 0 4 0 653419657 279392256 22729 33554432000 4194304 4964800 140734278257984 18446744073709551615 4668715 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 49.78
Current children cumulated vsize (KiB) 272844

[startup+25.501 s]
/proc/loadavg: 7.94 7.92 7.98 9/279 27637
/proc/meminfo: memFree=20325392/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=283548 CPUtime=100.86 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 30210 0 0 0 10073 13 0 0 25 0 4 0 653419628 290353152 27810 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670180 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 70887 27810 460 189 0 65236 0
[pid=27632/tid=27634] ppid=27630 vsize=283548 CPUtime=25.19 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 4047 0 0 0 2517 2 0 0 25 0 4 0 653419657 290353152 27810 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=283548 CPUtime=25.19 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 6766 0 0 0 2516 3 0 0 25 0 4 0 653419657 290353152 27810 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=283548 CPUtime=24.98 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 9508 0 0 0 2493 5 0 0 25 0 4 0 653419657 290353152 27810 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 100.86
Current children cumulated vsize (KiB) 283548

[startup+51.1005 s]
/proc/loadavg: 7.96 7.92 7.98 9/279 27638
/proc/meminfo: memFree=20211724/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=293852 CPUtime=203.05 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 36865 0 0 0 20286 19 0 0 25 0 4 0 653419628 300904448 34465 33554432000 4194304 4964800 140734278257984 18446744073709551615 4660444 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 73463 34465 460 189 0 67812 0
[pid=27632/tid=27634] ppid=27630 vsize=293852 CPUtime=50.78 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 4470 0 0 0 5076 2 0 0 25 0 4 0 653419657 300904448 34465 33554432000 4194304 4964800 140734278257984 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=293852 CPUtime=50.79 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 9304 0 0 0 5074 5 0 0 25 0 4 0 653419657 300904448 34465 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=293852 CPUtime=50.37 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 12384 0 0 0 5030 7 0 0 25 0 4 0 653419657 300904448 34465 33554432000 4194304 4964800 140734278257984 18446744073709551615 4363327 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 203.05
Current children cumulated vsize (KiB) 293852

[startup+102.31 s]
/proc/loadavg: 8.03 7.95 7.99 9/279 27639
/proc/meminfo: memFree=20006492/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=300476 CPUtime=407.46 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 49785 0 0 0 40714 32 0 0 25 0 4 0 653419628 307687424 47381 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 75119 47381 460 189 0 69468 0
[pid=27632/tid=27634] ppid=27630 vsize=300476 CPUtime=101.98 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 6057 0 0 0 10194 4 0 0 25 0 4 0 653419657 307687424 47381 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=300476 CPUtime=102 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 15220 0 0 0 10190 10 0 0 25 0 4 0 653419657 307687424 47381 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=300476 CPUtime=101.16 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 14857 0 0 0 10105 11 0 0 25 0 4 0 653419657 307687424 47381 33554432000 4194304 4964800 140734278257984 18446744073709551615 4320864 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 407.46
Current children cumulated vsize (KiB) 300476

[startup+162.301 s]
/proc/loadavg: 8.01 7.95 7.99 9/279 27641
/proc/meminfo: memFree=19884092/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=371652 CPUtime=646.92 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 67882 0 0 0 64643 49 0 0 25 0 4 0 653419628 380571648 65037 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 92913 65037 460 189 0 87262 0
[pid=27632/tid=27634] ppid=27630 vsize=371652 CPUtime=161.95 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 11385 0 0 0 16185 10 0 0 25 0 4 0 653419657 380571648 65037 33554432000 4194304 4964800 140734278257984 18446744073709551615 4659391 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=371652 CPUtime=162 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 19342 0 0 0 16187 13 0 0 25 0 4 0 653419657 380571648 65037 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=371652 CPUtime=160.67 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 20513 0 0 0 16050 17 0 0 25 0 4 0 653419657 380571648 65037 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670349 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 646.92
Current children cumulated vsize (KiB) 371652

[startup+222.301 s]
/proc/loadavg: 8.06 7.97 7.99 9/279 27642
/proc/meminfo: memFree=19742968/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=514588 CPUtime=886.41 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 77989 0 0 0 88582 59 0 0 25 0 4 0 653419628 526938112 74514 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670264 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 128647 74514 460 189 0 122996 0
[pid=27632/tid=27634] ppid=27630 vsize=514588 CPUtime=221.95 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 12333 0 0 0 22184 11 0 0 25 0 4 0 653419657 526938112 74514 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670239 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=514588 CPUtime=221.99 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 24677 0 0 0 22181 18 0 0 25 0 4 0 653419657 526938112 74514 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=514588 CPUtime=220.17 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 22514 0 0 0 21999 18 0 0 25 0 4 0 653419657 526938112 74514 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 886.41
Current children cumulated vsize (KiB) 514588

[startup+282.3 s]

################
# More data... #
################

Current children cumulated CPU time (s) 2562.93
Current children cumulated vsize (KiB) 656036

[startup+702.301 s]
/proc/loadavg: 8.08 8.07 8.01 9/279 27655
/proc/meminfo: memFree=18688080/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=656544 CPUtime=2802.4 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 138298 0 0 0 280118 122 0 0 25 0 4 0 653419628 672301056 134551 33554432000 4194304 4964800 140734278257984 18446744073709551615 4370264 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 164136 134551 460 189 0 158485 0
[pid=27632/tid=27634] ppid=27630 vsize=656544 CPUtime=701.89 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 26933 0 0 0 70166 23 0 0 25 0 4 0 653419657 672301056 134551 33554432000 4194304 4964800 140734278257984 18446744073709551615 4370288 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=656544 CPUtime=701.99 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 36449 0 0 0 70169 30 0 0 25 0 4 0 653419657 672301056 134551 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=656544 CPUtime=696.24 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 49926 0 0 0 69573 51 0 0 25 0 4 0 653419657 672301056 134551 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2802.4
Current children cumulated vsize (KiB) 656544

[startup+762.31 s]
/proc/loadavg: 8.16 8.08 8.02 9/279 27657
/proc/meminfo: memFree=18559480/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=656544 CPUtime=3041.93 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 138334 0 0 0 304071 122 0 0 25 0 4 0 653419628 672301056 134587 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 164136 134587 460 189 0 158485 0
[pid=27632/tid=27634] ppid=27630 vsize=656544 CPUtime=761.88 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 26965 0 0 0 76165 23 0 0 25 0 4 0 653419657 672301056 134587 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=656544 CPUtime=762 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 36453 0 0 0 76170 30 0 0 25 0 4 0 653419657 672301056 134587 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=656544 CPUtime=755.75 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 49926 0 0 0 75524 51 0 0 25 0 4 0 653419657 672301056 134587 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3041.93
Current children cumulated vsize (KiB) 656544

[startup+822.3 s]
/proc/loadavg: 8.11 8.08 8.02 9/279 27658
/proc/meminfo: memFree=18456180/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=656544 CPUtime=3281.4 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 138349 0 0 0 328017 123 0 0 25 0 4 0 653419628 672301056 134602 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670299 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 164136 134602 460 189 0 158485 0
[pid=27632/tid=27634] ppid=27630 vsize=656544 CPUtime=821.87 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 26965 0 0 0 82164 23 0 0 25 0 4 0 653419657 672301056 134602 33554432000 4194304 4964800 140734278257984 18446744073709551615 4362287 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=656544 CPUtime=821.99 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 36453 0 0 0 82169 30 0 0 25 0 4 0 653419657 672301056 134602 33554432000 4194304 4964800 140734278257984 18446744073709551615 4627213 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=656544 CPUtime=815.25 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 49941 0 0 0 81474 51 0 0 25 0 4 0 653419657 672301056 134602 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3281.4
Current children cumulated vsize (KiB) 656544

[startup+882.301 s]
/proc/loadavg: 8.16 8.10 8.02 9/279 27660
/proc/meminfo: memFree=18405428/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=669280 CPUtime=3520.89 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 141613 0 0 0 351964 125 0 0 25 0 4 0 653419628 685342720 137866 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 167320 137866 460 189 0 161669 0
[pid=27632/tid=27634] ppid=27630 vsize=669280 CPUtime=881.85 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 27002 0 0 0 88162 23 0 0 25 0 4 0 653419657 685342720 137866 33554432000 4194304 4964800 140734278257984 18446744073709551615 4370377 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=669280 CPUtime=881.99 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 39612 0 0 0 88167 32 0 0 25 0 4 0 653419657 685342720 137866 33554432000 4194304 4964800 140734278257984 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=669280 CPUtime=874.76 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 49953 0 0 0 87425 51 0 0 25 0 4 0 653419657 685342720 137866 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3520.89
Current children cumulated vsize (KiB) 669280

[startup+942.301 s]
/proc/loadavg: 8.12 8.09 8.02 9/279 27661
/proc/meminfo: memFree=18337340/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=675292 CPUtime=3760.39 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 143080 0 0 0 375913 126 0 0 25 0 4 0 653419628 691499008 139333 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670357 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 168823 139333 460 189 0 163172 0
[pid=27632/tid=27634] ppid=27630 vsize=675292 CPUtime=941.85 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 27002 0 0 0 94162 23 0 0 25 0 4 0 653419657 691499008 139333 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=675292 CPUtime=941.99 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 41029 0 0 0 94166 33 0 0 25 0 4 0 653419657 691499008 139333 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=675292 CPUtime=934.27 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 50002 0 0 0 93376 51 0 0 25 0 4 0 653419657 691499008 139333 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3760.39
Current children cumulated vsize (KiB) 675292

[startup+1002.3 s]
/proc/loadavg: 8.14 8.10 8.03 9/279 27663
/proc/meminfo: memFree=18289712/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=685256 CPUtime=3999.87 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 146663 0 0 0 399859 128 0 0 25 0 4 0 653419628 701702144 142916 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 171314 142916 460 189 0 165663 0
[pid=27632/tid=27634] ppid=27630 vsize=685256 CPUtime=1001.83 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 29303 0 0 0 100159 24 0 0 25 0 4 0 653419657 701702144 142916 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=685256 CPUtime=1001.99 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 42089 0 0 0 100165 34 0 0 25 0 4 0 653419657 701702144 142916 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=685256 CPUtime=993.77 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 50211 0 0 0 99326 51 0 0 25 0 4 0 653419657 701702144 142916 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3999.87
Current children cumulated vsize (KiB) 685256

[startup+1062.3 s]
/proc/loadavg: 8.15 8.11 8.03 9/279 27665
/proc/meminfo: memFree=18171900/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=685256 CPUtime=4239.39 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 147167 0 0 0 423810 129 0 0 25 0 4 0 653419628 701702144 143380 33554432000 4194304 4964800 140734278257984 18446744073709551615 4653186 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 171314 143380 460 189 0 165663 0
[pid=27632/tid=27634] ppid=27630 vsize=685256 CPUtime=1061.82 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 29303 0 0 0 106158 24 0 0 25 0 4 0 653419657 701702144 143380 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=685256 CPUtime=1061.99 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 42107 0 0 0 106165 34 0 0 25 0 4 0 653419657 701702144 143380 33554432000 4194304 4964800 140734278257984 18446744073709551615 4668753 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=685256 CPUtime=1053.29 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 50697 0 0 0 105277 52 0 0 25 0 4 0 653419657 701702144 143380 33554432000 4194304 4964800 140734278257984 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4239.39
Current children cumulated vsize (KiB) 685256

[startup+1122.3 s]
/proc/loadavg: 7.26 7.92 7.97 5/270 27684
/proc/meminfo: memFree=20345328/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=750792 CPUtime=4478.88 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 155757 0 0 0 447749 139 0 0 25 0 4 0 653419628 768811008 151970 33554432000 4194304 4964800 140734278257984 18446744073709551615 4653186 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 187698 151970 460 189 0 182047 0
[pid=27632/tid=27634] ppid=27630 vsize=750792 CPUtime=1121.81 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 29303 0 0 0 112157 24 0 0 25 0 4 0 653419657 768811008 151970 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=750792 CPUtime=1121.99 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 45671 0 0 0 112160 39 0 0 25 0 4 0 653419657 768811008 151970 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=750792 CPUtime=1112.8 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 52829 0 0 0 111226 54 0 0 25 0 4 0 653419657 768811008 151970 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4478.88
Current children cumulated vsize (KiB) 750792

[startup+1182.3 s]
/proc/loadavg: 5.19 7.21 7.72 5/270 27686
/proc/meminfo: memFree=20333488/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=752044 CPUtime=4718.39 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 158653 0 0 0 471696 143 0 0 25 0 4 0 653419628 770093056 154847 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 188011 154847 460 189 0 182360 0
[pid=27632/tid=27634] ppid=27630 vsize=752044 CPUtime=1181.8 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 29348 0 0 0 118156 24 0 0 25 0 4 0 653419657 770093056 154847 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=752044 CPUtime=1181.99 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 45950 0 0 0 118160 39 0 0 25 0 4 0 653419657 770093056 154847 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=752044 CPUtime=1172.31 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 52906 0 0 0 117177 54 0 0 25 0 4 0 653419657 770093056 154847 33554432000 4194304 4964800 140734278257984 18446744073709551615 4669429 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4718.39
Current children cumulated vsize (KiB) 752044



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.8 s]
/proc/loadavg: 4.85 7.00 7.64 5/270 27686
/proc/meminfo: memFree=20333460/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=752044 CPUtime=4800.24 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 158653 0 0 0 479881 143 0 0 25 0 4 0 653419628 770093056 154847 33554432000 4194304 4964800 140734278257984 18446744073709551615 4668702 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/27632/statm: 188011 154847 460 189 0 182360 0
[pid=27632/tid=27634] ppid=27630 vsize=752044 CPUtime=1202.31 cores=0,2,4,6
/proc/27632/task/27634/stat : 27634 (claspmt) R 27630 27632 26831 0 -1 4202560 29348 0 0 0 120207 24 0 0 25 0 4 0 653419657 770093056 154847 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=27632/tid=27635] ppid=27630 vsize=752044 CPUtime=1202.5 cores=0,2,4,6
/proc/27632/task/27635/stat : 27635 (claspmt) R 27630 27632 26831 0 -1 4202560 45950 0 0 0 120211 39 0 0 25 0 4 0 653419657 770093056 154847 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=27632/tid=27636] ppid=27630 vsize=752044 CPUtime=1192.64 cores=0,2,4,6
/proc/27632/task/27636/stat : 27636 (claspmt) R 27630 27632 26831 0 -1 4202560 52906 0 0 0 119210 54 0 0 25 0 4 0 653419657 770093056 154847 33554432000 4194304 4964800 140734278257984 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 752044

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202.92 s]
/proc/loadavg: 4.85 7.00 7.64 5/270 27686
/proc/meminfo: memFree=20333460/32950904 swapFree=67057400/67111528
[pid=27632] ppid=27630 vsize=740060 CPUtime=4800.37 cores=0,2,4,6
/proc/27632/stat : 27632 (claspmt) R 27630 27632 26831 0 -1 4202496 158664 0 0 0 479894 143 0 0 25 0 1 0 653419628 757821440 152005 33554432000 4194304 4964800 140734278257984 18446744073709551615 237278537490 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/27632/statm: 185015 152005 471 189 0 179364 0
Current children cumulated CPU time (s) 4800.37
Current children cumulated vsize (KiB) 740060

Child status: 0
Real time (s): 1202.95
CPU time (s): 4800.41
CPU user time (s): 4798.94
CPU system time (s): 1.46778
CPU usage (%): 399.052
Max. virtual memory (cumulated for all children) (KiB): 752044

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.94
system time used= 1.46778
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 158665
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4494
involuntary context switches= 22805

runsolver used 2.73458 second user time and 7.15991 second system time

The end

Launcher Data 

Begin job on node121 at 2011-05-15 02:09:59
IDJOB=3373461
IDBENCH=20529
IDSOLVER=1853
FILE ID=node121/3373461-1305418199
RUNJOBID= node121-1305418100-27577
PBS_JOBID= 13324525
Free space on /tmp= 73416 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT07/industrial/IBM_FV_2002_13_rule_1/SAT_dat.k20.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3373461-1305418199/watcher-3373461-1305418199 -o /tmp/evaluation-result-3373461-1305418199/solver-3373461-1305418199 -C 4800 -W 1300 -M 15500  HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373461-1305418199.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 2bee695ce4278b72ee9295631ad94252
RANDOM SEED=1002836104

node121.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950904 kB
MemFree:      20507304 kB
Buffers:        303232 kB
Cached:        5985980 kB
SwapCached:       7656 kB
Active:        6874532 kB
Inactive:      5217108 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950904 kB
LowFree:      20507304 kB
SwapTotal:    67111528 kB
SwapFree:     67057400 kB
Dirty:            3924 kB
Writeback:           8 kB
AnonPages:     5800096 kB
Mapped:          22528 kB
Slab:           265344 kB
PageTables:      25064 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586980 kB
Committed_AS: 31110000 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73428 MiB
End job on node121 at 2011-05-15 02:30:04