Trace number 3381390

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40002.5 5014.73

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m52-q1041.25-n50-p15-s99099953.cnf
MD5SUMdda220d08783aabf4e7a9e99eeeccae6
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2444
Number of clauses498568
Sum of the clauses size999480
Maximum clause length52
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2498518
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 550

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...514721/instance-3381390-1305514721.cnf
3.89/3.98	c Solving...
40000.22/5012.52	
40000.22/5012.52	*** INTERRUPTED! ***
40000.22/5012.52	*** Shutting down threads...

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3381390-1305514721/watcher-3381390-1305514721 -o /tmp/evaluation-result-3381390-1305514721/solver-3381390-1305514721 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381390-1305514721.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.38 7.89 7.96 2/396 29787
/proc/meminfo: memFree=14646492/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=23380 CPUtime=0.01 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 1292 0 0 0 1 0 0 0 25 0 1 0 741702826 23941120 1033 33554432000 4194304 4964800 140735099447616 18446744073709551615 256415391488 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/29787/statm: 5845 1035 386 189 0 719 0

[startup+0.108563 s]
/proc/loadavg: 7.38 7.89 7.96 2/396 29787
/proc/meminfo: memFree=14646492/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=42096 CPUtime=0.09 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 8078 0 0 0 9 0 0 0 25 0 1 0 741702826 43106304 5320 33554432000 4194304 4964800 140735099447616 18446744073709551615 256415391488 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/29787/statm: 10524 5322 386 189 0 5398 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 42096

[startup+0.200545 s]
/proc/loadavg: 7.38 7.89 7.96 2/396 29787
/proc/meminfo: memFree=14646492/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=54708 CPUtime=0.19 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 11085 0 0 0 18 1 0 0 25 0 1 0 741702826 56020992 8305 33554432000 4194304 4964800 140735099447616 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/29787/statm: 13677 8305 399 189 0 8551 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 54708

[startup+0.30051 s]
/proc/loadavg: 7.38 7.89 7.96 2/396 29787
/proc/meminfo: memFree=14646492/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=54708 CPUtime=0.29 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 11085 0 0 0 28 1 0 0 25 0 1 0 741702826 56020992 8305 33554432000 4194304 4964800 140735099447616 18446744073709551615 4607562 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/29787/statm: 13677 8305 399 189 0 8551 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 54708

[startup+0.700448 s]
/proc/loadavg: 7.38 7.89 7.96 2/396 29787
/proc/meminfo: memFree=14646492/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=54708 CPUtime=0.69 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 11085 0 0 0 68 1 0 0 25 0 1 0 741702826 56020992 8305 33554432000 4194304 4964800 140735099447616 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/29787/statm: 13677 8305 399 189 0 8551 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 54708

[startup+1.50029 s]
/proc/loadavg: 7.38 7.89 7.96 2/397 29788
/proc/meminfo: memFree=14614636/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=54708 CPUtime=1.49 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 11085 0 0 0 148 1 0 0 25 0 1 0 741702826 56020992 8305 33554432000 4194304 4964800 140735099447616 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/29787/statm: 13677 8305 399 189 0 8551 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 54708

[startup+3.10099 s]
/proc/loadavg: 7.38 7.89 7.96 2/397 29788
/proc/meminfo: memFree=14613768/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=55632 CPUtime=3.09 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 11338 0 0 0 308 1 0 0 25 0 1 0 741702826 56967168 8558 33554432000 4194304 4964800 140735099447616 18446744073709551615 4600312 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/29787/statm: 13908 8558 400 189 0 8782 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 55632

[startup+6.3003 s]
/proc/loadavg: 6.87 7.78 7.92 9/404 29795
/proc/meminfo: memFree=14551548/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=411464 CPUtime=20.35 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 33979 0 0 0 2020 15 0 0 18 0 8 0 741702826 421339136 30205 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/29787/statm: 102866 30205 464 189 0 97215 0
[pid=29787/tid=29789] ppid=29785 vsize=411464 CPUtime=1.98 cores=0-7
/proc/29787/task/29789/stat : 29789 (claspmt) R 29785 29787 29105 0 -1 4202560 2464 0 0 0 197 1 0 0 18 0 8 0 741703225 421339136 30205 33554432000 4194304 4964800 140735099447616 18446744073709551615 4627973 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29787/tid=29790] ppid=29785 vsize=411464 CPUtime=2.03 cores=0-7
/proc/29787/task/29790/stat : 29790 (claspmt) R 29785 29787 29105 0 -1 4202560 2729 0 0 0 202 1 0 0 18 0 8 0 741703225 421339136 30205 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670330 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=29787/tid=29791] ppid=29785 vsize=411464 CPUtime=2 cores=0-7
/proc/29787/task/29791/stat : 29791 (claspmt) R 29785 29787 29105 0 -1 4202560 3099 0 0 0 198 2 0 0 18 0 8 0 741703225 421339136 30205 33554432000 4194304 4964800 140735099447616 18446744073709551615 4627259 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29787/tid=29792] ppid=29785 vsize=411464 CPUtime=2.01 cores=0-7
/proc/29787/task/29792/stat : 29792 (claspmt) R 29785 29787 29105 0 -1 4202560 3315 0 0 0 199 2 0 0 18 0 8 0 741703225 421339136 30207 33554432000 4194304 4964800 140735099447616 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29787/tid=29793] ppid=29785 vsize=411464 CPUtime=2.29 cores=0-7
/proc/29787/task/29793/stat : 29793 (claspmt) R 29785 29787 29105 0 -1 4202560 1080 0 0 0 229 0 0 0 21 0 8 0 741703225 421339136 30207 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670261 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=29787/tid=29794] ppid=29785 vsize=411464 CPUtime=2.02 cores=0-7
/proc/29787/task/29794/stat : 29794 (claspmt) R 29785 29787 29105 0 -1 4202560 2587 0 0 0 201 1 0 0 21 0 8 0 741703225 421339136 30207 33554432000 4194304 4964800 140735099447616 18446744073709551615 256415052833 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29787/tid=29795] ppid=29785 vsize=411464 CPUtime=2.01 cores=0-7
/proc/29787/task/29795/stat : 29795 (claspmt) R 29785 29787 29105 0 -1 4202560 4214 0 0 0 199 2 0 0 18 0 8 0 741703225 421339136 30207 33554432000 4194304 4964800 140735099447616 18446744073709551615 256415052485 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 20.35
Current children cumulated vsize (KiB) 411464

[startup+12.7011 s]
/proc/loadavg: 6.96 7.78 7.92 9/404 29795
/proc/meminfo: memFree=14356400/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=504220 CPUtime=71.43 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 81969 0 0 0 7105 38 0 0 25 0 8 0 741702826 516321280 78195 33554432000 4194304 4964800 140735099447616 18446744073709551615 4320951 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/29787/statm: 126055 78195 466 189 0 120404 0
[pid=29787/tid=29789] ppid=29785 vsize=504220 CPUtime=8.3 cores=0-7
/proc/29787/task/29789/stat : 29789 (claspmt) R 29785 29787 29105 0 -1 4202560 8989 0 0 0 825 5 0 0 25 0 8 0 741703225 516321280 78195 33554432000 4194304 4964800 140735099447616 18446744073709551615 4334332 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29787/tid=29790] ppid=29785 vsize=504220 CPUtime=8.42 cores=0-7
/proc/29787/task/29790/stat : 29790 (claspmt) R 29785 29787 29105 0 -1 4202560 5963 0 0 0 840 2 0 0 25 0 8 0 741703225 516321280 78195 33554432000 4194304 4964800 140735099447616 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=29787/tid=29791] ppid=29785 vsize=504220 CPUtime=8.4 cores=0-7
/proc/29787/task/29791/stat : 29791 (claspmt) R 29785 29787 29105 0 -1 4202560 9040 0 0 0 835 5 0 0 25 0 8 0 741703225 516321280 78195 33554432000 4194304 4964800 140735099447616 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29787/tid=29792] ppid=29785 vsize=504220 CPUtime=8.4 cores=0-7
/proc/29787/task/29792/stat : 29792 (claspmt) R 29785 29787 29105 0 -1 4202560 9225 0 0 0 836 4 0 0 25 0 8 0 741703225 516321280 78195 33554432000 4194304 4964800 140735099447616 18446744073709551615 4637196 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29787/tid=29793] ppid=29785 vsize=504220 CPUtime=8.69 cores=0-7
/proc/29787/task/29793/stat : 29793 (claspmt) R 29785 29787 29105 0 -1 4202560 7392 0 0 0 867 2 0 0 25 0 8 0 741703225 516321280 78195 33554432000 4194304 4964800 140735099447616 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=29787/tid=29794] ppid=29785 vsize=504220 CPUtime=8.42 cores=0-7
/proc/29787/task/29794/stat : 29794 (claspmt) R 29785 29787 29105 0 -1 4202560 7678 0 0 0 839 3 0 0 25 0 8 0 741703225 516321280 78195 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29787/tid=29795] ppid=29785 vsize=504220 CPUtime=8.41 cores=0-7
/proc/29787/task/29795/stat : 29795 (claspmt) R 29785 29787 29105 0 -1 4202560 11908 0 0 0 834 7 0 0 25 0 8 0 741703225 516321280 78195 33554432000 4194304 4964800 140735099447616 18446744073709551615 4628021 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 71.43
Current children cumulated vsize (KiB) 504220

[startup+25.5007 s]
/proc/loadavg: 7.19 7.79 7.92 9/404 29795
/proc/meminfo: memFree=14089616/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=875044 CPUtime=173.64 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 145097 0 0 0 17291 73 0 0 25 0 8 0 741702826 896045056 141323 33554432000 4194304 4964800 140735099447616 18446744073709551615 256415052833 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/29787/statm: 218761 141323 466 189 0 213110 0
[pid=29787/tid=29789] ppid=29785 vsize=875044 CPUtime=20.92 cores=0-7
/proc/29787/task/29789/stat : 29789 (claspmt) R 29785 29787 29105 0 -1 4202560 15522 0 0 0 2083 9 0 0 25 0 8 0 741703225 896045056 141323 33554432000 4194304 4964800 140735099447616 18446744073709551615 4653595 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29787/tid=29790] ppid=29785 vsize=875044 CPUtime=21.22 cores=0-7
/proc/29787/task/29790/stat : 29790 (claspmt) R 29785 29787 29105 0 -1 4202560 9107 0 0 0 2118 4 0 0 25 0 8 0 741703225 896045056 141323 33554432000 4194304 4964800 140735099447616 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=29787/tid=29791] ppid=29785 vsize=875044 CPUtime=21.2 cores=0-7
/proc/29787/task/29791/stat : 29791 (claspmt) R 29785 29787 29105 0 -1 4202560 17718 0 0 0 2109 11 0 0 18 0 8 0 741703225 896045056 141323 33554432000 4194304 4964800 140735099447616 18446744073709551615 4342186 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29787/tid=29792] ppid=29785 vsize=875044 CPUtime=21.2 cores=0-7
/proc/29787/task/29792/stat : 29792 (claspmt) R 29785 29787 29105 0 -1 4202560 18320 0 0 0 2112 8 0 0 19 0 8 0 741703225 896045056 141323 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670321 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29787/tid=29793] ppid=29785 vsize=875044 CPUtime=21.48 cores=0-7
/proc/29787/task/29793/stat : 29793 (claspmt) R 29785 29787 29105 0 -1 4202560 14807 0 0 0 2142 6 0 0 25 0 8 0 741703225 896045056 141323 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=29787/tid=29794] ppid=29785 vsize=875044 CPUtime=21.22 cores=0-7
/proc/29787/task/29794/stat : 29794 (claspmt) R 29785 29787 29105 0 -1 4202560 12309 0 0 0 2117 5 0 0 25 0 8 0 741703225 896045056 141324 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29787/tid=29795] ppid=29785 vsize=875044 CPUtime=21.21 cores=0-7
/proc/29787/task/29795/stat : 29795 (claspmt) R 29785 29787 29105 0 -1 4202560 25987 0 0 0 2106 15 0 0 25 0 8 0 741703225 896045056 141324 33554432000 4194304 4964800 140735099447616 18446744073709551615 256415052833 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 173.64
Current children cumulated vsize (KiB) 875044

[startup+51.1008 s]
/proc/loadavg: 7.53 7.82 7.92 9/404 29796
/proc/meminfo: memFree=13750028/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=1113580 CPUtime=378.08 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 228900 0 0 0 37679 129 0 0 25 0 8 0 741702826 1140305920 225126 33554432000 4194304 4964800 140735099447616 18446744073709551615 4362211 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/29787/statm: 278395 225126 466 189 0 272744 0
[pid=29787/tid=29789] ppid=29785 vsize=1113580 CPUtime=46.19 cores=0-7
/proc/29787/task/29789/stat : 29789 (claspmt) R 29785 29787 29105 0 -1 4202560 27875 0 0 0 4600 19 0 0 25 0 8 0 741703225 1140305920 225126 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670361 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29787/tid=29790] ppid=29785 vsize=1113580 CPUtime=46.82 cores=0-7
/proc/29787/task/29790/stat : 29790 (claspmt) R 29785 29787 29105 0 -1 4202560 12948 0 0 0 4676 6 0 0 25 0 8 0 741703225 1140305920 225126 33554432000 4194304 4964800 140735099447616 18446744073709551615 4627536 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=29787/tid=29791] ppid=29785 vsize=1113580 CPUtime=46.8 cores=0-7
/proc/29787/task/29791/stat : 29791 (claspmt) R 29785 29787 29105 0 -1 4202560 24480 0 0 0 4664 16 0 0 25 0 8 0 741703225 1140305920 225126 33554432000 4194304 4964800 140735099447616 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29787/tid=29792] ppid=29785 vsize=1113580 CPUtime=46.8 cores=0-7
/proc/29787/task/29792/stat : 29792 (claspmt) R 29785 29787 29105 0 -1 4202560 23482 0 0 0 4668 12 0 0 25 0 8 0 741703225 1140305920 225126 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29787/tid=29793] ppid=29785 vsize=1113580 CPUtime=47.08 cores=0-7
/proc/29787/task/29793/stat : 29793 (claspmt) R 29785 29787 29105 0 -1 4202560 26163 0 0 0 4696 12 0 0 25 0 8 0 741703225 1140305920 225126 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=29787/tid=29794] ppid=29785 vsize=1113580 CPUtime=46.82 cores=0-7
/proc/29787/task/29794/stat : 29794 (claspmt) R 29785 29787 29105 0 -1 4202560 20304 0 0 0 4673 9 0 0 25 0 8 0 741703225 1140305920 225126 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29787/tid=29795] ppid=29785 vsize=1113580 CPUtime=46.8 cores=0-7
/proc/29787/task/29795/stat : 29795 (claspmt) R 29785 29787 29105 0 -1 4202560 39042 0 0 0 4655 25 0 0 25 0 8 0 741703225 1140305920 225126 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670264 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 378.08
Current children cumulated vsize (KiB) 1113580

[startup+102.301 s]
/proc/loadavg: 7.94 7.88 7.94 9/404 29797
/proc/meminfo: memFree=13574920/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=1401408 CPUtime=786.97 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 271868 0 0 0 78534 163 0 0 25 0 8 0 741702826 1435041792 268025 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/29787/statm: 350352 268025 466 189 0 344701 0
[pid=29787/tid=29789] ppid=29785 vsize=1401408 CPUtime=97.09 cores=0-7
/proc/29787/task/29789/stat : 29789 (claspmt) R 29785 29787 29105 0 -1 4202560 37376 0 0 0 9682 27 0 0 25 0 8 0 741703225 1435041792 268025 33554432000 4194304 4964800 140735099447616 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29787/tid=29790] ppid=29785 vsize=1401408 CPUtime=98.03 cores=0-7
/proc/29787/task/29790/stat : 29790 (claspmt) R 29785 29787 29105 0 -1 4202560 15129 0 0 0 9797 6 0 0 25 0 8 0 741703225 1435041792 268025 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=29787/tid=29791] ppid=29785 vsize=1401408 CPUtime=97.67 cores=0-7
/proc/29787/task/29791/stat : 29791 (claspmt) R 29785 29787 29105 0 -1 4202560 27009 0 0 0 9748 19 0 0 25 0 8 0 741703225 1435041792 268025 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29787/tid=29792] ppid=29785 vsize=1401408 CPUtime=97.95 cores=0-7
/proc/29787/task/29792/stat : 29792 (claspmt) R 29785 29787 29105 0 -1 4202560 25843 0 0 0 9782 13 0 0 25 0 8 0 741703225 1435041792 268025 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670368 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29787/tid=29793] ppid=29785 vsize=1401408 CPUtime=98.27 cores=0-7
/proc/29787/task/29793/stat : 29793 (claspmt) R 29785 29787 29105 0 -1 4202560 28777 0 0 0 9813 14 0 0 25 0 8 0 741703225 1435041792 268025 33554432000 4194304 4964800 140735099447616 18446744073709551615 4321093 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=29787/tid=29794] ppid=29785 vsize=1401408 CPUtime=97.96 cores=0-7
/proc/29787/task/29794/stat : 29794 (claspmt) R 29785 29787 29105 0 -1 4202560 22078 0 0 0 9785 11 0 0 25 0 8 0 741703225 1435041792 268025 33554432000 4194304 4964800 140735099447616 18446744073709551615 4627365 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29787/tid=29795] ppid=29785 vsize=1401408 CPUtime=98.01 cores=0-7
/proc/29787/task/29795/stat : 29795 (claspmt) R 29785 29787 29105 0 -1 4202560 51030 0 0 0 9765 36 0 0 25 0 8 0 741703225 1435041792 268025 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670361 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 786.97
Current children cumulated vsize (KiB) 1401408

[startup+162.301 s]
/proc/loadavg: 8.04 7.91 7.94 9/404 29808
/proc/meminfo: memFree=13320988/32950928 swapFree=67111200/67111528

################
# More data... #
################


[startup+4782.3 s]
/proc/loadavg: 8.08 8.06 8.00 9/404 29943
/proc/meminfo: memFree=9997888/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=4987148 CPUtime=38161.5 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 1169347 0 0 0 3814929 1220 0 0 25 0 8 0 741702826 5106839552 1160258 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/29787/statm: 1246787 1160258 466 189 0 1241136 0
[pid=29787/tid=29789] ppid=29785 vsize=4987148 CPUtime=4776.22 cores=0-7
/proc/29787/task/29789/stat : 29789 (claspmt) R 29785 29787 29105 0 -1 4202560 113504 0 0 0 477524 98 0 0 25 0 8 0 741703225 5106839552 1160258 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670351 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29787/tid=29790] ppid=29785 vsize=4987148 CPUtime=4778.09 cores=0-7
/proc/29787/task/29790/stat : 29790 (claspmt) R 29785 29787 29105 0 -1 4202560 52352 0 0 0 477777 32 0 0 25 0 8 0 741703225 5106839552 1160258 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=29787/tid=29791] ppid=29785 vsize=4987148 CPUtime=4729.88 cores=0-7
/proc/29787/task/29791/stat : 29791 (claspmt) R 29785 29787 29105 0 -1 4202560 92802 0 0 0 472879 109 0 0 25 0 8 0 741703225 5106839552 1160258 33554432000 4194304 4964800 140735099447616 18446744073709551615 4321097 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29787/tid=29792] ppid=29785 vsize=4987148 CPUtime=4771.24 cores=0-7
/proc/29787/task/29792/stat : 29792 (claspmt) R 29785 29787 29105 0 -1 4202560 68974 0 0 0 477050 74 0 0 25 0 8 0 741703225 5106839552 1160258 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=29787/tid=29793] ppid=29785 vsize=4987148 CPUtime=4776.99 cores=0-7
/proc/29787/task/29793/stat : 29793 (claspmt) R 29785 29787 29105 0 -1 4202560 81196 0 0 0 477634 65 0 0 25 0 8 0 741703225 5106839552 1160258 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29787/tid=29794] ppid=29785 vsize=4987148 CPUtime=4773.93 cores=0-7
/proc/29787/task/29794/stat : 29794 (claspmt) R 29785 29787 29105 0 -1 4202560 91860 0 0 0 477287 106 0 0 25 0 8 0 741703225 5106839552 1160258 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29787/tid=29795] ppid=29785 vsize=4987148 CPUtime=4774.15 cores=0-7
/proc/29787/task/29795/stat : 29795 (claspmt) R 29785 29787 29105 0 -1 4202560 346638 0 0 0 476985 430 0 0 25 0 8 0 741703225 5106839552 1160258 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38161.5
Current children cumulated vsize (KiB) 4987148

[startup+4842.32 s]
/proc/loadavg: 8.07 8.06 8.00 9/404 29945
/proc/meminfo: memFree=9996596/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=4987880 CPUtime=38640.8 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 1169610 0 0 0 3862863 1220 0 0 25 0 8 0 741702826 5107589120 1160521 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/29787/statm: 1246970 1160521 466 189 0 1241319 0
[pid=29787/tid=29789] ppid=29785 vsize=4987880 CPUtime=4836.23 cores=0-7
/proc/29787/task/29789/stat : 29789 (claspmt) R 29785 29787 29105 0 -1 4202560 113606 0 0 0 483525 98 0 0 25 0 8 0 741703225 5107589120 1160521 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670313 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29787/tid=29790] ppid=29785 vsize=4987880 CPUtime=4838.1 cores=0-7
/proc/29787/task/29790/stat : 29790 (claspmt) R 29785 29787 29105 0 -1 4202560 52507 0 0 0 483778 32 0 0 25 0 8 0 741703225 5107589120 1160521 33554432000 4194304 4964800 140735099447616 18446744073709551615 4321038 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=29787/tid=29791] ppid=29785 vsize=4987880 CPUtime=4789.13 cores=0-7
/proc/29787/task/29791/stat : 29791 (claspmt) R 29785 29787 29105 0 -1 4202560 92804 0 0 0 478804 109 0 0 25 0 8 0 741703225 5107589120 1160521 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29787/tid=29792] ppid=29785 vsize=4987880 CPUtime=4831.25 cores=0-7
/proc/29787/task/29792/stat : 29792 (claspmt) R 29785 29787 29105 0 -1 4202560 68974 0 0 0 483051 74 0 0 25 0 8 0 741703225 5107589120 1160521 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=29787/tid=29793] ppid=29785 vsize=4987880 CPUtime=4837.01 cores=0-7
/proc/29787/task/29793/stat : 29793 (claspmt) R 29785 29787 29105 0 -1 4202560 81200 0 0 0 483636 65 0 0 25 0 8 0 741703225 5107589120 1160521 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29787/tid=29794] ppid=29785 vsize=4987880 CPUtime=4833.95 cores=0-7
/proc/29787/task/29794/stat : 29794 (claspmt) R 29785 29787 29105 0 -1 4202560 91860 0 0 0 483289 106 0 0 25 0 8 0 741703225 5107589120 1160521 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29787/tid=29795] ppid=29785 vsize=4987880 CPUtime=4834.17 cores=0-7
/proc/29787/task/29795/stat : 29795 (claspmt) R 29785 29787 29105 0 -1 4202560 346638 0 0 0 482987 430 0 0 25 0 8 0 741703225 5107589120 1160521 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38640.8
Current children cumulated vsize (KiB) 4987880

[startup+4902.3 s]
/proc/loadavg: 8.11 8.08 8.01 9/404 29946
/proc/meminfo: memFree=9990460/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=4990040 CPUtime=39119.9 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 1171132 0 0 0 3910775 1220 0 0 25 0 8 0 741702826 5109800960 1162043 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/29787/statm: 1247510 1162043 466 189 0 1241859 0
[pid=29787/tid=29789] ppid=29785 vsize=4990040 CPUtime=4896.22 cores=0-7
/proc/29787/task/29789/stat : 29789 (claspmt) R 29785 29787 29105 0 -1 4202560 114824 0 0 0 489524 98 0 0 25 0 8 0 741703225 5109800960 1162043 33554432000 4194304 4964800 140735099447616 18446744073709551615 4627281 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29787/tid=29790] ppid=29785 vsize=4990040 CPUtime=4898.09 cores=0-7
/proc/29787/task/29790/stat : 29790 (claspmt) R 29785 29787 29105 0 -1 4202560 52702 0 0 0 489777 32 0 0 25 0 8 0 741703225 5109800960 1162043 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=29787/tid=29791] ppid=29785 vsize=4990040 CPUtime=4848.35 cores=0-7
/proc/29787/task/29791/stat : 29791 (claspmt) R 29785 29787 29105 0 -1 4202560 92815 0 0 0 484726 109 0 0 25 0 8 0 741703225 5109800960 1162043 33554432000 4194304 4964800 140735099447616 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29787/tid=29792] ppid=29785 vsize=4990040 CPUtime=4891.24 cores=0-7
/proc/29787/task/29792/stat : 29792 (claspmt) R 29785 29787 29105 0 -1 4202560 68974 0 0 0 489050 74 0 0 25 0 8 0 741703225 5109800960 1162043 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=29787/tid=29793] ppid=29785 vsize=4990040 CPUtime=4897 cores=0-7
/proc/29787/task/29793/stat : 29793 (claspmt) R 29785 29787 29105 0 -1 4202560 81205 0 0 0 489635 65 0 0 25 0 8 0 741703225 5109800960 1162043 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29787/tid=29794] ppid=29785 vsize=4990040 CPUtime=4893.94 cores=0-7
/proc/29787/task/29794/stat : 29794 (claspmt) R 29785 29787 29105 0 -1 4202560 91953 0 0 0 489288 106 0 0 25 0 8 0 741703225 5109800960 1162043 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29787/tid=29795] ppid=29785 vsize=4990040 CPUtime=4894.15 cores=0-7
/proc/29787/task/29795/stat : 29795 (claspmt) R 29785 29787 29105 0 -1 4202560 346638 0 0 0 488985 430 0 0 25 0 8 0 741703225 5109800960 1162043 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39119.9
Current children cumulated vsize (KiB) 4990040

[startup+4962.3 s]
/proc/loadavg: 8.08 8.08 8.01 9/404 29948
/proc/meminfo: memFree=9966356/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=4990040 CPUtime=39599.2 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 1177343 0 0 0 3958690 1227 0 0 25 0 8 0 741702826 5109800960 1168180 33554432000 4194304 4964800 140735099447616 18446744073709551615 4320951 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/29787/statm: 1247510 1168180 466 189 0 1241859 0
[pid=29787/tid=29789] ppid=29785 vsize=4990040 CPUtime=4956.22 cores=0-7
/proc/29787/task/29789/stat : 29789 (claspmt) R 29785 29787 29105 0 -1 4202560 119841 0 0 0 495518 104 0 0 25 0 8 0 741703225 5109800960 1168180 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29787/tid=29790] ppid=29785 vsize=4990040 CPUtime=4958.09 cores=0-7
/proc/29787/task/29790/stat : 29790 (claspmt) R 29785 29787 29105 0 -1 4202560 52707 0 0 0 495777 32 0 0 25 0 8 0 741703225 5109800960 1168180 33554432000 4194304 4964800 140735099447616 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=29787/tid=29791] ppid=29785 vsize=4990040 CPUtime=4907.58 cores=0-7
/proc/29787/task/29791/stat : 29791 (claspmt) R 29785 29787 29105 0 -1 4202560 92892 0 0 0 490648 110 0 0 25 0 8 0 741703225 5109800960 1168180 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29787/tid=29792] ppid=29785 vsize=4990040 CPUtime=4951.24 cores=0-7
/proc/29787/task/29792/stat : 29792 (claspmt) R 29785 29787 29105 0 -1 4202560 68974 0 0 0 495050 74 0 0 25 0 8 0 741703225 5109800960 1168180 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=29787/tid=29793] ppid=29785 vsize=4990040 CPUtime=4957 cores=0-7
/proc/29787/task/29793/stat : 29793 (claspmt) R 29785 29787 29105 0 -1 4202560 81980 0 0 0 495635 65 0 0 25 0 8 0 741703225 5109800960 1168180 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29787/tid=29794] ppid=29785 vsize=4990040 CPUtime=4953.94 cores=0-7
/proc/29787/task/29794/stat : 29794 (claspmt) R 29785 29787 29105 0 -1 4202560 92085 0 0 0 495288 106 0 0 25 0 8 0 741703225 5109800960 1168180 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29787/tid=29795] ppid=29785 vsize=4990040 CPUtime=4954.16 cores=0-7
/proc/29787/task/29795/stat : 29795 (claspmt) R 29785 29787 29105 0 -1 4202560 346683 0 0 0 494985 431 0 0 25 0 8 0 741703225 5109800960 1168180 33554432000 4194304 4964800 140735099447616 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39599.2
Current children cumulated vsize (KiB) 4990040



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5012.5 s]
/proc/loadavg: 8.03 8.06 8.00 9/404 29949
/proc/meminfo: memFree=9958632/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=4990040 CPUtime=40000.2 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4202496 1179348 0 0 0 3998795 1227 0 0 25 0 8 0 741702826 5109800960 1170175 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/29787/statm: 1247510 1170175 466 189 0 1241859 0
[pid=29787/tid=29789] ppid=29785 vsize=4990040 CPUtime=5006.44 cores=0-7
/proc/29787/task/29789/stat : 29789 (claspmt) R 29785 29787 29105 0 -1 4202560 120520 0 0 0 500540 104 0 0 25 0 8 0 741703225 5109800960 1170175 33554432000 4194304 4964800 140735099447616 18446744073709551615 4644872 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29787/tid=29790] ppid=29785 vsize=4990040 CPUtime=5008.31 cores=0-7
/proc/29787/task/29790/stat : 29790 (claspmt) R 29785 29787 29105 0 -1 4202560 52752 0 0 0 500799 32 0 0 25 0 8 0 741703225 5109800960 1170175 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=29787/tid=29791] ppid=29785 vsize=4990040 CPUtime=4957.14 cores=0-7
/proc/29787/task/29791/stat : 29791 (claspmt) R 29785 29787 29105 0 -1 4202560 92935 0 0 0 495604 110 0 0 25 0 8 0 741703225 5109800960 1170175 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29787/tid=29792] ppid=29785 vsize=4990040 CPUtime=5001.45 cores=0-7
/proc/29787/task/29792/stat : 29792 (claspmt) R 29785 29787 29105 0 -1 4202560 68974 0 0 0 500071 74 0 0 25 0 8 0 741703225 5109800960 1170175 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=29787/tid=29793] ppid=29785 vsize=4990040 CPUtime=5007.21 cores=0-7
/proc/29787/task/29793/stat : 29793 (claspmt) R 29785 29787 29105 0 -1 4202560 83178 0 0 0 500656 65 0 0 25 0 8 0 741703225 5109800960 1170175 33554432000 4194304 4964800 140735099447616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29787/tid=29794] ppid=29785 vsize=4990040 CPUtime=5004.15 cores=0-7
/proc/29787/task/29794/stat : 29794 (claspmt) R 29785 29787 29105 0 -1 4202560 92109 0 0 0 500309 106 0 0 25 0 8 0 741703225 5109800960 1170175 33554432000 4194304 4964800 140735099447616 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29787/tid=29795] ppid=29785 vsize=4990040 CPUtime=5004.38 cores=0-7
/proc/29787/task/29795/stat : 29795 (claspmt) R 29785 29787 29105 0 -1 4202560 346698 0 0 0 500007 431 0 0 25 0 8 0 741703225 5109800960 1170175 33554432000 4194304 4964800 140735099447616 18446744073709551615 4342208 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.2
Current children cumulated vsize (KiB) 4990040

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -29787

Solver just ended. Dumping a history of the last processes samples

[startup+5012.7 s]
/proc/loadavg: 7.47 7.94 7.96 2/399 29950
/proc/meminfo: memFree=9958672/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=4990040 CPUtime=40000.5 cores=0-7
/proc/29787/stat : 29787 (claspmt) S 29785 29787 29105 0 -1 4202496 1179350 0 0 0 3998822 1227 0 0 25 0 2 0 741702826 5109800960 1170177 33554432000 4194304 4964800 140735099447616 18446744073709551615 256427194341 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/29787/statm: 1247510 1170177 468 189 0 1241859 0
[pid=29787/tid=29789] ppid=29785 vsize=4990040 CPUtime=5006.64 cores=0-7
/proc/29787/task/29789/stat : 29789 (claspmt) R 29785 29787 29105 0 -1 4202560 120520 0 0 0 500560 104 0 0 25 0 2 0 741703225 5109800960 1170177 33554432000 4194304 4964800 140735099447616 18446744073709551615 256415089235 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 4990040

[startup+5013.5 s]
/proc/loadavg: 7.47 7.94 7.96 2/399 29950
/proc/meminfo: memFree=9958672/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=4990040 CPUtime=40001.3 cores=0-7
/proc/29787/stat : 29787 (claspmt) S 29785 29787 29105 0 -1 4202496 1179350 0 0 0 3998901 1227 0 0 25 0 2 0 741702826 5109800960 1170177 33554432000 4194304 4964800 140735099447616 18446744073709551615 256427194341 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/29787/statm: 1247510 1170177 468 189 0 1241859 0
[pid=29787/tid=29789] ppid=29785 vsize=4990040 CPUtime=5007.42 cores=0-7
/proc/29787/task/29789/stat : 29789 (claspmt) R 29785 29787 29105 0 -1 4202560 120520 0 0 0 500638 104 0 0 25 0 2 0 741703225 5109800960 1170177 33554432000 4194304 4964800 140735099447616 18446744073709551615 256415089266 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 40001.3
Current children cumulated vsize (KiB) 4990040

[startup+5014.3 s]
/proc/loadavg: 7.47 7.94 7.96 2/399 29950
/proc/meminfo: memFree=9958672/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=4990040 CPUtime=40002.1 cores=0-7
/proc/29787/stat : 29787 (claspmt) S 29785 29787 29105 0 -1 4202496 1179350 0 0 0 3998981 1227 0 0 25 0 2 0 741702826 5109800960 1170177 33554432000 4194304 4964800 140735099447616 18446744073709551615 256427194341 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/29787/statm: 1247510 1170177 468 189 0 1241859 0
[pid=29787/tid=29789] ppid=29785 vsize=4990040 CPUtime=5008.22 cores=0-7
/proc/29787/task/29789/stat : 29789 (claspmt) R 29785 29787 29105 0 -1 4202560 120520 0 0 0 500718 104 0 0 25 0 2 0 741703225 5109800960 1170177 33554432000 4194304 4964800 140735099447616 18446744073709551615 256415089235 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 40002.1
Current children cumulated vsize (KiB) 4990040

[startup+5014.7 s]
/proc/loadavg: 7.47 7.94 7.96 2/399 29950
/proc/meminfo: memFree=9958672/32950928 swapFree=67111200/67111528
[pid=29787] ppid=29785 vsize=0 CPUtime=40002.5 cores=0-7
/proc/29787/stat : 29787 (claspmt) R 29785 29787 29105 0 -1 4203524 1179350 0 0 0 3999004 1245 0 0 18 0 1 0 741702826 0 0 33554432000 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/29787/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 40002.5
Current children cumulated vsize (KiB) 0

Child ended because it received signal 9 (SIGKILL)
Real time (s): 5014.73
CPU time (s): 40002.5
CPU user time (s): 39990
CPU system time (s): 12.4811
CPU usage (%): 797.7
Max. virtual memory (cumulated for all children) (KiB): 4990040

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39990
system time used= 12.4811
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1179350
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 110341
involuntary context switches= 187694

runsolver used 17.6483 second user time and 48.0357 second system time

The end

Launcher Data 

Begin job on node115 at 2011-05-16 04:58:42
IDJOB=3381390
IDBENCH=88226
IDSOLVER=1865
FILE ID=node115/3381390-1305514721
RUNJOBID= node115-1305509699-29119
PBS_JOBID= 13325518
Free space on /tmp= 73352 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m52-q1041.25-n50-p15-s99099953.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3381390-1305514721/watcher-3381390-1305514721 -o /tmp/evaluation-result-3381390-1305514721/solver-3381390-1305514721 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381390-1305514721.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= dda220d08783aabf4e7a9e99eeeccae6
RANDOM SEED=622333394

node115.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.62
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      14646648 kB
Buffers:        277228 kB
Cached:        3045036 kB
SwapCached:          0 kB
Active:       14964508 kB
Inactive:      3019092 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      14646648 kB
SwapTotal:    67111528 kB
SwapFree:     67111200 kB
Dirty:            7384 kB
Writeback:           0 kB
AnonPages:    14661572 kB
Mapped:          34020 kB
Slab:           195564 kB
PageTables:      56936 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 100271048 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73352 MiB
End job on node115 at 2011-05-16 06:22:16