Trace number 3373145

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 4800.2 1203.64

General information on the benchmark

NameSAT04/handmade/marijn/equilarge-l/
equilarge_l4.used-as.sat04-896.cnf
MD5SUMfc993d9544ccd8eaeaa4882599387947
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark437.97
Satisfiable
(Un)Satisfiability was proved
Number of variables4499
Number of clauses18603
Sum of the clauses size54454
Maximum clause length3
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21355
Number of clauses of size 317248
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...430533/instance-3373145-1305430533.cnf
0.00/0.04	c Solving...
4800.12/1203.61	
4800.12/1203.61	*** INTERRUPTED! ***
4800.12/1203.61	*** Shutting down threads...
4800.12/1203.64	s UNKNOWN
4800.12/1203.64	
4800.12/1203.64	c INTERRUPTED   : 1
4800.12/1203.64	c Models        : 0+    
4800.12/1203.64	c Time          : 1203.616s (Solving: 1203.58s 1st Model: 0.00s Unsat: 0.00s)
4800.12/1203.64	c CPU Time      : 4800.160s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3373145-1305430533/watcher-3373145-1305430533 -o /tmp/evaluation-result-3373145-1305430533/solver-3373145-1305430533 -C 4800 -W 1300 -M 15500 HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373145-1305430533.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.84 6.58 7.32 5/367 11740
/proc/meminfo: memFree=25528428/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=23148 CPUtime=0 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 1172 0 0 0 0 0 0 0 25 0 1 0 635468958 23703552 989 33554432000 4194304 4964800 140733635359248 18446744073709551615 264726065861 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/11740/statm: 5787 991 396 189 0 661 0

[startup+0.851796 s]
/proc/loadavg: 5.84 6.58 7.32 5/367 11740
/proc/meminfo: memFree=25528428/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=239556 CPUtime=3.26 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 4663 0 0 0 325 1 0 0 25 0 4 0 635468958 245305344 4285 33554432000 4194304 4964800 140733635359248 18446744073709551615 4342199 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 59889 4285 458 189 0 54238 0
Current children cumulated CPU time (s) 3.26
Current children cumulated vsize (KiB) 239556

[startup+0.900788 s]
/proc/loadavg: 5.84 6.58 7.32 5/367 11740
/proc/meminfo: memFree=25528428/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=239556 CPUtime=3.46 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 4664 0 0 0 345 1 0 0 25 0 4 0 635468958 245305344 4286 33554432000 4194304 4964800 140733635359248 18446744073709551615 4333204 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 59889 4286 458 189 0 54238 0
Current children cumulated CPU time (s) 3.46
Current children cumulated vsize (KiB) 239556

[startup+1.00077 s]
/proc/loadavg: 5.84 6.58 7.32 5/367 11740
/proc/meminfo: memFree=25528428/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=239556 CPUtime=3.86 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 4702 0 0 0 385 1 0 0 25 0 4 0 635468958 245305344 4324 33554432000 4194304 4964800 140733635359248 18446744073709551615 4661130 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 59889 4324 458 189 0 54238 0
Current children cumulated CPU time (s) 3.86
Current children cumulated vsize (KiB) 239556

[startup+1.10075 s]
/proc/loadavg: 5.84 6.58 7.32 5/367 11740
/proc/meminfo: memFree=25528428/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=239556 CPUtime=4.26 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 4706 0 0 0 425 1 0 0 25 0 4 0 635468958 245305344 4328 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670330 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 59889 4328 458 189 0 54238 0
Current children cumulated CPU time (s) 4.26
Current children cumulated vsize (KiB) 239556

[startup+1.50067 s]
/proc/loadavg: 5.84 6.58 7.32 5/367 11740
/proc/meminfo: memFree=25528428/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=240372 CPUtime=5.85 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 5345 0 0 0 584 1 0 0 25 0 4 0 635468958 246140928 4967 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670225 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 60093 4967 458 189 0 54442 0
Current children cumulated CPU time (s) 5.85
Current children cumulated vsize (KiB) 240372

[startup+3.10037 s]
/proc/loadavg: 5.84 6.58 7.32 9/371 11744
/proc/meminfo: memFree=25500448/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=241360 CPUtime=12.22 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 6171 0 0 0 1221 1 0 0 25 0 4 0 635468958 247152640 5793 33554432000 4194304 4964800 140733635359248 18446744073709551615 4668568 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 60340 5793 458 189 0 54689 0
[pid=11740/tid=11742] ppid=11738 vsize=241360 CPUtime=3.05 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 1220 0 0 0 305 0 0 0 25 0 4 0 635468963 247152640 5793 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=241360 CPUtime=3.05 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 1145 0 0 0 305 0 0 0 25 0 4 0 635468963 247152640 5793 33554432000 4194304 4964800 140733635359248 18446744073709551615 4668998 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=241360 CPUtime=3.02 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 1000 0 0 0 302 0 0 0 25 0 4 0 635468963 247152640 5793 33554432000 4194304 4964800 140733635359248 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.22
Current children cumulated vsize (KiB) 241360

[startup+6.31377 s]
/proc/loadavg: 6.01 6.61 7.33 9/371 11745
/proc/meminfo: memFree=25489780/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=244204 CPUtime=25.04 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 8550 0 0 0 2502 2 0 0 25 0 4 0 635468958 250064896 8172 33554432000 4194304 4964800 140733635359248 18446744073709551615 4660526 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 61051 8172 460 189 0 55400 0
[pid=11740/tid=11742] ppid=11738 vsize=244204 CPUtime=6.26 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 1916 0 0 0 626 0 0 0 25 0 4 0 635468963 250064896 8172 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=244204 CPUtime=6.26 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 1709 0 0 0 626 0 0 0 25 0 4 0 635468963 250064896 8172 33554432000 4194304 4964800 140733635359248 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=244204 CPUtime=6.19 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 1601 0 0 0 619 0 0 0 25 0 4 0 635468963 250064896 8172 33554432000 4194304 4964800 140733635359248 18446744073709551615 4332324 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 25.04
Current children cumulated vsize (KiB) 244204

[startup+12.7006 s]
/proc/loadavg: 6.25 6.65 7.33 9/371 11745
/proc/meminfo: memFree=25479016/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=244900 CPUtime=50.52 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 10798 0 0 0 5048 4 0 0 25 0 4 0 635468958 250777600 10420 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670180 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 61225 10420 460 189 0 55574 0
[pid=11740/tid=11742] ppid=11738 vsize=244900 CPUtime=12.64 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 2147 0 0 0 1264 0 0 0 25 0 4 0 635468963 250777600 10420 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=244900 CPUtime=12.59 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 2314 0 0 0 1258 1 0 0 25 0 4 0 635468963 250777600 10420 33554432000 4194304 4964800 140733635359248 18446744073709551615 4378499 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=244900 CPUtime=12.57 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 2000 0 0 0 1257 0 0 0 25 0 4 0 635468963 250777600 10420 33554432000 4194304 4964800 140733635359248 18446744073709551615 4660835 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 50.52
Current children cumulated vsize (KiB) 244900

[startup+25.5011 s]
/proc/loadavg: 6.64 6.71 7.34 9/371 11745
/proc/meminfo: memFree=25456948/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=250284 CPUtime=101.56 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 14688 0 0 0 10149 7 0 0 25 0 4 0 635468958 256290816 14229 33554432000 4194304 4964800 140733635359248 18446744073709551615 4660390 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 62571 14229 460 189 0 56920 0
[pid=11740/tid=11742] ppid=11738 vsize=250284 CPUtime=25.45 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 3332 0 0 0 2544 1 0 0 25 0 4 0 635468963 256290816 14229 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=250284 CPUtime=25.23 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 2846 0 0 0 2522 1 0 0 25 0 4 0 635468963 256290816 14229 33554432000 4194304 4964800 140733635359248 18446744073709551615 4668689 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=250284 CPUtime=25.38 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 2864 0 0 0 2537 1 0 0 25 0 4 0 635468963 256290816 14229 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670678 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 101.56
Current children cumulated vsize (KiB) 250284

[startup+51.1003 s]
/proc/loadavg: 7.18 6.83 7.37 9/371 11746
/proc/meminfo: memFree=25443416/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=253284 CPUtime=203.66 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 17314 0 0 0 20355 11 0 0 25 0 4 0 635468958 259362816 16847 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670343 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 63321 16847 460 189 0 57670 0
[pid=11740/tid=11742] ppid=11738 vsize=253284 CPUtime=51.05 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 4097 0 0 0 5103 2 0 0 25 0 4 0 635468963 259362816 16847 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=253284 CPUtime=50.53 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 3124 0 0 0 5052 1 0 0 25 0 4 0 635468963 259362816 16847 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=253284 CPUtime=50.97 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 3596 0 0 0 5095 2 0 0 25 0 4 0 635468963 259362816 16847 33554432000 4194304 4964800 140733635359248 18446744073709551615 4320993 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 203.66
Current children cumulated vsize (KiB) 253284

[startup+102.301 s]
/proc/loadavg: 7.64 7.01 7.40 9/371 11747
/proc/meminfo: memFree=25421424/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=254924 CPUtime=407.85 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 21065 0 0 0 40769 16 0 0 25 0 4 0 635468958 261042176 20550 33554432000 4194304 4964800 140733635359248 18446744073709551615 4364544 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 63731 20550 460 189 0 58080 0
[pid=11740/tid=11742] ppid=11738 vsize=254924 CPUtime=102.25 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 4799 0 0 0 10222 3 0 0 25 0 4 0 635468963 261042176 20550 33554432000 4194304 4964800 140733635359248 18446744073709551615 4342213 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=254924 CPUtime=101.13 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 3235 0 0 0 10111 2 0 0 25 0 4 0 635468963 261042176 20550 33554432000 4194304 4964800 140733635359248 18446744073709551615 4321076 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=254924 CPUtime=102.17 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 4801 0 0 0 10213 4 0 0 25 0 4 0 635468963 261042176 20550 33554432000 4194304 4964800 140733635359248 18446744073709551615 4321020 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 407.85
Current children cumulated vsize (KiB) 254924

[startup+162.3 s]
/proc/loadavg: 8.02 7.23 7.45 9/371 11749
/proc/meminfo: memFree=25406012/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=256712 CPUtime=647.15 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 23664 0 0 0 64694 21 0 0 25 0 4 0 635468958 262873088 23063 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670346 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 64178 23063 460 189 0 58527 0
[pid=11740/tid=11742] ppid=11738 vsize=256712 CPUtime=162.25 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 5635 0 0 0 16221 4 0 0 25 0 4 0 635468963 262873088 23063 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670371 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=256712 CPUtime=160.42 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 3594 0 0 0 16040 2 0 0 25 0 4 0 635468963 262873088 23063 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670330 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=256712 CPUtime=162.17 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 4953 0 0 0 16213 4 0 0 25 0 4 0 635468963 262873088 23063 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 647.15
Current children cumulated vsize (KiB) 256712

[startup+222.301 s]
/proc/loadavg: 8.11 7.41 7.49 9/371 11750
/proc/meminfo: memFree=25387372/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=261436 CPUtime=886.44 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 27548 0 0 0 88618 26 0 0 25 0 4 0 635468958 267710464 26889 33554432000 4194304 4964800 140733635359248 18446744073709551615 4644872 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 65359 26889 460 189 0 59708 0
[pid=11740/tid=11742] ppid=11738 vsize=261436 CPUtime=222.24 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 7467 0 0 0 22219 5 0 0 25 0 4 0 635468963 267710464 26889 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=261436 CPUtime=219.71 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 4425 0 0 0 21967 4 0 0 25 0 4 0 635468963 267710464 26889 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670335 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=261436 CPUtime=222.17 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 5391 0 0 0 22212 5 0 0 25 0 4 0 635468963 267710464 26889 33554432000 4194304 4964800 140733635359248 18446744073709551615 4669276 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 886.44
Current children cumulated vsize (KiB) 261436

[startup+282.301 s]
/proc/loadavg: 8.04 7.52 7.52 9/371 11752
/proc/meminfo: memFree=25379132/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=262380 CPUtime=1125.76 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 28504 0 0 0 112549 27 0 0 25 0 4 0 635468958 268677120 27752 33554432000 4194304 4964800 140733635359248 18446744073709551615 4660155 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 65595 27752 460 189 0 59944 0
[pid=11740/tid=11742] ppid=11738 vsize=262380 CPUtime=282.25 cores=1,3,5,7

################
# More data... #
################

[pid=11740] ppid=11738 vsize=269916 CPUtime=2561.49 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 34689 0 0 0 256111 38 0 0 25 0 4 0 635468958 276393984 33704 33554432000 4194304 4964800 140733635359248 18446744073709551615 4320947 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 67479 33704 460 189 0 61828 0
[pid=11740/tid=11742] ppid=11738 vsize=269916 CPUtime=642.24 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 9027 0 0 0 64217 7 0 0 25 0 4 0 635468963 276393984 33704 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=269916 CPUtime=634.75 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 6751 0 0 0 63467 8 0 0 25 0 4 0 635468963 276393984 33704 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670351 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=269916 CPUtime=642.17 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 6107 0 0 0 64210 7 0 0 25 0 4 0 635468963 276393984 33704 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2561.49
Current children cumulated vsize (KiB) 269916

[startup+702.3 s]
/proc/loadavg: 8.18 7.96 7.72 9/371 11763
/proc/meminfo: memFree=25323376/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=271088 CPUtime=2800.79 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 35761 0 0 0 280039 40 0 0 25 0 4 0 635468958 277594112 34728 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670273 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 67772 34728 460 189 0 62121 0
[pid=11740/tid=11742] ppid=11738 vsize=271088 CPUtime=702.25 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 9503 0 0 0 70217 8 0 0 25 0 4 0 635468963 277594112 34728 33554432000 4194304 4964800 140733635359248 18446744073709551615 4660390 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=271088 CPUtime=694.05 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 6767 0 0 0 69397 8 0 0 25 0 4 0 635468963 277594112 34728 33554432000 4194304 4964800 140733635359248 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=271088 CPUtime=702.18 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 6242 0 0 0 70210 8 0 0 25 0 4 0 635468963 277594112 34728 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2800.79
Current children cumulated vsize (KiB) 271088

[startup+762.301 s]
/proc/loadavg: 8.10 7.98 7.74 9/371 11765
/proc/meminfo: memFree=25315148/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=273572 CPUtime=3040.08 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 36964 0 0 0 303965 43 0 0 25 0 4 0 635468958 280137728 35909 33554432000 4194304 4964800 140733635359248 18446744073709551615 4660449 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 68393 35909 460 189 0 62742 0
[pid=11740/tid=11742] ppid=11738 vsize=273572 CPUtime=762.25 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 10068 0 0 0 76216 9 0 0 25 0 4 0 635468963 280137728 35909 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=273572 CPUtime=753.34 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 7245 0 0 0 75325 9 0 0 25 0 4 0 635468963 280137728 35909 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670321 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=273572 CPUtime=762.18 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 6361 0 0 0 76210 8 0 0 25 0 4 0 635468963 280137728 35909 33554432000 4194304 4964800 140733635359248 18446744073709551615 4660507 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3040.08
Current children cumulated vsize (KiB) 273572

[startup+822.301 s]
/proc/loadavg: 8.08 7.99 7.75 9/371 11766
/proc/meminfo: memFree=25311868/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=273572 CPUtime=3279.39 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 37614 0 0 0 327895 44 0 0 25 0 4 0 635468958 280137728 36548 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670346 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 68393 36548 460 189 0 62742 0
[pid=11740/tid=11742] ppid=11738 vsize=273572 CPUtime=822.25 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 10143 0 0 0 82216 9 0 0 25 0 4 0 635468963 280137728 36548 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=273572 CPUtime=812.64 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 7753 0 0 0 81254 10 0 0 25 0 4 0 635468963 280137728 36548 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670514 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=273572 CPUtime=822.18 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 6424 0 0 0 82210 8 0 0 25 0 4 0 635468963 280137728 36548 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3279.39
Current children cumulated vsize (KiB) 273572

[startup+882.3 s]
/proc/loadavg: 8.13 8.02 7.78 9/371 11768
/proc/meminfo: memFree=25300548/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=273572 CPUtime=3518.68 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 39146 0 0 0 351820 48 0 0 25 0 4 0 635468958 280137728 37941 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670282 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 68393 37941 460 189 0 62742 0
[pid=11740/tid=11742] ppid=11738 vsize=273572 CPUtime=882.25 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 10786 0 0 0 88215 10 0 0 25 0 4 0 635468963 280137728 37941 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670264 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=273572 CPUtime=871.93 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 7868 0 0 0 87183 10 0 0 25 0 4 0 635468963 280137728 37941 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=273572 CPUtime=882.18 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 6985 0 0 0 88208 10 0 0 25 0 4 0 635468963 280137728 37941 33554432000 4194304 4964800 140733635359248 18446744073709551615 4668797 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3518.68
Current children cumulated vsize (KiB) 273572

[startup+942.301 s]
/proc/loadavg: 8.08 8.03 7.79 9/371 11769
/proc/meminfo: memFree=25298908/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=273572 CPUtime=3758.01 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 39168 0 0 0 375753 48 0 0 25 0 4 0 635468958 280137728 37963 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670534 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 68393 37963 460 189 0 62742 0
[pid=11740/tid=11742] ppid=11738 vsize=273572 CPUtime=942.26 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 10792 0 0 0 94216 10 0 0 25 0 4 0 635468963 280137728 37963 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=273572 CPUtime=931.23 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 7876 0 0 0 93113 10 0 0 25 0 4 0 635468963 280137728 37963 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670340 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=273572 CPUtime=942.2 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 6985 0 0 0 94210 10 0 0 25 0 4 0 635468963 280137728 37963 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670097 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3758.01
Current children cumulated vsize (KiB) 273572

[startup+1002.3 s]
/proc/loadavg: 8.07 8.04 7.81 9/371 11771
/proc/meminfo: memFree=25294028/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=273572 CPUtime=3997.27 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 39641 0 0 0 399678 49 0 0 25 0 4 0 635468958 280137728 38436 33554432000 4194304 4964800 140733635359248 18446744073709551615 4325321 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 68393 38436 460 189 0 62742 0
[pid=11740/tid=11742] ppid=11738 vsize=273572 CPUtime=1002.24 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 10931 0 0 0 100214 10 0 0 25 0 4 0 635468963 280137728 38436 33554432000 4194304 4964800 140733635359248 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=273572 CPUtime=990.51 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 8183 0 0 0 99040 11 0 0 25 0 4 0 635468963 280137728 38436 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=273572 CPUtime=1002.18 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 7011 0 0 0 100208 10 0 0 25 0 4 0 635468963 280137728 38436 33554432000 4194304 4964800 140733635359248 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3997.27
Current children cumulated vsize (KiB) 273572

[startup+1062.31 s]
/proc/loadavg: 8.14 8.06 7.83 9/371 11773
/proc/meminfo: memFree=25289880/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=274820 CPUtime=4236.61 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 39987 0 0 0 423612 49 0 0 25 0 4 0 635468958 281415680 38770 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 68705 38770 460 189 0 63054 0
[pid=11740/tid=11742] ppid=11738 vsize=274820 CPUtime=1062.26 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 11037 0 0 0 106216 10 0 0 25 0 4 0 635468963 281415680 38770 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=274820 CPUtime=1049.81 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 8419 0 0 0 104970 11 0 0 25 0 4 0 635468963 281415680 38770 33554432000 4194304 4964800 140733635359248 18446744073709551615 4669343 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=274820 CPUtime=1062.19 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 7014 0 0 0 106209 10 0 0 25 0 4 0 635468963 281415680 38770 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4236.61
Current children cumulated vsize (KiB) 274820

[startup+1122.3 s]
/proc/loadavg: 8.16 8.08 7.85 9/371 11774
/proc/meminfo: memFree=25285492/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=274820 CPUtime=4475.86 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 40255 0 0 0 447536 50 0 0 25 0 4 0 635468958 281415680 39026 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670323 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 68705 39026 460 189 0 63054 0
[pid=11740/tid=11742] ppid=11738 vsize=274820 CPUtime=1122.24 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 11280 0 0 0 112214 10 0 0 25 0 4 0 635468963 281415680 39026 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=274820 CPUtime=1109.1 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 8419 0 0 0 110899 11 0 0 25 0 4 0 635468963 281415680 39026 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=274820 CPUtime=1122.18 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 7016 0 0 0 112208 10 0 0 25 0 4 0 635468963 281415680 39026 33554432000 4194304 4964800 140733635359248 18446744073709551615 4627415 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4475.86
Current children cumulated vsize (KiB) 274820

[startup+1182.3 s]
/proc/loadavg: 8.12 8.07 7.86 9/371 11807
/proc/meminfo: memFree=25407156/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=275348 CPUtime=4715.14 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 41395 0 0 0 471462 52 0 0 25 0 4 0 635468958 281956352 40120 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670299 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 68837 40120 460 189 0 63186 0
[pid=11740/tid=11742] ppid=11738 vsize=275348 CPUtime=1182.25 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 12117 0 0 0 118213 12 0 0 25 0 4 0 635468963 281956352 40120 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=275348 CPUtime=1168.38 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 8421 0 0 0 116827 11 0 0 25 0 4 0 635468963 281956352 40120 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=275348 CPUtime=1182.18 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 7022 0 0 0 118208 10 0 0 25 0 4 0 635468963 281956352 40120 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4715.14
Current children cumulated vsize (KiB) 275348



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.6 s]
/proc/loadavg: 8.14 8.08 7.86 9/371 11807
/proc/meminfo: memFree=25399472/32951124 swapFree=63083164/67111528
[pid=11740] ppid=11738 vsize=275872 CPUtime=4800.12 cores=1,3,5,7
/proc/11740/stat : 11740 (claspmt) R 11738 11740 10794 0 -1 4202496 41583 0 0 0 479960 52 0 0 25 0 4 0 635468958 282492928 40294 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670323 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11740/statm: 68968 40294 460 189 0 63317 0
[pid=11740/tid=11742] ppid=11738 vsize=275872 CPUtime=1203.56 cores=1,3,5,7
/proc/11740/task/11742/stat : 11742 (claspmt) R 11738 11740 10794 0 -1 4202560 12160 0 0 0 120344 12 0 0 25 0 4 0 635468963 282492928 40294 33554432000 4194304 4964800 140733635359248 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11740/tid=11743] ppid=11738 vsize=275872 CPUtime=1189.43 cores=1,3,5,7
/proc/11740/task/11743/stat : 11743 (claspmt) R 11738 11740 10794 0 -1 4202560 8533 0 0 0 118932 11 0 0 25 0 4 0 635468963 282492928 40294 33554432000 4194304 4964800 140733635359248 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11740/tid=11744] ppid=11738 vsize=275872 CPUtime=1203.49 cores=1,3,5,7
/proc/11740/task/11744/stat : 11744 (claspmt) R 11738 11740 10794 0 -1 4202560 7027 0 0 0 120339 10 0 0 25 0 4 0 635468963 282492928 40294 33554432000 4194304 4964800 140733635359248 18446744073709551615 4669413 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.12
Current children cumulated vsize (KiB) 275872

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0
Real time (s): 1203.64
CPU time (s): 4800.2
CPU user time (s): 4799.66
CPU system time (s): 0.537918
CPU usage (%): 398.805
Max. virtual memory (cumulated for all children) (KiB): 275872

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4799.66
system time used= 0.537918
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 41595
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 13125
involuntary context switches= 22841

runsolver used 3.87241 second user time and 10.3614 second system time

The end

Launcher Data 

Begin job on node141 at 2011-05-15 05:35:33
IDJOB=3373145
IDBENCH=84762
IDSOLVER=1853
FILE ID=node141/3373145-1305430533
RUNJOBID= node141-1305430498-11687
PBS_JOBID= 13324603
Free space on /tmp= 73360 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT04/handmade/marijn/equilarge-l/equilarge_l4.used-as.sat04-896.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3373145-1305430533/watcher-3373145-1305430533 -o /tmp/evaluation-result-3373145-1305430533/solver-3373145-1305430533 -C 4800 -W 1300 -M 15500  HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373145-1305430533.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= fc993d9544ccd8eaeaa4882599387947
RANDOM SEED=1685379577

node141.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5362.96
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25528708 kB
Buffers:        247628 kB
Cached:        4908956 kB
SwapCached:      15708 kB
Active:        2103372 kB
Inactive:      5080096 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25528708 kB
SwapTotal:    67111528 kB
SwapFree:     63083164 kB
Dirty:            1480 kB
Writeback:           0 kB
AnonPages:     2017816 kB
Mapped:          13644 kB
Slab:           136700 kB
PageTables:      38264 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 86322380 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73360 MiB
End job on node141 at 2011-05-15 05:55:39