Trace number 3381005

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40001.8 5045.28

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-21x33.cnf
MD5SUMf7c29bbc3bef00d1e7cc799553ba4813
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2772
Number of clauses448371
Sum of the clauses size1785168
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 24158
Number of clauses of size 30
Number of clauses of size 4444213
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...500174/instance-3381005-1305500174.cnf
37.89/37.95	c Solving...
40000.22/5043.82	
40000.22/5043.82	*** INTERRUPTED! ***
40000.22/5043.82	*** Shutting down threads...
40001.67/5045.27	s UNKNOWN
40001.67/5045.27	
40001.67/5045.27	c INTERRUPTED   : 1
40001.67/5045.27	c Models        : 0+    
40001.67/5045.27	c Time          : 5043.820s (Solving: 5005.86s 1st Model: 0.00s Unsat: 0.00s)
40001.67/5045.27	c CPU Time      : 40000.290s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3381005-1305500174/watcher-3381005-1305500174 -o /tmp/evaluation-result-3381005-1305500174/solver-3381005-1305500174 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381005-1305500174.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.22 8.26 8.35 2/426 11622
/proc/meminfo: memFree=10665504/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=22940 CPUtime=0.01 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 1118 0 0 0 1 0 0 0 25 0 1 0 661616138 23490560 937 33554432000 4194304 4964800 140735322110752 18446744073709551615 242970211013 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/11622/statm: 5735 938 386 189 0 609 0

[startup+0.039218 s]
/proc/loadavg: 7.22 8.26 8.35 2/426 11622
/proc/meminfo: memFree=10665504/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=27528 CPUtime=0.03 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 2598 0 0 0 3 0 0 0 25 0 1 0 661616138 28188672 2034 33554432000 4194304 4964800 140735322110752 18446744073709551615 4567909 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/11622/statm: 6882 2036 386 189 0 1756 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 27528

[startup+0.100203 s]
/proc/loadavg: 7.22 8.26 8.35 2/426 11622
/proc/meminfo: memFree=10665504/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=37060 CPUtime=0.09 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 5685 0 0 0 8 1 0 0 25 0 1 0 661616138 37949440 4444 33554432000 4194304 4964800 140735322110752 18446744073709551615 4567832 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/11622/statm: 9265 4445 386 189 0 4139 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 37060

[startup+0.300167 s]
/proc/loadavg: 7.22 8.26 8.35 2/426 11622
/proc/meminfo: memFree=10665504/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=58060 CPUtime=0.29 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 11578 0 0 0 27 2 0 0 25 0 1 0 661616138 59453440 9703 33554432000 4194304 4964800 140735322110752 18446744073709551615 4607513 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/11622/statm: 14515 9703 399 189 0 9389 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 58060

[startup+0.701112 s]
/proc/loadavg: 7.22 8.26 8.35 2/426 11622
/proc/meminfo: memFree=10665504/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=58060 CPUtime=0.69 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 11578 0 0 0 67 2 0 0 25 0 1 0 661616138 59453440 9703 33554432000 4194304 4964800 140735322110752 18446744073709551615 4607644 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/11622/statm: 14515 9703 399 189 0 9389 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 58060

[startup+1.50096 s]
/proc/loadavg: 7.22 8.26 8.35 2/427 11623
/proc/meminfo: memFree=10628800/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=58060 CPUtime=1.49 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 11578 0 0 0 147 2 0 0 25 0 1 0 661616138 59453440 9703 33554432000 4194304 4964800 140735322110752 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/11622/statm: 14515 9703 399 189 0 9389 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 58060

[startup+3.10069 s]
/proc/loadavg: 7.22 8.26 8.35 2/427 11623
/proc/meminfo: memFree=10628056/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=58060 CPUtime=3.09 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 11578 0 0 0 307 2 0 0 25 0 1 0 661616138 59453440 9703 33554432000 4194304 4964800 140735322110752 18446744073709551615 4607604 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/11622/statm: 14515 9703 399 189 0 9389 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 58060

[startup+6.30014 s]
/proc/loadavg: 6.72 8.14 8.31 2/427 11623
/proc/meminfo: memFree=10625080/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=64148 CPUtime=6.29 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 13095 0 0 0 626 3 0 0 25 0 1 0 661616138 65687552 11220 33554432000 4194304 4964800 140735322110752 18446744073709551615 4600743 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/11622/statm: 16037 11220 400 189 0 10911 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 64148

[startup+12.7011 s]
/proc/loadavg: 6.26 8.02 8.27 2/427 11624
/proc/meminfo: memFree=10613140/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=76200 CPUtime=12.69 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 16098 0 0 0 1266 3 0 0 25 0 1 0 661616138 78028800 14223 33554432000 4194304 4964800 140735322110752 18446744073709551615 4600540 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/11622/statm: 19050 14223 400 189 0 13924 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 76200

[startup+25.5008 s]
/proc/loadavg: 5.10 7.68 8.15 2/427 11624
/proc/meminfo: memFree=10597840/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=92748 CPUtime=25.49 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 20255 0 0 0 2545 4 0 0 25 0 1 0 661616138 94973952 18380 33554432000 4194304 4964800 140735322110752 18446744073709551615 4600360 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/11622/statm: 23187 18380 400 189 0 18061 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 92748

[startup+51.1004 s]
/proc/loadavg: 5.32 7.50 8.08 9/434 11632
/proc/meminfo: memFree=10101720/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=954644 CPUtime=140.76 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 149182 0 0 0 14007 69 0 0 25 0 8 0 661616138 977555456 145459 33554432000 4194304 4964800 140735322110752 18446744073709551615 4654540 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11622/statm: 238661 145460 466 189 0 233010 0
[pid=11622/tid=11625] ppid=11620 vsize=954644 CPUtime=12.87 cores=0-7
/proc/11622/task/11625/stat : 11625 (claspmt) R 11620 11622 11562 0 -1 4202560 16034 0 0 0 1278 9 0 0 25 0 8 0 661619934 977555456 145460 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670363 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11622/tid=11626] ppid=11620 vsize=954644 CPUtime=12.86 cores=0-7
/proc/11622/task/11626/stat : 11626 (claspmt) R 11620 11622 11562 0 -1 4202560 16720 0 0 0 1279 7 0 0 25 0 8 0 661619934 977555456 145460 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11622/tid=11627] ppid=11620 vsize=954644 CPUtime=12.85 cores=0-7
/proc/11622/task/11627/stat : 11627 (claspmt) R 11620 11622 11562 0 -1 4202560 12861 0 0 0 1280 5 0 0 25 0 8 0 661619934 977555456 145460 33554432000 4194304 4964800 140735322110752 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11622/tid=11628] ppid=11620 vsize=954644 CPUtime=12.87 cores=0-7
/proc/11622/task/11628/stat : 11628 (claspmt) R 11620 11622 11562 0 -1 4202560 15681 0 0 0 1280 7 0 0 25 0 8 0 661619934 977555456 145461 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11622/tid=11629] ppid=11620 vsize=954644 CPUtime=12.8 cores=0-7
/proc/11622/task/11629/stat : 11629 (claspmt) R 11620 11622 11562 0 -1 4202560 14142 0 0 0 1273 7 0 0 25 0 8 0 661619934 977555456 145461 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11622/tid=11630] ppid=11620 vsize=954644 CPUtime=12.85 cores=0-7
/proc/11622/task/11630/stat : 11630 (claspmt) R 11620 11622 11562 0 -1 4202560 15086 0 0 0 1278 7 0 0 25 0 8 0 661619934 977555456 145461 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11622/tid=11631] ppid=11620 vsize=954644 CPUtime=12.85 cores=0-7
/proc/11622/task/11631/stat : 11631 (claspmt) R 11620 11622 11562 0 -1 4202560 19209 0 0 0 1275 10 0 0 24 0 8 0 661619934 977555456 145461 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 140.76
Current children cumulated vsize (KiB) 954644

[startup+102.317 s]
/proc/loadavg: 6.90 7.59 8.08 9/434 11633
/proc/meminfo: memFree=9873172/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=1054308 CPUtime=549.54 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 203431 0 0 0 54842 112 0 0 25 0 8 0 661616138 1079611392 199563 33554432000 4194304 4964800 140735322110752 18446744073709551615 4320981 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11622/statm: 263577 199563 466 189 0 257926 0
[pid=11622/tid=11625] ppid=11620 vsize=1054308 CPUtime=64.02 cores=0-7
/proc/11622/task/11625/stat : 11625 (claspmt) R 11620 11622 11562 0 -1 4202560 26267 0 0 0 6386 16 0 0 25 0 8 0 661619934 1079611392 199563 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11622/tid=11626] ppid=11620 vsize=1054308 CPUtime=64.05 cores=0-7
/proc/11622/task/11626/stat : 11626 (claspmt) R 11620 11622 11562 0 -1 4202560 23171 0 0 0 6392 13 0 0 25 0 8 0 661619934 1079611392 199563 33554432000 4194304 4964800 140735322110752 18446744073709551615 4684107 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11622/tid=11627] ppid=11620 vsize=1054308 CPUtime=64.05 cores=0-7
/proc/11622/task/11627/stat : 11627 (claspmt) R 11620 11622 11562 0 -1 4202560 16726 0 0 0 6396 9 0 0 25 0 8 0 661619934 1079611392 199563 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11622/tid=11628] ppid=11620 vsize=1054308 CPUtime=64.06 cores=0-7
/proc/11622/task/11628/stat : 11628 (claspmt) R 11620 11622 11562 0 -1 4202560 20285 0 0 0 6396 10 0 0 25 0 8 0 661619934 1079611392 199563 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11622/tid=11629] ppid=11620 vsize=1054308 CPUtime=63.25 cores=0-7
/proc/11622/task/11629/stat : 11629 (claspmt) R 11620 11622 11562 0 -1 4202560 19539 0 0 0 6314 11 0 0 25 0 8 0 661619934 1079611392 199563 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11622/tid=11630] ppid=11620 vsize=1054308 CPUtime=64.05 cores=0-7
/proc/11622/task/11630/stat : 11630 (claspmt) R 11620 11622 11562 0 -1 4202560 17797 0 0 0 6394 11 0 0 25 0 8 0 661619934 1079611392 199563 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11622/tid=11631] ppid=11620 vsize=1054308 CPUtime=64.05 cores=0-7
/proc/11622/task/11631/stat : 11631 (claspmt) R 11620 11622 11562 0 -1 4202560 30275 0 0 0 6386 19 0 0 25 0 8 0 661619934 1079611392 199563 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 549.54
Current children cumulated vsize (KiB) 1054308

[startup+162.301 s]
/proc/loadavg: 7.65 7.68 8.08 9/434 11635
/proc/meminfo: memFree=9686532/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=1200652 CPUtime=1028.42 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 249888 0 0 0 102686 156 0 0 25 0 8 0 661616138 1229467648 245848 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11622/statm: 300163 245848 466 189 0 294512 0
[pid=11622/tid=11625] ppid=11620 vsize=1200652 CPUtime=123.98 cores=0-7
/proc/11622/task/11625/stat : 11625 (claspmt) R 11620 11622 11562 0 -1 4202560 32847 0 0 0 12377 21 0 0 25 0 8 0 661619934 1229467648 245848 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11622/tid=11626] ppid=11620 vsize=1200652 CPUtime=124.02 cores=0-7
/proc/11622/task/11626/stat : 11626 (claspmt) R 11620 11622 11562 0 -1 4202560 29857 0 0 0 12382 20 0 0 25 0 8 0 661619934 1229467648 245848 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11622/tid=11627] ppid=11620 vsize=1200652 CPUtime=124.03 cores=0-7
/proc/11622/task/11627/stat : 11627 (claspmt) R 11620 11622 11562 0 -1 4202560 19450 0 0 0 12392 11 0 0 25 0 8 0 661619934 1229467648 245848 33554432000 4194304 4964800 140735322110752 18446744073709551615 4332324 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11622/tid=11628] ppid=11620 vsize=1200652 CPUtime=124.02 cores=0-7
/proc/11622/task/11628/stat : 11628 (claspmt) R 11620 11622 11562 0 -1 4202560 23662 0 0 0 12389 13 0 0 25 0 8 0 661619934 1229467648 245848 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11622/tid=11629] ppid=11620 vsize=1200652 CPUtime=122.38 cores=0-7
/proc/11622/task/11629/stat : 11629 (claspmt) R 11620 11622 11562 0 -1 4202560 23351 0 0 0 12223 15 0 0 25 0 8 0 661619934 1229467648 245848 33554432000 4194304 4964800 140735322110752 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11622/tid=11630] ppid=11620 vsize=1200652 CPUtime=124.01 cores=0-7
/proc/11622/task/11630/stat : 11630 (claspmt) R 11620 11622 11562 0 -1 4202560 20847 0 0 0 12387 14 0 0 25 0 8 0 661619934 1229467648 245848 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11622/tid=11631] ppid=11620 vsize=1200652 CPUtime=124 cores=0-7
/proc/11622/task/11631/stat : 11631 (claspmt) R 11620 11622 11562 0 -1 4202560 42345 0 0 0 12372 28 0 0 25 0 8 0 661619934 1229467648 245848 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1028.42
Current children cumulated vsize (KiB) 1200652

[startup+222.301 s]
/proc/loadavg: 7.92 7.75 8.08 9/434 11636
/proc/meminfo: memFree=9505600/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=1418828 CPUtime=1507.46 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 295399 0 0 0 150543 203 0 0 25 0 8 0 661616138 1452879872 291263 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11622/statm: 354707 291263 466 189 0 349056 0
[pid=11622/tid=11625] ppid=11620 vsize=1418828 CPUtime=183.96 cores=0-7
/proc/11622/task/11625/stat : 11625 (claspmt) R 11620 11622 11562 0 -1 4202560 38977 0 0 0 18369 27 0 0 25 0 8 0 661619934 1452879872 291263 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11622/tid=11626] ppid=11620 vsize=1418828 CPUtime=183.99 cores=0-7
/proc/11622/task/11626/stat : 11626 (claspmt) R 11620 11622 11562 0 -1 4202560 35614 0 0 0 18371 28 0 0 25 0 8 0 661619934 1452879872 291263 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670330 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11622/tid=11627] ppid=11620 vsize=1418828 CPUtime=184.03 cores=0-7
/proc/11622/task/11627/stat : 11627 (claspmt) R 11620 11622 11562 0 -1 4202560 24285 0 0 0 18388 15 0 0 25 0 8 0 661619934 1452879872 291263 33554432000 4194304 4964800 140735322110752 18446744073709551615 4642167 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11622/tid=11628] ppid=11620 vsize=1418828 CPUtime=184 cores=0-7
/proc/11622/task/11628/stat : 11628 (claspmt) R 11620 11622 11562 0 -1 4202560 26054 0 0 0 18386 14 0 0 25 0 8 0 661619934 1452879872 291263 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11622/tid=11629] ppid=11620 vsize=1418828 CPUtime=181.51 cores=0-7
/proc/11622/task/11629/stat : 11629 (claspmt) R 11620 11622 11562 0 -1 4202560 26543 0 0 0 18132 19 0 0 25 0 8 0 661619934 1452879872 291263 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670313 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11622/tid=11630] ppid=11620 vsize=1418828 CPUtime=184.01 cores=0-7
/proc/11622/task/11630/stat : 11630 (claspmt) R 11620 11622 11562 0 -1 4202560 23585 0 0 0 18384 17 0 0 25 0 8 0 661619934 1452879872 291263 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11622/tid=11631] ppid=11620 vsize=1418828 CPUtime=183.97 cores=0-7
/proc/11622/task/11631/stat : 11631 (claspmt) R 11620 11622 11562 0 -1 4202560 52997 0 0 0 18359 38 0 0 25 0 8 0 661619934 1452879872 291263 33554432000 4194304 4964800 140735322110752 18446744073709551615 4342199 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1507.46
Current children cumulated vsize (KiB) 1418828

################
# More data... #
################

[pid=11622] ppid=11620 vsize=3760024 CPUtime=37912.3 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 877647 0 0 0 3790073 1153 0 0 25 0 8 0 661616138 3850264576 862193 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670496 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/11622/statm: 940006 862193 466 189 0 934355 0
[pid=11622/tid=11625] ppid=11620 vsize=3760024 CPUtime=4743.12 cores=0-7
/proc/11622/task/11625/stat : 11625 (claspmt) R 11620 11622 11562 0 -1 4202560 83024 0 0 0 474221 91 0 0 25 0 8 0 661619934 3850264576 862193 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11622/tid=11626] ppid=11620 vsize=3760024 CPUtime=4738.45 cores=0-7
/proc/11622/task/11626/stat : 11626 (claspmt) R 11620 11622 11562 0 -1 4202560 132109 0 0 0 473679 166 0 0 25 0 8 0 661619934 3850264576 862193 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11622/tid=11627] ppid=11620 vsize=3760024 CPUtime=4709.82 cores=0-7
/proc/11622/task/11627/stat : 11627 (claspmt) R 11620 11622 11562 0 -1 4202560 62596 0 0 0 470903 79 0 0 25 0 8 0 661619934 3850264576 862193 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11622/tid=11628] ppid=11620 vsize=3760024 CPUtime=4743.62 cores=0-7
/proc/11622/task/11628/stat : 11628 (claspmt) R 11620 11622 11562 0 -1 4202560 53805 0 0 0 474309 53 0 0 25 0 8 0 661619934 3850264576 862193 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11622/tid=11629] ppid=11620 vsize=3760024 CPUtime=4739.04 cores=0-7
/proc/11622/task/11629/stat : 11629 (claspmt) R 11620 11622 11562 0 -1 4202560 61535 0 0 0 473823 81 0 0 25 0 8 0 661619934 3850264576 862193 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11622/tid=11630] ppid=11620 vsize=3760024 CPUtime=4730.67 cores=0-7
/proc/11622/task/11630/stat : 11630 (claspmt) R 11620 11622 11562 0 -1 4202560 72656 0 0 0 472945 122 0 0 25 0 8 0 661619934 3850264576 862193 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11622/tid=11631] ppid=11620 vsize=3760024 CPUtime=4731.6 cores=0-7
/proc/11622/task/11631/stat : 11631 (claspmt) R 11620 11622 11562 0 -1 4202560 180287 0 0 0 472880 280 0 0 25 0 8 0 661619934 3850264576 862193 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 37912.3
Current children cumulated vsize (KiB) 3760024

[startup+4842.3 s]
/proc/loadavg: 8.09 8.06 8.01 9/434 11783
/proc/meminfo: memFree=7190892/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=3761504 CPUtime=38391.3 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 884091 0 0 0 3837964 1168 0 0 25 0 8 0 661616138 3851780096 868637 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/11622/statm: 940376 868637 466 189 0 934725 0
[pid=11622/tid=11625] ppid=11620 vsize=3761504 CPUtime=4803.12 cores=0-7
/proc/11622/task/11625/stat : 11625 (claspmt) R 11620 11622 11562 0 -1 4202560 83057 0 0 0 480221 91 0 0 25 0 8 0 661619934 3851780096 868637 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11622/tid=11626] ppid=11620 vsize=3761504 CPUtime=4798.43 cores=0-7
/proc/11622/task/11626/stat : 11626 (claspmt) R 11620 11622 11562 0 -1 4202560 133088 0 0 0 479675 168 0 0 25 0 8 0 661619934 3851780096 868637 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11622/tid=11627] ppid=11620 vsize=3761504 CPUtime=4769.83 cores=0-7
/proc/11622/task/11627/stat : 11627 (claspmt) R 11620 11622 11562 0 -1 4202560 63052 0 0 0 476903 80 0 0 25 0 8 0 661619934 3851780096 868637 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11622/tid=11628] ppid=11620 vsize=3761504 CPUtime=4803.62 cores=0-7
/proc/11622/task/11628/stat : 11628 (claspmt) R 11620 11622 11562 0 -1 4202560 53812 0 0 0 480309 53 0 0 25 0 8 0 661619934 3851780096 868637 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11622/tid=11629] ppid=11620 vsize=3761504 CPUtime=4799.01 cores=0-7
/proc/11622/task/11629/stat : 11629 (claspmt) R 11620 11622 11562 0 -1 4202560 61612 0 0 0 479819 82 0 0 25 0 8 0 661619934 3851780096 868637 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11622/tid=11630] ppid=11620 vsize=3761504 CPUtime=4790.64 cores=0-7
/proc/11622/task/11630/stat : 11630 (claspmt) R 11620 11622 11562 0 -1 4202560 72748 0 0 0 478942 122 0 0 25 0 8 0 661619934 3851780096 868637 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11622/tid=11631] ppid=11620 vsize=3761504 CPUtime=4791.61 cores=0-7
/proc/11622/task/11631/stat : 11631 (claspmt) R 11620 11622 11562 0 -1 4202560 182129 0 0 0 478876 285 0 0 25 0 8 0 661619934 3851780096 868637 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38391.3
Current children cumulated vsize (KiB) 3761504

[startup+4902.3 s]
/proc/loadavg: 8.18 8.08 8.01 9/434 11785
/proc/meminfo: memFree=7190016/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=3761504 CPUtime=38870.4 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 884289 0 0 0 3885872 1169 0 0 25 0 8 0 661616138 3851780096 868835 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/11622/statm: 940376 868835 466 189 0 934725 0
[pid=11622/tid=11625] ppid=11620 vsize=3761504 CPUtime=4863.12 cores=0-7
/proc/11622/task/11625/stat : 11625 (claspmt) R 11620 11622 11562 0 -1 4202560 83155 0 0 0 486221 91 0 0 25 0 8 0 661619934 3851780096 868835 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11622/tid=11626] ppid=11620 vsize=3761504 CPUtime=4858.43 cores=0-7
/proc/11622/task/11626/stat : 11626 (claspmt) R 11620 11622 11562 0 -1 4202560 133093 0 0 0 485675 168 0 0 25 0 8 0 661619934 3851780096 868835 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11622/tid=11627] ppid=11620 vsize=3761504 CPUtime=4829.83 cores=0-7
/proc/11622/task/11627/stat : 11627 (claspmt) R 11620 11622 11562 0 -1 4202560 63053 0 0 0 482903 80 0 0 25 0 8 0 661619934 3851780096 868835 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11622/tid=11628] ppid=11620 vsize=3761504 CPUtime=4863.63 cores=0-7
/proc/11622/task/11628/stat : 11628 (claspmt) R 11620 11622 11562 0 -1 4202560 53816 0 0 0 486309 54 0 0 25 0 8 0 661619934 3851780096 868835 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11622/tid=11629] ppid=11620 vsize=3761504 CPUtime=4859 cores=0-7
/proc/11622/task/11629/stat : 11629 (claspmt) R 11620 11622 11562 0 -1 4202560 61677 0 0 0 485817 83 0 0 25 0 8 0 661619934 3851780096 868835 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11622/tid=11630] ppid=11620 vsize=3761504 CPUtime=4850.61 cores=0-7
/proc/11622/task/11630/stat : 11630 (claspmt) R 11620 11622 11562 0 -1 4202560 72762 0 0 0 484939 122 0 0 25 0 8 0 661619934 3851780096 868835 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11622/tid=11631] ppid=11620 vsize=3761504 CPUtime=4851.61 cores=0-7
/proc/11622/task/11631/stat : 11631 (claspmt) R 11620 11622 11562 0 -1 4202560 182140 0 0 0 484876 285 0 0 25 0 8 0 661619934 3851780096 868835 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38870.4
Current children cumulated vsize (KiB) 3761504

[startup+4962.3 s]
/proc/loadavg: 8.13 8.08 8.01 9/434 11787
/proc/meminfo: memFree=7188640/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=3761504 CPUtime=39349.4 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 885178 0 0 0 3933769 1173 0 0 25 0 8 0 661616138 3851780096 868996 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/11622/statm: 940376 868996 466 189 0 934725 0
[pid=11622/tid=11625] ppid=11620 vsize=3761504 CPUtime=4923.07 cores=0-7
/proc/11622/task/11625/stat : 11625 (claspmt) R 11620 11622 11562 0 -1 4202560 83975 0 0 0 492215 92 0 0 25 0 8 0 661619934 3851780096 868996 33554432000 4194304 4964800 140735322110752 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11622/tid=11626] ppid=11620 vsize=3761504 CPUtime=4918.43 cores=0-7
/proc/11622/task/11626/stat : 11626 (claspmt) R 11620 11622 11562 0 -1 4202560 133098 0 0 0 491675 168 0 0 25 0 8 0 661619934 3851780096 868996 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11622/tid=11627] ppid=11620 vsize=3761504 CPUtime=4889.83 cores=0-7
/proc/11622/task/11627/stat : 11627 (claspmt) R 11620 11622 11562 0 -1 4202560 63053 0 0 0 488903 80 0 0 25 0 8 0 661619934 3851780096 868996 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11622/tid=11628] ppid=11620 vsize=3761504 CPUtime=4923.63 cores=0-7
/proc/11622/task/11628/stat : 11628 (claspmt) R 11620 11622 11562 0 -1 4202560 53819 0 0 0 492309 54 0 0 25 0 8 0 661619934 3851780096 868996 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11622/tid=11629] ppid=11620 vsize=3761504 CPUtime=4918.98 cores=0-7
/proc/11622/task/11629/stat : 11629 (claspmt) R 11620 11622 11562 0 -1 4202560 61735 0 0 0 491813 85 0 0 25 0 8 0 661619934 3851780096 868996 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11622/tid=11630] ppid=11620 vsize=3761504 CPUtime=4910.59 cores=0-7
/proc/11622/task/11630/stat : 11630 (claspmt) R 11620 11622 11562 0 -1 4202560 72763 0 0 0 490937 122 0 0 25 0 8 0 661619934 3851780096 868996 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670349 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11622/tid=11631] ppid=11620 vsize=3761504 CPUtime=4911.59 cores=0-7
/proc/11622/task/11631/stat : 11631 (claspmt) R 11620 11622 11562 0 -1 4202560 182142 0 0 0 490874 285 0 0 25 0 8 0 661619934 3851780096 868996 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39349.4
Current children cumulated vsize (KiB) 3761504

[startup+5022.3 s]
/proc/loadavg: 8.05 8.06 8.01 9/434 11788
/proc/meminfo: memFree=7188384/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=3761504 CPUtime=39828.5 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 885250 0 0 0 3981673 1175 0 0 25 0 8 0 661616138 3851780096 869067 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670295 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/11622/statm: 940376 869067 466 189 0 934725 0
[pid=11622/tid=11625] ppid=11620 vsize=3761504 CPUtime=4983.07 cores=0-7
/proc/11622/task/11625/stat : 11625 (claspmt) R 11620 11622 11562 0 -1 4202560 83975 0 0 0 498215 92 0 0 25 0 8 0 661619934 3851780096 869067 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11622/tid=11626] ppid=11620 vsize=3761504 CPUtime=4978.43 cores=0-7
/proc/11622/task/11626/stat : 11626 (claspmt) R 11620 11622 11562 0 -1 4202560 133098 0 0 0 497675 168 0 0 25 0 8 0 661619934 3851780096 869067 33554432000 4194304 4964800 140735322110752 18446744073709551615 4684107 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11622/tid=11627] ppid=11620 vsize=3761504 CPUtime=4949.83 cores=0-7
/proc/11622/task/11627/stat : 11627 (claspmt) R 11620 11622 11562 0 -1 4202560 63054 0 0 0 494903 80 0 0 25 0 8 0 661619934 3851780096 869067 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11622/tid=11628] ppid=11620 vsize=3761504 CPUtime=4983.63 cores=0-7
/proc/11622/task/11628/stat : 11628 (claspmt) R 11620 11622 11562 0 -1 4202560 53819 0 0 0 498309 54 0 0 25 0 8 0 661619934 3851780096 869067 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11622/tid=11629] ppid=11620 vsize=3761504 CPUtime=4978.96 cores=0-7
/proc/11622/task/11629/stat : 11629 (claspmt) R 11620 11622 11562 0 -1 4202560 61795 0 0 0 497810 86 0 0 25 0 8 0 661619934 3851780096 869067 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11622/tid=11630] ppid=11620 vsize=3761504 CPUtime=4970.55 cores=0-7
/proc/11622/task/11630/stat : 11630 (claspmt) R 11620 11622 11562 0 -1 4202560 72763 0 0 0 496932 123 0 0 25 0 8 0 661619934 3851780096 869067 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11622/tid=11631] ppid=11620 vsize=3761504 CPUtime=4971.59 cores=0-7
/proc/11622/task/11631/stat : 11631 (claspmt) R 11620 11622 11562 0 -1 4202560 182153 0 0 0 496874 285 0 0 25 0 8 0 661619934 3851780096 869067 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39828.5
Current children cumulated vsize (KiB) 3761504



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5043.8 s]
/proc/loadavg: 8.03 8.05 8.01 9/434 11789
/proc/meminfo: memFree=7188132/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=3761504 CPUtime=40000.2 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 885272 0 0 0 3998847 1175 0 0 25 0 8 0 661616138 3851780096 869089 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670588 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/11622/statm: 940376 869089 466 189 0 934725 0
[pid=11622/tid=11625] ppid=11620 vsize=3761504 CPUtime=5004.58 cores=0-7
/proc/11622/task/11625/stat : 11625 (claspmt) R 11620 11622 11562 0 -1 4202560 83975 0 0 0 500366 92 0 0 25 0 8 0 661619934 3851780096 869089 33554432000 4194304 4964800 140735322110752 18446744073709551615 4332641 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11622/tid=11626] ppid=11620 vsize=3761504 CPUtime=4999.95 cores=0-7
/proc/11622/task/11626/stat : 11626 (claspmt) R 11620 11622 11562 0 -1 4202560 133098 0 0 0 499827 168 0 0 25 0 8 0 661619934 3851780096 869089 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670313 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11622/tid=11627] ppid=11620 vsize=3761504 CPUtime=4971.34 cores=0-7
/proc/11622/task/11627/stat : 11627 (claspmt) R 11620 11622 11562 0 -1 4202560 63054 0 0 0 497054 80 0 0 25 0 8 0 661619934 3851780096 869089 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11622/tid=11628] ppid=11620 vsize=3761504 CPUtime=5005.15 cores=0-7
/proc/11622/task/11628/stat : 11628 (claspmt) R 11620 11622 11562 0 -1 4202560 53819 0 0 0 500461 54 0 0 25 0 8 0 661619934 3851780096 869089 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11622/tid=11629] ppid=11620 vsize=3761504 CPUtime=5000.46 cores=0-7
/proc/11622/task/11629/stat : 11629 (claspmt) R 11620 11622 11562 0 -1 4202560 61816 0 0 0 499960 86 0 0 25 0 8 0 661619934 3851780096 869089 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11622/tid=11630] ppid=11620 vsize=3761504 CPUtime=4992.05 cores=0-7
/proc/11622/task/11630/stat : 11630 (claspmt) R 11620 11622 11562 0 -1 4202560 72763 0 0 0 499082 123 0 0 25 0 8 0 661619934 3851780096 869089 33554432000 4194304 4964800 140735322110752 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11622/tid=11631] ppid=11620 vsize=3761504 CPUtime=4993.11 cores=0-7
/proc/11622/task/11631/stat : 11631 (claspmt) R 11620 11622 11562 0 -1 4202560 182154 0 0 0 499026 285 0 0 25 0 8 0 661619934 3851780096 869089 33554432000 4194304 4964800 140735322110752 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.2
Current children cumulated vsize (KiB) 3761504

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5044.62 s]
/proc/loadavg: 8.03 8.05 8.01 2/428 11790
/proc/meminfo: memFree=7207028/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=3733252 CPUtime=40001.1 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 885282 0 0 0 3998933 1175 0 0 25 0 1 0 661616138 3822850048 863165 33554432000 4194304 4964800 140735322110752 18446744073709551615 4628834 0 0 4096 18946 0 0 0 17 3 0 0 0
/proc/11622/statm: 933313 863165 476 189 0 927662 0
Current children cumulated CPU time (s) 40001.1
Current children cumulated vsize (KiB) 3733252

[startup+5045 s]
/proc/loadavg: 8.03 8.05 8.01 2/428 11790
/proc/meminfo: memFree=7207028/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=3647612 CPUtime=40001.5 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 885282 0 0 0 3998971 1176 0 0 25 0 1 0 661616138 3735154688 857806 33554432000 4194304 4964800 140735322110752 18446744073709551615 242970207210 0 0 4096 18946 0 0 0 17 3 0 0 0
/proc/11622/statm: 911903 857806 476 189 0 906252 0
Current children cumulated CPU time (s) 40001.5
Current children cumulated vsize (KiB) 3647612

[startup+5045.2 s]
/proc/loadavg: 8.03 8.05 8.01 2/428 11790
/proc/meminfo: memFree=7207028/32951132 swapFree=67093844/67111528
[pid=11622] ppid=11620 vsize=2058884 CPUtime=40001.7 cores=0-7
/proc/11622/stat : 11622 (claspmt) R 11620 11622 11562 0 -1 4202496 885282 0 0 0 3998984 1183 0 0 25 0 1 0 661616138 2108297216 427567 33554432000 4194304 4964800 140735322110752 18446744073709551615 242970593415 0 0 4096 18946 0 0 0 17 3 0 0 0
/proc/11622/statm: 514721 427144 476 189 0 509070 0
Current children cumulated CPU time (s) 40001.7
Current children cumulated vsize (KiB) 2058884

Child status: 0
Real time (s): 5045.28
CPU time (s): 40001.8
CPU user time (s): 39989.8
CPU system time (s): 11.9132
CPU usage (%): 792.854
Max. virtual memory (cumulated for all children) (KiB): 3761504

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39989.8
system time used= 11.9132
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 885283
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 181679
involuntary context switches= 189137

runsolver used 20.4689 second user time and 52.313 second system time

The end

Launcher Data 

Begin job on node128 at 2011-05-16 00:56:15
IDJOB=3381005
IDBENCH=83165
IDSOLVER=1865
FILE ID=node128/3381005-1305500174
RUNJOBID= node128-1305500174-11576
PBS_JOBID= 13325453
Free space on /tmp= 73580 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-21x33.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3381005-1305500174/watcher-3381005-1305500174 -o /tmp/evaluation-result-3381005-1305500174/solver-3381005-1305500174 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381005-1305500174.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= f7c29bbc3bef00d1e7cc799553ba4813
RANDOM SEED=527909453

node128.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5434.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      10665660 kB
Buffers:        293336 kB
Cached:        3246376 kB
SwapCached:       1440 kB
Active:       19784772 kB
Inactive:      2167180 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      10665660 kB
SwapTotal:    67111528 kB
SwapFree:     67093844 kB
Dirty:           11716 kB
Writeback:           4 kB
AnonPages:    18411136 kB
Mapped:          36716 kB
Slab:           191792 kB
PageTables:      70068 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS: 123663392 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73580 MiB
End job on node128 at 2011-05-16 02:20:21