Trace number 3322704

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40000.8 5008.48

General information on the benchmark

NameSAT02/handmade/simon/
satex-challenges/c6288.shuffled.cnf
MD5SUM75bd8319132b47c2d2e6f0889735a248
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark0.774881
Satisfiable
(Un)Satisfiability was proved
Number of variables5008
Number of clauses14813
Sum of the clauses size34485
Maximum clause length33
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 29982
Number of clauses of size 34829
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 51

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...393168/instance-3322704-1304393168.cnf
0.02/0.03	c Solving...
40000.70/5008.41	
40000.70/5008.41	*** INTERRUPTED! ***
40000.70/5008.41	*** Shutting down threads...
40000.70/5008.47	s UNKNOWN
40000.70/5008.47	
40000.70/5008.47	c INTERRUPTED   : 1
40000.70/5008.47	c Models        : 0+    
40000.70/5008.47	c Time          : 5008.417s (Solving: 5008.39s 1st Model: 0.00s Unsat: 0.00s)
40000.70/5008.47	c CPU Time      : 40000.750s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3322704-1304393168/watcher-3322704-1304393168 -o /tmp/evaluation-result-3322704-1304393168/solver-3322704-1304393168 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322704-1304393168.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.82 7.79 7.92 2/359 18267
/proc/meminfo: memFree=6863792/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=23296 CPUtime=0 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 1218 0 0 0 0 0 0 0 23 0 1 0 629547453 23855104 1061 33554432000 4194304 4964704 140736728962272 18446744073709551615 4608680 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/18267/statm: 5824 1061 403 189 0 698 0

[startup+0.029278 s]
/proc/loadavg: 6.82 7.79 7.92 2/359 18267
/proc/meminfo: memFree=6863792/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=23824 CPUtime=0.02 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 1322 0 0 0 2 0 0 0 23 0 1 0 629547453 24395776 1165 33554432000 4194304 4964704 140736728962272 18446744073709551615 4623979 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/18267/statm: 5956 1165 404 189 0 830 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 23824

[startup+0.13965 s]
/proc/loadavg: 6.82 7.79 7.92 2/359 18267
/proc/meminfo: memFree=6863792/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=255324 CPUtime=0.75 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 3477 0 0 0 75 0 0 0 25 0 8 0 629547453 261451776 2965 33554432000 4194304 4964704 140736728962272 18446744073709551615 4660057 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/18267/statm: 63831 2965 464 189 0 58180 0
Current children cumulated CPU time (s) 0.75
Current children cumulated vsize (KiB) 255324

[startup+0.300192 s]
/proc/loadavg: 6.82 7.79 7.92 2/359 18267
/proc/meminfo: memFree=6863792/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=256288 CPUtime=2.05 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 4356 0 0 0 204 1 0 0 25 0 8 0 629547453 262438912 3844 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670219 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/18267/statm: 64072 3844 464 189 0 58421 0
Current children cumulated CPU time (s) 2.05
Current children cumulated vsize (KiB) 256288

[startup+0.700125 s]
/proc/loadavg: 6.82 7.79 7.92 2/359 18267
/proc/meminfo: memFree=6863792/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=256420 CPUtime=5.25 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 4699 0 0 0 524 1 0 0 25 0 8 0 629547453 262574080 4187 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670260 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/18267/statm: 64105 4187 464 189 0 58454 0
Current children cumulated CPU time (s) 5.25
Current children cumulated vsize (KiB) 256420

[startup+1.50099 s]
/proc/loadavg: 6.82 7.79 7.92 9/367 18275
/proc/meminfo: memFree=6847360/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=257344 CPUtime=11.64 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 5430 0 0 0 1163 1 0 0 25 0 8 0 629547453 263520256 4918 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669494 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/18267/statm: 64336 4918 464 189 0 58685 0
[pid=18267/tid=18269] ppid=18265 vsize=257344 CPUtime=1.45 cores=0-7
/proc/18267/task/18269/stat : 18269 (claspmt) R 18265 18267 18241 0 -1 4202560 544 0 0 0 145 0 0 0 19 0 8 0 629547457 263520256 4918 33554432000 4194304 4964704 140736728962272 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18267/tid=18270] ppid=18265 vsize=257344 CPUtime=1.45 cores=0-7
/proc/18267/task/18270/stat : 18270 (claspmt) R 18265 18267 18241 0 -1 4202560 591 0 0 0 145 0 0 0 20 0 8 0 629547457 263520256 4918 33554432000 4194304 4964704 140736728962272 18446744073709551615 4342208 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18267/tid=18271] ppid=18265 vsize=257344 CPUtime=1.44 cores=0-7
/proc/18267/task/18271/stat : 18271 (claspmt) R 18265 18267 18241 0 -1 4202560 487 0 0 0 144 0 0 0 20 0 8 0 629547457 263520256 4918 33554432000 4194304 4964704 140736728962272 18446744073709551615 4362996 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18267/tid=18272] ppid=18265 vsize=257344 CPUtime=1.45 cores=0-7
/proc/18267/task/18272/stat : 18272 (claspmt) R 18265 18267 18241 0 -1 4202560 490 0 0 0 145 0 0 0 19 0 8 0 629547457 263520256 4918 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668635 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18267/tid=18273] ppid=18265 vsize=257344 CPUtime=1.45 cores=0-7
/proc/18267/task/18273/stat : 18273 (claspmt) R 18265 18267 18241 0 -1 4202560 435 0 0 0 145 0 0 0 19 0 8 0 629547457 263520256 4918 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668632 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18267/tid=18274] ppid=18265 vsize=257344 CPUtime=1.45 cores=0-7
/proc/18267/task/18274/stat : 18274 (claspmt) R 18265 18267 18241 0 -1 4202560 487 0 0 0 145 0 0 0 20 0 8 0 629547457 263520256 4918 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668632 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18267/tid=18275] ppid=18265 vsize=257344 CPUtime=1.45 cores=0-7
/proc/18267/task/18275/stat : 18275 (claspmt) R 18265 18267 18241 0 -1 4202560 714 0 0 0 145 0 0 0 19 0 8 0 629547457 263520256 4918 33554432000 4194304 4964704 140736728962272 18446744073709551615 4365104 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.64
Current children cumulated vsize (KiB) 257344

[startup+3.10073 s]
/proc/loadavg: 6.82 7.79 7.92 9/367 18275
/proc/meminfo: memFree=6843404/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=258940 CPUtime=24.41 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 6746 0 0 0 2440 1 0 0 25 0 8 0 629547453 265154560 6234 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669200 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/18267/statm: 64735 6234 464 189 0 59084 0
[pid=18267/tid=18269] ppid=18265 vsize=258940 CPUtime=3.05 cores=0-7
/proc/18267/task/18269/stat : 18269 (claspmt) R 18265 18267 18241 0 -1 4202560 668 0 0 0 305 0 0 0 25 0 8 0 629547457 265154560 6234 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18267/tid=18270] ppid=18265 vsize=258940 CPUtime=3.05 cores=0-7
/proc/18267/task/18270/stat : 18270 (claspmt) R 18265 18267 18241 0 -1 4202560 874 0 0 0 305 0 0 0 25 0 8 0 629547457 265154560 6234 33554432000 4194304 4964704 140736728962272 18446744073709551615 4660088 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18267/tid=18271] ppid=18265 vsize=258940 CPUtime=3.03 cores=0-7
/proc/18267/task/18271/stat : 18271 (claspmt) R 18265 18267 18241 0 -1 4202560 570 0 0 0 303 0 0 0 25 0 8 0 629547457 265154560 6234 33554432000 4194304 4964704 140736728962272 18446744073709551615 4644854 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18267/tid=18272] ppid=18265 vsize=258940 CPUtime=3.05 cores=0-7
/proc/18267/task/18272/stat : 18272 (claspmt) R 18265 18267 18241 0 -1 4202560 637 0 0 0 305 0 0 0 25 0 8 0 629547457 265154560 6234 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670049 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18267/tid=18273] ppid=18265 vsize=258940 CPUtime=3.05 cores=0-7
/proc/18267/task/18273/stat : 18273 (claspmt) R 18265 18267 18241 0 -1 4202560 549 0 0 0 305 0 0 0 25 0 8 0 629547457 265154560 6234 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669333 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18267/tid=18274] ppid=18265 vsize=258940 CPUtime=3.05 cores=0-7
/proc/18267/task/18274/stat : 18274 (claspmt) R 18265 18267 18241 0 -1 4202560 586 0 0 0 305 0 0 0 25 0 8 0 629547457 265154560 6234 33554432000 4194304 4964704 140736728962272 18446744073709551615 4320996 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18267/tid=18275] ppid=18265 vsize=258940 CPUtime=3.04 cores=0-7
/proc/18267/task/18275/stat : 18275 (claspmt) R 18265 18267 18241 0 -1 4202560 1130 0 0 0 304 0 0 0 25 0 8 0 629547457 265154560 6234 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668657 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 24.41
Current children cumulated vsize (KiB) 258940

[startup+6.3012 s]
/proc/loadavg: 6.91 7.79 7.92 9/367 18275
/proc/meminfo: memFree=6833636/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=261576 CPUtime=49.97 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 9610 0 0 0 4994 3 0 0 25 0 8 0 629547453 267853824 9098 33554432000 4194304 4964704 140736728962272 18446744073709551615 4653272 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/18267/statm: 65394 9098 466 189 0 59743 0
[pid=18267/tid=18269] ppid=18265 vsize=261576 CPUtime=6.25 cores=0-7
/proc/18267/task/18269/stat : 18269 (claspmt) R 18265 18267 18241 0 -1 4202560 899 0 0 0 625 0 0 0 25 0 8 0 629547457 267853824 9098 33554432000 4194304 4964704 140736728962272 18446744073709551615 4647742 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18267/tid=18270] ppid=18265 vsize=261576 CPUtime=6.25 cores=0-7
/proc/18267/task/18270/stat : 18270 (claspmt) R 18265 18267 18241 0 -1 4202560 1440 0 0 0 625 0 0 0 25 0 8 0 629547457 267853824 9098 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18267/tid=18271] ppid=18265 vsize=261576 CPUtime=6.19 cores=0-7
/proc/18267/task/18271/stat : 18271 (claspmt) R 18265 18267 18241 0 -1 4202560 749 0 0 0 619 0 0 0 25 0 8 0 629547457 267853824 9098 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669893 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18267/tid=18272] ppid=18265 vsize=261576 CPUtime=6.25 cores=0-7
/proc/18267/task/18272/stat : 18272 (claspmt) R 18265 18267 18241 0 -1 4202560 984 0 0 0 625 0 0 0 25 0 8 0 629547457 267853824 9098 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670454 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18267/tid=18273] ppid=18265 vsize=261576 CPUtime=6.25 cores=0-7
/proc/18267/task/18273/stat : 18273 (claspmt) R 18265 18267 18241 0 -1 4202560 1012 0 0 0 625 0 0 0 25 0 8 0 629547457 267853824 9098 33554432000 4194304 4964704 140736728962272 18446744073709551615 4660085 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18267/tid=18274] ppid=18265 vsize=261576 CPUtime=6.25 cores=0-7
/proc/18267/task/18274/stat : 18274 (claspmt) R 18265 18267 18241 0 -1 4202560 775 0 0 0 625 0 0 0 21 0 8 0 629547457 267853824 9098 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18267/tid=18275] ppid=18265 vsize=261576 CPUtime=6.23 cores=0-7
/proc/18267/task/18275/stat : 18275 (claspmt) R 18265 18267 18241 0 -1 4202560 1821 0 0 0 623 0 0 0 25 0 8 0 629547457 267853824 9098 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670357 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 49.97
Current children cumulated vsize (KiB) 261576

[startup+12.7011 s]
/proc/loadavg: 7.00 7.80 7.92 10/367 18275
/proc/meminfo: memFree=6822972/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=263060 CPUtime=101.09 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 11302 0 0 0 10105 4 0 0 25 0 8 0 629547453 269373440 10736 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668618 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/18267/statm: 65765 10736 466 189 0 60114 0
[pid=18267/tid=18269] ppid=18265 vsize=263060 CPUtime=12.64 cores=0-7
/proc/18267/task/18269/stat : 18269 (claspmt) R 18265 18267 18241 0 -1 4202560 1082 0 0 0 1264 0 0 0 25 0 8 0 629547457 269373440 10736 33554432000 4194304 4964704 140736728962272 18446744073709551615 4683983 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18267/tid=18270] ppid=18265 vsize=263060 CPUtime=12.64 cores=0-7
/proc/18267/task/18270/stat : 18270 (claspmt) R 18265 18267 18241 0 -1 4202560 1900 0 0 0 1264 0 0 0 19 0 8 0 629547457 269373440 10736 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670263 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18267/tid=18271] ppid=18265 vsize=263060 CPUtime=12.51 cores=0-7
/proc/18267/task/18271/stat : 18271 (claspmt) R 18265 18267 18241 0 -1 4202560 887 0 0 0 1251 0 0 0 25 0 8 0 629547457 269373440 10736 33554432000 4194304 4964704 140736728962272 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18267/tid=18272] ppid=18265 vsize=263060 CPUtime=12.65 cores=0-7
/proc/18267/task/18272/stat : 18272 (claspmt) R 18265 18267 18241 0 -1 4202560 1187 0 0 0 1265 0 0 0 25 0 8 0 629547457 269373440 10736 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18267/tid=18273] ppid=18265 vsize=263060 CPUtime=12.65 cores=0-7
/proc/18267/task/18273/stat : 18273 (claspmt) R 18265 18267 18241 0 -1 4202560 1168 0 0 0 1265 0 0 0 25 0 8 0 629547457 269373440 10736 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670184 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18267/tid=18274] ppid=18265 vsize=263060 CPUtime=12.65 cores=0-7
/proc/18267/task/18274/stat : 18274 (claspmt) R 18265 18267 18241 0 -1 4202560 873 0 0 0 1265 0 0 0 25 0 8 0 629547457 269373440 10736 33554432000 4194304 4964704 140736728962272 18446744073709551615 4342213 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18267/tid=18275] ppid=18265 vsize=263060 CPUtime=12.63 cores=0-7
/proc/18267/task/18275/stat : 18275 (claspmt) R 18265 18267 18241 0 -1 4202560 2160 0 0 0 1263 0 0 0 25 0 8 0 629547457 269373440 10736 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669218 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 101.09
Current children cumulated vsize (KiB) 263060

[startup+25.501 s]
/proc/loadavg: 7.36 7.84 7.93 9/367 18275
/proc/meminfo: memFree=6816508/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=265176 CPUtime=203.32 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 13249 0 0 0 20327 5 0 0 25 0 8 0 629547453 271540224 12683 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670219 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/18267/statm: 66294 12683 466 189 0 60643 0
[pid=18267/tid=18269] ppid=18265 vsize=265176 CPUtime=25.44 cores=0-7
/proc/18267/task/18269/stat : 18269 (claspmt) R 18265 18267 18241 0 -1 4202560 1381 0 0 0 2544 0 0 0 25 0 8 0 629547457 271540224 12683 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669452 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18267/tid=18270] ppid=18265 vsize=265176 CPUtime=25.44 cores=0-7
/proc/18267/task/18270/stat : 18270 (claspmt) R 18265 18267 18241 0 -1 4202560 2177 0 0 0 2544 0 0 0 25 0 8 0 629547457 271540224 12683 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668654 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18267/tid=18271] ppid=18265 vsize=265176 CPUtime=25.15 cores=0-7
/proc/18267/task/18271/stat : 18271 (claspmt) R 18265 18267 18241 0 -1 4202560 981 0 0 0 2515 0 0 0 25 0 8 0 629547457 271540224 12683 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668652 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18267/tid=18272] ppid=18265 vsize=265176 CPUtime=25.44 cores=0-7
/proc/18267/task/18272/stat : 18272 (claspmt) R 18265 18267 18241 0 -1 4202560 1546 0 0 0 2544 0 0 0 25 0 8 0 629547457 271540224 12683 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670594 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18267/tid=18273] ppid=18265 vsize=265176 CPUtime=25.45 cores=0-7
/proc/18267/task/18273/stat : 18273 (claspmt) R 18265 18267 18241 0 -1 4202560 1321 0 0 0 2545 0 0 0 25 0 8 0 629547457 271540224 12683 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18267/tid=18274] ppid=18265 vsize=265176 CPUtime=25.44 cores=0-7
/proc/18267/task/18274/stat : 18274 (claspmt) R 18265 18267 18241 0 -1 4202560 1000 0 0 0 2544 0 0 0 25 0 8 0 629547457 271540224 12683 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670100 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18267/tid=18275] ppid=18265 vsize=265176 CPUtime=25.44 cores=0-7
/proc/18267/task/18275/stat : 18275 (claspmt) R 18265 18267 18241 0 -1 4202560 2636 0 0 0 2543 1 0 0 25 0 8 0 629547457 271540224 12683 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668714 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 203.32
Current children cumulated vsize (KiB) 265176

[startup+51.1007 s]
/proc/loadavg: 7.58 7.85 7.93 9/367 18277
/proc/meminfo: memFree=6800832/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=269996 CPUtime=407.78 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 17136 0 0 0 40769 9 0 0 25 0 8 0 629547453 276475904 16570 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669270 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/18267/statm: 67499 16570 466 189 0 61848 0
[pid=18267/tid=18269] ppid=18265 vsize=269996 CPUtime=51.03 cores=0-7
/proc/18267/task/18269/stat : 18269 (claspmt) R 18265 18267 18241 0 -1 4202560 1763 0 0 0 5103 0 0 0 25 0 8 0 629547457 276475904 16570 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669196 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18267/tid=18270] ppid=18265 vsize=269996 CPUtime=51.04 cores=0-7
/proc/18267/task/18270/stat : 18270 (claspmt) R 18265 18267 18241 0 -1 4202560 2929 0 0 0 5103 1 0 0 25 0 8 0 629547457 276475904 16570 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18267/tid=18271] ppid=18265 vsize=269996 CPUtime=50.45 cores=0-7
/proc/18267/task/18271/stat : 18271 (claspmt) R 18265 18267 18241 0 -1 4202560 1216 0 0 0 5045 0 0 0 25 0 8 0 629547457 276475904 16570 33554432000 4194304 4964704 140736728962272 18446744073709551615 4361365 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18267/tid=18272] ppid=18265 vsize=269996 CPUtime=51.03 cores=0-7
/proc/18267/task/18272/stat : 18272 (claspmt) R 18265 18267 18241 0 -1 4202560 1939 0 0 0 5103 0 0 0 25 0 8 0 629547457 276475904 16570 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670250 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18267/tid=18273] ppid=18265 vsize=269996 CPUtime=51.05 cores=0-7
/proc/18267/task/18273/stat : 18273 (claspmt) R 18265 18267 18241 0 -1 4202560 1798 0 0 0 5104 1 0 0 25 0 8 0 629547457 276475904 16570 33554432000 4194304 4964704 140736728962272 18446744073709551615 4320939 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18267/tid=18274] ppid=18265 vsize=269996 CPUtime=51.04 cores=0-7
/proc/18267/task/18274/stat : 18274 (claspmt) R 18265 18267 18241 0 -1 4202560 1144 0 0 0 5104 0 0 0 25 0 8 0 629547457 276475904 16570 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18267/tid=18275] ppid=18265 vsize=269996 CPUtime=51.02 cores=0-7
/proc/18267/task/18275/stat : 18275 (claspmt) R 18265 18267 18241 0 -1 4202560 3722 0 0 0 5101 1 0 0 25 0 8 0 629547457 276475904 16570 33554432000 4194304 4964704 140736728962272 18446744073709551615 4320890 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 407.78

################
# More data... #
################


[startup+4662.3 s]
/proc/loadavg: 8.02 8.06 8.00 9/367 18414
/proc/meminfo: memFree=6610764/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=375972 CPUtime=37236.3 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 64097 0 0 0 3723358 269 0 0 25 0 8 0 629547453 384995328 62882 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669270 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/18267/statm: 93993 62882 466 189 0 88342 0
[pid=18267/tid=18269] ppid=18265 vsize=375972 CPUtime=4630.84 cores=0-7
/proc/18267/task/18269/stat : 18269 (claspmt) R 18265 18267 18241 0 -1 4202560 8915 0 0 0 463049 35 0 0 25 0 8 0 629547457 384995328 62882 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669399 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18267/tid=18270] ppid=18265 vsize=375972 CPUtime=4661.55 cores=0-7
/proc/18267/task/18270/stat : 18270 (claspmt) R 18265 18267 18241 0 -1 4202560 9304 0 0 0 466118 37 0 0 25 0 8 0 629547457 384995328 62882 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669452 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18267/tid=18271] ppid=18265 vsize=375972 CPUtime=4645.2 cores=0-7
/proc/18267/task/18271/stat : 18271 (claspmt) R 18265 18267 18241 0 -1 4202560 3587 0 0 0 464492 28 0 0 25 0 8 0 629547457 384995328 62882 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18267/tid=18272] ppid=18265 vsize=375972 CPUtime=4661.51 cores=0-7
/proc/18267/task/18272/stat : 18272 (claspmt) R 18265 18267 18241 0 -1 4202560 7733 0 0 0 466117 34 0 0 25 0 8 0 629547457 384995328 62882 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668673 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18267/tid=18273] ppid=18265 vsize=375972 CPUtime=4661.75 cores=0-7
/proc/18267/task/18273/stat : 18273 (claspmt) R 18265 18267 18241 0 -1 4202560 8535 0 0 0 466146 29 0 0 25 0 8 0 629547457 384995328 62882 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18267/tid=18274] ppid=18265 vsize=375972 CPUtime=4661.42 cores=0-7
/proc/18267/task/18274/stat : 18274 (claspmt) R 18265 18267 18241 0 -1 4202560 4244 0 0 0 466117 25 0 0 25 0 8 0 629547457 384995328 62882 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18267/tid=18275] ppid=18265 vsize=375972 CPUtime=4660.86 cores=0-7
/proc/18267/task/18275/stat : 18275 (claspmt) R 18265 18267 18241 0 -1 4202560 16612 0 0 0 466039 47 0 0 25 0 8 0 629547457 384995328 62882 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37236.3
Current children cumulated vsize (KiB) 375972

[startup+4722.31 s]
/proc/loadavg: 8.03 8.06 8.00 9/367 18416
/proc/meminfo: memFree=6609584/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=376448 CPUtime=37715.6 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 64469 0 0 0 3771289 272 0 0 25 0 8 0 629547453 385482752 63254 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669270 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/18267/statm: 94112 63254 466 189 0 88461 0
[pid=18267/tid=18269] ppid=18265 vsize=376448 CPUtime=4690.83 cores=0-7
/proc/18267/task/18269/stat : 18269 (claspmt) R 18265 18267 18241 0 -1 4202560 8970 0 0 0 469048 35 0 0 25 0 8 0 629547457 385482752 63254 33554432000 4194304 4964704 140736728962272 18446744073709551615 4632224 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18267/tid=18270] ppid=18265 vsize=376448 CPUtime=4721.56 cores=0-7
/proc/18267/task/18270/stat : 18270 (claspmt) R 18265 18267 18241 0 -1 4202560 9336 0 0 0 472119 37 0 0 25 0 8 0 629547457 385482752 63254 33554432000 4194304 4964704 140736728962272 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18267/tid=18271] ppid=18265 vsize=376448 CPUtime=4704.49 cores=0-7
/proc/18267/task/18271/stat : 18271 (claspmt) R 18265 18267 18241 0 -1 4202560 3605 0 0 0 470421 28 0 0 25 0 8 0 629547457 385482752 63254 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18267/tid=18272] ppid=18265 vsize=376448 CPUtime=4721.52 cores=0-7
/proc/18267/task/18272/stat : 18272 (claspmt) R 18265 18267 18241 0 -1 4202560 7769 0 0 0 472118 34 0 0 25 0 8 0 629547457 385482752 63254 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18267/tid=18273] ppid=18265 vsize=376448 CPUtime=4721.77 cores=0-7
/proc/18267/task/18273/stat : 18273 (claspmt) R 18265 18267 18241 0 -1 4202560 8562 0 0 0 472147 30 0 0 25 0 8 0 629547457 385482752 63254 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18267/tid=18274] ppid=18265 vsize=376448 CPUtime=4721.44 cores=0-7
/proc/18267/task/18274/stat : 18274 (claspmt) R 18265 18267 18241 0 -1 4202560 4292 0 0 0 472118 26 0 0 25 0 8 0 629547457 385482752 63254 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18267/tid=18275] ppid=18265 vsize=376448 CPUtime=4720.87 cores=0-7
/proc/18267/task/18275/stat : 18275 (claspmt) R 18265 18267 18241 0 -1 4202560 16696 0 0 0 472039 48 0 0 25 0 8 0 629547457 385482752 63254 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37715.6
Current children cumulated vsize (KiB) 376448

[startup+4782.3 s]
/proc/loadavg: 8.08 8.06 8.00 9/367 18418
/proc/meminfo: memFree=6608668/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=376580 CPUtime=38194.7 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 64800 0 0 0 3819196 276 0 0 25 0 8 0 629547453 385617920 63585 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670260 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/18267/statm: 94145 63585 466 189 0 88494 0
[pid=18267/tid=18269] ppid=18265 vsize=376580 CPUtime=4750.8 cores=0-7
/proc/18267/task/18269/stat : 18269 (claspmt) R 18265 18267 18241 0 -1 4202560 9096 0 0 0 475044 36 0 0 25 0 8 0 629547457 385617920 63585 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18267/tid=18270] ppid=18265 vsize=376580 CPUtime=4781.54 cores=0-7
/proc/18267/task/18270/stat : 18270 (claspmt) R 18265 18267 18241 0 -1 4202560 9389 0 0 0 478117 37 0 0 25 0 8 0 629547457 385617920 63585 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18267/tid=18271] ppid=18265 vsize=376580 CPUtime=4763.76 cores=0-7
/proc/18267/task/18271/stat : 18271 (claspmt) R 18265 18267 18241 0 -1 4202560 3650 0 0 0 476347 29 0 0 25 0 8 0 629547457 385617920 63585 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669462 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18267/tid=18272] ppid=18265 vsize=376580 CPUtime=4781.5 cores=0-7
/proc/18267/task/18272/stat : 18272 (claspmt) R 18265 18267 18241 0 -1 4202560 7829 0 0 0 478115 35 0 0 25 0 8 0 629547457 385617920 63585 33554432000 4194304 4964704 140736728962272 18446744073709551615 4660077 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18267/tid=18273] ppid=18265 vsize=376580 CPUtime=4781.74 cores=0-7
/proc/18267/task/18273/stat : 18273 (claspmt) R 18265 18267 18241 0 -1 4202560 8566 0 0 0 478144 30 0 0 25 0 8 0 629547457 385617920 63585 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18267/tid=18274] ppid=18265 vsize=376580 CPUtime=4781.42 cores=0-7
/proc/18267/task/18274/stat : 18274 (claspmt) R 18265 18267 18241 0 -1 4202560 4312 0 0 0 478116 26 0 0 25 0 8 0 629547457 385617920 63585 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18267/tid=18275] ppid=18265 vsize=376580 CPUtime=4780.83 cores=0-7
/proc/18267/task/18275/stat : 18275 (claspmt) R 18265 18267 18241 0 -1 4202560 16710 0 0 0 478035 48 0 0 25 0 8 0 629547457 385617920 63585 33554432000 4194304 4964704 140736728962272 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38194.7
Current children cumulated vsize (KiB) 376580

[startup+4842.3 s]
/proc/loadavg: 8.09 8.06 8.00 9/367 18418
/proc/meminfo: memFree=6608004/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=376580 CPUtime=38674 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 65022 0 0 0 3867117 280 0 0 25 0 8 0 629547453 385617920 63807 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668488 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/18267/statm: 94145 63807 466 189 0 88494 0
[pid=18267/tid=18269] ppid=18265 vsize=376580 CPUtime=4810.8 cores=0-7
/proc/18267/task/18269/stat : 18269 (claspmt) R 18265 18267 18241 0 -1 4202560 9157 0 0 0 481043 37 0 0 25 0 8 0 629547457 385617920 63807 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18267/tid=18270] ppid=18265 vsize=376580 CPUtime=4841.53 cores=0-7
/proc/18267/task/18270/stat : 18270 (claspmt) R 18265 18267 18241 0 -1 4202560 9412 0 0 0 484116 37 0 0 25 0 8 0 629547457 385617920 63807 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668946 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18267/tid=18271] ppid=18265 vsize=376580 CPUtime=4823.04 cores=0-7
/proc/18267/task/18271/stat : 18271 (claspmt) R 18265 18267 18241 0 -1 4202560 3650 0 0 0 482275 29 0 0 25 0 8 0 629547457 385617920 63807 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18267/tid=18272] ppid=18265 vsize=376580 CPUtime=4841.5 cores=0-7
/proc/18267/task/18272/stat : 18272 (claspmt) R 18265 18267 18241 0 -1 4202560 7861 0 0 0 484115 35 0 0 25 0 8 0 629547457 385617920 63807 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668721 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18267/tid=18273] ppid=18265 vsize=376580 CPUtime=4841.75 cores=0-7
/proc/18267/task/18273/stat : 18273 (claspmt) R 18265 18267 18241 0 -1 4202560 8588 0 0 0 484144 31 0 0 25 0 8 0 629547457 385617920 63807 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669354 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18267/tid=18274] ppid=18265 vsize=376580 CPUtime=4841.41 cores=0-7
/proc/18267/task/18274/stat : 18274 (claspmt) R 18265 18267 18241 0 -1 4202560 4312 0 0 0 484115 26 0 0 25 0 8 0 629547457 385617920 63807 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18267/tid=18275] ppid=18265 vsize=376580 CPUtime=4840.82 cores=0-7
/proc/18267/task/18275/stat : 18275 (claspmt) R 18265 18267 18241 0 -1 4202560 16782 0 0 0 484033 49 0 0 25 0 8 0 629547457 385617920 63807 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668676 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38674
Current children cumulated vsize (KiB) 376580

[startup+4902.3 s]
/proc/loadavg: 8.10 8.07 8.00 9/367 18420
/proc/meminfo: memFree=6606220/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=377260 CPUtime=39153.2 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 65490 0 0 0 3915038 282 0 0 25 0 8 0 629547453 386314240 64236 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668654 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/18267/statm: 94315 64236 466 189 0 88664 0
[pid=18267/tid=18269] ppid=18265 vsize=377260 CPUtime=4870.79 cores=0-7
/proc/18267/task/18269/stat : 18269 (claspmt) R 18265 18267 18241 0 -1 4202560 9214 0 0 0 487042 37 0 0 25 0 8 0 629547457 386314240 64236 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670100 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18267/tid=18270] ppid=18265 vsize=377260 CPUtime=4901.54 cores=0-7
/proc/18267/task/18270/stat : 18270 (claspmt) R 18265 18267 18241 0 -1 4202560 9425 0 0 0 490116 38 0 0 25 0 8 0 629547457 386314240 64236 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670100 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18267/tid=18271] ppid=18265 vsize=377260 CPUtime=4882.33 cores=0-7
/proc/18267/task/18271/stat : 18271 (claspmt) R 18265 18267 18241 0 -1 4202560 3678 0 0 0 488203 30 0 0 25 0 8 0 629547457 386314240 64236 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18267/tid=18272] ppid=18265 vsize=377260 CPUtime=4901.49 cores=0-7
/proc/18267/task/18272/stat : 18272 (claspmt) R 18265 18267 18241 0 -1 4202560 7981 0 0 0 490114 35 0 0 25 0 8 0 629547457 386314240 64236 33554432000 4194304 4964704 140736728962272 18446744073709551615 4342199 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18267/tid=18273] ppid=18265 vsize=377260 CPUtime=4901.74 cores=0-7
/proc/18267/task/18273/stat : 18273 (claspmt) R 18265 18267 18241 0 -1 4202560 8648 0 0 0 490143 31 0 0 25 0 8 0 629547457 386314240 64236 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18267/tid=18274] ppid=18265 vsize=377260 CPUtime=4901.4 cores=0-7
/proc/18267/task/18274/stat : 18274 (claspmt) R 18265 18267 18241 0 -1 4202560 4377 0 0 0 490114 26 0 0 25 0 8 0 629547457 386314240 64236 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668907 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18267/tid=18275] ppid=18265 vsize=377260 CPUtime=4900.8 cores=0-7
/proc/18267/task/18275/stat : 18275 (claspmt) R 18265 18267 18241 0 -1 4202560 16843 0 0 0 490031 49 0 0 25 0 8 0 629547457 386314240 64236 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669277 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39153.2
Current children cumulated vsize (KiB) 377260

[startup+4962.3 s]
/proc/loadavg: 8.08 8.06 8.00 9/367 18422
/proc/meminfo: memFree=6603812/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=377260 CPUtime=39632.4 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 66136 0 0 0 3962955 287 0 0 25 0 8 0 629547453 386314240 64882 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670277 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/18267/statm: 94315 64882 466 189 0 88664 0
[pid=18267/tid=18269] ppid=18265 vsize=377260 CPUtime=4930.78 cores=0-7
/proc/18267/task/18269/stat : 18269 (claspmt) R 18265 18267 18241 0 -1 4202560 9290 0 0 0 493040 38 0 0 25 0 8 0 629547457 386314240 64882 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18267/tid=18270] ppid=18265 vsize=377260 CPUtime=4961.53 cores=0-7
/proc/18267/task/18270/stat : 18270 (claspmt) R 18265 18267 18241 0 -1 4202560 9569 0 0 0 496114 39 0 0 25 0 8 0 629547457 386314240 64882 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668766 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18267/tid=18271] ppid=18265 vsize=377260 CPUtime=4941.6 cores=0-7
/proc/18267/task/18271/stat : 18271 (claspmt) R 18265 18267 18241 0 -1 4202560 3775 0 0 0 494130 30 0 0 25 0 8 0 629547457 386314240 64882 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18267/tid=18272] ppid=18265 vsize=377260 CPUtime=4961.48 cores=0-7
/proc/18267/task/18272/stat : 18272 (claspmt) R 18265 18267 18241 0 -1 4202560 8002 0 0 0 496113 35 0 0 25 0 8 0 629547457 386314240 64882 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18267/tid=18273] ppid=18265 vsize=377260 CPUtime=4961.73 cores=0-7
/proc/18267/task/18273/stat : 18273 (claspmt) R 18265 18267 18241 0 -1 4202560 8731 0 0 0 496142 31 0 0 25 0 8 0 629547457 386314240 64882 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18267/tid=18274] ppid=18265 vsize=377260 CPUtime=4961.4 cores=0-7
/proc/18267/task/18274/stat : 18274 (claspmt) R 18265 18267 18241 0 -1 4202560 4377 0 0 0 496113 27 0 0 25 0 8 0 629547457 386314240 64882 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18267/tid=18275] ppid=18265 vsize=377260 CPUtime=4960.78 cores=0-7
/proc/18267/task/18275/stat : 18275 (claspmt) R 18265 18267 18241 0 -1 4202560 16973 0 0 0 496028 50 0 0 25 0 8 0 629547457 386314240 64882 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39632.4
Current children cumulated vsize (KiB) 377260



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5008.4 s]
/proc/loadavg: 8.17 8.09 8.01 9/367 18424
/proc/meminfo: memFree=6603540/32951124 swapFree=61020464/67111528
[pid=18267] ppid=18265 vsize=377260 CPUtime=40000.7 cores=0-7
/proc/18267/stat : 18267 (claspmt) R 18265 18267 18241 0 -1 4202496 66196 0 0 0 3999778 292 0 0 25 0 8 0 629547453 386314240 64936 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668488 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/18267/statm: 94315 64936 466 189 0 88664 0
[pid=18267/tid=18269] ppid=18265 vsize=377260 CPUtime=4976.87 cores=0-7
/proc/18267/task/18269/stat : 18269 (claspmt) R 18265 18267 18241 0 -1 4202560 9311 0 0 0 497649 38 0 0 21 0 8 0 629547457 386314240 64936 33554432000 4194304 4964704 140736728962272 18446744073709551615 4654417 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18267/tid=18270] ppid=18265 vsize=377260 CPUtime=5007.63 cores=0-7
/proc/18267/task/18270/stat : 18270 (claspmt) R 18265 18267 18241 0 -1 4202560 9569 0 0 0 500724 39 0 0 25 0 8 0 629547457 386314240 64936 33554432000 4194304 4964704 140736728962272 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18267/tid=18271] ppid=18265 vsize=377260 CPUtime=4987.14 cores=0-7
/proc/18267/task/18271/stat : 18271 (claspmt) R 18265 18267 18241 0 -1 4202560 3791 0 0 0 498683 31 0 0 25 0 8 0 629547457 386314240 64936 33554432000 4194304 4964704 140736728962272 18446744073709551615 4668648 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18267/tid=18272] ppid=18265 vsize=377260 CPUtime=5007.59 cores=0-7
/proc/18267/task/18272/stat : 18272 (claspmt) R 18265 18267 18241 0 -1 4202560 8007 0 0 0 500724 35 0 0 25 0 8 0 629547457 386314240 64936 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18267/tid=18273] ppid=18265 vsize=377260 CPUtime=5007.84 cores=0-7
/proc/18267/task/18273/stat : 18273 (claspmt) R 18265 18267 18241 0 -1 4202560 8734 0 0 0 500752 32 0 0 25 0 8 0 629547457 386314240 64936 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669472 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18267/tid=18274] ppid=18265 vsize=377260 CPUtime=5007.5 cores=0-7
/proc/18267/task/18274/stat : 18274 (claspmt) R 18265 18267 18241 0 -1 4202560 4382 0 0 0 500722 28 0 0 25 0 8 0 629547457 386314240 64936 33554432000 4194304 4964704 140736728962272 18446744073709551615 4379632 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18267/tid=18275] ppid=18265 vsize=377260 CPUtime=5006.89 cores=0-7
/proc/18267/task/18275/stat : 18275 (claspmt) R 18265 18267 18241 0 -1 4202560 16973 0 0 0 500638 51 0 0 25 0 8 0 629547457 386314240 64936 33554432000 4194304 4964704 140736728962272 18446744073709551615 4669399 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 377260

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0
Real time (s): 5008.48
CPU time (s): 40000.8
CPU user time (s): 39997.9
CPU system time (s): 2.93855
CPU usage (%): 798.662
Max. virtual memory (cumulated for all children) (KiB): 377260

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39997.9
system time used= 2.93855
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 66207
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 326088
involuntary context switches= 192781

runsolver used 17.1474 second user time and 42.0606 second system time

The end

Launcher Data 

Begin job on node118 at 2011-05-03 05:26:08
IDJOB=3322704
IDBENCH=87734
IDSOLVER=1711
FILE ID=node118/3322704-1304393168
RUNJOBID= node118-1304393168-18255
PBS_JOBID= 13181856
Free space on /tmp= 73556 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT02/handmade/simon/satex-challenges/c6288.shuffled.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322704-1304393168/watcher-3322704-1304393168 -o /tmp/evaluation-result-3322704-1304393168/solver-3322704-1304393168 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322704-1304393168.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 75bd8319132b47c2d2e6f0889735a248
RANDOM SEED=612948829

node118.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5374.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:       6864188 kB
Buffers:         53116 kB
Cached:         195500 kB
SwapCached:      96888 kB
Active:       12765448 kB
Inactive:     13079180 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:       6864188 kB
SwapTotal:    67111528 kB
SwapFree:     61020464 kB
Dirty:            1284 kB
Writeback:           4 kB
AnonPages:    25510552 kB
Mapped:          16884 kB
Slab:            87140 kB
PageTables:      89656 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 88624632 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73552 MiB
End job on node118 at 2011-05-03 06:49:38