Trace number 3373015

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 4800.15 1206.34

General information on the benchmark

Namecrafted/kullmann/GreenTao/
GreenTaok1k2/GreenTao_2-4-4_512.cnf
MD5SUMc1f047ec179094963b21d7b90509d2b9
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables510
Number of clauses4492
Sum of the clauses size17968
Maximum clause length4
Minimum clause length4
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 44492
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...425928/instance-3373015-1305425928.cnf
0.00/0.00	c Solving...
4800.10/1206.32	
4800.10/1206.32	*** INTERRUPTED! ***
4800.10/1206.32	*** Shutting down threads...
4800.10/1206.33	s UNKNOWN
4800.10/1206.33	
4800.10/1206.33	c INTERRUPTED   : 1
4800.10/1206.33	c Models        : 0+    
4800.10/1206.33	c Time          : 1206.327s (Solving: 1206.32s 1st Model: 0.00s Unsat: 0.00s)
4800.10/1206.33	c CPU Time      : 4800.130s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3373015-1305425928/watcher-3373015-1305425928 -o /tmp/evaluation-result-3373015-1305425928/solver-3373015-1305425928 -C 4800 -W 1300 -M 15500 HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373015-1305425928.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.83 9.01 8.46 5/338 24609
/proc/meminfo: memFree=22110780/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=21440 CPUtime=0 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 663 0 0 0 0 0 0 0 25 0 1 0 635008402 21954560 588 33554432000 4194304 4964800 140735224750144 18446744073709551615 4623600 0 0 4096 18946 0 0 0 17 3 0 0 0
/proc/24609/statm: 5360 588 399 189 0 234 0
[pid=24609/tid=24610] ppid=24607 vsize=103512 CPUtime=0 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 46 0 0 0 0 0 0 0 25 0 4 0 635008403 105996288 945 33554432000 4194304 4964800 140735224750144 18446744073709551615 270338488814 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=103512 CPUtime=0 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 98 0 0 0 0 0 0 0 25 0 4 0 635008403 105996288 945 33554432000 4194304 4964800 140735224750144 18446744073709551615 270338488863 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=103512 CPUtime=0 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 97 0 0 0 0 0 0 0 25 0 4 0 635008403 105996288 946 33554432000 4194304 4964800 140735224750144 18446744073709551615 270338488814 0 0 4096 18946 0 0 0 -1 1 0 0 0

[startup+0.154654 s]
/proc/loadavg: 8.83 9.01 8.46 5/338 24609
/proc/meminfo: memFree=22110780/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=169708 CPUtime=0.87 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 1708 0 0 0 87 0 0 0 18 0 4 0 635008402 173780992 1622 33554432000 4194304 4964800 140735224750144 18446744073709551615 270338488863 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 42559 1906 458 189 0 36908 0
[pid=24609/tid=24610] ppid=24607 vsize=170500 CPUtime=0.62 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 396 0 0 0 62 0 0 0 25 0 4 0 635008403 174592000 2267 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670295 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=170916 CPUtime=0.8 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 548 0 0 0 80 0 0 0 18 0 4 0 635008403 175017984 2483 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670538 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=170916 CPUtime=0.81 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 481 0 0 0 81 0 0 0 19 0 4 0 635008403 175017984 2483 33554432000 4194304 4964800 140735224750144 18446744073709551615 4321020 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 0.87
Current children cumulated vsize (KiB) 169708

[startup+0.911392 s]
/proc/loadavg: 8.83 9.01 8.46 5/338 24609
/proc/meminfo: memFree=22110780/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=171052 CPUtime=3.59 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 2794 0 0 0 359 0 0 0 19 0 4 0 635008402 175157248 2708 33554432000 4194304 4964800 140735224750144 18446744073709551615 270338488863 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 42763 2712 458 189 0 37112 0
[pid=24609/tid=24610] ppid=24607 vsize=171052 CPUtime=0.9 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 478 0 0 0 90 0 0 0 25 0 4 0 635008403 175157248 2712 33554432000 4194304 4964800 140735224750144 18446744073709551615 4334787 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=171052 CPUtime=0.9 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 621 0 0 0 90 0 0 0 19 0 4 0 635008403 175157248 2712 33554432000 4194304 4964800 140735224750144 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=171052 CPUtime=0.9 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 544 0 0 0 90 0 0 0 19 0 4 0 635008403 175157248 2712 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3.59
Current children cumulated vsize (KiB) 171052

[startup+1.02352 s]
/proc/loadavg: 8.83 9.01 8.46 5/338 24609
/proc/meminfo: memFree=22110780/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=171196 CPUtime=4.05 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 2835 0 0 0 404 1 0 0 19 0 4 0 635008402 175304704 2749 33554432000 4194304 4964800 140735224750144 18446744073709551615 4320955 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 42799 2749 458 189 0 37148 0
[pid=24609/tid=24610] ppid=24607 vsize=171196 CPUtime=1 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 485 0 0 0 100 0 0 0 25 0 4 0 635008403 175304704 2749 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670357 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=171196 CPUtime=1 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 646 0 0 0 100 0 0 0 19 0 4 0 635008403 175304704 2749 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670437 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=171196 CPUtime=1 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 549 0 0 0 100 0 0 0 19 0 4 0 635008403 175304704 2749 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670335 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4.05
Current children cumulated vsize (KiB) 171196

[startup+1.1235 s]
/proc/loadavg: 8.83 9.01 8.46 5/338 24609
/proc/meminfo: memFree=22110780/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=171328 CPUtime=4.45 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 2898 0 0 0 444 1 0 0 19 0 4 0 635008402 175439872 2812 33554432000 4194304 4964800 140735224750144 18446744073709551615 4320848 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 42832 2812 458 189 0 37181 0
[pid=24609/tid=24610] ppid=24607 vsize=171328 CPUtime=1.1 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 503 0 0 0 110 0 0 0 25 0 4 0 635008403 175439872 2812 33554432000 4194304 4964800 140735224750144 18446744073709551615 4332230 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=171328 CPUtime=1.1 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 683 0 0 0 110 0 0 0 19 0 4 0 635008403 175439872 2812 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=171328 CPUtime=1.1 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 557 0 0 0 110 0 0 0 19 0 4 0 635008403 175439872 2812 33554432000 4194304 4964800 140735224750144 18446744073709551615 4321122 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4.45
Current children cumulated vsize (KiB) 171328

[startup+1.50043 s]
/proc/loadavg: 8.83 9.01 8.46 5/338 24609
/proc/meminfo: memFree=22110780/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=172016 CPUtime=5.96 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 3236 0 0 0 595 1 0 0 20 0 4 0 635008402 176144384 3150 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670330 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 43004 3150 458 189 0 37353 0
[pid=24609/tid=24610] ppid=24607 vsize=172016 CPUtime=1.48 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 597 0 0 0 148 0 0 0 25 0 4 0 635008403 176144384 3150 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670264 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=172016 CPUtime=1.48 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 773 0 0 0 148 0 0 0 20 0 4 0 635008403 176144384 3150 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=172016 CPUtime=1.48 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 631 0 0 0 148 0 0 0 20 0 4 0 635008403 176144384 3150 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.96
Current children cumulated vsize (KiB) 172016

[startup+3.10111 s]
/proc/loadavg: 8.83 9.01 8.46 10/342 24613
/proc/meminfo: memFree=22095748/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=172832 CPUtime=12.33 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 3919 0 0 0 1232 1 0 0 25 0 4 0 635008402 176979968 3833 33554432000 4194304 4964800 140735224750144 18446744073709551615 4321013 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 43208 3833 458 189 0 37557 0
[pid=24609/tid=24610] ppid=24607 vsize=172832 CPUtime=3.08 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 771 0 0 0 308 0 0 0 25 0 4 0 635008403 176979968 3833 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=172832 CPUtime=3.08 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 1086 0 0 0 308 0 0 0 25 0 4 0 635008403 176979968 3833 33554432000 4194304 4964800 140735224750144 18446744073709551615 4320905 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=172832 CPUtime=3.06 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 733 0 0 0 306 0 0 0 25 0 4 0 635008403 176979968 3833 33554432000 4194304 4964800 140735224750144 18446744073709551615 4312185 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.33
Current children cumulated vsize (KiB) 172832

[startup+6.31248 s]
/proc/loadavg: 8.84 9.01 8.46 5/337 24615
/proc/meminfo: memFree=23228216/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=173960 CPUtime=25.14 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 4582 0 0 0 2512 2 0 0 19 0 4 0 635008402 178135040 4496 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670299 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 43490 4496 458 189 0 37839 0
[pid=24609/tid=24610] ppid=24607 vsize=173960 CPUtime=6.29 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 943 0 0 0 629 0 0 0 25 0 4 0 635008403 178135040 4496 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=173960 CPUtime=6.29 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 1401 0 0 0 629 0 0 0 25 0 4 0 635008403 178135040 4496 33554432000 4194304 4964800 140735224750144 18446744073709551615 4321054 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=173960 CPUtime=6.23 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 829 0 0 0 623 0 0 0 25 0 4 0 635008403 178135040 4496 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 25.14
Current children cumulated vsize (KiB) 173960

[startup+12.7002 s]
/proc/loadavg: 8.85 9.01 8.46 9/337 24714
/proc/meminfo: memFree=23141336/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=174764 CPUtime=50.62 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 5364 0 0 0 5059 3 0 0 25 0 4 0 635008402 178958336 5278 33554432000 4194304 4964800 140735224750144 18446744073709551615 4660425 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 43691 5278 458 189 0 38040 0
[pid=24609/tid=24610] ppid=24607 vsize=174764 CPUtime=12.67 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 1302 0 0 0 1267 0 0 0 25 0 4 0 635008403 178958336 5278 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=174764 CPUtime=12.68 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 1659 0 0 0 1268 0 0 0 25 0 4 0 635008403 178958336 5278 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670295 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=174764 CPUtime=12.55 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 911 0 0 0 1255 0 0 0 25 0 4 0 635008403 178958336 5278 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670313 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.62
Current children cumulated vsize (KiB) 174764

[startup+25.5007 s]
/proc/loadavg: 8.72 8.98 8.46 9/337 24715
/proc/meminfo: memFree=23085472/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=176236 CPUtime=101.69 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 6408 0 0 0 10164 5 0 0 25 0 4 0 635008402 180465664 6322 33554432000 4194304 4964800 140735224750144 18446744073709551615 4321116 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 44059 6322 458 189 0 38408 0
[pid=24609/tid=24610] ppid=24607 vsize=176236 CPUtime=25.48 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 1753 0 0 0 2547 1 0 0 25 0 4 0 635008403 180465664 6322 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=176236 CPUtime=25.47 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 1966 0 0 0 2547 0 0 0 25 0 4 0 635008403 180465664 6322 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670361 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=176236 CPUtime=25.21 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 1030 0 0 0 2521 0 0 0 18 0 4 0 635008403 180465664 6322 33554432000 4194304 4964800 140735224750144 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.69
Current children cumulated vsize (KiB) 176236

[startup+51.1006 s]
/proc/loadavg: 8.49 8.90 8.45 9/337 24715
/proc/meminfo: memFree=23028448/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=178072 CPUtime=203.8 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 7755 0 0 0 20371 9 0 0 25 0 4 0 635008402 182345728 7669 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670264 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 44518 7669 458 189 0 38867 0
[pid=24609/tid=24610] ppid=24607 vsize=178072 CPUtime=51.07 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 2361 0 0 0 5105 2 0 0 25 0 4 0 635008403 182345728 7669 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=178072 CPUtime=51.07 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 2361 0 0 0 5106 1 0 0 25 0 4 0 635008403 182345728 7669 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=178072 CPUtime=50.55 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 1169 0 0 0 5053 2 0 0 25 0 4 0 635008403 182345728 7669 33554432000 4194304 4964800 140735224750144 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.8
Current children cumulated vsize (KiB) 178072

[startup+102.3 s]
/proc/loadavg: 8.33 8.79 8.43 9/337 24717
/proc/meminfo: memFree=22928072/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=180752 CPUtime=408.04 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 9439 0 0 0 40789 15 0 0 25 0 4 0 635008402 185090048 9353 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 45188 9353 458 189 0 39537 0
[pid=24609/tid=24610] ppid=24607 vsize=180752 CPUtime=102.28 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 3318 0 0 0 10224 4 0 0 25 0 4 0 635008403 185090048 9353 33554432000 4194304 4964800 140735224750144 18446744073709551615 4675938 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=180752 CPUtime=102.27 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 2829 0 0 0 10224 3 0 0 25 0 4 0 635008403 185090048 9353 33554432000 4194304 4964800 140735224750144 18446744073709551615 4320947 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=180752 CPUtime=101.18 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 1274 0 0 0 10115 3 0 0 25 0 4 0 635008403 185090048 9353 33554432000 4194304 4964800 140735224750144 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 408.04
Current children cumulated vsize (KiB) 180752

[startup+162.301 s]

################
# More data... #
################

[pid=24609] ppid=24607 vsize=253616 CPUtime=2795.49 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 16195 0 0 0 279477 72 0 0 25 0 4 0 635008402 259702784 16109 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670403 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 63404 16109 458 189 0 57753 0
[pid=24609/tid=24610] ppid=24607 vsize=253616 CPUtime=696.52 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 6011 0 0 0 69633 19 0 0 25 0 4 0 635008403 259702784 16109 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670330 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=253616 CPUtime=701.36 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 4619 0 0 0 70119 17 0 0 25 0 4 0 635008403 259702784 16109 33554432000 4194304 4964800 140735224750144 18446744073709551615 4361118 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=253616 CPUtime=696.27 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 2453 0 0 0 69609 18 0 0 25 0 4 0 635008403 259702784 16109 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670361 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2795.49
Current children cumulated vsize (KiB) 253616

[startup+762.3 s]
/proc/loadavg: 9.32 9.17 8.78 10/343 5939
/proc/meminfo: memFree=22590768/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=253784 CPUtime=3034.15 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 16467 0 0 0 303340 75 0 0 25 0 4 0 635008402 259874816 16381 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670343 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 63446 16381 458 189 0 57795 0
[pid=24609/tid=24610] ppid=24607 vsize=253784 CPUtime=755.81 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 6103 0 0 0 75562 19 0 0 25 0 4 0 635008403 259874816 16381 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=253784 CPUtime=761.23 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 4664 0 0 0 76105 18 0 0 25 0 4 0 635008403 259874816 16381 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=253784 CPUtime=755.83 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 2567 0 0 0 75564 19 0 0 25 0 4 0 635008403 259874816 16381 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3034.15
Current children cumulated vsize (KiB) 253784

[startup+822.301 s]
/proc/loadavg: 9.42 9.22 8.82 11/345 7462
/proc/meminfo: memFree=22573924/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=253924 CPUtime=3272.75 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 16735 0 0 0 327194 81 0 0 25 0 4 0 635008402 260018176 16649 33554432000 4194304 4964800 140735224750144 18446744073709551615 270338050976 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/24609/statm: 63481 16649 458 189 0 57830 0
[pid=24609/tid=24610] ppid=24607 vsize=253924 CPUtime=815.6 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 6166 0 0 0 81539 21 0 0 25 0 4 0 635008403 260018176 16649 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=253924 CPUtime=821.12 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 4726 0 0 0 82092 20 0 0 25 0 4 0 635008403 260018176 16649 33554432000 4194304 4964800 140735224750144 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=253924 CPUtime=815.1 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 2669 0 0 0 81490 20 0 0 25 0 4 0 635008403 260018176 16649 33554432000 4194304 4964800 140735224750144 18446744073709551615 4361330 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3272.75
Current children cumulated vsize (KiB) 253924

[startup+882.301 s]
/proc/loadavg: 9.33 9.22 8.84 10/343 9083
/proc/meminfo: memFree=22548948/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=254236 CPUtime=3511.23 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 16991 0 0 0 351039 84 0 0 25 0 4 0 635008402 260337664 16905 33554432000 4194304 4964800 140735224750144 18446744073709551615 4321102 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/24609/statm: 63559 16905 458 189 0 57908 0
[pid=24609/tid=24610] ppid=24607 vsize=254236 CPUtime=875.43 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 6200 0 0 0 87521 22 0 0 25 0 4 0 635008403 260337664 16905 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=254236 CPUtime=881.06 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 4808 0 0 0 88085 21 0 0 25 0 4 0 635008403 260337664 16905 33554432000 4194304 4964800 140735224750144 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=254236 CPUtime=874.41 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 2766 0 0 0 87420 21 0 0 25 0 4 0 635008403 260337664 16905 33554432000 4194304 4964800 140735224750144 18446744073709551615 4321010 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3511.23
Current children cumulated vsize (KiB) 254236

[startup+942.301 s]
/proc/loadavg: 9.30 9.23 8.86 11/344 10596
/proc/meminfo: memFree=22502188/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=254376 CPUtime=3749.89 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 17228 0 0 0 374902 87 0 0 25 0 4 0 635008402 260481024 17142 33554432000 4194304 4964800 140735224750144 18446744073709551615 4359865 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/24609/statm: 63594 17142 458 189 0 57943 0
[pid=24609/tid=24610] ppid=24607 vsize=254376 CPUtime=935.37 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 6212 0 0 0 93514 23 0 0 25 0 4 0 635008403 260481024 17142 33554432000 4194304 4964800 140735224750144 18446744073709551615 4660442 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=254376 CPUtime=940.72 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 4901 0 0 0 94051 21 0 0 25 0 4 0 635008403 260481024 17142 33554432000 4194304 4964800 140735224750144 18446744073709551615 4642167 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=254376 CPUtime=933.93 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 2882 0 0 0 93372 21 0 0 25 0 4 0 635008403 260481024 17142 33554432000 4194304 4964800 140735224750144 18446744073709551615 270338488814 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3749.89
Current children cumulated vsize (KiB) 254376

[startup+1002.3 s]
/proc/loadavg: 9.22 9.21 8.88 10/344 12129
/proc/meminfo: memFree=22473404/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=254824 CPUtime=3988.53 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 17438 0 0 0 398761 92 0 0 25 0 4 0 635008402 260939776 17352 33554432000 4194304 4964800 140735224750144 18446744073709551615 4653193 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/24609/statm: 63706 17352 458 189 0 58055 0
[pid=24609/tid=24610] ppid=24607 vsize=254824 CPUtime=995.3 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 6223 0 0 0 99506 24 0 0 25 0 4 0 635008403 260939776 17352 33554432000 4194304 4964800 140735224750144 18446744073709551615 4627365 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=254824 CPUtime=999.99 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 4918 0 0 0 99977 22 0 0 25 0 4 0 635008403 260939776 17352 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670437 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=254824 CPUtime=993.75 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 2989 0 0 0 99352 23 0 0 25 0 4 0 635008403 260939776 17352 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3988.53
Current children cumulated vsize (KiB) 254824

[startup+1062.31 s]
/proc/loadavg: 9.16 9.20 8.90 10/343 13671
/proc/meminfo: memFree=22471360/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=255192 CPUtime=4227.19 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 17725 0 0 0 422622 97 0 0 25 0 4 0 635008402 261316608 17639 33554432000 4194304 4964800 140735224750144 18446744073709551615 4660163 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/24609/statm: 63798 17639 458 189 0 58147 0
[pid=24609/tid=24610] ppid=24607 vsize=255192 CPUtime=1055.24 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 6263 0 0 0 105499 25 0 0 25 0 4 0 635008403 261316608 17639 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=255192 CPUtime=1059.29 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 5027 0 0 0 105905 24 0 0 25 0 4 0 635008403 261316608 17639 33554432000 4194304 4964800 140735224750144 18446744073709551615 4321069 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=255192 CPUtime=1053.7 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 3076 0 0 0 105346 24 0 0 25 0 4 0 635008403 261316608 17639 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4227.19
Current children cumulated vsize (KiB) 255192

[startup+1122.3 s]
/proc/loadavg: 9.27 9.22 8.92 10/345 15175
/proc/meminfo: memFree=22464044/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=255332 CPUtime=4465.76 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 17906 0 0 0 446476 100 0 0 25 0 4 0 635008402 261459968 17820 33554432000 4194304 4964800 140735224750144 18446744073709551615 4325305 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 63833 17820 458 189 0 58182 0
[pid=24609/tid=24610] ppid=24607 vsize=255332 CPUtime=1114.89 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 6292 0 0 0 111463 26 0 0 25 0 4 0 635008403 261459968 17820 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670264 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=255332 CPUtime=1118.59 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 5105 0 0 0 111834 25 0 0 25 0 4 0 635008403 261459968 17820 33554432000 4194304 4964800 140735224750144 18446744073709551615 4321232 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=255332 CPUtime=1113.59 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 3099 0 0 0 111334 25 0 0 25 0 4 0 635008403 261459968 17820 33554432000 4194304 4964800 140735224750144 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4465.76
Current children cumulated vsize (KiB) 255332

[startup+1182.3 s]
/proc/loadavg: 9.27 9.23 8.94 10/345 16635
/proc/meminfo: memFree=22423900/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=255836 CPUtime=4704.5 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 18102 0 0 0 470344 106 0 0 25 0 4 0 635008402 261976064 18016 33554432000 4194304 4964800 140735224750144 18446744073709551615 4362289 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 63959 18016 458 189 0 58308 0
[pid=24609/tid=24610] ppid=24607 vsize=255836 CPUtime=1174.57 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 6324 0 0 0 117430 27 0 0 25 0 4 0 635008403 261976064 18016 33554432000 4194304 4964800 140735224750144 18446744073709551615 4321076 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=255836 CPUtime=1177.84 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 5174 0 0 0 117758 26 0 0 25 0 4 0 635008403 261976064 18016 33554432000 4194304 4964800 140735224750144 18446744073709551615 4363081 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=255836 CPUtime=1173.49 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 3156 0 0 0 117322 27 0 0 25 0 4 0 635008403 261976064 18016 33554432000 4194304 4964800 140735224750144 18446744073709551615 270338081410 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4704.5
Current children cumulated vsize (KiB) 255836



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1206.3 s]
/proc/loadavg: 9.33 9.24 8.95 11/345 17145
/proc/meminfo: memFree=22424104/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=255836 CPUtime=4800.1 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 18132 0 0 0 479901 109 0 0 25 0 4 0 635008402 261976064 18046 33554432000 4194304 4964800 140735224750144 18446744073709551615 4320934 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 63959 18046 458 189 0 58308 0
[pid=24609/tid=24610] ppid=24607 vsize=255836 CPUtime=1198.29 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 6325 0 0 0 119801 28 0 0 25 0 4 0 635008403 261976064 18046 33554432000 4194304 4964800 140735224750144 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=255836 CPUtime=1201.76 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 5174 0 0 0 120150 26 0 0 18 0 4 0 635008403 261976064 18046 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=255836 CPUtime=1197.48 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 3184 0 0 0 119721 27 0 0 25 0 4 0 635008403 261976064 18046 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.1
Current children cumulated vsize (KiB) 255836

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1206.3 s]
/proc/loadavg: 9.33 9.24 8.95 11/345 17145
/proc/meminfo: memFree=22424104/32951124 swapFree=59367104/67111528
[pid=24609] ppid=24607 vsize=255836 CPUtime=4800.1 cores=1,3,5,7
/proc/24609/stat : 24609 (claspmt) R 24607 24609 22057 0 -1 4202496 18132 0 0 0 479901 109 0 0 25 0 4 0 635008402 261976064 18046 33554432000 4194304 4964800 140735224750144 18446744073709551615 4320934 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/24609/statm: 63959 18046 458 189 0 58308 0
[pid=24609/tid=24610] ppid=24607 vsize=255836 CPUtime=1198.29 cores=1,3,5,7
/proc/24609/task/24610/stat : 24610 (claspmt) R 24607 24609 22057 0 -1 4202560 6325 0 0 0 119801 28 0 0 25 0 4 0 635008403 261976064 18046 33554432000 4194304 4964800 140735224750144 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=24609/tid=24611] ppid=24607 vsize=255836 CPUtime=1201.76 cores=1,3,5,7
/proc/24609/task/24611/stat : 24611 (claspmt) R 24607 24609 22057 0 -1 4202560 5174 0 0 0 120150 26 0 0 18 0 4 0 635008403 261976064 18046 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24609/tid=24612] ppid=24607 vsize=255836 CPUtime=1197.48 cores=1,3,5,7
/proc/24609/task/24612/stat : 24612 (claspmt) R 24607 24609 22057 0 -1 4202560 3184 0 0 0 119721 27 0 0 25 0 4 0 635008403 261976064 18046 33554432000 4194304 4964800 140735224750144 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.1
Current children cumulated vsize (KiB) 255836

Child status: 0
Real time (s): 1206.34
CPU time (s): 4800.15
CPU user time (s): 4799.05
CPU system time (s): 1.09583
CPU usage (%): 397.91
Max. virtual memory (cumulated for all children) (KiB): 255836

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4799.05
system time used= 1.09583
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 18145
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 86804
involuntary context switches= 50253

runsolver used 3.36949 second user time and 9.77851 second system time

The end

Launcher Data 

Begin job on node143 at 2011-05-15 04:18:48
IDJOB=3373015
IDBENCH=83247
IDSOLVER=1853
FILE ID=node143/3373015-1305425928
RUNJOBID= node143-1305423515-22076
PBS_JOBID= 13324594
Free space on /tmp= 73412 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/crafted/kullmann/GreenTao/GreenTaok1k2/GreenTao_2-4-4_512.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3373015-1305425928/watcher-3373015-1305425928 -o /tmp/evaluation-result-3373015-1305425928/solver-3373015-1305425928 -C 4800 -W 1300 -M 15500  HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373015-1305425928.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= c1f047ec179094963b21d7b90509d2b9
RANDOM SEED=1419652734

node143.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      22111052 kB
Buffers:        180172 kB
Cached:         761604 kB
SwapCached:      67400 kB
Active:        1256428 kB
Inactive:      9345572 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      22111052 kB
SwapTotal:    67111528 kB
SwapFree:     59367104 kB
Dirty:            1040 kB
Writeback:           0 kB
AnonPages:     9592784 kB
Mapped:          23720 kB
Slab:           118112 kB
PageTables:      55696 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 71143852 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73548 MiB
End job on node143 at 2011-05-15 04:38:56