Trace number 3380967

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40001 5010.34

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-bottom18.cnf
MD5SUM9ece8f569c93ceee834817ef3350ed8e
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables36410
Number of clauses121578
Sum of the clauses size309786
Maximum clause length1188
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 262692
Number of clauses of size 358247
Number of clauses of size 4170
Number of clauses of size 516
Number of clauses of size over 5452

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...497587/instance-3380967-1305497587.cnf
0.28/0.35	c Solving...
40000.57/5010.02	
40000.57/5010.02	*** INTERRUPTED! ***
40000.57/5010.02	*** Shutting down threads...
40000.95/5010.33	s UNKNOWN
40000.95/5010.33	
40000.95/5010.33	c INTERRUPTED   : 1
40000.95/5010.33	c Models        : 0+    
40000.95/5010.33	c Time          : 5010.025s (Solving: 5009.67s 1st Model: 0.00s Unsat: 0.00s)
40000.95/5010.33	c CPU Time      : 40000.670s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3380967-1305497587/watcher-3380967-1305497587 -o /tmp/evaluation-result-3380967-1305497587/solver-3380967-1305497587 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380967-1305497587.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.83 7.81 7.94 2/438 11333
/proc/meminfo: memFree=21601260/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=28136 CPUtime=0 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 2581 0 0 0 0 0 0 0 22 0 1 0 661359084 28811264 2227 33554432000 4194304 4964800 140736631714016 18446744073709551615 253640556299 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/11333/statm: 7034 2227 389 189 0 1908 0

[startup+0.0480851 s]
/proc/loadavg: 6.83 7.81 7.94 2/438 11333
/proc/meminfo: memFree=21601260/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=35216 CPUtime=0.03 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 5144 0 0 0 3 0 0 0 22 0 1 0 661359084 36061184 3962 33554432000 4194304 4964800 140736631714016 18446744073709551615 4615486 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/11333/statm: 8804 3965 399 189 0 3678 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 35216

[startup+0.101053 s]
/proc/loadavg: 6.83 7.81 7.94 2/438 11333
/proc/meminfo: memFree=21601260/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=39560 CPUtime=0.09 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 6191 0 0 0 9 0 0 0 22 0 1 0 661359084 40509440 5009 33554432000 4194304 4964800 140736631714016 18446744073709551615 4623021 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/11333/statm: 9890 5009 404 189 0 4764 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 39560

[startup+0.30102 s]
/proc/loadavg: 6.83 7.81 7.94 2/438 11333
/proc/meminfo: memFree=21601260/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=43784 CPUtime=0.28 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 7233 0 0 0 28 0 0 0 23 0 1 0 661359084 44834816 6051 33554432000 4194304 4964800 140736631714016 18446744073709551615 4622615 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/11333/statm: 10946 6051 404 189 0 5820 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 43784

[startup+0.700911 s]
/proc/loadavg: 6.83 7.81 7.94 2/438 11333
/proc/meminfo: memFree=21601260/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=482944 CPUtime=2.32 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 18450 0 0 0 226 6 0 0 25 0 8 0 661359084 494534656 14698 33554432000 4194304 4964800 140736631714016 18446744073709551615 4342223 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/11333/statm: 120736 14698 464 189 0 115085 0
Current children cumulated CPU time (s) 2.32
Current children cumulated vsize (KiB) 482944

[startup+1.50076 s]
/proc/loadavg: 6.83 7.81 7.94 10/446 11341
/proc/meminfo: memFree=21540472/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=484576 CPUtime=8.7 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 21769 0 0 0 862 8 0 0 25 0 8 0 661359084 496205824 18017 33554432000 4194304 4964800 140736631714016 18446744073709551615 4669512 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/11333/statm: 121144 18017 464 189 0 115493 0
[pid=11333/tid=11335] ppid=11331 vsize=484576 CPUtime=1.03 cores=0-7
/proc/11333/task/11335/stat : 11335 (claspmt) R 11331 11333 11307 0 -1 4202560 1639 0 0 0 103 0 0 0 18 0 8 0 661359120 496205824 18017 33554432000 4194304 4964800 140736631714016 18446744073709551615 4668725 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11333/tid=11336] ppid=11331 vsize=484576 CPUtime=1.03 cores=0-7
/proc/11333/task/11336/stat : 11336 (claspmt) R 11331 11333 11307 0 -1 4202560 2312 0 0 0 102 1 0 0 18 0 8 0 661359120 496205824 18017 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11333/tid=11337] ppid=11331 vsize=484576 CPUtime=1.03 cores=0-7
/proc/11333/task/11337/stat : 11337 (claspmt) R 11331 11333 11307 0 -1 4202560 2134 0 0 0 103 0 0 0 18 0 8 0 661359120 496205824 18017 33554432000 4194304 4964800 140736631714016 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11333/tid=11338] ppid=11331 vsize=484576 CPUtime=1.03 cores=0-7
/proc/11333/task/11338/stat : 11338 (claspmt) R 11331 11333 11307 0 -1 4202560 1828 0 0 0 103 0 0 0 18 0 8 0 661359120 496205824 18017 33554432000 4194304 4964800 140736631714016 18446744073709551615 4333168 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11333/tid=11339] ppid=11331 vsize=484576 CPUtime=1.12 cores=0-7
/proc/11333/task/11339/stat : 11339 (claspmt) R 11331 11333 11307 0 -1 4202560 1335 0 0 0 112 0 0 0 19 0 8 0 661359120 496205824 18017 33554432000 4194304 4964800 140736631714016 18446744073709551615 4668987 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11333/tid=11340] ppid=11331 vsize=484576 CPUtime=1.03 cores=0-7
/proc/11333/task/11340/stat : 11340 (claspmt) R 11331 11333 11307 0 -1 4202560 1863 0 0 0 102 1 0 0 18 0 8 0 661359120 496205824 18017 33554432000 4194304 4964800 140736631714016 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11333/tid=11341] ppid=11331 vsize=484576 CPUtime=1.03 cores=0-7
/proc/11333/task/11341/stat : 11341 (claspmt) R 11331 11333 11307 0 -1 4202560 2610 0 0 0 102 1 0 0 18 0 8 0 661359120 496205824 18017 33554432000 4194304 4964800 140736631714016 18446744073709551615 4627168 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 8.7
Current children cumulated vsize (KiB) 484576

[startup+3.10047 s]
/proc/loadavg: 6.83 7.81 7.94 9/446 11341
/proc/meminfo: memFree=21524848/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=487008 CPUtime=21.49 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 26639 0 0 0 2137 12 0 0 25 0 8 0 661359084 498696192 22887 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/11333/statm: 121752 22887 464 189 0 116101 0
[pid=11333/tid=11335] ppid=11331 vsize=487008 CPUtime=2.62 cores=0-7
/proc/11333/task/11335/stat : 11335 (claspmt) R 11331 11333 11307 0 -1 4202560 1984 0 0 0 262 0 0 0 21 0 8 0 661359120 498696192 22887 33554432000 4194304 4964800 140736631714016 18446744073709551615 253640555961 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11333/tid=11336] ppid=11331 vsize=487008 CPUtime=2.62 cores=0-7
/proc/11333/task/11336/stat : 11336 (claspmt) R 11331 11333 11307 0 -1 4202560 3049 0 0 0 261 1 0 0 20 0 8 0 661359120 498696192 22887 33554432000 4194304 4964800 140736631714016 18446744073709551615 4342144 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11333/tid=11337] ppid=11331 vsize=487008 CPUtime=2.63 cores=0-7
/proc/11333/task/11337/stat : 11337 (claspmt) R 11331 11333 11307 0 -1 4202560 2871 0 0 0 262 1 0 0 20 0 8 0 661359120 498696192 22887 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670190 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11333/tid=11338] ppid=11331 vsize=487008 CPUtime=2.63 cores=0-7
/proc/11333/task/11338/stat : 11338 (claspmt) R 11331 11333 11307 0 -1 4202560 2491 0 0 0 262 1 0 0 18 0 8 0 661359120 498696192 22887 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670264 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11333/tid=11339] ppid=11331 vsize=487008 CPUtime=2.72 cores=0-7
/proc/11333/task/11339/stat : 11339 (claspmt) R 11331 11333 11307 0 -1 4202560 1798 0 0 0 272 0 0 0 23 0 8 0 661359120 498696192 22887 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11333/tid=11340] ppid=11331 vsize=487008 CPUtime=2.62 cores=0-7
/proc/11333/task/11340/stat : 11340 (claspmt) R 11331 11333 11307 0 -1 4202560 2364 0 0 0 261 1 0 0 20 0 8 0 661359120 498696192 22887 33554432000 4194304 4964800 140736631714016 18446744073709551615 4658177 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11333/tid=11341] ppid=11331 vsize=487008 CPUtime=2.63 cores=0-7
/proc/11333/task/11341/stat : 11341 (claspmt) R 11331 11333 11307 0 -1 4202560 3655 0 0 0 261 2 0 0 21 0 8 0 661359120 498696192 22887 33554432000 4194304 4964800 140736631714016 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 21.49
Current children cumulated vsize (KiB) 487008

[startup+6.30089 s]
/proc/loadavg: 6.93 7.82 7.94 9/446 11341
/proc/meminfo: memFree=21501060/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=489384 CPUtime=47.04 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 29983 0 0 0 4689 15 0 0 25 0 8 0 661359084 501129216 26231 33554432000 4194304 4964800 140736631714016 18446744073709551615 4654668 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/11333/statm: 122346 26231 464 189 0 116695 0
[pid=11333/tid=11335] ppid=11331 vsize=489384 CPUtime=5.83 cores=0-7
/proc/11333/task/11335/stat : 11335 (claspmt) R 11331 11333 11307 0 -1 4202560 2337 0 0 0 582 1 0 0 25 0 8 0 661359120 501129216 26231 33554432000 4194304 4964800 140736631714016 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11333/tid=11336] ppid=11331 vsize=489384 CPUtime=5.83 cores=0-7
/proc/11333/task/11336/stat : 11336 (claspmt) R 11331 11333 11307 0 -1 4202560 3433 0 0 0 581 2 0 0 25 0 8 0 661359120 501129216 26231 33554432000 4194304 4964800 140736631714016 18446744073709551615 253640520737 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11333/tid=11337] ppid=11331 vsize=489384 CPUtime=5.83 cores=0-7
/proc/11333/task/11337/stat : 11337 (claspmt) R 11331 11333 11307 0 -1 4202560 3254 0 0 0 582 1 0 0 25 0 8 0 661359120 501129216 26231 33554432000 4194304 4964800 140736631714016 18446744073709551615 4321020 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11333/tid=11338] ppid=11331 vsize=489384 CPUtime=5.83 cores=0-7
/proc/11333/task/11338/stat : 11338 (claspmt) R 11331 11333 11307 0 -1 4202560 2856 0 0 0 582 1 0 0 25 0 8 0 661359120 501129216 26232 33554432000 4194304 4964800 140736631714016 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11333/tid=11339] ppid=11331 vsize=489384 CPUtime=5.93 cores=0-7
/proc/11333/task/11339/stat : 11339 (claspmt) R 11331 11333 11307 0 -1 4202560 2411 0 0 0 592 1 0 0 25 0 8 0 661359120 501129216 26232 33554432000 4194304 4964800 140736631714016 18446744073709551615 4370462 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11333/tid=11340] ppid=11331 vsize=489384 CPUtime=5.82 cores=0-7
/proc/11333/task/11340/stat : 11340 (claspmt) R 11331 11333 11307 0 -1 4202560 2627 0 0 0 581 1 0 0 25 0 8 0 661359120 501129216 26232 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11333/tid=11341] ppid=11331 vsize=489384 CPUtime=5.83 cores=0-7
/proc/11333/task/11341/stat : 11341 (claspmt) R 11331 11333 11307 0 -1 4202560 4336 0 0 0 580 3 0 0 25 0 8 0 661359120 501129216 26232 33554432000 4194304 4964800 140736631714016 18446744073709551615 253640520737 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 47.04
Current children cumulated vsize (KiB) 489384

[startup+12.7007 s]
/proc/loadavg: 7.01 7.82 7.94 9/446 11341
/proc/meminfo: memFree=21476400/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=491280 CPUtime=98.14 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 36089 0 0 0 9794 20 0 0 25 0 8 0 661359084 503070720 32337 33554432000 4194304 4964800 140736631714016 18446744073709551615 4669509 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/11333/statm: 122820 32337 466 189 0 117169 0
[pid=11333/tid=11335] ppid=11331 vsize=491280 CPUtime=12.23 cores=0-7
/proc/11333/task/11335/stat : 11335 (claspmt) R 11331 11333 11307 0 -1 4202560 2753 0 0 0 1222 1 0 0 25 0 8 0 661359120 503070720 32337 33554432000 4194304 4964800 140736631714016 18446744073709551615 46936009002960 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11333/tid=11336] ppid=11331 vsize=491280 CPUtime=12.23 cores=0-7
/proc/11333/task/11336/stat : 11336 (claspmt) R 11331 11333 11307 0 -1 4202560 5050 0 0 0 1220 3 0 0 25 0 8 0 661359120 503070720 32337 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670313 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11333/tid=11337] ppid=11331 vsize=491280 CPUtime=12.23 cores=0-7
/proc/11333/task/11337/stat : 11337 (claspmt) R 11331 11333 11307 0 -1 4202560 3757 0 0 0 1221 2 0 0 25 0 8 0 661359120 503070720 32337 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11333/tid=11338] ppid=11331 vsize=491280 CPUtime=12.22 cores=0-7
/proc/11333/task/11338/stat : 11338 (claspmt) R 11331 11333 11307 0 -1 4202560 3383 0 0 0 1221 1 0 0 25 0 8 0 661359120 503070720 32337 33554432000 4194304 4964800 140736631714016 18446744073709551615 4668672 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11333/tid=11339] ppid=11331 vsize=491280 CPUtime=12.32 cores=0-7
/proc/11333/task/11339/stat : 11339 (claspmt) R 11331 11333 11307 0 -1 4202560 3147 0 0 0 1231 1 0 0 25 0 8 0 661359120 503070720 32337 33554432000 4194304 4964800 140736631714016 18446744073709551615 4669033 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11333/tid=11340] ppid=11331 vsize=491280 CPUtime=12.22 cores=0-7
/proc/11333/task/11340/stat : 11340 (claspmt) R 11331 11333 11307 0 -1 4202560 3345 0 0 0 1220 2 0 0 25 0 8 0 661359120 503070720 32337 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11333/tid=11341] ppid=11331 vsize=491280 CPUtime=12.23 cores=0-7
/proc/11333/task/11341/stat : 11341 (claspmt) R 11331 11333 11307 0 -1 4202560 5554 0 0 0 1219 4 0 0 25 0 8 0 661359120 503070720 32338 33554432000 4194304 4964800 140736631714016 18446744073709551615 253640520737 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 98.14
Current children cumulated vsize (KiB) 491280

[startup+25.5004 s]
/proc/loadavg: 7.23 7.83 7.94 9/446 11342
/proc/meminfo: memFree=21419660/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=500364 CPUtime=200.32 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 50159 0 0 0 20002 30 0 0 25 0 8 0 661359084 512372736 46407 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670180 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/11333/statm: 125091 46407 466 189 0 119440 0
[pid=11333/tid=11335] ppid=11331 vsize=500364 CPUtime=25.02 cores=0-7
/proc/11333/task/11335/stat : 11335 (claspmt) R 11331 11333 11307 0 -1 4202560 3663 0 0 0 2500 2 0 0 25 0 8 0 661359120 512372736 46407 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11333/tid=11336] ppid=11331 vsize=500364 CPUtime=25.03 cores=0-7
/proc/11333/task/11336/stat : 11336 (claspmt) R 11331 11333 11307 0 -1 4202560 8315 0 0 0 2498 5 0 0 25 0 8 0 661359120 512372736 46407 33554432000 4194304 4964800 140736631714016 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11333/tid=11337] ppid=11331 vsize=500364 CPUtime=25.03 cores=0-7
/proc/11333/task/11337/stat : 11337 (claspmt) R 11331 11333 11307 0 -1 4202560 5387 0 0 0 2500 3 0 0 25 0 8 0 661359120 512372736 46407 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670137 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11333/tid=11338] ppid=11331 vsize=500364 CPUtime=25.02 cores=0-7
/proc/11333/task/11338/stat : 11338 (claspmt) R 11331 11333 11307 0 -1 4202560 5221 0 0 0 2499 3 0 0 25 0 8 0 661359120 512372736 46407 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11333/tid=11339] ppid=11331 vsize=500364 CPUtime=25.12 cores=0-7
/proc/11333/task/11339/stat : 11339 (claspmt) R 11331 11333 11307 0 -1 4202560 4440 0 0 0 2510 2 0 0 25 0 8 0 661359120 512372736 46407 33554432000 4194304 4964800 140736631714016 18446744073709551615 253640557117 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11333/tid=11340] ppid=11331 vsize=500364 CPUtime=25.01 cores=0-7
/proc/11333/task/11340/stat : 11340 (claspmt) R 11331 11333 11307 0 -1 4202560 4122 0 0 0 2498 3 0 0 25 0 8 0 661359120 512372736 46407 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11333/tid=11341] ppid=11331 vsize=500364 CPUtime=25.04 cores=0-7
/proc/11333/task/11341/stat : 11341 (claspmt) R 11331 11333 11307 0 -1 4202560 9158 0 0 0 2497 7 0 0 25 0 8 0 661359120 512372736 46407 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670020 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 200.32
Current children cumulated vsize (KiB) 500364

[startup+51.1007 s]
/proc/loadavg: 7.56 7.85 7.94 9/446 11342
/proc/meminfo: memFree=21362052/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=509048 CPUtime=404.75 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 64560 0 0 0 40431 44 0 0 25 0 8 0 661359084 521265152 60808 33554432000 4194304 4964800 140736631714016 18446744073709551615 4654668 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/11333/statm: 127262 60808 466 189 0 121611 0
[pid=11333/tid=11335] ppid=11331 vsize=509048 CPUtime=50.62 cores=0-7
/proc/11333/task/11335/stat : 11335 (claspmt) R 11331 11333 11307 0 -1 4202560 4567 0 0 0 5059 3 0 0 25 0 8 0 661359120 521265152 60808 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11333/tid=11336] ppid=11331 vsize=509048 CPUtime=50.62 cores=0-7
/proc/11333/task/11336/stat : 11336 (claspmt) R 11331 11333 11307 0 -1 4202560 10692 0 0 0 5056 6 0 0 25 0 8 0 661359120 521265152 60808 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670343 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11333/tid=11337] ppid=11331 vsize=509048 CPUtime=50.62 cores=0-7
/proc/11333/task/11337/stat : 11337 (claspmt) R 11331 11333 11307 0 -1 4202560 6663 0 0 0 5058 4 0 0 25 0 8 0 661359120 521265152 60808 33554432000 4194304 4964800 140736631714016 18446744073709551615 4342228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11333/tid=11338] ppid=11331 vsize=509048 CPUtime=50.63 cores=0-7
/proc/11333/task/11338/stat : 11338 (claspmt) R 11331 11333 11307 0 -1 4202560 7220 0 0 0 5058 5 0 0 25 0 8 0 661359120 521265152 60808 33554432000 4194304 4964800 140736631714016 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11333/tid=11339] ppid=11331 vsize=509048 CPUtime=50.72 cores=0-7
/proc/11333/task/11339/stat : 11339 (claspmt) R 11331 11333 11307 0 -1 4202560 6386 0 0 0 5068 4 0 0 25 0 8 0 661359120 521265152 60808 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670340 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11333/tid=11340] ppid=11331 vsize=509048 CPUtime=50.61 cores=0-7
/proc/11333/task/11340/stat : 11340 (claspmt) R 11331 11333 11307 0 -1 4202560 5110 0 0 0 5058 3 0 0 25 0 8 0 661359120 521265152 60808 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670264 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11333/tid=11341] ppid=11331 vsize=509048 CPUtime=50.63 cores=0-7
/proc/11333/task/11341/stat : 11341 (claspmt) R 11331 11333 11307 0 -1 4202560 12948 0 0 0 5052 11 0 0 25 0 8 0 661359120 521265152 60808 33554432000 4194304 4964800 140736631714016 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 404.75

################
# More data... #
################

/proc/11333/task/11340/stat : 11340 (claspmt) R 11331 11333 11307 0 -1 4202560 25767 0 0 0 466027 36 0 0 25 0 8 0 661359120 1465241600 300584 33554432000 4194304 4964800 140736631714016 18446744073709551615 4668654 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11333/tid=11341] ppid=11331 vsize=1430900 CPUtime=4659.98 cores=0-7
/proc/11333/task/11341/stat : 11341 (claspmt) R 11331 11333 11307 0 -1 4202560 85397 0 0 0 465889 109 0 0 25 0 8 0 661359120 1465241600 300584 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37224.1
Current children cumulated vsize (KiB) 1430900

[startup+4722.32 s]
/proc/loadavg: 8.00 8.02 8.00 9/446 11479
/proc/meminfo: memFree=20396844/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=1430900 CPUtime=37703.4 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 309844 0 0 0 3769924 412 0 0 25 0 8 0 661359084 1465241600 300784 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670437 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11333/statm: 357725 300784 466 189 0 352074 0
[pid=11333/tid=11335] ppid=11331 vsize=1430900 CPUtime=4666.64 cores=0-7
/proc/11333/task/11335/stat : 11335 (claspmt) R 11331 11333 11307 0 -1 4202560 23394 0 0 0 466628 36 0 0 25 0 8 0 661359120 1465241600 300784 33554432000 4194304 4964800 140736631714016 18446744073709551615 4669413 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11333/tid=11336] ppid=11331 vsize=1430900 CPUtime=4721.8 cores=0-7
/proc/11333/task/11336/stat : 11336 (claspmt) R 11331 11333 11307 0 -1 4202560 47863 0 0 0 472130 50 0 0 25 0 8 0 661359120 1465241600 300784 33554432000 4194304 4964800 140736631714016 18446744073709551615 4668753 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11333/tid=11337] ppid=11331 vsize=1430900 CPUtime=4721.69 cores=0-7
/proc/11333/task/11337/stat : 11337 (claspmt) R 11331 11333 11307 0 -1 4202560 32026 0 0 0 472125 44 0 0 25 0 8 0 661359120 1465241600 300784 33554432000 4194304 4964800 140736631714016 18446744073709551615 4669532 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11333/tid=11338] ppid=11331 vsize=1430900 CPUtime=4721.75 cores=0-7
/proc/11333/task/11338/stat : 11338 (claspmt) R 11331 11333 11307 0 -1 4202560 35143 0 0 0 472125 50 0 0 25 0 8 0 661359120 1465241600 300784 33554432000 4194304 4964800 140736631714016 18446744073709551615 4669093 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11333/tid=11339] ppid=11331 vsize=1430900 CPUtime=4715.88 cores=0-7
/proc/11333/task/11339/stat : 11339 (claspmt) R 11331 11333 11307 0 -1 4202560 30925 0 0 0 471547 41 0 0 25 0 8 0 661359120 1465241600 300784 33554432000 4194304 4964800 140736631714016 18446744073709551615 4668737 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11333/tid=11340] ppid=11331 vsize=1430900 CPUtime=4720.63 cores=0-7
/proc/11333/task/11340/stat : 11340 (claspmt) R 11331 11333 11307 0 -1 4202560 25821 0 0 0 472027 36 0 0 25 0 8 0 661359120 1465241600 300784 33554432000 4194304 4964800 140736631714016 18446744073709551615 4332656 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11333/tid=11341] ppid=11331 vsize=1430900 CPUtime=4720 cores=0-7
/proc/11333/task/11341/stat : 11341 (claspmt) R 11331 11333 11307 0 -1 4202560 85405 0 0 0 471891 109 0 0 25 0 8 0 661359120 1465241600 300784 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37703.4
Current children cumulated vsize (KiB) 1430900

[startup+4782.3 s]
/proc/loadavg: 8.00 8.02 8.00 9/446 11480
/proc/meminfo: memFree=20383692/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=1430900 CPUtime=38182.3 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 313498 0 0 0 3817816 418 0 0 25 0 8 0 661359084 1465241600 304438 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11333/statm: 357725 304438 466 189 0 352074 0
[pid=11333/tid=11335] ppid=11331 vsize=1430900 CPUtime=4725.75 cores=0-7
/proc/11333/task/11335/stat : 11335 (claspmt) R 11331 11333 11307 0 -1 4202560 23607 0 0 0 472539 36 0 0 25 0 8 0 661359120 1465241600 304438 33554432000 4194304 4964800 140736631714016 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11333/tid=11336] ppid=11331 vsize=1430900 CPUtime=4781.79 cores=0-7
/proc/11333/task/11336/stat : 11336 (claspmt) R 11331 11333 11307 0 -1 4202560 48194 0 0 0 478128 51 0 0 25 0 8 0 661359120 1465241600 304438 33554432000 4194304 4964800 140736631714016 18446744073709551615 4660444 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11333/tid=11337] ppid=11331 vsize=1430900 CPUtime=4781.67 cores=0-7
/proc/11333/task/11337/stat : 11337 (claspmt) R 11331 11333 11307 0 -1 4202560 32089 0 0 0 478123 44 0 0 25 0 8 0 661359120 1465241600 304438 33554432000 4194304 4964800 140736631714016 18446744073709551615 4684107 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11333/tid=11338] ppid=11331 vsize=1430900 CPUtime=4781.74 cores=0-7
/proc/11333/task/11338/stat : 11338 (claspmt) R 11331 11333 11307 0 -1 4202560 35143 0 0 0 478124 50 0 0 25 0 8 0 661359120 1465241600 304438 33554432000 4194304 4964800 140736631714016 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11333/tid=11339] ppid=11331 vsize=1430900 CPUtime=4775.86 cores=0-7
/proc/11333/task/11339/stat : 11339 (claspmt) R 11331 11333 11307 0 -1 4202560 31138 0 0 0 477544 42 0 0 25 0 8 0 661359120 1465241600 304438 33554432000 4194304 4964800 140736631714016 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11333/tid=11340] ppid=11331 vsize=1430900 CPUtime=4780.61 cores=0-7
/proc/11333/task/11340/stat : 11340 (claspmt) R 11331 11333 11307 0 -1 4202560 26209 0 0 0 478024 37 0 0 25 0 8 0 661359120 1465241600 304438 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11333/tid=11341] ppid=11331 vsize=1430900 CPUtime=4779.99 cores=0-7
/proc/11333/task/11341/stat : 11341 (claspmt) R 11331 11333 11307 0 -1 4202560 86393 0 0 0 477888 111 0 0 25 0 8 0 661359120 1465241600 304438 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38182.3
Current children cumulated vsize (KiB) 1430900

[startup+4842.3 s]
/proc/loadavg: 8.00 8.01 8.00 9/446 11482
/proc/meminfo: memFree=20365076/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=1430900 CPUtime=38661.4 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 318123 0 0 0 3865718 425 0 0 25 0 8 0 661359084 1465241600 309063 33554432000 4194304 4964800 140736631714016 18446744073709551615 4320909 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11333/statm: 357725 309063 466 189 0 352074 0
[pid=11333/tid=11335] ppid=11331 vsize=1430900 CPUtime=4784.88 cores=0-7
/proc/11333/task/11335/stat : 11335 (claspmt) R 11331 11333 11307 0 -1 4202560 23996 0 0 0 478451 37 0 0 25 0 8 0 661359120 1465241600 309063 33554432000 4194304 4964800 140736631714016 18446744073709551615 4668709 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11333/tid=11336] ppid=11331 vsize=1430900 CPUtime=4841.77 cores=0-7
/proc/11333/task/11336/stat : 11336 (claspmt) R 11331 11333 11307 0 -1 4202560 48925 0 0 0 484126 51 0 0 25 0 8 0 661359120 1465241600 309063 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11333/tid=11337] ppid=11331 vsize=1430900 CPUtime=4841.67 cores=0-7
/proc/11333/task/11337/stat : 11337 (claspmt) R 11331 11333 11307 0 -1 4202560 32089 0 0 0 484123 44 0 0 25 0 8 0 661359120 1465241600 309063 33554432000 4194304 4964800 140736631714016 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11333/tid=11338] ppid=11331 vsize=1430900 CPUtime=4841.74 cores=0-7
/proc/11333/task/11338/stat : 11338 (claspmt) R 11331 11333 11307 0 -1 4202560 35211 0 0 0 484124 50 0 0 25 0 8 0 661359120 1465241600 309063 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11333/tid=11339] ppid=11331 vsize=1430900 CPUtime=4835.86 cores=0-7
/proc/11333/task/11339/stat : 11339 (claspmt) R 11331 11333 11307 0 -1 4202560 31377 0 0 0 483543 43 0 0 25 0 8 0 661359120 1465241600 309063 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11333/tid=11340] ppid=11331 vsize=1430900 CPUtime=4840.58 cores=0-7
/proc/11333/task/11340/stat : 11340 (claspmt) R 11331 11333 11307 0 -1 4202560 27468 0 0 0 484020 38 0 0 25 0 8 0 661359120 1465241600 309063 33554432000 4194304 4964800 140736631714016 18446744073709551615 4668987 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11333/tid=11341] ppid=11331 vsize=1430900 CPUtime=4839.99 cores=0-7
/proc/11333/task/11341/stat : 11341 (claspmt) R 11331 11333 11307 0 -1 4202560 87465 0 0 0 483886 113 0 0 25 0 8 0 661359120 1465241600 309063 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38661.4
Current children cumulated vsize (KiB) 1430900

[startup+4902.3 s]
/proc/loadavg: 8.11 8.04 8.01 9/446 11484
/proc/meminfo: memFree=20359360/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=1432108 CPUtime=39140.6 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 319667 0 0 0 3913627 428 0 0 25 0 8 0 661359084 1466478592 310568 33554432000 4194304 4964800 140736631714016 18446744073709551615 4669350 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11333/statm: 358027 310568 466 189 0 352376 0
[pid=11333/tid=11335] ppid=11331 vsize=1432108 CPUtime=4844.01 cores=0-7
/proc/11333/task/11335/stat : 11335 (claspmt) R 11331 11333 11307 0 -1 4202560 24449 0 0 0 484363 38 0 0 25 0 8 0 661359120 1466478592 310568 33554432000 4194304 4964800 140736631714016 18446744073709551615 4668900 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11333/tid=11336] ppid=11331 vsize=1432108 CPUtime=4901.78 cores=0-7
/proc/11333/task/11336/stat : 11336 (claspmt) R 11331 11333 11307 0 -1 4202560 49598 0 0 0 490126 52 0 0 25 0 8 0 661359120 1466478592 310568 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11333/tid=11337] ppid=11331 vsize=1432108 CPUtime=4901.67 cores=0-7
/proc/11333/task/11337/stat : 11337 (claspmt) R 11331 11333 11307 0 -1 4202560 32260 0 0 0 490122 45 0 0 25 0 8 0 661359120 1466478592 310568 33554432000 4194304 4964800 140736631714016 18446744073709551615 253640557154 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11333/tid=11338] ppid=11331 vsize=1432108 CPUtime=4901.73 cores=0-7
/proc/11333/task/11338/stat : 11338 (claspmt) R 11331 11333 11307 0 -1 4202560 35408 0 0 0 490123 50 0 0 25 0 8 0 661359120 1466478592 310568 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11333/tid=11339] ppid=11331 vsize=1432108 CPUtime=4895.86 cores=0-7
/proc/11333/task/11339/stat : 11339 (claspmt) R 11331 11333 11307 0 -1 4202560 31404 0 0 0 489543 43 0 0 25 0 8 0 661359120 1466478592 310568 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11333/tid=11340] ppid=11331 vsize=1432108 CPUtime=4900.57 cores=0-7
/proc/11333/task/11340/stat : 11340 (claspmt) R 11331 11333 11307 0 -1 4202560 27484 0 0 0 490019 38 0 0 25 0 8 0 661359120 1466478592 310568 33554432000 4194304 4964800 140736631714016 18446744073709551615 4332327 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11333/tid=11341] ppid=11331 vsize=1432108 CPUtime=4899.98 cores=0-7
/proc/11333/task/11341/stat : 11341 (claspmt) R 11331 11333 11307 0 -1 4202560 87465 0 0 0 489885 113 0 0 25 0 8 0 661359120 1466478592 310568 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39140.6
Current children cumulated vsize (KiB) 1432108

[startup+4962.3 s]
/proc/loadavg: 8.04 8.03 8.00 9/446 11485
/proc/meminfo: memFree=20350668/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=1433600 CPUtime=39619.6 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 321816 0 0 0 3961534 430 0 0 25 0 8 0 661359084 1468006400 312681 33554432000 4194304 4964800 140736631714016 18446744073709551615 4321102 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11333/statm: 358400 312681 466 189 0 352749 0
[pid=11333/tid=11335] ppid=11331 vsize=1433600 CPUtime=4903.12 cores=0-7
/proc/11333/task/11335/stat : 11335 (claspmt) R 11331 11333 11307 0 -1 4202560 24449 0 0 0 490274 38 0 0 25 0 8 0 661359120 1468006400 312681 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11333/tid=11336] ppid=11331 vsize=1433600 CPUtime=4961.77 cores=0-7
/proc/11333/task/11336/stat : 11336 (claspmt) R 11331 11333 11307 0 -1 4202560 50536 0 0 0 496124 53 0 0 25 0 8 0 661359120 1468006400 312681 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11333/tid=11337] ppid=11331 vsize=1433600 CPUtime=4961.67 cores=0-7
/proc/11333/task/11337/stat : 11337 (claspmt) R 11331 11333 11307 0 -1 4202560 32260 0 0 0 496122 45 0 0 25 0 8 0 661359120 1468006400 312681 33554432000 4194304 4964800 140736631714016 18446744073709551615 4669476 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11333/tid=11338] ppid=11331 vsize=1433600 CPUtime=4961.73 cores=0-7
/proc/11333/task/11338/stat : 11338 (claspmt) R 11331 11333 11307 0 -1 4202560 35764 0 0 0 496123 50 0 0 25 0 8 0 661359120 1468006400 312681 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11333/tid=11339] ppid=11331 vsize=1433600 CPUtime=4955.86 cores=0-7
/proc/11333/task/11339/stat : 11339 (claspmt) R 11331 11333 11307 0 -1 4202560 31409 0 0 0 495543 43 0 0 25 0 8 0 661359120 1468006400 312681 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11333/tid=11340] ppid=11331 vsize=1433600 CPUtime=4960.56 cores=0-7
/proc/11333/task/11340/stat : 11340 (claspmt) R 11331 11333 11307 0 -1 4202560 27520 0 0 0 496018 38 0 0 25 0 8 0 661359120 1468006400 312681 33554432000 4194304 4964800 140736631714016 18446744073709551615 4361114 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11333/tid=11341] ppid=11331 vsize=1433600 CPUtime=4959.98 cores=0-7
/proc/11333/task/11341/stat : 11341 (claspmt) R 11331 11333 11307 0 -1 4202560 88175 0 0 0 495884 114 0 0 25 0 8 0 661359120 1468006400 312681 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670538 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39619.6
Current children cumulated vsize (KiB) 1433600



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010 s]
/proc/loadavg: 7.97 8.01 7.99 9/446 11486
/proc/meminfo: memFree=20350412/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=1433600 CPUtime=40000.6 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 321829 0 0 0 3999626 431 0 0 25 0 8 0 661359084 1468006400 312694 33554432000 4194304 4964800 140736631714016 18446744073709551615 4362220 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11333/statm: 358400 312694 466 189 0 352749 0
[pid=11333/tid=11335] ppid=11331 vsize=1433600 CPUtime=4950.13 cores=0-7
/proc/11333/task/11335/stat : 11335 (claspmt) R 11331 11333 11307 0 -1 4202560 24449 0 0 0 494975 38 0 0 25 0 8 0 661359120 1468006400 312694 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11333/tid=11336] ppid=11331 vsize=1433600 CPUtime=5009.49 cores=0-7
/proc/11333/task/11336/stat : 11336 (claspmt) R 11331 11333 11307 0 -1 4202560 50536 0 0 0 500896 53 0 0 25 0 8 0 661359120 1468006400 312694 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11333/tid=11337] ppid=11331 vsize=1433600 CPUtime=5009.38 cores=0-7
/proc/11333/task/11337/stat : 11337 (claspmt) R 11331 11333 11307 0 -1 4202560 32260 0 0 0 500893 45 0 0 25 0 8 0 661359120 1468006400 312694 33554432000 4194304 4964800 140736631714016 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11333/tid=11338] ppid=11331 vsize=1433600 CPUtime=5009.44 cores=0-7
/proc/11333/task/11338/stat : 11338 (claspmt) R 11331 11333 11307 0 -1 4202560 35772 0 0 0 500894 50 0 0 25 0 8 0 661359120 1468006400 312694 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670321 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11333/tid=11339] ppid=11331 vsize=1433600 CPUtime=5003.57 cores=0-7
/proc/11333/task/11339/stat : 11339 (claspmt) R 11331 11333 11307 0 -1 4202560 31409 0 0 0 500314 43 0 0 25 0 8 0 661359120 1468006400 312694 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11333/tid=11340] ppid=11331 vsize=1433600 CPUtime=5008.25 cores=0-7
/proc/11333/task/11340/stat : 11340 (claspmt) R 11331 11333 11307 0 -1 4202560 27520 0 0 0 500787 38 0 0 25 0 8 0 661359120 1468006400 312694 33554432000 4194304 4964800 140736631714016 18446744073709551615 4669471 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11333/tid=11341] ppid=11331 vsize=1433600 CPUtime=5007.7 cores=0-7
/proc/11333/task/11341/stat : 11341 (claspmt) R 11331 11333 11307 0 -1 4202560 88175 0 0 0 500656 114 0 0 25 0 8 0 661359120 1468006400 312694 33554432000 4194304 4964800 140736631714016 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 1433600

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.2 s]
/proc/loadavg: 7.97 8.01 7.99 9/446 11486
/proc/meminfo: memFree=20350412/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=1417260 CPUtime=40000.9 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 321839 0 0 0 3999653 432 0 0 25 0 1 0 661359084 1451274240 308378 33554432000 4194304 4964800 140736631714016 18446744073709551615 253640514826 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/11333/statm: 354315 308378 476 189 0 348664 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 1417260

[startup+5010.3 s]
/proc/loadavg: 7.97 8.01 7.99 9/446 11486
/proc/meminfo: memFree=20350412/32951124 swapFree=56449232/67111528
[pid=11333] ppid=11331 vsize=997888 CPUtime=40000.9 cores=0-7
/proc/11333/stat : 11333 (claspmt) R 11331 11333 11307 0 -1 4202496 321839 0 0 0 3999661 434 0 0 25 0 1 0 661359084 1021837312 182798 33554432000 4194304 4964800 140736631714016 18446744073709551615 253640903015 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/11333/statm: 249472 182798 476 189 0 243821 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 997888

Child status: 0
Real time (s): 5010.34
CPU time (s): 40001
CPU user time (s): 39996.6
CPU system time (s): 4.37633
CPU usage (%): 798.37
Max. virtual memory (cumulated for all children) (KiB): 1433600

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39996.6
system time used= 4.37633
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 321840
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 92551
involuntary context switches= 140885

runsolver used 19.787 second user time and 53.1349 second system time

The end

Launcher Data 

Begin job on node106 at 2011-05-16 00:13:07
IDJOB=3380967
IDBENCH=83127
IDSOLVER=1865
FILE ID=node106/3380967-1305497587
RUNJOBID= node106-1305497587-11321
PBS_JOBID= 13325443
Free space on /tmp= 73384 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-bottom18.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3380967-1305497587/watcher-3380967-1305497587 -o /tmp/evaluation-result-3380967-1305497587/solver-3380967-1305497587 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380967-1305497587.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 9ece8f569c93ceee834817ef3350ed8e
RANDOM SEED=1399649380

node106.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.20
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21601416 kB
Buffers:        307116 kB
Cached:        6240252 kB
SwapCached:      27536 kB
Active:        2747140 kB
Inactive:      8342840 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21601416 kB
SwapTotal:    67111528 kB
SwapFree:     56449232 kB
Dirty:            3256 kB
Writeback:           0 kB
AnonPages:     4524328 kB
Mapped:          19476 kB
Slab:           129580 kB
PageTables:      59744 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 123524756 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73384 MiB
End job on node106 at 2011-05-16 01:36:39