Trace number 3381243

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 33276.3 5102.38

General information on the benchmark

NameSAT04/industrial/velev/vliw_unsat_2.0/
9dlx_vliw_at_b_iq9.used-as.sat04-719.cnf
MD5SUM9a43349587a435e7109b46b3aceb156e
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark1144.59
Satisfiable
(Un)Satisfiability was proved
Number of variables482093
Number of clauses9676386
Sum of the clauses size28195028
Maximum clause length226
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 29152157
Number of clauses of size 3120945
Number of clauses of size 410067
Number of clauses of size 528986
Number of clauses of size over 5364230

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...510342/instance-3381243-1305510342.cnf
3.09/3.13	c Solving...
33273.87/5100.01	
33273.87/5100.01	*** INTERRUPTED! ***
33273.87/5100.01	*** Shutting down threads...

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3381243-1305510342/watcher-3381243-1305510342 -o /tmp/evaluation-result-3381243-1305510342/solver-3381243-1305510342 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381243-1305510342.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.53 7.79 7.98 2/294 12561
/proc/meminfo: memFree=21879968/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=82084 CPUtime=0 cores=0-7
/proc/12561/stat : 12561 (claspmt) R 12559 12561 12535 0 -1 4202496 722 0 0 0 0 0 0 0 25 0 1 0 643450722 84054016 653 33554432000 4194304 4964800 140734808896640 18446744073709551615 4686341 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/12561/statm: 20521 654 375 189 0 15395 0

[startup+0.013162 s]
/proc/loadavg: 6.53 7.79 7.98 2/294 12561
/proc/meminfo: memFree=21879968/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=82084 CPUtime=0 cores=0-7
/proc/12561/stat : 12561 (claspmt) R 12559 12561 12535 0 -1 4202496 3100 0 0 0 0 0 0 0 25 0 1 0 643450722 84054016 3031 33554432000 4194304 4964800 140734808896640 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/12561/statm: 20521 3046 376 189 0 15395 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 82084

[startup+0.101139 s]
/proc/loadavg: 6.53 7.79 7.98 2/294 12561
/proc/meminfo: memFree=21879968/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=117760 CPUtime=0.09 cores=0-7
/proc/12561/stat : 12561 (claspmt) R 12559 12561 12535 0 -1 4202496 28050 0 0 0 7 2 0 0 25 0 1 0 643450722 120586240 24215 33554432000 4194304 4964800 140734808896640 18446744073709551615 4581636 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/12561/statm: 29440 24216 388 189 0 24314 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 117760

[startup+0.301103 s]
/proc/loadavg: 6.53 7.79 7.98 2/294 12561
/proc/meminfo: memFree=21879968/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=132300 CPUtime=0.29 cores=0-7
/proc/12561/stat : 12561 (claspmt) R 12559 12561 12535 0 -1 4202496 31800 0 0 0 26 3 0 0 25 0 1 0 643450722 135475200 27807 33554432000 4194304 4964800 140734808896640 18446744073709551615 220815897285 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/12561/statm: 33075 27808 388 189 0 27949 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 132300

[startup+0.701049 s]
/proc/loadavg: 6.53 7.79 7.98 2/294 12561
/proc/meminfo: memFree=21879968/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=159556 CPUtime=0.69 cores=0-7
/proc/12561/stat : 12561 (claspmt) R 12559 12561 12535 0 -1 4202496 38865 0 0 0 64 5 0 0 25 0 1 0 643450722 163385344 34593 33554432000 4194304 4964800 140734808896640 18446744073709551615 220815897285 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/12561/statm: 39889 34594 388 189 0 34763 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 159556

[startup+1.50088 s]
/proc/loadavg: 6.53 7.79 7.98 2/295 12562
/proc/meminfo: memFree=21729952/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=210900 CPUtime=1.49 cores=0-7
/proc/12561/stat : 12561 (claspmt) R 12559 12561 12535 0 -1 4202496 52149 0 0 0 141 8 0 0 25 0 1 0 643450722 215961600 47250 33554432000 4194304 4964800 140734808896640 18446744073709551615 220816236288 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/12561/statm: 52725 47251 388 189 0 47599 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 210900

[startup+3.10059 s]
/proc/loadavg: 6.08 7.67 7.94 2/295 12562
/proc/meminfo: memFree=21660140/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=325032 CPUtime=3.09 cores=0-7
/proc/12561/stat : 12561 (claspmt) R 12559 12561 12535 0 -1 4202496 85906 0 0 0 292 17 0 0 25 0 1 0 643450722 332832768 75618 33554432000 4194304 4964800 140734808896640 18446744073709551615 4629529 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/12561/statm: 81258 75618 401 189 0 76132 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 325032

[startup+6.301 s]
/proc/loadavg: 6.08 7.67 7.94 2/302 12569
/proc/meminfo: memFree=20893208/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=1243180 CPUtime=8.73 cores=0-7
/proc/12561/stat : 12561 (claspmt) S 12559 12561 12535 0 -1 4202496 290714 0 0 0 717 156 0 0 17 0 8 0 643450722 1273016320 248726 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/12561/statm: 310795 248726 436 189 0 305144 0
[pid=12561/tid=12563] ppid=12559 vsize=1243180 CPUtime=0.28 cores=0-7
/proc/12561/task/12563/stat : 12563 (claspmt) S 12559 12561 12535 0 -1 4202560 25040 0 0 0 12 16 0 0 16 0 8 0 643451036 1273016320 248726 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 6 0 0 0
[pid=12561/tid=12564] ppid=12559 vsize=1243180 CPUtime=0.66 cores=0-7
/proc/12561/task/12564/stat : 12564 (claspmt) S 12559 12561 12535 0 -1 4202560 31437 0 0 0 49 17 0 0 18 0 8 0 643451036 1273016320 248726 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 5 0 0 0
[pid=12561/tid=12565] ppid=12559 vsize=1243180 CPUtime=0.59 cores=0-7
/proc/12561/task/12565/stat : 12565 (claspmt) S 12559 12561 12535 0 -1 4202560 31654 0 0 0 38 21 0 0 17 0 8 0 643451036 1273016320 248726 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 0 0 0 0
[pid=12561/tid=12566] ppid=12559 vsize=1243180 CPUtime=0.22 cores=0-7
/proc/12561/task/12566/stat : 12566 (claspmt) S 12559 12561 12535 0 -1 4202560 25207 0 0 0 5 17 0 0 16 0 8 0 643451036 1273016320 248726 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 2 0 0 0
[pid=12561/tid=12567] ppid=12559 vsize=1243180 CPUtime=2.49 cores=0-7
/proc/12561/task/12567/stat : 12567 (claspmt) R 12559 12561 12535 0 -1 4202560 30439 0 0 0 228 21 0 0 20 0 8 0 643451036 1273016320 248726 33554432000 4194304 4964800 140734808896640 18446744073709551615 4670160 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12561/tid=12568] ppid=12559 vsize=1243180 CPUtime=0.65 cores=0-7
/proc/12561/task/12568/stat : 12568 (claspmt) S 12559 12561 12535 0 -1 4202560 30239 0 0 0 48 17 0 0 18 0 8 0 643451036 1273016320 248726 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 3 0 0 0
[pid=12561/tid=12569] ppid=12559 vsize=1243180 CPUtime=0.63 cores=0-7
/proc/12561/task/12569/stat : 12569 (claspmt) S 12559 12561 12535 0 -1 4202560 30716 0 0 0 37 26 0 0 17 0 8 0 643451037 1273016320 248726 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 8.73
Current children cumulated vsize (KiB) 1243180

[startup+12.7008 s]
/proc/loadavg: 5.30 7.45 7.87 2/302 12570
/proc/meminfo: memFree=20894004/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=1243180 CPUtime=15.13 cores=0-7
/proc/12561/stat : 12561 (claspmt) S 12559 12561 12535 0 -1 4202496 290715 0 0 0 1357 156 0 0 17 0 8 0 643450722 1273016320 248727 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/12561/statm: 310795 248727 436 189 0 305144 0
[pid=12561/tid=12563] ppid=12559 vsize=1243180 CPUtime=0.28 cores=0-7
/proc/12561/task/12563/stat : 12563 (claspmt) S 12559 12561 12535 0 -1 4202560 25040 0 0 0 12 16 0 0 16 0 8 0 643451036 1273016320 248727 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 6 0 0 0
[pid=12561/tid=12564] ppid=12559 vsize=1243180 CPUtime=0.66 cores=0-7
/proc/12561/task/12564/stat : 12564 (claspmt) S 12559 12561 12535 0 -1 4202560 31437 0 0 0 49 17 0 0 18 0 8 0 643451036 1273016320 248727 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 5 0 0 0
[pid=12561/tid=12565] ppid=12559 vsize=1243180 CPUtime=0.59 cores=0-7
/proc/12561/task/12565/stat : 12565 (claspmt) S 12559 12561 12535 0 -1 4202560 31654 0 0 0 38 21 0 0 17 0 8 0 643451036 1273016320 248727 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 0 0 0 0
[pid=12561/tid=12566] ppid=12559 vsize=1243180 CPUtime=0.22 cores=0-7
/proc/12561/task/12566/stat : 12566 (claspmt) S 12559 12561 12535 0 -1 4202560 25207 0 0 0 5 17 0 0 16 0 8 0 643451036 1273016320 248727 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 2 0 0 0
[pid=12561/tid=12567] ppid=12559 vsize=1243180 CPUtime=8.89 cores=0-7
/proc/12561/task/12567/stat : 12567 (claspmt) R 12559 12561 12535 0 -1 4202560 30440 0 0 0 868 21 0 0 25 0 8 0 643451036 1273016320 248727 33554432000 4194304 4964800 140734808896640 18446744073709551615 4668552 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12561/tid=12568] ppid=12559 vsize=1243180 CPUtime=0.65 cores=0-7
/proc/12561/task/12568/stat : 12568 (claspmt) S 12559 12561 12535 0 -1 4202560 30239 0 0 0 48 17 0 0 18 0 8 0 643451036 1273016320 248727 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 3 0 0 0
[pid=12561/tid=12569] ppid=12559 vsize=1243180 CPUtime=0.63 cores=0-7
/proc/12561/task/12569/stat : 12569 (claspmt) S 12559 12561 12535 0 -1 4202560 30716 0 0 0 37 26 0 0 17 0 8 0 643451037 1273016320 248727 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 15.13
Current children cumulated vsize (KiB) 1243180

[startup+25.5005 s]
/proc/loadavg: 4.64 7.24 7.79 2/302 12570
/proc/meminfo: memFree=20894924/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=1243180 CPUtime=27.93 cores=0-7
/proc/12561/stat : 12561 (claspmt) S 12559 12561 12535 0 -1 4202496 290715 0 0 0 2637 156 0 0 17 0 8 0 643450722 1273016320 248727 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/12561/statm: 310795 248727 436 189 0 305144 0
[pid=12561/tid=12563] ppid=12559 vsize=1243180 CPUtime=0.28 cores=0-7
/proc/12561/task/12563/stat : 12563 (claspmt) S 12559 12561 12535 0 -1 4202560 25040 0 0 0 12 16 0 0 16 0 8 0 643451036 1273016320 248727 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 6 0 0 0
[pid=12561/tid=12564] ppid=12559 vsize=1243180 CPUtime=0.66 cores=0-7
/proc/12561/task/12564/stat : 12564 (claspmt) S 12559 12561 12535 0 -1 4202560 31437 0 0 0 49 17 0 0 18 0 8 0 643451036 1273016320 248727 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 5 0 0 0
[pid=12561/tid=12565] ppid=12559 vsize=1243180 CPUtime=0.59 cores=0-7
/proc/12561/task/12565/stat : 12565 (claspmt) S 12559 12561 12535 0 -1 4202560 31654 0 0 0 38 21 0 0 17 0 8 0 643451036 1273016320 248727 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 0 0 0 0
[pid=12561/tid=12566] ppid=12559 vsize=1243180 CPUtime=0.22 cores=0-7
/proc/12561/task/12566/stat : 12566 (claspmt) S 12559 12561 12535 0 -1 4202560 25207 0 0 0 5 17 0 0 16 0 8 0 643451036 1273016320 248727 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 2 0 0 0
[pid=12561/tid=12567] ppid=12559 vsize=1243180 CPUtime=21.69 cores=0-7
/proc/12561/task/12567/stat : 12567 (claspmt) R 12559 12561 12535 0 -1 4202560 30440 0 0 0 2148 21 0 0 25 0 8 0 643451036 1273016320 248727 33554432000 4194304 4964800 140734808896640 18446744073709551615 4399339 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12561/tid=12568] ppid=12559 vsize=1243180 CPUtime=0.65 cores=0-7
/proc/12561/task/12568/stat : 12568 (claspmt) S 12559 12561 12535 0 -1 4202560 30239 0 0 0 48 17 0 0 18 0 8 0 643451036 1273016320 248727 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 3 0 0 0
[pid=12561/tid=12569] ppid=12559 vsize=1243180 CPUtime=0.63 cores=0-7
/proc/12561/task/12569/stat : 12569 (claspmt) S 12559 12561 12535 0 -1 4202560 30716 0 0 0 37 26 0 0 17 0 8 0 643451037 1273016320 248727 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 27.93
Current children cumulated vsize (KiB) 1243180

[startup+51.1007 s]
/proc/loadavg: 3.40 6.74 7.61 2/302 12571
/proc/meminfo: memFree=20894872/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=1243180 CPUtime=53.54 cores=0-7
/proc/12561/stat : 12561 (claspmt) S 12559 12561 12535 0 -1 4202496 290775 0 0 0 5198 156 0 0 17 0 8 0 643450722 1273016320 248787 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/12561/statm: 310795 248787 436 189 0 305144 0
[pid=12561/tid=12563] ppid=12559 vsize=1243180 CPUtime=0.28 cores=0-7
/proc/12561/task/12563/stat : 12563 (claspmt) S 12559 12561 12535 0 -1 4202560 25040 0 0 0 12 16 0 0 16 0 8 0 643451036 1273016320 248787 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 6 0 0 0
[pid=12561/tid=12564] ppid=12559 vsize=1243180 CPUtime=0.66 cores=0-7
/proc/12561/task/12564/stat : 12564 (claspmt) S 12559 12561 12535 0 -1 4202560 31437 0 0 0 49 17 0 0 18 0 8 0 643451036 1273016320 248787 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 5 0 0 0
[pid=12561/tid=12565] ppid=12559 vsize=1243180 CPUtime=0.59 cores=0-7
/proc/12561/task/12565/stat : 12565 (claspmt) S 12559 12561 12535 0 -1 4202560 31654 0 0 0 38 21 0 0 17 0 8 0 643451036 1273016320 248787 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 0 0 0 0
[pid=12561/tid=12566] ppid=12559 vsize=1243180 CPUtime=0.22 cores=0-7
/proc/12561/task/12566/stat : 12566 (claspmt) S 12559 12561 12535 0 -1 4202560 25207 0 0 0 5 17 0 0 16 0 8 0 643451036 1273016320 248787 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 2 0 0 0
[pid=12561/tid=12567] ppid=12559 vsize=1243180 CPUtime=47.29 cores=0-7
/proc/12561/task/12567/stat : 12567 (claspmt) R 12559 12561 12535 0 -1 4202560 30500 0 0 0 4708 21 0 0 25 0 8 0 643451036 1273016320 248787 33554432000 4194304 4964800 140734808896640 18446744073709551615 4399625 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12561/tid=12568] ppid=12559 vsize=1243180 CPUtime=0.65 cores=0-7
/proc/12561/task/12568/stat : 12568 (claspmt) S 12559 12561 12535 0 -1 4202560 30239 0 0 0 48 17 0 0 18 0 8 0 643451036 1273016320 248787 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 3 0 0 0
[pid=12561/tid=12569] ppid=12559 vsize=1243180 CPUtime=0.63 cores=0-7
/proc/12561/task/12569/stat : 12569 (claspmt) S 12559 12561 12535 0 -1 4202560 30716 0 0 0 37 26 0 0 17 0 8 0 643451037 1273016320 248787 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 53.54
Current children cumulated vsize (KiB) 1243180

[startup+102.311 s]
/proc/loadavg: 1.95 5.77 7.23 2/302 12572
/proc/meminfo: memFree=20894328/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=1243712 CPUtime=104.75 cores=0-7
/proc/12561/stat : 12561 (claspmt) S 12559 12561 12535 0 -1 4202496 290914 0 0 0 10319 156 0 0 17 0 8 0 643450722 1273561088 248926 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/12561/statm: 310928 248926 436 189 0 305277 0
[pid=12561/tid=12563] ppid=12559 vsize=1243712 CPUtime=0.28 cores=0-7
/proc/12561/task/12563/stat : 12563 (claspmt) S 12559 12561 12535 0 -1 4202560 25040 0 0 0 12 16 0 0 16 0 8 0 643451036 1273561088 248926 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 6 0 0 0
[pid=12561/tid=12564] ppid=12559 vsize=1243712 CPUtime=0.66 cores=0-7
/proc/12561/task/12564/stat : 12564 (claspmt) S 12559 12561 12535 0 -1 4202560 31437 0 0 0 49 17 0 0 18 0 8 0 643451036 1273561088 248926 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 5 0 0 0
[pid=12561/tid=12565] ppid=12559 vsize=1243712 CPUtime=0.59 cores=0-7
/proc/12561/task/12565/stat : 12565 (claspmt) S 12559 12561 12535 0 -1 4202560 31654 0 0 0 38 21 0 0 17 0 8 0 643451036 1273561088 248926 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 0 0 0 0
[pid=12561/tid=12566] ppid=12559 vsize=1243712 CPUtime=0.22 cores=0-7
/proc/12561/task/12566/stat : 12566 (claspmt) S 12559 12561 12535 0 -1 4202560 25207 0 0 0 5 17 0 0 16 0 8 0 643451036 1273561088 248926 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 2 0 0 0
[pid=12561/tid=12567] ppid=12559 vsize=1243712 CPUtime=98.51 cores=0-7
/proc/12561/task/12567/stat : 12567 (claspmt) R 12559 12561 12535 0 -1 4202560 30639 0 0 0 9830 21 0 0 25 0 8 0 643451036 1273561088 248926 33554432000 4194304 4964800 140734808896640 18446744073709551615 4669131 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12561/tid=12568] ppid=12559 vsize=1243712 CPUtime=0.65 cores=0-7
/proc/12561/task/12568/stat : 12568 (claspmt) S 12559 12561 12535 0 -1 4202560 30239 0 0 0 48 17 0 0 18 0 8 0 643451036 1273561088 248926 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 3 0 0 0
[pid=12561/tid=12569] ppid=12559 vsize=1243712 CPUtime=0.63 cores=0-7
/proc/12561/task/12569/stat : 12569 (claspmt) S 12559 12561 12535 0 -1 4202560 30716 0 0 0 37 26 0 0 17 0 8 0 643451037 1273561088 248926 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828052377 0 0 4096 18946 18446604451722304320 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 104.75
Current children cumulated vsize (KiB) 1243712

[startup+162.301 s]
/proc/loadavg: 1.35 4.90 6.83 2/302 12574
/proc/meminfo: memFree=20893440/32951124 swapFree=63485424/67111528

################
# More data... #
################

/proc/12561/stat : 12561 (claspmt) R 12559 12561 12535 0 -1 4202496 2394710 0 0 0 3167448 1971 0 0 25 0 8 0 643450722 9875673088 2352267 33554432000 4194304 4964800 140734808896640 18446744073709551615 4371928 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/12561/statm: 2411053 2352267 460 189 0 2405402 0
[pid=12561/tid=12563] ppid=12559 vsize=9644212 CPUtime=3832.74 cores=0-7
/proc/12561/task/12563/stat : 12563 (claspmt) R 12559 12561 12535 0 -1 4202560 40777 0 0 0 383237 37 0 0 25 0 8 0 643451036 9875673088 2352267 33554432000 4194304 4964800 140734808896640 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=12561/tid=12564] ppid=12559 vsize=9644212 CPUtime=3832.84 cores=0-7
/proc/12561/task/12564/stat : 12564 (claspmt) R 12559 12561 12535 0 -1 4202560 1263693 0 0 0 382352 932 0 0 25 0 8 0 643451036 9875673088 2352267 33554432000 4194304 4964800 140734808896640 18446744073709551615 4313804 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12561/tid=12565] ppid=12559 vsize=9644212 CPUtime=3833.38 cores=0-7
/proc/12561/task/12565/stat : 12565 (claspmt) R 12559 12561 12535 0 -1 4202560 142838 0 0 0 383210 128 0 0 25 0 8 0 643451036 9875673088 2352267 33554432000 4194304 4964800 140734808896640 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12561/tid=12566] ppid=12559 vsize=9644212 CPUtime=3833 cores=0-7
/proc/12561/task/12566/stat : 12566 (claspmt) R 12559 12561 12535 0 -1 4202560 378535 0 0 0 382962 338 0 0 25 0 8 0 643451036 9875673088 2352267 33554432000 4194304 4964800 140734808896640 18446744073709551615 4668753 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=12561/tid=12567] ppid=12559 vsize=9644212 CPUtime=4898.44 cores=0-7
/proc/12561/task/12567/stat : 12567 (claspmt) R 12559 12561 12535 0 -1 4202560 32481 0 0 0 489822 22 0 0 25 0 8 0 643451036 9875673088 2352267 33554432000 4194304 4964800 140734808896640 18446744073709551615 4669413 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12561/tid=12568] ppid=12559 vsize=9644212 CPUtime=3802.52 cores=0-7
/proc/12561/task/12568/stat : 12568 (claspmt) R 12559 12561 12535 0 -1 4202560 146618 0 0 0 380075 177 0 0 25 0 8 0 643451036 9875673088 2352267 33554432000 4194304 4964800 140734808896640 18446744073709551615 4632420 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12561/tid=12569] ppid=12559 vsize=9644212 CPUtime=3833.36 cores=0-7
/proc/12561/task/12569/stat : 12569 (claspmt) R 12559 12561 12535 0 -1 4202560 260107 0 0 0 383071 265 0 0 25 0 8 0 643451037 9875673088 2352267 33554432000 4194304 4964800 140734808896640 18446744073709551615 4365162 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 31694.2
Current children cumulated vsize (KiB) 9644212

[startup+4962.3 s]
/proc/loadavg: 8.05 8.05 7.93 9/302 13195
/proc/meminfo: memFree=12414840/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=9709748 CPUtime=32173.6 cores=0-7
/proc/12561/stat : 12561 (claspmt) R 12559 12561 12535 0 -1 4202496 2407029 0 0 0 3215381 1979 0 0 25 0 8 0 643450722 9942781952 2364586 33554432000 4194304 4964800 140734808896640 18446744073709551615 4669011 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/12561/statm: 2427437 2364586 460 189 0 2421786 0
[pid=12561/tid=12563] ppid=12559 vsize=9709748 CPUtime=3892.74 cores=0-7
/proc/12561/task/12563/stat : 12563 (claspmt) R 12559 12561 12535 0 -1 4202560 40777 0 0 0 389237 37 0 0 25 0 8 0 643451036 9942781952 2364586 33554432000 4194304 4964800 140734808896640 18446744073709551615 4668900 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=12561/tid=12564] ppid=12559 vsize=9709748 CPUtime=3892.84 cores=0-7
/proc/12561/task/12564/stat : 12564 (claspmt) R 12559 12561 12535 0 -1 4202560 1275966 0 0 0 388345 939 0 0 25 0 8 0 643451036 9942781952 2364586 33554432000 4194304 4964800 140734808896640 18446744073709551615 4313736 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12561/tid=12565] ppid=12559 vsize=9709748 CPUtime=3893.38 cores=0-7
/proc/12561/task/12565/stat : 12565 (claspmt) R 12559 12561 12535 0 -1 4202560 142839 0 0 0 389210 128 0 0 25 0 8 0 643451036 9942781952 2364586 33554432000 4194304 4964800 140734808896640 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12561/tid=12566] ppid=12559 vsize=9709748 CPUtime=3893 cores=0-7
/proc/12561/task/12566/stat : 12566 (claspmt) R 12559 12561 12535 0 -1 4202560 378570 0 0 0 388962 338 0 0 25 0 8 0 643451036 9942781952 2364586 33554432000 4194304 4964800 140734808896640 18446744073709551615 4670437 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=12561/tid=12567] ppid=12559 vsize=9709748 CPUtime=4958.44 cores=0-7
/proc/12561/task/12567/stat : 12567 (claspmt) R 12559 12561 12535 0 -1 4202560 32481 0 0 0 495822 22 0 0 25 0 8 0 643451036 9942781952 2364586 33554432000 4194304 4964800 140734808896640 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12561/tid=12568] ppid=12559 vsize=9709748 CPUtime=3862.51 cores=0-7
/proc/12561/task/12568/stat : 12568 (claspmt) R 12559 12561 12535 0 -1 4202560 146626 0 0 0 386074 177 0 0 25 0 8 0 643451036 9942781952 2364586 33554432000 4194304 4964800 140734808896640 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12561/tid=12569] ppid=12559 vsize=9709748 CPUtime=3893.37 cores=0-7
/proc/12561/task/12569/stat : 12569 (claspmt) R 12559 12561 12535 0 -1 4202560 260109 0 0 0 389072 265 0 0 25 0 8 0 643451037 9942781952 2364586 33554432000 4194304 4964800 140734808896640 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 32173.6
Current children cumulated vsize (KiB) 9709748

[startup+5022.3 s]
/proc/loadavg: 8.13 8.07 7.94 9/302 13196
/proc/meminfo: memFree=12375632/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=9709748 CPUtime=32653 cores=0-7
/proc/12561/stat : 12561 (claspmt) R 12559 12561 12535 0 -1 4202496 2417978 0 0 0 3263313 1988 0 0 25 0 8 0 643450722 9942781952 2375535 33554432000 4194304 4964800 140734808896640 18446744073709551615 4370264 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/12561/statm: 2427437 2375535 460 189 0 2421786 0
[pid=12561/tid=12563] ppid=12559 vsize=9709748 CPUtime=3952.75 cores=0-7
/proc/12561/task/12563/stat : 12563 (claspmt) R 12559 12561 12535 0 -1 4202560 40895 0 0 0 395237 38 0 0 25 0 8 0 643451036 9942781952 2375535 33554432000 4194304 4964800 140734808896640 18446744073709551615 4370264 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=12561/tid=12564] ppid=12559 vsize=9709748 CPUtime=3952.85 cores=0-7
/proc/12561/task/12564/stat : 12564 (claspmt) R 12559 12561 12535 0 -1 4202560 1286792 0 0 0 394337 948 0 0 25 0 8 0 643451036 9942781952 2375535 33554432000 4194304 4964800 140734808896640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12561/tid=12565] ppid=12559 vsize=9709748 CPUtime=3953.38 cores=0-7
/proc/12561/task/12565/stat : 12565 (claspmt) R 12559 12561 12535 0 -1 4202560 142840 0 0 0 395210 128 0 0 25 0 8 0 643451036 9942781952 2375535 33554432000 4194304 4964800 140734808896640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12561/tid=12566] ppid=12559 vsize=9709748 CPUtime=3953 cores=0-7
/proc/12561/task/12566/stat : 12566 (claspmt) R 12559 12561 12535 0 -1 4202560 378571 0 0 0 394962 338 0 0 25 0 8 0 643451036 9942781952 2375535 33554432000 4194304 4964800 140734808896640 18446744073709551615 4364587 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=12561/tid=12567] ppid=12559 vsize=9709748 CPUtime=5018.44 cores=0-7
/proc/12561/task/12567/stat : 12567 (claspmt) R 12559 12561 12535 0 -1 4202560 32481 0 0 0 501822 22 0 0 25 0 8 0 643451036 9942781952 2375535 33554432000 4194304 4964800 140734808896640 18446744073709551615 4668987 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12561/tid=12568] ppid=12559 vsize=9709748 CPUtime=3922.5 cores=0-7
/proc/12561/task/12568/stat : 12568 (claspmt) R 12559 12561 12535 0 -1 4202560 146629 0 0 0 392073 177 0 0 25 0 8 0 643451036 9942781952 2375535 33554432000 4194304 4964800 140734808896640 18446744073709551615 4321038 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12561/tid=12569] ppid=12559 vsize=9709748 CPUtime=3953.37 cores=0-7
/proc/12561/task/12569/stat : 12569 (claspmt) R 12559 12561 12535 0 -1 4202560 260109 0 0 0 395072 265 0 0 25 0 8 0 643451037 9942781952 2375535 33554432000 4194304 4964800 140734808896640 18446744073709551615 4651367 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 32653
Current children cumulated vsize (KiB) 9709748

[startup+5082.3 s]
/proc/loadavg: 8.19 8.09 7.95 9/302 13198
/proc/meminfo: memFree=12199288/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=9906356 CPUtime=33132.4 cores=0-7
/proc/12561/stat : 12561 (claspmt) R 12559 12561 12535 0 -1 4202496 2460803 0 0 0 3311220 2020 0 0 25 0 8 0 643450722 10144108544 2418360 33554432000 4194304 4964800 140734808896640 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/12561/statm: 2476589 2418360 460 189 0 2470938 0
[pid=12561/tid=12563] ppid=12559 vsize=9906356 CPUtime=4012.75 cores=0-7
/proc/12561/task/12563/stat : 12563 (claspmt) R 12559 12561 12535 0 -1 4202560 42188 0 0 0 401234 41 0 0 25 0 8 0 643451036 10144108544 2418360 33554432000 4194304 4964800 140734808896640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=12561/tid=12564] ppid=12559 vsize=9906356 CPUtime=4012.85 cores=0-7
/proc/12561/task/12564/stat : 12564 (claspmt) R 12559 12561 12535 0 -1 4202560 1328316 0 0 0 400308 977 0 0 25 0 8 0 643451036 10144108544 2418360 33554432000 4194304 4964800 140734808896640 18446744073709551615 4313801 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12561/tid=12565] ppid=12559 vsize=9906356 CPUtime=4013.38 cores=0-7
/proc/12561/task/12565/stat : 12565 (claspmt) R 12559 12561 12535 0 -1 4202560 142844 0 0 0 401210 128 0 0 25 0 8 0 643451036 10144108544 2418360 33554432000 4194304 4964800 140734808896640 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12561/tid=12566] ppid=12559 vsize=9906356 CPUtime=4013.01 cores=0-7
/proc/12561/task/12566/stat : 12566 (claspmt) R 12559 12561 12535 0 -1 4202560 378571 0 0 0 400963 338 0 0 25 0 8 0 643451036 10144108544 2418360 33554432000 4194304 4964800 140734808896640 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=12561/tid=12567] ppid=12559 vsize=9906356 CPUtime=5078.44 cores=0-7
/proc/12561/task/12567/stat : 12567 (claspmt) R 12559 12561 12535 0 -1 4202560 32481 0 0 0 507822 22 0 0 25 0 8 0 643451036 10144108544 2418360 33554432000 4194304 4964800 140734808896640 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12561/tid=12568] ppid=12559 vsize=9906356 CPUtime=3982.49 cores=0-7
/proc/12561/task/12568/stat : 12568 (claspmt) R 12559 12561 12535 0 -1 4202560 146633 0 0 0 398072 177 0 0 25 0 8 0 643451036 10144108544 2418360 33554432000 4194304 4964800 140734808896640 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12561/tid=12569] ppid=12559 vsize=9906356 CPUtime=4013.37 cores=0-7
/proc/12561/task/12569/stat : 12569 (claspmt) R 12559 12561 12535 0 -1 4202560 260109 0 0 0 401072 265 0 0 25 0 8 0 643451037 10144108544 2418360 33554432000 4194304 4964800 140734808896640 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 33132.4
Current children cumulated vsize (KiB) 9906356



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 8.15 8.08 7.95 9/302 13198
/proc/meminfo: memFree=12193696/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=9906356 CPUtime=33273.9 cores=0-7
/proc/12561/stat : 12561 (claspmt) R 12559 12561 12535 0 -1 4202496 2462218 0 0 0 3325366 2021 0 0 25 0 8 0 643450722 10144108544 2419775 33554432000 4194304 4964800 140734808896640 18446744073709551615 4669017 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/12561/statm: 2476589 2419775 460 189 0 2470938 0
[pid=12561/tid=12563] ppid=12559 vsize=9906356 CPUtime=4030.46 cores=0-7
/proc/12561/task/12563/stat : 12563 (claspmt) R 12559 12561 12535 0 -1 4202560 42265 0 0 0 403005 41 0 0 25 0 8 0 643451036 10144108544 2419775 33554432000 4194304 4964800 140734808896640 18446744073709551615 4670691 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=12561/tid=12564] ppid=12559 vsize=9906356 CPUtime=4030.56 cores=0-7
/proc/12561/task/12564/stat : 12564 (claspmt) R 12559 12561 12535 0 -1 4202560 1329651 0 0 0 402078 978 0 0 25 0 8 0 643451036 10144108544 2419775 33554432000 4194304 4964800 140734808896640 18446744073709551615 4313753 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12561/tid=12565] ppid=12559 vsize=9906356 CPUtime=4031.09 cores=0-7
/proc/12561/task/12565/stat : 12565 (claspmt) R 12559 12561 12535 0 -1 4202560 142847 0 0 0 402981 128 0 0 25 0 8 0 643451036 10144108544 2419775 33554432000 4194304 4964800 140734808896640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12561/tid=12566] ppid=12559 vsize=9906356 CPUtime=4030.72 cores=0-7
/proc/12561/task/12566/stat : 12566 (claspmt) R 12559 12561 12535 0 -1 4202560 378571 0 0 0 402734 338 0 0 25 0 8 0 643451036 10144108544 2419775 33554432000 4194304 4964800 140734808896640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=12561/tid=12567] ppid=12559 vsize=9906356 CPUtime=5096.15 cores=0-7
/proc/12561/task/12567/stat : 12567 (claspmt) R 12559 12561 12535 0 -1 4202560 32481 0 0 0 509593 22 0 0 25 0 8 0 643451036 10144108544 2419775 33554432000 4194304 4964800 140734808896640 18446744073709551615 4669280 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12561/tid=12568] ppid=12559 vsize=9906356 CPUtime=4000.2 cores=0-7
/proc/12561/task/12568/stat : 12568 (claspmt) R 12559 12561 12535 0 -1 4202560 146633 0 0 0 399843 177 0 0 25 0 8 0 643451036 10144108544 2419775 33554432000 4194304 4964800 140734808896640 18446744073709551615 4668998 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12561/tid=12569] ppid=12559 vsize=9906356 CPUtime=4031.08 cores=0-7
/proc/12561/task/12569/stat : 12569 (claspmt) R 12559 12561 12535 0 -1 4202560 260109 0 0 0 402843 265 0 0 25 0 8 0 643451037 10144108544 2419775 33554432000 4194304 4964800 140734808896640 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 33273.9
Current children cumulated vsize (KiB) 9906356

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -12561

Solver just ended. Dumping a history of the last processes samples

[startup+5100.6 s]
/proc/loadavg: 7.58 7.96 7.92 2/297 13199
/proc/meminfo: memFree=12193856/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=9906356 CPUtime=33274.6 cores=0-7
/proc/12561/stat : 12561 (claspmt) S 12559 12561 12535 0 -1 4202496 2462220 0 0 0 3325435 2021 0 0 25 0 2 0 643450722 10144108544 2419777 33554432000 4194304 4964800 140734808896640 18446744073709551615 220828039141 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/12561/statm: 2476589 2419777 462 189 0 2470938 0
[pid=12561/tid=12564] ppid=12559 vsize=9906356 CPUtime=4031.16 cores=0-7
/proc/12561/task/12564/stat : 12564 (claspmt) R 12559 12561 12535 0 -1 4202560 1329651 0 0 0 402138 978 0 0 25 0 2 0 643451036 10144108544 2419777 33554432000 4194304 4964800 140734808896640 18446744073709551615 4313853 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 33274.6
Current children cumulated vsize (KiB) 9906356

[startup+5101.4 s]
/proc/loadavg: 7.58 7.96 7.92 2/297 13199
/proc/meminfo: memFree=12193856/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=9823020 CPUtime=33275.4 cores=0-7
/proc/12561/stat : 12561 (claspmt) R 12559 12561 12535 0 -1 4202496 2462228 0 0 0 3325514 2022 0 0 18 0 1 0 643450722 10058772480 2398951 33554432000 4194304 4964800 140734808896640 18446744073709551615 220815893041 0 0 4096 18946 0 0 0 17 7 0 0 0
/proc/12561/statm: 2455755 2398951 470 189 0 2450104 0
Current children cumulated CPU time (s) 33275.4
Current children cumulated vsize (KiB) 9823020

[startup+5101.81 s]
/proc/loadavg: 7.58 7.96 7.92 2/296 13199
/proc/meminfo: memFree=12276820/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=9800420 CPUtime=33275.8 cores=0-7
/proc/12561/stat : 12561 (claspmt) R 12559 12561 12535 0 -1 4202496 2462228 0 0 0 3325555 2022 0 0 19 0 1 0 643450722 10035630080 2393269 33554432000 4194304 4964800 140734808896640 18446744073709551615 220815885400 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/12561/statm: 2450105 2393269 470 189 0 2444454 0
Current children cumulated CPU time (s) 33275.8
Current children cumulated vsize (KiB) 9800420

[startup+5102.2 s]
/proc/loadavg: 7.58 7.96 7.92 2/296 13199
/proc/meminfo: memFree=12276820/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=0 CPUtime=33276.1 cores=0-7
/proc/12561/stat : 12561 (claspmt) R 12559 12561 12535 0 -1 4203524 2462228 0 0 0 3325575 2040 0 0 19 0 1 0 643450722 0 0 33554432000 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/12561/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 33276.1
Current children cumulated vsize (KiB) 0

[startup+5102.3 s]
/proc/loadavg: 7.58 7.96 7.92 2/296 13199
/proc/meminfo: memFree=12276820/32951124 swapFree=63485424/67111528
[pid=12561] ppid=12559 vsize=0 CPUtime=33276.2 cores=0-7
/proc/12561/stat : 12561 (claspmt) R 12559 12561 12535 0 -1 4203524 2462228 0 0 0 3325575 2050 0 0 19 0 1 0 643450722 0 0 33554432000 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/12561/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 33276.2
Current children cumulated vsize (KiB) 0

Child ended because it received signal 9 (SIGKILL)
Real time (s): 5102.38
CPU time (s): 33276.3
CPU user time (s): 33255.8
CPU system time (s): 20.5819
CPU usage (%): 652.173
Max. virtual memory (cumulated for all children) (KiB): 9906356

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 33255.8
system time used= 20.5819
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2462228
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 153451
involuntary context switches= 110176

runsolver used 12.994 second user time and 37.3103 second system time

The end

Launcher Data 

Begin job on node147 at 2011-05-16 03:45:48
IDJOB=3381243
IDBENCH=85008
IDSOLVER=1865
FILE ID=node147/3381243-1305510342
RUNJOBID= node147-1305510342-12549
PBS_JOBID= 13325522
Free space on /tmp= 72948 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT04/industrial/velev/vliw_unsat_2.0/9dlx_vliw_at_b_iq9.used-as.sat04-719.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3381243-1305510342/watcher-3381243-1305510342 -o /tmp/evaluation-result-3381243-1305510342/solver-3381243-1305510342 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381243-1305510342.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 9a43349587a435e7109b46b3aceb156e
RANDOM SEED=1694261219

node147.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21880248 kB
Buffers:        319632 kB
Cached:        5489284 kB
SwapCached:      13132 kB
Active:        4443988 kB
Inactive:      6364004 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21880248 kB
SwapTotal:    67111528 kB
SwapFree:     63485424 kB
Dirty:          214492 kB
Writeback:           0 kB
AnonPages:     4994540 kB
Mapped:          16708 kB
Slab:           163296 kB
PageTables:      32696 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 52918064 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72980 MiB
End job on node147 at 2011-05-16 05:10:51