Trace number 3380705

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40001 5004.45

General information on the benchmark

NameCRAFTED/rbsat/random/
forced/rbsat-v1150c84314gyes1.cnf
MD5SUMd6097a9814089eeede1d027388c17a02
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark114.827
Satisfiable
(Un)Satisfiability was proved
Number of variables1150
Number of clauses84314
Sum of the clauses size169678
Maximum clause length23
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 284264
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 550

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...659441/instance-3380705-1307659441.cnf
0.19/0.25	c Solving...
40000.61/5004.11	
40000.61/5004.11	*** INTERRUPTED! ***
40000.61/5004.11	*** Shutting down threads...
40000.94/5004.45	s UNKNOWN
40000.94/5004.45	
40000.94/5004.45	c INTERRUPTED   : 1
40000.94/5004.45	c Models        : 0+    
40000.94/5004.45	c Time          : 5004.110s (Solving: 5003.86s 1st Model: 0.00s Unsat: 0.00s)
40000.94/5004.45	c CPU Time      : 40000.650s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3380705-1307659441/watcher-3380705-1307659441 -o /tmp/evaluation-result-3380705-1307659441/solver-3380705-1307659441 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380705-1307659441.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.00 7.69 5.91 2/170 15445
/proc/meminfo: memFree=32196800/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=21480 CPUtime=0 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 673 0 0 0 0 0 0 0 25 0 1 0 956174815 21995520 594 33554432000 4194304 4964800 140734759264704 18446744073709551615 224284925696 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/15445/statm: 5370 594 386 189 0 244 0

[startup+0.034494 s]
/proc/loadavg: 8.00 7.69 5.91 2/170 15445
/proc/meminfo: memFree=32196800/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=26348 CPUtime=0.03 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 2169 0 0 0 3 0 0 0 25 0 1 0 956174815 26980352 1788 33554432000 4194304 4964800 140734759264704 18446744073709551615 4607562 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/15445/statm: 6587 1788 399 189 0 1461 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 26348

[startup+0.100464 s]
/proc/loadavg: 8.00 7.69 5.91 2/170 15445
/proc/meminfo: memFree=32196800/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=26348 CPUtime=0.09 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 2169 0 0 0 9 0 0 0 25 0 1 0 956174815 26980352 1788 33554432000 4194304 4964800 140734759264704 18446744073709551615 4607212 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/15445/statm: 6587 1788 399 189 0 1461 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 26348

[startup+0.300443 s]
/proc/loadavg: 8.00 7.69 5.91 2/170 15445
/proc/meminfo: memFree=32196800/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=451956 CPUtime=0.3 cores=0-7
/proc/15445/stat : 15445 (claspmt) S 15443 15445 15286 0 -1 4202496 3281 0 0 0 30 0 0 0 25 0 8 0 956174815 462802944 2707 33554432000 4194304 4964800 140734759264704 18446744073709551615 224296741785 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/15445/statm: 112989 2707 437 189 0 107338 0
Current children cumulated CPU time (s) 0.3
Current children cumulated vsize (KiB) 451956

[startup+0.700328 s]
/proc/loadavg: 8.00 7.69 5.91 2/170 15445
/proc/meminfo: memFree=32196800/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=452256 CPUtime=3.41 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 7556 0 0 0 339 2 0 0 18 0 8 0 956174815 463110144 6981 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/15445/statm: 113064 6982 464 189 0 107413 0
Current children cumulated CPU time (s) 3.41
Current children cumulated vsize (KiB) 452256

[startup+1.5012 s]
/proc/loadavg: 8.00 7.69 5.91 9/178 15453
/proc/meminfo: memFree=32164396/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=455184 CPUtime=9.82 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 11984 0 0 0 978 4 0 0 19 0 8 0 956174815 466108416 11409 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670323 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/15445/statm: 113796 11409 464 189 0 108145 0
[pid=15445/tid=15447] ppid=15443 vsize=455184 CPUtime=1.18 cores=0-7
/proc/15445/task/15447/stat : 15447 (claspmt) R 15443 15445 15286 0 -1 4202560 1064 0 0 0 118 0 0 0 19 0 8 0 956174840 466108416 11409 33554432000 4194304 4964800 140734759264704 18446744073709551615 224284587041 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15445/tid=15448] ppid=15443 vsize=455184 CPUtime=1.18 cores=0-7
/proc/15445/task/15448/stat : 15448 (claspmt) R 15443 15445 15286 0 -1 4202560 1279 0 0 0 118 0 0 0 19 0 8 0 956174840 466108416 11409 33554432000 4194304 4964800 140734759264704 18446744073709551615 4660390 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15445/tid=15449] ppid=15443 vsize=455184 CPUtime=1.17 cores=0-7
/proc/15445/task/15449/stat : 15449 (claspmt) R 15443 15445 15286 0 -1 4202560 1070 0 0 0 117 0 0 0 19 0 8 0 956174840 466108416 11409 33554432000 4194304 4964800 140734759264704 18446744073709551615 4363327 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=15445/tid=15450] ppid=15443 vsize=455184 CPUtime=1.18 cores=0-7
/proc/15445/task/15450/stat : 15450 (claspmt) R 15443 15445 15286 0 -1 4202560 1233 0 0 0 118 0 0 0 19 0 8 0 956174840 466108416 11409 33554432000 4194304 4964800 140734759264704 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15445/tid=15451] ppid=15443 vsize=455184 CPUtime=1.23 cores=0-7
/proc/15445/task/15451/stat : 15451 (claspmt) R 15443 15445 15286 0 -1 4202560 375 0 0 0 123 0 0 0 19 0 8 0 956174840 466108416 11409 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15445/tid=15452] ppid=15443 vsize=455184 CPUtime=1.18 cores=0-7
/proc/15445/task/15452/stat : 15452 (claspmt) R 15443 15445 15286 0 -1 4202560 1373 0 0 0 118 0 0 0 19 0 8 0 956174840 466108416 11409 33554432000 4194304 4964800 140734759264704 18446744073709551615 4627536 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15445/tid=15453] ppid=15443 vsize=455184 CPUtime=1.18 cores=0-7
/proc/15445/task/15453/stat : 15453 (claspmt) R 15443 15445 15286 0 -1 4202560 2151 0 0 0 118 0 0 0 19 0 8 0 956174840 466108416 11409 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 9.82
Current children cumulated vsize (KiB) 455184

[startup+3.10093 s]
/proc/loadavg: 8.00 7.69 5.91 9/178 15453
/proc/meminfo: memFree=32143812/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=458240 CPUtime=22.6 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 16204 0 0 0 2254 6 0 0 23 0 8 0 956174815 469237760 15629 33554432000 4194304 4964800 140734759264704 18446744073709551615 4627536 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/15445/statm: 114560 15629 466 189 0 108909 0
[pid=15445/tid=15447] ppid=15443 vsize=458240 CPUtime=2.77 cores=0-7
/proc/15445/task/15447/stat : 15447 (claspmt) R 15443 15445 15286 0 -1 4202560 1597 0 0 0 277 0 0 0 22 0 8 0 956174840 469237760 15629 33554432000 4194304 4964800 140734759264704 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15445/tid=15448] ppid=15443 vsize=458240 CPUtime=2.78 cores=0-7
/proc/15445/task/15448/stat : 15448 (claspmt) R 15443 15445 15286 0 -1 4202560 1831 0 0 0 277 1 0 0 23 0 8 0 956174840 469237760 15629 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15445/tid=15449] ppid=15443 vsize=458240 CPUtime=2.77 cores=0-7
/proc/15445/task/15449/stat : 15449 (claspmt) R 15443 15445 15286 0 -1 4202560 1414 0 0 0 277 0 0 0 23 0 8 0 956174840 469237760 15629 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=15445/tid=15450] ppid=15443 vsize=458240 CPUtime=2.78 cores=0-7
/proc/15445/task/15450/stat : 15450 (claspmt) R 15443 15445 15286 0 -1 4202560 1614 0 0 0 278 0 0 0 22 0 8 0 956174840 469237760 15629 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670437 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15445/tid=15451] ppid=15443 vsize=458240 CPUtime=2.83 cores=0-7
/proc/15445/task/15451/stat : 15451 (claspmt) R 15443 15445 15286 0 -1 4202560 449 0 0 0 283 0 0 0 23 0 8 0 956174840 469237760 15629 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15445/tid=15452] ppid=15443 vsize=458240 CPUtime=2.77 cores=0-7
/proc/15445/task/15452/stat : 15452 (claspmt) R 15443 15445 15286 0 -1 4202560 1742 0 0 0 277 0 0 0 22 0 8 0 956174840 469237760 15629 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15445/tid=15453] ppid=15443 vsize=458240 CPUtime=2.79 cores=0-7
/proc/15445/task/15453/stat : 15453 (claspmt) R 15443 15445 15286 0 -1 4202560 3651 0 0 0 278 1 0 0 23 0 8 0 956174840 469237760 15629 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 22.6
Current children cumulated vsize (KiB) 458240

[startup+6.3004 s]
/proc/loadavg: 8.00 7.70 5.92 9/178 15453
/proc/meminfo: memFree=32112448/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=460220 CPUtime=48.17 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 23610 0 0 0 4807 10 0 0 25 0 8 0 956174815 471265280 23035 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/15445/statm: 115055 23036 466 189 0 109404 0
[pid=15445/tid=15447] ppid=15443 vsize=460220 CPUtime=5.97 cores=0-7
/proc/15445/task/15447/stat : 15447 (claspmt) R 15443 15445 15286 0 -1 4202560 2363 0 0 0 596 1 0 0 25 0 8 0 956174840 471265280 23036 33554432000 4194304 4964800 140734759264704 18446744073709551615 4627257 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15445/tid=15448] ppid=15443 vsize=460220 CPUtime=5.98 cores=0-7
/proc/15445/task/15448/stat : 15448 (claspmt) R 15443 15445 15286 0 -1 4202560 2696 0 0 0 597 1 0 0 25 0 8 0 956174840 471265280 23036 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15445/tid=15449] ppid=15443 vsize=460220 CPUtime=5.97 cores=0-7
/proc/15445/task/15449/stat : 15449 (claspmt) R 15443 15445 15286 0 -1 4202560 1921 0 0 0 597 0 0 0 25 0 8 0 956174840 471265280 23036 33554432000 4194304 4964800 140734759264704 18446744073709551615 4627964 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=15445/tid=15450] ppid=15443 vsize=460220 CPUtime=5.98 cores=0-7
/proc/15445/task/15450/stat : 15450 (claspmt) R 15443 15445 15286 0 -1 4202560 2636 0 0 0 597 1 0 0 25 0 8 0 956174840 471265280 23036 33554432000 4194304 4964800 140734759264704 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15445/tid=15451] ppid=15443 vsize=460220 CPUtime=6.03 cores=0-7
/proc/15445/task/15451/stat : 15451 (claspmt) R 15443 15445 15286 0 -1 4202560 1459 0 0 0 603 0 0 0 25 0 8 0 956174840 471265280 23036 33554432000 4194304 4964800 140734759264704 18446744073709551615 4653198 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15445/tid=15452] ppid=15443 vsize=460220 CPUtime=5.95 cores=0-7
/proc/15445/task/15452/stat : 15452 (claspmt) R 15443 15445 15286 0 -1 4202560 2114 0 0 0 595 0 0 0 25 0 8 0 956174840 471265280 23036 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15445/tid=15453] ppid=15443 vsize=460220 CPUtime=5.99 cores=0-7
/proc/15445/task/15453/stat : 15453 (claspmt) R 15443 15445 15286 0 -1 4202560 5648 0 0 0 597 2 0 0 25 0 8 0 956174840 471265280 23036 33554432000 4194304 4964800 140734759264704 18446744073709551615 224284587041 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 48.17
Current children cumulated vsize (KiB) 460220

[startup+12.7003 s]
/proc/loadavg: 8.00 7.70 5.93 9/178 15453
/proc/meminfo: memFree=32091776/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=464756 CPUtime=99.33 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 27859 0 0 0 9920 13 0 0 25 0 8 0 956174815 475910144 27284 33554432000 4194304 4964800 140734759264704 18446744073709551615 4627823 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/15445/statm: 116189 27284 466 189 0 110538 0
[pid=15445/tid=15447] ppid=15443 vsize=464756 CPUtime=12.38 cores=0-7
/proc/15445/task/15447/stat : 15447 (claspmt) R 15443 15445 15286 0 -1 4202560 2842 0 0 0 1236 2 0 0 25 0 8 0 956174840 475910144 27284 33554432000 4194304 4964800 140734759264704 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15445/tid=15448] ppid=15443 vsize=464756 CPUtime=12.37 cores=0-7
/proc/15445/task/15448/stat : 15448 (claspmt) R 15443 15445 15286 0 -1 4202560 3218 0 0 0 1236 1 0 0 25 0 8 0 956174840 475910144 27284 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15445/tid=15449] ppid=15443 vsize=464756 CPUtime=12.36 cores=0-7
/proc/15445/task/15449/stat : 15449 (claspmt) R 15443 15445 15286 0 -1 4202560 2002 0 0 0 1236 0 0 0 25 0 8 0 956174840 475910144 27284 33554432000 4194304 4964800 140734759264704 18446744073709551615 4627536 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=15445/tid=15450] ppid=15443 vsize=464756 CPUtime=12.38 cores=0-7
/proc/15445/task/15450/stat : 15450 (claspmt) R 15443 15445 15286 0 -1 4202560 2915 0 0 0 1237 1 0 0 25 0 8 0 956174840 475910144 27284 33554432000 4194304 4964800 140734759264704 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15445/tid=15451] ppid=15443 vsize=464756 CPUtime=12.43 cores=0-7
/proc/15445/task/15451/stat : 15451 (claspmt) R 15443 15445 15286 0 -1 4202560 1698 0 0 0 1243 0 0 0 25 0 8 0 956174840 475910144 27284 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15445/tid=15452] ppid=15443 vsize=464756 CPUtime=12.32 cores=0-7
/proc/15445/task/15452/stat : 15452 (claspmt) R 15443 15445 15286 0 -1 4202560 2343 0 0 0 1231 1 0 0 25 0 8 0 956174840 475910144 27284 33554432000 4194304 4964800 140734759264704 18446744073709551615 4320848 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15445/tid=15453] ppid=15443 vsize=464756 CPUtime=12.38 cores=0-7
/proc/15445/task/15453/stat : 15453 (claspmt) R 15443 15445 15286 0 -1 4202560 7643 0 0 0 1235 3 0 0 25 0 8 0 956174840 475910144 27284 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 99.33
Current children cumulated vsize (KiB) 464756

[startup+25.5012 s]
/proc/loadavg: 8.00 7.72 5.97 9/178 15455
/proc/meminfo: memFree=32066140/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=470644 CPUtime=201.66 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 34773 0 0 0 20147 19 0 0 25 0 8 0 956174815 481939456 34198 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670310 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/15445/statm: 117661 34198 466 189 0 112010 0
[pid=15445/tid=15447] ppid=15443 vsize=470644 CPUtime=25.17 cores=0-7
/proc/15445/task/15447/stat : 15447 (claspmt) R 15443 15445 15286 0 -1 4202560 4049 0 0 0 2514 3 0 0 25 0 8 0 956174840 481939456 34198 33554432000 4194304 4964800 140734759264704 18446744073709551615 4320864 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15445/tid=15448] ppid=15443 vsize=470644 CPUtime=25.18 cores=0-7
/proc/15445/task/15448/stat : 15448 (claspmt) R 15443 15445 15286 0 -1 4202560 3848 0 0 0 2516 2 0 0 25 0 8 0 956174840 481939456 34198 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15445/tid=15449] ppid=15443 vsize=470644 CPUtime=25.17 cores=0-7
/proc/15445/task/15449/stat : 15449 (claspmt) R 15443 15445 15286 0 -1 4202560 2490 0 0 0 2516 1 0 0 25 0 8 0 956174840 481939456 34198 33554432000 4194304 4964800 140734759264704 18446744073709551615 4627362 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=15445/tid=15450] ppid=15443 vsize=470644 CPUtime=25.18 cores=0-7
/proc/15445/task/15450/stat : 15450 (claspmt) R 15443 15445 15286 0 -1 4202560 3386 0 0 0 2517 1 0 0 25 0 8 0 956174840 481939456 34198 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15445/tid=15451] ppid=15443 vsize=470644 CPUtime=25.23 cores=0-7
/proc/15445/task/15451/stat : 15451 (claspmt) R 15443 15445 15286 0 -1 4202560 2714 0 0 0 2522 1 0 0 25 0 8 0 956174840 481939456 34198 33554432000 4194304 4964800 140734759264704 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15445/tid=15452] ppid=15443 vsize=470644 CPUtime=25.04 cores=0-7
/proc/15445/task/15452/stat : 15452 (claspmt) R 15443 15445 15286 0 -1 4202560 2904 0 0 0 2503 1 0 0 18 0 8 0 956174840 481939456 34199 33554432000 4194304 4964800 140734759264704 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15445/tid=15453] ppid=15443 vsize=470644 CPUtime=25.19 cores=0-7
/proc/15445/task/15453/stat : 15453 (claspmt) R 15443 15445 15286 0 -1 4202560 9809 0 0 0 2515 4 0 0 25 0 8 0 956174840 481939456 34199 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 201.66
Current children cumulated vsize (KiB) 470644

[startup+51.1009 s]
/proc/loadavg: 8.07 7.76 6.03 9/178 15455
/proc/meminfo: memFree=32002764/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=476824 CPUtime=406.29 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 50253 0 0 0 40596 33 0 0 25 0 8 0 956174815 488267776 49653 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/15445/statm: 119206 49653 466 189 0 113555 0
[pid=15445/tid=15447] ppid=15443 vsize=476824 CPUtime=50.76 cores=0-7
/proc/15445/task/15447/stat : 15447 (claspmt) R 15443 15445 15286 0 -1 4202560 6151 0 0 0 5071 5 0 0 25 0 8 0 956174840 488267776 49653 33554432000 4194304 4964800 140734759264704 18446744073709551615 4320921 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15445/tid=15448] ppid=15443 vsize=476824 CPUtime=50.77 cores=0-7
/proc/15445/task/15448/stat : 15448 (claspmt) R 15443 15445 15286 0 -1 4202560 5475 0 0 0 5074 3 0 0 25 0 8 0 956174840 488267776 49653 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670349 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15445/tid=15449] ppid=15443 vsize=476824 CPUtime=50.76 cores=0-7
/proc/15445/task/15449/stat : 15449 (claspmt) R 15443 15445 15286 0 -1 4202560 3722 0 0 0 5074 2 0 0 25 0 8 0 956174840 488267776 49653 33554432000 4194304 4964800 140734759264704 18446744073709551615 4668900 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=15445/tid=15450] ppid=15443 vsize=476824 CPUtime=50.78 cores=0-7
/proc/15445/task/15450/stat : 15450 (claspmt) R 15443 15445 15286 0 -1 4202560 4887 0 0 0 5075 3 0 0 25 0 8 0 956174840 488267776 49653 33554432000 4194304 4964800 140734759264704 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15445/tid=15451] ppid=15443 vsize=476824 CPUtime=50.83 cores=0-7
/proc/15445/task/15451/stat : 15451 (claspmt) R 15443 15445 15286 0 -1 4202560 4707 0 0 0 5081 2 0 0 25 0 8 0 956174840 488267776 49653 33554432000 4194304 4964800 140734759264704 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15445/tid=15452] ppid=15443 vsize=476824 CPUtime=50.5 cores=0-7
/proc/15445/task/15452/stat : 15452 (claspmt) R 15443 15445 15286 0 -1 4202560 3933 0 0 0 5048 2 0 0 25 0 8 0 956174840 488267776 49653 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15445/tid=15453] ppid=15443 vsize=476824 CPUtime=50.78 cores=0-7
/proc/15445/task/15453/stat : 15453 (claspmt) R 15443 15445 15286 0 -1 4202560 14751 0 0 0 5070 8 0 0 25 0 8 0 956174840 488267776 49653 33554432000 4194304 4964800 140734759264704 18446744073709551615 4627293 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 406.29

################
# More data... #
################

/proc/15445/task/15452/stat : 15452 (claspmt) R 15443 15445 15286 0 -1 4202560 18123 0 0 0 465630 33 0 0 25 0 8 0 956174840 1420611584 247917 33554432000 4194304 4964800 140734759264704 18446744073709551615 4321088 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15445/tid=15453] ppid=15443 vsize=1387316 CPUtime=4653.13 cores=0-7
/proc/15445/task/15453/stat : 15453 (claspmt) R 15443 15445 15286 0 -1 4202560 112670 0 0 0 465173 140 0 0 25 0 8 0 956174840 1420611584 247917 33554432000 4194304 4964800 140734759264704 18446744073709551615 4365674 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 37268.1
Current children cumulated vsize (KiB) 1387316

[startup+4722.31 s]
/proc/loadavg: 8.07 8.04 8.00 9/178 15600
/proc/meminfo: memFree=31180824/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=1387316 CPUtime=37747.8 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 264957 0 0 0 3774463 320 0 0 25 0 8 0 956174815 1420611584 253821 33554432000 4194304 4964800 140734759264704 18446744073709551615 4320865 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/15445/statm: 346829 253821 466 189 0 341178 0
[pid=15445/tid=15447] ppid=15443 vsize=1387316 CPUtime=4709.87 cores=0-7
/proc/15445/task/15447/stat : 15447 (claspmt) R 15443 15445 15286 0 -1 4202560 24890 0 0 0 470961 26 0 0 25 0 8 0 956174840 1420611584 253821 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=15445/tid=15448] ppid=15443 vsize=1387316 CPUtime=4721.93 cores=0-7
/proc/15445/task/15448/stat : 15448 (claspmt) R 15443 15445 15286 0 -1 4202560 27076 0 0 0 472176 17 0 0 25 0 8 0 956174840 1420611584 253821 33554432000 4194304 4964800 140734759264704 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15445/tid=15449] ppid=15443 vsize=1387316 CPUtime=4721.21 cores=0-7
/proc/15445/task/15449/stat : 15449 (claspmt) R 15443 15445 15286 0 -1 4202560 19912 0 0 0 472097 24 0 0 25 0 8 0 956174840 1420611584 253821 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=15445/tid=15450] ppid=15443 vsize=1387316 CPUtime=4721.75 cores=0-7
/proc/15445/task/15450/stat : 15450 (claspmt) R 15443 15445 15286 0 -1 4202560 16633 0 0 0 472158 17 0 0 25 0 8 0 956174840 1420611584 253821 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15445/tid=15451] ppid=15443 vsize=1387316 CPUtime=4721.43 cores=0-7
/proc/15445/task/15451/stat : 15451 (claspmt) R 15443 15445 15286 0 -1 4202560 18199 0 0 0 472124 19 0 0 25 0 8 0 956174840 1420611584 253821 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670321 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15445/tid=15452] ppid=15443 vsize=1387316 CPUtime=4716.64 cores=0-7
/proc/15445/task/15452/stat : 15452 (claspmt) R 15443 15445 15286 0 -1 4202560 18200 0 0 0 471631 33 0 0 25 0 8 0 956174840 1420611584 253821 33554432000 4194304 4964800 140734759264704 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15445/tid=15453] ppid=15443 vsize=1387316 CPUtime=4712.81 cores=0-7
/proc/15445/task/15453/stat : 15453 (claspmt) R 15443 15445 15286 0 -1 4202560 117255 0 0 0 471132 149 0 0 25 0 8 0 956174840 1420611584 253821 33554432000 4194304 4964800 140734759264704 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 37747.8
Current children cumulated vsize (KiB) 1387316

[startup+4782.3 s]
/proc/loadavg: 8.07 8.05 8.00 9/178 15602
/proc/meminfo: memFree=31179172/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=1387316 CPUtime=38227.4 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 265442 0 0 0 3822420 322 0 0 25 0 8 0 956174815 1420611584 254089 33554432000 4194304 4964800 140734759264704 18446744073709551615 4653244 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/15445/statm: 346829 254089 466 189 0 341178 0
[pid=15445/tid=15447] ppid=15443 vsize=1387316 CPUtime=4769.85 cores=0-7
/proc/15445/task/15447/stat : 15447 (claspmt) R 15443 15445 15286 0 -1 4202560 25077 0 0 0 476959 26 0 0 25 0 8 0 956174840 1420611584 254089 33554432000 4194304 4964800 140734759264704 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=15445/tid=15448] ppid=15443 vsize=1387316 CPUtime=4781.92 cores=0-7
/proc/15445/task/15448/stat : 15448 (claspmt) R 15443 15445 15286 0 -1 4202560 27178 0 0 0 478174 18 0 0 25 0 8 0 956174840 1420611584 254089 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15445/tid=15449] ppid=15443 vsize=1387316 CPUtime=4781.21 cores=0-7
/proc/15445/task/15449/stat : 15449 (claspmt) R 15443 15445 15286 0 -1 4202560 20016 0 0 0 478096 25 0 0 25 0 8 0 956174840 1420611584 254089 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670264 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=15445/tid=15450] ppid=15443 vsize=1387316 CPUtime=4781.74 cores=0-7
/proc/15445/task/15450/stat : 15450 (claspmt) R 15443 15445 15286 0 -1 4202560 16674 0 0 0 478157 17 0 0 25 0 8 0 956174840 1420611584 254089 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15445/tid=15451] ppid=15443 vsize=1387316 CPUtime=4781.42 cores=0-7
/proc/15445/task/15451/stat : 15451 (claspmt) R 15443 15445 15286 0 -1 4202560 18205 0 0 0 478123 19 0 0 25 0 8 0 956174840 1420611584 254089 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15445/tid=15452] ppid=15443 vsize=1387316 CPUtime=4776.62 cores=0-7
/proc/15445/task/15452/stat : 15452 (claspmt) R 15443 15445 15286 0 -1 4202560 18200 0 0 0 477629 33 0 0 25 0 8 0 956174840 1420611584 254089 33554432000 4194304 4964800 140734759264704 18446744073709551615 4320989 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15445/tid=15453] ppid=15443 vsize=1387316 CPUtime=4772.48 cores=0-7
/proc/15445/task/15453/stat : 15453 (claspmt) R 15443 15445 15286 0 -1 4202560 117298 0 0 0 477099 149 0 0 25 0 8 0 956174840 1420611584 254089 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38227.4
Current children cumulated vsize (KiB) 1387316

[startup+4842.3 s]
/proc/loadavg: 8.06 8.05 8.00 9/178 15604
/proc/meminfo: memFree=31178504/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=1387316 CPUtime=38707.1 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 265812 0 0 0 3870386 323 0 0 25 0 8 0 956174815 1420611584 254341 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/15445/statm: 346829 254341 466 189 0 341178 0
[pid=15445/tid=15447] ppid=15443 vsize=1387316 CPUtime=4829.85 cores=0-7
/proc/15445/task/15447/stat : 15447 (claspmt) R 15443 15445 15286 0 -1 4202560 25077 0 0 0 482959 26 0 0 25 0 8 0 956174840 1420611584 254341 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670321 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=15445/tid=15448] ppid=15443 vsize=1387316 CPUtime=4841.92 cores=0-7
/proc/15445/task/15448/stat : 15448 (claspmt) R 15443 15445 15286 0 -1 4202560 27495 0 0 0 484174 18 0 0 25 0 8 0 956174840 1420611584 254341 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15445/tid=15449] ppid=15443 vsize=1387316 CPUtime=4841.2 cores=0-7
/proc/15445/task/15449/stat : 15449 (claspmt) R 15443 15445 15286 0 -1 4202560 20039 0 0 0 484095 25 0 0 25 0 8 0 956174840 1420611584 254341 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=15445/tid=15450] ppid=15443 vsize=1387316 CPUtime=4841.74 cores=0-7
/proc/15445/task/15450/stat : 15450 (claspmt) R 15443 15445 15286 0 -1 4202560 16679 0 0 0 484157 17 0 0 25 0 8 0 956174840 1420611584 254341 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15445/tid=15451] ppid=15443 vsize=1387316 CPUtime=4841.42 cores=0-7
/proc/15445/task/15451/stat : 15451 (claspmt) R 15443 15445 15286 0 -1 4202560 18230 0 0 0 484123 19 0 0 25 0 8 0 956174840 1420611584 254341 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15445/tid=15452] ppid=15443 vsize=1387316 CPUtime=4836.62 cores=0-7
/proc/15445/task/15452/stat : 15452 (claspmt) R 15443 15445 15286 0 -1 4202560 18200 0 0 0 483629 33 0 0 25 0 8 0 956174840 1420611584 254341 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15445/tid=15453] ppid=15443 vsize=1387316 CPUtime=4832.15 cores=0-7
/proc/15445/task/15453/stat : 15453 (claspmt) R 15443 15445 15286 0 -1 4202560 117298 0 0 0 483066 149 0 0 25 0 8 0 956174840 1420611584 254341 33554432000 4194304 4964800 140734759264704 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38707.1
Current children cumulated vsize (KiB) 1387316

[startup+4902.3 s]
/proc/loadavg: 8.06 8.05 8.00 9/178 15606
/proc/meminfo: memFree=31176472/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=1388064 CPUtime=39186.7 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 266426 0 0 0 3918351 323 0 0 25 0 8 0 956174815 1421377536 254955 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/15445/statm: 347016 254955 466 189 0 341365 0
[pid=15445/tid=15447] ppid=15443 vsize=1388064 CPUtime=4889.84 cores=0-7
/proc/15445/task/15447/stat : 15447 (claspmt) R 15443 15445 15286 0 -1 4202560 25265 0 0 0 488958 26 0 0 25 0 8 0 956174840 1421377536 254955 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=15445/tid=15448] ppid=15443 vsize=1388064 CPUtime=4901.92 cores=0-7
/proc/15445/task/15448/stat : 15448 (claspmt) R 15443 15445 15286 0 -1 4202560 27508 0 0 0 490174 18 0 0 25 0 8 0 956174840 1421377536 254955 33554432000 4194304 4964800 140734759264704 18446744073709551615 4683952 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15445/tid=15449] ppid=15443 vsize=1388064 CPUtime=4901.19 cores=0-7
/proc/15445/task/15449/stat : 15449 (claspmt) R 15443 15445 15286 0 -1 4202560 20076 0 0 0 490094 25 0 0 25 0 8 0 956174840 1421377536 254955 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=15445/tid=15450] ppid=15443 vsize=1388064 CPUtime=4901.74 cores=0-7
/proc/15445/task/15450/stat : 15450 (claspmt) R 15443 15445 15286 0 -1 4202560 16939 0 0 0 490157 17 0 0 25 0 8 0 956174840 1421377536 254955 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15445/tid=15451] ppid=15443 vsize=1388064 CPUtime=4901.41 cores=0-7
/proc/15445/task/15451/stat : 15451 (claspmt) R 15443 15445 15286 0 -1 4202560 18242 0 0 0 490122 19 0 0 25 0 8 0 956174840 1421377536 254955 33554432000 4194304 4964800 140734759264704 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15445/tid=15452] ppid=15443 vsize=1388064 CPUtime=4896.61 cores=0-7
/proc/15445/task/15452/stat : 15452 (claspmt) R 15443 15445 15286 0 -1 4202560 18303 0 0 0 489628 33 0 0 25 0 8 0 956174840 1421377536 254955 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15445/tid=15453] ppid=15443 vsize=1388064 CPUtime=4891.82 cores=0-7
/proc/15445/task/15453/stat : 15453 (claspmt) R 15443 15445 15286 0 -1 4202560 117299 0 0 0 489033 149 0 0 25 0 8 0 956174840 1421377536 254955 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39186.7
Current children cumulated vsize (KiB) 1388064

[startup+4962.3 s]
/proc/loadavg: 8.06 8.06 8.00 9/178 15606
/proc/meminfo: memFree=31175804/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=1388064 CPUtime=39666.4 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 266584 0 0 0 3966316 324 0 0 25 0 8 0 956174815 1421377536 255023 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670349 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/15445/statm: 347016 255023 466 189 0 341365 0
[pid=15445/tid=15447] ppid=15443 vsize=1388064 CPUtime=4949.84 cores=0-7
/proc/15445/task/15447/stat : 15447 (claspmt) R 15443 15445 15286 0 -1 4202560 25270 0 0 0 494958 26 0 0 25 0 8 0 956174840 1421377536 255023 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=15445/tid=15448] ppid=15443 vsize=1388064 CPUtime=4961.92 cores=0-7
/proc/15445/task/15448/stat : 15448 (claspmt) R 15443 15445 15286 0 -1 4202560 27590 0 0 0 496174 18 0 0 25 0 8 0 956174840 1421377536 255023 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15445/tid=15449] ppid=15443 vsize=1388064 CPUtime=4961.19 cores=0-7
/proc/15445/task/15449/stat : 15449 (claspmt) R 15443 15445 15286 0 -1 4202560 20110 0 0 0 496094 25 0 0 25 0 8 0 956174840 1421377536 255023 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=15445/tid=15450] ppid=15443 vsize=1388064 CPUtime=4961.74 cores=0-7
/proc/15445/task/15450/stat : 15450 (claspmt) R 15443 15445 15286 0 -1 4202560 16976 0 0 0 496157 17 0 0 25 0 8 0 956174840 1421377536 255023 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15445/tid=15451] ppid=15443 vsize=1388064 CPUtime=4961.41 cores=0-7
/proc/15445/task/15451/stat : 15451 (claspmt) R 15443 15445 15286 0 -1 4202560 18242 0 0 0 496122 19 0 0 25 0 8 0 956174840 1421377536 255023 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15445/tid=15452] ppid=15443 vsize=1388064 CPUtime=4956.61 cores=0-7
/proc/15445/task/15452/stat : 15452 (claspmt) R 15443 15445 15286 0 -1 4202560 18303 0 0 0 495628 33 0 0 25 0 8 0 956174840 1421377536 255023 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15445/tid=15453] ppid=15443 vsize=1388064 CPUtime=4951.49 cores=0-7
/proc/15445/task/15453/stat : 15453 (claspmt) R 15443 15445 15286 0 -1 4202560 117299 0 0 0 495000 149 0 0 25 0 8 0 956174840 1421377536 255023 33554432000 4194304 4964800 140734759264704 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39666.4
Current children cumulated vsize (KiB) 1388064



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5004.1 s]
/proc/loadavg: 8.07 8.06 8.00 9/178 15608
/proc/meminfo: memFree=31174896/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=1388064 CPUtime=40000.6 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 266863 0 0 0 3999737 324 0 0 25 0 8 0 956174815 1421377536 255302 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670361 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/15445/statm: 347016 255302 466 189 0 341365 0
[pid=15445/tid=15447] ppid=15443 vsize=1388064 CPUtime=4991.65 cores=0-7
/proc/15445/task/15447/stat : 15447 (claspmt) R 15443 15445 15286 0 -1 4202560 25273 0 0 0 499139 26 0 0 25 0 8 0 956174840 1421377536 255302 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=15445/tid=15448] ppid=15443 vsize=1388064 CPUtime=5003.73 cores=0-7
/proc/15445/task/15448/stat : 15448 (claspmt) R 15443 15445 15286 0 -1 4202560 27803 0 0 0 500355 18 0 0 25 0 8 0 956174840 1421377536 255302 33554432000 4194304 4964800 140734759264704 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15445/tid=15449] ppid=15443 vsize=1388064 CPUtime=5002.99 cores=0-7
/proc/15445/task/15449/stat : 15449 (claspmt) R 15443 15445 15286 0 -1 4202560 20114 0 0 0 500274 25 0 0 25 0 8 0 956174840 1421377536 255302 33554432000 4194304 4964800 140734759264704 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=15445/tid=15450] ppid=15443 vsize=1388064 CPUtime=5003.55 cores=0-7
/proc/15445/task/15450/stat : 15450 (claspmt) R 15443 15445 15286 0 -1 4202560 16986 0 0 0 500338 17 0 0 25 0 8 0 956174840 1421377536 255302 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15445/tid=15451] ppid=15443 vsize=1388064 CPUtime=5003.22 cores=0-7
/proc/15445/task/15451/stat : 15451 (claspmt) R 15443 15445 15286 0 -1 4202560 18242 0 0 0 500303 19 0 0 25 0 8 0 956174840 1421377536 255302 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15445/tid=15452] ppid=15443 vsize=1388064 CPUtime=4998.42 cores=0-7
/proc/15445/task/15452/stat : 15452 (claspmt) R 15443 15445 15286 0 -1 4202560 18352 0 0 0 499809 33 0 0 25 0 8 0 956174840 1421377536 255302 33554432000 4194304 4964800 140734759264704 18446744073709551615 4627290 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15445/tid=15453] ppid=15443 vsize=1388064 CPUtime=4993.07 cores=0-7
/proc/15445/task/15453/stat : 15453 (claspmt) R 15443 15445 15286 0 -1 4202560 117299 0 0 0 499158 149 0 0 25 0 8 0 956174840 1421377536 255302 33554432000 4194304 4964800 140734759264704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 1388064

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5004.21 s]
/proc/loadavg: 8.07 8.06 8.00 9/178 15608
/proc/meminfo: memFree=31174896/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=1385488 CPUtime=40000.8 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 266873 0 0 0 3999751 324 0 0 25 0 1 0 956174815 1418739712 254748 33554432000 4194304 4964800 140734759264704 18446744073709551615 224284581067 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/15445/statm: 346372 254748 476 189 0 340721 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 1385488

[startup+5004.4 s]
/proc/loadavg: 8.07 8.06 8.00 9/178 15608
/proc/meminfo: memFree=31174896/32951124 swapFree=67049456/67111528
[pid=15445] ppid=15443 vsize=1378192 CPUtime=40000.9 cores=0-7
/proc/15445/stat : 15445 (claspmt) R 15443 15445 15286 0 -1 4202496 266873 0 0 0 3999770 324 0 0 25 0 1 0 956174815 1411268608 247765 33554432000 4194304 4964800 140734759264704 18446744073709551615 224284969319 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/15445/statm: 344548 247765 476 189 0 338897 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 1378192

Child status: 0
Real time (s): 5004.45
CPU time (s): 40001
CPU user time (s): 39997.7
CPU system time (s): 3.3005
CPU usage (%): 799.308
Max. virtual memory (cumulated for all children) (KiB): 1388064

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39997.7
system time used= 3.3005
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 266874
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 63391
involuntary context switches= 191677

runsolver used 8.16076 second user time and 19.1421 second system time

The end

Launcher Data 

Begin job on node118 at 2011-06-10 00:44:02
IDJOB=3380705
IDBENCH=71433
IDSOLVER=1865
FILE ID=node118/3380705-1307659441
RUNJOBID= node118-1307658060-15300
PBS_JOBID= 13549425
Free space on /tmp= 74176 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/random/forced/rbsat-v1150c84314gyes1.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3380705-1307659441/watcher-3380705-1307659441 -o /tmp/evaluation-result-3380705-1307659441/solver-3380705-1307659441 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380705-1307659441.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= d6097a9814089eeede1d027388c17a02
RANDOM SEED=969724469

node118.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5374.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      32196972 kB
Buffers:        195116 kB
Cached:         344768 kB
SwapCached:      15436 kB
Active:         302968 kB
Inactive:       255856 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      32196972 kB
SwapTotal:    67111528 kB
SwapFree:     67049456 kB
Dirty:            1980 kB
Writeback:           0 kB
AnonPages:       16560 kB
Mapped:           8712 kB
Slab:           128804 kB
PageTables:       4700 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   186848 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74176 MiB
End job on node118 at 2011-06-10 02:07:28