Trace number 3322622

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40001 5007.63

General information on the benchmark

NameSAT04/handmade/marijn/equilarge-l/
equilarge_l4.used-as.sat04-896.cnf
MD5SUMfc993d9544ccd8eaeaa4882599387947
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark821.55
Satisfiable
(Un)Satisfiability was proved
Number of variables4499
Number of clauses18603
Sum of the clauses size54454
Maximum clause length3
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21355
Number of clauses of size 317248
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...391752/instance-3322622-1304391752.cnf
0.04/0.04	c Solving...
40000.85/5007.51	
40000.85/5007.51	*** INTERRUPTED! ***
40000.85/5007.51	*** Shutting down threads...
40001.01/5007.62	s UNKNOWN
40001.01/5007.62	
40001.01/5007.62	c INTERRUPTED   : 1
40001.01/5007.62	c Models        : 0+    
40001.01/5007.62	c Time          : 5007.517s (Solving: 5007.48s 1st Model: 0.00s Unsat: 0.00s)
40001.01/5007.62	c CPU Time      : 40000.910s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3322622-1304391752/watcher-3322622-1304391752 -o /tmp/evaluation-result-3322622-1304391752/solver-3322622-1304391752 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322622-1304391752.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.44 7.92 7.97 2/342 15646
/proc/meminfo: memFree=6310760/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=23440 CPUtime=0.01 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 1264 0 0 0 1 0 0 0 18 0 1 0 550777808 24002560 1080 33554432000 4194304 4964704 140734277623728 18446744073709551615 4625035 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/15646/statm: 5860 1081 397 189 0 734 0

[startup+0.0424361 s]
/proc/loadavg: 7.44 7.92 7.97 2/342 15646
/proc/meminfo: memFree=6310760/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=25080 CPUtime=0.04 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 1651 0 0 0 4 0 0 0 18 0 1 0 550777808 25681920 1467 33554432000 4194304 4964704 140734277623728 18446744073709551615 4685854 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/15646/statm: 6270 1467 402 189 0 1144 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 25080

[startup+0.129509 s]
/proc/loadavg: 7.44 7.92 7.97 2/342 15646
/proc/meminfo: memFree=6310760/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=451800 CPUtime=1.4 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 5333 0 0 0 138 2 0 0 18 0 8 0 550777808 462643200 4822 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670219 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/15646/statm: 112950 4822 464 189 0 107299 0
Current children cumulated CPU time (s) 1.4
Current children cumulated vsize (KiB) 451800

[startup+0.408009 s]
/proc/loadavg: 7.44 7.92 7.97 2/342 15646
/proc/meminfo: memFree=6310760/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=452804 CPUtime=6.56 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 7607 0 0 0 653 3 0 0 18 0 8 0 550777808 463671296 7096 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670184 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/15646/statm: 113234 7361 464 189 0 107583 0
Current children cumulated CPU time (s) 6.56
Current children cumulated vsize (KiB) 452804

[startup+1.17216 s]
/proc/loadavg: 7.44 7.92 7.97 2/342 15646
/proc/meminfo: memFree=6310760/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=453200 CPUtime=8.91 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 8130 0 0 0 888 3 0 0 19 0 8 0 550777808 464076800 7619 33554432000 4194304 4964704 140734277623728 18446744073709551615 4342180 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/15646/statm: 113300 7619 464 189 0 107649 0
Current children cumulated CPU time (s) 8.91
Current children cumulated vsize (KiB) 453200

[startup+1.57209 s]
/proc/loadavg: 7.44 7.92 7.97 2/342 15646
/proc/meminfo: memFree=6310760/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=453464 CPUtime=12.1 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 8666 0 0 0 1207 3 0 0 20 0 8 0 550777808 464347136 8155 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/15646/statm: 113366 8155 464 189 0 107715 0
Current children cumulated CPU time (s) 12.1
Current children cumulated vsize (KiB) 453464

[startup+3.10082 s]
/proc/loadavg: 7.44 7.92 7.97 9/350 15654
/proc/meminfo: memFree=6275128/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=454312 CPUtime=24.31 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 10263 0 0 0 2427 4 0 0 25 0 8 0 550777808 465215488 9752 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670100 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/15646/statm: 113578 9752 464 189 0 107927 0
[pid=15646/tid=15648] ppid=15644 vsize=454312 CPUtime=3.03 cores=0-7
/proc/15646/task/15648/stat : 15648 (claspmt) R 15644 15646 15620 0 -1 4202560 1007 0 0 0 303 0 0 0 25 0 8 0 550777812 465215488 9752 33554432000 4194304 4964704 140734277623728 18446744073709551615 4371537 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15646/tid=15649] ppid=15644 vsize=454312 CPUtime=3.03 cores=0-7
/proc/15646/task/15649/stat : 15649 (claspmt) R 15644 15646 15620 0 -1 4202560 1017 0 0 0 303 0 0 0 25 0 8 0 550777812 465215488 9752 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15646/tid=15650] ppid=15644 vsize=454312 CPUtime=3 cores=0-7
/proc/15646/task/15650/stat : 15650 (claspmt) R 15644 15646 15620 0 -1 4202560 1121 0 0 0 300 0 0 0 24 0 8 0 550777812 465215488 9752 33554432000 4194304 4964704 140734277623728 18446744073709551615 4660933 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15646/tid=15651] ppid=15644 vsize=454312 CPUtime=3.03 cores=0-7
/proc/15646/task/15651/stat : 15651 (claspmt) R 15644 15646 15620 0 -1 4202560 984 0 0 0 303 0 0 0 25 0 8 0 550777812 465215488 9752 33554432000 4194304 4964704 140734277623728 18446744073709551615 4660057 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=15646/tid=15652] ppid=15644 vsize=454312 CPUtime=3.03 cores=0-7
/proc/15646/task/15652/stat : 15652 (claspmt) R 15644 15646 15620 0 -1 4202560 1090 0 0 0 303 0 0 0 25 0 8 0 550777812 465215488 9752 33554432000 4194304 4964704 140734277623728 18446744073709551615 4362329 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15646/tid=15653] ppid=15644 vsize=454312 CPUtime=3.03 cores=0-7
/proc/15646/task/15653/stat : 15653 (claspmt) R 15644 15646 15620 0 -1 4202560 990 0 0 0 303 0 0 0 25 0 8 0 550777812 465215488 9752 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670454 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15646/tid=15654] ppid=15644 vsize=454312 CPUtime=3.02 cores=0-7
/proc/15646/task/15654/stat : 15654 (claspmt) R 15644 15646 15620 0 -1 4202560 1577 0 0 0 302 0 0 0 24 0 8 0 550777812 465215488 9752 33554432000 4194304 4964704 140734277623728 18446744073709551615 4320919 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 24.31
Current children cumulated vsize (KiB) 454312

[startup+6.31325 s]
/proc/loadavg: 7.49 7.92 7.97 9/350 15654
/proc/meminfo: memFree=6261240/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=456708 CPUtime=49.98 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 13764 0 0 0 4992 6 0 0 25 0 8 0 550777808 467668992 13253 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670215 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/15646/statm: 114177 13253 466 189 0 108526 0
[pid=15646/tid=15648] ppid=15644 vsize=456708 CPUtime=6.24 cores=0-7
/proc/15646/task/15648/stat : 15648 (claspmt) R 15644 15646 15620 0 -1 4202560 1394 0 0 0 624 0 0 0 25 0 8 0 550777812 467668992 13253 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670277 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15646/tid=15649] ppid=15644 vsize=456708 CPUtime=6.24 cores=0-7
/proc/15646/task/15649/stat : 15649 (claspmt) R 15644 15646 15620 0 -1 4202560 1423 0 0 0 624 0 0 0 25 0 8 0 550777812 467668992 13253 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670215 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15646/tid=15650] ppid=15644 vsize=456708 CPUtime=6.17 cores=0-7
/proc/15646/task/15650/stat : 15650 (claspmt) R 15644 15646 15620 0 -1 4202560 1543 0 0 0 617 0 0 0 25 0 8 0 550777812 467668992 13253 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15646/tid=15651] ppid=15644 vsize=456708 CPUtime=6.24 cores=0-7
/proc/15646/task/15651/stat : 15651 (claspmt) R 15644 15646 15620 0 -1 4202560 1381 0 0 0 624 0 0 0 25 0 8 0 550777812 467668992 13253 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670114 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=15646/tid=15652] ppid=15644 vsize=456708 CPUtime=6.24 cores=0-7
/proc/15646/task/15652/stat : 15652 (claspmt) R 15644 15646 15620 0 -1 4202560 1536 0 0 0 624 0 0 0 25 0 8 0 550777812 467668992 13253 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15646/tid=15653] ppid=15644 vsize=456708 CPUtime=6.24 cores=0-7
/proc/15646/task/15653/stat : 15653 (claspmt) R 15644 15646 15620 0 -1 4202560 1217 0 0 0 624 0 0 0 25 0 8 0 550777812 467668992 13253 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670598 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15646/tid=15654] ppid=15644 vsize=456708 CPUtime=6.23 cores=0-7
/proc/15646/task/15654/stat : 15654 (claspmt) R 15644 15646 15620 0 -1 4202560 2271 0 0 0 623 0 0 0 25 0 8 0 550777812 467668992 13253 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670255 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 49.98
Current children cumulated vsize (KiB) 456708

[startup+12.7011 s]
/proc/loadavg: 7.53 7.92 7.97 9/350 15654
/proc/meminfo: memFree=6249228/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=459240 CPUtime=101 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 17173 0 0 0 10092 8 0 0 25 0 8 0 550777808 470261760 16662 33554432000 4194304 4964704 140734277623728 18446744073709551615 4370264 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/15646/statm: 114810 16662 466 189 0 109159 0
[pid=15646/tid=15648] ppid=15644 vsize=459240 CPUtime=12.63 cores=0-7
/proc/15646/task/15648/stat : 15648 (claspmt) R 15644 15646 15620 0 -1 4202560 1891 0 0 0 1263 0 0 0 25 0 8 0 550777812 470261760 16662 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670271 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15646/tid=15649] ppid=15644 vsize=459240 CPUtime=12.63 cores=0-7
/proc/15646/task/15649/stat : 15649 (claspmt) R 15644 15646 15620 0 -1 4202560 1699 0 0 0 1262 1 0 0 25 0 8 0 550777812 470261760 16662 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15646/tid=15650] ppid=15644 vsize=459240 CPUtime=12.51 cores=0-7
/proc/15646/task/15650/stat : 15650 (claspmt) R 15644 15646 15620 0 -1 4202560 1918 0 0 0 1250 1 0 0 25 0 8 0 550777812 470261760 16662 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15646/tid=15651] ppid=15644 vsize=459240 CPUtime=12.63 cores=0-7
/proc/15646/task/15651/stat : 15651 (claspmt) R 15644 15646 15620 0 -1 4202560 1898 0 0 0 1263 0 0 0 25 0 8 0 550777812 470261760 16662 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=15646/tid=15652] ppid=15644 vsize=459240 CPUtime=12.63 cores=0-7
/proc/15646/task/15652/stat : 15652 (claspmt) R 15644 15646 15620 0 -1 4202560 2148 0 0 0 1262 1 0 0 22 0 8 0 550777812 470261760 16662 33554432000 4194304 4964704 140734277623728 18446744073709551615 4364704 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15646/tid=15653] ppid=15644 vsize=459240 CPUtime=12.63 cores=0-7
/proc/15646/task/15653/stat : 15653 (claspmt) R 15644 15646 15620 0 -1 4202560 1504 0 0 0 1263 0 0 0 25 0 8 0 550777812 470261760 16662 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15646/tid=15654] ppid=15644 vsize=459240 CPUtime=12.62 cores=0-7
/proc/15646/task/15654/stat : 15654 (claspmt) R 15644 15646 15620 0 -1 4202560 2813 0 0 0 1261 1 0 0 25 0 8 0 550777812 470261760 16662 33554432000 4194304 4964704 140734277623728 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 101
Current children cumulated vsize (KiB) 459240

[startup+25.5009 s]
/proc/loadavg: 7.70 7.94 7.97 9/350 15654
/proc/meminfo: memFree=6223544/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=462460 CPUtime=203.24 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 23713 0 0 0 20311 13 0 0 25 0 8 0 550777808 473559040 23121 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670202 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/15646/statm: 115615 23121 466 189 0 109964 0
[pid=15646/tid=15648] ppid=15644 vsize=462460 CPUtime=25.43 cores=0-7
/proc/15646/task/15648/stat : 15648 (claspmt) R 15644 15646 15620 0 -1 4202560 2687 0 0 0 2542 1 0 0 25 0 8 0 550777812 473559040 23121 33554432000 4194304 4964704 140734277623728 18446744073709551615 4320848 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15646/tid=15649] ppid=15644 vsize=462460 CPUtime=25.43 cores=0-7
/proc/15646/task/15649/stat : 15649 (claspmt) R 15644 15646 15620 0 -1 4202560 2397 0 0 0 2542 1 0 0 25 0 8 0 550777812 473559040 23121 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670193 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15646/tid=15650] ppid=15644 vsize=462460 CPUtime=25.16 cores=0-7
/proc/15646/task/15650/stat : 15650 (claspmt) R 15644 15646 15620 0 -1 4202560 2677 0 0 0 2515 1 0 0 25 0 8 0 550777812 473559040 23121 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670269 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15646/tid=15651] ppid=15644 vsize=462460 CPUtime=25.43 cores=0-7
/proc/15646/task/15651/stat : 15651 (claspmt) R 15644 15646 15620 0 -1 4202560 2442 0 0 0 2543 0 0 0 25 0 8 0 550777812 473559040 23121 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670271 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=15646/tid=15652] ppid=15644 vsize=462460 CPUtime=25.42 cores=0-7
/proc/15646/task/15652/stat : 15652 (claspmt) R 15644 15646 15620 0 -1 4202560 3009 0 0 0 2541 1 0 0 25 0 8 0 550777812 473559040 23121 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15646/tid=15653] ppid=15644 vsize=462460 CPUtime=25.42 cores=0-7
/proc/15646/task/15653/stat : 15653 (claspmt) R 15644 15646 15620 0 -1 4202560 2381 0 0 0 2541 1 0 0 25 0 8 0 550777812 473559040 23121 33554432000 4194304 4964704 140734277623728 18446744073709551615 4668820 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15646/tid=15654] ppid=15644 vsize=462460 CPUtime=25.41 cores=0-7
/proc/15646/task/15654/stat : 15654 (claspmt) R 15644 15646 15620 0 -1 4202560 3846 0 0 0 2540 1 0 0 25 0 8 0 550777812 473559040 23121 33554432000 4194304 4964704 140734277623728 18446744073709551615 4669416 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 203.24
Current children cumulated vsize (KiB) 462460

[startup+51.1004 s]
/proc/loadavg: 7.80 7.95 7.97 9/350 15655
/proc/meminfo: memFree=6193628/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=468548 CPUtime=407.75 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 31104 0 0 0 40757 18 0 0 25 0 8 0 550777808 479793152 30507 33554432000 4194304 4964704 140734277623728 18446744073709551615 4342157 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/15646/statm: 117137 30507 466 189 0 111486 0
[pid=15646/tid=15648] ppid=15644 vsize=468548 CPUtime=51.02 cores=0-7
/proc/15646/task/15648/stat : 15648 (claspmt) R 15644 15646 15620 0 -1 4202560 3457 0 0 0 5101 1 0 0 25 0 8 0 550777812 479793152 30507 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15646/tid=15649] ppid=15644 vsize=468548 CPUtime=51.03 cores=0-7
/proc/15646/task/15649/stat : 15649 (claspmt) R 15644 15646 15620 0 -1 4202560 2824 0 0 0 5101 2 0 0 25 0 8 0 550777812 479793152 30507 33554432000 4194304 4964704 140734277623728 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15646/tid=15650] ppid=15644 vsize=468548 CPUtime=50.48 cores=0-7
/proc/15646/task/15650/stat : 15650 (claspmt) R 15644 15646 15620 0 -1 4202560 3289 0 0 0 5047 1 0 0 25 0 8 0 550777812 479793152 30507 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15646/tid=15651] ppid=15644 vsize=468548 CPUtime=51.03 cores=0-7
/proc/15646/task/15651/stat : 15651 (claspmt) R 15644 15646 15620 0 -1 4202560 3122 0 0 0 5102 1 0 0 25 0 8 0 550777812 479793152 30507 33554432000 4194304 4964704 140734277623728 18446744073709551615 4364587 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=15646/tid=15652] ppid=15644 vsize=468548 CPUtime=51.02 cores=0-7
/proc/15646/task/15652/stat : 15652 (claspmt) R 15644 15646 15620 0 -1 4202560 4315 0 0 0 5100 2 0 0 25 0 8 0 550777812 479793152 30507 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670291 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15646/tid=15653] ppid=15644 vsize=468548 CPUtime=51.02 cores=0-7
/proc/15646/task/15653/stat : 15653 (claspmt) R 15644 15646 15620 0 -1 4202560 3888 0 0 0 5099 3 0 0 25 0 8 0 550777812 479793152 30507 33554432000 4194304 4964704 140734277623728 18446744073709551615 4653533 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15646/tid=15654] ppid=15644 vsize=468548 CPUtime=51.01 cores=0-7
/proc/15646/task/15654/stat : 15654 (claspmt) R 15644 15646 15620 0 -1 4202560 5424 0 0 0 5098 3 0 0 25 0 8 0 550777812 479793152 30507 33554432000 4194304 4964704 140734277623728 18446744073709551615 216634212994 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 407.75
Current children cumulated vsize (KiB) 468548

[startup+102.3 s]
/proc/loadavg: 8.12 8.01 8.00 9/350 15659
/proc/meminfo: memFree=6149548/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=473944 CPUtime=815.94 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 41977 0 0 0 81561 33 0 0 25 0 8 0 550777808 485318656 41313 33554432000 4194304 4964704 140734277623728 18446744073709551615 4338823 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/15646/statm: 118486 41313 466 189 0 112835 0
[pid=15646/tid=15648] ppid=15644 vsize=473944 CPUtime=102.22 cores=0-7
/proc/15646/task/15648/stat : 15648 (claspmt) R 15644 15646 15620 0 -1 4202560 4930 0 0 0 10219 3 0 0 25 0 8 0 550777812 485318656 41313 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15646/tid=15649] ppid=15644 vsize=473944 CPUtime=101.93 cores=0-7
/proc/15646/task/15649/stat : 15649 (claspmt) R 15644 15646 15620 0 -1 4202560 3309 0 0 0 10191 2 0 0 25 0 8 0 550777812 485318656 41313 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15646/tid=15650] ppid=15644 vsize=473944 CPUtime=101.12 cores=0-7
/proc/15646/task/15650/stat : 15650 (claspmt) R 15644 15646 15620 0 -1 4202560 4132 0 0 0 10109 3 0 0 25 0 8 0 550777812 485318656 41313 33554432000 4194304 4964704 140734277623728 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 7 0 0 0

################
# More data... #
################

[pid=15646/tid=15649] ppid=15644 vsize=922324 CPUtime=4656.29 cores=0-7
/proc/15646/task/15649/stat : 15649 (claspmt) R 15644 15646 15620 0 -1 4202560 10986 0 0 0 465607 22 0 0 25 0 8 0 550777812 944459776 128591 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15646/tid=15650] ppid=15644 vsize=922324 CPUtime=4645.5 cores=0-7
/proc/15646/task/15650/stat : 15650 (claspmt) R 15644 15646 15620 0 -1 4202560 11604 0 0 0 464516 34 0 0 25 0 8 0 550777812 944459776 128591 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670230 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15646/tid=15651] ppid=15644 vsize=922324 CPUtime=4661.47 cores=0-7
/proc/15646/task/15651/stat : 15651 (claspmt) R 15644 15646 15620 0 -1 4202560 14000 0 0 0 466112 35 0 0 25 0 8 0 550777812 944459776 128591 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670269 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=15646/tid=15652] ppid=15644 vsize=922324 CPUtime=4651.81 cores=0-7
/proc/15646/task/15652/stat : 15652 (claspmt) R 15644 15646 15620 0 -1 4202560 17039 0 0 0 465148 33 0 0 25 0 8 0 550777812 944459776 128591 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15646/tid=15653] ppid=15644 vsize=922324 CPUtime=4661.96 cores=0-7
/proc/15646/task/15653/stat : 15653 (claspmt) R 15644 15646 15620 0 -1 4202560 12058 0 0 0 466162 34 0 0 25 0 8 0 550777812 944459776 128591 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670271 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15646/tid=15654] ppid=15644 vsize=922324 CPUtime=4662.05 cores=0-7
/proc/15646/task/15654/stat : 15654 (claspmt) R 15644 15646 15620 0 -1 4202560 30294 0 0 0 466142 63 0 0 25 0 8 0 550777812 944459776 128591 33554432000 4194304 4964704 140734277623728 18446744073709551615 216634212958 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 37243.1
Current children cumulated vsize (KiB) 922324

[startup+4722.3 s]
/proc/loadavg: 8.06 8.07 8.02 9/350 15794
/proc/meminfo: memFree=5796180/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=922324 CPUtime=37722.4 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 134582 0 0 0 3771945 299 0 0 25 0 8 0 550777808 944459776 129322 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/15646/statm: 230581 129322 466 189 0 224930 0
[pid=15646/tid=15648] ppid=15644 vsize=922324 CPUtime=4702.11 cores=0-7
/proc/15646/task/15648/stat : 15648 (claspmt) R 15644 15646 15620 0 -1 4202560 22773 0 0 0 470171 40 0 0 25 0 8 0 550777812 944459776 129322 33554432000 4194304 4964704 140734277623728 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15646/tid=15649] ppid=15644 vsize=922324 CPUtime=4716.29 cores=0-7
/proc/15646/task/15649/stat : 15649 (claspmt) R 15644 15646 15620 0 -1 4202560 10993 0 0 0 471607 22 0 0 25 0 8 0 550777812 944459776 129322 33554432000 4194304 4964704 140734277623728 18446744073709551615 4379614 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15646/tid=15650] ppid=15644 vsize=922324 CPUtime=4705.5 cores=0-7
/proc/15646/task/15650/stat : 15650 (claspmt) R 15644 15646 15620 0 -1 4202560 11713 0 0 0 470516 34 0 0 25 0 8 0 550777812 944459776 129322 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15646/tid=15651] ppid=15644 vsize=922324 CPUtime=4721.47 cores=0-7
/proc/15646/task/15651/stat : 15651 (claspmt) R 15644 15646 15620 0 -1 4202560 14162 0 0 0 472112 35 0 0 25 0 8 0 550777812 944459776 129322 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=15646/tid=15652] ppid=15644 vsize=922324 CPUtime=4711.81 cores=0-7
/proc/15646/task/15652/stat : 15652 (claspmt) R 15644 15646 15620 0 -1 4202560 17039 0 0 0 471148 33 0 0 25 0 8 0 550777812 944459776 129322 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15646/tid=15653] ppid=15644 vsize=922324 CPUtime=4721.96 cores=0-7
/proc/15646/task/15653/stat : 15653 (claspmt) R 15644 15646 15620 0 -1 4202560 12066 0 0 0 472162 34 0 0 25 0 8 0 550777812 944459776 129322 33554432000 4194304 4964704 140734277623728 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15646/tid=15654] ppid=15644 vsize=922324 CPUtime=4722.05 cores=0-7
/proc/15646/task/15654/stat : 15654 (claspmt) R 15644 15646 15620 0 -1 4202560 30683 0 0 0 472141 64 0 0 25 0 8 0 550777812 944459776 129322 33554432000 4194304 4964704 140734277623728 18446744073709551615 4653154 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 37722.4
Current children cumulated vsize (KiB) 922324

[startup+4782.3 s]
/proc/loadavg: 8.02 8.06 8.01 9/350 15796
/proc/meminfo: memFree=5794784/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=922324 CPUtime=38201.8 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 134989 0 0 0 3819873 304 0 0 25 0 8 0 550777808 944459776 129707 33554432000 4194304 4964704 140734277623728 18446744073709551615 4659143 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/15646/statm: 230581 129707 466 189 0 224930 0
[pid=15646/tid=15648] ppid=15644 vsize=922324 CPUtime=4761.46 cores=0-7
/proc/15646/task/15648/stat : 15648 (claspmt) R 15644 15646 15620 0 -1 4202560 22859 0 0 0 476106 40 0 0 25 0 8 0 550777812 944459776 129707 33554432000 4194304 4964704 140734277623728 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15646/tid=15649] ppid=15644 vsize=922324 CPUtime=4776.28 cores=0-7
/proc/15646/task/15649/stat : 15649 (claspmt) R 15644 15646 15620 0 -1 4202560 11002 0 0 0 477606 22 0 0 25 0 8 0 550777812 944459776 129707 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15646/tid=15650] ppid=15644 vsize=922324 CPUtime=4765.5 cores=0-7
/proc/15646/task/15650/stat : 15650 (claspmt) R 15644 15646 15620 0 -1 4202560 11789 0 0 0 476515 35 0 0 25 0 8 0 550777812 944459776 129707 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15646/tid=15651] ppid=15644 vsize=922324 CPUtime=4781.47 cores=0-7
/proc/15646/task/15651/stat : 15651 (claspmt) R 15644 15646 15620 0 -1 4202560 14170 0 0 0 478111 36 0 0 25 0 8 0 550777812 944459776 129707 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=15646/tid=15652] ppid=15644 vsize=922324 CPUtime=4771.81 cores=0-7
/proc/15646/task/15652/stat : 15652 (claspmt) R 15644 15646 15620 0 -1 4202560 17084 0 0 0 477148 33 0 0 25 0 8 0 550777812 944459776 129707 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15646/tid=15653] ppid=15644 vsize=922324 CPUtime=4781.96 cores=0-7
/proc/15646/task/15653/stat : 15653 (claspmt) R 15644 15646 15620 0 -1 4202560 12107 0 0 0 478161 35 0 0 25 0 8 0 550777812 944459776 129707 33554432000 4194304 4964704 140734277623728 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15646/tid=15654] ppid=15644 vsize=922324 CPUtime=4782.05 cores=0-7
/proc/15646/task/15654/stat : 15654 (claspmt) R 15644 15646 15620 0 -1 4202560 30825 0 0 0 478140 65 0 0 25 0 8 0 550777812 944459776 129707 33554432000 4194304 4964704 140734277623728 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38201.8
Current children cumulated vsize (KiB) 922324

[startup+4842.3 s]
/proc/loadavg: 8.00 8.05 8.00 9/350 15798
/proc/meminfo: memFree=5793996/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=922324 CPUtime=38681.1 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 135144 0 0 0 3867799 306 0 0 25 0 8 0 550777808 944459776 129840 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/15646/statm: 230581 129840 466 189 0 224930 0
[pid=15646/tid=15648] ppid=15644 vsize=922324 CPUtime=4820.79 cores=0-7
/proc/15646/task/15648/stat : 15648 (claspmt) R 15644 15646 15620 0 -1 4202560 22991 0 0 0 482039 40 0 0 25 0 8 0 550777812 944459776 129840 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15646/tid=15649] ppid=15644 vsize=922324 CPUtime=4836.29 cores=0-7
/proc/15646/task/15649/stat : 15649 (claspmt) R 15644 15646 15620 0 -1 4202560 11025 0 0 0 483606 23 0 0 25 0 8 0 550777812 944459776 129840 33554432000 4194304 4964704 140734277623728 18446744073709551615 4669455 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15646/tid=15650] ppid=15644 vsize=922324 CPUtime=4825.49 cores=0-7
/proc/15646/task/15650/stat : 15650 (claspmt) R 15644 15646 15620 0 -1 4202560 11789 0 0 0 482514 35 0 0 25 0 8 0 550777812 944459776 129840 33554432000 4194304 4964704 140734277623728 18446744073709551615 4361283 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15646/tid=15651] ppid=15644 vsize=922324 CPUtime=4841.46 cores=0-7
/proc/15646/task/15651/stat : 15651 (claspmt) R 15644 15646 15620 0 -1 4202560 14170 0 0 0 484110 36 0 0 25 0 8 0 550777812 944459776 129840 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=15646/tid=15652] ppid=15644 vsize=922324 CPUtime=4831.8 cores=0-7
/proc/15646/task/15652/stat : 15652 (claspmt) R 15644 15646 15620 0 -1 4202560 17084 0 0 0 483147 33 0 0 25 0 8 0 550777812 944459776 129840 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15646/tid=15653] ppid=15644 vsize=922324 CPUtime=4841.94 cores=0-7
/proc/15646/task/15653/stat : 15653 (claspmt) R 15644 15646 15620 0 -1 4202560 12107 0 0 0 484159 35 0 0 25 0 8 0 550777812 944459776 129840 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15646/tid=15654] ppid=15644 vsize=922324 CPUtime=4842.05 cores=0-7
/proc/15646/task/15654/stat : 15654 (claspmt) R 15644 15646 15620 0 -1 4202560 30825 0 0 0 484139 66 0 0 25 0 8 0 550777812 944459776 129840 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670260 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38681
Current children cumulated vsize (KiB) 922324

[startup+4902.3 s]
/proc/loadavg: 8.00 8.03 8.00 9/350 15799
/proc/meminfo: memFree=5791860/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=922324 CPUtime=39160.5 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 135718 0 0 0 3915738 309 0 0 25 0 8 0 550777808 944459776 130317 33554432000 4194304 4964704 140734277623728 18446744073709551615 4668472 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/15646/statm: 230581 130317 466 189 0 224930 0
[pid=15646/tid=15648] ppid=15644 vsize=922324 CPUtime=4880.14 cores=0-7
/proc/15646/task/15648/stat : 15648 (claspmt) R 15644 15646 15620 0 -1 4202560 23012 0 0 0 487973 41 0 0 25 0 8 0 550777812 944459776 130317 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670260 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15646/tid=15649] ppid=15644 vsize=922324 CPUtime=4896.3 cores=0-7
/proc/15646/task/15649/stat : 15649 (claspmt) R 15644 15646 15620 0 -1 4202560 11064 0 0 0 489607 23 0 0 25 0 8 0 550777812 944459776 130317 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15646/tid=15650] ppid=15644 vsize=922324 CPUtime=4885.51 cores=0-7
/proc/15646/task/15650/stat : 15650 (claspmt) R 15644 15646 15620 0 -1 4202560 11798 0 0 0 488515 36 0 0 25 0 8 0 550777812 944459776 130317 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15646/tid=15651] ppid=15644 vsize=922324 CPUtime=4901.48 cores=0-7
/proc/15646/task/15651/stat : 15651 (claspmt) R 15644 15646 15620 0 -1 4202560 14170 0 0 0 490111 37 0 0 25 0 8 0 550777812 944459776 130317 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=15646/tid=15652] ppid=15644 vsize=922324 CPUtime=4891.82 cores=0-7
/proc/15646/task/15652/stat : 15652 (claspmt) R 15644 15646 15620 0 -1 4202560 17092 0 0 0 489148 34 0 0 25 0 8 0 550777812 944459776 130317 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15646/tid=15653] ppid=15644 vsize=922324 CPUtime=4901.95 cores=0-7
/proc/15646/task/15653/stat : 15653 (claspmt) R 15644 15646 15620 0 -1 4202560 12153 0 0 0 490160 35 0 0 25 0 8 0 550777812 944459776 130317 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15646/tid=15654] ppid=15644 vsize=922324 CPUtime=4902.06 cores=0-7
/proc/15646/task/15654/stat : 15654 (claspmt) R 15644 15646 15620 0 -1 4202560 31247 0 0 0 490139 67 0 0 25 0 8 0 550777812 944459776 130317 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670263 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39160.5
Current children cumulated vsize (KiB) 922324

[startup+4962.3 s]
/proc/loadavg: 8.08 8.04 8.01 9/350 15801
/proc/meminfo: memFree=5785116/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=922324 CPUtime=39639.7 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 137680 0 0 0 3963654 315 0 0 25 0 8 0 550777808 944459776 132213 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670248 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/15646/statm: 230581 132213 466 189 0 224930 0
[pid=15646/tid=15648] ppid=15644 vsize=922324 CPUtime=4939.48 cores=0-7
/proc/15646/task/15648/stat : 15648 (claspmt) R 15644 15646 15620 0 -1 4202560 23205 0 0 0 493907 41 0 0 25 0 8 0 550777812 944459776 132213 33554432000 4194304 4964704 140734277623728 18446744073709551615 4332387 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15646/tid=15649] ppid=15644 vsize=922324 CPUtime=4956.29 cores=0-7
/proc/15646/task/15649/stat : 15649 (claspmt) R 15644 15646 15620 0 -1 4202560 11288 0 0 0 495605 24 0 0 25 0 8 0 550777812 944459776 132213 33554432000 4194304 4964704 140734277623728 18446744073709551615 4361315 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15646/tid=15650] ppid=15644 vsize=922324 CPUtime=4945.49 cores=0-7
/proc/15646/task/15650/stat : 15650 (claspmt) R 15644 15646 15620 0 -1 4202560 12278 0 0 0 494512 37 0 0 25 0 8 0 550777812 944459776 132213 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15646/tid=15651] ppid=15644 vsize=922324 CPUtime=4961.46 cores=0-7
/proc/15646/task/15651/stat : 15651 (claspmt) R 15644 15646 15620 0 -1 4202560 14308 0 0 0 496109 37 0 0 25 0 8 0 550777812 944459776 132213 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=15646/tid=15652] ppid=15644 vsize=922324 CPUtime=4951.8 cores=0-7
/proc/15646/task/15652/stat : 15652 (claspmt) R 15644 15646 15620 0 -1 4202560 17097 0 0 0 495146 34 0 0 25 0 8 0 550777812 944459776 132213 33554432000 4194304 4964704 140734277623728 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15646/tid=15653] ppid=15644 vsize=922324 CPUtime=4961.94 cores=0-7
/proc/15646/task/15653/stat : 15653 (claspmt) R 15644 15646 15620 0 -1 4202560 12315 0 0 0 496158 36 0 0 25 0 8 0 550777812 944459776 132213 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15646/tid=15654] ppid=15644 vsize=922324 CPUtime=4962.04 cores=0-7
/proc/15646/task/15654/stat : 15654 (claspmt) R 15644 15646 15620 0 -1 4202560 31608 0 0 0 496136 68 0 0 25 0 8 0 550777812 944459776 132213 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39639.7
Current children cumulated vsize (KiB) 922324



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5007.5 s]
/proc/loadavg: 8.04 8.03 8.00 9/350 15802
/proc/meminfo: memFree=5784852/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=922324 CPUtime=40000.9 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 137745 0 0 0 3999769 316 0 0 25 0 8 0 550777808 944459776 132277 33554432000 4194304 4964704 140734277623728 18446744073709551615 4668488 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/15646/statm: 230581 132277 466 189 0 224930 0
[pid=15646/tid=15648] ppid=15644 vsize=922324 CPUtime=4984.18 cores=0-7
/proc/15646/task/15648/stat : 15648 (claspmt) R 15644 15646 15620 0 -1 4202560 23211 0 0 0 498377 41 0 0 25 0 8 0 550777812 944459776 132277 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670263 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=15646/tid=15649] ppid=15644 vsize=922324 CPUtime=5001.49 cores=0-7
/proc/15646/task/15649/stat : 15649 (claspmt) R 15644 15646 15620 0 -1 4202560 11297 0 0 0 500125 24 0 0 25 0 8 0 550777812 944459776 132277 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=15646/tid=15650] ppid=15644 vsize=922324 CPUtime=4990.7 cores=0-7
/proc/15646/task/15650/stat : 15650 (claspmt) R 15644 15646 15620 0 -1 4202560 12278 0 0 0 499033 37 0 0 25 0 8 0 550777812 944459776 132277 33554432000 4194304 4964704 140734277623728 18446744073709551615 4669429 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=15646/tid=15651] ppid=15644 vsize=922324 CPUtime=5006.67 cores=0-7
/proc/15646/task/15651/stat : 15651 (claspmt) R 15644 15646 15620 0 -1 4202560 14347 0 0 0 500630 37 0 0 25 0 8 0 550777812 944459776 132277 33554432000 4194304 4964704 140734277623728 18446744073709551615 4362349 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=15646/tid=15652] ppid=15644 vsize=922324 CPUtime=4997.01 cores=0-7
/proc/15646/task/15652/stat : 15652 (claspmt) R 15644 15646 15620 0 -1 4202560 17097 0 0 0 499667 34 0 0 25 0 8 0 550777812 944459776 132277 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=15646/tid=15653] ppid=15644 vsize=922324 CPUtime=5007.15 cores=0-7
/proc/15646/task/15653/stat : 15653 (claspmt) R 15644 15646 15620 0 -1 4202560 12317 0 0 0 500679 36 0 0 25 0 8 0 550777812 944459776 132277 33554432000 4194304 4964704 140734277623728 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=15646/tid=15654] ppid=15644 vsize=922324 CPUtime=5007.25 cores=0-7
/proc/15646/task/15654/stat : 15654 (claspmt) R 15644 15646 15620 0 -1 4202560 31617 0 0 0 500657 68 0 0 25 0 8 0 550777812 944459776 132277 33554432000 4194304 4964704 140734277623728 18446744073709551615 4668683 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 922324

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5007.62 s]
/proc/loadavg: 8.04 8.03 8.00 9/350 15802
/proc/meminfo: memFree=5784852/32951124 swapFree=67111524/67111528
[pid=15646] ppid=15644 vsize=500484 CPUtime=40001 cores=0-7
/proc/15646/stat : 15646 (claspmt) R 15644 15646 15620 0 -1 4202496 137755 0 0 0 3999783 318 0 0 25 0 1 0 550777808 512495616 39109 33554432000 4194304 4964704 140734277623728 18446744073709551615 216634541162 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/15646/statm: 125121 39109 476 189 0 119470 0
Current children cumulated CPU time (s) 40001
Current children cumulated vsize (KiB) 500484

Child status: 0
Real time (s): 5007.63
CPU time (s): 40001
CPU user time (s): 39997.8
CPU system time (s): 3.19351
CPU usage (%): 798.802
Max. virtual memory (cumulated for all children) (KiB): 922324

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39997.8
system time used= 3.19351
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 137756
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 185018
involuntary context switches= 191744

runsolver used 14.4418 second user time and 40.1989 second system time

The end

Launcher Data 

Begin job on node103 at 2011-05-03 05:02:32
IDJOB=3322622
IDBENCH=84762
IDSOLVER=1711
FILE ID=node103/3322622-1304391752
RUNJOBID= node103-1304391752-15634
PBS_JOBID= 13181846
Free space on /tmp= 73448 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT04/handmade/marijn/equilarge-l/equilarge_l4.used-as.sat04-896.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322622-1304391752/watcher-3322622-1304391752 -o /tmp/evaluation-result-3322622-1304391752/solver-3322622-1304391752 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322622-1304391752.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= fc993d9544ccd8eaeaa4882599387947
RANDOM SEED=1770047092

node103.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:       6310916 kB
Buffers:       2141236 kB
Cached:        8321240 kB
SwapCached:          4 kB
Active:       17349540 kB
Inactive:      8647868 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:       6310916 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:            1204 kB
Writeback:           0 kB
AnonPages:    15535344 kB
Mapped:          30076 kB
Slab:           521516 kB
PageTables:      52876 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 78288608 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73448 MiB
End job on node103 at 2011-05-03 06:26:01