Trace number 3380693

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40001.1 5012.34

General information on the benchmark

NameCRAFTED/rbsat/random/
forced/rbsat-v2640c305320gyes2.cnf
MD5SUMa931e6258afb576eab04f38421c07518
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2640
Number of clauses305320
Sum of the clauses size613120
Maximum clause length33
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2305240
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 580

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...487202/instance-3380693-1305487202.cnf
1.69/1.72	c Solving...
40000.14/5011.42	
40000.14/5011.42	*** INTERRUPTED! ***
40000.14/5011.42	*** Shutting down threads...
40001.06/5012.33	s UNKNOWN
40001.06/5012.33	
40001.06/5012.33	c INTERRUPTED   : 1
40001.06/5012.33	c Models        : 0+    
40001.06/5012.33	c Time          : 5011.421s (Solving: 5009.70s 1st Model: 0.00s Unsat: 0.00s)
40001.06/5012.33	c CPU Time      : 40000.180s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3380693-1305487202/watcher-3380693-1305487202 -o /tmp/evaluation-result-3380693-1305487202/solver-3380693-1305487202 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380693-1305487202.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.20 8.07 8.10 2/425 24476
/proc/meminfo: memFree=17175036/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=23676 CPUtime=0.01 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 1384 0 0 0 1 0 0 0 21 0 1 0 738950865 24244224 1123 33554432000 4194304 4964800 140736255868224 18446744073709551615 224284586693 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/24476/statm: 5919 1124 386 189 0 793 0

[startup+0.050366 s]
/proc/loadavg: 7.20 8.07 8.10 2/425 24476
/proc/meminfo: memFree=17175036/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=30772 CPUtime=0.04 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 3968 0 0 0 4 0 0 0 21 0 1 0 738950865 31510528 2728 33554432000 4194304 4964800 140736255868224 18446744073709551615 224284586693 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/24476/statm: 7693 2730 386 189 0 2567 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 30772

[startup+0.100355 s]
/proc/loadavg: 7.20 8.07 8.10 2/425 24476
/proc/meminfo: memFree=17175036/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=39516 CPUtime=0.09 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 6332 0 0 0 9 0 0 0 21 0 1 0 738950865 40464384 5068 33554432000 4194304 4964800 140736255868224 18446744073709551615 224284586693 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/24476/statm: 9879 5072 397 189 0 4753 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 39516

[startup+0.300334 s]
/proc/loadavg: 7.20 8.07 8.10 2/425 24476
/proc/meminfo: memFree=17175036/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=39780 CPUtime=0.29 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 6433 0 0 0 29 0 0 0 22 0 1 0 738950865 40734720 5169 33554432000 4194304 4964800 140736255868224 18446744073709551615 4607575 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/24476/statm: 9945 5169 399 189 0 4819 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 39780

[startup+0.700257 s]
/proc/loadavg: 7.20 8.07 8.10 2/425 24476
/proc/meminfo: memFree=17175036/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=39780 CPUtime=0.69 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 6433 0 0 0 69 0 0 0 23 0 1 0 738950865 40734720 5169 33554432000 4194304 4964800 140736255868224 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/24476/statm: 9945 5169 399 189 0 4819 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 39780

[startup+1.50112 s]
/proc/loadavg: 6.70 7.95 8.07 2/426 24477
/proc/meminfo: memFree=17154476/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=42948 CPUtime=1.49 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 7220 0 0 0 149 0 0 0 25 0 1 0 738950865 43978752 5956 33554432000 4194304 4964800 140736255868224 18446744073709551615 4600334 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/24476/statm: 10737 5956 400 189 0 5611 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 42948

[startup+3.10082 s]
/proc/loadavg: 6.70 7.95 8.07 9/433 24484
/proc/meminfo: memFree=17137440/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=401280 CPUtime=11.04 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 20132 0 0 0 1094 10 0 0 25 0 8 0 738950865 410910720 18229 33554432000 4194304 4964800 140736255868224 18446744073709551615 4668984 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/24476/statm: 100320 18229 464 189 0 94669 0
[pid=24476/tid=24478] ppid=24474 vsize=401280 CPUtime=1.14 cores=0-7
/proc/24476/task/24478/stat : 24478 (claspmt) R 24474 24476 24450 0 -1 4202560 1330 0 0 0 113 1 0 0 17 0 8 0 738951037 410910720 18229 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24476/tid=24479] ppid=24474 vsize=401280 CPUtime=1.13 cores=0-7
/proc/24476/task/24479/stat : 24479 (claspmt) R 24474 24476 24450 0 -1 4202560 1757 0 0 0 112 1 0 0 17 0 8 0 738951037 410910720 18229 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670437 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=24476/tid=24480] ppid=24474 vsize=401280 CPUtime=1.14 cores=0-7
/proc/24476/task/24480/stat : 24480 (claspmt) R 24474 24476 24450 0 -1 4202560 1563 0 0 0 113 1 0 0 19 0 8 0 738951037 410910720 18229 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670440 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=24476/tid=24481] ppid=24474 vsize=401280 CPUtime=1.14 cores=0-7
/proc/24476/task/24481/stat : 24481 (claspmt) R 24474 24476 24450 0 -1 4202560 1575 0 0 0 113 1 0 0 17 0 8 0 738951037 410910720 18229 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=24476/tid=24482] ppid=24474 vsize=401280 CPUtime=1.36 cores=0-7
/proc/24476/task/24482/stat : 24482 (claspmt) R 24474 24476 24450 0 -1 4202560 1110 0 0 0 136 0 0 0 19 0 8 0 738951037 410910720 18229 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670155 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=24476/tid=24483] ppid=24474 vsize=401280 CPUtime=1.13 cores=0-7
/proc/24476/task/24483/stat : 24483 (claspmt) R 24474 24476 24450 0 -1 4202560 1668 0 0 0 113 0 0 0 17 0 8 0 738951037 410910720 18229 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24476/tid=24484] ppid=24474 vsize=401280 CPUtime=1.12 cores=0-7
/proc/24476/task/24484/stat : 24484 (claspmt) R 24474 24476 24450 0 -1 4202560 2475 0 0 0 111 1 0 0 17 0 8 0 738951037 410910720 18229 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.04
Current children cumulated vsize (KiB) 401280

[startup+6.30129 s]
/proc/loadavg: 6.70 7.95 8.07 9/433 24484
/proc/meminfo: memFree=17059840/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=409612 CPUtime=36.59 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 36167 0 0 0 3641 18 0 0 25 0 8 0 738950865 419442688 34264 33554432000 4194304 4964800 140736255868224 18446744073709551615 4660255 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/24476/statm: 102403 34264 464 189 0 96752 0
[pid=24476/tid=24478] ppid=24474 vsize=409612 CPUtime=4.33 cores=0-7
/proc/24476/task/24478/stat : 24478 (claspmt) R 24474 24476 24450 0 -1 4202560 3003 0 0 0 432 1 0 0 23 0 8 0 738951037 419442688 34264 33554432000 4194304 4964800 140736255868224 18446744073709551615 4627272 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24476/tid=24479] ppid=24474 vsize=409612 CPUtime=4.34 cores=0-7
/proc/24476/task/24479/stat : 24479 (claspmt) R 24474 24476 24450 0 -1 4202560 3592 0 0 0 431 3 0 0 23 0 8 0 738951037 419442688 34264 33554432000 4194304 4964800 140736255868224 18446744073709551615 4660155 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=24476/tid=24480] ppid=24474 vsize=409612 CPUtime=4.34 cores=0-7
/proc/24476/task/24480/stat : 24480 (claspmt) R 24474 24476 24450 0 -1 4202560 3092 0 0 0 432 2 0 0 25 0 8 0 738951037 419442688 34264 33554432000 4194304 4964800 140736255868224 18446744073709551615 4668987 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=24476/tid=24481] ppid=24474 vsize=409612 CPUtime=4.33 cores=0-7
/proc/24476/task/24481/stat : 24481 (claspmt) R 24474 24476 24450 0 -1 4202560 3529 0 0 0 432 1 0 0 23 0 8 0 738951037 419442688 34264 33554432000 4194304 4964800 140736255868224 18446744073709551615 4660168 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=24476/tid=24482] ppid=24474 vsize=409612 CPUtime=4.56 cores=0-7
/proc/24476/task/24482/stat : 24482 (claspmt) R 24474 24476 24450 0 -1 4202560 1481 0 0 0 456 0 0 0 25 0 8 0 738951037 419442688 34265 33554432000 4194304 4964800 140736255868224 18446744073709551615 4632345 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=24476/tid=24483] ppid=24474 vsize=409612 CPUtime=4.33 cores=0-7
/proc/24476/task/24483/stat : 24483 (claspmt) R 24474 24476 24450 0 -1 4202560 3212 0 0 0 432 1 0 0 23 0 8 0 738951037 419442688 34265 33554432000 4194304 4964800 140736255868224 18446744073709551615 224284586693 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24476/tid=24484] ppid=24474 vsize=409612 CPUtime=4.28 cores=0-7
/proc/24476/task/24484/stat : 24484 (claspmt) R 24474 24476 24450 0 -1 4202560 7981 0 0 0 423 5 0 0 23 0 8 0 738951037 419442688 34265 33554432000 4194304 4964800 140736255868224 18446744073709551615 4365674 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 36.59
Current children cumulated vsize (KiB) 409612

[startup+12.7012 s]
/proc/loadavg: 6.90 7.95 8.07 9/433 24484
/proc/meminfo: memFree=16943048/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=428660 CPUtime=87.67 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 63488 0 0 0 8733 34 0 0 25 0 8 0 738950865 438947840 61585 33554432000 4194304 4964800 140736255868224 18446744073709551615 4627263 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/24476/statm: 107165 61585 466 189 0 101514 0
[pid=24476/tid=24478] ppid=24474 vsize=428660 CPUtime=10.72 cores=0-7
/proc/24476/task/24478/stat : 24478 (claspmt) R 24474 24476 24450 0 -1 4202560 6459 0 0 0 1070 2 0 0 19 0 8 0 738951037 438947840 61585 33554432000 4194304 4964800 140736255868224 18446744073709551615 4669131 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24476/tid=24479] ppid=24474 vsize=428660 CPUtime=10.72 cores=0-7
/proc/24476/task/24479/stat : 24479 (claspmt) R 24474 24476 24450 0 -1 4202560 6575 0 0 0 1068 4 0 0 25 0 8 0 738951037 438947840 61585 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=24476/tid=24480] ppid=24474 vsize=428660 CPUtime=10.74 cores=0-7
/proc/24476/task/24480/stat : 24480 (claspmt) R 24474 24476 24450 0 -1 4202560 5634 0 0 0 1070 4 0 0 25 0 8 0 738951037 438947840 61585 33554432000 4194304 4964800 140736255868224 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=24476/tid=24481] ppid=24474 vsize=428660 CPUtime=10.72 cores=0-7
/proc/24476/task/24481/stat : 24481 (claspmt) R 24474 24476 24450 0 -1 4202560 6700 0 0 0 1070 2 0 0 19 0 8 0 738951037 438947840 61585 33554432000 4194304 4964800 140736255868224 18446744073709551615 4668900 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=24476/tid=24482] ppid=24474 vsize=428660 CPUtime=10.96 cores=0-7
/proc/24476/task/24482/stat : 24482 (claspmt) R 24474 24476 24450 0 -1 4202560 4494 0 0 0 1094 2 0 0 25 0 8 0 738951037 438947840 61585 33554432000 4194304 4964800 140736255868224 18446744073709551615 4658574 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=24476/tid=24483] ppid=24474 vsize=428660 CPUtime=10.74 cores=0-7
/proc/24476/task/24483/stat : 24483 (claspmt) R 24474 24476 24450 0 -1 4202560 4865 0 0 0 1071 3 0 0 25 0 8 0 738951037 438947840 61585 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24476/tid=24484] ppid=24474 vsize=428660 CPUtime=10.58 cores=0-7
/proc/24476/task/24484/stat : 24484 (claspmt) R 24474 24476 24450 0 -1 4202560 15837 0 0 0 1049 9 0 0 25 0 8 0 738951037 438947840 61585 33554432000 4194304 4964800 140736255868224 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 87.67
Current children cumulated vsize (KiB) 428660

[startup+25.5011 s]
/proc/loadavg: 7.07 7.95 8.06 9/433 24484
/proc/meminfo: memFree=16834148/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=447880 CPUtime=189.87 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 88456 0 0 0 18937 50 0 0 25 0 8 0 738950865 458629120 86541 33554432000 4194304 4964800 140736255868224 18446744073709551615 4683980 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/24476/statm: 111970 86541 466 189 0 106319 0
[pid=24476/tid=24478] ppid=24474 vsize=447880 CPUtime=23.52 cores=0-7
/proc/24476/task/24478/stat : 24478 (claspmt) R 24474 24476 24450 0 -1 4202560 9754 0 0 0 2347 5 0 0 25 0 8 0 738951037 458629120 86541 33554432000 4194304 4964800 140736255868224 18446744073709551615 4627259 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24476/tid=24479] ppid=24474 vsize=447880 CPUtime=23.52 cores=0-7
/proc/24476/task/24479/stat : 24479 (claspmt) R 24474 24476 24450 0 -1 4202560 9555 0 0 0 2346 6 0 0 25 0 8 0 738951037 458629120 86541 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=24476/tid=24480] ppid=24474 vsize=447880 CPUtime=23.54 cores=0-7
/proc/24476/task/24480/stat : 24480 (claspmt) R 24474 24476 24450 0 -1 4202560 7091 0 0 0 2349 5 0 0 25 0 8 0 738951037 458629120 86541 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=24476/tid=24481] ppid=24474 vsize=447880 CPUtime=23.53 cores=0-7
/proc/24476/task/24481/stat : 24481 (claspmt) R 24474 24476 24450 0 -1 4202560 8600 0 0 0 2350 3 0 0 25 0 8 0 738951037 458629120 86541 33554432000 4194304 4964800 140736255868224 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=24476/tid=24482] ppid=24474 vsize=447880 CPUtime=23.76 cores=0-7
/proc/24476/task/24482/stat : 24482 (claspmt) R 24474 24476 24450 0 -1 4202560 7319 0 0 0 2372 4 0 0 25 0 8 0 738951037 458629120 86541 33554432000 4194304 4964800 140736255868224 18446744073709551615 4684107 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=24476/tid=24483] ppid=24474 vsize=447880 CPUtime=23.54 cores=0-7
/proc/24476/task/24483/stat : 24483 (claspmt) R 24474 24476 24450 0 -1 4202560 6828 0 0 0 2350 4 0 0 25 0 8 0 738951037 458629120 86541 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670340 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24476/tid=24484] ppid=24474 vsize=447880 CPUtime=23.19 cores=0-7
/proc/24476/task/24484/stat : 24484 (claspmt) R 24474 24476 24450 0 -1 4202560 23631 0 0 0 2305 14 0 0 25 0 8 0 738951037 458629120 86541 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 189.87
Current children cumulated vsize (KiB) 447880

[startup+51.1008 s]
/proc/loadavg: 7.38 7.96 8.06 9/433 24486
/proc/meminfo: memFree=16804600/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=514496 CPUtime=394.28 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 97331 0 0 0 39369 59 0 0 25 0 8 0 738950865 526843904 95412 33554432000 4194304 4964800 140736255868224 18446744073709551615 4627204 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/24476/statm: 128624 95412 466 189 0 122973 0
[pid=24476/tid=24478] ppid=24474 vsize=514496 CPUtime=49.11 cores=0-7
/proc/24476/task/24478/stat : 24478 (claspmt) R 24474 24476 24450 0 -1 4202560 11171 0 0 0 4905 6 0 0 25 0 8 0 738951037 526843904 95412 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24476/tid=24479] ppid=24474 vsize=514496 CPUtime=49.11 cores=0-7
/proc/24476/task/24479/stat : 24479 (claspmt) R 24474 24476 24450 0 -1 4202560 10509 0 0 0 4904 7 0 0 25 0 8 0 738951037 526843904 95412 33554432000 4194304 4964800 140736255868224 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=24476/tid=24480] ppid=24474 vsize=514496 CPUtime=49.14 cores=0-7
/proc/24476/task/24480/stat : 24480 (claspmt) R 24474 24476 24450 0 -1 4202560 7757 0 0 0 4908 6 0 0 25 0 8 0 738951037 526843904 95412 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=24476/tid=24481] ppid=24474 vsize=514496 CPUtime=49.12 cores=0-7
/proc/24476/task/24481/stat : 24481 (claspmt) R 24474 24476 24450 0 -1 4202560 9172 0 0 0 4909 3 0 0 25 0 8 0 738951037 526843904 95412 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=24476/tid=24482] ppid=24474 vsize=514496 CPUtime=49.36 cores=0-7
/proc/24476/task/24482/stat : 24482 (claspmt) R 24474 24476 24450 0 -1 4202560 7708 0 0 0 4931 5 0 0 25 0 8 0 738951037 526843904 95412 33554432000 4194304 4964800 140736255868224 18446744073709551615 4669119 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=24476/tid=24483] ppid=24474 vsize=514496 CPUtime=49.14 cores=0-7
/proc/24476/task/24483/stat : 24483 (claspmt) R 24474 24476 24450 0 -1 4202560 7780 0 0 0 4909 5 0 0 25 0 8 0 738951037 526843904 95412 33554432000 4194304 4964800 140736255868224 18446744073709551615 4627299 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24476/tid=24484] ppid=24474 vsize=514496 CPUtime=48.42 cores=0-7
/proc/24476/task/24484/stat : 24484 (claspmt) R 24474 24476 24450 0 -1 4202560 26906 0 0 0 4826 16 0 0 25 0 8 0 738951037 526843904 95412 33554432000 4194304 4964800 140736255868224 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 394.28
Current children cumulated vsize (KiB) 514496

[startup+102.316 s]
/proc/loadavg: 7.79 7.97 8.06 9/433 24486
/proc/meminfo: memFree=16676356/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=794980 CPUtime=803.18 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 128002 0 0 0 80230 88 0 0 25 0 8 0 738950865 814059520 126071 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670371 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/24476/statm: 198745 126071 466 189 0 193094 0
[pid=24476/tid=24478] ppid=24474 vsize=794980 CPUtime=100.31 cores=0-7
/proc/24476/task/24478/stat : 24478 (claspmt) R 24474 24476 24450 0 -1 4202560 16054 0 0 0 10020 11 0 0 25 0 8 0 738951037 814059520 126071 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24476/tid=24479] ppid=24474 vsize=794980 CPUtime=100.31 cores=0-7
/proc/24476/task/24479/stat : 24479 (claspmt) R 24474 24476 24450 0 -1 4202560 12987 0 0 0 10021 10 0 0 25 0 8 0 738951037 814059520 126071 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=24476/tid=24480] ppid=24474 vsize=794980 CPUtime=100.35 cores=0-7
/proc/24476/task/24480/stat : 24480 (claspmt) R 24474 24476 24450 0 -1 4202560 9828 0 0 0 10027 8 0 0 25 0 8 0 738951037 814059520 126071 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 0 0 0 0

################
# More data... #
################

[pid=24476] ppid=24474 vsize=2573496 CPUtime=37692.5 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 602693 0 0 0 3768565 686 0 0 25 0 8 0 738950865 2635259904 592077 33554432000 4194304 4964800 140736255868224 18446744073709551615 4320951 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/24476/statm: 643374 592077 466 189 0 637723 0
[pid=24476/tid=24478] ppid=24474 vsize=2573496 CPUtime=4713.68 cores=0-7
/proc/24476/task/24478/stat : 24478 (claspmt) R 24474 24476 24450 0 -1 4202560 69161 0 0 0 471297 71 0 0 25 0 8 0 738951037 2635259904 592077 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=24476/tid=24479] ppid=24474 vsize=2573496 CPUtime=4718.5 cores=0-7
/proc/24476/task/24479/stat : 24479 (claspmt) R 24474 24476 24450 0 -1 4202560 44808 0 0 0 471812 38 0 0 25 0 8 0 738951037 2635259904 592077 33554432000 4194304 4964800 140736255868224 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=24476/tid=24480] ppid=24474 vsize=2573496 CPUtime=4720.07 cores=0-7
/proc/24476/task/24480/stat : 24480 (claspmt) R 24474 24476 24450 0 -1 4202560 28495 0 0 0 471967 40 0 0 25 0 8 0 738951037 2635259904 592077 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=24476/tid=24481] ppid=24474 vsize=2573496 CPUtime=4720.04 cores=0-7
/proc/24476/task/24481/stat : 24481 (claspmt) R 24474 24476 24450 0 -1 4202560 37274 0 0 0 471968 36 0 0 25 0 8 0 738951037 2635259904 592077 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=24476/tid=24482] ppid=24474 vsize=2573496 CPUtime=4720.48 cores=0-7
/proc/24476/task/24482/stat : 24482 (claspmt) R 24474 24476 24450 0 -1 4202560 34036 0 0 0 472015 33 0 0 25 0 8 0 738951037 2635259904 592077 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=24476/tid=24483] ppid=24474 vsize=2573496 CPUtime=4690.66 cores=0-7
/proc/24476/task/24483/stat : 24483 (claspmt) R 24474 24476 24450 0 -1 4202560 29227 0 0 0 469031 35 0 0 25 0 8 0 738951037 2635259904 592077 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24476/tid=24484] ppid=24474 vsize=2573496 CPUtime=4699.19 cores=0-7
/proc/24476/task/24484/stat : 24484 (claspmt) R 24474 24476 24450 0 -1 4202560 257189 0 0 0 469622 297 0 0 25 0 8 0 738951037 2635259904 592077 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37692.5
Current children cumulated vsize (KiB) 2573496

[startup+4782.3 s]
/proc/loadavg: 8.26 8.11 8.02 9/433 24627
/proc/meminfo: memFree=14805364/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=2573496 CPUtime=38170.7 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 602796 0 0 0 3816382 687 0 0 25 0 8 0 738950865 2635259904 592180 33554432000 4194304 4964800 140736255868224 18446744073709551615 4321016 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/24476/statm: 643374 592180 466 189 0 637723 0
[pid=24476/tid=24478] ppid=24474 vsize=2573496 CPUtime=4772.8 cores=0-7
/proc/24476/task/24478/stat : 24478 (claspmt) R 24474 24476 24450 0 -1 4202560 69161 0 0 0 477209 71 0 0 25 0 8 0 738951037 2635259904 592180 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670434 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=24476/tid=24479] ppid=24474 vsize=2573496 CPUtime=4778.45 cores=0-7
/proc/24476/task/24479/stat : 24479 (claspmt) R 24474 24476 24450 0 -1 4202560 44808 0 0 0 477807 38 0 0 25 0 8 0 738951037 2635259904 592180 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=24476/tid=24480] ppid=24474 vsize=2573496 CPUtime=4780.05 cores=0-7
/proc/24476/task/24480/stat : 24480 (claspmt) R 24474 24476 24450 0 -1 4202560 28496 0 0 0 477965 40 0 0 25 0 8 0 738951037 2635259904 592180 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=24476/tid=24481] ppid=24474 vsize=2573496 CPUtime=4779.23 cores=0-7
/proc/24476/task/24481/stat : 24481 (claspmt) R 24474 24476 24450 0 -1 4202560 37371 0 0 0 477886 37 0 0 25 0 8 0 738951037 2635259904 592180 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=24476/tid=24482] ppid=24474 vsize=2573496 CPUtime=4780.47 cores=0-7
/proc/24476/task/24482/stat : 24482 (claspmt) R 24474 24476 24450 0 -1 4202560 34037 0 0 0 478014 33 0 0 25 0 8 0 738951037 2635259904 592180 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670588 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=24476/tid=24483] ppid=24474 vsize=2573496 CPUtime=4750.65 cores=0-7
/proc/24476/task/24483/stat : 24483 (claspmt) R 24474 24476 24450 0 -1 4202560 29231 0 0 0 475030 35 0 0 25 0 8 0 738951037 2635259904 592180 33554432000 4194304 4964800 140736255868224 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24476/tid=24484] ppid=24474 vsize=2573496 CPUtime=4759.17 cores=0-7
/proc/24476/task/24484/stat : 24484 (claspmt) R 24474 24476 24450 0 -1 4202560 257189 0 0 0 475620 297 0 0 25 0 8 0 738951037 2635259904 592180 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38170.7
Current children cumulated vsize (KiB) 2573496

[startup+4842.3 s]
/proc/loadavg: 8.14 8.10 8.02 9/433 24629
/proc/meminfo: memFree=14805080/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=2573496 CPUtime=38649.8 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 602910 0 0 0 3864290 688 0 0 25 0 8 0 738950865 2635259904 592294 33554432000 4194304 4964800 140736255868224 18446744073709551615 4627272 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/24476/statm: 643374 592294 466 189 0 637723 0
[pid=24476/tid=24478] ppid=24474 vsize=2573496 CPUtime=4831.92 cores=0-7
/proc/24476/task/24478/stat : 24478 (claspmt) R 24474 24476 24450 0 -1 4202560 69161 0 0 0 483121 71 0 0 25 0 8 0 738951037 2635259904 592294 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=24476/tid=24479] ppid=24474 vsize=2573496 CPUtime=4838.44 cores=0-7
/proc/24476/task/24479/stat : 24479 (claspmt) R 24474 24476 24450 0 -1 4202560 44808 0 0 0 483806 38 0 0 25 0 8 0 738951037 2635259904 592294 33554432000 4194304 4964800 140736255868224 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=24476/tid=24480] ppid=24474 vsize=2573496 CPUtime=4840.03 cores=0-7
/proc/24476/task/24480/stat : 24480 (claspmt) R 24474 24476 24450 0 -1 4202560 28498 0 0 0 483962 41 0 0 25 0 8 0 738951037 2635259904 592294 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=24476/tid=24481] ppid=24474 vsize=2573496 CPUtime=4839.23 cores=0-7
/proc/24476/task/24481/stat : 24481 (claspmt) R 24474 24476 24450 0 -1 4202560 37482 0 0 0 483886 37 0 0 25 0 8 0 738951037 2635259904 592294 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=24476/tid=24482] ppid=24474 vsize=2573496 CPUtime=4840.47 cores=0-7
/proc/24476/task/24482/stat : 24482 (claspmt) R 24474 24476 24450 0 -1 4202560 34038 0 0 0 484014 33 0 0 25 0 8 0 738951037 2635259904 592294 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=24476/tid=24483] ppid=24474 vsize=2573496 CPUtime=4810.64 cores=0-7
/proc/24476/task/24483/stat : 24483 (claspmt) R 24474 24476 24450 0 -1 4202560 29231 0 0 0 481029 35 0 0 25 0 8 0 738951037 2635259904 592294 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24476/tid=24484] ppid=24474 vsize=2573496 CPUtime=4819.18 cores=0-7
/proc/24476/task/24484/stat : 24484 (claspmt) R 24474 24476 24450 0 -1 4202560 257189 0 0 0 481620 298 0 0 25 0 8 0 738951037 2635259904 592294 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38649.8
Current children cumulated vsize (KiB) 2573496

[startup+4902.3 s]
/proc/loadavg: 8.09 8.09 8.02 9/433 24629
/proc/meminfo: memFree=14805152/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=2573496 CPUtime=39128.8 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 603088 0 0 0 3912195 689 0 0 25 0 8 0 738950865 2635259904 592472 33554432000 4194304 4964800 140736255868224 18446744073709551615 4627204 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/24476/statm: 643374 592472 466 189 0 637723 0
[pid=24476/tid=24478] ppid=24474 vsize=2573496 CPUtime=4891.43 cores=0-7
/proc/24476/task/24478/stat : 24478 (claspmt) R 24474 24476 24450 0 -1 4202560 69298 0 0 0 489072 71 0 0 25 0 8 0 738951037 2635259904 592472 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24476/tid=24479] ppid=24474 vsize=2573496 CPUtime=4898.43 cores=0-7
/proc/24476/task/24479/stat : 24479 (claspmt) R 24474 24476 24450 0 -1 4202560 44837 0 0 0 489805 38 0 0 25 0 8 0 738951037 2635259904 592472 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=24476/tid=24480] ppid=24474 vsize=2573496 CPUtime=4900.03 cores=0-7
/proc/24476/task/24480/stat : 24480 (claspmt) R 24474 24476 24450 0 -1 4202560 28498 0 0 0 489962 41 0 0 25 0 8 0 738951037 2635259904 592472 33554432000 4194304 4964800 140736255868224 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=24476/tid=24481] ppid=24474 vsize=2573496 CPUtime=4899.17 cores=0-7
/proc/24476/task/24481/stat : 24481 (claspmt) R 24474 24476 24450 0 -1 4202560 37494 0 0 0 489880 37 0 0 25 0 8 0 738951037 2635259904 592472 33554432000 4194304 4964800 140736255868224 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=24476/tid=24482] ppid=24474 vsize=2573496 CPUtime=4900.47 cores=0-7
/proc/24476/task/24482/stat : 24482 (claspmt) R 24474 24476 24450 0 -1 4202560 34038 0 0 0 490014 33 0 0 25 0 8 0 738951037 2635259904 592472 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=24476/tid=24483] ppid=24474 vsize=2573496 CPUtime=4870.64 cores=0-7
/proc/24476/task/24483/stat : 24483 (claspmt) R 24474 24476 24450 0 -1 4202560 29231 0 0 0 487029 35 0 0 25 0 8 0 738951037 2635259904 592472 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24476/tid=24484] ppid=24474 vsize=2573496 CPUtime=4879.18 cores=0-7
/proc/24476/task/24484/stat : 24484 (claspmt) R 24474 24476 24450 0 -1 4202560 257189 0 0 0 487620 298 0 0 25 0 8 0 738951037 2635259904 592472 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39128.8
Current children cumulated vsize (KiB) 2573496

[startup+4962.3 s]
/proc/loadavg: 8.11 8.10 8.03 9/433 24631
/proc/meminfo: memFree=14804612/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=2573496 CPUtime=39607.9 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 603256 0 0 0 3960104 690 0 0 25 0 8 0 738950865 2635259904 592640 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/24476/statm: 643374 592640 466 189 0 637723 0
[pid=24476/tid=24478] ppid=24474 vsize=2573496 CPUtime=4951.43 cores=0-7
/proc/24476/task/24478/stat : 24478 (claspmt) R 24474 24476 24450 0 -1 4202560 69344 0 0 0 495072 71 0 0 25 0 8 0 738951037 2635259904 592640 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670349 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24476/tid=24479] ppid=24474 vsize=2573496 CPUtime=4958.41 cores=0-7
/proc/24476/task/24479/stat : 24479 (claspmt) R 24474 24476 24450 0 -1 4202560 44914 0 0 0 495803 38 0 0 25 0 8 0 738951037 2635259904 592640 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=24476/tid=24480] ppid=24474 vsize=2573496 CPUtime=4960.03 cores=0-7
/proc/24476/task/24480/stat : 24480 (claspmt) R 24474 24476 24450 0 -1 4202560 28498 0 0 0 495962 41 0 0 25 0 8 0 738951037 2635259904 592640 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=24476/tid=24481] ppid=24474 vsize=2573496 CPUtime=4959.17 cores=0-7
/proc/24476/task/24481/stat : 24481 (claspmt) R 24474 24476 24450 0 -1 4202560 37494 0 0 0 495880 37 0 0 25 0 8 0 738951037 2635259904 592640 33554432000 4194304 4964800 140736255868224 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=24476/tid=24482] ppid=24474 vsize=2573496 CPUtime=4960.47 cores=0-7
/proc/24476/task/24482/stat : 24482 (claspmt) R 24474 24476 24450 0 -1 4202560 34082 0 0 0 496014 33 0 0 25 0 8 0 738951037 2635259904 592640 33554432000 4194304 4964800 140736255868224 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=24476/tid=24483] ppid=24474 vsize=2573496 CPUtime=4930.64 cores=0-7
/proc/24476/task/24483/stat : 24483 (claspmt) R 24474 24476 24450 0 -1 4202560 29232 0 0 0 493029 35 0 0 25 0 8 0 738951037 2635259904 592640 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24476/tid=24484] ppid=24474 vsize=2573496 CPUtime=4939.18 cores=0-7
/proc/24476/task/24484/stat : 24484 (claspmt) R 24474 24476 24450 0 -1 4202560 257189 0 0 0 493620 298 0 0 25 0 8 0 738951037 2635259904 592640 33554432000 4194304 4964800 140736255868224 18446744073709551615 4320916 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39607.9
Current children cumulated vsize (KiB) 2573496



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5011.4 s]
/proc/loadavg: 8.09 8.10 8.03 9/433 24633
/proc/meminfo: memFree=14792928/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=2583904 CPUtime=40000.1 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 606315 0 0 0 3999323 691 0 0 25 0 8 0 738950865 2645917696 595699 33554432000 4194304 4964800 140736255868224 18446744073709551615 4659391 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/24476/statm: 645976 595699 466 189 0 640325 0
[pid=24476/tid=24478] ppid=24474 vsize=2583904 CPUtime=5000.55 cores=0-7
/proc/24476/task/24478/stat : 24478 (claspmt) R 24474 24476 24450 0 -1 4202560 69555 0 0 0 499983 72 0 0 25 0 8 0 738951037 2645917696 595699 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24476/tid=24479] ppid=24474 vsize=2583904 CPUtime=5007.52 cores=0-7
/proc/24476/task/24479/stat : 24479 (claspmt) R 24474 24476 24450 0 -1 4202560 45032 0 0 0 500714 38 0 0 25 0 8 0 738951037 2645917696 595699 33554432000 4194304 4964800 140736255868224 18446744073709551615 4362938 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=24476/tid=24480] ppid=24474 vsize=2583904 CPUtime=5009.15 cores=0-7
/proc/24476/task/24480/stat : 24480 (claspmt) R 24474 24476 24450 0 -1 4202560 28512 0 0 0 500873 42 0 0 25 0 8 0 738951037 2645917696 595699 33554432000 4194304 4964800 140736255868224 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=24476/tid=24481] ppid=24474 vsize=2583904 CPUtime=5008.29 cores=0-7
/proc/24476/task/24481/stat : 24481 (claspmt) R 24474 24476 24450 0 -1 4202560 37719 0 0 0 500792 37 0 0 25 0 8 0 738951037 2645917696 595699 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=24476/tid=24482] ppid=24474 vsize=2583904 CPUtime=5009.59 cores=0-7
/proc/24476/task/24482/stat : 24482 (claspmt) R 24474 24476 24450 0 -1 4202560 34205 0 0 0 500926 33 0 0 25 0 8 0 738951037 2645917696 595699 33554432000 4194304 4964800 140736255868224 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=24476/tid=24483] ppid=24474 vsize=2583904 CPUtime=4979.76 cores=0-7
/proc/24476/task/24483/stat : 24483 (claspmt) R 24474 24476 24450 0 -1 4202560 29368 0 0 0 497941 35 0 0 25 0 8 0 738951037 2645917696 595699 33554432000 4194304 4964800 140736255868224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24476/tid=24484] ppid=24474 vsize=2583904 CPUtime=4988.29 cores=0-7
/proc/24476/task/24484/stat : 24484 (claspmt) R 24474 24476 24450 0 -1 4202560 259388 0 0 0 498530 299 0 0 25 0 8 0 738951037 2645917696 595699 33554432000 4194304 4964800 140736255868224 18446744073709551615 4365711 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 2583904

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5011.8 s]
/proc/loadavg: 8.09 8.10 8.03 2/427 24634
/proc/meminfo: memFree=14792976/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=2575368 CPUtime=40000.6 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 606325 0 0 0 3999365 691 0 0 25 0 1 0 738950865 2637176832 593499 33554432000 4194304 4964800 140736255868224 18446744073709551615 224284582894 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/24476/statm: 643842 593499 476 189 0 638191 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 2575368

[startup+5012.2 s]
/proc/loadavg: 8.09 8.10 8.03 2/427 24634
/proc/meminfo: memFree=14792976/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=2567164 CPUtime=40000.9 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 606325 0 0 0 3999404 691 0 0 25 0 1 0 738950865 2628775936 591236 33554432000 4194304 4964800 140736255868224 18446744073709551615 224284582043 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/24476/statm: 641791 591236 476 189 0 636140 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 2567164

[startup+5012.3 s]
/proc/loadavg: 8.09 8.10 8.03 2/427 24634
/proc/meminfo: memFree=14792976/32951124 swapFree=45193372/67111528
[pid=24476] ppid=24474 vsize=1056308 CPUtime=40001.1 cores=0-7
/proc/24476/stat : 24476 (claspmt) R 24474 24476 24450 0 -1 4202496 606325 0 0 0 3999408 698 0 0 25 0 1 0 738950865 1081659392 189671 33554432000 4194304 4964800 140736255868224 18446744073709551615 224284969095 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/24476/statm: 264077 189671 476 189 0 258426 0
Current children cumulated CPU time (s) 40001.1
Current children cumulated vsize (KiB) 1056308

Child status: 0
Real time (s): 5012.34
CPU time (s): 40001.1
CPU user time (s): 39994.1
CPU system time (s): 7.02093
CPU usage (%): 798.053
Max. virtual memory (cumulated for all children) (KiB): 2583904

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39994.1
system time used= 7.02093
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 606326
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 87602
involuntary context switches= 189261

runsolver used 20.3059 second user time and 52.583 second system time

The end

Launcher Data 

Begin job on node118 at 2011-05-15 21:20:02
IDJOB=3380693
IDBENCH=71345
IDSOLVER=1865
FILE ID=node118/3380693-1305487202
RUNJOBID= node118-1305487202-24464
PBS_JOBID= 13325385
Free space on /tmp= 73264 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/random/forced/rbsat-v2640c305320gyes2.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3380693-1305487202/watcher-3380693-1305487202 -o /tmp/evaluation-result-3380693-1305487202/solver-3380693-1305487202 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380693-1305487202.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= a931e6258afb576eab04f38421c07518
RANDOM SEED=823487786

node118.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5374.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      17175308 kB
Buffers:        311084 kB
Cached:        4719416 kB
SwapCached:      22076 kB
Active:       12811232 kB
Inactive:      2635804 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      17175308 kB
SwapTotal:    67111528 kB
SwapFree:     45193372 kB
Dirty:            4792 kB
Writeback:           0 kB
AnonPages:    10407184 kB
Mapped:          21764 kB
Slab:           159040 kB
PageTables:      98076 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 125841112 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73260 MiB
End job on node118 at 2011-05-15 22:43:35