Trace number 3322657

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40000.5 5007.95

General information on the benchmark

NameSAT05/crafted/sabharwal05/counting/fphp/unsat/
harder/fphp-020-019.shuffled-as.sat05-1232.cnf
MD5SUMabb32bff5eb954e790612a6209864b5d
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark2.08668
Satisfiable
(Un)Satisfiability was proved
Number of variables380
Number of clauses7050
Sum of the clauses size14440
Maximum clause length19
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 27030
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 520

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...392316/instance-3322657-1304392316.cnf
0.00/0.02	c Solving...
40000.45/5007.91	
40000.45/5007.91	*** INTERRUPTED! ***
40000.45/5007.91	*** Shutting down threads...
40000.45/5007.94	s UNKNOWN
40000.45/5007.94	
40000.45/5007.94	c INTERRUPTED   : 1
40000.45/5007.94	c Models        : 0+    
40000.45/5007.94	c Time          : 5007.913s (Solving: 5007.89s 1st Model: 0.00s Unsat: 0.00s)
40000.45/5007.94	c CPU Time      : 40000.490s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3322657-1304392316/watcher-3322657-1304392316 -o /tmp/evaluation-result-3322657-1304392316/solver-3322657-1304392316 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322657-1304392316.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.06 8.06 8.02 1/274 23020
/proc/meminfo: memFree=17402376/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=21472 CPUtime=0 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 665 0 0 0 0 0 0 0 25 0 1 0 550829605 21987328 592 33554432000 4194304 4964704 140734854192896 18446744073709551615 4607736 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/23020/statm: 5368 593 400 189 0 242 0

[startup+0.0515849 s]
/proc/loadavg: 8.06 8.06 8.02 1/274 23020
/proc/meminfo: memFree=17402376/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=382104 CPUtime=0.19 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 1900 0 0 0 18 1 0 0 25 0 8 0 550829605 391274496 1820 33554432000 4194304 4964704 140734854192896 18446744073709551615 4653356 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/23020/statm: 95526 1821 464 189 0 89875 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 382104

[startup+0.103294 s]
/proc/loadavg: 8.06 8.06 8.02 1/274 23020
/proc/meminfo: memFree=17402376/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=382368 CPUtime=0.53 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 2497 0 0 0 51 2 0 0 25 0 8 0 550829605 391544832 2417 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320970 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/23020/statm: 95592 2417 464 189 0 89941 0
Current children cumulated CPU time (s) 0.53
Current children cumulated vsize (KiB) 382368

[startup+0.300227 s]
/proc/loadavg: 8.06 8.06 8.02 1/274 23020
/proc/meminfo: memFree=17402376/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=383560 CPUtime=2.09 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 3858 0 0 0 207 2 0 0 18 0 8 0 550829605 392765440 3778 33554432000 4194304 4964704 140734854192896 18446744073709551615 4644813 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/23020/statm: 95890 3778 464 189 0 90239 0
Current children cumulated CPU time (s) 2.09
Current children cumulated vsize (KiB) 383560

[startup+0.700157 s]
/proc/loadavg: 8.06 8.06 8.02 1/274 23020
/proc/meminfo: memFree=17402376/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=449624 CPUtime=5.28 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 6120 0 0 0 524 4 0 0 18 0 8 0 550829605 460414976 6040 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670215 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/23020/statm: 112406 6040 464 189 0 106755 0
Current children cumulated CPU time (s) 5.28
Current children cumulated vsize (KiB) 449624

[startup+1.50101 s]
/proc/loadavg: 8.06 8.06 8.02 9/282 23028
/proc/meminfo: memFree=17378636/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=450944 CPUtime=11.66 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 8192 0 0 0 1160 6 0 0 19 0 8 0 550829605 461766656 8112 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320966 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/23020/statm: 112736 8112 464 189 0 107085 0
[pid=23020/tid=23022] ppid=23018 vsize=450944 CPUtime=1.46 cores=0-7
/proc/23020/task/23022/stat : 23022 (claspmt) R 23018 23020 22776 0 -1 4202560 870 0 0 0 146 0 0 0 20 0 8 0 550829607 461766656 8113 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670241 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23020/tid=23023] ppid=23018 vsize=450944 CPUtime=1.46 cores=0-7
/proc/23020/task/23023/stat : 23023 (claspmt) R 23018 23020 22776 0 -1 4202560 1126 0 0 0 145 1 0 0 20 0 8 0 550829607 461766656 8114 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=23020/tid=23024] ppid=23018 vsize=450944 CPUtime=1.43 cores=0-7
/proc/23020/task/23024/stat : 23024 (claspmt) R 23018 23020 22776 0 -1 4202560 971 0 0 0 143 0 0 0 25 0 8 0 550829607 461766656 8114 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23020/tid=23025] ppid=23018 vsize=450944 CPUtime=1.45 cores=0-7
/proc/23020/task/23025/stat : 23025 (claspmt) R 23018 23020 22776 0 -1 4202560 1111 0 0 0 145 0 0 0 19 0 8 0 550829607 461766656 8114 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670288 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23020/tid=23026] ppid=23018 vsize=450944 CPUtime=1.46 cores=0-7
/proc/23020/task/23026/stat : 23026 (claspmt) R 23018 23020 22776 0 -1 4202560 901 0 0 0 145 1 0 0 19 0 8 0 550829607 461766656 8114 33554432000 4194304 4964704 140734854192896 18446744073709551615 228122375201 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23020/tid=23027] ppid=23018 vsize=450944 CPUtime=1.46 cores=0-7
/proc/23020/task/23027/stat : 23027 (claspmt) R 23018 23020 22776 0 -1 4202560 1006 0 0 0 146 0 0 0 19 0 8 0 550829607 461766656 8115 33554432000 4194304 4964704 140734854192896 18446744073709551615 228122375201 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23020/tid=23028] ppid=23018 vsize=450944 CPUtime=1.46 cores=0-7
/proc/23020/task/23028/stat : 23028 (claspmt) R 23018 23020 22776 0 -1 4202560 800 0 0 0 146 0 0 0 18 0 8 0 550829607 461766656 8115 33554432000 4194304 4964704 140734854192896 18446744073709551615 4673205 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 11.66
Current children cumulated vsize (KiB) 450944

[startup+3.10073 s]
/proc/loadavg: 8.06 8.06 8.02 9/282 23028
/proc/meminfo: memFree=17364800/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=453584 CPUtime=24.44 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 12335 0 0 0 2436 8 0 0 19 0 8 0 550829605 464470016 12255 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320976 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/23020/statm: 113396 12255 464 189 0 107745 0
[pid=23020/tid=23022] ppid=23018 vsize=453584 CPUtime=3.05 cores=0-7
/proc/23020/task/23022/stat : 23022 (claspmt) R 23018 23020 22776 0 -1 4202560 1255 0 0 0 305 0 0 0 24 0 8 0 550829607 464470016 12255 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23020/tid=23023] ppid=23018 vsize=453584 CPUtime=3.06 cores=0-7
/proc/23020/task/23023/stat : 23023 (claspmt) R 23018 23020 22776 0 -1 4202560 1617 0 0 0 305 1 0 0 25 0 8 0 550829607 464470016 12255 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=23020/tid=23024] ppid=23018 vsize=453584 CPUtime=3.02 cores=0-7
/proc/23020/task/23024/stat : 23024 (claspmt) R 23018 23020 22776 0 -1 4202560 1694 0 0 0 302 0 0 0 25 0 8 0 550829607 464470016 12255 33554432000 4194304 4964704 140734854192896 18446744073709551615 4361296 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23020/tid=23025] ppid=23018 vsize=453584 CPUtime=3.06 cores=0-7
/proc/23020/task/23025/stat : 23025 (claspmt) R 23018 23020 22776 0 -1 4202560 1612 0 0 0 305 1 0 0 23 0 8 0 550829607 464470016 12255 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23020/tid=23026] ppid=23018 vsize=453584 CPUtime=3.06 cores=0-7
/proc/23020/task/23026/stat : 23026 (claspmt) R 23018 23020 22776 0 -1 4202560 1400 0 0 0 305 1 0 0 23 0 8 0 550829607 464470016 12255 33554432000 4194304 4964704 140734854192896 18446744073709551615 4668942 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23020/tid=23027] ppid=23018 vsize=453584 CPUtime=3.05 cores=0-7
/proc/23020/task/23027/stat : 23027 (claspmt) R 23018 23020 22776 0 -1 4202560 1449 0 0 0 305 0 0 0 22 0 8 0 550829607 464470016 12255 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23020/tid=23028] ppid=23018 vsize=453584 CPUtime=3.05 cores=0-7
/proc/23020/task/23028/stat : 23028 (claspmt) R 23018 23020 22776 0 -1 4202560 1195 0 0 0 305 0 0 0 21 0 8 0 550829607 464470016 12255 33554432000 4194304 4964704 140734854192896 18446744073709551615 4365714 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 24.44
Current children cumulated vsize (KiB) 453584

[startup+6.30116 s]
/proc/loadavg: 8.06 8.06 8.02 9/282 23028
/proc/meminfo: memFree=17349308/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=454856 CPUtime=50 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 14793 0 0 0 4989 11 0 0 25 0 8 0 550829605 465772544 14681 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670266 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/23020/statm: 113714 14681 464 189 0 108063 0
[pid=23020/tid=23022] ppid=23018 vsize=454856 CPUtime=6.26 cores=0-7
/proc/23020/task/23022/stat : 23022 (claspmt) R 23018 23020 22776 0 -1 4202560 1466 0 0 0 625 1 0 0 19 0 8 0 550829607 465772544 14681 33554432000 4194304 4964704 140734854192896 18446744073709551615 4332324 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23020/tid=23023] ppid=23018 vsize=454856 CPUtime=6.25 cores=0-7
/proc/23020/task/23023/stat : 23023 (claspmt) R 23018 23020 22776 0 -1 4202560 2035 0 0 0 624 1 0 0 18 0 8 0 550829607 465772544 14681 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=23020/tid=23024] ppid=23018 vsize=454856 CPUtime=6.23 cores=0-7
/proc/23020/task/23024/stat : 23024 (claspmt) R 23018 23020 22776 0 -1 4202560 2206 0 0 0 622 1 0 0 25 0 8 0 550829607 465772544 14681 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23020/tid=23025] ppid=23018 vsize=454856 CPUtime=6.26 cores=0-7
/proc/23020/task/23025/stat : 23025 (claspmt) R 23018 23020 22776 0 -1 4202560 1766 0 0 0 625 1 0 0 25 0 8 0 550829607 465772544 14681 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23020/tid=23026] ppid=23018 vsize=454856 CPUtime=6.26 cores=0-7
/proc/23020/task/23026/stat : 23026 (claspmt) R 23018 23020 22776 0 -1 4202560 1683 0 0 0 625 1 0 0 25 0 8 0 550829607 465772544 14681 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23020/tid=23027] ppid=23018 vsize=454856 CPUtime=6.26 cores=0-7
/proc/23020/task/23027/stat : 23027 (claspmt) R 23018 23020 22776 0 -1 4202560 1798 0 0 0 625 1 0 0 25 0 8 0 550829607 465772544 14681 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23020/tid=23028] ppid=23018 vsize=454856 CPUtime=6.26 cores=0-7
/proc/23020/task/23028/stat : 23028 (claspmt) R 23018 23020 22776 0 -1 4202560 1408 0 0 0 625 1 0 0 25 0 8 0 550829607 465772544 14681 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 50
Current children cumulated vsize (KiB) 454856

[startup+12.701 s]
/proc/loadavg: 8.05 8.06 8.02 9/282 23028
/proc/meminfo: memFree=17331516/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=457468 CPUtime=101.1 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 18670 0 0 0 10095 15 0 0 25 0 8 0 550829605 468447232 18558 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670260 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/23020/statm: 114367 18558 464 189 0 108716 0
[pid=23020/tid=23022] ppid=23018 vsize=457468 CPUtime=12.65 cores=0-7
/proc/23020/task/23022/stat : 23022 (claspmt) R 23018 23020 22776 0 -1 4202560 1797 0 0 0 1263 2 0 0 25 0 8 0 550829607 468447232 18558 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23020/tid=23023] ppid=23018 vsize=457468 CPUtime=12.64 cores=0-7
/proc/23020/task/23023/stat : 23023 (claspmt) R 23018 23020 22776 0 -1 4202560 2512 0 0 0 1263 1 0 0 18 0 8 0 550829607 468447232 18558 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=23020/tid=23024] ppid=23018 vsize=457468 CPUtime=12.62 cores=0-7
/proc/23020/task/23024/stat : 23024 (claspmt) R 23018 23020 22776 0 -1 4202560 3053 0 0 0 1261 1 0 0 25 0 8 0 550829607 468447232 18558 33554432000 4194304 4964704 140734854192896 18446744073709551615 4660115 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23020/tid=23025] ppid=23018 vsize=457468 CPUtime=12.66 cores=0-7
/proc/23020/task/23025/stat : 23025 (claspmt) R 23018 23020 22776 0 -1 4202560 2457 0 0 0 1264 2 0 0 25 0 8 0 550829607 468447232 18558 33554432000 4194304 4964704 140734854192896 18446744073709551615 4642509 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23020/tid=23026] ppid=23018 vsize=457468 CPUtime=12.66 cores=0-7
/proc/23020/task/23026/stat : 23026 (claspmt) R 23018 23020 22776 0 -1 4202560 2071 0 0 0 1264 2 0 0 25 0 8 0 550829607 468447232 18558 33554432000 4194304 4964704 140734854192896 18446744073709551615 4338784 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23020/tid=23027] ppid=23018 vsize=457468 CPUtime=12.65 cores=0-7
/proc/23020/task/23027/stat : 23027 (claspmt) R 23018 23020 22776 0 -1 4202560 2129 0 0 0 1264 1 0 0 25 0 8 0 550829607 468447232 18558 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23020/tid=23028] ppid=23018 vsize=457468 CPUtime=12.65 cores=0-7
/proc/23020/task/23028/stat : 23028 (claspmt) R 23018 23020 22776 0 -1 4202560 1613 0 0 0 1264 1 0 0 25 0 8 0 550829607 468447232 18558 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 101.1
Current children cumulated vsize (KiB) 457468

[startup+25.5007 s]
/proc/loadavg: 8.04 8.06 8.02 9/282 23028
/proc/meminfo: memFree=17307348/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=461088 CPUtime=203.33 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 24657 0 0 0 20312 21 0 0 25 0 8 0 550829605 472154112 24533 33554432000 4194304 4964704 140734854192896 18446744073709551615 4342199 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/23020/statm: 115272 24533 464 189 0 109621 0
[pid=23020/tid=23022] ppid=23018 vsize=461088 CPUtime=25.43 cores=0-7
/proc/23020/task/23022/stat : 23022 (claspmt) R 23018 23020 22776 0 -1 4202560 2328 0 0 0 2540 3 0 0 25 0 8 0 550829607 472154112 24533 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670416 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23020/tid=23023] ppid=23018 vsize=461088 CPUtime=25.44 cores=0-7
/proc/23020/task/23023/stat : 23023 (claspmt) R 23018 23020 22776 0 -1 4202560 3151 0 0 0 2542 2 0 0 25 0 8 0 550829607 472154112 24533 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670466 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=23020/tid=23024] ppid=23018 vsize=461088 CPUtime=25.42 cores=0-7
/proc/23020/task/23024/stat : 23024 (claspmt) R 23018 23020 22776 0 -1 4202560 4305 0 0 0 2539 3 0 0 25 0 8 0 550829607 472154112 24533 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23020/tid=23025] ppid=23018 vsize=461088 CPUtime=25.45 cores=0-7
/proc/23020/task/23025/stat : 23025 (claspmt) R 23018 23020 22776 0 -1 4202560 3310 0 0 0 2542 3 0 0 25 0 8 0 550829607 472154112 24533 33554432000 4194304 4964704 140734854192896 18446744073709551615 4683929 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23020/tid=23026] ppid=23018 vsize=461088 CPUtime=25.45 cores=0-7
/proc/23020/task/23026/stat : 23026 (claspmt) R 23018 23020 22776 0 -1 4202560 2762 0 0 0 2542 3 0 0 25 0 8 0 550829607 472154112 24533 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320934 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23020/tid=23027] ppid=23018 vsize=461088 CPUtime=25.45 cores=0-7
/proc/23020/task/23027/stat : 23027 (claspmt) R 23018 23020 22776 0 -1 4202560 3038 0 0 0 2543 2 0 0 25 0 8 0 550829607 472154112 24533 33554432000 4194304 4964704 140734854192896 18446744073709551615 4342232 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23020/tid=23028] ppid=23018 vsize=461088 CPUtime=25.45 cores=0-7
/proc/23020/task/23028/stat : 23028 (claspmt) R 23018 23020 22776 0 -1 4202560 2115 0 0 0 2544 1 0 0 18 0 8 0 550829607 472154112 24533 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 203.33
Current children cumulated vsize (KiB) 461088

[startup+51.1011 s]
/proc/loadavg: 8.02 8.05 8.01 9/282 23029
/proc/meminfo: memFree=17281312/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=464556 CPUtime=407.79 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 31146 0 0 0 40746 33 0 0 25 0 8 0 550829605 475705344 31014 33554432000 4194304 4964704 140734854192896 18446744073709551615 4325305 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/23020/statm: 116139 31014 464 189 0 110488 0
[pid=23020/tid=23022] ppid=23018 vsize=464556 CPUtime=50.98 cores=0-7
/proc/23020/task/23022/stat : 23022 (claspmt) R 23018 23020 22776 0 -1 4202560 3259 0 0 0 5094 4 0 0 25 0 8 0 550829607 475705344 31014 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23020/tid=23023] ppid=23018 vsize=464556 CPUtime=51.02 cores=0-7
/proc/23020/task/23023/stat : 23023 (claspmt) R 23018 23020 22776 0 -1 4202560 4022 0 0 0 5099 3 0 0 25 0 8 0 550829607 475705344 31014 33554432000 4194304 4964704 140734854192896 18446744073709551615 4363366 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=23020/tid=23024] ppid=23018 vsize=464556 CPUtime=51 cores=0-7
/proc/23020/task/23024/stat : 23024 (claspmt) R 23018 23020 22776 0 -1 4202560 5476 0 0 0 5096 4 0 0 25 0 8 0 550829607 475705344 31014 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23020/tid=23025] ppid=23018 vsize=464556 CPUtime=51.02 cores=0-7
/proc/23020/task/23025/stat : 23025 (claspmt) R 23018 23020 22776 0 -1 4202560 4244 0 0 0 5098 4 0 0 25 0 8 0 550829607 475705344 31014 33554432000 4194304 4964704 140734854192896 18446744073709551615 4327267 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=23020/tid=23026] ppid=23018 vsize=464556 CPUtime=51.04 cores=0-7
/proc/23020/task/23026/stat : 23026 (claspmt) R 23018 23020 22776 0 -1 4202560 3950 0 0 0 5099 5 0 0 25 0 8 0 550829607 475705344 31014 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23020/tid=23027] ppid=23018 vsize=464556 CPUtime=51.03 cores=0-7
/proc/23020/task/23027/stat : 23027 (claspmt) R 23018 23020 22776 0 -1 4202560 3751 0 0 0 5100 3 0 0 25 0 8 0 550829607 475705344 31014 33554432000 4194304 4964704 140734854192896 18446744073709551615 4332387 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23020/tid=23028] ppid=23018 vsize=464556 CPUtime=51.04 cores=0-7
/proc/23020/task/23028/stat : 23028 (claspmt) R 23018 23020 22776 0 -1 4202560 2397 0 0 0 5102 2 0 0 25 0 8 0 550829607 475705344 31014 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670215 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 407.79

################
# More data... #
################


[startup+4722.31 s]
/proc/loadavg: 8.15 8.09 8.01 9/282 23165
/proc/meminfo: memFree=17094636/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=492300 CPUtime=37719.2 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 113979 0 0 0 3770497 1423 0 0 25 0 8 0 550829605 504115200 76672 33554432000 4194304 4964704 140734854192896 18446744073709551615 4332351 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/23020/statm: 123075 76672 464 189 0 117424 0
[pid=23020/tid=23022] ppid=23018 vsize=492300 CPUtime=4707.99 cores=0-7
/proc/23020/task/23022/stat : 23022 (claspmt) R 23018 23020 22776 0 -1 4202560 15204 0 0 0 470530 269 0 0 18 0 8 0 550829607 504115200 76672 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=23020/tid=23023] ppid=23018 vsize=492300 CPUtime=4718.5 cores=0-7
/proc/23020/task/23023/stat : 23023 (claspmt) R 23018 23020 22776 0 -1 4202560 17101 0 0 0 471666 184 0 0 25 0 8 0 550829607 504115200 76672 33554432000 4194304 4964704 140734854192896 18446744073709551615 4660077 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23020/tid=23024] ppid=23018 vsize=492300 CPUtime=4720.24 cores=0-7
/proc/23020/task/23024/stat : 23024 (claspmt) R 23018 23020 22776 0 -1 4202560 19769 0 0 0 471854 170 0 0 19 0 8 0 550829607 504115200 76672 33554432000 4194304 4964704 140734854192896 18446744073709551615 4659143 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23020/tid=23025] ppid=23018 vsize=492300 CPUtime=4720.07 cores=0-7
/proc/23020/task/23025/stat : 23025 (claspmt) R 23018 23020 22776 0 -1 4202560 13311 0 0 0 471853 154 0 0 18 0 8 0 550829607 504115200 76672 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23020/tid=23026] ppid=23018 vsize=492300 CPUtime=4719.99 cores=0-7
/proc/23020/task/23026/stat : 23026 (claspmt) R 23018 23020 22776 0 -1 4202560 11728 0 0 0 471840 159 0 0 21 0 8 0 550829607 504115200 76672 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320919 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23020/tid=23027] ppid=23018 vsize=492300 CPUtime=4700.65 cores=0-7
/proc/23020/task/23027/stat : 23027 (claspmt) R 23018 23020 22776 0 -1 4202560 15582 0 0 0 469888 177 0 0 25 0 8 0 550829607 504115200 76672 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23020/tid=23028] ppid=23018 vsize=492300 CPUtime=4718.73 cores=0-7
/proc/23020/task/23028/stat : 23028 (claspmt) R 23018 23020 22776 0 -1 4202560 10224 0 0 0 471719 154 0 0 25 0 8 0 550829607 504115200 76672 33554432000 4194304 4964704 140734854192896 18446744073709551615 4675387 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37719.2
Current children cumulated vsize (KiB) 492300

[startup+4782.3 s]
/proc/loadavg: 8.06 8.07 8.01 9/282 23167
/proc/meminfo: memFree=17095728/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=491860 CPUtime=38198.4 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 114129 0 0 0 3818400 1441 0 0 25 0 8 0 550829605 503664640 76618 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320955 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/23020/statm: 122965 76618 464 189 0 117314 0
[pid=23020/tid=23022] ppid=23018 vsize=491860 CPUtime=4767.92 cores=0-7
/proc/23020/task/23022/stat : 23022 (claspmt) R 23018 23020 22776 0 -1 4202560 15204 0 0 0 476520 272 0 0 25 0 8 0 550829607 503664640 76618 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670243 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=23020/tid=23023] ppid=23018 vsize=491860 CPUtime=4778.48 cores=0-7
/proc/23020/task/23023/stat : 23023 (claspmt) R 23018 23020 22776 0 -1 4202560 17120 0 0 0 477661 187 0 0 25 0 8 0 550829607 503664640 76618 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23020/tid=23024] ppid=23018 vsize=491860 CPUtime=4780.22 cores=0-7
/proc/23020/task/23024/stat : 23024 (claspmt) R 23018 23020 22776 0 -1 4202560 19769 0 0 0 477849 173 0 0 25 0 8 0 550829607 503664640 76618 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23020/tid=23025] ppid=23018 vsize=491860 CPUtime=4780.03 cores=0-7
/proc/23020/task/23025/stat : 23025 (claspmt) R 23018 23020 22776 0 -1 4202560 13311 0 0 0 477846 157 0 0 25 0 8 0 550829607 503664640 76618 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23020/tid=23026] ppid=23018 vsize=491860 CPUtime=4779.96 cores=0-7
/proc/23020/task/23026/stat : 23026 (claspmt) R 23018 23020 22776 0 -1 4202560 11728 0 0 0 477836 160 0 0 25 0 8 0 550829607 503664640 76618 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23020/tid=23027] ppid=23018 vsize=491860 CPUtime=4760.13 cores=0-7
/proc/23020/task/23027/stat : 23027 (claspmt) R 23018 23020 22776 0 -1 4202560 15713 0 0 0 475833 180 0 0 18 0 8 0 550829607 503664640 76618 33554432000 4194304 4964704 140734854192896 18446744073709551615 4321020 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23020/tid=23028] ppid=23018 vsize=491860 CPUtime=4778.7 cores=0-7
/proc/23020/task/23028/stat : 23028 (claspmt) R 23018 23020 22776 0 -1 4202560 10224 0 0 0 477714 156 0 0 25 0 8 0 550829607 503664640 76618 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670458 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38198.4
Current children cumulated vsize (KiB) 491860

[startup+4842.3 s]
/proc/loadavg: 8.06 8.07 8.01 9/282 23169
/proc/meminfo: memFree=17095792/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=491860 CPUtime=38677.6 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 114196 0 0 0 3866308 1457 0 0 25 0 8 0 550829605 503664640 76574 33554432000 4194304 4964704 140734854192896 18446744073709551615 4312171 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/23020/statm: 122965 76574 464 189 0 117314 0
[pid=23020/tid=23022] ppid=23018 vsize=491860 CPUtime=4827.86 cores=0-7
/proc/23020/task/23022/stat : 23022 (claspmt) R 23018 23020 22776 0 -1 4202560 15215 0 0 0 482512 274 0 0 25 0 8 0 550829607 503664640 76574 33554432000 4194304 4964704 140734854192896 18446744073709551615 4660085 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=23020/tid=23023] ppid=23018 vsize=491860 CPUtime=4838.46 cores=0-7
/proc/23020/task/23023/stat : 23023 (claspmt) R 23018 23020 22776 0 -1 4202560 17121 0 0 0 483657 189 0 0 25 0 8 0 550829607 503664640 76574 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23020/tid=23024] ppid=23018 vsize=491860 CPUtime=4840.18 cores=0-7
/proc/23020/task/23024/stat : 23024 (claspmt) R 23018 23020 22776 0 -1 4202560 19769 0 0 0 483844 174 0 0 25 0 8 0 550829607 503664640 76574 33554432000 4194304 4964704 140734854192896 18446744073709551615 4660162 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23020/tid=23025] ppid=23018 vsize=491860 CPUtime=4840 cores=0-7
/proc/23020/task/23025/stat : 23025 (claspmt) R 23018 23020 22776 0 -1 4202560 13312 0 0 0 483840 160 0 0 18 0 8 0 550829607 503664640 76574 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23020/tid=23026] ppid=23018 vsize=491860 CPUtime=4839.93 cores=0-7
/proc/23020/task/23026/stat : 23026 (claspmt) R 23018 23020 22776 0 -1 4202560 11763 0 0 0 483831 162 0 0 25 0 8 0 550829607 503664640 76574 33554432000 4194304 4964704 140734854192896 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23020/tid=23027] ppid=23018 vsize=491860 CPUtime=4819.6 cores=0-7
/proc/23020/task/23027/stat : 23027 (claspmt) R 23018 23020 22776 0 -1 4202560 15721 0 0 0 481778 182 0 0 25 0 8 0 550829607 503664640 76574 33554432000 4194304 4964704 140734854192896 18446744073709551615 4654500 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23020/tid=23028] ppid=23018 vsize=491860 CPUtime=4838.68 cores=0-7
/proc/23020/task/23028/stat : 23028 (claspmt) R 23018 23020 22776 0 -1 4202560 10224 0 0 0 483710 158 0 0 25 0 8 0 550829607 503664640 76574 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38677.6
Current children cumulated vsize (KiB) 491860

[startup+4902.3 s]
/proc/loadavg: 8.13 8.08 8.01 9/282 23170
/proc/meminfo: memFree=17094388/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=491860 CPUtime=39156.9 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 114998 0 0 0 3914215 1476 0 0 25 0 8 0 550829605 503664640 77046 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670248 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/23020/statm: 122965 77046 464 189 0 117314 0
[pid=23020/tid=23022] ppid=23018 vsize=491860 CPUtime=4887.79 cores=0-7
/proc/23020/task/23022/stat : 23022 (claspmt) R 23018 23020 22776 0 -1 4202560 15413 0 0 0 488502 277 0 0 25 0 8 0 550829607 503664640 77046 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=23020/tid=23023] ppid=23018 vsize=491860 CPUtime=4898.43 cores=0-7
/proc/23020/task/23023/stat : 23023 (claspmt) R 23018 23020 22776 0 -1 4202560 17263 0 0 0 489652 191 0 0 25 0 8 0 550829607 503664640 77046 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23020/tid=23024] ppid=23018 vsize=491860 CPUtime=4900.14 cores=0-7
/proc/23020/task/23024/stat : 23024 (claspmt) R 23018 23020 22776 0 -1 4202560 19797 0 0 0 489837 177 0 0 25 0 8 0 550829607 503664640 77046 33554432000 4194304 4964704 140734854192896 18446744073709551615 4660156 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23020/tid=23025] ppid=23018 vsize=491860 CPUtime=4899.98 cores=0-7
/proc/23020/task/23025/stat : 23025 (claspmt) R 23018 23020 22776 0 -1 4202560 13563 0 0 0 489835 163 0 0 19 0 8 0 550829607 503664640 77046 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23020/tid=23026] ppid=23018 vsize=491860 CPUtime=4899.9 cores=0-7
/proc/23020/task/23026/stat : 23026 (claspmt) R 23018 23020 22776 0 -1 4202560 11805 0 0 0 489825 165 0 0 25 0 8 0 550829607 503664640 77046 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670193 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23020/tid=23027] ppid=23018 vsize=491860 CPUtime=4879.07 cores=0-7
/proc/23020/task/23027/stat : 23027 (claspmt) R 23018 23020 22776 0 -1 4202560 15804 0 0 0 487724 183 0 0 22 0 8 0 550829607 503664640 77046 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670255 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23020/tid=23028] ppid=23018 vsize=491860 CPUtime=4898.66 cores=0-7
/proc/23020/task/23028/stat : 23028 (claspmt) R 23018 23020 22776 0 -1 4202560 10247 0 0 0 489706 160 0 0 25 0 8 0 550829607 503664640 77046 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39156.9
Current children cumulated vsize (KiB) 491860

[startup+4962.3 s]
/proc/loadavg: 8.12 8.09 8.02 9/282 23172
/proc/meminfo: memFree=17093580/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=491860 CPUtime=39636.1 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 115817 0 0 0 3962119 1496 0 0 25 0 8 0 550829605 503664640 76928 33554432000 4194304 4964704 140734854192896 18446744073709551615 4669051 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/23020/statm: 122965 76928 464 189 0 117314 0
[pid=23020/tid=23022] ppid=23018 vsize=491860 CPUtime=4947.74 cores=0-7
/proc/23020/task/23022/stat : 23022 (claspmt) R 23018 23020 22776 0 -1 4202560 15663 0 0 0 494493 281 0 0 20 0 8 0 550829607 503664640 76928 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=23020/tid=23023] ppid=23018 vsize=491860 CPUtime=4958.4 cores=0-7
/proc/23020/task/23023/stat : 23023 (claspmt) R 23018 23020 22776 0 -1 4202560 17263 0 0 0 495646 194 0 0 25 0 8 0 550829607 503664640 76928 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23020/tid=23024] ppid=23018 vsize=491860 CPUtime=4960.12 cores=0-7
/proc/23020/task/23024/stat : 23024 (claspmt) R 23018 23020 22776 0 -1 4202560 19995 0 0 0 495832 180 0 0 25 0 8 0 550829607 503664640 76928 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23020/tid=23025] ppid=23018 vsize=491860 CPUtime=4959.97 cores=0-7
/proc/23020/task/23025/stat : 23025 (claspmt) R 23018 23020 22776 0 -1 4202560 13723 0 0 0 495831 166 0 0 22 0 8 0 550829607 503664640 76928 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670132 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23020/tid=23026] ppid=23018 vsize=491860 CPUtime=4959.87 cores=0-7
/proc/23020/task/23026/stat : 23026 (claspmt) R 23018 23020 22776 0 -1 4202560 11805 0 0 0 495820 167 0 0 20 0 8 0 550829607 503664640 76928 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670454 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23020/tid=23027] ppid=23018 vsize=491860 CPUtime=4938.55 cores=0-7
/proc/23020/task/23027/stat : 23027 (claspmt) R 23018 23020 22776 0 -1 4202560 15975 0 0 0 493669 186 0 0 25 0 8 0 550829607 503664640 76928 33554432000 4194304 4964704 140734854192896 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23020/tid=23028] ppid=23018 vsize=491860 CPUtime=4958.63 cores=0-7
/proc/23020/task/23028/stat : 23028 (claspmt) R 23018 23020 22776 0 -1 4202560 10287 0 0 0 495701 162 0 0 25 0 8 0 550829607 503664640 76928 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39636.2
Current children cumulated vsize (KiB) 491860



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5007.9 s]
/proc/loadavg: 8.10 8.09 8.02 9/282 23173
/proc/meminfo: memFree=17092732/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=491960 CPUtime=40000.4 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 116611 0 0 0 3998534 1511 0 0 25 0 8 0 550829605 503767040 77357 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670454 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/23020/statm: 122990 77357 464 189 0 117339 0
[pid=23020/tid=23022] ppid=23018 vsize=491960 CPUtime=4993.3 cores=0-7
/proc/23020/task/23022/stat : 23022 (claspmt) R 23018 23020 22776 0 -1 4202560 15748 0 0 0 499047 283 0 0 22 0 8 0 550829607 503767040 77357 33554432000 4194304 4964704 140734854192896 18446744073709551615 4653264 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=23020/tid=23023] ppid=23018 vsize=491960 CPUtime=5003.99 cores=0-7
/proc/23020/task/23023/stat : 23023 (claspmt) R 23018 23020 22776 0 -1 4202560 17316 0 0 0 500203 196 0 0 25 0 8 0 550829607 503767040 77357 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23020/tid=23024] ppid=23018 vsize=491960 CPUtime=5005.71 cores=0-7
/proc/23020/task/23024/stat : 23024 (claspmt) R 23018 23020 22776 0 -1 4202560 20165 0 0 0 500390 181 0 0 25 0 8 0 550829607 503767040 77357 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23020/tid=23025] ppid=23018 vsize=491960 CPUtime=5005.17 cores=0-7
/proc/23020/task/23025/stat : 23025 (claspmt) R 23018 23020 22776 0 -1 4202560 13755 0 0 0 500349 168 0 0 25 0 8 0 550829607 503767040 77357 33554432000 4194304 4964704 140734854192896 18446744073709551615 4658370 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23020/tid=23026] ppid=23018 vsize=491960 CPUtime=5005.44 cores=0-7
/proc/23020/task/23026/stat : 23026 (claspmt) R 23018 23020 22776 0 -1 4202560 11835 0 0 0 500375 169 0 0 25 0 8 0 550829607 503767040 77357 33554432000 4194304 4964704 140734854192896 18446744073709551615 4325338 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23020/tid=23027] ppid=23018 vsize=491960 CPUtime=4984.13 cores=0-7
/proc/23020/task/23027/stat : 23027 (claspmt) R 23018 23020 22776 0 -1 4202560 16066 0 0 0 498226 187 0 0 25 0 8 0 550829607 503767040 77357 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23020/tid=23028] ppid=23018 vsize=491960 CPUtime=5004.23 cores=0-7
/proc/23020/task/23028/stat : 23028 (claspmt) R 23018 23020 22776 0 -1 4202560 10560 0 0 0 500259 164 0 0 25 0 8 0 550829607 503767040 77357 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 491960

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5007.9 s]
/proc/loadavg: 8.10 8.09 8.02 9/282 23173
/proc/meminfo: memFree=17092732/32951124 swapFree=67111444/67111528
[pid=23020] ppid=23018 vsize=491960 CPUtime=40000.4 cores=0-7
/proc/23020/stat : 23020 (claspmt) R 23018 23020 22776 0 -1 4202496 116611 0 0 0 3998534 1511 0 0 25 0 8 0 550829605 503767040 77357 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670454 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/23020/statm: 122990 77357 464 189 0 117339 0
[pid=23020/tid=23022] ppid=23018 vsize=491960 CPUtime=4993.3 cores=0-7
/proc/23020/task/23022/stat : 23022 (claspmt) R 23018 23020 22776 0 -1 4202560 15748 0 0 0 499047 283 0 0 22 0 8 0 550829607 503767040 77357 33554432000 4194304 4964704 140734854192896 18446744073709551615 4653264 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=23020/tid=23023] ppid=23018 vsize=491960 CPUtime=5003.99 cores=0-7
/proc/23020/task/23023/stat : 23023 (claspmt) R 23018 23020 22776 0 -1 4202560 17316 0 0 0 500203 196 0 0 25 0 8 0 550829607 503767040 77357 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=23020/tid=23024] ppid=23018 vsize=491960 CPUtime=5005.71 cores=0-7
/proc/23020/task/23024/stat : 23024 (claspmt) R 23018 23020 22776 0 -1 4202560 20165 0 0 0 500390 181 0 0 25 0 8 0 550829607 503767040 77357 33554432000 4194304 4964704 140734854192896 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=23020/tid=23025] ppid=23018 vsize=491960 CPUtime=5005.17 cores=0-7
/proc/23020/task/23025/stat : 23025 (claspmt) R 23018 23020 22776 0 -1 4202560 13755 0 0 0 500349 168 0 0 25 0 8 0 550829607 503767040 77357 33554432000 4194304 4964704 140734854192896 18446744073709551615 4658370 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=23020/tid=23026] ppid=23018 vsize=491960 CPUtime=5005.44 cores=0-7
/proc/23020/task/23026/stat : 23026 (claspmt) R 23018 23020 22776 0 -1 4202560 11835 0 0 0 500375 169 0 0 25 0 8 0 550829607 503767040 77357 33554432000 4194304 4964704 140734854192896 18446744073709551615 4325338 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=23020/tid=23027] ppid=23018 vsize=491960 CPUtime=4984.13 cores=0-7
/proc/23020/task/23027/stat : 23027 (claspmt) R 23018 23020 22776 0 -1 4202560 16066 0 0 0 498226 187 0 0 25 0 8 0 550829607 503767040 77357 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=23020/tid=23028] ppid=23018 vsize=491960 CPUtime=5004.23 cores=0-7
/proc/23020/task/23028/stat : 23028 (claspmt) R 23018 23020 22776 0 -1 4202560 10560 0 0 0 500259 164 0 0 25 0 8 0 550829607 503767040 77357 33554432000 4194304 4964704 140734854192896 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 491960

Child status: 0
Real time (s): 5007.95
CPU time (s): 40000.5
CPU user time (s): 39985.4
CPU system time (s): 15.1347
CPU usage (%): 798.741
Max. virtual memory (cumulated for all children) (KiB): 493772

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39985.4
system time used= 15.1347
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 116622
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2863041
involuntary context switches= 191439

runsolver used 12.6521 second user time and 29.1006 second system time

The end

Launcher Data 

Begin job on node138 at 2011-05-03 05:11:56
IDJOB=3322657
IDBENCH=85542
IDSOLVER=1711
FILE ID=node138/3322657-1304392316
RUNJOBID= node138-1304389717-22790
PBS_JOBID= 13181827
Free space on /tmp= 73532 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT05/crafted/sabharwal05/counting/fphp/unsat/harder/fphp-020-019.shuffled-as.sat05-1232.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322657-1304392316/watcher-3322657-1304392316 -o /tmp/evaluation-result-3322657-1304392316/solver-3322657-1304392316 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322657-1304392316.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= abb32bff5eb954e790612a6209864b5d
RANDOM SEED=733388889

node138.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      17402780 kB
Buffers:        751432 kB
Cached:        2690728 kB
SwapCached:         84 kB
Active:       12258352 kB
Inactive:      2847980 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      17402780 kB
SwapTotal:    67111528 kB
SwapFree:     67111444 kB
Dirty:             816 kB
Writeback:           0 kB
AnonPages:    11664552 kB
Mapped:          25116 kB
Slab:           341428 kB
PageTables:      38208 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 38306776 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73528 MiB
End job on node138 at 2011-05-03 06:35:26