Trace number 3381259

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40000.9 5009.28

General information on the benchmark

NameSAT05/crafted/sabharwal05/counting/php/unsat/
harder/php-040-039.shuffled-as.sat05-1194.cnf
MD5SUM8c84d9b9f2d75de33656119bc03a9afb
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark45.1361
Satisfiable
(Un)Satisfiability was proved
Number of variables1560
Number of clauses30460
Sum of the clauses size62400
Maximum clause length39
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 230420
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 540

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...510883/instance-3381259-1305510883.cnf
0.00/0.08	c Solving...
40000.50/5009.01	
40000.50/5009.01	*** INTERRUPTED! ***
40000.50/5009.01	*** Shutting down threads...
40000.78/5009.27	s UNKNOWN
40000.78/5009.27	
40000.78/5009.27	c INTERRUPTED   : 1
40000.78/5009.27	c Models        : 0+    
40000.78/5009.27	c Time          : 5009.017s (Solving: 5008.93s 1st Model: 0.00s Unsat: 0.00s)
40000.78/5009.27	c CPU Time      : 40000.590s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3381259-1305510883/watcher-3381259-1305510883 -o /tmp/evaluation-result-3381259-1305510883/solver-3381259-1305510883 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381259-1305510883.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.98 7.96 8.07 2/340 1020
/proc/meminfo: memFree=29313580/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=21520 CPUtime=0 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 666 0 0 0 0 0 0 0 25 0 1 0 662688713 22036480 584 33554432000 4194304 4964800 140735384121808 18446744073709551615 4599599 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/1020/statm: 5380 584 386 189 0 254 0

[startup+0.0984409 s]
/proc/loadavg: 7.98 7.96 8.07 2/340 1020
/proc/meminfo: memFree=29313580/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=252556 CPUtime=0.11 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 2014 0 0 0 11 0 0 0 25 0 8 0 662688713 258617344 1697 33554432000 4194304 4964800 140735384121808 18446744073709551615 230804212852 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/1020/statm: 63139 1698 453 189 0 57488 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 252556

[startup+0.100806 s]
/proc/loadavg: 7.98 7.96 8.07 2/340 1020
/proc/meminfo: memFree=29313580/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=252556 CPUtime=0.14 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 2138 0 0 0 14 0 0 0 25 0 8 0 662688713 258617344 1821 33554432000 4194304 4964800 140735384121808 18446744073709551615 230812695331 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/1020/statm: 63139 2001 464 189 0 57488 0
Current children cumulated CPU time (s) 0.14
Current children cumulated vsize (KiB) 252556

[startup+0.30038 s]
/proc/loadavg: 7.98 7.96 8.07 2/340 1020
/proc/meminfo: memFree=29313580/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=518664 CPUtime=1.44 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 8901 0 0 0 135 9 0 0 25 0 8 0 662688713 531111936 8584 33554432000 4194304 4964800 140735384121808 18446744073709551615 4652110 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/1020/statm: 129666 8584 464 189 0 124015 0
Current children cumulated CPU time (s) 1.44
Current children cumulated vsize (KiB) 518664

[startup+0.700273 s]
/proc/loadavg: 7.98 7.96 8.07 2/340 1020
/proc/meminfo: memFree=29313580/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=523068 CPUtime=4.4 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 19802 0 0 0 421 19 0 0 25 0 8 0 662688713 535621632 19485 33554432000 4194304 4964800 140735384121808 18446744073709551615 4654532 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/1020/statm: 130767 19486 464 189 0 125116 0
Current children cumulated CPU time (s) 4.4
Current children cumulated vsize (KiB) 523068

[startup+1.50113 s]
/proc/loadavg: 7.98 7.96 8.07 9/348 1030
/proc/meminfo: memFree=29212796/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=527724 CPUtime=10.74 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 29945 0 0 0 1049 25 0 0 25 0 8 0 662688713 540389376 29628 33554432000 4194304 4964800 140735384121808 18446744073709551615 4653507 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/1020/statm: 131931 29628 464 189 0 126280 0
[pid=1020/tid=1022] ppid=1018 vsize=527724 CPUtime=1.38 cores=0-7
/proc/1020/task/1022/stat : 1022 (claspmt) R 1018 1020 884 0 -1 4202560 3057 0 0 0 136 2 0 0 19 0 8 0 662688722 540389376 29628 33554432000 4194304 4964800 140735384121808 18446744073709551615 4342935 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1020/tid=1023] ppid=1018 vsize=527724 CPUtime=1.37 cores=0-7
/proc/1020/task/1023/stat : 1023 (claspmt) R 1018 1020 884 0 -1 4202560 4799 0 0 0 135 2 0 0 18 0 8 0 662688722 540389376 29628 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=1020/tid=1024] ppid=1018 vsize=527724 CPUtime=1.38 cores=0-7
/proc/1020/task/1024/stat : 1024 (claspmt) R 1018 1020 884 0 -1 4202560 2799 0 0 0 137 1 0 0 19 0 8 0 662688722 540389376 29628 33554432000 4194304 4964800 140735384121808 18446744073709551615 4660165 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=1020/tid=1025] ppid=1018 vsize=527724 CPUtime=1.34 cores=0-7
/proc/1020/task/1025/stat : 1025 (claspmt) R 1018 1020 884 0 -1 4202560 4060 0 0 0 130 4 0 0 19 0 8 0 662688722 540389376 29628 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=1020/tid=1027] ppid=1018 vsize=527724 CPUtime=1.37 cores=0-7
/proc/1020/task/1027/stat : 1027 (claspmt) R 1018 1020 884 0 -1 4202560 2996 0 0 0 136 1 0 0 18 0 8 0 662688722 540389376 29628 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=1020/tid=1028] ppid=1018 vsize=527724 CPUtime=1.36 cores=0-7
/proc/1020/task/1028/stat : 1028 (claspmt) R 1018 1020 884 0 -1 4202560 3334 0 0 0 132 4 0 0 19 0 8 0 662688722 540389376 29628 33554432000 4194304 4964800 140735384121808 18446744073709551615 4627293 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=1020/tid=1030] ppid=1018 vsize=527724 CPUtime=1.35 cores=0-7
/proc/1020/task/1030/stat : 1030 (claspmt) R 1018 1020 884 0 -1 4202560 5794 0 0 0 128 7 0 0 19 0 8 0 662688722 540389376 29628 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 10.74
Current children cumulated vsize (KiB) 527724

[startup+3.10083 s]
/proc/loadavg: 7.98 7.96 8.07 9/348 1030
/proc/meminfo: memFree=29179440/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=537360 CPUtime=23.5 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 48740 0 0 0 2316 34 0 0 25 0 8 0 662688713 550256640 48423 33554432000 4194304 4964800 140735384121808 18446744073709551615 4653296 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/1020/statm: 134340 48424 466 189 0 128689 0
[pid=1020/tid=1022] ppid=1018 vsize=537360 CPUtime=2.97 cores=0-7
/proc/1020/task/1022/stat : 1022 (claspmt) R 1018 1020 884 0 -1 4202560 4743 0 0 0 295 2 0 0 22 0 8 0 662688722 550256640 48424 33554432000 4194304 4964800 140735384121808 18446744073709551615 4653542 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1020/tid=1023] ppid=1018 vsize=537360 CPUtime=2.96 cores=0-7
/proc/1020/task/1023/stat : 1023 (claspmt) R 1018 1020 884 0 -1 4202560 8002 0 0 0 292 4 0 0 21 0 8 0 662688722 550256640 48425 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=1020/tid=1024] ppid=1018 vsize=537360 CPUtime=2.98 cores=0-7
/proc/1020/task/1024/stat : 1024 (claspmt) R 1018 1020 884 0 -1 4202560 4903 0 0 0 296 2 0 0 24 0 8 0 662688722 550256640 48425 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670332 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=1020/tid=1025] ppid=1018 vsize=537360 CPUtime=2.94 cores=0-7
/proc/1020/task/1025/stat : 1025 (claspmt) R 1018 1020 884 0 -1 4202560 6956 0 0 0 289 5 0 0 22 0 8 0 662688722 550256640 48425 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670295 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=1020/tid=1027] ppid=1018 vsize=537360 CPUtime=2.96 cores=0-7
/proc/1020/task/1027/stat : 1027 (claspmt) R 1018 1020 884 0 -1 4202560 6696 0 0 0 293 3 0 0 18 0 8 0 662688722 550256640 48425 33554432000 4194304 4964800 140735384121808 18446744073709551615 4668900 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=1020/tid=1028] ppid=1018 vsize=537360 CPUtime=2.96 cores=0-7
/proc/1020/task/1028/stat : 1028 (claspmt) R 1018 1020 884 0 -1 4202560 5767 0 0 0 291 5 0 0 19 0 8 0 662688722 550256640 48426 33554432000 4194304 4964800 140735384121808 18446744073709551615 4332491 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=1020/tid=1030] ppid=1018 vsize=537360 CPUtime=2.94 cores=0-7
/proc/1020/task/1030/stat : 1030 (claspmt) R 1018 1020 884 0 -1 4202560 7905 0 0 0 286 8 0 0 19 0 8 0 662688722 550256640 48426 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 23.5
Current children cumulated vsize (KiB) 537360

[startup+6.30125 s]
/proc/loadavg: 7.98 7.96 8.07 9/348 1030
/proc/meminfo: memFree=29091172/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=553020 CPUtime=49.06 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 65441 0 0 0 4862 44 0 0 25 0 8 0 662688713 566292480 65124 33554432000 4194304 4964800 140735384121808 18446744073709551615 4653464 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/1020/statm: 138255 65124 466 189 0 132604 0
[pid=1020/tid=1022] ppid=1018 vsize=553020 CPUtime=6.17 cores=0-7
/proc/1020/task/1022/stat : 1022 (claspmt) R 1018 1020 884 0 -1 4202560 6325 0 0 0 614 3 0 0 25 0 8 0 662688722 566292480 65124 33554432000 4194304 4964800 140735384121808 18446744073709551615 4631252 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1020/tid=1023] ppid=1018 vsize=553020 CPUtime=6.17 cores=0-7
/proc/1020/task/1023/stat : 1023 (claspmt) R 1018 1020 884 0 -1 4202560 10283 0 0 0 611 6 0 0 25 0 8 0 662688722 566292480 65124 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=1020/tid=1024] ppid=1018 vsize=553020 CPUtime=6.18 cores=0-7
/proc/1020/task/1024/stat : 1024 (claspmt) R 1018 1020 884 0 -1 4202560 6486 0 0 0 615 3 0 0 25 0 8 0 662688722 566292480 65126 33554432000 4194304 4964800 140735384121808 18446744073709551615 4332465 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=1020/tid=1025] ppid=1018 vsize=553020 CPUtime=6.14 cores=0-7
/proc/1020/task/1025/stat : 1025 (claspmt) R 1018 1020 884 0 -1 4202560 8858 0 0 0 608 6 0 0 25 0 8 0 662688722 566292480 65127 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=1020/tid=1027] ppid=1018 vsize=553020 CPUtime=6.16 cores=0-7
/proc/1020/task/1027/stat : 1027 (claspmt) R 1018 1020 884 0 -1 4202560 9444 0 0 0 612 4 0 0 25 0 8 0 662688722 566292480 65127 33554432000 4194304 4964800 140735384121808 18446744073709551615 4343440 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=1020/tid=1028] ppid=1018 vsize=553020 CPUtime=6.15 cores=0-7
/proc/1020/task/1028/stat : 1028 (claspmt) R 1018 1020 884 0 -1 4202560 8549 0 0 0 609 6 0 0 25 0 8 0 662688722 566292480 65128 33554432000 4194304 4964800 140735384121808 18446744073709551615 230808826913 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=1020/tid=1030] ppid=1018 vsize=553020 CPUtime=6.14 cores=0-7
/proc/1020/task/1030/stat : 1030 (claspmt) R 1018 1020 884 0 -1 4202560 9755 0 0 0 605 9 0 0 25 0 8 0 662688722 566292480 65128 33554432000 4194304 4964800 140735384121808 18446744073709551615 4321020 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 49.06
Current children cumulated vsize (KiB) 553020

[startup+12.7011 s]
/proc/loadavg: 7.99 7.96 8.07 9/348 1031
/proc/meminfo: memFree=28944456/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=560344 CPUtime=100.15 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 93882 0 0 0 9955 60 0 0 25 0 8 0 662688713 573792256 93444 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/1020/statm: 140086 93444 466 189 0 134435 0
[pid=1020/tid=1022] ppid=1018 vsize=560344 CPUtime=12.56 cores=0-7
/proc/1020/task/1022/stat : 1022 (claspmt) R 1018 1020 884 0 -1 4202560 10509 0 0 0 1251 5 0 0 25 0 8 0 662688722 573792256 93444 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1020/tid=1023] ppid=1018 vsize=560344 CPUtime=12.56 cores=0-7
/proc/1020/task/1023/stat : 1023 (claspmt) R 1018 1020 884 0 -1 4202560 13968 0 0 0 1248 8 0 0 21 0 8 0 662688722 573792256 93444 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=1020/tid=1024] ppid=1018 vsize=560344 CPUtime=12.57 cores=0-7
/proc/1020/task/1024/stat : 1024 (claspmt) R 1018 1020 884 0 -1 4202560 11385 0 0 0 1252 5 0 0 25 0 8 0 662688722 573792256 93444 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=1020/tid=1025] ppid=1018 vsize=560344 CPUtime=12.53 cores=0-7
/proc/1020/task/1025/stat : 1025 (claspmt) R 1018 1020 884 0 -1 4202560 12501 0 0 0 1246 7 0 0 25 0 8 0 662688722 573792256 93444 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=1020/tid=1027] ppid=1018 vsize=560344 CPUtime=12.55 cores=0-7
/proc/1020/task/1027/stat : 1027 (claspmt) R 1018 1020 884 0 -1 4202560 14921 0 0 0 1248 7 0 0 25 0 8 0 662688722 573792256 93444 33554432000 4194304 4964800 140735384121808 18446744073709551615 4342778 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=1020/tid=1028] ppid=1018 vsize=560344 CPUtime=12.56 cores=0-7
/proc/1020/task/1028/stat : 1028 (claspmt) R 1018 1020 884 0 -1 4202560 11537 0 0 0 1247 9 0 0 25 0 8 0 662688722 573792256 93444 33554432000 4194304 4964800 140735384121808 18446744073709551615 4332514 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=1020/tid=1030] ppid=1018 vsize=560344 CPUtime=12.54 cores=0-7
/proc/1020/task/1030/stat : 1030 (claspmt) R 1018 1020 884 0 -1 4202560 10366 0 0 0 1245 9 0 0 25 0 8 0 662688722 573792256 93444 33554432000 4194304 4964800 140735384121808 18446744073709551615 4658589 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 100.15
Current children cumulated vsize (KiB) 560344

[startup+25.5007 s]
/proc/loadavg: 7.99 7.96 8.07 9/348 1031
/proc/meminfo: memFree=28821516/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=707480 CPUtime=202.36 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 126434 0 0 0 20157 79 0 0 25 0 8 0 662688713 724459520 125875 33554432000 4194304 4964800 140735384121808 18446744073709551615 4654540 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/1020/statm: 176870 125875 466 189 0 171219 0
[pid=1020/tid=1022] ppid=1018 vsize=707480 CPUtime=25.35 cores=0-7
/proc/1020/task/1022/stat : 1022 (claspmt) R 1018 1020 884 0 -1 4202560 16479 0 0 0 2527 8 0 0 25 0 8 0 662688722 724459520 125875 33554432000 4194304 4964800 140735384121808 18446744073709551615 4653326 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1020/tid=1023] ppid=1018 vsize=707480 CPUtime=25.36 cores=0-7
/proc/1020/task/1023/stat : 1023 (claspmt) R 1018 1020 884 0 -1 4202560 16194 0 0 0 2526 10 0 0 25 0 8 0 662688722 724459520 125875 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=1020/tid=1024] ppid=1018 vsize=707480 CPUtime=25.37 cores=0-7
/proc/1020/task/1024/stat : 1024 (claspmt) R 1018 1020 884 0 -1 4202560 18819 0 0 0 2527 10 0 0 25 0 8 0 662688722 724459520 125875 33554432000 4194304 4964800 140735384121808 18446744073709551615 230808826913 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=1020/tid=1025] ppid=1018 vsize=707480 CPUtime=25.33 cores=0-7
/proc/1020/task/1025/stat : 1025 (claspmt) R 1018 1020 884 0 -1 4202560 14093 0 0 0 2525 8 0 0 25 0 8 0 662688722 724459520 125875 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=1020/tid=1027] ppid=1018 vsize=707480 CPUtime=25.35 cores=0-7
/proc/1020/task/1027/stat : 1027 (claspmt) R 1018 1020 884 0 -1 4202560 22654 0 0 0 2523 12 0 0 25 0 8 0 662688722 724459520 125875 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=1020/tid=1028] ppid=1018 vsize=707480 CPUtime=25.35 cores=0-7
/proc/1020/task/1028/stat : 1028 (claspmt) R 1018 1020 884 0 -1 4202560 13634 0 0 0 2526 9 0 0 25 0 8 0 662688722 724459520 125876 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=1020/tid=1030] ppid=1018 vsize=707480 CPUtime=25.33 cores=0-7
/proc/1020/task/1030/stat : 1030 (claspmt) R 1018 1020 884 0 -1 4202560 13511 0 0 0 2521 12 0 0 18 0 8 0 662688722 724459520 125876 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670514 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 202.36
Current children cumulated vsize (KiB) 707480

[startup+51.101 s]
/proc/loadavg: 8.06 7.98 8.07 9/348 1033
/proc/meminfo: memFree=28618516/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=1064392 CPUtime=406.8 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 177126 0 0 0 40577 103 0 0 25 0 8 0 662688713 1089937408 176355 33554432000 4194304 4964800 140735384121808 18446744073709551615 4332465 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/1020/statm: 266098 176355 466 189 0 260447 0
[pid=1020/tid=1022] ppid=1018 vsize=1064392 CPUtime=50.93 cores=0-7
/proc/1020/task/1022/stat : 1022 (claspmt) R 1018 1020 884 0 -1 4202560 19545 0 0 0 5083 10 0 0 25 0 8 0 662688722 1089937408 176355 33554432000 4194304 4964800 140735384121808 18446744073709551615 4342918 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1020/tid=1023] ppid=1018 vsize=1064392 CPUtime=50.95 cores=0-7
/proc/1020/task/1023/stat : 1023 (claspmt) R 1018 1020 884 0 -1 4202560 20673 0 0 0 5083 12 0 0 25 0 8 0 662688722 1089937408 176355 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=1020/tid=1024] ppid=1018 vsize=1064392 CPUtime=50.97 cores=0-7
/proc/1020/task/1024/stat : 1024 (claspmt) R 1018 1020 884 0 -1 4202560 24496 0 0 0 5085 12 0 0 25 0 8 0 662688722 1089937408 176355 33554432000 4194304 4964800 140735384121808 18446744073709551615 4653388 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=1020/tid=1025] ppid=1018 vsize=1064392 CPUtime=50.93 cores=0-7
/proc/1020/task/1025/stat : 1025 (claspmt) R 1018 1020 884 0 -1 4202560 15364 0 0 0 5084 9 0 0 25 0 8 0 662688722 1089937408 176355 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=1020/tid=1027] ppid=1018 vsize=1064392 CPUtime=50.94 cores=0-7
/proc/1020/task/1027/stat : 1027 (claspmt) R 1018 1020 884 0 -1 4202560 33102 0 0 0 5077 17 0 0 25 0 8 0 662688722 1089937408 176355 33554432000 4194304 4964800 140735384121808 18446744073709551615 4343440 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=1020/tid=1028] ppid=1018 vsize=1064392 CPUtime=50.93 cores=0-7
/proc/1020/task/1028/stat : 1028 (claspmt) R 1018 1020 884 0 -1 4202560 27647 0 0 0 5077 16 0 0 25 0 8 0 662688722 1089937408 176355 33554432000 4194304 4964800 140735384121808 18446744073709551615 4342887 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=1020/tid=1030] ppid=1018 vsize=1064392 CPUtime=50.93 cores=0-7
/proc/1020/task/1030/stat : 1030 (claspmt) R 1018 1020 884 0 -1 4202560 17853 0 0 0 5079 14 0 0 25 0 8 0 662688722 1089937408 176355 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 406.8

################
# More data... #
################

[pid=1020/tid=1023] ppid=1018 vsize=3331440 CPUtime=4653.96 cores=0-7
/proc/1020/task/1023/stat : 1023 (claspmt) R 1018 1020 884 0 -1 4202560 75425 0 0 0 465324 72 0 0 25 0 8 0 662688722 3411394560 765447 33554432000 4194304 4964800 140735384121808 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1020/tid=1024] ppid=1018 vsize=3331440 CPUtime=4659.75 cores=0-7
/proc/1020/task/1024/stat : 1024 (claspmt) R 1018 1020 884 0 -1 4202560 143829 0 0 0 465866 109 0 0 25 0 8 0 662688722 3411394560 765447 33554432000 4194304 4964800 140735384121808 18446744073709551615 4660390 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=1020/tid=1025] ppid=1018 vsize=3331440 CPUtime=4646.08 cores=0-7
/proc/1020/task/1025/stat : 1025 (claspmt) R 1018 1020 884 0 -1 4202560 56508 0 0 0 464547 61 0 0 25 0 8 0 662688722 3411394560 765447 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=1020/tid=1027] ppid=1018 vsize=3331440 CPUtime=4643.09 cores=0-7
/proc/1020/task/1027/stat : 1027 (claspmt) R 1018 1020 884 0 -1 4202560 152998 0 0 0 464185 124 0 0 25 0 8 0 662688722 3411394560 765447 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=1020/tid=1028] ppid=1018 vsize=3331440 CPUtime=4660.25 cores=0-7
/proc/1020/task/1028/stat : 1028 (claspmt) R 1018 1020 884 0 -1 4202560 149553 0 0 0 465909 116 0 0 25 0 8 0 662688722 3411394560 765447 33554432000 4194304 4964800 140735384121808 18446744073709551615 4333097 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=1020/tid=1030] ppid=1018 vsize=3331440 CPUtime=4660.64 cores=0-7
/proc/1020/task/1030/stat : 1030 (claspmt) R 1018 1020 884 0 -1 4202560 43070 0 0 0 466009 55 0 0 25 0 8 0 662688722 3411394560 765447 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37231.5
Current children cumulated vsize (KiB) 3331440

[startup+4722.31 s]
/proc/loadavg: 8.03 8.05 8.00 9/348 1712
/proc/meminfo: memFree=26245452/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=3331440 CPUtime=37710.8 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 786198 0 0 0 3770343 735 0 0 25 0 8 0 662688713 3411394560 765266 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/1020/statm: 832860 765266 466 189 0 827209 0
[pid=1020/tid=1022] ppid=1018 vsize=3331440 CPUtime=4710.12 cores=0-7
/proc/1020/task/1022/stat : 1022 (claspmt) R 1018 1020 884 0 -1 4202560 69257 0 0 0 470914 98 0 0 25 0 8 0 662688722 3411394560 765266 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1020/tid=1023] ppid=1018 vsize=3331440 CPUtime=4713.29 cores=0-7
/proc/1020/task/1023/stat : 1023 (claspmt) R 1018 1020 884 0 -1 4202560 75426 0 0 0 471256 73 0 0 25 0 8 0 662688722 3411394560 765266 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1020/tid=1024] ppid=1018 vsize=3331440 CPUtime=4719.75 cores=0-7
/proc/1020/task/1024/stat : 1024 (claspmt) R 1018 1020 884 0 -1 4202560 143829 0 0 0 471865 110 0 0 25 0 8 0 662688722 3411394560 765266 33554432000 4194304 4964800 140735384121808 18446744073709551615 4321010 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=1020/tid=1025] ppid=1018 vsize=3331440 CPUtime=4706.07 cores=0-7
/proc/1020/task/1025/stat : 1025 (claspmt) R 1018 1020 884 0 -1 4202560 56508 0 0 0 470546 61 0 0 25 0 8 0 662688722 3411394560 765266 33554432000 4194304 4964800 140735384121808 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=1020/tid=1027] ppid=1018 vsize=3331440 CPUtime=4703.09 cores=0-7
/proc/1020/task/1027/stat : 1027 (claspmt) R 1018 1020 884 0 -1 4202560 152998 0 0 0 470185 124 0 0 25 0 8 0 662688722 3411394560 765266 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=1020/tid=1028] ppid=1018 vsize=3331440 CPUtime=4720.26 cores=0-7
/proc/1020/task/1028/stat : 1028 (claspmt) R 1018 1020 884 0 -1 4202560 149553 0 0 0 471909 117 0 0 25 0 8 0 662688722 3411394560 765266 33554432000 4194304 4964800 140735384121808 18446744073709551615 4627168 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=1020/tid=1030] ppid=1018 vsize=3331440 CPUtime=4720.65 cores=0-7
/proc/1020/task/1030/stat : 1030 (claspmt) R 1018 1020 884 0 -1 4202560 43070 0 0 0 472009 56 0 0 25 0 8 0 662688722 3411394560 765266 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37710.8
Current children cumulated vsize (KiB) 3331440

[startup+4782.3 s]
/proc/loadavg: 8.06 8.05 8.00 9/348 1714
/proc/meminfo: memFree=26247044/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=3331664 CPUtime=38189.9 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 786241 0 0 0 3818250 740 0 0 25 0 8 0 662688713 3411623936 764794 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/1020/statm: 832916 764794 466 189 0 827265 0
[pid=1020/tid=1022] ppid=1018 vsize=3331664 CPUtime=4770.08 cores=0-7
/proc/1020/task/1022/stat : 1022 (claspmt) R 1018 1020 884 0 -1 4202560 69291 0 0 0 476909 99 0 0 25 0 8 0 662688722 3411623936 764794 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1020/tid=1023] ppid=1018 vsize=3331664 CPUtime=4772.63 cores=0-7
/proc/1020/task/1023/stat : 1023 (claspmt) R 1018 1020 884 0 -1 4202560 75429 0 0 0 477189 74 0 0 25 0 8 0 662688722 3411623936 764794 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1020/tid=1024] ppid=1018 vsize=3331664 CPUtime=4779.73 cores=0-7
/proc/1020/task/1024/stat : 1024 (claspmt) R 1018 1020 884 0 -1 4202560 143829 0 0 0 477863 110 0 0 25 0 8 0 662688722 3411623936 764794 33554432000 4194304 4964800 140735384121808 18446744073709551615 4342232 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=1020/tid=1025] ppid=1018 vsize=3331664 CPUtime=4766.06 cores=0-7
/proc/1020/task/1025/stat : 1025 (claspmt) R 1018 1020 884 0 -1 4202560 56508 0 0 0 476544 62 0 0 25 0 8 0 662688722 3411623936 764794 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=1020/tid=1027] ppid=1018 vsize=3331664 CPUtime=4763.06 cores=0-7
/proc/1020/task/1027/stat : 1027 (claspmt) R 1018 1020 884 0 -1 4202560 153004 0 0 0 476181 125 0 0 25 0 8 0 662688722 3411623936 764794 33554432000 4194304 4964800 140735384121808 18446744073709551615 4342939 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=1020/tid=1028] ppid=1018 vsize=3331664 CPUtime=4780.22 cores=0-7
/proc/1020/task/1028/stat : 1028 (claspmt) R 1018 1020 884 0 -1 4202560 149553 0 0 0 477905 117 0 0 25 0 8 0 662688722 3411623936 764794 33554432000 4194304 4964800 140735384121808 18446744073709551615 4332510 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=1020/tid=1030] ppid=1018 vsize=3331664 CPUtime=4780.62 cores=0-7
/proc/1020/task/1030/stat : 1030 (claspmt) R 1018 1020 884 0 -1 4202560 43070 0 0 0 478006 56 0 0 25 0 8 0 662688722 3411623936 764794 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38189.9
Current children cumulated vsize (KiB) 3331664

[startup+4842.3 s]
/proc/loadavg: 8.10 8.07 8.01 9/348 1715
/proc/meminfo: memFree=26231132/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=3331664 CPUtime=38669.1 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 790378 0 0 0 3866161 746 0 0 25 0 8 0 662688713 3411623936 768880 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670310 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/1020/statm: 832916 768880 466 189 0 827265 0
[pid=1020/tid=1022] ppid=1018 vsize=3331664 CPUtime=4830.04 cores=0-7
/proc/1020/task/1022/stat : 1022 (claspmt) R 1018 1020 884 0 -1 4202560 69292 0 0 0 482904 100 0 0 25 0 8 0 662688722 3411623936 768880 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670295 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1020/tid=1023] ppid=1018 vsize=3331664 CPUtime=4831.95 cores=0-7
/proc/1020/task/1023/stat : 1023 (claspmt) R 1018 1020 884 0 -1 4202560 75434 0 0 0 483121 74 0 0 19 0 8 0 662688722 3411623936 768880 33554432000 4194304 4964800 140735384121808 18446744073709551615 4653144 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1020/tid=1024] ppid=1018 vsize=3331664 CPUtime=4839.71 cores=0-7
/proc/1020/task/1024/stat : 1024 (claspmt) R 1018 1020 884 0 -1 4202560 143829 0 0 0 483860 111 0 0 25 0 8 0 662688722 3411623936 768880 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=1020/tid=1025] ppid=1018 vsize=3331664 CPUtime=4826.04 cores=0-7
/proc/1020/task/1025/stat : 1025 (claspmt) R 1018 1020 884 0 -1 4202560 56508 0 0 0 482542 62 0 0 25 0 8 0 662688722 3411623936 768880 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=1020/tid=1027] ppid=1018 vsize=3331664 CPUtime=4823.03 cores=0-7
/proc/1020/task/1027/stat : 1027 (claspmt) R 1018 1020 884 0 -1 4202560 153037 0 0 0 482178 125 0 0 25 0 8 0 662688722 3411623936 768880 33554432000 4194304 4964800 140735384121808 18446744073709551615 230808863346 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=1020/tid=1028] ppid=1018 vsize=3331664 CPUtime=4840.21 cores=0-7
/proc/1020/task/1028/stat : 1028 (claspmt) R 1018 1020 884 0 -1 4202560 153596 0 0 0 483901 120 0 0 25 0 8 0 662688722 3411623936 768880 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=1020/tid=1030] ppid=1018 vsize=3331664 CPUtime=4840.59 cores=0-7
/proc/1020/task/1030/stat : 1030 (claspmt) R 1018 1020 884 0 -1 4202560 43070 0 0 0 484003 56 0 0 25 0 8 0 662688722 3411623936 768880 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38669.1
Current children cumulated vsize (KiB) 3331664

[startup+4902.3 s]
/proc/loadavg: 8.08 8.07 8.01 9/348 1717
/proc/meminfo: memFree=26214952/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=3397200 CPUtime=39148.3 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 794432 0 0 0 3914079 750 0 0 25 0 8 0 662688713 3478732800 772924 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670343 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/1020/statm: 849300 772924 466 189 0 843649 0
[pid=1020/tid=1022] ppid=1018 vsize=3397200 CPUtime=4890.02 cores=0-7
/proc/1020/task/1022/stat : 1022 (claspmt) R 1018 1020 884 0 -1 4202560 69310 0 0 0 488901 101 0 0 25 0 8 0 662688722 3478732800 772924 33554432000 4194304 4964800 140735384121808 18446744073709551615 4332496 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1020/tid=1023] ppid=1018 vsize=3397200 CPUtime=4891.3 cores=0-7
/proc/1020/task/1023/stat : 1023 (claspmt) R 1018 1020 884 0 -1 4202560 75435 0 0 0 489055 75 0 0 25 0 8 0 662688722 3478732800 772924 33554432000 4194304 4964800 140735384121808 18446744073709551615 4660157 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1020/tid=1024] ppid=1018 vsize=3397200 CPUtime=4899.69 cores=0-7
/proc/1020/task/1024/stat : 1024 (claspmt) R 1018 1020 884 0 -1 4202560 147855 0 0 0 489857 112 0 0 25 0 8 0 662688722 3478732800 772924 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=1020/tid=1025] ppid=1018 vsize=3397200 CPUtime=4886.04 cores=0-7
/proc/1020/task/1025/stat : 1025 (claspmt) R 1018 1020 884 0 -1 4202560 56508 0 0 0 488541 63 0 0 25 0 8 0 662688722 3478732800 772924 33554432000 4194304 4964800 140735384121808 18446744073709551615 4362326 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=1020/tid=1027] ppid=1018 vsize=3397200 CPUtime=4883.02 cores=0-7
/proc/1020/task/1027/stat : 1027 (claspmt) R 1018 1020 884 0 -1 4202560 153043 0 0 0 488176 126 0 0 25 0 8 0 662688722 3478732800 772924 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=1020/tid=1028] ppid=1018 vsize=3397200 CPUtime=4900.19 cores=0-7
/proc/1020/task/1028/stat : 1028 (claspmt) R 1018 1020 884 0 -1 4202560 153596 0 0 0 489899 120 0 0 25 0 8 0 662688722 3478732800 772924 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=1020/tid=1030] ppid=1018 vsize=3397200 CPUtime=4900.58 cores=0-7
/proc/1020/task/1030/stat : 1030 (claspmt) R 1018 1020 884 0 -1 4202560 43070 0 0 0 490002 56 0 0 25 0 8 0 662688722 3478732800 772924 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39148.3
Current children cumulated vsize (KiB) 3397200

[startup+4962.3 s]
/proc/loadavg: 8.06 8.07 8.01 9/348 1719
/proc/meminfo: memFree=26201028/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=3401292 CPUtime=39627.5 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 798293 0 0 0 3961992 756 0 0 25 0 8 0 662688713 3482923008 776327 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/1020/statm: 850323 776327 466 189 0 844672 0
[pid=1020/tid=1022] ppid=1018 vsize=3401292 CPUtime=4949.97 cores=0-7
/proc/1020/task/1022/stat : 1022 (claspmt) R 1018 1020 884 0 -1 4202560 69310 0 0 0 494896 101 0 0 25 0 8 0 662688722 3482923008 776327 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1020/tid=1023] ppid=1018 vsize=3401292 CPUtime=4950.63 cores=0-7
/proc/1020/task/1023/stat : 1023 (claspmt) R 1018 1020 884 0 -1 4202560 78168 0 0 0 494986 77 0 0 25 0 8 0 662688722 3482923008 776327 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1020/tid=1024] ppid=1018 vsize=3401292 CPUtime=4959.67 cores=0-7
/proc/1020/task/1024/stat : 1024 (claspmt) R 1018 1020 884 0 -1 4202560 147856 0 0 0 495854 113 0 0 25 0 8 0 662688722 3482923008 776327 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=1020/tid=1025] ppid=1018 vsize=3401292 CPUtime=4946.03 cores=0-7
/proc/1020/task/1025/stat : 1025 (claspmt) R 1018 1020 884 0 -1 4202560 56508 0 0 0 494540 63 0 0 25 0 8 0 662688722 3482923008 776327 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=1020/tid=1027] ppid=1018 vsize=3401292 CPUtime=4942.99 cores=0-7
/proc/1020/task/1027/stat : 1027 (claspmt) R 1018 1020 884 0 -1 4202560 153052 0 0 0 494173 126 0 0 25 0 8 0 662688722 3482923008 776327 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=1020/tid=1028] ppid=1018 vsize=3401292 CPUtime=4960.19 cores=0-7
/proc/1020/task/1028/stat : 1028 (claspmt) R 1018 1020 884 0 -1 4202560 153596 0 0 0 495898 121 0 0 25 0 8 0 662688722 3482923008 776327 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=1020/tid=1030] ppid=1018 vsize=3401292 CPUtime=4960.56 cores=0-7
/proc/1020/task/1030/stat : 1030 (claspmt) R 1018 1020 884 0 -1 4202560 43070 0 0 0 496000 56 0 0 25 0 8 0 662688722 3482923008 776327 33554432000 4194304 4964800 140735384121808 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39627.5
Current children cumulated vsize (KiB) 3401292



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5009 s]
/proc/loadavg: 8.07 8.07 8.01 9/348 1720
/proc/meminfo: memFree=26208312/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=3401292 CPUtime=40000.5 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 798306 0 0 0 3999293 757 0 0 25 0 8 0 662688713 3482923008 774478 33554432000 4194304 4964800 140735384121808 18446744073709551615 4332566 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/1020/statm: 850323 774478 466 189 0 844672 0
[pid=1020/tid=1022] ppid=1018 vsize=3401292 CPUtime=4996.64 cores=0-7
/proc/1020/task/1022/stat : 1022 (claspmt) R 1018 1020 884 0 -1 4202560 69322 0 0 0 499563 101 0 0 25 0 8 0 662688722 3482923008 774478 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670335 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1020/tid=1023] ppid=1018 vsize=3401292 CPUtime=4996.82 cores=0-7
/proc/1020/task/1023/stat : 1023 (claspmt) R 1018 1020 884 0 -1 4202560 78168 0 0 0 499605 77 0 0 25 0 8 0 662688722 3482923008 774478 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1020/tid=1024] ppid=1018 vsize=3401292 CPUtime=5006.37 cores=0-7
/proc/1020/task/1024/stat : 1024 (claspmt) R 1018 1020 884 0 -1 4202560 147856 0 0 0 500524 113 0 0 25 0 8 0 662688722 3482923008 774478 33554432000 4194304 4964800 140735384121808 18446744073709551615 4332545 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=1020/tid=1025] ppid=1018 vsize=3401292 CPUtime=4992.74 cores=0-7
/proc/1020/task/1025/stat : 1025 (claspmt) R 1018 1020 884 0 -1 4202560 56508 0 0 0 499211 63 0 0 25 0 8 0 662688722 3482923008 774478 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670538 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=1020/tid=1027] ppid=1018 vsize=3401292 CPUtime=4989.7 cores=0-7
/proc/1020/task/1027/stat : 1027 (claspmt) R 1018 1020 884 0 -1 4202560 153052 0 0 0 498844 126 0 0 25 0 8 0 662688722 3482923008 774478 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=1020/tid=1028] ppid=1018 vsize=3401292 CPUtime=5006.88 cores=0-7
/proc/1020/task/1028/stat : 1028 (claspmt) R 1018 1020 884 0 -1 4202560 153596 0 0 0 500567 121 0 0 25 0 8 0 662688722 3482923008 774478 33554432000 4194304 4964800 140735384121808 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=1020/tid=1030] ppid=1018 vsize=3401292 CPUtime=5007.28 cores=0-7
/proc/1020/task/1030/stat : 1030 (claspmt) R 1018 1020 884 0 -1 4202560 43071 0 0 0 500671 57 0 0 25 0 8 0 662688722 3482923008 774478 33554432000 4194304 4964800 140735384121808 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 3401292

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5009.2 s]
/proc/loadavg: 8.07 8.07 8.01 9/348 1720
/proc/meminfo: memFree=26208312/32951124 swapFree=37160400/67111528
[pid=1020] ppid=1018 vsize=1890532 CPUtime=40000.8 cores=0-7
/proc/1020/stat : 1020 (claspmt) R 1018 1020 884 0 -1 4202496 798316 0 0 0 3999315 763 0 0 25 0 1 0 662688713 1935904768 408218 33554432000 4194304 4964800 140735384121808 18446744073709551615 230809208967 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/1020/statm: 472633 408183 476 189 0 466982 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 1890532

Child status: 0
Real time (s): 5009.28
CPU time (s): 40000.9
CPU user time (s): 39993.2
CPU system time (s): 7.71283
CPU usage (%): 798.536
Max. virtual memory (cumulated for all children) (KiB): 3401292

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39993.2
system time used= 7.71283
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 798317
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 480706
involuntary context switches= 195974

runsolver used 15.8446 second user time and 39.0681 second system time

The end

Launcher Data 

Begin job on node104 at 2011-05-16 03:54:43
IDJOB=3381259
IDBENCH=85456
IDSOLVER=1865
FILE ID=node104/3381259-1305510883
RUNJOBID= node104-1305510705-898
PBS_JOBID= 13325527
Free space on /tmp= 73428 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT05/crafted/sabharwal05/counting/php/unsat/harder/php-040-039.shuffled-as.sat05-1194.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3381259-1305510883/watcher-3381259-1305510883 -o /tmp/evaluation-result-3381259-1305510883/solver-3381259-1305510883 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381259-1305510883.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 8c84d9b9f2d75de33656119bc03a9afb
RANDOM SEED=837181365

node104.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5293.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      29313984 kB
Buffers:        290320 kB
Cached:        2229164 kB
SwapCached:     152320 kB
Active:         114008 kB
Inactive:      3249916 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      29313984 kB
SwapTotal:    67111528 kB
SwapFree:     37160400 kB
Dirty:            1192 kB
Writeback:           0 kB
AnonPages:      705092 kB
Mapped:           8316 kB
Slab:           122944 kB
PageTables:      84640 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 74804008 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73460 MiB
End job on node104 at 2011-05-16 05:18:14