Trace number 3322776

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40001.7 5021.76

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m62-q1327.5-n60-p15-s1351253.cnf
MD5SUMb23c52475e3d098cb9d5a92c025d6982
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.465928
Satisfiable
(Un)Satisfiability was proved
Number of variables3420
Number of clauses1120760
Sum of the clauses size2244820
Maximum clause length62
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21120700
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 560

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...395051/instance-3322776-1304395051.cnf
15.58/15.69	c Solving...
40000.16/5020.32	
40000.16/5020.32	*** INTERRUPTED! ***
40000.16/5020.32	*** Shutting down threads...
40001.60/5021.73	s UNKNOWN
40001.60/5021.73	
40001.60/5021.73	c INTERRUPTED   : 1
40001.60/5021.73	c Models        : 0+    
40001.60/5021.73	c Time          : 5020.515s (Solving: 5004.82s 1st Model: 0.00s Unsat: 0.00s)
40001.60/5021.73	c CPU Time      : 40000.420s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3322776-1304395051/watcher-3322776-1304395051 -o /tmp/evaluation-result-3322776-1304395051/solver-3322776-1304395051 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322776-1304395051.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.01 8.04 8.00 2/265 4099
/proc/meminfo: memFree=15285960/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=22384 CPUtime=0 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 940 0 0 0 0 0 0 0 24 0 1 0 551104906 22921216 807 33554432000 4194304 4964704 140735565610784 18446744073709551615 237278540485 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/4099/statm: 5596 807 386 189 0 470 0

[startup+0.080019 s]
/proc/loadavg: 8.01 8.04 8.00 2/265 4099
/proc/meminfo: memFree=15285960/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=36376 CPUtime=0.06 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 5944 0 0 0 6 0 0 0 24 0 1 0 551104906 37249024 4089 33554432000 4194304 4964704 140735565610784 18446744073709551615 4599599 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/4099/statm: 9094 4090 386 189 0 3968 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 36376

[startup+0.101017 s]
/proc/loadavg: 8.01 8.04 8.00 2/265 4099
/proc/meminfo: memFree=15285960/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=39544 CPUtime=0.09 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 6937 0 0 0 8 1 0 0 24 0 1 0 551104906 40493056 5082 33554432000 4194304 4964704 140735565610784 18446744073709551615 237278540485 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/4099/statm: 9886 5084 386 189 0 4760 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 39544

[startup+0.300981 s]
/proc/loadavg: 8.01 8.04 8.00 2/265 4099
/proc/meminfo: memFree=15285960/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=81688 CPUtime=0.29 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 20794 0 0 0 27 2 0 0 25 0 1 0 551104906 83648512 14575 33554432000 4194304 4964704 140735565610784 18446744073709551615 4618467 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/4099/statm: 20422 14575 397 189 0 15296 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 81688

[startup+0.70091 s]
/proc/loadavg: 8.01 8.04 8.00 2/265 4099
/proc/meminfo: memFree=15285960/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=96224 CPUtime=0.68 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 24166 0 0 0 66 2 0 0 25 0 1 0 551104906 98533376 17947 33554432000 4194304 4964704 140735565610784 18446744073709551615 4607673 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/4099/statm: 24056 17947 399 189 0 18930 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 96224

[startup+1.50077 s]
/proc/loadavg: 8.01 8.04 8.00 2/266 4100
/proc/meminfo: memFree=15215404/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=96224 CPUtime=1.48 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 24166 0 0 0 146 2 0 0 25 0 1 0 551104906 98533376 17947 33554432000 4194304 4964704 140735565610784 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/4099/statm: 24056 17947 399 189 0 18930 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 96224

[startup+3.10048 s]
/proc/loadavg: 8.01 8.04 8.00 2/266 4100
/proc/meminfo: memFree=15214924/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=96224 CPUtime=3.08 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 24166 0 0 0 306 2 0 0 25 0 1 0 551104906 98533376 17947 33554432000 4194304 4964704 140735565610784 18446744073709551615 4607472 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/4099/statm: 24056 17947 399 189 0 18930 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 96224

[startup+6.30091 s]
/proc/loadavg: 7.45 7.92 7.96 2/266 4100
/proc/meminfo: memFree=15214856/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=96224 CPUtime=6.29 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 24166 0 0 0 626 3 0 0 25 0 1 0 551104906 98533376 17947 33554432000 4194304 4964704 140735565610784 18446744073709551615 4607604 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/4099/statm: 24056 17947 399 189 0 18930 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 96224

[startup+12.7008 s]
/proc/loadavg: 6.93 7.81 7.92 2/266 4101
/proc/meminfo: memFree=15212548/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=99816 CPUtime=12.69 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 25081 0 0 0 1266 3 0 0 25 0 1 0 551104906 102211584 18862 33554432000 4194304 4964704 140735565610784 18446744073709551615 4600360 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/4099/statm: 24954 18862 400 189 0 19828 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 99816

[startup+25.5005 s]
/proc/loadavg: 6.70 7.70 7.89 9/273 4108
/proc/meminfo: memFree=15014748/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=515552 CPUtime=86.16 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 78429 0 0 0 8581 35 0 0 25 0 8 0 551104906 527925248 70009 33554432000 4194304 4964704 140735565610784 18446744073709551615 4320970 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/4099/statm: 128888 70010 466 189 0 123237 0
[pid=4099/tid=4102] ppid=4097 vsize=515552 CPUtime=8.67 cores=0-7
/proc/4099/task/4102/stat : 4102 (claspmt) R 4097 4099 3873 0 -1 4202560 5019 0 0 0 864 3 0 0 25 0 8 0 551106476 527925248 70010 33554432000 4194304 4964704 140735565610784 18446744073709551615 4325321 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4099/tid=4103] ppid=4097 vsize=515552 CPUtime=8.69 cores=0-7
/proc/4099/task/4103/stat : 4103 (claspmt) R 4097 4099 3873 0 -1 4202560 3550 0 0 0 867 2 0 0 25 0 8 0 551106476 527925248 70010 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4099/tid=4104] ppid=4097 vsize=515552 CPUtime=8.68 cores=0-7
/proc/4099/task/4104/stat : 4104 (claspmt) R 4097 4099 3873 0 -1 4202560 6708 0 0 0 864 4 0 0 25 0 8 0 551106476 527925248 70010 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4099/tid=4105] ppid=4097 vsize=515552 CPUtime=8.67 cores=0-7
/proc/4099/task/4105/stat : 4105 (claspmt) R 4097 4099 3873 0 -1 4202560 5813 0 0 0 864 3 0 0 25 0 8 0 551106476 527925248 70010 33554432000 4194304 4964704 140735565610784 18446744073709551615 4627412 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4099/tid=4106] ppid=4097 vsize=515552 CPUtime=9.75 cores=0-7
/proc/4099/task/4106/stat : 4106 (claspmt) R 4097 4099 3873 0 -1 4202560 7560 0 0 0 971 4 0 0 25 0 8 0 551106476 527925248 70010 33554432000 4194304 4964704 140735565610784 18446744073709551615 4627259 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=4099/tid=4107] ppid=4097 vsize=515552 CPUtime=8.69 cores=0-7
/proc/4099/task/4107/stat : 4107 (claspmt) R 4097 4099 3873 0 -1 4202560 6873 0 0 0 866 3 0 0 25 0 8 0 551106476 527925248 70010 33554432000 4194304 4964704 140735565610784 18446744073709551615 4378499 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4099/tid=4108] ppid=4097 vsize=515552 CPUtime=8.61 cores=0-7
/proc/4099/task/4108/stat : 4108 (claspmt) R 4097 4099 3873 0 -1 4202560 9868 0 0 0 856 5 0 0 25 0 8 0 551106476 527925248 70010 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 86.16
Current children cumulated vsize (KiB) 515552

[startup+51.101 s]
/proc/loadavg: 7.14 7.73 7.89 9/273 4109
/proc/meminfo: memFree=14705744/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=947176 CPUtime=290.73 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 155249 0 0 0 28985 88 0 0 25 0 8 0 551104906 969908224 146829 33554432000 4194304 4964704 140735565610784 18446744073709551615 237278540485 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/4099/statm: 236794 146829 466 189 0 231143 0
[pid=4099/tid=4102] ppid=4097 vsize=947176 CPUtime=34.26 cores=0-7
/proc/4099/task/4102/stat : 4102 (claspmt) R 4097 4099 3873 0 -1 4202560 11431 0 0 0 3418 8 0 0 25 0 8 0 551106476 969908224 146829 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670288 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4099/tid=4103] ppid=4097 vsize=947176 CPUtime=34.29 cores=0-7
/proc/4099/task/4103/stat : 4103 (claspmt) R 4097 4099 3873 0 -1 4202560 7249 0 0 0 3425 4 0 0 25 0 8 0 551106476 969908224 146829 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4099/tid=4104] ppid=4097 vsize=947176 CPUtime=34.28 cores=0-7
/proc/4099/task/4104/stat : 4104 (claspmt) R 4097 4099 3873 0 -1 4202560 15350 0 0 0 3419 9 0 0 25 0 8 0 551106476 969908224 146829 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4099/tid=4105] ppid=4097 vsize=947176 CPUtime=34.24 cores=0-7
/proc/4099/task/4105/stat : 4105 (claspmt) R 4097 4099 3873 0 -1 4202560 12487 0 0 0 3414 10 0 0 25 0 8 0 551106476 969908224 146829 33554432000 4194304 4964704 140735565610784 18446744073709551615 4668942 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4099/tid=4106] ppid=4097 vsize=947176 CPUtime=35.34 cores=0-7
/proc/4099/task/4106/stat : 4106 (claspmt) R 4097 4099 3873 0 -1 4202560 19019 0 0 0 3522 12 0 0 25 0 8 0 551106476 969908224 146829 33554432000 4194304 4964704 140735565610784 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=4099/tid=4107] ppid=4097 vsize=947176 CPUtime=34.28 cores=0-7
/proc/4099/task/4107/stat : 4107 (claspmt) R 4097 4099 3873 0 -1 4202560 15378 0 0 0 3420 8 0 0 25 0 8 0 551106476 969908224 146829 33554432000 4194304 4964704 140735565610784 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4099/tid=4108] ppid=4097 vsize=947176 CPUtime=34.03 cores=0-7
/proc/4099/task/4108/stat : 4108 (claspmt) R 4097 4099 3873 0 -1 4202560 24672 0 0 0 3389 14 0 0 25 0 8 0 551106476 969908224 146829 33554432000 4194304 4964704 140735565610784 18446744073709551615 4654400 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 290.73
Current children cumulated vsize (KiB) 947176

[startup+102.311 s]
/proc/loadavg: 7.68 7.78 7.90 9/273 4110
/proc/meminfo: memFree=14293164/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=1329904 CPUtime=699.95 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 257221 0 0 0 69837 158 0 0 25 0 8 0 551104906 1361821696 248800 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/4099/statm: 332476 248800 466 189 0 326825 0
[pid=4099/tid=4102] ppid=4097 vsize=1329904 CPUtime=85.47 cores=0-7
/proc/4099/task/4102/stat : 4102 (claspmt) R 4097 4099 3873 0 -1 4202560 15803 0 0 0 8534 13 0 0 25 0 8 0 551106476 1361821696 248800 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670260 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4099/tid=4103] ppid=4097 vsize=1329904 CPUtime=85.5 cores=0-7
/proc/4099/task/4103/stat : 4103 (claspmt) R 4097 4099 3873 0 -1 4202560 11688 0 0 0 8545 5 0 0 25 0 8 0 551106476 1361821696 248801 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4099/tid=4104] ppid=4097 vsize=1329904 CPUtime=85.48 cores=0-7
/proc/4099/task/4104/stat : 4104 (claspmt) R 4097 4099 3873 0 -1 4202560 23244 0 0 0 8534 14 0 0 25 0 8 0 551106476 1361821696 248801 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4099/tid=4105] ppid=4097 vsize=1329904 CPUtime=85.02 cores=0-7
/proc/4099/task/4105/stat : 4105 (claspmt) R 4097 4099 3873 0 -1 4202560 21060 0 0 0 8487 15 0 0 25 0 8 0 551106476 1361821696 248801 33554432000 4194304 4964704 140735565610784 18446744073709551615 4627257 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4099/tid=4106] ppid=4097 vsize=1329904 CPUtime=86.55 cores=0-7
/proc/4099/task/4106/stat : 4106 (claspmt) R 4097 4099 3873 0 -1 4202560 30004 0 0 0 8636 19 0 0 25 0 8 0 551106476 1361821696 248801 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=4099/tid=4107] ppid=4097 vsize=1329904 CPUtime=85.5 cores=0-7
/proc/4099/task/4107/stat : 4107 (claspmt) R 4097 4099 3873 0 -1 4202560 24649 0 0 0 8537 13 0 0 25 0 8 0 551106476 1361821696 248801 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4099/tid=4108] ppid=4097 vsize=1329904 CPUtime=85.24 cores=0-7
/proc/4099/task/4108/stat : 4108 (claspmt) R 4097 4099 3873 0 -1 4202560 47910 0 0 0 8491 33 0 0 25 0 8 0 551106476 1361821696 248801 33554432000 4194304 4964704 140735565610784 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 699.95
Current children cumulated vsize (KiB) 1329904

[startup+162.301 s]
/proc/loadavg: 7.93 7.83 7.91 9/273 4112
/proc/meminfo: memFree=14011780/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=1499536 CPUtime=1179.37 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 327230 0 0 0 117717 220 0 0 25 0 8 0 551104906 1535524864 318802 33554432000 4194304 4964704 140735565610784 18446744073709551615 4333097 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/4099/statm: 374884 318802 466 189 0 369233 0
[pid=4099/tid=4102] ppid=4097 vsize=1499536 CPUtime=145.45 cores=0-7
/proc/4099/task/4102/stat : 4102 (claspmt) R 4097 4099 3873 0 -1 4202560 21152 0 0 0 14527 18 0 0 24 0 8 0 551106476 1535524864 318802 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670431 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4099/tid=4103] ppid=4097 vsize=1499536 CPUtime=145.49 cores=0-7
/proc/4099/task/4103/stat : 4103 (claspmt) R 4097 4099 3873 0 -1 4202560 13638 0 0 0 14542 7 0 0 25 0 8 0 551106476 1535524864 318802 33554432000 4194304 4964704 140735565610784 18446744073709551615 237278538484 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4099/tid=4104] ppid=4097 vsize=1499536 CPUtime=145.46 cores=0-7
/proc/4099/task/4104/stat : 4104 (claspmt) R 4097 4099 3873 0 -1 4202560 26408 0 0 0 14530 16 0 0 25 0 8 0 551106476 1535524864 318802 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4099/tid=4105] ppid=4097 vsize=1499536 CPUtime=144.52 cores=0-7
/proc/4099/task/4105/stat : 4105 (claspmt) R 4097 4099 3873 0 -1 4202560 27944 0 0 0 14428 24 0 0 25 0 8 0 551106476 1535524864 318802 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4099/tid=4106] ppid=4097 vsize=1499536 CPUtime=146.54 cores=0-7
/proc/4099/task/4106/stat : 4106 (claspmt) R 4097 4099 3873 0 -1 4202560 35152 0 0 0 14631 23 0 0 25 0 8 0 551106476 1535524864 318802 33554432000 4194304 4964704 140735565610784 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=4099/tid=4107] ppid=4097 vsize=1499536 CPUtime=145.49 cores=0-7
/proc/4099/task/4107/stat : 4107 (claspmt) R 4097 4099 3873 0 -1 4202560 26040 0 0 0 14535 14 0 0 25 0 8 0 551106476 1535524864 318802 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670215 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4099/tid=4108] ppid=4097 vsize=1499536 CPUtime=145.23 cores=0-7
/proc/4099/task/4108/stat : 4108 (claspmt) R 4097 4099 3873 0 -1 4202560 67159 0 0 0 14473 50 0 0 25 0 8 0 551106476 1535524864 318802 33554432000 4194304 4964704 140735565610784 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1179.37
Current children cumulated vsize (KiB) 1499536

[startup+222.301 s]
/proc/loadavg: 8.02 7.88 7.91 9/273 4122
/proc/meminfo: memFree=13744828/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=1872016 CPUtime=1658.82 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 394143 0 0 0 165606 276 0 0 25 0 8 0 551104906 1916944384 385714 33554432000 4194304 4964704 140735565610784 18446744073709551615 4320955 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/4099/statm: 468004 385714 466 189 0 462353 0
[pid=4099/tid=4102] ppid=4097 vsize=1872016 CPUtime=205.41 cores=0-7
/proc/4099/task/4102/stat : 4102 (claspmt) R 4097 4099 3873 0 -1 4202560 26267 0 0 0 20519 22 0 0 25 0 8 0 551106476 1916944384 385714 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0

################
# More data... #
################

[pid=4099/tid=4105] ppid=4097 vsize=5668144 CPUtime=4700.77 cores=0-7
/proc/4099/task/4105/stat : 4105 (claspmt) R 4097 4099 3873 0 -1 4202560 93513 0 0 0 469969 108 0 0 25 0 8 0 551106476 5804179456 1339221 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4099/tid=4106] ppid=4097 vsize=5668144 CPUtime=4706.41 cores=0-7
/proc/4099/task/4106/stat : 4106 (claspmt) R 4097 4099 3873 0 -1 4202560 73663 0 0 0 470582 59 0 0 25 0 8 0 551106476 5804179456 1339221 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670241 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=4099/tid=4107] ppid=4097 vsize=5668144 CPUtime=4695.95 cores=0-7
/proc/4099/task/4107/stat : 4107 (claspmt) R 4097 4099 3873 0 -1 4202560 79743 0 0 0 469508 87 0 0 25 0 8 0 551106476 5804179456 1339221 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4099/tid=4108] ppid=4097 vsize=5668144 CPUtime=4691.23 cores=0-7
/proc/4099/task/4108/stat : 4108 (claspmt) R 4097 4099 3873 0 -1 4202560 457349 0 0 0 468554 569 0 0 25 0 8 0 551106476 5804179456 1339221 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37618.6
Current children cumulated vsize (KiB) 5668144

[startup+4782.3 s]
/proc/loadavg: 8.02 8.04 8.00 9/273 4259
/proc/meminfo: memFree=9909240/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=5733680 CPUtime=38098.1 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 1354389 0 0 0 3808408 1400 0 0 25 0 8 0 551104906 5871288320 1342217 33554432000 4194304 4964704 140735565610784 18446744073709551615 4342232 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/4099/statm: 1433420 1342217 466 189 0 1427769 0
[pid=4099/tid=4102] ppid=4097 vsize=5733680 CPUtime=4754.45 cores=0-7
/proc/4099/task/4102/stat : 4102 (claspmt) R 4097 4099 3873 0 -1 4202560 82316 0 0 0 475345 100 0 0 25 0 8 0 551106476 5871288320 1342217 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4099/tid=4103] ppid=4097 vsize=5733680 CPUtime=4765.36 cores=0-7
/proc/4099/task/4103/stat : 4103 (claspmt) R 4097 4099 3873 0 -1 4202560 79287 0 0 0 476506 30 0 0 25 0 8 0 551106476 5871288320 1342217 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4099/tid=4104] ppid=4097 vsize=5733680 CPUtime=4763.83 cores=0-7
/proc/4099/task/4104/stat : 4104 (claspmt) R 4097 4099 3873 0 -1 4202560 71615 0 0 0 476325 58 0 0 25 0 8 0 551106476 5871288320 1342217 33554432000 4194304 4964704 140735565610784 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4099/tid=4105] ppid=4097 vsize=5733680 CPUtime=4760.76 cores=0-7
/proc/4099/task/4105/stat : 4105 (claspmt) R 4097 4099 3873 0 -1 4202560 93515 0 0 0 475968 108 0 0 25 0 8 0 551106476 5871288320 1342217 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4099/tid=4106] ppid=4097 vsize=5733680 CPUtime=4766.41 cores=0-7
/proc/4099/task/4106/stat : 4106 (claspmt) R 4097 4099 3873 0 -1 4202560 73687 0 0 0 476581 60 0 0 25 0 8 0 551106476 5871288320 1342217 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=4099/tid=4107] ppid=4097 vsize=5733680 CPUtime=4755.46 cores=0-7
/proc/4099/task/4107/stat : 4107 (claspmt) R 4097 4099 3873 0 -1 4202560 79743 0 0 0 475459 87 0 0 25 0 8 0 551106476 5871288320 1342217 33554432000 4194304 4964704 140735565610784 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4099/tid=4108] ppid=4097 vsize=5733680 CPUtime=4751.22 cores=0-7
/proc/4099/task/4108/stat : 4108 (claspmt) R 4097 4099 3873 0 -1 4202560 457349 0 0 0 474553 569 0 0 25 0 8 0 551106476 5871288320 1342217 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38098.1
Current children cumulated vsize (KiB) 5733680

[startup+4842.3 s]
/proc/loadavg: 8.00 8.03 8.00 9/273 4261
/proc/meminfo: memFree=9892492/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=5733680 CPUtime=38577.6 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 1358549 0 0 0 3856355 1403 0 0 25 0 8 0 551104906 5871288320 1346377 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/4099/statm: 1433420 1346377 466 189 0 1427769 0
[pid=4099/tid=4102] ppid=4097 vsize=5733680 CPUtime=4814.45 cores=0-7
/proc/4099/task/4102/stat : 4102 (claspmt) R 4097 4099 3873 0 -1 4202560 83732 0 0 0 481345 100 0 0 25 0 8 0 551106476 5871288320 1346377 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670230 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4099/tid=4103] ppid=4097 vsize=5733680 CPUtime=4825.36 cores=0-7
/proc/4099/task/4103/stat : 4103 (claspmt) R 4097 4099 3873 0 -1 4202560 79443 0 0 0 482506 30 0 0 25 0 8 0 551106476 5871288320 1346377 33554432000 4194304 4964704 140735565610784 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4099/tid=4104] ppid=4097 vsize=5733680 CPUtime=4823.83 cores=0-7
/proc/4099/task/4104/stat : 4104 (claspmt) R 4097 4099 3873 0 -1 4202560 71615 0 0 0 482325 58 0 0 25 0 8 0 551106476 5871288320 1346377 33554432000 4194304 4964704 140735565610784 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4099/tid=4105] ppid=4097 vsize=5733680 CPUtime=4820.76 cores=0-7
/proc/4099/task/4105/stat : 4105 (claspmt) R 4097 4099 3873 0 -1 4202560 93522 0 0 0 481968 108 0 0 25 0 8 0 551106476 5871288320 1346377 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4099/tid=4106] ppid=4097 vsize=5733680 CPUtime=4826.41 cores=0-7
/proc/4099/task/4106/stat : 4106 (claspmt) R 4097 4099 3873 0 -1 4202560 73710 0 0 0 482581 60 0 0 25 0 8 0 551106476 5871288320 1346377 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=4099/tid=4107] ppid=4097 vsize=5733680 CPUtime=4814.96 cores=0-7
/proc/4099/task/4107/stat : 4107 (claspmt) R 4097 4099 3873 0 -1 4202560 79743 0 0 0 481409 87 0 0 25 0 8 0 551106476 5871288320 1346377 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4099/tid=4108] ppid=4097 vsize=5733680 CPUtime=4811.22 cores=0-7
/proc/4099/task/4108/stat : 4108 (claspmt) R 4097 4099 3873 0 -1 4202560 457349 0 0 0 480553 569 0 0 25 0 8 0 551106476 5871288320 1346377 33554432000 4194304 4964704 140735565610784 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38577.6
Current children cumulated vsize (KiB) 5733680

[startup+4902.3 s]
/proc/loadavg: 8.00 8.03 8.00 9/273 4262
/proc/meminfo: memFree=9878352/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=5733680 CPUtime=39057 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 1362607 0 0 0 3904290 1410 0 0 25 0 8 0 551104906 5871288320 1349948 33554432000 4194304 4964704 140735565610784 18446744073709551615 4338784 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/4099/statm: 1433420 1349948 466 189 0 1427769 0
[pid=4099/tid=4102] ppid=4097 vsize=5733680 CPUtime=4874.35 cores=0-7
/proc/4099/task/4102/stat : 4102 (claspmt) R 4097 4099 3873 0 -1 4202560 84264 0 0 0 487334 101 0 0 25 0 8 0 551106476 5871288320 1349948 33554432000 4194304 4964704 140735565610784 18446744073709551615 4627338 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4099/tid=4103] ppid=4097 vsize=5733680 CPUtime=4885.36 cores=0-7
/proc/4099/task/4103/stat : 4103 (claspmt) R 4097 4099 3873 0 -1 4202560 79785 0 0 0 488506 30 0 0 25 0 8 0 551106476 5871288320 1349948 33554432000 4194304 4964704 140735565610784 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4099/tid=4104] ppid=4097 vsize=5733680 CPUtime=4883.82 cores=0-7
/proc/4099/task/4104/stat : 4104 (claspmt) R 4097 4099 3873 0 -1 4202560 71622 0 0 0 488324 58 0 0 25 0 8 0 551106476 5871288320 1349948 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4099/tid=4105] ppid=4097 vsize=5733680 CPUtime=4880.76 cores=0-7
/proc/4099/task/4105/stat : 4105 (claspmt) R 4097 4099 3873 0 -1 4202560 93525 0 0 0 487968 108 0 0 25 0 8 0 551106476 5871288320 1349948 33554432000 4194304 4964704 140735565610784 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4099/tid=4106] ppid=4097 vsize=5733680 CPUtime=4886.41 cores=0-7
/proc/4099/task/4106/stat : 4106 (claspmt) R 4097 4099 3873 0 -1 4202560 73778 0 0 0 488581 60 0 0 25 0 8 0 551106476 5871288320 1349948 33554432000 4194304 4964704 140735565610784 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=4099/tid=4107] ppid=4097 vsize=5733680 CPUtime=4874.47 cores=0-7
/proc/4099/task/4107/stat : 4107 (claspmt) R 4097 4099 3873 0 -1 4202560 80593 0 0 0 487359 88 0 0 25 0 8 0 551106476 5871288320 1349948 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4099/tid=4108] ppid=4097 vsize=5733680 CPUtime=4871.23 cores=0-7
/proc/4099/task/4108/stat : 4108 (claspmt) R 4097 4099 3873 0 -1 4202560 459284 0 0 0 486550 573 0 0 25 0 8 0 551106476 5871288320 1349948 33554432000 4194304 4964704 140735565610784 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39057
Current children cumulated vsize (KiB) 5733680

[startup+4962.3 s]
/proc/loadavg: 8.07 8.04 8.00 9/273 4264
/proc/meminfo: memFree=9853044/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=5733680 CPUtime=39536.5 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 1369043 0 0 0 3952227 1423 0 0 25 0 8 0 551104906 5871288320 1356384 33554432000 4194304 4964704 140735565610784 18446744073709551615 4342144 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/4099/statm: 1433420 1356384 466 189 0 1427769 0
[pid=4099/tid=4102] ppid=4097 vsize=5733680 CPUtime=4934.36 cores=0-7
/proc/4099/task/4102/stat : 4102 (claspmt) R 4097 4099 3873 0 -1 4202560 85228 0 0 0 493333 103 0 0 25 0 8 0 551106476 5871288320 1356384 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4099/tid=4103] ppid=4097 vsize=5733680 CPUtime=4945.36 cores=0-7
/proc/4099/task/4103/stat : 4103 (claspmt) R 4097 4099 3873 0 -1 4202560 79900 0 0 0 494506 30 0 0 25 0 8 0 551106476 5871288320 1356384 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4099/tid=4104] ppid=4097 vsize=5733680 CPUtime=4943.81 cores=0-7
/proc/4099/task/4104/stat : 4104 (claspmt) R 4097 4099 3873 0 -1 4202560 71622 0 0 0 494323 58 0 0 25 0 8 0 551106476 5871288320 1356384 33554432000 4194304 4964704 140735565610784 18446744073709551615 4343157 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4099/tid=4105] ppid=4097 vsize=5733680 CPUtime=4940.77 cores=0-7
/proc/4099/task/4105/stat : 4105 (claspmt) R 4097 4099 3873 0 -1 4202560 93525 0 0 0 493969 108 0 0 25 0 8 0 551106476 5871288320 1356384 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4099/tid=4106] ppid=4097 vsize=5733680 CPUtime=4946.41 cores=0-7
/proc/4099/task/4106/stat : 4106 (claspmt) R 4097 4099 3873 0 -1 4202560 73804 0 0 0 494581 60 0 0 25 0 8 0 551106476 5871288320 1356384 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670243 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=4099/tid=4107] ppid=4097 vsize=5733680 CPUtime=4933.98 cores=0-7
/proc/4099/task/4107/stat : 4107 (claspmt) R 4097 4099 3873 0 -1 4202560 80593 0 0 0 493310 88 0 0 25 0 8 0 551106476 5871288320 1356384 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670269 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4099/tid=4108] ppid=4097 vsize=5733680 CPUtime=4931.23 cores=0-7
/proc/4099/task/4108/stat : 4108 (claspmt) R 4097 4099 3873 0 -1 4202560 464613 0 0 0 492539 584 0 0 25 0 8 0 551106476 5871288320 1356384 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39536.5
Current children cumulated vsize (KiB) 5733680



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5020.3 s]
/proc/loadavg: 8.08 8.04 8.00 10/273 4265
/proc/meminfo: memFree=9808144/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=5799216 CPUtime=40000.2 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 1380257 0 0 0 3998574 1442 0 0 25 0 8 0 551104906 5938397184 1367598 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670219 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/4099/statm: 1449804 1367598 466 189 0 1444153 0
[pid=4099/tid=4102] ppid=4097 vsize=5799216 CPUtime=4992.38 cores=0-7
/proc/4099/task/4102/stat : 4102 (claspmt) R 4097 4099 3873 0 -1 4202560 89461 0 0 0 499127 111 0 0 25 0 8 0 551106476 5938397184 1367598 33554432000 4194304 4964704 140735565610784 18446744073709551615 4321020 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4099/tid=4103] ppid=4097 vsize=5799216 CPUtime=5003.38 cores=0-7
/proc/4099/task/4103/stat : 4103 (claspmt) R 4097 4099 3873 0 -1 4202560 80129 0 0 0 500308 30 0 0 25 0 8 0 551106476 5938397184 1367598 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4099/tid=4104] ppid=4097 vsize=5799216 CPUtime=5001.83 cores=0-7
/proc/4099/task/4104/stat : 4104 (claspmt) R 4097 4099 3873 0 -1 4202560 71676 0 0 0 500124 59 0 0 25 0 8 0 551106476 5938397184 1367598 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4099/tid=4105] ppid=4097 vsize=5799216 CPUtime=4998.79 cores=0-7
/proc/4099/task/4105/stat : 4105 (claspmt) R 4097 4099 3873 0 -1 4202560 93528 0 0 0 499771 108 0 0 25 0 8 0 551106476 5938397184 1367598 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4099/tid=4106] ppid=4097 vsize=5799216 CPUtime=5004.43 cores=0-7
/proc/4099/task/4106/stat : 4106 (claspmt) R 4097 4099 3873 0 -1 4202560 73830 0 0 0 500383 60 0 0 25 0 8 0 551106476 5938397184 1367598 33554432000 4194304 4964704 140735565610784 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=4099/tid=4107] ppid=4097 vsize=5799216 CPUtime=4991.51 cores=0-7
/proc/4099/task/4107/stat : 4107 (claspmt) R 4097 4099 3873 0 -1 4202560 80593 0 0 0 499063 88 0 0 25 0 8 0 551106476 5938397184 1367598 33554432000 4194304 4964704 140735565610784 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4099/tid=4108] ppid=4097 vsize=5799216 CPUtime=4989.25 cores=0-7
/proc/4099/task/4108/stat : 4108 (claspmt) R 4097 4099 3873 0 -1 4202560 471051 0 0 0 498331 594 0 0 25 0 8 0 551106476 5938397184 1367598 33554432000 4194304 4964704 140735565610784 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.2
Current children cumulated vsize (KiB) 5799216

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5020.6 s]
/proc/loadavg: 8.08 8.04 8.00 10/273 4265
/proc/meminfo: memFree=9808144/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=5798788 CPUtime=40000.5 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 1380267 0 0 0 3998608 1442 0 0 24 0 1 0 551104906 5937958912 1365350 33554432000 4194304 4964704 140735565610784 18446744073709551615 237278534884 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/4099/statm: 1449697 1365350 476 189 0 1444046 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 5798788

[startup+5021.4 s]
/proc/loadavg: 8.08 8.04 8.00 2/267 4266
/proc/meminfo: memFree=9825676/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=5733252 CPUtime=40001.3 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 1380267 0 0 0 3998689 1442 0 0 25 0 1 0 551104906 5870850048 1362941 33554432000 4194304 4964704 140735565610784 18446744073709551615 237278536682 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/4099/statm: 1433313 1362941 476 189 0 1427662 0
Current children cumulated CPU time (s) 40001.3
Current children cumulated vsize (KiB) 5733252

[startup+5021.6 s]
/proc/loadavg: 8.08 8.04 8.00 2/267 4266
/proc/meminfo: memFree=9825676/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=4619140 CPUtime=40001.5 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 1380267 0 0 0 3998704 1446 0 0 25 0 1 0 551104906 4729999360 1076523 33554432000 4194304 4964704 140735565610784 18446744073709551615 237278922887 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/4099/statm: 1154785 1076523 476 189 0 1149134 0
Current children cumulated CPU time (s) 40001.5
Current children cumulated vsize (KiB) 4619140

[startup+5021.7 s]
/proc/loadavg: 8.08 8.04 8.00 2/267 4266
/proc/meminfo: memFree=9825676/32950904 swapFree=67082284/67111528
[pid=4099] ppid=4097 vsize=1932164 CPUtime=40001.6 cores=0-7
/proc/4099/stat : 4099 (claspmt) R 4097 4099 3873 0 -1 4202496 1380267 0 0 0 3998704 1456 0 0 25 0 1 0 551104906 1978535936 397012 33554432000 4194304 4964704 140735565610784 18446744073709551615 237278923111 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/4099/statm: 483041 396548 476 189 0 477390 0
Current children cumulated CPU time (s) 40001.6
Current children cumulated vsize (KiB) 1932164

Child status: 0
Real time (s): 5021.76
CPU time (s): 40001.7
CPU user time (s): 39987
CPU system time (s): 14.6238
CPU usage (%): 796.566
Max. virtual memory (cumulated for all children) (KiB): 5799216

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39987
system time used= 14.6238
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1380268
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 61642
involuntary context switches= 186748

runsolver used 11.9122 second user time and 29.4095 second system time

The end

Launcher Data 

Begin job on node121 at 2011-05-03 05:57:32
IDJOB=3322776
IDBENCH=88217
IDSOLVER=1711
FILE ID=node121/3322776-1304395051
RUNJOBID= node121-1304390537-3887
PBS_JOBID= 13181833
Free space on /tmp= 73616 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m62-q1327.5-n60-p15-s1351253.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322776-1304395051/watcher-3322776-1304395051 -o /tmp/evaluation-result-3322776-1304395051/solver-3322776-1304395051 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322776-1304395051.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= b23c52475e3d098cb9d5a92c025d6982
RANDOM SEED=1905237947

node121.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950904 kB
MemFree:      15286116 kB
Buffers:       2200880 kB
Cached:        9534608 kB
SwapCached:       2776 kB
Active:       14120892 kB
Inactive:      3004228 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950904 kB
LowFree:      15286116 kB
SwapTotal:    67111528 kB
SwapFree:     67082284 kB
Dirty:           15564 kB
Writeback:           0 kB
AnonPages:     5387612 kB
Mapped:          22644 kB
Slab:           452564 kB
PageTables:      23496 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586980 kB
Committed_AS: 30639032 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73612 MiB
End job on node121 at 2011-05-03 07:21:14