Trace number 3381064

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40001.2 5011.07

General information on the benchmark

Namecrafted/kullmann/GreenTao/
GreenTaok1k2/GreenTao_2-3-6_2072.cnf
MD5SUMd2ef996f95bbf68a3c980c797dfdc364
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2071
Number of clauses159039
Sum of the clauses size479052
Maximum clause length6
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3158394
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 5645

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...502599/instance-3381064-1305502599.cnf
1.19/1.21	c Solving...
40000.74/5010.72	
40000.74/5010.72	*** INTERRUPTED! ***
40000.74/5010.72	*** Shutting down threads...
40001.10/5011.06	s UNKNOWN
40001.10/5011.06	
40001.10/5011.06	c INTERRUPTED   : 1
40001.10/5011.06	c Models        : 0+    
40001.10/5011.06	c Time          : 5010.722s (Solving: 5009.51s 1st Model: 0.00s Unsat: 0.00s)
40001.10/5011.06	c CPU Time      : 40000.830s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3381064-1305502599/watcher-3381064-1305502599 -o /tmp/evaluation-result-3381064-1305502599/solver-3381064-1305502599 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381064-1305502599.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.41 7.89 7.96 1/438 11514
/proc/meminfo: memFree=21600308/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=22624 CPUtime=0 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 985 0 0 0 0 0 0 0 24 0 1 0 661860314 23166976 863 33554432000 4194304 4964800 140735275735312 18446744073709551615 253640859392 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/11514/statm: 5656 864 386 189 0 530 0

[startup+0.0489171 s]
/proc/loadavg: 7.41 7.89 7.96 1/438 11514
/proc/meminfo: memFree=21600308/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=30664 CPUtime=0.04 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 3644 0 0 0 4 0 0 0 24 0 1 0 661860314 31399936 2795 33554432000 4194304 4964800 140735275735312 18446744073709551615 4615530 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/11514/statm: 7666 2795 398 189 0 2540 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 30664

[startup+0.100908 s]
/proc/loadavg: 7.41 7.89 7.96 1/438 11514
/proc/meminfo: memFree=21600308/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=31716 CPUtime=0.09 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 3906 0 0 0 9 0 0 0 24 0 1 0 661860314 32477184 3057 33554432000 4194304 4964800 140735275735312 18446744073709551615 4607513 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/11514/statm: 7929 3057 402 189 0 2803 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 31716

[startup+0.300869 s]
/proc/loadavg: 7.41 7.89 7.96 1/438 11514
/proc/meminfo: memFree=21600308/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=31716 CPUtime=0.29 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 3907 0 0 0 29 0 0 0 25 0 1 0 661860314 32477184 3058 33554432000 4194304 4964800 140735275735312 18446744073709551615 4600360 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/11514/statm: 7929 3058 403 189 0 2803 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 31716

[startup+0.700807 s]
/proc/loadavg: 7.41 7.89 7.96 1/438 11514
/proc/meminfo: memFree=21600308/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=33696 CPUtime=0.69 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 4410 0 0 0 69 0 0 0 25 0 1 0 661860314 34504704 3561 33554432000 4194304 4964800 140735275735312 18446744073709551615 4600360 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/11514/statm: 8424 3561 403 189 0 3298 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 33696

[startup+1.50061 s]
/proc/loadavg: 7.41 7.89 7.96 2/439 11515
/proc/meminfo: memFree=21586048/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=399308 CPUtime=3.17 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 11995 0 0 0 310 7 0 0 18 0 8 0 661860314 408891392 10770 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670299 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11514/statm: 99827 10770 464 189 0 94176 0
Current children cumulated CPU time (s) 3.17
Current children cumulated vsize (KiB) 399308

[startup+3.10032 s]
/proc/loadavg: 7.41 7.89 7.96 9/446 11522
/proc/meminfo: memFree=21548424/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=400904 CPUtime=15.95 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 17652 0 0 0 1585 10 0 0 21 0 8 0 661860314 410525696 16427 33554432000 4194304 4964800 140735275735312 18446744073709551615 4660449 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11514/statm: 100226 16427 464 189 0 94575 0
[pid=11514/tid=11516] ppid=11512 vsize=400904 CPUtime=1.85 cores=0-7
/proc/11514/task/11516/stat : 11516 (claspmt) R 11512 11514 11307 0 -1 4202560 1300 0 0 0 184 1 0 0 20 0 8 0 661860436 410525696 16427 33554432000 4194304 4964800 140735275735312 18446744073709551615 4668702 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11514/tid=11517] ppid=11512 vsize=400904 CPUtime=1.83 cores=0-7
/proc/11514/task/11517/stat : 11517 (claspmt) R 11512 11514 11307 0 -1 4202560 1720 0 0 0 182 1 0 0 20 0 8 0 661860436 410525696 16427 33554432000 4194304 4964800 140735275735312 18446744073709551615 4668770 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11514/tid=11518] ppid=11512 vsize=400904 CPUtime=1.83 cores=0-7
/proc/11514/task/11518/stat : 11518 (claspmt) R 11512 11514 11307 0 -1 4202560 1490 0 0 0 183 0 0 0 20 0 8 0 661860436 410525696 16427 33554432000 4194304 4964800 140735275735312 18446744073709551615 4631102 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11514/tid=11519] ppid=11512 vsize=400904 CPUtime=1.84 cores=0-7
/proc/11514/task/11519/stat : 11519 (claspmt) R 11512 11514 11307 0 -1 4202560 1699 0 0 0 183 1 0 0 20 0 8 0 661860436 410525696 16427 33554432000 4194304 4964800 140735275735312 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11514/tid=11520] ppid=11512 vsize=400904 CPUtime=1.85 cores=0-7
/proc/11514/task/11520/stat : 11520 (claspmt) R 11512 11514 11307 0 -1 4202560 1359 0 0 0 185 0 0 0 20 0 8 0 661860436 410525696 16427 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11514/tid=11521] ppid=11512 vsize=400904 CPUtime=1.83 cores=0-7
/proc/11514/task/11521/stat : 11521 (claspmt) R 11512 11514 11307 0 -1 4202560 1818 0 0 0 182 1 0 0 20 0 8 0 661860436 410525696 16427 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670403 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11514/tid=11522] ppid=11512 vsize=400904 CPUtime=1.83 cores=0-7
/proc/11514/task/11522/stat : 11522 (claspmt) R 11512 11514 11307 0 -1 4202560 1732 0 0 0 182 1 0 0 20 0 8 0 661860436 410525696 16427 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 15.95
Current children cumulated vsize (KiB) 400904

[startup+6.30074 s]
/proc/loadavg: 7.46 7.89 7.96 9/446 11522
/proc/meminfo: memFree=21516676/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=405112 CPUtime=41.49 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 22894 0 0 0 4137 12 0 0 25 0 8 0 661860314 414834688 21669 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11514/statm: 101278 21669 466 189 0 95627 0
[pid=11514/tid=11516] ppid=11512 vsize=405112 CPUtime=5.05 cores=0-7
/proc/11514/task/11516/stat : 11516 (claspmt) R 11512 11514 11307 0 -1 4202560 1849 0 0 0 504 1 0 0 25 0 8 0 661860436 414834688 21669 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11514/tid=11517] ppid=11512 vsize=405112 CPUtime=5.04 cores=0-7
/proc/11514/task/11517/stat : 11517 (claspmt) R 11512 11514 11307 0 -1 4202560 2609 0 0 0 502 2 0 0 25 0 8 0 661860436 414834688 21669 33554432000 4194304 4964800 140735275735312 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11514/tid=11518] ppid=11512 vsize=405112 CPUtime=4.99 cores=0-7
/proc/11514/task/11518/stat : 11518 (claspmt) R 11512 11514 11307 0 -1 4202560 2105 0 0 0 498 1 0 0 25 0 8 0 661860436 414834688 21669 33554432000 4194304 4964800 140735275735312 18446744073709551615 4627281 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11514/tid=11519] ppid=11512 vsize=405112 CPUtime=5.03 cores=0-7
/proc/11514/task/11519/stat : 11519 (claspmt) R 11512 11514 11307 0 -1 4202560 2486 0 0 0 502 1 0 0 25 0 8 0 661860436 414834688 21669 33554432000 4194304 4964800 140735275735312 18446744073709551615 4342186 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11514/tid=11520] ppid=11512 vsize=405112 CPUtime=5.05 cores=0-7
/proc/11514/task/11520/stat : 11520 (claspmt) R 11512 11514 11307 0 -1 4202560 2057 0 0 0 505 0 0 0 25 0 8 0 661860436 414834688 21669 33554432000 4194304 4964800 140735275735312 18446744073709551615 4320872 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11514/tid=11521] ppid=11512 vsize=405112 CPUtime=5.03 cores=0-7
/proc/11514/task/11521/stat : 11521 (claspmt) R 11512 11514 11307 0 -1 4202560 2346 0 0 0 502 1 0 0 25 0 8 0 661860436 414834688 21669 33554432000 4194304 4964800 140735275735312 18446744073709551615 4668689 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11514/tid=11522] ppid=11512 vsize=405112 CPUtime=5.04 cores=0-7
/proc/11514/task/11522/stat : 11522 (claspmt) R 11512 11514 11307 0 -1 4202560 2458 0 0 0 502 2 0 0 24 0 8 0 661860436 414834688 21669 33554432000 4194304 4964800 140735275735312 18446744073709551615 4668763 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 41.49
Current children cumulated vsize (KiB) 405112

[startup+12.7006 s]
/proc/loadavg: 7.50 7.89 7.96 9/446 11522
/proc/meminfo: memFree=21484700/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=409404 CPUtime=92.6 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 31176 0 0 0 9242 18 0 0 25 0 8 0 661860314 419229696 29951 33554432000 4194304 4964800 140735275735312 18446744073709551615 4627204 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11514/statm: 102351 29951 466 189 0 96700 0
[pid=11514/tid=11516] ppid=11512 vsize=409404 CPUtime=11.45 cores=0-7
/proc/11514/task/11516/stat : 11516 (claspmt) R 11512 11514 11307 0 -1 4202560 2775 0 0 0 1143 2 0 0 25 0 8 0 661860436 419229696 29951 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670437 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11514/tid=11517] ppid=11512 vsize=409404 CPUtime=11.44 cores=0-7
/proc/11514/task/11517/stat : 11517 (claspmt) R 11512 11514 11307 0 -1 4202560 4189 0 0 0 1141 3 0 0 25 0 8 0 661860436 419229696 29951 33554432000 4194304 4964800 140735275735312 18446744073709551615 4668689 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11514/tid=11518] ppid=11512 vsize=409404 CPUtime=11.29 cores=0-7
/proc/11514/task/11518/stat : 11518 (claspmt) R 11512 11514 11307 0 -1 4202560 2788 0 0 0 1128 1 0 0 25 0 8 0 661860436 419229696 29951 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11514/tid=11519] ppid=11512 vsize=409404 CPUtime=11.43 cores=0-7
/proc/11514/task/11519/stat : 11519 (claspmt) R 11512 11514 11307 0 -1 4202560 3458 0 0 0 1142 1 0 0 25 0 8 0 661860436 419229696 29951 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11514/tid=11520] ppid=11512 vsize=409404 CPUtime=11.45 cores=0-7
/proc/11514/task/11520/stat : 11520 (claspmt) R 11512 11514 11307 0 -1 4202560 3004 0 0 0 1144 1 0 0 25 0 8 0 661860436 419229696 29951 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11514/tid=11521] ppid=11512 vsize=409404 CPUtime=11.44 cores=0-7
/proc/11514/task/11521/stat : 11521 (claspmt) R 11512 11514 11307 0 -1 4202560 3259 0 0 0 1142 2 0 0 25 0 8 0 661860436 419229696 29951 33554432000 4194304 4964800 140735275735312 18446744073709551615 4632224 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11514/tid=11522] ppid=11512 vsize=409404 CPUtime=11.44 cores=0-7
/proc/11514/task/11522/stat : 11522 (claspmt) R 11512 11514 11307 0 -1 4202560 3866 0 0 0 1141 3 0 0 25 0 8 0 661860436 419229696 29951 33554432000 4194304 4964800 140735275735312 18446744073709551615 4660099 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 92.6
Current children cumulated vsize (KiB) 409404

[startup+25.5012 s]
/proc/loadavg: 7.68 7.91 7.96 9/446 11522
/proc/meminfo: memFree=21451100/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=414416 CPUtime=194.8 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 39260 0 0 0 19457 23 0 0 25 0 8 0 661860314 424361984 38035 33554432000 4194304 4964800 140735275735312 18446744073709551615 4627204 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11514/statm: 103604 38035 466 189 0 97953 0
[pid=11514/tid=11516] ppid=11512 vsize=414416 CPUtime=24.24 cores=0-7
/proc/11514/task/11516/stat : 11516 (claspmt) R 11512 11514 11307 0 -1 4202560 3637 0 0 0 2422 2 0 0 25 0 8 0 661860436 424361984 38035 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11514/tid=11517] ppid=11512 vsize=414416 CPUtime=24.24 cores=0-7
/proc/11514/task/11517/stat : 11517 (claspmt) R 11512 11514 11307 0 -1 4202560 5918 0 0 0 2421 3 0 0 25 0 8 0 661860436 424361984 38035 33554432000 4194304 4964800 140735275735312 18446744073709551615 4654500 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11514/tid=11518] ppid=11512 vsize=414416 CPUtime=23.9 cores=0-7
/proc/11514/task/11518/stat : 11518 (claspmt) R 11512 11514 11307 0 -1 4202560 3767 0 0 0 2388 2 0 0 25 0 8 0 661860436 424361984 38035 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11514/tid=11519] ppid=11512 vsize=414416 CPUtime=24.23 cores=0-7
/proc/11514/task/11519/stat : 11519 (claspmt) R 11512 11514 11307 0 -1 4202560 4316 0 0 0 2421 2 0 0 25 0 8 0 661860436 424361984 38035 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670403 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11514/tid=11520] ppid=11512 vsize=414416 CPUtime=24.24 cores=0-7
/proc/11514/task/11520/stat : 11520 (claspmt) R 11512 11514 11307 0 -1 4202560 3931 0 0 0 2423 1 0 0 25 0 8 0 661860436 424361984 38035 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11514/tid=11521] ppid=11512 vsize=414416 CPUtime=24.24 cores=0-7
/proc/11514/task/11521/stat : 11521 (claspmt) R 11512 11514 11307 0 -1 4202560 4009 0 0 0 2421 3 0 0 25 0 8 0 661860436 424361984 38035 33554432000 4194304 4964800 140735275735312 18446744073709551615 4654532 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11514/tid=11522] ppid=11512 vsize=414416 CPUtime=24.24 cores=0-7
/proc/11514/task/11522/stat : 11522 (claspmt) R 11512 11514 11307 0 -1 4202560 5181 0 0 0 2420 4 0 0 25 0 8 0 661860436 424361984 38035 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 194.8
Current children cumulated vsize (KiB) 414416

[startup+51.1006 s]
/proc/loadavg: 7.79 7.92 7.96 9/446 11523
/proc/meminfo: memFree=21397872/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=427488 CPUtime=399.2 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 52573 0 0 0 39886 34 0 0 25 0 8 0 661860314 437747712 51324 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11514/statm: 106872 51324 466 189 0 101221 0
[pid=11514/tid=11516] ppid=11512 vsize=427488 CPUtime=49.83 cores=0-7
/proc/11514/task/11516/stat : 11516 (claspmt) R 11512 11514 11307 0 -1 4202560 5104 0 0 0 4980 3 0 0 25 0 8 0 661860436 437747712 51324 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670340 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11514/tid=11517] ppid=11512 vsize=427488 CPUtime=49.84 cores=0-7
/proc/11514/task/11517/stat : 11517 (claspmt) R 11512 11514 11307 0 -1 4202560 9036 0 0 0 4979 5 0 0 25 0 8 0 661860436 437747712 51324 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11514/tid=11518] ppid=11512 vsize=427488 CPUtime=49.13 cores=0-7
/proc/11514/task/11518/stat : 11518 (claspmt) R 11512 11514 11307 0 -1 4202560 5056 0 0 0 4910 3 0 0 25 0 8 0 661860436 437747712 51324 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11514/tid=11519] ppid=11512 vsize=427488 CPUtime=49.82 cores=0-7
/proc/11514/task/11519/stat : 11519 (claspmt) R 11512 11514 11307 0 -1 4202560 5629 0 0 0 4979 3 0 0 25 0 8 0 661860436 437747712 51324 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670264 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11514/tid=11520] ppid=11512 vsize=427488 CPUtime=49.85 cores=0-7
/proc/11514/task/11520/stat : 11520 (claspmt) R 11512 11514 11307 0 -1 4202560 5339 0 0 0 4982 3 0 0 25 0 8 0 661860436 437747712 51324 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11514/tid=11521] ppid=11512 vsize=427488 CPUtime=49.83 cores=0-7
/proc/11514/task/11521/stat : 11521 (claspmt) R 11512 11514 11307 0 -1 4202560 5536 0 0 0 4980 3 0 0 25 0 8 0 661860436 437747712 51324 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11514/tid=11522] ppid=11512 vsize=427488 CPUtime=49.83 cores=0-7
/proc/11514/task/11522/stat : 11522 (claspmt) R 11512 11514 11307 0 -1 4202560 6776 0 0 0 4978 5 0 0 25 0 8 0 661860436 437747712 51324 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 399.2
Current children cumulated vsize (KiB) 427488

[startup+102.317 s]
/proc/loadavg: 7.96 7.94 7.97 9/446 11524
/proc/meminfo: memFree=21346944/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=435696 CPUtime=808.13 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 65357 0 0 0 80769 44 0 0 25 0 8 0 661860314 446152704 64017 33554432000 4194304 4964800 140735275735312 18446744073709551615 4668725 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/11514/statm: 108924 64017 466 189 0 103273 0
[pid=11514/tid=11516] ppid=11512 vsize=435696 CPUtime=101.03 cores=0-7
/proc/11514/task/11516/stat : 11516 (claspmt) R 11512 11514 11307 0 -1 4202560 6472 0 0 0 10098 5 0 0 25 0 8 0 661860436 446152704 64017 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670321 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11514/tid=11517] ppid=11512 vsize=435696 CPUtime=101.05 cores=0-7
/proc/11514/task/11517/stat : 11517 (claspmt) R 11512 11514 11307 0 -1 4202560 10615 0 0 0 10099 6 0 0 25 0 8 0 661860436 446152704 64017 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11514/tid=11518] ppid=11512 vsize=435696 CPUtime=99.59 cores=0-7
/proc/11514/task/11518/stat : 11518 (claspmt) R 11512 11514 11307 0 -1 4202560 6453 0 0 0 9955 4 0 0 25 0 8 0 661860436 446152704 64017 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 7 0 0 0

################
# More data... #
################

[pid=11514/tid=11517] ppid=11512 vsize=1232696 CPUtime=4660.45 cores=0-7
/proc/11514/task/11517/stat : 11517 (claspmt) R 11512 11514 11307 0 -1 4202560 44957 0 0 0 466015 30 0 0 25 0 8 0 661860436 1262280704 250270 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11514/tid=11518] ppid=11512 vsize=1232696 CPUtime=4635.42 cores=0-7
/proc/11514/task/11518/stat : 11518 (claspmt) R 11512 11514 11307 0 -1 4202560 26422 0 0 0 463513 29 0 0 25 0 8 0 661860436 1262280704 250270 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11514/tid=11519] ppid=11512 vsize=1232696 CPUtime=4659.97 cores=0-7
/proc/11514/task/11519/stat : 11519 (claspmt) R 11512 11514 11307 0 -1 4202560 35880 0 0 0 465973 24 0 0 25 0 8 0 661860436 1262280704 250270 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11514/tid=11520] ppid=11512 vsize=1232696 CPUtime=4651.83 cores=0-7
/proc/11514/task/11520/stat : 11520 (claspmt) R 11512 11514 11307 0 -1 4202560 30609 0 0 0 465159 24 0 0 25 0 8 0 661860436 1262280704 250270 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11514/tid=11521] ppid=11512 vsize=1232696 CPUtime=4660.99 cores=0-7
/proc/11514/task/11521/stat : 11521 (claspmt) R 11512 11514 11307 0 -1 4202560 29259 0 0 0 466073 26 0 0 25 0 8 0 661860436 1262280704 250270 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11514/tid=11522] ppid=11512 vsize=1232696 CPUtime=4661.01 cores=0-7
/proc/11514/task/11522/stat : 11522 (claspmt) R 11512 11514 11307 0 -1 4202560 37599 0 0 0 466055 46 0 0 25 0 8 0 661860436 1262280704 250270 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 37218.7
Current children cumulated vsize (KiB) 1232696

[startup+4722.32 s]
/proc/loadavg: 8.11 8.03 8.01 9/446 11659
/proc/meminfo: memFree=20597112/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=1233516 CPUtime=37697.9 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 266019 0 0 0 3769538 254 0 0 25 0 8 0 661860314 1263120384 250709 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670403 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/11514/statm: 308379 250709 466 189 0 302728 0
[pid=11514/tid=11516] ppid=11512 vsize=1233516 CPUtime=4707.63 cores=0-7
/proc/11514/task/11516/stat : 11516 (claspmt) R 11512 11514 11307 0 -1 4202560 35087 0 0 0 470734 29 0 0 25 0 8 0 661860436 1263120384 250709 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11514/tid=11517] ppid=11512 vsize=1233516 CPUtime=4720.47 cores=0-7
/proc/11514/task/11517/stat : 11517 (claspmt) R 11512 11514 11307 0 -1 4202560 44957 0 0 0 472017 30 0 0 25 0 8 0 661860436 1263120384 250709 33554432000 4194304 4964800 140735275735312 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11514/tid=11518] ppid=11512 vsize=1233516 CPUtime=4695.43 cores=0-7
/proc/11514/task/11518/stat : 11518 (claspmt) R 11512 11514 11307 0 -1 4202560 26495 0 0 0 469514 29 0 0 25 0 8 0 661860436 1263120384 250709 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11514/tid=11519] ppid=11512 vsize=1233516 CPUtime=4719.98 cores=0-7
/proc/11514/task/11519/stat : 11519 (claspmt) R 11512 11514 11307 0 -1 4202560 35963 0 0 0 471973 25 0 0 25 0 8 0 661860436 1263120384 250709 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11514/tid=11520] ppid=11512 vsize=1233516 CPUtime=4711.85 cores=0-7
/proc/11514/task/11520/stat : 11520 (claspmt) R 11512 11514 11307 0 -1 4202560 30609 0 0 0 471160 25 0 0 25 0 8 0 661860436 1263120384 250709 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11514/tid=11521] ppid=11512 vsize=1233516 CPUtime=4721.01 cores=0-7
/proc/11514/task/11521/stat : 11521 (claspmt) R 11512 11514 11307 0 -1 4202560 29259 0 0 0 472075 26 0 0 25 0 8 0 661860436 1263120384 250709 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11514/tid=11522] ppid=11512 vsize=1233516 CPUtime=4721.03 cores=0-7
/proc/11514/task/11522/stat : 11522 (claspmt) R 11512 11514 11307 0 -1 4202560 37748 0 0 0 472056 47 0 0 25 0 8 0 661860436 1263120384 250709 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 37697.9
Current children cumulated vsize (KiB) 1233516

[startup+4782.3 s]
/proc/loadavg: 8.11 8.04 8.00 9/446 11661
/proc/meminfo: memFree=20594244/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=1233516 CPUtime=38176.9 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 266967 0 0 0 3817433 256 0 0 25 0 8 0 661860314 1263120384 251571 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670264 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/11514/statm: 308379 251571 466 189 0 302728 0
[pid=11514/tid=11516] ppid=11512 vsize=1233516 CPUtime=4767.28 cores=0-7
/proc/11514/task/11516/stat : 11516 (claspmt) R 11512 11514 11307 0 -1 4202560 35087 0 0 0 476699 29 0 0 25 0 8 0 661860436 1263120384 251571 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11514/tid=11517] ppid=11512 vsize=1233516 CPUtime=4779.9 cores=0-7
/proc/11514/task/11517/stat : 11517 (claspmt) R 11512 11514 11307 0 -1 4202560 44966 0 0 0 477960 30 0 0 25 0 8 0 661860436 1263120384 251571 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11514/tid=11518] ppid=11512 vsize=1233516 CPUtime=4755.42 cores=0-7
/proc/11514/task/11518/stat : 11518 (claspmt) R 11512 11514 11307 0 -1 4202560 26534 0 0 0 475512 30 0 0 25 0 8 0 661860436 1263120384 251571 33554432000 4194304 4964800 140735275735312 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11514/tid=11519] ppid=11512 vsize=1233516 CPUtime=4779.94 cores=0-7
/proc/11514/task/11519/stat : 11519 (claspmt) R 11512 11514 11307 0 -1 4202560 36389 0 0 0 477969 25 0 0 25 0 8 0 661860436 1263120384 251571 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11514/tid=11520] ppid=11512 vsize=1233516 CPUtime=4771.84 cores=0-7
/proc/11514/task/11520/stat : 11520 (claspmt) R 11512 11514 11307 0 -1 4202560 30730 0 0 0 477159 25 0 0 25 0 8 0 661860436 1263120384 251571 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11514/tid=11521] ppid=11512 vsize=1233516 CPUtime=4780.99 cores=0-7
/proc/11514/task/11521/stat : 11521 (claspmt) R 11512 11514 11307 0 -1 4202560 29468 0 0 0 478073 26 0 0 25 0 8 0 661860436 1263120384 251571 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11514/tid=11522] ppid=11512 vsize=1233516 CPUtime=4781.01 cores=0-7
/proc/11514/task/11522/stat : 11522 (claspmt) R 11512 11514 11307 0 -1 4202560 37855 0 0 0 478054 47 0 0 25 0 8 0 661860436 1263120384 251571 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38176.9
Current children cumulated vsize (KiB) 1233516

[startup+4842.3 s]
/proc/loadavg: 8.25 8.08 8.02 9/446 11663
/proc/meminfo: memFree=20590640/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=1233676 CPUtime=38656 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 267820 0 0 0 3865339 258 0 0 25 0 8 0 661860314 1263284224 252088 33554432000 4194304 4964800 140735275735312 18446744073709551615 4320909 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/11514/statm: 308419 252088 466 189 0 302768 0
[pid=11514/tid=11516] ppid=11512 vsize=1233676 CPUtime=4827.27 cores=0-7
/proc/11514/task/11516/stat : 11516 (claspmt) R 11512 11514 11307 0 -1 4202560 35306 0 0 0 482698 29 0 0 25 0 8 0 661860436 1263284224 252088 33554432000 4194304 4964800 140735275735312 18446744073709551615 4668770 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11514/tid=11517] ppid=11512 vsize=1233676 CPUtime=4839.02 cores=0-7
/proc/11514/task/11517/stat : 11517 (claspmt) R 11512 11514 11307 0 -1 4202560 44967 0 0 0 483872 30 0 0 25 0 8 0 661860436 1263284224 252088 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11514/tid=11518] ppid=11512 vsize=1233676 CPUtime=4815.42 cores=0-7
/proc/11514/task/11518/stat : 11518 (claspmt) R 11512 11514 11307 0 -1 4202560 26543 0 0 0 481512 30 0 0 25 0 8 0 661860436 1263284224 252088 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11514/tid=11519] ppid=11512 vsize=1233676 CPUtime=4839.93 cores=0-7
/proc/11514/task/11519/stat : 11519 (claspmt) R 11512 11514 11307 0 -1 4202560 36524 0 0 0 483967 26 0 0 25 0 8 0 661860436 1263284224 252088 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11514/tid=11520] ppid=11512 vsize=1233676 CPUtime=4831.83 cores=0-7
/proc/11514/task/11520/stat : 11520 (claspmt) R 11512 11514 11307 0 -1 4202560 31119 0 0 0 483158 25 0 0 25 0 8 0 661860436 1263284224 252088 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11514/tid=11521] ppid=11512 vsize=1233676 CPUtime=4840.99 cores=0-7
/proc/11514/task/11521/stat : 11521 (claspmt) R 11512 11514 11307 0 -1 4202560 29468 0 0 0 484072 27 0 0 25 0 8 0 661860436 1263284224 252088 33554432000 4194304 4964800 140735275735312 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11514/tid=11522] ppid=11512 vsize=1233676 CPUtime=4841.02 cores=0-7
/proc/11514/task/11522/stat : 11522 (claspmt) R 11512 11514 11307 0 -1 4202560 37940 0 0 0 484054 48 0 0 25 0 8 0 661860436 1263284224 252088 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670256 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38656
Current children cumulated vsize (KiB) 1233676

[startup+4902.3 s]
/proc/loadavg: 8.16 8.08 8.01 9/446 11664
/proc/meminfo: memFree=20586632/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=1233864 CPUtime=39135.1 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 269028 0 0 0 3913251 259 0 0 25 0 8 0 661860314 1263476736 253210 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/11514/statm: 308466 253210 466 189 0 302815 0
[pid=11514/tid=11516] ppid=11512 vsize=1233864 CPUtime=4887.27 cores=0-7
/proc/11514/task/11516/stat : 11516 (claspmt) R 11512 11514 11307 0 -1 4202560 35378 0 0 0 488698 29 0 0 25 0 8 0 661860436 1263476736 253210 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11514/tid=11517] ppid=11512 vsize=1233864 CPUtime=4898.16 cores=0-7
/proc/11514/task/11517/stat : 11517 (claspmt) R 11512 11514 11307 0 -1 4202560 45422 0 0 0 489785 31 0 0 25 0 8 0 661860436 1263476736 253210 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11514/tid=11518] ppid=11512 vsize=1233864 CPUtime=4875.41 cores=0-7
/proc/11514/task/11518/stat : 11518 (claspmt) R 11512 11514 11307 0 -1 4202560 26781 0 0 0 487511 30 0 0 25 0 8 0 661860436 1263476736 253210 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11514/tid=11519] ppid=11512 vsize=1233864 CPUtime=4899.92 cores=0-7
/proc/11514/task/11519/stat : 11519 (claspmt) R 11512 11514 11307 0 -1 4202560 36536 0 0 0 489966 26 0 0 25 0 8 0 661860436 1263476736 253210 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11514/tid=11520] ppid=11512 vsize=1233864 CPUtime=4891.83 cores=0-7
/proc/11514/task/11520/stat : 11520 (claspmt) R 11512 11514 11307 0 -1 4202560 31352 0 0 0 489158 25 0 0 25 0 8 0 661860436 1263476736 253210 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11514/tid=11521] ppid=11512 vsize=1233864 CPUtime=4900.99 cores=0-7
/proc/11514/task/11521/stat : 11521 (claspmt) R 11512 11514 11307 0 -1 4202560 29481 0 0 0 490072 27 0 0 25 0 8 0 661860436 1263476736 253210 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11514/tid=11522] ppid=11512 vsize=1233864 CPUtime=4901.02 cores=0-7
/proc/11514/task/11522/stat : 11522 (claspmt) R 11512 11514 11307 0 -1 4202560 38069 0 0 0 490054 48 0 0 25 0 8 0 661860436 1263476736 253210 33554432000 4194304 4964800 140735275735312 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39135.1
Current children cumulated vsize (KiB) 1233864

[startup+4962.3 s]
/proc/loadavg: 8.13 8.08 8.01 9/446 11666
/proc/meminfo: memFree=20582636/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=1234788 CPUtime=39614.2 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 270301 0 0 0 3961158 261 0 0 25 0 8 0 661860314 1264422912 254235 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/11514/statm: 308697 254235 466 189 0 303046 0
[pid=11514/tid=11516] ppid=11512 vsize=1234788 CPUtime=4947.25 cores=0-7
/proc/11514/task/11516/stat : 11516 (claspmt) R 11512 11514 11307 0 -1 4202560 35696 0 0 0 494696 29 0 0 25 0 8 0 661860436 1264422912 254235 33554432000 4194304 4964800 140735275735312 18446744073709551615 4653186 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11514/tid=11517] ppid=11512 vsize=1234788 CPUtime=4957.27 cores=0-7
/proc/11514/task/11517/stat : 11517 (claspmt) R 11512 11514 11307 0 -1 4202560 45509 0 0 0 495696 31 0 0 25 0 8 0 661860436 1264422912 254235 33554432000 4194304 4964800 140735275735312 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11514/tid=11518] ppid=11512 vsize=1234788 CPUtime=4935.41 cores=0-7
/proc/11514/task/11518/stat : 11518 (claspmt) R 11512 11514 11307 0 -1 4202560 26814 0 0 0 493511 30 0 0 25 0 8 0 661860436 1264422912 254235 33554432000 4194304 4964800 140735275735312 18446744073709551615 4668689 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11514/tid=11519] ppid=11512 vsize=1234788 CPUtime=4959.9 cores=0-7
/proc/11514/task/11519/stat : 11519 (claspmt) R 11512 11514 11307 0 -1 4202560 37047 0 0 0 495964 26 0 0 25 0 8 0 661860436 1264422912 254235 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11514/tid=11520] ppid=11512 vsize=1234788 CPUtime=4951.84 cores=0-7
/proc/11514/task/11520/stat : 11520 (claspmt) R 11512 11514 11307 0 -1 4202560 31594 0 0 0 495158 26 0 0 25 0 8 0 661860436 1264422912 254235 33554432000 4194304 4964800 140735275735312 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11514/tid=11521] ppid=11512 vsize=1234788 CPUtime=4960.99 cores=0-7
/proc/11514/task/11521/stat : 11521 (claspmt) R 11512 11514 11307 0 -1 4202560 29481 0 0 0 496072 27 0 0 25 0 8 0 661860436 1264422912 254235 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11514/tid=11522] ppid=11512 vsize=1234788 CPUtime=4961.02 cores=0-7
/proc/11514/task/11522/stat : 11522 (claspmt) R 11512 11514 11307 0 -1 4202560 38147 0 0 0 496054 48 0 0 25 0 8 0 661860436 1264422912 254235 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39614.2
Current children cumulated vsize (KiB) 1234788



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.7 s]
/proc/loadavg: 8.13 8.08 8.01 9/446 11667
/proc/meminfo: memFree=20580016/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=1235612 CPUtime=40000.7 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 271000 0 0 0 3999811 263 0 0 25 0 8 0 661860314 1265266688 254847 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/11514/statm: 308903 254847 466 189 0 303252 0
[pid=11514/tid=11516] ppid=11512 vsize=1235612 CPUtime=4995.67 cores=0-7
/proc/11514/task/11516/stat : 11516 (claspmt) R 11512 11514 11307 0 -1 4202560 35789 0 0 0 499538 29 0 0 25 0 8 0 661860436 1265266688 254847 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11514/tid=11517] ppid=11512 vsize=1235612 CPUtime=5004.96 cores=0-7
/proc/11514/task/11517/stat : 11517 (claspmt) R 11512 11514 11307 0 -1 4202560 45610 0 0 0 500465 31 0 0 25 0 8 0 661860436 1265266688 254847 33554432000 4194304 4964800 140735275735312 18446744073709551615 4668709 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11514/tid=11518] ppid=11512 vsize=1235612 CPUtime=4983.83 cores=0-7
/proc/11514/task/11518/stat : 11518 (claspmt) R 11512 11514 11307 0 -1 4202560 26850 0 0 0 498353 30 0 0 25 0 8 0 661860436 1265266688 254847 33554432000 4194304 4964800 140735275735312 18446744073709551615 4668794 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11514/tid=11519] ppid=11512 vsize=1235612 CPUtime=5008.3 cores=0-7
/proc/11514/task/11519/stat : 11519 (claspmt) R 11512 11514 11307 0 -1 4202560 37048 0 0 0 500804 26 0 0 25 0 8 0 661860436 1265266688 254847 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11514/tid=11520] ppid=11512 vsize=1235612 CPUtime=5000.26 cores=0-7
/proc/11514/task/11520/stat : 11520 (claspmt) R 11512 11514 11307 0 -1 4202560 31690 0 0 0 500000 26 0 0 25 0 8 0 661860436 1265266688 254847 33554432000 4194304 4964800 140735275735312 18446744073709551615 4321020 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11514/tid=11521] ppid=11512 vsize=1235612 CPUtime=5009.41 cores=0-7
/proc/11514/task/11521/stat : 11521 (claspmt) R 11512 11514 11307 0 -1 4202560 29633 0 0 0 500914 27 0 0 25 0 8 0 661860436 1265266688 254847 33554432000 4194304 4964800 140735275735312 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11514/tid=11522] ppid=11512 vsize=1235612 CPUtime=5009.43 cores=0-7
/proc/11514/task/11522/stat : 11522 (claspmt) R 11512 11514 11307 0 -1 4202560 38366 0 0 0 500895 48 0 0 25 0 8 0 661860436 1265266688 254847 33554432000 4194304 4964800 140735275735312 18446744073709551615 4668681 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 1235612

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5011 s]
/proc/loadavg: 8.13 8.08 8.01 9/446 11667
/proc/meminfo: memFree=20580016/32951124 swapFree=56449232/67111528
[pid=11514] ppid=11512 vsize=1225624 CPUtime=40001.1 cores=0-7
/proc/11514/stat : 11514 (claspmt) R 11512 11514 11307 0 -1 4202496 271010 0 0 0 3999847 263 0 0 25 0 1 0 661860314 1255038976 248599 33554432000 4194304 4964800 140735275735312 18446744073709551615 253640517390 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/11514/statm: 306406 248599 476 189 0 300755 0
Current children cumulated CPU time (s) 40001.1
Current children cumulated vsize (KiB) 1225624

Child status: 0
Real time (s): 5011.07
CPU time (s): 40001.2
CPU user time (s): 39998.5
CPU system time (s): 2.68559
CPU usage (%): 798.257
Max. virtual memory (cumulated for all children) (KiB): 1235612

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39998.5
system time used= 2.68559
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 271011
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 74586
involuntary context switches= 139065

runsolver used 19.982 second user time and 53.2959 second system time

The end

Launcher Data 

Begin job on node106 at 2011-05-16 01:36:39
IDJOB=3381064
IDBENCH=83245
IDSOLVER=1865
FILE ID=node106/3381064-1305502599
RUNJOBID= node106-1305497587-11321
PBS_JOBID= 13325443
Free space on /tmp= 73384 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/crafted/kullmann/GreenTao/GreenTaok1k2/GreenTao_2-3-6_2072.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3381064-1305502599/watcher-3381064-1305502599 -o /tmp/evaluation-result-3381064-1305502599/solver-3381064-1305502599 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381064-1305502599.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= d2ef996f95bbf68a3c980c797dfdc364
RANDOM SEED=251995869

node106.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.20
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21600836 kB
Buffers:        307500 kB
Cached:        6240984 kB
SwapCached:      27536 kB
Active:        2748500 kB
Inactive:      8342968 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21600836 kB
SwapTotal:    67111528 kB
SwapFree:     56449232 kB
Dirty:            3112 kB
Writeback:           0 kB
AnonPages:     4524296 kB
Mapped:          20060 kB
Slab:           129432 kB
PageTables:      59704 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 123523316 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73384 MiB
End job on node106 at 2011-05-16 03:00:12