Trace number 3380702

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40001.2 5012.21

General information on the benchmark

NameCRAFTED/rbsat/random/
forced/rbsat-v2640c305320gyes3.cnf
MD5SUMd6877629cc818d7e31dc245fb95f0588
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2640
Number of clauses305320
Sum of the clauses size613120
Maximum clause length33
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2305240
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 580

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...487526/instance-3380702-1305487526.cnf
1.69/1.70	c Solving...
40000.07/5011.22	
40000.07/5011.22	*** INTERRUPTED! ***
40000.07/5011.22	*** Shutting down threads...
40001.14/5012.20	s UNKNOWN
40001.14/5012.20	
40001.14/5012.20	c INTERRUPTED   : 1
40001.14/5012.20	c Models        : 0+    
40001.14/5012.20	c Time          : 5011.223s (Solving: 5009.52s 1st Model: 0.00s Unsat: 0.00s)
40001.14/5012.20	c CPU Time      : 40000.170s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3380702-1305487526/watcher-3380702-1305487526 -o /tmp/evaluation-result-3380702-1305487526/solver-3380702-1305487526 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380702-1305487526.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.03 7.95 7.94 2/438 10937
/proc/meminfo: memFree=21602692/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=22352 CPUtime=0 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 942 0 0 0 0 0 0 0 23 0 1 0 660353020 22888448 816 33554432000 4194304 4964800 140733597440640 18446744073709551615 253640520389 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/10937/statm: 5588 818 386 189 0 462 0

[startup+0.111468 s]
/proc/loadavg: 8.03 7.95 7.94 2/438 10937
/proc/meminfo: memFree=21602692/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=39840 CPUtime=0.1 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 6438 0 0 0 10 0 0 0 24 0 1 0 660353020 40796160 5174 33554432000 4194304 4964800 140733597440640 18446744073709551615 4607575 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/10937/statm: 9960 5174 399 189 0 4834 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 39840

[startup+0.200432 s]
/proc/loadavg: 8.03 7.95 7.94 2/438 10937
/proc/meminfo: memFree=21602692/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=39840 CPUtime=0.19 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 6438 0 0 0 19 0 0 0 24 0 1 0 660353020 40796160 5174 33554432000 4194304 4964800 140733597440640 18446744073709551615 4607585 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/10937/statm: 9960 5174 399 189 0 4834 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 39840

[startup+0.300415 s]
/proc/loadavg: 8.03 7.95 7.94 2/438 10937
/proc/meminfo: memFree=21602692/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=39840 CPUtime=0.29 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 6438 0 0 0 29 0 0 0 25 0 1 0 660353020 40796160 5174 33554432000 4194304 4964800 140733597440640 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/10937/statm: 9960 5174 399 189 0 4834 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 39840

[startup+0.70034 s]
/proc/loadavg: 8.03 7.95 7.94 2/438 10937
/proc/meminfo: memFree=21602692/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=39840 CPUtime=0.69 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 6438 0 0 0 69 0 0 0 25 0 1 0 660353020 40796160 5174 33554432000 4194304 4964800 140733597440640 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/10937/statm: 9960 5174 399 189 0 4834 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 39840

[startup+1.50021 s]
/proc/loadavg: 7.47 7.83 7.90 2/439 10938
/proc/meminfo: memFree=21581488/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=43140 CPUtime=1.49 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 7257 0 0 0 148 1 0 0 25 0 1 0 660353020 44175360 5993 33554432000 4194304 4964800 140733597440640 18446744073709551615 4600360 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/10937/statm: 10785 5993 400 189 0 5659 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 43140

[startup+3.10087 s]
/proc/loadavg: 7.47 7.83 7.90 9/446 10945
/proc/meminfo: memFree=21558616/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=466872 CPUtime=11.05 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 20234 0 0 0 1098 7 0 0 17 0 8 0 660353020 478076928 18352 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670403 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/10937/statm: 116718 18352 464 189 0 111067 0
[pid=10937/tid=10939] ppid=10935 vsize=466872 CPUtime=1.14 cores=0-7
/proc/10937/task/10939/stat : 10939 (claspmt) R 10935 10937 10868 0 -1 4202560 1664 0 0 0 114 0 0 0 17 0 8 0 660353191 478076928 18352 33554432000 4194304 4964800 140733597440640 18446744073709551615 4627168 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=10937/tid=10940] ppid=10935 vsize=466872 CPUtime=1.15 cores=0-7
/proc/10937/task/10940/stat : 10940 (claspmt) R 10935 10937 10868 0 -1 4202560 1213 0 0 0 115 0 0 0 19 0 8 0 660353191 478076928 18352 33554432000 4194304 4964800 140733597440640 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10937/tid=10941] ppid=10935 vsize=466872 CPUtime=1.14 cores=0-7
/proc/10937/task/10941/stat : 10941 (claspmt) R 10935 10937 10868 0 -1 4202560 1346 0 0 0 114 0 0 0 17 0 8 0 660353191 478076928 18352 33554432000 4194304 4964800 140733597440640 18446744073709551615 4627248 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10937/tid=10942] ppid=10935 vsize=466872 CPUtime=1.14 cores=0-7
/proc/10937/task/10942/stat : 10942 (claspmt) R 10935 10937 10868 0 -1 4202560 1524 0 0 0 114 0 0 0 17 0 8 0 660353191 478076928 18352 33554432000 4194304 4964800 140733597440640 18446744073709551615 4660425 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10937/tid=10943] ppid=10935 vsize=466872 CPUtime=1.28 cores=0-7
/proc/10937/task/10943/stat : 10943 (claspmt) R 10935 10937 10868 0 -1 4202560 960 0 0 0 128 0 0 0 19 0 8 0 660353191 478076928 18352 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670371 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=10937/tid=10944] ppid=10935 vsize=466872 CPUtime=1.15 cores=0-7
/proc/10937/task/10944/stat : 10944 (claspmt) R 10935 10937 10868 0 -1 4202560 1826 0 0 0 114 1 0 0 17 0 8 0 660353191 478076928 18352 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10937/tid=10945] ppid=10935 vsize=466872 CPUtime=1.13 cores=0-7
/proc/10937/task/10945/stat : 10945 (claspmt) R 10935 10937 10868 0 -1 4202560 2778 0 0 0 113 0 0 0 17 0 8 0 660353191 478076928 18352 33554432000 4194304 4964800 140733597440640 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.05
Current children cumulated vsize (KiB) 466872

[startup+6.30128 s]
/proc/loadavg: 7.51 7.83 7.90 9/446 10945
/proc/meminfo: memFree=21484148/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=474020 CPUtime=36.6 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 36998 0 0 0 3643 17 0 0 25 0 8 0 660353020 485396480 35116 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/10937/statm: 118505 35116 464 189 0 112854 0
[pid=10937/tid=10939] ppid=10935 vsize=474020 CPUtime=4.34 cores=0-7
/proc/10937/task/10939/stat : 10939 (claspmt) R 10935 10937 10868 0 -1 4202560 3770 0 0 0 433 1 0 0 23 0 8 0 660353191 485396480 35116 33554432000 4194304 4964800 140733597440640 18446744073709551615 4342144 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=10937/tid=10940] ppid=10935 vsize=474020 CPUtime=4.35 cores=0-7
/proc/10937/task/10940/stat : 10940 (claspmt) R 10935 10937 10868 0 -1 4202560 3445 0 0 0 434 1 0 0 25 0 8 0 660353191 485396480 35117 33554432000 4194304 4964800 140733597440640 18446744073709551615 4660195 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10937/tid=10941] ppid=10935 vsize=474020 CPUtime=4.34 cores=0-7
/proc/10937/task/10941/stat : 10941 (claspmt) R 10935 10937 10868 0 -1 4202560 2706 0 0 0 433 1 0 0 23 0 8 0 660353191 485396480 35118 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10937/tid=10942] ppid=10935 vsize=474020 CPUtime=4.34 cores=0-7
/proc/10937/task/10942/stat : 10942 (claspmt) R 10935 10937 10868 0 -1 4202560 3735 0 0 0 432 2 0 0 23 0 8 0 660353191 485396480 35118 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10937/tid=10943] ppid=10935 vsize=474020 CPUtime=4.48 cores=0-7
/proc/10937/task/10943/stat : 10943 (claspmt) R 10935 10937 10868 0 -1 4202560 1779 0 0 0 448 0 0 0 25 0 8 0 660353191 485396480 35118 33554432000 4194304 4964800 140733597440640 18446744073709551615 4320939 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=10937/tid=10944] ppid=10935 vsize=474020 CPUtime=4.35 cores=0-7
/proc/10937/task/10944/stat : 10944 (claspmt) R 10935 10937 10868 0 -1 4202560 2956 0 0 0 433 2 0 0 23 0 8 0 660353191 485396480 35118 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670381 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10937/tid=10945] ppid=10935 vsize=474020 CPUtime=4.29 cores=0-7
/proc/10937/task/10945/stat : 10945 (claspmt) R 10935 10937 10868 0 -1 4202560 7954 0 0 0 426 3 0 0 23 0 8 0 660353191 485396480 35118 33554432000 4194304 4964800 140733597440640 18446744073709551615 4627252 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 36.6
Current children cumulated vsize (KiB) 474020

[startup+12.7011 s]
/proc/loadavg: 7.55 7.83 7.90 9/446 10945
/proc/meminfo: memFree=21367536/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=491676 CPUtime=87.69 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 61651 0 0 0 8739 30 0 0 25 0 8 0 660353020 503476224 59769 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670534 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/10937/statm: 122919 59769 466 189 0 117268 0
[pid=10937/tid=10939] ppid=10935 vsize=491676 CPUtime=10.74 cores=0-7
/proc/10937/task/10939/stat : 10939 (claspmt) R 10935 10937 10868 0 -1 4202560 6949 0 0 0 1070 4 0 0 25 0 8 0 660353191 503476224 59769 33554432000 4194304 4964800 140733597440640 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=10937/tid=10940] ppid=10935 vsize=491676 CPUtime=10.74 cores=0-7
/proc/10937/task/10940/stat : 10940 (claspmt) R 10935 10937 10868 0 -1 4202560 6372 0 0 0 1072 2 0 0 25 0 8 0 660353191 503476224 59769 33554432000 4194304 4964800 140733597440640 18446744073709551615 4669008 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10937/tid=10941] ppid=10935 vsize=491676 CPUtime=10.74 cores=0-7
/proc/10937/task/10941/stat : 10941 (claspmt) R 10935 10937 10868 0 -1 4202560 5273 0 0 0 1072 2 0 0 25 0 8 0 660353191 503476224 59769 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670538 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10937/tid=10942] ppid=10935 vsize=491676 CPUtime=10.74 cores=0-7
/proc/10937/task/10942/stat : 10942 (claspmt) R 10935 10937 10868 0 -1 4202560 6066 0 0 0 1070 4 0 0 25 0 8 0 660353191 503476224 59769 33554432000 4194304 4964800 140733597440640 18446744073709551615 4653186 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10937/tid=10943] ppid=10935 vsize=491676 CPUtime=10.88 cores=0-7
/proc/10937/task/10943/stat : 10943 (claspmt) R 10935 10937 10868 0 -1 4202560 4794 0 0 0 1086 2 0 0 25 0 8 0 660353191 503476224 59769 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=10937/tid=10944] ppid=10935 vsize=491676 CPUtime=10.75 cores=0-7
/proc/10937/task/10944/stat : 10944 (claspmt) R 10935 10937 10868 0 -1 4202560 4989 0 0 0 1072 3 0 0 25 0 8 0 660353191 503476224 59769 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670332 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10937/tid=10945] ppid=10935 vsize=491676 CPUtime=10.58 cores=0-7
/proc/10937/task/10945/stat : 10945 (claspmt) R 10935 10937 10868 0 -1 4202560 14536 0 0 0 1053 5 0 0 25 0 8 0 660353191 503476224 59769 33554432000 4194304 4964800 140733597440640 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 87.69
Current children cumulated vsize (KiB) 491676

[startup+25.5008 s]
/proc/loadavg: 7.62 7.84 7.90 9/446 10945
/proc/meminfo: memFree=21253368/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=508972 CPUtime=189.88 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 89886 0 0 0 18940 48 0 0 25 0 8 0 660353020 521187328 88004 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/10937/statm: 127243 88004 466 189 0 121592 0
[pid=10937/tid=10939] ppid=10935 vsize=508972 CPUtime=23.54 cores=0-7
/proc/10937/task/10939/stat : 10939 (claspmt) R 10935 10937 10868 0 -1 4202560 10078 0 0 0 2347 7 0 0 25 0 8 0 660353191 521187328 88004 33554432000 4194304 4964800 140733597440640 18446744073709551615 4651386 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=10937/tid=10940] ppid=10935 vsize=508972 CPUtime=23.54 cores=0-7
/proc/10937/task/10940/stat : 10940 (claspmt) R 10935 10937 10868 0 -1 4202560 9018 0 0 0 2351 3 0 0 25 0 8 0 660353191 521187328 88004 33554432000 4194304 4964800 140733597440640 18446744073709551615 4378499 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10937/tid=10941] ppid=10935 vsize=508972 CPUtime=23.54 cores=0-7
/proc/10937/task/10941/stat : 10941 (claspmt) R 10935 10937 10868 0 -1 4202560 7372 0 0 0 2351 3 0 0 25 0 8 0 660353191 521187328 88004 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10937/tid=10942] ppid=10935 vsize=508972 CPUtime=23.54 cores=0-7
/proc/10937/task/10942/stat : 10942 (claspmt) R 10935 10937 10868 0 -1 4202560 9783 0 0 0 2348 6 0 0 25 0 8 0 660353191 521187328 88004 33554432000 4194304 4964800 140733597440640 18446744073709551615 4627401 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10937/tid=10943] ppid=10935 vsize=508972 CPUtime=23.68 cores=0-7
/proc/10937/task/10943/stat : 10943 (claspmt) R 10935 10937 10868 0 -1 4202560 7032 0 0 0 2364 4 0 0 25 0 8 0 660353191 521187328 88004 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670304 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=10937/tid=10944] ppid=10935 vsize=508972 CPUtime=23.55 cores=0-7
/proc/10937/task/10944/stat : 10944 (claspmt) R 10935 10937 10868 0 -1 4202560 7249 0 0 0 2350 5 0 0 25 0 8 0 660353191 521187328 88004 33554432000 4194304 4964800 140733597440640 18446744073709551615 253640556173 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10937/tid=10945] ppid=10935 vsize=508972 CPUtime=23.19 cores=0-7
/proc/10937/task/10945/stat : 10945 (claspmt) R 10935 10937 10868 0 -1 4202560 23347 0 0 0 2308 11 0 0 25 0 8 0 660353191 521187328 88004 33554432000 4194304 4964800 140733597440640 18446744073709551615 4631259 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 189.88
Current children cumulated vsize (KiB) 508972

[startup+51.1011 s]
/proc/loadavg: 7.77 7.85 7.91 9/446 10946
/proc/meminfo: memFree=21212748/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=511348 CPUtime=394.28 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 100576 0 0 0 39371 57 0 0 25 0 8 0 660353020 523620352 98694 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670335 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/10937/statm: 127837 98694 466 189 0 122186 0
[pid=10937/tid=10939] ppid=10935 vsize=511348 CPUtime=49.13 cores=0-7
/proc/10937/task/10939/stat : 10939 (claspmt) R 10935 10937 10868 0 -1 4202560 12048 0 0 0 4904 9 0 0 25 0 8 0 660353191 523620352 98694 33554432000 4194304 4964800 140733597440640 18446744073709551615 4627341 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=10937/tid=10940] ppid=10935 vsize=511348 CPUtime=49.14 cores=0-7
/proc/10937/task/10940/stat : 10940 (claspmt) R 10935 10937 10868 0 -1 4202560 10617 0 0 0 4910 4 0 0 25 0 8 0 660353191 523620352 98694 33554432000 4194304 4964800 140733597440640 18446744073709551615 4321010 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10937/tid=10941] ppid=10935 vsize=511348 CPUtime=49.13 cores=0-7
/proc/10937/task/10941/stat : 10941 (claspmt) R 10935 10937 10868 0 -1 4202560 8311 0 0 0 4909 4 0 0 25 0 8 0 660353191 523620352 98694 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10937/tid=10942] ppid=10935 vsize=511348 CPUtime=49.14 cores=0-7
/proc/10937/task/10942/stat : 10942 (claspmt) R 10935 10937 10868 0 -1 4202560 10113 0 0 0 4908 6 0 0 25 0 8 0 660353191 523620352 98694 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10937/tid=10943] ppid=10935 vsize=511348 CPUtime=49.27 cores=0-7
/proc/10937/task/10943/stat : 10943 (claspmt) R 10935 10937 10868 0 -1 4202560 7597 0 0 0 4923 4 0 0 25 0 8 0 660353191 523620352 98694 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=10937/tid=10944] ppid=10935 vsize=511348 CPUtime=49.14 cores=0-7
/proc/10937/task/10944/stat : 10944 (claspmt) R 10935 10937 10868 0 -1 4202560 7628 0 0 0 4909 5 0 0 25 0 8 0 660353191 523620352 98694 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10937/tid=10945] ppid=10935 vsize=511348 CPUtime=48.42 cores=0-7
/proc/10937/task/10945/stat : 10945 (claspmt) R 10935 10937 10868 0 -1 4202560 27743 0 0 0 4827 15 0 0 25 0 8 0 660353191 523620352 98694 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 394.28
Current children cumulated vsize (KiB) 511348

[startup+102.301 s]
/proc/loadavg: 7.90 7.87 7.91 9/446 10947
/proc/meminfo: memFree=21111804/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=843532 CPUtime=803.13 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 125044 0 0 0 80231 82 0 0 25 0 8 0 660353020 863776768 123112 33554432000 4194304 4964800 140733597440640 18446744073709551615 4627204 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/10937/statm: 210883 123112 466 189 0 205232 0
[pid=10937/tid=10939] ppid=10935 vsize=843532 CPUtime=100.33 cores=0-7
/proc/10937/task/10939/stat : 10939 (claspmt) R 10935 10937 10868 0 -1 4202560 13852 0 0 0 10021 12 0 0 25 0 8 0 660353191 863776768 123120 33554432000 4194304 4964800 140733597440640 18446744073709551615 4321010 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=10937/tid=10940] ppid=10935 vsize=843532 CPUtime=100.34 cores=0-7
/proc/10937/task/10940/stat : 10940 (claspmt) R 10935 10937 10868 0 -1 4202560 13752 0 0 0 10028 6 0 0 25 0 8 0 660353191 863776768 123120 33554432000 4194304 4964800 140733597440640 18446744073709551615 4342144 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10937/tid=10941] ppid=10935 vsize=843532 CPUtime=100.33 cores=0-7
/proc/10937/task/10941/stat : 10941 (claspmt) R 10935 10937 10868 0 -1 4202560 9056 0 0 0 10029 4 0 0 25 0 8 0 660353191 863776768 123120 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0

################
# More data... #
################

[pid=10937] ppid=10935 vsize=2892776 CPUtime=37693.1 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 671262 0 0 0 3768563 752 0 0 25 0 8 0 660353020 2962202624 662002 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/10937/statm: 723194 662002 466 189 0 717543 0
[pid=10937/tid=10939] ppid=10935 vsize=2892776 CPUtime=4662.02 cores=0-7
/proc/10937/task/10939/stat : 10939 (claspmt) R 10935 10937 10868 0 -1 4202560 53727 0 0 0 466129 73 0 0 25 0 8 0 660353191 2962202624 662002 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=10937/tid=10940] ppid=10935 vsize=2892776 CPUtime=4720.36 cores=0-7
/proc/10937/task/10940/stat : 10940 (claspmt) R 10935 10937 10868 0 -1 4202560 34987 0 0 0 472005 31 0 0 25 0 8 0 660353191 2962202624 662002 33554432000 4194304 4964800 140733597440640 18446744073709551615 4644900 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10937/tid=10941] ppid=10935 vsize=2892776 CPUtime=4719.21 cores=0-7
/proc/10937/task/10941/stat : 10941 (claspmt) R 10935 10937 10868 0 -1 4202560 25217 0 0 0 471894 27 0 0 25 0 8 0 660353191 2962202624 662002 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10937/tid=10942] ppid=10935 vsize=2892776 CPUtime=4715.21 cores=0-7
/proc/10937/task/10942/stat : 10942 (claspmt) R 10935 10937 10868 0 -1 4202560 34553 0 0 0 471479 42 0 0 25 0 8 0 660353191 2962202624 662002 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=10937/tid=10943] ppid=10935 vsize=2892776 CPUtime=4719.47 cores=0-7
/proc/10937/task/10943/stat : 10943 (claspmt) R 10935 10937 10868 0 -1 4202560 26062 0 0 0 471921 26 0 0 25 0 8 0 660353191 2962202624 662002 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10937/tid=10944] ppid=10935 vsize=2892776 CPUtime=4719.3 cores=0-7
/proc/10937/task/10944/stat : 10944 (claspmt) R 10935 10937 10868 0 -1 4202560 25138 0 0 0 471896 34 0 0 25 0 8 0 660353191 2962202624 662002 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10937/tid=10945] ppid=10935 vsize=2892776 CPUtime=4715.51 cores=0-7
/proc/10937/task/10945/stat : 10945 (claspmt) R 10935 10937 10868 0 -1 4202560 395784 0 0 0 471122 429 0 0 25 0 8 0 660353191 2962202624 662002 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 37693.1
Current children cumulated vsize (KiB) 2892776

[startup+4782.3 s]
/proc/loadavg: 8.14 8.08 8.05 9/446 11084
/proc/meminfo: memFree=18950608/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=2892776 CPUtime=38172.2 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 671272 0 0 0 3816464 754 0 0 25 0 8 0 660353020 2962202624 662012 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670499 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/10937/statm: 723194 662012 466 189 0 717543 0
[pid=10937/tid=10939] ppid=10935 vsize=2892776 CPUtime=4722 cores=0-7
/proc/10937/task/10939/stat : 10939 (claspmt) R 10935 10937 10868 0 -1 4202560 53729 0 0 0 472127 73 0 0 25 0 8 0 660353191 2962202624 662012 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670538 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=10937/tid=10940] ppid=10935 vsize=2892776 CPUtime=4780.33 cores=0-7
/proc/10937/task/10940/stat : 10940 (claspmt) R 10935 10937 10868 0 -1 4202560 34987 0 0 0 478001 32 0 0 25 0 8 0 660353191 2962202624 662012 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10937/tid=10941] ppid=10935 vsize=2892776 CPUtime=4779.19 cores=0-7
/proc/10937/task/10941/stat : 10941 (claspmt) R 10935 10937 10868 0 -1 4202560 25217 0 0 0 477892 27 0 0 25 0 8 0 660353191 2962202624 662012 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10937/tid=10942] ppid=10935 vsize=2892776 CPUtime=4775.2 cores=0-7
/proc/10937/task/10942/stat : 10942 (claspmt) R 10935 10937 10868 0 -1 4202560 34553 0 0 0 477478 42 0 0 25 0 8 0 660353191 2962202624 662012 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=10937/tid=10943] ppid=10935 vsize=2892776 CPUtime=4779.46 cores=0-7
/proc/10937/task/10943/stat : 10943 (claspmt) R 10935 10937 10868 0 -1 4202560 26062 0 0 0 477920 26 0 0 25 0 8 0 660353191 2962202624 662012 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10937/tid=10944] ppid=10935 vsize=2892776 CPUtime=4779.27 cores=0-7
/proc/10937/task/10944/stat : 10944 (claspmt) R 10935 10937 10868 0 -1 4202560 25146 0 0 0 477893 34 0 0 25 0 8 0 660353191 2962202624 662012 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10937/tid=10945] ppid=10935 vsize=2892776 CPUtime=4774.64 cores=0-7
/proc/10937/task/10945/stat : 10945 (claspmt) R 10935 10937 10868 0 -1 4202560 395784 0 0 0 477034 430 0 0 25 0 8 0 660353191 2962202624 662012 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38172.2
Current children cumulated vsize (KiB) 2892776

[startup+4842.32 s]
/proc/loadavg: 8.12 8.08 8.05 9/446 11086
/proc/meminfo: memFree=18950232/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=2892776 CPUtime=38651.4 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 671274 0 0 0 3864386 754 0 0 25 0 8 0 660353020 2962202624 662013 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/10937/statm: 723194 662013 466 189 0 717543 0
[pid=10937/tid=10939] ppid=10935 vsize=2892776 CPUtime=4782.02 cores=0-7
/proc/10937/task/10939/stat : 10939 (claspmt) R 10935 10937 10868 0 -1 4202560 53731 0 0 0 478129 73 0 0 25 0 8 0 660353191 2962202624 662013 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=10937/tid=10940] ppid=10935 vsize=2892776 CPUtime=4840.34 cores=0-7
/proc/10937/task/10940/stat : 10940 (claspmt) R 10935 10937 10868 0 -1 4202560 34987 0 0 0 484002 32 0 0 25 0 8 0 660353191 2962202624 662013 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10937/tid=10941] ppid=10935 vsize=2892776 CPUtime=4839.18 cores=0-7
/proc/10937/task/10941/stat : 10941 (claspmt) R 10935 10937 10868 0 -1 4202560 25217 0 0 0 483891 27 0 0 25 0 8 0 660353191 2962202624 662013 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670349 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10937/tid=10942] ppid=10935 vsize=2892776 CPUtime=4835.21 cores=0-7
/proc/10937/task/10942/stat : 10942 (claspmt) R 10935 10937 10868 0 -1 4202560 34553 0 0 0 483479 42 0 0 25 0 8 0 660353191 2962202624 662013 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=10937/tid=10943] ppid=10935 vsize=2892776 CPUtime=4839.47 cores=0-7
/proc/10937/task/10943/stat : 10943 (claspmt) R 10935 10937 10868 0 -1 4202560 26062 0 0 0 483921 26 0 0 25 0 8 0 660353191 2962202624 662013 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10937/tid=10944] ppid=10935 vsize=2892776 CPUtime=4839.29 cores=0-7
/proc/10937/task/10944/stat : 10944 (claspmt) R 10935 10937 10868 0 -1 4202560 25146 0 0 0 483895 34 0 0 25 0 8 0 660353191 2962202624 662013 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10937/tid=10945] ppid=10935 vsize=2892776 CPUtime=4833.77 cores=0-7
/proc/10937/task/10945/stat : 10945 (claspmt) R 10935 10937 10868 0 -1 4202560 395784 0 0 0 482947 430 0 0 25 0 8 0 660353191 2962202624 662013 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38651.4
Current children cumulated vsize (KiB) 2892776

[startup+4902.3 s]
/proc/loadavg: 8.11 8.07 8.05 9/446 11087
/proc/meminfo: memFree=18950092/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=2892776 CPUtime=39130.4 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 671295 0 0 0 3912286 754 0 0 25 0 8 0 660353020 2962202624 662034 33554432000 4194304 4964800 140733597440640 18446744073709551615 4644912 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/10937/statm: 723194 662034 466 189 0 717543 0
[pid=10937/tid=10939] ppid=10935 vsize=2892776 CPUtime=4842 cores=0-7
/proc/10937/task/10939/stat : 10939 (claspmt) R 10935 10937 10868 0 -1 4202560 53732 0 0 0 484127 73 0 0 25 0 8 0 660353191 2962202624 662034 33554432000 4194304 4964800 140733597440640 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=10937/tid=10940] ppid=10935 vsize=2892776 CPUtime=4900.33 cores=0-7
/proc/10937/task/10940/stat : 10940 (claspmt) R 10935 10937 10868 0 -1 4202560 34988 0 0 0 490001 32 0 0 25 0 8 0 660353191 2962202624 662034 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10937/tid=10941] ppid=10935 vsize=2892776 CPUtime=4899.15 cores=0-7
/proc/10937/task/10941/stat : 10941 (claspmt) R 10935 10937 10868 0 -1 4202560 25217 0 0 0 489888 27 0 0 25 0 8 0 660353191 2962202624 662034 33554432000 4194304 4964800 140733597440640 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10937/tid=10942] ppid=10935 vsize=2892776 CPUtime=4895.2 cores=0-7
/proc/10937/task/10942/stat : 10942 (claspmt) R 10935 10937 10868 0 -1 4202560 34571 0 0 0 489478 42 0 0 25 0 8 0 660353191 2962202624 662034 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=10937/tid=10943] ppid=10935 vsize=2892776 CPUtime=4899.46 cores=0-7
/proc/10937/task/10943/stat : 10943 (claspmt) R 10935 10937 10868 0 -1 4202560 26062 0 0 0 489920 26 0 0 25 0 8 0 660353191 2962202624 662034 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10937/tid=10944] ppid=10935 vsize=2892776 CPUtime=4899.28 cores=0-7
/proc/10937/task/10944/stat : 10944 (claspmt) R 10935 10937 10868 0 -1 4202560 25146 0 0 0 489894 34 0 0 25 0 8 0 660353191 2962202624 662034 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10937/tid=10945] ppid=10935 vsize=2892776 CPUtime=4892.89 cores=0-7
/proc/10937/task/10945/stat : 10945 (claspmt) R 10935 10937 10868 0 -1 4202560 395785 0 0 0 488859 430 0 0 25 0 8 0 660353191 2962202624 662034 33554432000 4194304 4964800 140733597440640 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39130.4
Current children cumulated vsize (KiB) 2892776

[startup+4962.3 s]
/proc/loadavg: 8.10 8.07 8.05 9/446 11089
/proc/meminfo: memFree=18947476/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=2892776 CPUtime=39609.5 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 672240 0 0 0 3960197 755 0 0 25 0 8 0 660353020 2962202624 662979 33554432000 4194304 4964800 140733597440640 18446744073709551615 4321016 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/10937/statm: 723194 662979 466 189 0 717543 0
[pid=10937/tid=10939] ppid=10935 vsize=2892776 CPUtime=4902 cores=0-7
/proc/10937/task/10939/stat : 10939 (claspmt) R 10935 10937 10868 0 -1 4202560 53732 0 0 0 490127 73 0 0 25 0 8 0 660353191 2962202624 662979 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=10937/tid=10940] ppid=10935 vsize=2892776 CPUtime=4960.33 cores=0-7
/proc/10937/task/10940/stat : 10940 (claspmt) R 10935 10937 10868 0 -1 4202560 35202 0 0 0 496001 32 0 0 25 0 8 0 660353191 2962202624 662979 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10937/tid=10941] ppid=10935 vsize=2892776 CPUtime=4959.14 cores=0-7
/proc/10937/task/10941/stat : 10941 (claspmt) R 10935 10937 10868 0 -1 4202560 25218 0 0 0 495887 27 0 0 25 0 8 0 660353191 2962202624 662979 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10937/tid=10942] ppid=10935 vsize=2892776 CPUtime=4955.2 cores=0-7
/proc/10937/task/10942/stat : 10942 (claspmt) R 10935 10937 10868 0 -1 4202560 34622 0 0 0 495478 42 0 0 25 0 8 0 660353191 2962202624 662979 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=10937/tid=10943] ppid=10935 vsize=2892776 CPUtime=4959.46 cores=0-7
/proc/10937/task/10943/stat : 10943 (claspmt) R 10935 10937 10868 0 -1 4202560 26062 0 0 0 495920 26 0 0 25 0 8 0 660353191 2962202624 662979 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10937/tid=10944] ppid=10935 vsize=2892776 CPUtime=4959.28 cores=0-7
/proc/10937/task/10944/stat : 10944 (claspmt) R 10935 10937 10868 0 -1 4202560 25146 0 0 0 495894 34 0 0 25 0 8 0 660353191 2962202624 662979 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10937/tid=10945] ppid=10935 vsize=2892776 CPUtime=4952 cores=0-7
/proc/10937/task/10945/stat : 10945 (claspmt) R 10935 10937 10868 0 -1 4202560 396464 0 0 0 494770 430 0 0 25 0 8 0 660353191 2962202624 662979 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39609.5
Current children cumulated vsize (KiB) 2892776



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5011.2 s]
/proc/loadavg: 8.11 8.08 8.05 9/446 11090
/proc/meminfo: memFree=18946720/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=2892776 CPUtime=40000.1 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 672406 0 0 0 3999252 755 0 0 25 0 8 0 660353020 2962202624 663071 33554432000 4194304 4964800 140733597440640 18446744073709551615 4320944 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/10937/statm: 723194 663071 466 189 0 717543 0
[pid=10937/tid=10939] ppid=10935 vsize=2892776 CPUtime=4950.92 cores=0-7
/proc/10937/task/10939/stat : 10939 (claspmt) R 10935 10937 10868 0 -1 4202560 53804 0 0 0 495019 73 0 0 25 0 8 0 660353191 2962202624 663071 33554432000 4194304 4964800 140733597440640 18446744073709551615 4320916 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=10937/tid=10940] ppid=10935 vsize=2892776 CPUtime=5009.24 cores=0-7
/proc/10937/task/10940/stat : 10940 (claspmt) R 10935 10937 10868 0 -1 4202560 35268 0 0 0 500892 32 0 0 25 0 8 0 660353191 2962202624 663071 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10937/tid=10941] ppid=10935 vsize=2892776 CPUtime=5008.05 cores=0-7
/proc/10937/task/10941/stat : 10941 (claspmt) R 10935 10937 10868 0 -1 4202560 25218 0 0 0 500778 27 0 0 25 0 8 0 660353191 2962202624 663071 33554432000 4194304 4964800 140733597440640 18446744073709551615 4342174 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10937/tid=10942] ppid=10935 vsize=2892776 CPUtime=5004.12 cores=0-7
/proc/10937/task/10942/stat : 10942 (claspmt) R 10935 10937 10868 0 -1 4202560 34650 0 0 0 500369 43 0 0 25 0 8 0 660353191 2962202624 663071 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670357 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=10937/tid=10943] ppid=10935 vsize=2892776 CPUtime=5008.38 cores=0-7
/proc/10937/task/10943/stat : 10943 (claspmt) R 10935 10937 10868 0 -1 4202560 26062 0 0 0 500812 26 0 0 25 0 8 0 660353191 2962202624 663071 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10937/tid=10944] ppid=10935 vsize=2892776 CPUtime=5008.2 cores=0-7
/proc/10937/task/10944/stat : 10944 (claspmt) R 10935 10937 10868 0 -1 4202560 25146 0 0 0 500786 34 0 0 25 0 8 0 660353191 2962202624 663071 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10937/tid=10945] ppid=10935 vsize=2892776 CPUtime=5000.2 cores=0-7
/proc/10937/task/10945/stat : 10945 (claspmt) R 10935 10937 10868 0 -1 4202560 396464 0 0 0 499590 430 0 0 25 0 8 0 660353191 2962202624 663071 33554432000 4194304 4964800 140733597440640 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 2892776

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5011.9 s]
/proc/loadavg: 8.11 8.08 8.05 2/440 11091
/proc/meminfo: memFree=18956440/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=2876036 CPUtime=40000.8 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 672416 0 0 0 3999329 755 0 0 25 0 1 0 660353020 2945060864 657803 33554432000 4194304 4964800 140733597440640 18446744073709551615 253640516145 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/10937/statm: 719009 657803 476 189 0 713358 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 2876036

[startup+5012.1 s]
/proc/loadavg: 8.11 8.08 8.05 2/440 11091
/proc/meminfo: memFree=18956440/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=2867832 CPUtime=40001.1 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 672416 0 0 0 3999349 756 0 0 25 0 1 0 660353020 2936659968 644279 33554432000 4194304 4964800 140733597440640 18446744073709551615 253640514913 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/10937/statm: 716958 644279 476 189 0 711307 0
Current children cumulated CPU time (s) 40001
Current children cumulated vsize (KiB) 2867832

[startup+5012.2 s]
/proc/loadavg: 8.11 8.08 8.05 2/440 11091
/proc/meminfo: memFree=18956440/32951124 swapFree=56449232/67111528
[pid=10937] ppid=10935 vsize=637552 CPUtime=40001.1 cores=0-7
/proc/10937/stat : 10937 (claspmt) R 10935 10937 10868 0 -1 4202496 672416 0 0 0 3999349 765 0 0 25 0 1 0 660353020 652853248 53059 33554432000 4194304 4964800 140733597440640 18446744073709551615 253640903015 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/10937/statm: 159388 53021 476 189 0 153737 0
Current children cumulated CPU time (s) 40001.1
Current children cumulated vsize (KiB) 637552

Child status: 0
Real time (s): 5012.21
CPU time (s): 40001.2
CPU user time (s): 39993.5
CPU system time (s): 7.66683
CPU usage (%): 798.074
Max. virtual memory (cumulated for all children) (KiB): 2892776

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39993.5
system time used= 7.66683
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 672417
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 69023
involuntary context switches= 140292

runsolver used 19.2051 second user time and 53.7788 second system time

The end

Launcher Data 

Begin job on node106 at 2011-05-15 21:25:26
IDJOB=3380702
IDBENCH=71423
IDSOLVER=1865
FILE ID=node106/3380702-1305487526
RUNJOBID= node106-1305487216-10882
PBS_JOBID= 13325387
Free space on /tmp= 73384 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/random/forced/rbsat-v2640c305320gyes3.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3380702-1305487526/watcher-3380702-1305487526 -o /tmp/evaluation-result-3380702-1305487526/solver-3380702-1305487526 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380702-1305487526.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= d6877629cc818d7e31dc245fb95f0588
RANDOM SEED=1313035753

node106.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.20
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21602972 kB
Buffers:        306356 kB
Cached:        6240340 kB
SwapCached:      27536 kB
Active:        2746996 kB
Inactive:      8342584 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21602972 kB
SwapTotal:    67111528 kB
SwapFree:     56449232 kB
Dirty:            4712 kB
Writeback:           0 kB
AnonPages:     4524080 kB
Mapped:          20280 kB
Slab:           129328 kB
PageTables:      60052 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 123521904 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73380 MiB
End job on node106 at 2011-05-15 22:49:00