Trace number 3321498

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40001.7 5008.18

General information on the benchmark

NameCRAFTED/rbsat/random/forced/
rbsat-v2640c305320gyes10.cnf
MD5SUMe4e4b653016e98f210aa8da2214e783a
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2640
Number of clauses305320
Sum of the clauses size613120
Maximum clause length33
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2305240
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 580

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...344422/instance-3321498-1304344422.cnf
1.69/1.70	c Solving...
40000.69/5007.22	
40000.69/5007.22	*** INTERRUPTED! ***
40000.69/5007.22	*** Shutting down threads...
40001.61/5008.17	s UNKNOWN
40001.61/5008.17	
40001.61/5008.17	c INTERRUPTED   : 1
40001.61/5008.17	c Models        : 0+    
40001.61/5008.17	c Time          : 5007.223s (Solving: 5005.52s 1st Model: 0.00s Unsat: 0.00s)
40001.61/5008.17	c CPU Time      : 40000.740s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321498-1304344422/watcher-3321498-1304344422 -o /tmp/evaluation-result-3321498-1304344422/solver-3321498-1304344422 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321498-1304344422.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.92 2.17 3.64 2/253 17621
/proc/meminfo: memFree=27268680/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=22352 CPUtime=0 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 926 0 0 0 0 0 0 0 23 0 1 0 624672945 22888448 800 33554432000 4194304 4964704 140733548474560 18446744073709551615 256415391488 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/17621/statm: 5588 801 386 189 0 462 0

[startup+0.096409 s]
/proc/loadavg: 1.92 2.17 3.64 2/253 17621
/proc/meminfo: memFree=27268680/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=39084 CPUtime=0.08 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 6178 0 0 0 8 0 0 0 23 0 1 0 624672945 40022016 4914 33554432000 4194304 4964704 140733548474560 18446744073709551615 4618467 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/17621/statm: 9771 4914 397 189 0 4645 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 39084

[startup+0.10039 s]
/proc/loadavg: 1.92 2.17 3.64 2/253 17621
/proc/meminfo: memFree=27268680/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=39348 CPUtime=0.09 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 6311 0 0 0 9 0 0 0 23 0 1 0 624672945 40292352 5047 33554432000 4194304 4964704 140733548474560 18446744073709551615 256415052485 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/17621/statm: 9870 5051 397 189 0 4744 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 39348

[startup+0.300351 s]
/proc/loadavg: 1.92 2.17 3.64 2/253 17621
/proc/meminfo: memFree=27268680/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=39876 CPUtime=0.29 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 6442 0 0 0 29 0 0 0 25 0 1 0 624672945 40833024 5178 33554432000 4194304 4964704 140733548474560 18446744073709551615 4607486 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/17621/statm: 9969 5178 399 189 0 4843 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 39876

[startup+0.700288 s]
/proc/loadavg: 1.92 2.17 3.64 2/253 17621
/proc/meminfo: memFree=27268680/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=39876 CPUtime=0.69 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 6442 0 0 0 69 0 0 0 25 0 1 0 624672945 40833024 5178 33554432000 4194304 4964704 140733548474560 18446744073709551615 4607585 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/17621/statm: 9969 5178 399 189 0 4843 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 39876

[startup+1.50115 s]
/proc/loadavg: 1.92 2.17 3.64 2/254 17622
/proc/meminfo: memFree=27248428/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=43176 CPUtime=1.49 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 7296 0 0 0 149 0 0 0 25 0 1 0 624672945 44212224 6032 33554432000 4194304 4964704 140733548474560 18446744073709551615 4615447 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/17621/statm: 10794 6032 400 189 0 5668 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 43176

[startup+3.10075 s]
/proc/loadavg: 1.92 2.17 3.64 9/261 17629
/proc/meminfo: memFree=27231632/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=401512 CPUtime=10.95 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 19948 0 0 0 1084 11 0 0 18 0 8 0 624672945 411148288 18065 33554432000 4194304 4964704 140733548474560 18446744073709551615 4320984 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/17621/statm: 100378 18065 464 189 0 94727 0
[pid=17621/tid=17623] ppid=17619 vsize=401512 CPUtime=1.15 cores=0-7
/proc/17621/task/17623/stat : 17623 (claspmt) R 17619 17621 17370 0 -1 4202560 1408 0 0 0 115 0 0 0 17 0 8 0 624673116 411148288 18065 33554432000 4194304 4964704 140733548474560 18446744073709551615 4660367 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=17621/tid=17624] ppid=17619 vsize=401512 CPUtime=1.14 cores=0-7
/proc/17621/task/17624/stat : 17624 (claspmt) R 17619 17621 17370 0 -1 4202560 1690 0 0 0 112 2 0 0 17 0 8 0 624673116 411148288 18065 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670466 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=17621/tid=17625] ppid=17619 vsize=401512 CPUtime=1.14 cores=0-7
/proc/17621/task/17625/stat : 17625 (claspmt) R 17619 17621 17370 0 -1 4202560 1203 0 0 0 114 0 0 0 19 0 8 0 624673116 411148288 18065 33554432000 4194304 4964704 140733548474560 18446744073709551615 4627615 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=17621/tid=17626] ppid=17619 vsize=401512 CPUtime=1.14 cores=0-7
/proc/17621/task/17626/stat : 17626 (claspmt) R 17619 17621 17370 0 -1 4202560 1768 0 0 0 113 1 0 0 17 0 8 0 624673116 411148288 18065 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670230 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=17621/tid=17627] ppid=17619 vsize=401512 CPUtime=1.37 cores=0-7
/proc/17621/task/17627/stat : 17627 (claspmt) R 17619 17621 17370 0 -1 4202560 936 0 0 0 137 0 0 0 19 0 8 0 624673116 411148288 18065 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=17621/tid=17628] ppid=17619 vsize=401512 CPUtime=1.14 cores=0-7
/proc/17621/task/17628/stat : 17628 (claspmt) R 17619 17621 17370 0 -1 4202560 1928 0 0 0 112 2 0 0 17 0 8 0 624673116 411148288 18065 33554432000 4194304 4964704 140733548474560 18446744073709551615 4669051 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=17621/tid=17629] ppid=17619 vsize=401512 CPUtime=0.96 cores=0-7
/proc/17621/task/17629/stat : 17629 (claspmt) R 17619 17621 17370 0 -1 4202560 2256 0 0 0 95 1 0 0 18 0 8 0 624673116 411148288 18065 33554432000 4194304 4964704 140733548474560 18446744073709551615 4660129 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 10.95
Current children cumulated vsize (KiB) 401512

[startup+6.30114 s]
/proc/loadavg: 2.41 2.27 3.66 9/261 17630
/proc/meminfo: memFree=27151652/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=409444 CPUtime=36.51 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 36249 0 0 0 3631 20 0 0 25 0 8 0 624672945 419270656 34366 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670248 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/17621/statm: 102361 34366 464 189 0 96710 0
[pid=17621/tid=17623] ppid=17619 vsize=409444 CPUtime=4.35 cores=0-7
/proc/17621/task/17623/stat : 17623 (claspmt) R 17619 17621 17370 0 -1 4202560 2921 0 0 0 434 1 0 0 23 0 8 0 624673116 419270656 34366 33554432000 4194304 4964704 140733548474560 18446744073709551615 4627263 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=17621/tid=17624] ppid=17619 vsize=409444 CPUtime=4.34 cores=0-7
/proc/17621/task/17624/stat : 17624 (claspmt) R 17619 17621 17370 0 -1 4202560 3798 0 0 0 431 3 0 0 23 0 8 0 624673116 419270656 34366 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670215 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=17621/tid=17625] ppid=17619 vsize=409444 CPUtime=4.35 cores=0-7
/proc/17621/task/17625/stat : 17625 (claspmt) R 17619 17621 17370 0 -1 4202560 2783 0 0 0 434 1 0 0 25 0 8 0 624673116 419270656 34366 33554432000 4194304 4964704 140733548474560 18446744073709551615 4659143 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=17621/tid=17626] ppid=17619 vsize=409444 CPUtime=4.33 cores=0-7
/proc/17621/task/17626/stat : 17626 (claspmt) R 17619 17621 17370 0 -1 4202560 3795 0 0 0 431 2 0 0 23 0 8 0 624673116 419270656 34366 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=17621/tid=17627] ppid=17619 vsize=409444 CPUtime=4.57 cores=0-7
/proc/17621/task/17627/stat : 17627 (claspmt) R 17619 17621 17370 0 -1 4202560 1737 0 0 0 457 0 0 0 25 0 8 0 624673116 419270656 34366 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=17621/tid=17628] ppid=17619 vsize=409444 CPUtime=4.34 cores=0-7
/proc/17621/task/17628/stat : 17628 (claspmt) R 17619 17621 17370 0 -1 4202560 3778 0 0 0 431 3 0 0 23 0 8 0 624673116 419270656 34366 33554432000 4194304 4964704 140733548474560 18446744073709551615 4653272 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=17621/tid=17629] ppid=17619 vsize=409444 CPUtime=4.13 cores=0-7
/proc/17621/task/17629/stat : 17629 (claspmt) R 17619 17621 17370 0 -1 4202560 7128 0 0 0 409 4 0 0 25 0 8 0 624673116 419270656 34366 33554432000 4194304 4964704 140733548474560 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 36.51
Current children cumulated vsize (KiB) 409444

[startup+12.7009 s]
/proc/loadavg: 2.85 2.36 3.69 9/261 17630
/proc/meminfo: memFree=27028912/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=493236 CPUtime=87.61 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 64701 0 0 0 8727 34 0 0 25 0 8 0 624672945 505073664 62818 33554432000 4194304 4964704 140733548474560 18446744073709551615 4627269 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/17621/statm: 123309 62818 466 189 0 117658 0
[pid=17621/tid=17623] ppid=17619 vsize=493236 CPUtime=10.74 cores=0-7
/proc/17621/task/17623/stat : 17623 (claspmt) R 17619 17621 17370 0 -1 4202560 6300 0 0 0 1072 2 0 0 25 0 8 0 624673116 505073664 62818 33554432000 4194304 4964704 140733548474560 18446744073709551615 4627443 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=17621/tid=17624] ppid=17619 vsize=493236 CPUtime=10.74 cores=0-7
/proc/17621/task/17624/stat : 17624 (claspmt) R 17619 17621 17370 0 -1 4202560 6625 0 0 0 1069 5 0 0 25 0 8 0 624673116 505073664 62818 33554432000 4194304 4964704 140733548474560 18446744073709551615 4627281 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=17621/tid=17625] ppid=17619 vsize=493236 CPUtime=10.74 cores=0-7
/proc/17621/task/17625/stat : 17625 (claspmt) R 17619 17621 17370 0 -1 4202560 5661 0 0 0 1072 2 0 0 25 0 8 0 624673116 505073664 62818 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670458 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=17621/tid=17626] ppid=17619 vsize=493236 CPUtime=10.73 cores=0-7
/proc/17621/task/17626/stat : 17626 (claspmt) R 17619 17621 17370 0 -1 4202560 7103 0 0 0 1069 4 0 0 25 0 8 0 624673116 505073664 62818 33554432000 4194304 4964704 140733548474560 18446744073709551615 4668464 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=17621/tid=17627] ppid=17619 vsize=493236 CPUtime=10.96 cores=0-7
/proc/17621/task/17627/stat : 17627 (claspmt) R 17619 17621 17370 0 -1 4202560 4719 0 0 0 1095 1 0 0 25 0 8 0 624673116 505073664 62818 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=17621/tid=17628] ppid=17619 vsize=493236 CPUtime=10.74 cores=0-7
/proc/17621/task/17628/stat : 17628 (claspmt) R 17619 17621 17370 0 -1 4202560 5694 0 0 0 1070 4 0 0 25 0 8 0 624673116 505073664 62818 33554432000 4194304 4964704 140733548474560 18446744073709551615 4627263 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=17621/tid=17629] ppid=17619 vsize=493236 CPUtime=10.46 cores=0-7
/proc/17621/task/17629/stat : 17629 (claspmt) R 17619 17621 17370 0 -1 4202560 15909 0 0 0 1038 8 0 0 25 0 8 0 624673116 505073664 62818 33554432000 4194304 4964704 140733548474560 18446744073709551615 4342199 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 87.61
Current children cumulated vsize (KiB) 493236

[startup+25.5005 s]
/proc/loadavg: 3.99 2.64 3.75 9/261 17630
/proc/meminfo: memFree=26914856/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=505820 CPUtime=189.89 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 91339 0 0 0 18937 52 0 0 25 0 8 0 624672945 517959680 89456 33554432000 4194304 4964704 140733548474560 18446744073709551615 4627982 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/17621/statm: 126455 89456 466 189 0 120804 0
[pid=17621/tid=17623] ppid=17619 vsize=505820 CPUtime=23.54 cores=0-7
/proc/17621/task/17623/stat : 17623 (claspmt) R 17619 17621 17370 0 -1 4202560 9427 0 0 0 2350 4 0 0 25 0 8 0 624673116 517959680 89456 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670193 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=17621/tid=17624] ppid=17619 vsize=505820 CPUtime=23.54 cores=0-7
/proc/17621/task/17624/stat : 17624 (claspmt) R 17619 17621 17370 0 -1 4202560 9683 0 0 0 2347 7 0 0 25 0 8 0 624673116 517959680 89456 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=17621/tid=17625] ppid=17619 vsize=505820 CPUtime=23.53 cores=0-7
/proc/17621/task/17625/stat : 17625 (claspmt) R 17619 17621 17370 0 -1 4202560 8031 0 0 0 2350 3 0 0 18 0 8 0 624673116 517959680 89456 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=17621/tid=17626] ppid=17619 vsize=505820 CPUtime=23.53 cores=0-7
/proc/17621/task/17626/stat : 17626 (claspmt) R 17619 17621 17370 0 -1 4202560 10307 0 0 0 2346 7 0 0 25 0 8 0 624673116 517959680 89456 33554432000 4194304 4964704 140733548474560 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=17621/tid=17627] ppid=17619 vsize=505820 CPUtime=23.76 cores=0-7
/proc/17621/task/17627/stat : 17627 (claspmt) R 17619 17621 17370 0 -1 4202560 6847 0 0 0 2373 3 0 0 25 0 8 0 624673116 517959680 89456 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=17621/tid=17628] ppid=17619 vsize=505820 CPUtime=23.53 cores=0-7
/proc/17621/task/17628/stat : 17628 (claspmt) R 17619 17621 17370 0 -1 4202560 7607 0 0 0 2348 5 0 0 25 0 8 0 624673116 517959680 89456 33554432000 4194304 4964704 140733548474560 18446744073709551615 4332387 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=17621/tid=17629] ppid=17619 vsize=505820 CPUtime=23.16 cores=0-7
/proc/17621/task/17629/stat : 17629 (claspmt) R 17619 17621 17370 0 -1 4202560 24301 0 0 0 2302 14 0 0 25 0 8 0 624673116 517959680 89456 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 189.89
Current children cumulated vsize (KiB) 505820

[startup+51.1007 s]
/proc/loadavg: 5.36 3.07 3.87 9/261 17631
/proc/meminfo: memFree=26871120/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=511528 CPUtime=394.45 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 102086 0 0 0 39384 61 0 0 25 0 8 0 624672945 523804672 100199 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/17621/statm: 127882 100199 466 189 0 122231 0
[pid=17621/tid=17623] ppid=17619 vsize=511528 CPUtime=48.96 cores=0-7
/proc/17621/task/17623/stat : 17623 (claspmt) R 17619 17621 17370 0 -1 4202560 10433 0 0 0 4891 5 0 0 25 0 8 0 624673116 523804672 100200 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=17621/tid=17624] ppid=17619 vsize=511528 CPUtime=49.14 cores=0-7
/proc/17621/task/17624/stat : 17624 (claspmt) R 17619 17621 17370 0 -1 4202560 12842 0 0 0 4904 10 0 0 25 0 8 0 624673116 523804672 100200 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=17621/tid=17625] ppid=17619 vsize=511528 CPUtime=49.13 cores=0-7
/proc/17621/task/17625/stat : 17625 (claspmt) R 17619 17621 17370 0 -1 4202560 8441 0 0 0 4909 4 0 0 25 0 8 0 624673116 523804672 100200 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670458 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=17621/tid=17626] ppid=17619 vsize=511528 CPUtime=49.13 cores=0-7
/proc/17621/task/17626/stat : 17626 (claspmt) R 17619 17621 17370 0 -1 4202560 11336 0 0 0 4905 8 0 0 25 0 8 0 624673116 523804672 100200 33554432000 4194304 4964704 140733548474560 18446744073709551615 256415052833 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=17621/tid=17627] ppid=17619 vsize=511528 CPUtime=49.36 cores=0-7
/proc/17621/task/17627/stat : 17627 (claspmt) R 17619 17621 17370 0 -1 4202560 7342 0 0 0 4933 3 0 0 25 0 8 0 624673116 523804672 100200 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=17621/tid=17628] ppid=17619 vsize=511528 CPUtime=49.12 cores=0-7
/proc/17621/task/17628/stat : 17628 (claspmt) R 17619 17621 17370 0 -1 4202560 8427 0 0 0 4906 6 0 0 25 0 8 0 624673116 523804672 100200 33554432000 4194304 4964704 140733548474560 18446744073709551615 4319906 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=17621/tid=17629] ppid=17619 vsize=511528 CPUtime=48.71 cores=0-7
/proc/17621/task/17629/stat : 17629 (claspmt) R 17619 17621 17370 0 -1 4202560 27806 0 0 0 4855 16 0 0 25 0 8 0 624673116 523804672 100200 33554432000 4194304 4964704 140733548474560 18446744073709551615 4660755 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 394.45
Current children cumulated vsize (KiB) 511528

[startup+102.31 s]
/proc/loadavg: 6.85 3.83 4.08 9/261 17632
/proc/meminfo: memFree=26747836/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=848496 CPUtime=803.68 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 133012 0 0 0 80275 93 0 0 25 0 8 0 624672945 868859904 131101 33554432000 4194304 4964704 140733548474560 18446744073709551615 4627365 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/17621/statm: 212124 131101 466 189 0 206473 0
[pid=17621/tid=17623] ppid=17619 vsize=848496 CPUtime=99.73 cores=0-7
/proc/17621/task/17623/stat : 17623 (claspmt) R 17619 17621 17370 0 -1 4202560 14582 0 0 0 9962 11 0 0 25 0 8 0 624673116 868859904 131101 33554432000 4194304 4964704 140733548474560 18446744073709551615 256415089290 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=17621/tid=17624] ppid=17619 vsize=848496 CPUtime=100.34 cores=0-7
/proc/17621/task/17624/stat : 17624 (claspmt) R 17619 17621 17370 0 -1 4202560 14683 0 0 0 10022 12 0 0 25 0 8 0 624673116 868859904 131101 33554432000 4194304 4964704 140733548474560 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=17621/tid=17625] ppid=17619 vsize=848496 CPUtime=100.35 cores=0-7
/proc/17621/task/17625/stat : 17625 (claspmt) R 17619 17621 17370 0 -1 4202560 9589 0 0 0 10029 6 0 0 25 0 8 0 624673116 868859904 131101 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 3 0 0 0

################
# More data... #
################

[pid=17621] ppid=17619 vsize=3059608 CPUtime=37723.8 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 724452 0 0 0 3771559 824 0 0 25 0 8 0 624672945 3133038592 716199 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670243 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/17621/statm: 764902 716199 466 189 0 759251 0
[pid=17621/tid=17623] ppid=17619 vsize=3059608 CPUtime=4695.15 cores=0-7
/proc/17621/task/17623/stat : 17623 (claspmt) R 17619 17621 17370 0 -1 4202560 52801 0 0 0 469453 62 0 0 25 0 8 0 624673116 3133038592 716199 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670260 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=17621/tid=17624] ppid=17619 vsize=3059608 CPUtime=4720.39 cores=0-7
/proc/17621/task/17624/stat : 17624 (claspmt) R 17619 17621 17370 0 -1 4202560 36490 0 0 0 472005 34 0 0 25 0 8 0 624673116 3133038592 716199 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=17621/tid=17625] ppid=17619 vsize=3059608 CPUtime=4717.51 cores=0-7
/proc/17621/task/17625/stat : 17625 (claspmt) R 17619 17621 17370 0 -1 4202560 22357 0 0 0 471725 26 0 0 25 0 8 0 624673116 3133038592 716199 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=17621/tid=17626] ppid=17619 vsize=3059608 CPUtime=4719.57 cores=0-7
/proc/17621/task/17626/stat : 17626 (claspmt) R 17619 17621 17370 0 -1 4202560 37746 0 0 0 471919 38 0 0 25 0 8 0 624673116 3133038592 716199 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=17621/tid=17627] ppid=17619 vsize=3059608 CPUtime=4720.44 cores=0-7
/proc/17621/task/17627/stat : 17627 (claspmt) R 17619 17621 17370 0 -1 4202560 37682 0 0 0 471999 45 0 0 25 0 8 0 624673116 3133038592 716199 33554432000 4194304 4964704 140733548474560 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=17621/tid=17628] ppid=17619 vsize=3059608 CPUtime=4719.6 cores=0-7
/proc/17621/task/17628/stat : 17628 (claspmt) R 17619 17621 17370 0 -1 4202560 25162 0 0 0 471926 34 0 0 25 0 8 0 624673116 3133038592 716199 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=17621/tid=17629] ppid=17619 vsize=3059608 CPUtime=4717.7 cores=0-7
/proc/17621/task/17629/stat : 17629 (claspmt) R 17619 17621 17370 0 -1 4202560 440795 0 0 0 471279 491 0 0 25 0 8 0 624673116 3133038592 716199 33554432000 4194304 4964704 140733548474560 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37723.8
Current children cumulated vsize (KiB) 3059608

[startup+4782.3 s]
/proc/loadavg: 8.03 8.04 7.98 9/261 17778
/proc/meminfo: memFree=24326624/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=3190680 CPUtime=38203.3 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 742608 0 0 0 3819474 854 0 0 25 0 8 0 624672945 3267256320 734355 33554432000 4194304 4964704 140733548474560 18446744073709551615 4320944 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/17621/statm: 797670 734355 466 189 0 792019 0
[pid=17621/tid=17623] ppid=17619 vsize=3190680 CPUtime=4755.15 cores=0-7
/proc/17621/task/17623/stat : 17623 (claspmt) R 17619 17621 17370 0 -1 4202560 52801 0 0 0 475453 62 0 0 25 0 8 0 624673116 3267256320 734355 33554432000 4194304 4964704 140733548474560 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=17621/tid=17624] ppid=17619 vsize=3190680 CPUtime=4780.38 cores=0-7
/proc/17621/task/17624/stat : 17624 (claspmt) R 17619 17621 17370 0 -1 4202560 36490 0 0 0 478004 34 0 0 25 0 8 0 624673116 3267256320 734355 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=17621/tid=17625] ppid=17619 vsize=3190680 CPUtime=4777.51 cores=0-7
/proc/17621/task/17625/stat : 17625 (claspmt) R 17619 17621 17370 0 -1 4202560 22357 0 0 0 477725 26 0 0 25 0 8 0 624673116 3267256320 734355 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=17621/tid=17626] ppid=17619 vsize=3190680 CPUtime=4779.57 cores=0-7
/proc/17621/task/17626/stat : 17626 (claspmt) R 17619 17621 17370 0 -1 4202560 37812 0 0 0 477919 38 0 0 25 0 8 0 624673116 3267256320 734355 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=17621/tid=17627] ppid=17619 vsize=3190680 CPUtime=4780.43 cores=0-7
/proc/17621/task/17627/stat : 17627 (claspmt) R 17619 17621 17370 0 -1 4202560 37693 0 0 0 477998 45 0 0 25 0 8 0 624673116 3267256320 734355 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=17621/tid=17628] ppid=17619 vsize=3190680 CPUtime=4779.59 cores=0-7
/proc/17621/task/17628/stat : 17628 (claspmt) R 17619 17621 17370 0 -1 4202560 25167 0 0 0 477925 34 0 0 25 0 8 0 624673116 3267256320 734355 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=17621/tid=17629] ppid=17619 vsize=3190680 CPUtime=4777.69 cores=0-7
/proc/17621/task/17629/stat : 17629 (claspmt) R 17619 17621 17370 0 -1 4202560 456840 0 0 0 477252 517 0 0 25 0 8 0 624673116 3267256320 734355 33554432000 4194304 4964704 140733548474560 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38203.3
Current children cumulated vsize (KiB) 3190680

[startup+4842.3 s]
/proc/loadavg: 8.07 8.05 7.99 9/261 17780
/proc/meminfo: memFree=24284152/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=3256216 CPUtime=38682.8 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 753247 0 0 0 3867409 869 0 0 25 0 8 0 624672945 3334365184 744994 33554432000 4194304 4964704 140733548474560 18446744073709551615 4321016 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/17621/statm: 814054 744994 466 189 0 808403 0
[pid=17621/tid=17623] ppid=17619 vsize=3256216 CPUtime=4815.15 cores=0-7
/proc/17621/task/17623/stat : 17623 (claspmt) R 17619 17621 17370 0 -1 4202560 52808 0 0 0 481453 62 0 0 25 0 8 0 624673116 3334365184 744994 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=17621/tid=17624] ppid=17619 vsize=3256216 CPUtime=4840.39 cores=0-7
/proc/17621/task/17624/stat : 17624 (claspmt) R 17619 17621 17370 0 -1 4202560 36490 0 0 0 484005 34 0 0 25 0 8 0 624673116 3334365184 744994 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=17621/tid=17625] ppid=17619 vsize=3256216 CPUtime=4837.51 cores=0-7
/proc/17621/task/17625/stat : 17625 (claspmt) R 17619 17621 17370 0 -1 4202560 22383 0 0 0 483725 26 0 0 25 0 8 0 624673116 3334365184 744994 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=17621/tid=17626] ppid=17619 vsize=3256216 CPUtime=4839.57 cores=0-7
/proc/17621/task/17626/stat : 17626 (claspmt) R 17619 17621 17370 0 -1 4202560 37870 0 0 0 483919 38 0 0 25 0 8 0 624673116 3334365184 744994 33554432000 4194304 4964704 140733548474560 18446744073709551615 4632342 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=17621/tid=17627] ppid=17619 vsize=3256216 CPUtime=4840.43 cores=0-7
/proc/17621/task/17627/stat : 17627 (claspmt) R 17619 17621 17370 0 -1 4202560 37693 0 0 0 483998 45 0 0 25 0 8 0 624673116 3334365184 744994 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=17621/tid=17628] ppid=17619 vsize=3256216 CPUtime=4839.58 cores=0-7
/proc/17621/task/17628/stat : 17628 (claspmt) R 17619 17621 17370 0 -1 4202560 25561 0 0 0 483923 35 0 0 25 0 8 0 624673116 3334365184 744994 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=17621/tid=17629] ppid=17619 vsize=3256216 CPUtime=4837.7 cores=0-7
/proc/17621/task/17629/stat : 17629 (claspmt) R 17619 17621 17370 0 -1 4202560 463080 0 0 0 483245 525 0 0 25 0 8 0 624673116 3334365184 744994 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38682.8
Current children cumulated vsize (KiB) 3256216

[startup+4902.3 s]
/proc/loadavg: 8.02 8.04 7.99 9/261 17781
/proc/meminfo: memFree=24199264/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=3321752 CPUtime=39162.3 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 774981 0 0 0 3915333 896 0 0 25 0 8 0 624672945 3401474048 766713 33554432000 4194304 4964704 140733548474560 18446744073709551615 4320951 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/17621/statm: 830438 766713 466 189 0 824787 0
[pid=17621/tid=17623] ppid=17619 vsize=3321752 CPUtime=4875.14 cores=0-7
/proc/17621/task/17623/stat : 17623 (claspmt) R 17619 17621 17370 0 -1 4202560 53185 0 0 0 487452 62 0 0 25 0 8 0 624673116 3401474048 766713 33554432000 4194304 4964704 140733548474560 18446744073709551615 4653475 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=17621/tid=17624] ppid=17619 vsize=3321752 CPUtime=4900.39 cores=0-7
/proc/17621/task/17624/stat : 17624 (claspmt) R 17619 17621 17370 0 -1 4202560 36490 0 0 0 490005 34 0 0 25 0 8 0 624673116 3401474048 766713 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=17621/tid=17625] ppid=17619 vsize=3321752 CPUtime=4897.51 cores=0-7
/proc/17621/task/17625/stat : 17625 (claspmt) R 17619 17621 17370 0 -1 4202560 22384 0 0 0 489724 27 0 0 25 0 8 0 624673116 3401474048 766713 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670271 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=17621/tid=17626] ppid=17619 vsize=3321752 CPUtime=4899.57 cores=0-7
/proc/17621/task/17626/stat : 17626 (claspmt) R 17619 17621 17370 0 -1 4202560 38460 0 0 0 489918 39 0 0 25 0 8 0 624673116 3401474048 766713 33554432000 4194304 4964704 140733548474560 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=17621/tid=17627] ppid=17619 vsize=3321752 CPUtime=4900.43 cores=0-7
/proc/17621/task/17627/stat : 17627 (claspmt) R 17619 17621 17370 0 -1 4202560 37834 0 0 0 489998 45 0 0 25 0 8 0 624673116 3401474048 766713 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=17621/tid=17628] ppid=17619 vsize=3321752 CPUtime=4899.58 cores=0-7
/proc/17621/task/17628/stat : 17628 (claspmt) R 17619 17621 17370 0 -1 4202560 25581 0 0 0 489923 35 0 0 25 0 8 0 624673116 3401474048 766713 33554432000 4194304 4964704 140733548474560 18446744073709551615 4320849 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=17621/tid=17629] ppid=17619 vsize=3321752 CPUtime=4897.7 cores=0-7
/proc/17621/task/17629/stat : 17629 (claspmt) R 17619 17621 17370 0 -1 4202560 483655 0 0 0 489219 551 0 0 25 0 8 0 624673116 3401474048 766713 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39162.3
Current children cumulated vsize (KiB) 3321752

[startup+4962.3 s]
/proc/loadavg: 8.06 8.04 7.99 9/261 17783
/proc/meminfo: memFree=24131260/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=3452824 CPUtime=39641.7 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 791596 0 0 0 3963250 923 0 0 25 0 8 0 624672945 3535691776 782685 33554432000 4194304 4964704 140733548474560 18446744073709551615 4659104 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/17621/statm: 863206 782685 466 189 0 857555 0
[pid=17621/tid=17623] ppid=17619 vsize=3452824 CPUtime=4935.09 cores=0-7
/proc/17621/task/17623/stat : 17623 (claspmt) R 17619 17621 17370 0 -1 4202560 54431 0 0 0 493446 63 0 0 18 0 8 0 624673116 3535691776 782685 33554432000 4194304 4964704 140733548474560 18446744073709551615 4653526 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=17621/tid=17624] ppid=17619 vsize=3452824 CPUtime=4960.39 cores=0-7
/proc/17621/task/17624/stat : 17624 (claspmt) R 17619 17621 17370 0 -1 4202560 37069 0 0 0 496004 35 0 0 25 0 8 0 624673116 3535691776 782685 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=17621/tid=17625] ppid=17619 vsize=3452824 CPUtime=4957.51 cores=0-7
/proc/17621/task/17625/stat : 17625 (claspmt) R 17619 17621 17370 0 -1 4202560 22399 0 0 0 495724 27 0 0 25 0 8 0 624673116 3535691776 782685 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=17621/tid=17626] ppid=17619 vsize=3452824 CPUtime=4959.57 cores=0-7
/proc/17621/task/17626/stat : 17626 (claspmt) R 17619 17621 17370 0 -1 4202560 39057 0 0 0 495917 40 0 0 25 0 8 0 624673116 3535691776 782685 33554432000 4194304 4964704 140733548474560 18446744073709551615 4627937 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=17621/tid=17627] ppid=17619 vsize=3452824 CPUtime=4960.43 cores=0-7
/proc/17621/task/17627/stat : 17627 (claspmt) R 17619 17621 17370 0 -1 4202560 37846 0 0 0 495998 45 0 0 25 0 8 0 624673116 3535691776 782685 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=17621/tid=17628] ppid=17619 vsize=3452824 CPUtime=4959.55 cores=0-7
/proc/17621/task/17628/stat : 17628 (claspmt) R 17619 17621 17370 0 -1 4202560 25581 0 0 0 495920 35 0 0 25 0 8 0 624673116 3535691776 782685 33554432000 4194304 4964704 140733548474560 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=17621/tid=17629] ppid=17619 vsize=3452824 CPUtime=4957.7 cores=0-7
/proc/17621/task/17629/stat : 17629 (claspmt) R 17619 17621 17370 0 -1 4202560 495764 0 0 0 495199 571 0 0 25 0 8 0 624673116 3535691776 782685 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39641.7
Current children cumulated vsize (KiB) 3452824



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5007.21 s]
/proc/loadavg: 8.02 8.03 7.99 9/261 17784
/proc/meminfo: memFree=24120304/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=3452824 CPUtime=40000.7 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 794814 0 0 0 3999142 927 0 0 25 0 8 0 624672945 3535691776 785872 33554432000 4194304 4964704 140733548474560 18446744073709551615 4465573 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/17621/statm: 863206 785872 466 189 0 857555 0
[pid=17621/tid=17623] ppid=17619 vsize=3452824 CPUtime=4980.01 cores=0-7
/proc/17621/task/17623/stat : 17623 (claspmt) R 17619 17621 17370 0 -1 4202560 54431 0 0 0 497938 63 0 0 25 0 8 0 624673116 3535691776 785872 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=17621/tid=17624] ppid=17619 vsize=3452824 CPUtime=5005.31 cores=0-7
/proc/17621/task/17624/stat : 17624 (claspmt) R 17619 17621 17370 0 -1 4202560 37069 0 0 0 500496 35 0 0 25 0 8 0 624673116 3535691776 785872 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=17621/tid=17625] ppid=17619 vsize=3452824 CPUtime=5002.43 cores=0-7
/proc/17621/task/17625/stat : 17625 (claspmt) R 17619 17621 17370 0 -1 4202560 22399 0 0 0 500216 27 0 0 25 0 8 0 624673116 3535691776 785872 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=17621/tid=17626] ppid=17619 vsize=3452824 CPUtime=5004.49 cores=0-7
/proc/17621/task/17626/stat : 17626 (claspmt) R 17619 17621 17370 0 -1 4202560 39099 0 0 0 500409 40 0 0 25 0 8 0 624673116 3535691776 785872 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=17621/tid=17627] ppid=17619 vsize=3452824 CPUtime=5005.35 cores=0-7
/proc/17621/task/17627/stat : 17627 (claspmt) R 17619 17621 17370 0 -1 4202560 37847 0 0 0 500490 45 0 0 25 0 8 0 624673116 3535691776 785872 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=17621/tid=17628] ppid=17619 vsize=3452824 CPUtime=5004.47 cores=0-7
/proc/17621/task/17628/stat : 17628 (claspmt) R 17619 17621 17370 0 -1 4202560 25581 0 0 0 500412 35 0 0 25 0 8 0 624673116 3535691776 785872 33554432000 4194304 4964704 140733548474560 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=17621/tid=17629] ppid=17619 vsize=3452824 CPUtime=5002.61 cores=0-7
/proc/17621/task/17629/stat : 17629 (claspmt) R 17619 17621 17370 0 -1 4202560 498939 0 0 0 499686 575 0 0 25 0 8 0 624673116 3535691776 785872 33554432000 4194304 4964704 140733548474560 18446744073709551615 256415088395 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 3452824

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5007.8 s]
/proc/loadavg: 8.02 8.03 7.99 9/261 17784
/proc/meminfo: memFree=24120304/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=3436084 CPUtime=40001.3 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 794829 0 0 0 3999204 927 0 0 25 0 1 0 624672945 3518550016 775827 33554432000 4194304 4964704 140733548474560 18446744073709551615 256415048682 0 0 4096 18946 0 0 0 17 7 0 0 0
/proc/17621/statm: 859021 775827 476 189 0 853370 0
Current children cumulated CPU time (s) 40001.3
Current children cumulated vsize (KiB) 3436084

[startup+5008 s]
/proc/loadavg: 8.02 8.03 7.99 9/261 17784
/proc/meminfo: memFree=24120304/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=3427880 CPUtime=40001.5 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 794829 0 0 0 3999224 927 0 0 25 0 1 0 624672945 3510149120 772922 33554432000 4194304 4964704 140733548474560 18446744073709551615 256415048090 0 0 4096 18946 0 0 0 17 7 0 0 0
/proc/17621/statm: 856970 772922 476 189 0 851319 0
Current children cumulated CPU time (s) 40001.5
Current children cumulated vsize (KiB) 3427880

[startup+5008.1 s]
/proc/loadavg: 8.02 8.03 7.99 9/261 17784
/proc/meminfo: memFree=24120304/32950928 swapFree=67111364/67111528
[pid=17621] ppid=17619 vsize=2032060 CPUtime=40001.6 cores=0-7
/proc/17621/stat : 17621 (claspmt) R 17619 17621 17370 0 -1 4202496 794829 0 0 0 3999229 932 0 0 25 0 1 0 624672945 2080829440 438569 33554432000 4194304 4964704 140733548474560 18446744073709551615 256415435111 0 0 4096 18946 0 0 0 17 7 0 0 0
/proc/17621/statm: 508015 438569 476 189 0 502364 0
Current children cumulated CPU time (s) 40001.6
Current children cumulated vsize (KiB) 2032060

Child status: 0
Real time (s): 5008.18
CPU time (s): 40001.7
CPU user time (s): 39992.3
CPU system time (s): 9.40157
CPU usage (%): 798.728
Max. virtual memory (cumulated for all children) (KiB): 3452824

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39992.3
system time used= 9.40157
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 794830
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 66104
involuntary context switches= 189535

runsolver used 10.8094 second user time and 30.1804 second system time

The end

Launcher Data 

Begin job on node115 at 2011-05-02 15:53:43
IDJOB=3321498
IDBENCH=71425
IDSOLVER=1711
FILE ID=node115/3321498-1304344422
RUNJOBID= node115-1304341073-17384
PBS_JOBID= 13178058
Free space on /tmp= 73516 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT09/CRAFTED/rbsat/random/forced/rbsat-v2640c305320gyes10.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321498-1304344422/watcher-3321498-1304344422 -o /tmp/evaluation-result-3321498-1304344422/solver-3321498-1304344422 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321498-1304344422.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= e4e4b653016e98f210aa8da2214e783a
RANDOM SEED=358854786

node115.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.62
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      27269208 kB
Buffers:        892736 kB
Cached:        2517412 kB
SwapCached:          0 kB
Active:        2240332 kB
Inactive:      2990880 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      27269208 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:            4696 kB
Writeback:           0 kB
AnonPages:     1820824 kB
Mapped:          20660 kB
Slab:           370248 kB
PageTables:      17444 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 37878820 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73512 MiB
End job on node115 at 2011-05-02 17:17:12