Trace number 3321485

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40000.7 5006.23

General information on the benchmark

NameCRAFTED/rbsat/random/unforced/
rbsat-v1375c111739g6.cnf
MD5SUMc7c04151ac14a13eb4a03259d055e119
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1375
Number of clauses111739
Sum of the clauses size224743
Maximum clause length25
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2111684
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 555

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...342800/instance-3321485-1304342800.cnf
0.29/0.38	c Solving...
40000.08/5005.72	
40000.08/5005.72	*** INTERRUPTED! ***
40000.08/5005.72	*** Shutting down threads...
40000.62/5006.22	s UNKNOWN
40000.62/5006.22	
40000.62/5006.22	c INTERRUPTED   : 1
40000.62/5006.22	c Models        : 0+    
40000.62/5006.22	c Time          : 5005.722s (Solving: 5005.34s 1st Model: 0.00s Unsat: 0.00s)
40000.62/5006.22	c CPU Time      : 40000.150s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321485-1304342800/watcher-3321485-1304342800 -o /tmp/evaluation-result-3321485-1304342800/solver-3321485-1304342800 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321485-1304342800.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.03 6.68 7.50 2/246 5095
/proc/meminfo: memFree=21869736/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=21624 CPUtime=0 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 705 0 0 0 0 0 0 0 25 0 1 0 526696169 22142976 624 33554432000 4194304 4964704 140737383394704 18446744073709551615 255106429637 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/5095/statm: 5406 625 386 189 0 280 0

[startup+0.0574549 s]
/proc/loadavg: 6.03 6.68 7.50 2/246 5095
/proc/meminfo: memFree=21869736/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=28304 CPUtime=0.05 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 2787 0 0 0 5 0 0 0 25 0 1 0 526696169 28983296 2220 33554432000 4194304 4964704 140737383394704 18446744073709551615 4607604 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/5095/statm: 7076 2220 399 189 0 1950 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 28304

[startup+0.100437 s]
/proc/loadavg: 6.03 6.68 7.50 2/246 5095
/proc/meminfo: memFree=21869736/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=28304 CPUtime=0.09 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 2787 0 0 0 9 0 0 0 25 0 1 0 526696169 28983296 2220 33554432000 4194304 4964704 140737383394704 18446744073709551615 4607641 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/5095/statm: 7076 2220 399 189 0 1950 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 28304

[startup+0.300401 s]
/proc/loadavg: 6.03 6.68 7.50 2/246 5095
/proc/meminfo: memFree=21869736/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=29096 CPUtime=0.29 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 3007 0 0 0 29 0 0 0 25 0 1 0 526696169 29794304 2440 33554432000 4194304 4964704 140737383394704 18446744073709551615 4600357 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/5095/statm: 7274 2440 400 189 0 2148 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 29096

[startup+1.33553 s]
/proc/loadavg: 6.03 6.68 7.50 2/246 5095
/proc/meminfo: memFree=21869736/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=390940 CPUtime=7.45 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 10718 0 0 0 741 4 0 0 19 0 8 0 526696169 400322560 9898 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670271 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/5095/statm: 97735 9898 464 189 0 92084 0
Current children cumulated CPU time (s) 7.45
Current children cumulated vsize (KiB) 390940

[startup+1.50015 s]
/proc/loadavg: 6.03 6.68 7.50 2/246 5095
/proc/meminfo: memFree=21869736/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=391072 CPUtime=8.77 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 11731 0 0 0 873 4 0 0 19 0 8 0 526696169 400457728 10911 33554432000 4194304 4964704 140737383394704 18446744073709551615 4320902 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/5095/statm: 97768 10911 464 189 0 92117 0
Current children cumulated CPU time (s) 8.77
Current children cumulated vsize (KiB) 391072

[startup+3.10087 s]
/proc/loadavg: 6.03 6.68 7.50 9/254 5103
/proc/meminfo: memFree=21805332/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=395948 CPUtime=21.55 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 18132 0 0 0 2148 7 0 0 23 0 8 0 526696169 405450752 17312 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670454 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/5095/statm: 98987 17312 464 189 0 93336 0
[pid=5095/tid=5097] ppid=5093 vsize=395948 CPUtime=2.63 cores=0-7
/proc/5095/task/5097/stat : 5097 (claspmt) R 5093 5095 5034 0 -1 4202560 1643 0 0 0 263 0 0 0 21 0 8 0 526696207 405450752 17312 33554432000 4194304 4964704 140737383394704 18446744073709551615 4627290 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5095/tid=5098] ppid=5093 vsize=395948 CPUtime=2.63 cores=0-7
/proc/5095/task/5098/stat : 5098 (claspmt) R 5093 5095 5034 0 -1 4202560 2112 0 0 0 263 0 0 0 21 0 8 0 526696207 405450752 17312 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670337 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=5095/tid=5099] ppid=5093 vsize=395948 CPUtime=2.62 cores=0-7
/proc/5095/task/5099/stat : 5099 (claspmt) R 5093 5095 5034 0 -1 4202560 2005 0 0 0 262 0 0 0 21 0 8 0 526696207 405450752 17312 33554432000 4194304 4964704 140737383394704 18446744073709551615 4627248 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=5095/tid=5100] ppid=5093 vsize=395948 CPUtime=2.63 cores=0-7
/proc/5095/task/5100/stat : 5100 (claspmt) R 5093 5095 5034 0 -1 4202560 1827 0 0 0 263 0 0 0 21 0 8 0 526696207 405450752 17312 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=5095/tid=5101] ppid=5093 vsize=395948 CPUtime=2.71 cores=0-7
/proc/5095/task/5101/stat : 5101 (claspmt) R 5093 5095 5034 0 -1 4202560 267 0 0 0 271 0 0 0 25 0 8 0 526696207 405450752 17312 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670184 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=5095/tid=5102] ppid=5093 vsize=395948 CPUtime=2.63 cores=0-7
/proc/5095/task/5102/stat : 5102 (claspmt) R 5093 5095 5034 0 -1 4202560 1739 0 0 0 263 0 0 0 21 0 8 0 526696207 405450752 17312 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670250 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5095/tid=5103] ppid=5093 vsize=395948 CPUtime=2.63 cores=0-7
/proc/5095/task/5103/stat : 5103 (claspmt) R 5093 5095 5034 0 -1 4202560 3673 0 0 0 261 2 0 0 21 0 8 0 526696208 405450752 17312 33554432000 4194304 4964704 140737383394704 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 21.55
Current children cumulated vsize (KiB) 395948

[startup+6.30131 s]
/proc/loadavg: 6.19 6.70 7.50 9/254 5103
/proc/meminfo: memFree=21778304/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=399232 CPUtime=47.12 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 24400 0 0 0 4702 10 0 0 25 0 8 0 526696169 408813568 23580 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/5095/statm: 99808 23580 466 189 0 94157 0
[pid=5095/tid=5097] ppid=5093 vsize=399232 CPUtime=5.83 cores=0-7
/proc/5095/task/5097/stat : 5097 (claspmt) R 5093 5095 5034 0 -1 4202560 2245 0 0 0 583 0 0 0 25 0 8 0 526696207 408813568 23580 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5095/tid=5098] ppid=5093 vsize=399232 CPUtime=5.84 cores=0-7
/proc/5095/task/5098/stat : 5098 (claspmt) R 5093 5095 5034 0 -1 4202560 2792 0 0 0 583 1 0 0 25 0 8 0 526696207 408813568 23580 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670263 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=5095/tid=5099] ppid=5093 vsize=399232 CPUtime=5.8 cores=0-7
/proc/5095/task/5099/stat : 5099 (claspmt) R 5093 5095 5034 0 -1 4202560 2516 0 0 0 580 0 0 0 25 0 8 0 526696207 408813568 23580 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670241 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5095/tid=5100] ppid=5093 vsize=399232 CPUtime=5.83 cores=0-7
/proc/5095/task/5100/stat : 5100 (claspmt) R 5093 5095 5034 0 -1 4202560 2632 0 0 0 582 1 0 0 25 0 8 0 526696207 408813568 23580 33554432000 4194304 4964704 140737383394704 18446744073709551615 4660415 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=5095/tid=5101] ppid=5093 vsize=399232 CPUtime=5.91 cores=0-7
/proc/5095/task/5101/stat : 5101 (claspmt) R 5093 5095 5034 0 -1 4202560 883 0 0 0 591 0 0 0 25 0 8 0 526696207 408813568 23580 33554432000 4194304 4964704 140737383394704 18446744073709551615 4627290 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=5095/tid=5102] ppid=5093 vsize=399232 CPUtime=5.82 cores=0-7
/proc/5095/task/5102/stat : 5102 (claspmt) R 5093 5095 5034 0 -1 4202560 2463 0 0 0 582 0 0 0 25 0 8 0 526696207 408813568 23580 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=5095/tid=5103] ppid=5093 vsize=399232 CPUtime=5.83 cores=0-7
/proc/5095/task/5103/stat : 5103 (claspmt) R 5093 5095 5034 0 -1 4202560 5315 0 0 0 580 3 0 0 23 0 8 0 526696208 408813568 23580 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 47.12
Current children cumulated vsize (KiB) 399232

[startup+12.7012 s]
/proc/loadavg: 6.34 6.72 7.50 9/254 5103
/proc/meminfo: memFree=21758480/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=403352 CPUtime=98.32 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 29475 0 0 0 9819 13 0 0 25 0 8 0 526696169 413032448 28655 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670266 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/5095/statm: 100838 28655 466 189 0 95187 0
[pid=5095/tid=5097] ppid=5093 vsize=403352 CPUtime=12.24 cores=0-7
/proc/5095/task/5097/stat : 5097 (claspmt) R 5093 5095 5034 0 -1 4202560 2768 0 0 0 1223 1 0 0 25 0 8 0 526696207 413032448 28658 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5095/tid=5098] ppid=5093 vsize=403352 CPUtime=12.25 cores=0-7
/proc/5095/task/5098/stat : 5098 (claspmt) R 5093 5095 5034 0 -1 4202560 3207 0 0 0 1224 1 0 0 25 0 8 0 526696207 413032448 28658 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=5095/tid=5099] ppid=5093 vsize=403352 CPUtime=12.21 cores=0-7
/proc/5095/task/5099/stat : 5099 (claspmt) R 5093 5095 5034 0 -1 4202560 3057 0 0 0 1220 1 0 0 25 0 8 0 526696207 413032448 28658 33554432000 4194304 4964704 140737383394704 18446744073709551615 4627252 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5095/tid=5100] ppid=5093 vsize=403352 CPUtime=12.24 cores=0-7
/proc/5095/task/5100/stat : 5100 (claspmt) R 5093 5095 5034 0 -1 4202560 3015 0 0 0 1223 1 0 0 25 0 8 0 526696207 413032448 28658 33554432000 4194304 4964704 140737383394704 18446744073709551615 4320939 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=5095/tid=5101] ppid=5093 vsize=403352 CPUtime=12.31 cores=0-7
/proc/5095/task/5101/stat : 5101 (claspmt) R 5093 5095 5034 0 -1 4202560 1400 0 0 0 1231 0 0 0 25 0 8 0 526696207 413032448 28658 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=5095/tid=5102] ppid=5093 vsize=403352 CPUtime=12.18 cores=0-7
/proc/5095/task/5102/stat : 5102 (claspmt) R 5093 5095 5034 0 -1 4202560 2917 0 0 0 1218 0 0 0 25 0 8 0 526696207 413032448 28659 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670255 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=5095/tid=5103] ppid=5093 vsize=403352 CPUtime=12.24 cores=0-7
/proc/5095/task/5103/stat : 5103 (claspmt) R 5093 5095 5034 0 -1 4202560 7059 0 0 0 1220 4 0 0 25 0 8 0 526696208 413032448 28659 33554432000 4194304 4964704 140737383394704 18446744073709551615 255106429985 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 98.32
Current children cumulated vsize (KiB) 403352

[startup+25.5009 s]
/proc/loadavg: 6.70 6.78 7.51 9/254 5104
/proc/meminfo: memFree=21729172/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=408364 CPUtime=200.56 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 38291 0 0 0 20038 18 0 0 25 0 8 0 526696169 418164736 37471 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670458 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/5095/statm: 102091 37471 466 189 0 96440 0
[pid=5095/tid=5097] ppid=5093 vsize=408364 CPUtime=25.03 cores=0-7
/proc/5095/task/5097/stat : 5097 (claspmt) R 5093 5095 5034 0 -1 4202560 3416 0 0 0 2502 1 0 0 25 0 8 0 526696207 418164736 37471 33554432000 4194304 4964704 140737383394704 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5095/tid=5098] ppid=5093 vsize=408364 CPUtime=25.04 cores=0-7
/proc/5095/task/5098/stat : 5098 (claspmt) R 5093 5095 5034 0 -1 4202560 4090 0 0 0 2503 1 0 0 25 0 8 0 526696207 418164736 37471 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670230 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=5095/tid=5099] ppid=5093 vsize=408364 CPUtime=25 cores=0-7
/proc/5095/task/5099/stat : 5099 (claspmt) R 5093 5095 5034 0 -1 4202560 3640 0 0 0 2499 1 0 0 25 0 8 0 526696207 418164736 37471 33554432000 4194304 4964704 140737383394704 18446744073709551615 255106427309 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5095/tid=5100] ppid=5093 vsize=408364 CPUtime=25.03 cores=0-7
/proc/5095/task/5100/stat : 5100 (claspmt) R 5093 5095 5034 0 -1 4202560 3957 0 0 0 2502 1 0 0 25 0 8 0 526696207 418164736 37471 33554432000 4194304 4964704 140737383394704 18446744073709551615 4320875 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=5095/tid=5101] ppid=5093 vsize=408364 CPUtime=25.1 cores=0-7
/proc/5095/task/5101/stat : 5101 (claspmt) R 5093 5095 5034 0 -1 4202560 2041 0 0 0 2510 0 0 0 25 0 8 0 526696207 418164736 37471 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670255 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=5095/tid=5102] ppid=5093 vsize=408364 CPUtime=24.88 cores=0-7
/proc/5095/task/5102/stat : 5102 (claspmt) R 5093 5095 5034 0 -1 4202560 3274 0 0 0 2487 1 0 0 25 0 8 0 526696207 418164736 37471 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=5095/tid=5103] ppid=5093 vsize=408364 CPUtime=25.03 cores=0-7
/proc/5095/task/5103/stat : 5103 (claspmt) R 5093 5095 5034 0 -1 4202560 10986 0 0 0 2497 6 0 0 25 0 8 0 526696208 418164736 37471 33554432000 4194304 4964704 140737383394704 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 200.56
Current children cumulated vsize (KiB) 408364

[startup+51.1004 s]
/proc/loadavg: 7.14 6.88 7.52 9/254 5104
/proc/meminfo: memFree=21673548/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=416812 CPUtime=405.16 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 51890 0 0 0 40486 30 0 0 25 0 8 0 526696169 426815488 51063 33554432000 4194304 4964704 140737383394704 18446744073709551615 4627290 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/5095/statm: 104203 51063 466 189 0 98552 0
[pid=5095/tid=5097] ppid=5093 vsize=416812 CPUtime=50.61 cores=0-7
/proc/5095/task/5097/stat : 5097 (claspmt) R 5093 5095 5034 0 -1 4202560 5521 0 0 0 5059 2 0 0 25 0 8 0 526696207 426815488 51063 33554432000 4194304 4964704 140737383394704 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5095/tid=5098] ppid=5093 vsize=416812 CPUtime=50.63 cores=0-7
/proc/5095/task/5098/stat : 5098 (claspmt) R 5093 5095 5034 0 -1 4202560 5474 0 0 0 5061 2 0 0 25 0 8 0 526696207 426815488 51063 33554432000 4194304 4964704 140737383394704 18446744073709551615 4668931 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=5095/tid=5099] ppid=5093 vsize=416812 CPUtime=50.6 cores=0-7
/proc/5095/task/5099/stat : 5099 (claspmt) R 5093 5095 5034 0 -1 4202560 4660 0 0 0 5058 2 0 0 25 0 8 0 526696207 426815488 51063 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670269 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5095/tid=5100] ppid=5093 vsize=416812 CPUtime=50.63 cores=0-7
/proc/5095/task/5100/stat : 5100 (claspmt) R 5093 5095 5034 0 -1 4202560 5114 0 0 0 5061 2 0 0 25 0 8 0 526696207 426815488 51063 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=5095/tid=5101] ppid=5093 vsize=416812 CPUtime=50.69 cores=0-7
/proc/5095/task/5101/stat : 5101 (claspmt) R 5093 5095 5034 0 -1 4202560 3353 0 0 0 5068 1 0 0 25 0 8 0 526696207 426815488 51063 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=5095/tid=5102] ppid=5093 vsize=416812 CPUtime=50.3 cores=0-7
/proc/5095/task/5102/stat : 5102 (claspmt) R 5093 5095 5034 0 -1 4202560 4205 0 0 0 5028 2 0 0 25 0 8 0 526696207 426815488 51063 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=5095/tid=5103] ppid=5093 vsize=416812 CPUtime=50.63 cores=0-7
/proc/5095/task/5103/stat : 5103 (claspmt) R 5093 5095 5034 0 -1 4202560 15343 0 0 0 5053 10 0 0 25 0 8 0 526696208 426815488 51063 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 405.16
Current children cumulated vsize (KiB) 416812

[startup+102.3 s]
/proc/loadavg: 7.63 7.05 7.55 9/254 5106
/proc/meminfo: memFree=21588224/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=432504 CPUtime=814.31 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 72271 0 0 0 81383 48 0 0 25 0 8 0 526696169 442884096 71407 33554432000 4194304 4964704 140737383394704 18446744073709551615 4332288 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/5095/statm: 108126 71407 466 189 0 102475 0
[pid=5095/tid=5097] ppid=5093 vsize=432504 CPUtime=101.8 cores=0-7
/proc/5095/task/5097/stat : 5097 (claspmt) R 5093 5095 5034 0 -1 4202560 7180 0 0 0 10177 3 0 0 25 0 8 0 526696207 442884096 71407 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670179 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5095/tid=5098] ppid=5093 vsize=432504 CPUtime=101.84 cores=0-7
/proc/5095/task/5098/stat : 5098 (claspmt) R 5093 5095 5034 0 -1 4202560 6772 0 0 0 10181 3 0 0 25 0 8 0 526696207 442884096 71407 33554432000 4194304 4964704 140737383394704 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=5095/tid=5099] ppid=5093 vsize=432504 CPUtime=101.79 cores=0-7
/proc/5095/task/5099/stat : 5099 (claspmt) R 5093 5095 5034 0 -1 4202560 6126 0 0 0 10175 4 0 0 25 0 8 0 526696207 442884096 71407 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0

################
# More data... #
################

/proc/5095/task/5102/stat : 5102 (claspmt) R 5093 5095 5034 0 -1 4202560 20711 0 0 0 465626 33 0 0 25 0 8 0 526696207 1399873536 288046 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5095/tid=5103] ppid=5093 vsize=1367064 CPUtime=4661.8 cores=0-7
/proc/5095/task/5103/stat : 5103 (claspmt) R 5093 5095 5034 0 -1 4202560 137427 0 0 0 465997 183 0 0 25 0 8 0 526696208 1399873536 288046 33554432000 4194304 4964704 140737383394704 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 37255.6
Current children cumulated vsize (KiB) 1367064

[startup+4722.3 s]
/proc/loadavg: 8.00 8.03 8.00 9/254 5241
/proc/meminfo: memFree=20717724/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=1367528 CPUtime=37735.1 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 299217 0 0 0 3773148 367 0 0 25 0 8 0 526696169 1400348672 288421 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670248 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/5095/statm: 341882 288421 466 189 0 336231 0
[pid=5095/tid=5097] ppid=5093 vsize=1367528 CPUtime=4689.2 cores=0-7
/proc/5095/task/5097/stat : 5097 (claspmt) R 5093 5095 5034 0 -1 4202560 27148 0 0 0 468884 36 0 0 25 0 8 0 526696207 1400348672 288421 33554432000 4194304 4964704 140737383394704 18446744073709551615 4659143 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=5095/tid=5098] ppid=5093 vsize=1367528 CPUtime=4721.82 cores=0-7
/proc/5095/task/5098/stat : 5098 (claspmt) R 5093 5095 5034 0 -1 4202560 30068 0 0 0 472163 19 0 0 25 0 8 0 526696207 1400348672 288421 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=5095/tid=5099] ppid=5093 vsize=1367528 CPUtime=4721.5 cores=0-7
/proc/5095/task/5099/stat : 5099 (claspmt) R 5093 5095 5034 0 -1 4202560 18104 0 0 0 472126 24 0 0 25 0 8 0 526696207 1400348672 288421 33554432000 4194304 4964704 140737383394704 18446744073709551615 4320902 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5095/tid=5100] ppid=5093 vsize=1367528 CPUtime=4721.75 cores=0-7
/proc/5095/task/5100/stat : 5100 (claspmt) R 5093 5095 5034 0 -1 4202560 19160 0 0 0 472159 16 0 0 25 0 8 0 526696207 1400348672 288421 33554432000 4194304 4964704 140737383394704 18446744073709551615 4628019 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=5095/tid=5101] ppid=5093 vsize=1367528 CPUtime=4721.68 cores=0-7
/proc/5095/task/5101/stat : 5101 (claspmt) R 5093 5095 5034 0 -1 4202560 17702 0 0 0 472152 16 0 0 25 0 8 0 526696207 1400348672 288421 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=5095/tid=5102] ppid=5093 vsize=1367528 CPUtime=4716.59 cores=0-7
/proc/5095/task/5102/stat : 5102 (claspmt) R 5093 5095 5034 0 -1 4202560 20711 0 0 0 471626 33 0 0 25 0 8 0 526696207 1400348672 288421 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5095/tid=5103] ppid=5093 vsize=1367528 CPUtime=4721.8 cores=0-7
/proc/5095/task/5103/stat : 5103 (claspmt) R 5093 5095 5034 0 -1 4202560 137430 0 0 0 471997 183 0 0 25 0 8 0 526696208 1400348672 288421 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 37735.2
Current children cumulated vsize (KiB) 1367528

[startup+4782.3 s]
/proc/loadavg: 8.06 8.04 8.00 9/254 5242
/proc/meminfo: memFree=20716100/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=1367528 CPUtime=38214.7 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 299797 0 0 0 3821101 369 0 0 25 0 8 0 526696169 1400348672 288784 33554432000 4194304 4964704 140737383394704 18446744073709551615 4683933 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/5095/statm: 341882 288784 466 189 0 336231 0
[pid=5095/tid=5097] ppid=5093 vsize=1367528 CPUtime=4749.02 cores=0-7
/proc/5095/task/5097/stat : 5097 (claspmt) R 5093 5095 5034 0 -1 4202560 27445 0 0 0 474865 37 0 0 25 0 8 0 526696207 1400348672 288784 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5095/tid=5098] ppid=5093 vsize=1367528 CPUtime=4781.83 cores=0-7
/proc/5095/task/5098/stat : 5098 (claspmt) R 5093 5095 5034 0 -1 4202560 30250 0 0 0 478164 19 0 0 25 0 8 0 526696207 1400348672 288784 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=5095/tid=5099] ppid=5093 vsize=1367528 CPUtime=4781.23 cores=0-7
/proc/5095/task/5099/stat : 5099 (claspmt) R 5093 5095 5034 0 -1 4202560 18127 0 0 0 478098 25 0 0 25 0 8 0 526696207 1400348672 288784 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=5095/tid=5100] ppid=5093 vsize=1367528 CPUtime=4781.76 cores=0-7
/proc/5095/task/5100/stat : 5100 (claspmt) R 5093 5095 5034 0 -1 4202560 19161 0 0 0 478160 16 0 0 25 0 8 0 526696207 1400348672 288784 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=5095/tid=5101] ppid=5093 vsize=1367528 CPUtime=4781.68 cores=0-7
/proc/5095/task/5101/stat : 5101 (claspmt) R 5093 5095 5034 0 -1 4202560 17760 0 0 0 478151 17 0 0 25 0 8 0 526696207 1400348672 288784 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=5095/tid=5102] ppid=5093 vsize=1367528 CPUtime=4776.6 cores=0-7
/proc/5095/task/5102/stat : 5102 (claspmt) R 5093 5095 5034 0 -1 4202560 20711 0 0 0 477627 33 0 0 25 0 8 0 526696207 1400348672 288784 33554432000 4194304 4964704 140737383394704 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5095/tid=5103] ppid=5093 vsize=1367528 CPUtime=4781.8 cores=0-7
/proc/5095/task/5103/stat : 5103 (claspmt) R 5093 5095 5034 0 -1 4202560 137447 0 0 0 477997 183 0 0 25 0 8 0 526696208 1400348672 288784 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38214.7
Current children cumulated vsize (KiB) 1367528

[startup+4842.3 s]
/proc/loadavg: 8.07 8.04 8.00 9/254 5244
/proc/meminfo: memFree=20714224/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=1368240 CPUtime=38694.1 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 300258 0 0 0 3869043 370 0 0 25 0 8 0 526696169 1401077760 289245 33554432000 4194304 4964704 140737383394704 18446744073709551615 4320951 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/5095/statm: 342060 289245 466 189 0 336409 0
[pid=5095/tid=5097] ppid=5093 vsize=1368240 CPUtime=4809.01 cores=0-7
/proc/5095/task/5097/stat : 5097 (claspmt) R 5093 5095 5034 0 -1 4202560 27525 0 0 0 480864 37 0 0 25 0 8 0 526696207 1401077760 289245 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5095/tid=5098] ppid=5093 vsize=1368240 CPUtime=4841.82 cores=0-7
/proc/5095/task/5098/stat : 5098 (claspmt) R 5093 5095 5034 0 -1 4202560 30396 0 0 0 484163 19 0 0 25 0 8 0 526696207 1401077760 289245 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=5095/tid=5099] ppid=5093 vsize=1368240 CPUtime=4840.73 cores=0-7
/proc/5095/task/5099/stat : 5099 (claspmt) R 5093 5095 5034 0 -1 4202560 18127 0 0 0 484048 25 0 0 25 0 8 0 526696207 1401077760 289245 33554432000 4194304 4964704 140737383394704 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=5095/tid=5100] ppid=5093 vsize=1368240 CPUtime=4841.73 cores=0-7
/proc/5095/task/5100/stat : 5100 (claspmt) R 5093 5095 5034 0 -1 4202560 19205 0 0 0 484157 16 0 0 25 0 8 0 526696207 1401077760 289245 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=5095/tid=5101] ppid=5093 vsize=1368240 CPUtime=4841.68 cores=0-7
/proc/5095/task/5101/stat : 5101 (claspmt) R 5093 5095 5034 0 -1 4202560 17760 0 0 0 484151 17 0 0 25 0 8 0 526696207 1401077760 289245 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=5095/tid=5102] ppid=5093 vsize=1368240 CPUtime=4836.58 cores=0-7
/proc/5095/task/5102/stat : 5102 (claspmt) R 5093 5095 5034 0 -1 4202560 20881 0 0 0 483625 33 0 0 25 0 8 0 526696207 1401077760 289245 33554432000 4194304 4964704 140737383394704 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5095/tid=5103] ppid=5093 vsize=1368240 CPUtime=4841.8 cores=0-7
/proc/5095/task/5103/stat : 5103 (claspmt) R 5093 5095 5034 0 -1 4202560 137468 0 0 0 483997 183 0 0 25 0 8 0 526696208 1401077760 289245 33554432000 4194304 4964704 140737383394704 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38694.1
Current children cumulated vsize (KiB) 1368240

[startup+4902.31 s]
/proc/loadavg: 8.07 8.05 8.00 9/254 5246
/proc/meminfo: memFree=20712724/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=1368924 CPUtime=39173.7 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 300579 0 0 0 3916997 371 0 0 25 0 8 0 526696169 1401778176 289566 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/5095/statm: 342231 289566 466 189 0 336580 0
[pid=5095/tid=5097] ppid=5093 vsize=1368924 CPUtime=4868.99 cores=0-7
/proc/5095/task/5097/stat : 5097 (claspmt) R 5093 5095 5034 0 -1 4202560 27527 0 0 0 486862 37 0 0 25 0 8 0 526696207 1401778176 289566 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5095/tid=5098] ppid=5093 vsize=1368924 CPUtime=4901.83 cores=0-7
/proc/5095/task/5098/stat : 5098 (claspmt) R 5093 5095 5034 0 -1 4202560 30412 0 0 0 490164 19 0 0 25 0 8 0 526696207 1401778176 289566 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=5095/tid=5099] ppid=5093 vsize=1368924 CPUtime=4900.25 cores=0-7
/proc/5095/task/5099/stat : 5099 (claspmt) R 5093 5095 5034 0 -1 4202560 18127 0 0 0 490000 25 0 0 25 0 8 0 526696207 1401778176 289566 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=5095/tid=5100] ppid=5093 vsize=1368924 CPUtime=4901.74 cores=0-7
/proc/5095/task/5100/stat : 5100 (claspmt) R 5093 5095 5034 0 -1 4202560 19244 0 0 0 490158 16 0 0 25 0 8 0 526696207 1401778176 289566 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=5095/tid=5101] ppid=5093 vsize=1368924 CPUtime=4901.69 cores=0-7
/proc/5095/task/5101/stat : 5101 (claspmt) R 5093 5095 5034 0 -1 4202560 17760 0 0 0 490152 17 0 0 25 0 8 0 526696207 1401778176 289566 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=5095/tid=5102] ppid=5093 vsize=1368924 CPUtime=4896.6 cores=0-7
/proc/5095/task/5102/stat : 5102 (claspmt) R 5093 5095 5034 0 -1 4202560 20965 0 0 0 489626 34 0 0 25 0 8 0 526696207 1401778176 289566 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670250 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5095/tid=5103] ppid=5093 vsize=1368924 CPUtime=4901.8 cores=0-7
/proc/5095/task/5103/stat : 5103 (claspmt) R 5093 5095 5034 0 -1 4202560 137611 0 0 0 489997 183 0 0 25 0 8 0 526696208 1401778176 289566 33554432000 4194304 4964704 140737383394704 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39173.7
Current children cumulated vsize (KiB) 1368924

[startup+4962.3 s]
/proc/loadavg: 8.07 8.05 8.00 9/254 5247
/proc/meminfo: memFree=20706504/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=1375092 CPUtime=39653.1 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 302366 0 0 0 3964940 372 0 0 25 0 8 0 526696169 1408094208 291207 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/5095/statm: 343773 291207 466 189 0 338122 0
[pid=5095/tid=5097] ppid=5093 vsize=1375092 CPUtime=4928.98 cores=0-7
/proc/5095/task/5097/stat : 5097 (claspmt) R 5093 5095 5034 0 -1 4202560 27529 0 0 0 492861 37 0 0 25 0 8 0 526696207 1408094208 291207 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5095/tid=5098] ppid=5093 vsize=1375092 CPUtime=4961.83 cores=0-7
/proc/5095/task/5098/stat : 5098 (claspmt) R 5093 5095 5034 0 -1 4202560 30972 0 0 0 496163 20 0 0 25 0 8 0 526696207 1408094208 291207 33554432000 4194304 4964704 140737383394704 18446744073709551615 4627338 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=5095/tid=5099] ppid=5093 vsize=1375092 CPUtime=4959.75 cores=0-7
/proc/5095/task/5099/stat : 5099 (claspmt) R 5093 5095 5034 0 -1 4202560 18142 0 0 0 495950 25 0 0 25 0 8 0 526696207 1408094208 291207 33554432000 4194304 4964704 140737383394704 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=5095/tid=5100] ppid=5093 vsize=1375092 CPUtime=4961.74 cores=0-7
/proc/5095/task/5100/stat : 5100 (claspmt) R 5093 5095 5034 0 -1 4202560 19246 0 0 0 496158 16 0 0 25 0 8 0 526696207 1408094208 291207 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=5095/tid=5101] ppid=5093 vsize=1375092 CPUtime=4961.68 cores=0-7
/proc/5095/task/5101/stat : 5101 (claspmt) R 5093 5095 5034 0 -1 4202560 18157 0 0 0 496151 17 0 0 25 0 8 0 526696207 1408094208 291207 33554432000 4194304 4964704 140737383394704 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=5095/tid=5102] ppid=5093 vsize=1375092 CPUtime=4956.59 cores=0-7
/proc/5095/task/5102/stat : 5102 (claspmt) R 5093 5095 5034 0 -1 4202560 21026 0 0 0 495625 34 0 0 25 0 8 0 526696207 1408094208 291207 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5095/tid=5103] ppid=5093 vsize=1375092 CPUtime=4961.8 cores=0-7
/proc/5095/task/5103/stat : 5103 (claspmt) R 5093 5095 5034 0 -1 4202560 138344 0 0 0 495996 184 0 0 25 0 8 0 526696208 1408094208 291207 33554432000 4194304 4964704 140737383394704 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39653.1
Current children cumulated vsize (KiB) 1375092



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5005.71 s]
/proc/loadavg: 8.08 8.06 8.00 9/254 5248
/proc/meminfo: memFree=20693848/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=1383116 CPUtime=40000.1 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 305711 0 0 0 3999632 376 0 0 25 0 8 0 526696169 1416310784 294461 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670255 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/5095/statm: 345779 294461 466 189 0 340128 0
[pid=5095/tid=5097] ppid=5093 vsize=1383116 CPUtime=4972.4 cores=0-7
/proc/5095/task/5097/stat : 5097 (claspmt) R 5093 5095 5034 0 -1 4202560 28428 0 0 0 497202 38 0 0 25 0 8 0 526696207 1416310784 294461 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5095/tid=5098] ppid=5093 vsize=1383116 CPUtime=5005.24 cores=0-7
/proc/5095/task/5098/stat : 5098 (claspmt) R 5093 5095 5034 0 -1 4202560 31254 0 0 0 500504 20 0 0 25 0 8 0 526696207 1416310784 294461 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=5095/tid=5099] ppid=5093 vsize=1383116 CPUtime=5002.81 cores=0-7
/proc/5095/task/5099/stat : 5099 (claspmt) R 5093 5095 5034 0 -1 4202560 18784 0 0 0 500255 26 0 0 25 0 8 0 526696207 1416310784 294461 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=5095/tid=5100] ppid=5093 vsize=1383116 CPUtime=5005.15 cores=0-7
/proc/5095/task/5100/stat : 5100 (claspmt) R 5093 5095 5034 0 -1 4202560 19323 0 0 0 500499 16 0 0 25 0 8 0 526696207 1416310784 294461 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=5095/tid=5101] ppid=5093 vsize=1383116 CPUtime=5005.1 cores=0-7
/proc/5095/task/5101/stat : 5101 (claspmt) R 5093 5095 5034 0 -1 4202560 18186 0 0 0 500493 17 0 0 25 0 8 0 526696207 1416310784 294461 33554432000 4194304 4964704 140737383394704 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=5095/tid=5102] ppid=5093 vsize=1383116 CPUtime=5000.01 cores=0-7
/proc/5095/task/5102/stat : 5102 (claspmt) R 5093 5095 5034 0 -1 4202560 22066 0 0 0 499966 35 0 0 25 0 8 0 526696207 1416310784 294461 33554432000 4194304 4964704 140737383394704 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5095/tid=5103] ppid=5093 vsize=1383116 CPUtime=5005.21 cores=0-7
/proc/5095/task/5103/stat : 5103 (claspmt) R 5093 5095 5034 0 -1 4202560 138452 0 0 0 500337 184 0 0 25 0 8 0 526696208 1416310784 294461 33554432000 4194304 4964704 140737383394704 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 1383116

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5006.1 s]
/proc/loadavg: 8.08 8.06 8.00 9/254 5248
/proc/meminfo: memFree=20693848/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=1371996 CPUtime=40000.5 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 305722 0 0 0 3999676 376 0 0 25 0 1 0 526696169 1404923904 292223 33554432000 4194304 4964704 140737383394704 18446744073709551615 255106424708 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/5095/statm: 342999 292223 476 189 0 337348 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 1371996

[startup+5006.2 s]
/proc/loadavg: 8.08 8.06 8.00 9/254 5248
/proc/meminfo: memFree=20693848/32951124 swapFree=67111448/67111528
[pid=5095] ppid=5093 vsize=730500 CPUtime=40000.6 cores=0-7
/proc/5095/stat : 5095 (claspmt) R 5093 5095 5034 0 -1 4202496 305722 0 0 0 3999683 379 0 0 25 0 1 0 526696169 748032000 119974 33554432000 4194304 4964704 140737383394704 18446744073709551615 255106812263 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/5095/statm: 182625 119974 476 189 0 176974 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 730500

Child status: 0
Real time (s): 5006.23
CPU time (s): 40000.7
CPU user time (s): 39996.8
CPU system time (s): 3.82542
CPU usage (%): 799.018
Max. virtual memory (cumulated for all children) (KiB): 1383116

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39996.8
system time used= 3.82542
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 305723
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 71274
involuntary context switches= 190221

runsolver used 11.4813 second user time and 28.0537 second system time

The end

Launcher Data 

Begin job on node144 at 2011-05-02 15:26:40
IDJOB=3321485
IDBENCH=71315
IDSOLVER=1711
FILE ID=node144/3321485-1304342800
RUNJOBID= node144-1304342734-5048
PBS_JOBID= 13178064
Free space on /tmp= 73624 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT09/CRAFTED/rbsat/random/unforced/rbsat-v1375c111739g6.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321485-1304342800/watcher-3321485-1304342800 -o /tmp/evaluation-result-3321485-1304342800/solver-3321485-1304342800 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321485-1304342800.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= c7c04151ac14a13eb4a03259d055e119
RANDOM SEED=834508227

node144.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.16
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21870008 kB
Buffers:       2149560 kB
Cached:        5012412 kB
SwapCached:         80 kB
Active:        3909028 kB
Inactive:      6603788 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21870008 kB
SwapTotal:    67111528 kB
SwapFree:     67111448 kB
Dirty:            2312 kB
Writeback:           0 kB
AnonPages:     3350448 kB
Mapped:          21516 kB
Slab:           485920 kB
PageTables:      18936 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 37122848 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73620 MiB
End job on node144 at 2011-05-02 16:50:08