Trace number 3321511

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40000.7 5006.17

General information on the benchmark

NameCRAFTED/rbsat/crafted/forced/
rbsat-v945c61409gyes10.cnf
MD5SUM601a386973128161e9173e2bdf583672
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2.38564
Satisfiable
(Un)Satisfiability was proved
Number of variables945
Number of clauses61409
Sum of the clauses size123673
Maximum clause length21
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 261364
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...357453/instance-3321511-1304357453.cnf
0.09/0.18	c Solving...
40000.43/5005.91	
40000.43/5005.91	*** INTERRUPTED! ***
40000.43/5005.91	*** Shutting down threads...
40000.67/5006.16	s UNKNOWN
40000.67/5006.16	
40000.67/5006.16	c INTERRUPTED   : 1
40000.67/5006.16	c Models        : 0+    
40000.67/5006.16	c Time          : 5005.913s (Solving: 5005.73s 1st Model: 0.00s Unsat: 0.00s)
40000.67/5006.16	c CPU Time      : 40000.480s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321511-1304357453/watcher-3321511-1304357453 -o /tmp/evaluation-result-3321511-1304357453/solver-3321511-1304357453 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321511-1304357453.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.24 8.05 7.68 1/261 21242
/proc/meminfo: memFree=23598188/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=22148 CPUtime=0 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 865 0 0 0 0 0 0 0 23 0 1 0 625976103 22679552 752 33554432000 4194304 4964704 140736755562704 18446744073709551615 215988253381 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/21242/statm: 5537 753 386 189 0 411 0

[startup+0.030143 s]
/proc/loadavg: 8.24 8.05 7.68 1/261 21242
/proc/meminfo: memFree=23598188/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=24800 CPUtime=0.02 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 1688 0 0 0 2 0 0 0 23 0 1 0 625976103 25395200 1431 33554432000 4194304 4964704 140736755562704 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/21242/statm: 6200 1431 399 189 0 1074 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 24800

[startup+0.100131 s]
/proc/loadavg: 8.24 8.05 7.68 1/261 21242
/proc/meminfo: memFree=23598188/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=24932 CPUtime=0.09 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 1717 0 0 0 9 0 0 0 23 0 1 0 625976103 25530368 1460 33554432000 4194304 4964704 140736755562704 18446744073709551615 215988253381 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/21242/statm: 6233 1460 400 189 0 1107 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 24932

[startup+0.300098 s]
/proc/loadavg: 8.24 8.05 7.68 1/261 21242
/proc/meminfo: memFree=23598188/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=385096 CPUtime=0.8 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 3858 0 0 0 79 1 0 0 18 0 8 0 625976103 394338304 3471 33554432000 4194304 4964704 140736755562704 18446744073709551615 4379771 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/21242/statm: 96274 3471 464 189 0 90623 0
Current children cumulated CPU time (s) 0.8
Current children cumulated vsize (KiB) 385096

[startup+0.70103 s]
/proc/loadavg: 8.24 8.05 7.68 1/261 21242
/proc/meminfo: memFree=23598188/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=387632 CPUtime=4 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 8298 0 0 0 397 3 0 0 18 0 8 0 625976103 396935168 7911 33554432000 4194304 4964704 140736755562704 18446744073709551615 4627864 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/21242/statm: 96908 7911 464 189 0 91257 0
Current children cumulated CPU time (s) 4
Current children cumulated vsize (KiB) 387632

[startup+1.50089 s]
/proc/loadavg: 8.24 8.05 7.68 9/269 21250
/proc/meminfo: memFree=23565908/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=389912 CPUtime=10.38 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 11610 0 0 0 1034 4 0 0 19 0 8 0 625976103 399269888 11223 33554432000 4194304 4964704 140736755562704 18446744073709551615 4320989 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/21242/statm: 97478 11224 464 189 0 91827 0
[pid=21242/tid=21244] ppid=21240 vsize=389912 CPUtime=1.27 cores=0-7
/proc/21242/task/21244/stat : 21244 (claspmt) R 21240 21242 20692 0 -1 4202560 1088 0 0 0 127 0 0 0 19 0 8 0 625976121 399269888 11224 33554432000 4194304 4964704 140736755562704 18446744073709551615 4660328 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=21242/tid=21245] ppid=21240 vsize=389912 CPUtime=1.23 cores=0-7
/proc/21242/task/21245/stat : 21245 (claspmt) R 21240 21242 20692 0 -1 4202560 1311 0 0 0 123 0 0 0 25 0 8 0 625976121 399269888 11225 33554432000 4194304 4964704 140736755562704 18446744073709551615 215988253729 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=21242/tid=21246] ppid=21240 vsize=389912 CPUtime=1.26 cores=0-7
/proc/21242/task/21246/stat : 21246 (claspmt) R 21240 21242 20692 0 -1 4202560 1067 0 0 0 126 0 0 0 19 0 8 0 625976121 399269888 11225 33554432000 4194304 4964704 140736755562704 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21242/tid=21247] ppid=21240 vsize=389912 CPUtime=1.27 cores=0-7
/proc/21242/task/21247/stat : 21247 (claspmt) R 21240 21242 20692 0 -1 4202560 1337 0 0 0 127 0 0 0 25 0 8 0 625976121 399269888 11226 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=21242/tid=21248] ppid=21240 vsize=389912 CPUtime=1.3 cores=0-7
/proc/21242/task/21248/stat : 21248 (claspmt) R 21240 21242 20692 0 -1 4202560 529 0 0 0 130 0 0 0 25 0 8 0 625976121 399269888 11226 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=21242/tid=21249] ppid=21240 vsize=389912 CPUtime=1.26 cores=0-7
/proc/21242/task/21249/stat : 21249 (claspmt) R 21240 21242 20692 0 -1 4202560 1442 0 0 0 126 0 0 0 19 0 8 0 625976121 399269888 11226 33554432000 4194304 4964704 140736755562704 18446744073709551615 4627443 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21242/tid=21250] ppid=21240 vsize=389912 CPUtime=1.26 cores=0-7
/proc/21242/task/21250/stat : 21250 (claspmt) R 21240 21242 20692 0 -1 4202560 1872 0 0 0 126 0 0 0 19 0 8 0 625976121 399269888 11226 33554432000 4194304 4964704 140736755562704 18446744073709551615 215988253729 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 10.38
Current children cumulated vsize (KiB) 389912

[startup+3.10061 s]
/proc/loadavg: 8.24 8.05 7.68 9/269 21250
/proc/meminfo: memFree=23553260/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=390884 CPUtime=23.17 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 13148 0 0 0 2312 5 0 0 24 0 8 0 625976103 400265216 12761 33554432000 4194304 4964704 140736755562704 18446744073709551615 4362240 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/21242/statm: 97721 12761 466 189 0 92070 0
[pid=21242/tid=21244] ppid=21240 vsize=390884 CPUtime=2.87 cores=0-7
/proc/21242/task/21244/stat : 21244 (claspmt) R 21240 21242 20692 0 -1 4202560 1274 0 0 0 287 0 0 0 22 0 8 0 625976121 400265216 12761 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=21242/tid=21245] ppid=21240 vsize=390884 CPUtime=2.83 cores=0-7
/proc/21242/task/21245/stat : 21245 (claspmt) R 21240 21242 20692 0 -1 4202560 1511 0 0 0 283 0 0 0 25 0 8 0 625976121 400265216 12761 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=21242/tid=21246] ppid=21240 vsize=390884 CPUtime=2.85 cores=0-7
/proc/21242/task/21246/stat : 21246 (claspmt) R 21240 21242 20692 0 -1 4202560 1148 0 0 0 285 0 0 0 23 0 8 0 625976121 400265216 12761 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670243 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21242/tid=21247] ppid=21240 vsize=390884 CPUtime=2.87 cores=0-7
/proc/21242/task/21247/stat : 21247 (claspmt) R 21240 21242 20692 0 -1 4202560 1520 0 0 0 287 0 0 0 25 0 8 0 625976121 400265216 12761 33554432000 4194304 4964704 140736755562704 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=21242/tid=21248] ppid=21240 vsize=390884 CPUtime=2.9 cores=0-7
/proc/21242/task/21248/stat : 21248 (claspmt) R 21240 21242 20692 0 -1 4202560 778 0 0 0 290 0 0 0 25 0 8 0 625976121 400265216 12761 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=21242/tid=21249] ppid=21240 vsize=390884 CPUtime=2.86 cores=0-7
/proc/21242/task/21249/stat : 21249 (claspmt) R 21240 21242 20692 0 -1 4202560 1615 0 0 0 286 0 0 0 22 0 8 0 625976121 400265216 12761 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670458 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21242/tid=21250] ppid=21240 vsize=390884 CPUtime=2.86 cores=0-7
/proc/21242/task/21250/stat : 21250 (claspmt) R 21240 21242 20692 0 -1 4202560 2229 0 0 0 286 0 0 0 24 0 8 0 625976121 400265216 12761 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 23.17
Current children cumulated vsize (KiB) 390884

[startup+6.30105 s]
/proc/loadavg: 8.22 8.05 7.68 9/269 21251
/proc/meminfo: memFree=23537148/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=393536 CPUtime=48.75 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 17372 0 0 0 4867 8 0 0 25 0 8 0 625976103 402980864 16985 33554432000 4194304 4964704 140736755562704 18446744073709551615 4627365 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/21242/statm: 98384 16985 466 189 0 92733 0
[pid=21242/tid=21244] ppid=21240 vsize=393536 CPUtime=6.06 cores=0-7
/proc/21242/task/21244/stat : 21244 (claspmt) R 21240 21242 20692 0 -1 4202560 1819 0 0 0 606 0 0 0 25 0 8 0 625976121 402980864 16985 33554432000 4194304 4964704 140736755562704 18446744073709551615 4627254 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=21242/tid=21245] ppid=21240 vsize=393536 CPUtime=6.04 cores=0-7
/proc/21242/task/21245/stat : 21245 (claspmt) R 21240 21242 20692 0 -1 4202560 2134 0 0 0 603 1 0 0 25 0 8 0 625976121 402980864 16985 33554432000 4194304 4964704 140736755562704 18446744073709551615 4627278 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=21242/tid=21246] ppid=21240 vsize=393536 CPUtime=6.02 cores=0-7
/proc/21242/task/21246/stat : 21246 (claspmt) R 21240 21242 20692 0 -1 4202560 1463 0 0 0 602 0 0 0 25 0 8 0 625976121 402980864 16985 33554432000 4194304 4964704 140736755562704 18446744073709551615 4320886 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21242/tid=21247] ppid=21240 vsize=393536 CPUtime=6.06 cores=0-7
/proc/21242/task/21247/stat : 21247 (claspmt) R 21240 21242 20692 0 -1 4202560 1918 0 0 0 606 0 0 0 25 0 8 0 625976121 402980864 16985 33554432000 4194304 4964704 140736755562704 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=21242/tid=21248] ppid=21240 vsize=393536 CPUtime=6.1 cores=0-7
/proc/21242/task/21248/stat : 21248 (claspmt) R 21240 21242 20692 0 -1 4202560 1255 0 0 0 610 0 0 0 25 0 8 0 625976121 402980864 16985 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=21242/tid=21249] ppid=21240 vsize=393536 CPUtime=6.07 cores=0-7
/proc/21242/task/21249/stat : 21249 (claspmt) R 21240 21242 20692 0 -1 4202560 1901 0 0 0 606 1 0 0 25 0 8 0 625976121 402980864 16985 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21242/tid=21250] ppid=21240 vsize=393536 CPUtime=6.07 cores=0-7
/proc/21242/task/21250/stat : 21250 (claspmt) R 21240 21242 20692 0 -1 4202560 3450 0 0 0 606 1 0 0 25 0 8 0 625976121 402980864 16985 33554432000 4194304 4964704 140736755562704 18446744073709551615 4627170 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 48.75
Current children cumulated vsize (KiB) 393536

[startup+12.7009 s]
/proc/loadavg: 8.20 8.05 7.68 9/269 21251
/proc/meminfo: memFree=23507444/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=398528 CPUtime=99.88 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 24516 0 0 0 9976 12 0 0 25 0 8 0 625976103 408092672 24129 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/21242/statm: 99632 24129 466 189 0 93981 0
[pid=21242/tid=21244] ppid=21240 vsize=398528 CPUtime=12.47 cores=0-7
/proc/21242/task/21244/stat : 21244 (claspmt) R 21240 21242 20692 0 -1 4202560 2784 0 0 0 1246 1 0 0 25 0 8 0 625976121 408092672 24129 33554432000 4194304 4964704 140736755562704 18446744073709551615 4627299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=21242/tid=21245] ppid=21240 vsize=398528 CPUtime=12.43 cores=0-7
/proc/21242/task/21245/stat : 21245 (claspmt) R 21240 21242 20692 0 -1 4202560 3074 0 0 0 1242 1 0 0 25 0 8 0 625976121 408092672 24129 33554432000 4194304 4964704 140736755562704 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=21242/tid=21246] ppid=21240 vsize=398528 CPUtime=12.39 cores=0-7
/proc/21242/task/21246/stat : 21246 (claspmt) R 21240 21242 20692 0 -1 4202560 2056 0 0 0 1239 0 0 0 25 0 8 0 625976121 408092672 24129 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21242/tid=21247] ppid=21240 vsize=398528 CPUtime=12.46 cores=0-7
/proc/21242/task/21247/stat : 21247 (claspmt) R 21240 21242 20692 0 -1 4202560 2876 0 0 0 1245 1 0 0 25 0 8 0 625976121 408092672 24129 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=21242/tid=21248] ppid=21240 vsize=398528 CPUtime=12.5 cores=0-7
/proc/21242/task/21248/stat : 21248 (claspmt) R 21240 21242 20692 0 -1 4202560 2059 0 0 0 1250 0 0 0 25 0 8 0 625976121 408092672 24129 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=21242/tid=21249] ppid=21240 vsize=398528 CPUtime=12.43 cores=0-7
/proc/21242/task/21249/stat : 21249 (claspmt) R 21240 21242 20692 0 -1 4202560 2511 0 0 0 1242 1 0 0 25 0 8 0 625976121 408092672 24129 33554432000 4194304 4964704 140736755562704 18446744073709551615 4653140 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21242/tid=21250] ppid=21240 vsize=398528 CPUtime=12.47 cores=0-7
/proc/21242/task/21250/stat : 21250 (claspmt) R 21240 21242 20692 0 -1 4202560 5254 0 0 0 1245 2 0 0 25 0 8 0 625976121 408092672 24129 33554432000 4194304 4964704 140736755562704 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 99.88
Current children cumulated vsize (KiB) 398528

[startup+25.5007 s]
/proc/loadavg: 8.16 8.05 7.68 9/269 21251
/proc/meminfo: memFree=23478336/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=404096 CPUtime=202.15 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 32301 0 0 0 20197 18 0 0 25 0 8 0 625976103 413794304 31913 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670248 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/21242/statm: 101024 31913 466 189 0 95373 0
[pid=21242/tid=21244] ppid=21240 vsize=404096 CPUtime=25.27 cores=0-7
/proc/21242/task/21244/stat : 21244 (claspmt) R 21240 21242 20692 0 -1 4202560 3824 0 0 0 2525 2 0 0 25 0 8 0 625976121 413794304 31913 33554432000 4194304 4964704 140736755562704 18446744073709551615 4669024 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=21242/tid=21245] ppid=21240 vsize=404096 CPUtime=25.23 cores=0-7
/proc/21242/task/21245/stat : 21245 (claspmt) R 21240 21242 20692 0 -1 4202560 4090 0 0 0 2521 2 0 0 25 0 8 0 625976121 413794304 31913 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=21242/tid=21246] ppid=21240 vsize=404096 CPUtime=25.19 cores=0-7
/proc/21242/task/21246/stat : 21246 (claspmt) R 21240 21242 20692 0 -1 4202560 2660 0 0 0 2518 1 0 0 25 0 8 0 625976121 413794304 31913 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21242/tid=21247] ppid=21240 vsize=404096 CPUtime=25.26 cores=0-7
/proc/21242/task/21247/stat : 21247 (claspmt) R 21240 21242 20692 0 -1 4202560 3647 0 0 0 2524 2 0 0 25 0 8 0 625976121 413794304 31913 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670431 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=21242/tid=21248] ppid=21240 vsize=404096 CPUtime=25.3 cores=0-7
/proc/21242/task/21248/stat : 21248 (claspmt) R 21240 21242 20692 0 -1 4202560 2728 0 0 0 2529 1 0 0 25 0 8 0 625976121 413794304 31913 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=21242/tid=21249] ppid=21240 vsize=404096 CPUtime=25.11 cores=0-7
/proc/21242/task/21249/stat : 21249 (claspmt) R 21240 21242 20692 0 -1 4202560 3091 0 0 0 2510 1 0 0 25 0 8 0 625976121 413794304 31913 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670243 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21242/tid=21250] ppid=21240 vsize=404096 CPUtime=25.26 cores=0-7
/proc/21242/task/21250/stat : 21250 (claspmt) R 21240 21242 20692 0 -1 4202560 7797 0 0 0 2522 4 0 0 25 0 8 0 625976121 413794304 31913 33554432000 4194304 4964704 140736755562704 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 202.15
Current children cumulated vsize (KiB) 404096

[startup+51.1012 s]
/proc/loadavg: 8.18 8.06 7.70 9/269 21252
/proc/meminfo: memFree=23413696/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=415736 CPUtime=406.71 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 48744 0 0 0 40640 31 0 0 25 0 8 0 625976103 425713664 47855 33554432000 4194304 4964704 140736755562704 18446744073709551615 4668942 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/21242/statm: 103934 47855 466 189 0 98283 0
[pid=21242/tid=21244] ppid=21240 vsize=415736 CPUtime=50.86 cores=0-7
/proc/21242/task/21244/stat : 21244 (claspmt) R 21240 21242 20692 0 -1 4202560 5287 0 0 0 5083 3 0 0 25 0 8 0 625976121 425713664 47855 33554432000 4194304 4964704 140736755562704 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=21242/tid=21245] ppid=21240 vsize=415736 CPUtime=50.83 cores=0-7
/proc/21242/task/21245/stat : 21245 (claspmt) R 21240 21242 20692 0 -1 4202560 5927 0 0 0 5080 3 0 0 25 0 8 0 625976121 425713664 47855 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=21242/tid=21246] ppid=21240 vsize=415736 CPUtime=50.79 cores=0-7
/proc/21242/task/21246/stat : 21246 (claspmt) R 21240 21242 20692 0 -1 4202560 3821 0 0 0 5077 2 0 0 25 0 8 0 625976121 425713664 47855 33554432000 4194304 4964704 140736755562704 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21242/tid=21247] ppid=21240 vsize=415736 CPUtime=50.85 cores=0-7
/proc/21242/task/21247/stat : 21247 (claspmt) R 21240 21242 20692 0 -1 4202560 4798 0 0 0 5083 2 0 0 25 0 8 0 625976121 425713664 47855 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=21242/tid=21248] ppid=21240 vsize=415736 CPUtime=50.89 cores=0-7
/proc/21242/task/21248/stat : 21248 (claspmt) R 21240 21242 20692 0 -1 4202560 3788 0 0 0 5088 1 0 0 25 0 8 0 625976121 425713664 47855 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=21242/tid=21249] ppid=21240 vsize=415736 CPUtime=50.51 cores=0-7
/proc/21242/task/21249/stat : 21249 (claspmt) R 21240 21242 20692 0 -1 4202560 4535 0 0 0 5048 3 0 0 25 0 8 0 625976121 425713664 47855 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21242/tid=21250] ppid=21240 vsize=415736 CPUtime=50.86 cores=0-7
/proc/21242/task/21250/stat : 21250 (claspmt) R 21240 21242 20692 0 -1 4202560 14951 0 0 0 5077 9 0 0 25 0 8 0 625976121 425713664 47855 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 406.71

################
# More data... #
################

/proc/21242/task/21249/stat : 21249 (claspmt) R 21240 21242 20692 0 -1 4202560 15605 0 0 0 464768 25 0 0 25 0 8 0 625976121 1215950848 211205 33554432000 4194304 4964704 140736755562704 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=21242/tid=21250] ppid=21240 vsize=1187452 CPUtime=4660.15 cores=0-7
/proc/21242/task/21250/stat : 21250 (claspmt) R 21240 21242 20692 0 -1 4202560 48876 0 0 0 465949 66 0 0 25 0 8 0 625976121 1215950848 211205 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37254.6
Current children cumulated vsize (KiB) 1187452

[startup+4722.31 s]
/proc/loadavg: 8.02 8.05 8.00 9/269 21391
/proc/meminfo: memFree=22754772/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=1188568 CPUtime=37734.1 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 225551 0 0 0 3773150 259 0 0 25 0 8 0 625976103 1217093632 211762 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670454 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/21242/statm: 297142 211762 466 189 0 291491 0
[pid=21242/tid=21244] ppid=21240 vsize=1188568 CPUtime=4698.28 cores=0-7
/proc/21242/task/21244/stat : 21244 (claspmt) R 21240 21242 20692 0 -1 4202560 36003 0 0 0 469795 33 0 0 25 0 8 0 625976121 1217093632 211762 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21242/tid=21245] ppid=21240 vsize=1188568 CPUtime=4721.29 cores=0-7
/proc/21242/task/21245/stat : 21245 (claspmt) R 21240 21242 20692 0 -1 4202560 41577 0 0 0 472107 22 0 0 25 0 8 0 625976121 1217093632 211762 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=21242/tid=21246] ppid=21240 vsize=1188568 CPUtime=4720.53 cores=0-7
/proc/21242/task/21246/stat : 21246 (claspmt) R 21240 21242 20692 0 -1 4202560 20064 0 0 0 472021 32 0 0 25 0 8 0 625976121 1217093632 211762 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21242/tid=21247] ppid=21240 vsize=1188568 CPUtime=4721.94 cores=0-7
/proc/21242/task/21247/stat : 21247 (claspmt) R 21240 21242 20692 0 -1 4202560 21718 0 0 0 472172 22 0 0 25 0 8 0 625976121 1217093632 211762 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=21242/tid=21248] ppid=21240 vsize=1188568 CPUtime=4721.99 cores=0-7
/proc/21242/task/21248/stat : 21248 (claspmt) R 21240 21242 20692 0 -1 4202560 22044 0 0 0 472174 25 0 0 25 0 8 0 625976121 1217093632 211762 33554432000 4194304 4964704 140736755562704 18446744073709551615 4321010 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=21242/tid=21249] ppid=21240 vsize=1188568 CPUtime=4707.94 cores=0-7
/proc/21242/task/21249/stat : 21249 (claspmt) R 21240 21242 20692 0 -1 4202560 15613 0 0 0 470769 25 0 0 25 0 8 0 625976121 1217093632 211762 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=21242/tid=21250] ppid=21240 vsize=1188568 CPUtime=4720.16 cores=0-7
/proc/21242/task/21250/stat : 21250 (claspmt) R 21240 21242 20692 0 -1 4202560 49022 0 0 0 471949 67 0 0 25 0 8 0 625976121 1217093632 211762 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37734.1
Current children cumulated vsize (KiB) 1188568

[startup+4782.3 s]
/proc/loadavg: 8.06 8.05 8.00 9/269 21393
/proc/meminfo: memFree=22753492/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=1188568 CPUtime=38213.5 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 225931 0 0 0 3821092 260 0 0 25 0 8 0 625976103 1217093632 212012 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670230 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/21242/statm: 297142 212012 466 189 0 291491 0
[pid=21242/tid=21244] ppid=21240 vsize=1188568 CPUtime=4757.78 cores=0-7
/proc/21242/task/21244/stat : 21244 (claspmt) R 21240 21242 20692 0 -1 4202560 36112 0 0 0 475745 33 0 0 25 0 8 0 625976121 1217093632 212012 33554432000 4194304 4964704 140736755562704 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21242/tid=21245] ppid=21240 vsize=1188568 CPUtime=4781.27 cores=0-7
/proc/21242/task/21245/stat : 21245 (claspmt) R 21240 21242 20692 0 -1 4202560 41716 0 0 0 478105 22 0 0 25 0 8 0 625976121 1217093632 212012 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=21242/tid=21246] ppid=21240 vsize=1188568 CPUtime=4780.52 cores=0-7
/proc/21242/task/21246/stat : 21246 (claspmt) R 21240 21242 20692 0 -1 4202560 20064 0 0 0 478020 32 0 0 25 0 8 0 625976121 1217093632 212012 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21242/tid=21247] ppid=21240 vsize=1188568 CPUtime=4781.93 cores=0-7
/proc/21242/task/21247/stat : 21247 (claspmt) R 21240 21242 20692 0 -1 4202560 21754 0 0 0 478171 22 0 0 25 0 8 0 625976121 1217093632 212012 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=21242/tid=21248] ppid=21240 vsize=1188568 CPUtime=4781.98 cores=0-7
/proc/21242/task/21248/stat : 21248 (claspmt) R 21240 21242 20692 0 -1 4202560 22060 0 0 0 478173 25 0 0 25 0 8 0 625976121 1217093632 212012 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=21242/tid=21249] ppid=21240 vsize=1188568 CPUtime=4767.93 cores=0-7
/proc/21242/task/21249/stat : 21249 (claspmt) R 21240 21242 20692 0 -1 4202560 15656 0 0 0 476768 25 0 0 25 0 8 0 625976121 1217093632 212012 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=21242/tid=21250] ppid=21240 vsize=1188568 CPUtime=4780.15 cores=0-7
/proc/21242/task/21250/stat : 21250 (claspmt) R 21240 21242 20692 0 -1 4202560 49059 0 0 0 477948 67 0 0 25 0 8 0 625976121 1217093632 212012 33554432000 4194304 4964704 140736755562704 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38213.5
Current children cumulated vsize (KiB) 1188568

[startup+4842.3 s]
/proc/loadavg: 8.02 8.04 8.00 9/269 21395
/proc/meminfo: memFree=22751680/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=1189720 CPUtime=38693 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 226533 0 0 0 3869037 261 0 0 25 0 8 0 625976103 1218273280 212505 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670243 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/21242/statm: 297430 212505 466 189 0 291779 0
[pid=21242/tid=21244] ppid=21240 vsize=1189720 CPUtime=4817.28 cores=0-7
/proc/21242/task/21244/stat : 21244 (claspmt) R 21240 21242 20692 0 -1 4202560 36458 0 0 0 481695 33 0 0 25 0 8 0 625976121 1218273280 212505 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21242/tid=21245] ppid=21240 vsize=1189720 CPUtime=4841.27 cores=0-7
/proc/21242/task/21245/stat : 21245 (claspmt) R 21240 21242 20692 0 -1 4202560 41804 0 0 0 484104 23 0 0 25 0 8 0 625976121 1218273280 212505 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=21242/tid=21246] ppid=21240 vsize=1189720 CPUtime=4840.52 cores=0-7
/proc/21242/task/21246/stat : 21246 (claspmt) R 21240 21242 20692 0 -1 4202560 20065 0 0 0 484020 32 0 0 25 0 8 0 625976121 1218273280 212505 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21242/tid=21247] ppid=21240 vsize=1189720 CPUtime=4841.93 cores=0-7
/proc/21242/task/21247/stat : 21247 (claspmt) R 21240 21242 20692 0 -1 4202560 21783 0 0 0 484171 22 0 0 25 0 8 0 625976121 1218273280 212505 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=21242/tid=21248] ppid=21240 vsize=1189720 CPUtime=4841.98 cores=0-7
/proc/21242/task/21248/stat : 21248 (claspmt) R 21240 21242 20692 0 -1 4202560 22077 0 0 0 484173 25 0 0 25 0 8 0 625976121 1218273280 212505 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=21242/tid=21249] ppid=21240 vsize=1189720 CPUtime=4827.92 cores=0-7
/proc/21242/task/21249/stat : 21249 (claspmt) R 21240 21242 20692 0 -1 4202560 15726 0 0 0 482767 25 0 0 25 0 8 0 625976121 1218273280 212505 33554432000 4194304 4964704 140736755562704 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=21242/tid=21250] ppid=21240 vsize=1189720 CPUtime=4840.15 cores=0-7
/proc/21242/task/21250/stat : 21250 (claspmt) R 21240 21242 20692 0 -1 4202560 49110 0 0 0 483948 67 0 0 25 0 8 0 625976121 1218273280 212505 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38693
Current children cumulated vsize (KiB) 1189720

[startup+4902.3 s]
/proc/loadavg: 8.05 8.05 8.00 9/269 21396
/proc/meminfo: memFree=22746796/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=1190516 CPUtime=39172.5 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 228011 0 0 0 3916982 265 0 0 25 0 8 0 625976103 1219088384 213711 33554432000 4194304 4964704 140736755562704 18446744073709551615 4320951 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/21242/statm: 297629 213711 466 189 0 291978 0
[pid=21242/tid=21244] ppid=21240 vsize=1190516 CPUtime=4876.78 cores=0-7
/proc/21242/task/21244/stat : 21244 (claspmt) R 21240 21242 20692 0 -1 4202560 37105 0 0 0 487644 34 0 0 25 0 8 0 625976121 1219088384 213711 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21242/tid=21245] ppid=21240 vsize=1190516 CPUtime=4901.25 cores=0-7
/proc/21242/task/21245/stat : 21245 (claspmt) R 21240 21242 20692 0 -1 4202560 42157 0 0 0 490102 23 0 0 25 0 8 0 625976121 1219088384 213711 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=21242/tid=21246] ppid=21240 vsize=1190516 CPUtime=4900.52 cores=0-7
/proc/21242/task/21246/stat : 21246 (claspmt) R 21240 21242 20692 0 -1 4202560 20264 0 0 0 490019 33 0 0 25 0 8 0 625976121 1219088384 213711 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21242/tid=21247] ppid=21240 vsize=1190516 CPUtime=4901.92 cores=0-7
/proc/21242/task/21247/stat : 21247 (claspmt) R 21240 21242 20692 0 -1 4202560 21886 0 0 0 490170 22 0 0 25 0 8 0 625976121 1219088384 213711 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670230 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=21242/tid=21248] ppid=21240 vsize=1190516 CPUtime=4901.98 cores=0-7
/proc/21242/task/21248/stat : 21248 (claspmt) R 21240 21242 20692 0 -1 4202560 22172 0 0 0 490173 25 0 0 25 0 8 0 625976121 1219088384 213711 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670193 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=21242/tid=21249] ppid=21240 vsize=1190516 CPUtime=4887.93 cores=0-7
/proc/21242/task/21249/stat : 21249 (claspmt) R 21240 21242 20692 0 -1 4202560 15728 0 0 0 488767 26 0 0 25 0 8 0 625976121 1219088384 213711 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=21242/tid=21250] ppid=21240 vsize=1190516 CPUtime=4900.16 cores=0-7
/proc/21242/task/21250/stat : 21250 (claspmt) R 21240 21242 20692 0 -1 4202560 49142 0 0 0 489948 68 0 0 25 0 8 0 625976121 1219088384 213711 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39172.5
Current children cumulated vsize (KiB) 1190516

[startup+4962.3 s]
/proc/loadavg: 8.02 8.04 8.00 9/269 21398
/proc/meminfo: memFree=22738456/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=1196176 CPUtime=39651.9 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 230545 0 0 0 3964927 267 0 0 25 0 8 0 625976103 1224884224 215811 33554432000 4194304 4964704 140736755562704 18446744073709551615 4332343 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/21242/statm: 299044 215811 466 189 0 293393 0
[pid=21242/tid=21244] ppid=21240 vsize=1196176 CPUtime=4936.28 cores=0-7
/proc/21242/task/21244/stat : 21244 (claspmt) R 21240 21242 20692 0 -1 4202560 37259 0 0 0 493593 35 0 0 25 0 8 0 625976121 1224884224 215811 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21242/tid=21245] ppid=21240 vsize=1196176 CPUtime=4961.24 cores=0-7
/proc/21242/task/21245/stat : 21245 (claspmt) R 21240 21242 20692 0 -1 4202560 42702 0 0 0 496101 23 0 0 25 0 8 0 625976121 1224884224 215811 33554432000 4194304 4964704 140736755562704 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=21242/tid=21246] ppid=21240 vsize=1196176 CPUtime=4960.51 cores=0-7
/proc/21242/task/21246/stat : 21246 (claspmt) R 21240 21242 20692 0 -1 4202560 20590 0 0 0 496018 33 0 0 25 0 8 0 625976121 1224884224 215811 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21242/tid=21247] ppid=21240 vsize=1196176 CPUtime=4961.92 cores=0-7
/proc/21242/task/21247/stat : 21247 (claspmt) R 21240 21242 20692 0 -1 4202560 21976 0 0 0 496170 22 0 0 25 0 8 0 625976121 1224884224 215811 33554432000 4194304 4964704 140736755562704 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=21242/tid=21248] ppid=21240 vsize=1196176 CPUtime=4961.97 cores=0-7
/proc/21242/task/21248/stat : 21248 (claspmt) R 21240 21242 20692 0 -1 4202560 22271 0 0 0 496172 25 0 0 25 0 8 0 625976121 1224884224 215811 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=21242/tid=21249] ppid=21240 vsize=1196176 CPUtime=4947.93 cores=0-7
/proc/21242/task/21249/stat : 21249 (claspmt) R 21240 21242 20692 0 -1 4202560 15841 0 0 0 494767 26 0 0 25 0 8 0 625976121 1224884224 215811 33554432000 4194304 4964704 140736755562704 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=21242/tid=21250] ppid=21240 vsize=1196176 CPUtime=4960.15 cores=0-7
/proc/21242/task/21250/stat : 21250 (claspmt) R 21240 21242 20692 0 -1 4202560 50222 0 0 0 495946 69 0 0 25 0 8 0 625976121 1224884224 215811 33554432000 4194304 4964704 140736755562704 18446744073709551615 4321029 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39651.9
Current children cumulated vsize (KiB) 1196176



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5005.9 s]
/proc/loadavg: 8.06 8.04 8.00 9/269 21399
/proc/meminfo: memFree=22737348/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=1196584 CPUtime=40000.4 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 230941 0 0 0 3999775 268 0 0 25 0 8 0 625976103 1225302016 216084 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/21242/statm: 299146 216084 466 189 0 293495 0
[pid=21242/tid=21244] ppid=21240 vsize=1196584 CPUtime=4979.53 cores=0-7
/proc/21242/task/21244/stat : 21244 (claspmt) R 21240 21242 20692 0 -1 4202560 37413 0 0 0 497918 35 0 0 25 0 8 0 625976121 1225302016 216084 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21242/tid=21245] ppid=21240 vsize=1196584 CPUtime=5004.84 cores=0-7
/proc/21242/task/21245/stat : 21245 (claspmt) R 21240 21242 20692 0 -1 4202560 42793 0 0 0 500461 23 0 0 25 0 8 0 625976121 1225302016 216084 33554432000 4194304 4964704 140736755562704 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=21242/tid=21246] ppid=21240 vsize=1196584 CPUtime=5004.12 cores=0-7
/proc/21242/task/21246/stat : 21246 (claspmt) R 21240 21242 20692 0 -1 4202560 20590 0 0 0 500379 33 0 0 25 0 8 0 625976121 1225302016 216084 33554432000 4194304 4964704 140736755562704 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21242/tid=21247] ppid=21240 vsize=1196584 CPUtime=5005.52 cores=0-7
/proc/21242/task/21247/stat : 21247 (claspmt) R 21240 21242 20692 0 -1 4202560 21976 0 0 0 500530 22 0 0 25 0 8 0 625976121 1225302016 216084 33554432000 4194304 4964704 140736755562704 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=21242/tid=21248] ppid=21240 vsize=1196584 CPUtime=5005.58 cores=0-7
/proc/21242/task/21248/stat : 21248 (claspmt) R 21240 21242 20692 0 -1 4202560 22319 0 0 0 500533 25 0 0 25 0 8 0 625976121 1225302016 216084 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=21242/tid=21249] ppid=21240 vsize=1196584 CPUtime=4991.54 cores=0-7
/proc/21242/task/21249/stat : 21249 (claspmt) R 21240 21242 20692 0 -1 4202560 15938 0 0 0 499128 26 0 0 25 0 8 0 625976121 1225302016 216084 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=21242/tid=21250] ppid=21240 vsize=1196584 CPUtime=5003.76 cores=0-7
/proc/21242/task/21250/stat : 21250 (claspmt) R 21240 21242 20692 0 -1 4202560 50226 0 0 0 500307 69 0 0 25 0 8 0 625976121 1225302016 216084 33554432000 4194304 4964704 140736755562704 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 1196584

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5006.02 s]
/proc/loadavg: 8.06 8.04 8.00 2/263 21400
/proc/meminfo: memFree=22738636/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=1194960 CPUtime=40000.6 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 230951 0 0 0 3999791 268 0 0 25 0 1 0 625976103 1223639040 215630 33554432000 4194304 4964704 140736755562704 18446744073709551615 4646575 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/21242/statm: 298740 215630 476 189 0 293089 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 1194960

[startup+5006.11 s]
/proc/loadavg: 8.06 8.04 8.00 2/263 21400
/proc/meminfo: memFree=22738636/32950940 swapFree=67111364/67111528
[pid=21242] ppid=21240 vsize=1190268 CPUtime=40000.7 cores=0-7
/proc/21242/stat : 21242 (claspmt) R 21240 21242 20692 0 -1 4202496 230951 0 0 0 3999799 268 0 0 25 0 1 0 625976103 1218834432 214215 33554432000 4194304 4964704 140736755562704 18446744073709551615 215988247642 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/21242/statm: 297567 214215 476 189 0 291916 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 1190268

Child status: 0
Real time (s): 5006.17
CPU time (s): 40000.7
CPU user time (s): 39998
CPU system time (s): 2.72658
CPU usage (%): 799.029
Max. virtual memory (cumulated for all children) (KiB): 1196584

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39998
system time used= 2.72659
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 230952
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 92796
involuntary context switches= 190445

runsolver used 11.1773 second user time and 29.7985 second system time

The end

Launcher Data 

Begin job on node117 at 2011-05-02 19:30:53
IDJOB=3321511
IDBENCH=71570
IDSOLVER=1711
FILE ID=node117/3321511-1304357453
RUNJOBID= node117-1304349834-20706
PBS_JOBID= 13178674
Free space on /tmp= 72672 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/forced/rbsat-v945c61409gyes10.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321511-1304357453/watcher-3321511-1304357453 -o /tmp/evaluation-result-3321511-1304357453/solver-3321511-1304357453 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321511-1304357453.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 601a386973128161e9173e2bdf583672
RANDOM SEED=1361377496

node117.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5267.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950940 kB
MemFree:      23598468 kB
Buffers:       1019924 kB
Cached:        4465968 kB
SwapCached:        160 kB
Active:        3781672 kB
Inactive:      5077284 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950940 kB
LowFree:      23598468 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:            1600 kB
Writeback:           0 kB
AnonPages:     3373720 kB
Mapped:          23436 kB
Slab:           408588 kB
PageTables:      19596 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586996 kB
Committed_AS: 37497996 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72672 MiB
End job on node117 at 2011-05-02 20:54:21