Trace number 3321383

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40001.4 5005.08

General information on the benchmark

NameAPPLICATIONS/bitverif/
smulo/smulo064.cnf
MD5SUM472e9d10d7fd1a82d06da0809d0002df
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables47129
Number of clauses141002
Sum of the clauses size329002
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 294000
Number of clauses of size 347000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...337727/instance-3321383-1304337727.cnf
0.29/0.30	c Solving...
40000.57/5004.31	
40000.57/5004.31	*** INTERRUPTED! ***
40000.57/5004.31	*** Shutting down threads...
40001.32/5005.07	s UNKNOWN
40001.32/5005.07	
40001.32/5005.07	c INTERRUPTED   : 1
40001.32/5005.07	c Models        : 0+    
40001.32/5005.07	c Time          : 5004.314s (Solving: 5004.01s 1st Model: 0.00s Unsat: 0.00s)
40001.32/5005.07	c CPU Time      : 40000.640s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321383-1304337727/watcher-3321383-1304337727 -o /tmp/evaluation-result-3321383-1304337727/solver-3321383-1304337727 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321383-1304337727.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.45 7.91 7.94 1/186 8567
/proc/meminfo: memFree=25086136/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=26924 CPUtime=0 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 1409 0 0 0 0 0 0 0 25 0 1 0 545372360 27570176 1338 33554432000 4194304 4964704 140737107311472 18446744073709551615 4700332 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/8567/statm: 6731 1346 376 189 0 1605 0

[startup+0.0682531 s]
/proc/loadavg: 7.45 7.91 7.94 1/186 8567
/proc/meminfo: memFree=25086136/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=41896 CPUtime=0.06 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 7109 0 0 0 6 0 0 0 25 0 1 0 545372360 42901504 5466 33554432000 4194304 4964704 140737107311472 18446744073709551615 208079929463 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/8567/statm: 10474 5476 399 189 0 5348 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 41896

[startup+0.100245 s]
/proc/loadavg: 7.45 7.91 7.94 1/186 8567
/proc/meminfo: memFree=25086136/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=44704 CPUtime=0.09 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 7760 0 0 0 9 0 0 0 25 0 1 0 545372360 45776896 6117 33554432000 4194304 4964704 140737107311472 18446744073709551615 4615610 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/8567/statm: 11176 6117 404 189 0 6050 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 44704

[startup+0.300208 s]
/proc/loadavg: 7.45 7.91 7.94 1/186 8567
/proc/meminfo: memFree=25086136/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=49324 CPUtime=0.29 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 8939 0 0 0 29 0 0 0 25 0 1 0 545372360 50507776 7296 33554432000 4194304 4964704 140737107311472 18446744073709551615 208079890194 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/8567/statm: 12331 7296 406 189 0 7205 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 49324

[startup+1.56897 s]
/proc/loadavg: 7.45 7.91 7.94 1/186 8567
/proc/meminfo: memFree=25086136/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=425220 CPUtime=9.94 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 25697 0 0 0 987 7 0 0 25 0 8 0 545372360 435425280 20732 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668907 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/8567/statm: 106305 20732 464 189 0 100654 0
Current children cumulated CPU time (s) 9.94
Current children cumulated vsize (KiB) 425220

[startup+1.66895 s]
/proc/loadavg: 7.45 7.91 7.94 1/186 8567
/proc/meminfo: memFree=25086136/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=425484 CPUtime=10.74 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 25960 0 0 0 1067 7 0 0 25 0 8 0 545372360 435695616 20995 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668673 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/8567/statm: 106371 20996 464 189 0 100720 0
Current children cumulated CPU time (s) 10.74
Current children cumulated vsize (KiB) 425484

[startup+3.10068 s]
/proc/loadavg: 7.66 7.94 7.95 9/194 8575
/proc/meminfo: memFree=24995216/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=427524 CPUtime=22.23 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 28422 0 0 0 2214 9 0 0 25 0 8 0 545372360 437784576 23457 33554432000 4194304 4964704 140737107311472 18446744073709551615 4642509 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/8567/statm: 106881 23457 464 189 0 101230 0
[pid=8567/tid=8569] ppid=8565 vsize=427524 CPUtime=2.73 cores=0-7
/proc/8567/task/8569/stat : 8569 (claspmt) R 8565 8567 8541 0 -1 4202560 2100 0 0 0 273 0 0 0 21 0 8 0 545372390 437784576 23458 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=8567/tid=8570] ppid=8565 vsize=427524 CPUtime=2.73 cores=0-7
/proc/8567/task/8570/stat : 8570 (claspmt) R 8565 8567 8541 0 -1 4202560 2842 0 0 0 273 0 0 0 22 0 8 0 545372390 437784576 23458 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=8567/tid=8571] ppid=8565 vsize=427524 CPUtime=2.72 cores=0-7
/proc/8567/task/8571/stat : 8571 (claspmt) R 8565 8567 8541 0 -1 4202560 2703 0 0 0 271 1 0 0 22 0 8 0 545372390 437784576 23458 33554432000 4194304 4964704 140737107311472 18446744073709551615 4632224 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=8567/tid=8572] ppid=8565 vsize=427524 CPUtime=2.73 cores=0-7
/proc/8567/task/8572/stat : 8572 (claspmt) R 8565 8567 8541 0 -1 4202560 2300 0 0 0 273 0 0 0 21 0 8 0 545372390 437784576 23458 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=8567/tid=8573] ppid=8565 vsize=427524 CPUtime=2.77 cores=0-7
/proc/8567/task/8573/stat : 8573 (claspmt) R 8565 8567 8541 0 -1 4202560 2118 0 0 0 277 0 0 0 23 0 8 0 545372390 437784576 23458 33554432000 4194304 4964704 140737107311472 18446744073709551615 4669192 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=8567/tid=8574] ppid=8565 vsize=427524 CPUtime=2.73 cores=0-7
/proc/8567/task/8574/stat : 8574 (claspmt) R 8565 8567 8541 0 -1 4202560 2750 0 0 0 272 1 0 0 22 0 8 0 545372390 437784576 23458 33554432000 4194304 4964704 140737107311472 18446744073709551615 208080337439 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=8567/tid=8575] ppid=8565 vsize=427524 CPUtime=2.73 cores=0-7
/proc/8567/task/8575/stat : 8575 (claspmt) R 8565 8567 8541 0 -1 4202560 3474 0 0 0 272 1 0 0 22 0 8 0 545372390 437784576 23458 33554432000 4194304 4964704 140737107311472 18446744073709551615 208079893537 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 22.23
Current children cumulated vsize (KiB) 427524

[startup+6.30107 s]
/proc/loadavg: 7.66 7.94 7.95 9/194 8575
/proc/meminfo: memFree=24978112/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=430380 CPUtime=47.77 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 34119 0 0 0 4764 13 0 0 25 0 8 0 545372360 440709120 29154 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/8567/statm: 107595 29154 464 189 0 101944 0
[pid=8567/tid=8569] ppid=8565 vsize=430380 CPUtime=5.92 cores=0-7
/proc/8567/task/8569/stat : 8569 (claspmt) R 8565 8567 8541 0 -1 4202560 2301 0 0 0 592 0 0 0 25 0 8 0 545372390 440709120 29154 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670281 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=8567/tid=8570] ppid=8565 vsize=430380 CPUtime=5.93 cores=0-7
/proc/8567/task/8570/stat : 8570 (claspmt) R 8565 8567 8541 0 -1 4202560 3853 0 0 0 592 1 0 0 25 0 8 0 545372390 440709120 29154 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=8567/tid=8571] ppid=8565 vsize=430380 CPUtime=5.9 cores=0-7
/proc/8567/task/8571/stat : 8571 (claspmt) R 8565 8567 8541 0 -1 4202560 3274 0 0 0 589 1 0 0 25 0 8 0 545372390 440709120 29154 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670281 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=8567/tid=8572] ppid=8565 vsize=430380 CPUtime=5.92 cores=0-7
/proc/8567/task/8572/stat : 8572 (claspmt) R 8565 8567 8541 0 -1 4202560 3098 0 0 0 591 1 0 0 25 0 8 0 545372390 440709120 29154 33554432000 4194304 4964704 140737107311472 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=8567/tid=8573] ppid=8565 vsize=430380 CPUtime=5.96 cores=0-7
/proc/8567/task/8573/stat : 8573 (claspmt) R 8565 8567 8541 0 -1 4202560 2162 0 0 0 596 0 0 0 25 0 8 0 545372390 440709120 29154 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668866 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=8567/tid=8574] ppid=8565 vsize=430380 CPUtime=5.94 cores=0-7
/proc/8567/task/8574/stat : 8574 (claspmt) R 8565 8567 8541 0 -1 4202560 3407 0 0 0 592 2 0 0 25 0 8 0 545372390 440709120 29154 33554432000 4194304 4964704 140737107311472 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=8567/tid=8575] ppid=8565 vsize=430380 CPUtime=5.93 cores=0-7
/proc/8567/task/8575/stat : 8575 (claspmt) R 8565 8567 8541 0 -1 4202560 5445 0 0 0 590 3 0 0 25 0 8 0 545372390 440709120 29154 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668844 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 47.77
Current children cumulated vsize (KiB) 430380

[startup+12.7009 s]
/proc/loadavg: 7.71 7.94 7.95 9/194 8575
/proc/meminfo: memFree=24948448/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=433068 CPUtime=98.93 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 41966 0 0 0 9873 20 0 0 25 0 8 0 545372360 443461632 37001 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/8567/statm: 108267 37002 464 189 0 102616 0
[pid=8567/tid=8569] ppid=8565 vsize=433068 CPUtime=12.32 cores=0-7
/proc/8567/task/8569/stat : 8569 (claspmt) R 8565 8567 8541 0 -1 4202560 2724 0 0 0 1232 0 0 0 20 0 8 0 545372390 443461632 37002 33554432000 4194304 4964704 140737107311472 18446744073709551615 4342232 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=8567/tid=8570] ppid=8565 vsize=433068 CPUtime=12.32 cores=0-7
/proc/8567/task/8570/stat : 8570 (claspmt) R 8565 8567 8541 0 -1 4202560 5334 0 0 0 1230 2 0 0 25 0 8 0 545372390 443461632 37002 33554432000 4194304 4964704 140737107311472 18446744073709551615 208079893537 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=8567/tid=8571] ppid=8565 vsize=433068 CPUtime=12.26 cores=0-7
/proc/8567/task/8571/stat : 8571 (claspmt) R 8565 8567 8541 0 -1 4202560 4122 0 0 0 1224 2 0 0 25 0 8 0 545372390 443461632 37002 33554432000 4194304 4964704 140737107311472 18446744073709551615 4653394 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=8567/tid=8572] ppid=8565 vsize=433068 CPUtime=12.33 cores=0-7
/proc/8567/task/8572/stat : 8572 (claspmt) R 8565 8567 8541 0 -1 4202560 3560 0 0 0 1231 2 0 0 25 0 8 0 545372390 443461632 37002 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668709 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=8567/tid=8573] ppid=8565 vsize=433068 CPUtime=12.36 cores=0-7
/proc/8567/task/8573/stat : 8573 (claspmt) R 8565 8567 8541 0 -1 4202560 2206 0 0 0 1236 0 0 0 25 0 8 0 545372390 443461632 37002 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670100 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=8567/tid=8574] ppid=8565 vsize=433068 CPUtime=12.32 cores=0-7
/proc/8567/task/8574/stat : 8574 (claspmt) R 8565 8567 8541 0 -1 4202560 4120 0 0 0 1230 2 0 0 25 0 8 0 545372390 443461632 37002 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670100 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=8567/tid=8575] ppid=8565 vsize=433068 CPUtime=12.32 cores=0-7
/proc/8567/task/8575/stat : 8575 (claspmt) R 8565 8567 8541 0 -1 4202560 8953 0 0 0 1227 5 0 0 25 0 8 0 545372390 443461632 37002 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668618 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 98.93
Current children cumulated vsize (KiB) 433068

[startup+25.5004 s]
/proc/loadavg: 7.75 7.94 7.95 9/194 8576
/proc/meminfo: memFree=24914152/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=437716 CPUtime=201.25 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 49497 0 0 0 20099 26 0 0 25 0 8 0 545372360 448221184 44532 33554432000 4194304 4964704 140737107311472 18446744073709551615 4660223 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/8567/statm: 109429 44532 466 189 0 103778 0
[pid=8567/tid=8569] ppid=8565 vsize=437716 CPUtime=25.11 cores=0-7
/proc/8567/task/8569/stat : 8569 (claspmt) R 8565 8567 8541 0 -1 4202560 2867 0 0 0 2511 0 0 0 25 0 8 0 545372390 448221184 44532 33554432000 4194304 4964704 140737107311472 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=8567/tid=8570] ppid=8565 vsize=437716 CPUtime=25.12 cores=0-7
/proc/8567/task/8570/stat : 8570 (claspmt) R 8565 8567 8541 0 -1 4202560 6449 0 0 0 2509 3 0 0 25 0 8 0 545372390 448221184 44532 33554432000 4194304 4964704 140737107311472 18446744073709551615 4362958 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=8567/tid=8571] ppid=8565 vsize=437716 CPUtime=24.99 cores=0-7
/proc/8567/task/8571/stat : 8571 (claspmt) R 8565 8567 8541 0 -1 4202560 4545 0 0 0 2497 2 0 0 25 0 8 0 545372390 448221184 44532 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=8567/tid=8572] ppid=8565 vsize=437716 CPUtime=25.12 cores=0-7
/proc/8567/task/8572/stat : 8572 (claspmt) R 8565 8567 8541 0 -1 4202560 5707 0 0 0 2509 3 0 0 25 0 8 0 545372390 448221184 44532 33554432000 4194304 4964704 140737107311472 18446744073709551615 4669044 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=8567/tid=8573] ppid=8565 vsize=437716 CPUtime=25.16 cores=0-7
/proc/8567/task/8573/stat : 8573 (claspmt) R 8565 8567 8541 0 -1 4202560 2238 0 0 0 2516 0 0 0 25 0 8 0 545372390 448221184 44532 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668907 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=8567/tid=8574] ppid=8565 vsize=437716 CPUtime=25.13 cores=0-7
/proc/8567/task/8574/stat : 8574 (claspmt) R 8565 8567 8541 0 -1 4202560 4530 0 0 0 2510 3 0 0 25 0 8 0 545372390 448221184 44532 33554432000 4194304 4964704 140737107311472 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=8567/tid=8575] ppid=8565 vsize=437716 CPUtime=25.13 cores=0-7
/proc/8567/task/8575/stat : 8575 (claspmt) R 8565 8567 8541 0 -1 4202560 11776 0 0 0 2505 8 0 0 25 0 8 0 545372390 448221184 44532 33554432000 4194304 4964704 140737107311472 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 201.25
Current children cumulated vsize (KiB) 437716

[startup+51.1005 s]
/proc/loadavg: 7.89 7.96 7.96 9/194 8576
/proc/meminfo: memFree=24825828/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=455880 CPUtime=405.89 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 71543 0 0 0 40548 41 0 0 25 0 8 0 545372360 466821120 66578 33554432000 4194304 4964704 140737107311472 18446744073709551615 4684100 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/8567/statm: 113970 66578 466 189 0 108319 0
[pid=8567/tid=8569] ppid=8565 vsize=455880 CPUtime=50.71 cores=0-7
/proc/8567/task/8569/stat : 8569 (claspmt) R 8565 8567 8541 0 -1 4202560 3454 0 0 0 5070 1 0 0 25 0 8 0 545372390 466821120 66578 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=8567/tid=8570] ppid=8565 vsize=455880 CPUtime=50.73 cores=0-7
/proc/8567/task/8570/stat : 8570 (claspmt) R 8565 8567 8541 0 -1 4202560 9390 0 0 0 5068 5 0 0 25 0 8 0 545372390 466821120 66578 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=8567/tid=8571] ppid=8565 vsize=455880 CPUtime=50.45 cores=0-7
/proc/8567/task/8571/stat : 8571 (claspmt) R 8565 8567 8541 0 -1 4202560 5569 0 0 0 5042 3 0 0 25 0 8 0 545372390 466821120 66578 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=8567/tid=8572] ppid=8565 vsize=455880 CPUtime=50.72 cores=0-7
/proc/8567/task/8572/stat : 8572 (claspmt) R 8565 8567 8541 0 -1 4202560 10088 0 0 0 5066 6 0 0 25 0 8 0 545372390 466821120 66578 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=8567/tid=8573] ppid=8565 vsize=455880 CPUtime=50.77 cores=0-7
/proc/8567/task/8573/stat : 8573 (claspmt) R 8565 8567 8541 0 -1 4202560 2902 0 0 0 5076 1 0 0 25 0 8 0 545372390 466821120 66578 33554432000 4194304 4964704 140737107311472 18446744073709551615 4465569 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=8567/tid=8574] ppid=8565 vsize=455880 CPUtime=50.72 cores=0-7
/proc/8567/task/8574/stat : 8574 (claspmt) R 8565 8567 8541 0 -1 4202560 5735 0 0 0 5068 4 0 0 25 0 8 0 545372390 466821120 66578 33554432000 4194304 4964704 140737107311472 18446744073709551615 4320879 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=8567/tid=8575] ppid=8565 vsize=455880 CPUtime=50.72 cores=0-7
/proc/8567/task/8575/stat : 8575 (claspmt) R 8565 8567 8541 0 -1 4202560 22387 0 0 0 5056 16 0 0 25 0 8 0 545372390 466821120 66578 33554432000 4194304 4964704 140737107311472 18446744073709551615 4360553 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 405.89
Current children cumulated vsize (KiB) 455880

[startup+102.301 s]
/proc/loadavg: 7.95 7.97 7.96 9/194 8578
/proc/meminfo: memFree=24652836/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=681384 CPUtime=815.19 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 114561 0 0 0 81450 69 0 0 25 0 8 0 545372360 697737216 109419 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668668 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/8567/statm: 170346 109419 466 189 0 164695 0
[pid=8567/tid=8569] ppid=8565 vsize=681384 CPUtime=101.9 cores=0-7
/proc/8567/task/8569/stat : 8569 (claspmt) R 8565 8567 8541 0 -1 4202560 3809 0 0 0 10189 1 0 0 25 0 8 0 545372390 697737216 109419 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=8567/tid=8570] ppid=8565 vsize=681384 CPUtime=101.92 cores=0-7
/proc/8567/task/8570/stat : 8570 (claspmt) R 8565 8567 8541 0 -1 4202560 11606 0 0 0 10186 6 0 0 25 0 8 0 545372390 697737216 109419 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670184 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=8567/tid=8571] ppid=8565 vsize=681384 CPUtime=101.36 cores=0-7
/proc/8567/task/8571/stat : 8571 (claspmt) R 8565 8567 8541 0 -1 4202560 5977 0 0 0 10133 3 0 0 25 0 8 0 545372390 697737216 109419 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 7 0 0 0

################
# More data... #
################

/proc/8567/task/8574/stat : 8574 (claspmt) R 8565 8567 8541 0 -1 4202560 12479 0 0 0 466154 21 0 0 25 0 8 0 545372390 2677739520 580947 33554432000 4194304 4964704 140737107311472 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=8567/tid=8575] ppid=8565 vsize=2614980 CPUtime=4660.99 cores=0-7
/proc/8567/task/8575/stat : 8575 (claspmt) R 8565 8567 8541 0 -1 4202560 362513 0 0 0 465820 279 0 0 25 0 8 0 545372390 2677739520 580947 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 37266.6
Current children cumulated vsize (KiB) 2614980

[startup+4722.3 s]
/proc/loadavg: 8.00 8.00 8.00 9/194 8716
/proc/meminfo: memFree=22753652/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=2614980 CPUtime=37746.2 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 592443 0 0 0 3774073 546 0 0 25 0 8 0 545372360 2677739520 582906 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/8567/statm: 653745 582906 466 189 0 648094 0
[pid=8567/tid=8569] ppid=8565 vsize=2614980 CPUtime=4721.05 cores=0-7
/proc/8567/task/8569/stat : 8569 (claspmt) R 8565 8567 8541 0 -1 4202560 10326 0 0 0 472092 13 0 0 25 0 8 0 545372390 2677739520 582906 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670271 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=8567/tid=8570] ppid=8565 vsize=2614980 CPUtime=4721.8 cores=0-7
/proc/8567/task/8570/stat : 8570 (claspmt) R 8565 8567 8541 0 -1 4202560 51745 0 0 0 472129 51 0 0 25 0 8 0 545372390 2677739520 582906 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=8567/tid=8571] ppid=8565 vsize=2614980 CPUtime=4702.03 cores=0-7
/proc/8567/task/8571/stat : 8571 (claspmt) R 8565 8567 8541 0 -1 4202560 15519 0 0 0 470181 22 0 0 25 0 8 0 545372390 2677739520 582906 33554432000 4194304 4964704 140737107311472 18446744073709551615 4342208 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=8567/tid=8572] ppid=8565 vsize=2614980 CPUtime=4718.73 cores=0-7
/proc/8567/task/8572/stat : 8572 (claspmt) R 8565 8567 8541 0 -1 4202560 43971 0 0 0 471831 42 0 0 25 0 8 0 545372390 2677739520 582906 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668907 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=8567/tid=8573] ppid=8565 vsize=2614980 CPUtime=4717.74 cores=0-7
/proc/8567/task/8573/stat : 8573 (claspmt) R 8565 8567 8541 0 -1 4202560 68575 0 0 0 471689 85 0 0 25 0 8 0 545372390 2677739520 582906 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=8567/tid=8574] ppid=8565 vsize=2614980 CPUtime=4721.75 cores=0-7
/proc/8567/task/8574/stat : 8574 (claspmt) R 8565 8567 8541 0 -1 4202560 12495 0 0 0 472153 22 0 0 25 0 8 0 545372390 2677739520 582906 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=8567/tid=8575] ppid=8565 vsize=2614980 CPUtime=4720.99 cores=0-7
/proc/8567/task/8575/stat : 8575 (claspmt) R 8565 8567 8541 0 -1 4202560 364549 0 0 0 471818 281 0 0 25 0 8 0 545372390 2677739520 582906 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668907 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 37746.2
Current children cumulated vsize (KiB) 2614980

[startup+4782.3 s]
/proc/loadavg: 8.00 8.00 8.00 9/194 8718
/proc/meminfo: memFree=22752892/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=2614980 CPUtime=38225.8 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 592618 0 0 0 3822037 547 0 0 25 0 8 0 545372360 2677739520 583072 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670243 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/8567/statm: 653745 583072 466 189 0 648094 0
[pid=8567/tid=8569] ppid=8565 vsize=2614980 CPUtime=4781.05 cores=0-7
/proc/8567/task/8569/stat : 8569 (claspmt) R 8565 8567 8541 0 -1 4202560 10326 0 0 0 478092 13 0 0 25 0 8 0 545372390 2677739520 583072 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670100 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=8567/tid=8570] ppid=8565 vsize=2614980 CPUtime=4781.8 cores=0-7
/proc/8567/task/8570/stat : 8570 (claspmt) R 8565 8567 8541 0 -1 4202560 51894 0 0 0 478129 51 0 0 25 0 8 0 545372390 2677739520 583072 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=8567/tid=8571] ppid=8565 vsize=2614980 CPUtime=4761.71 cores=0-7
/proc/8567/task/8571/stat : 8571 (claspmt) R 8565 8567 8541 0 -1 4202560 15519 0 0 0 476149 22 0 0 25 0 8 0 545372390 2677739520 583072 33554432000 4194304 4964704 140737107311472 18446744073709551615 4669214 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=8567/tid=8572] ppid=8565 vsize=2614980 CPUtime=4778.73 cores=0-7
/proc/8567/task/8572/stat : 8572 (claspmt) R 8565 8567 8541 0 -1 4202560 43976 0 0 0 477831 42 0 0 25 0 8 0 545372390 2677739520 583072 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670193 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=8567/tid=8573] ppid=8565 vsize=2614980 CPUtime=4777.74 cores=0-7
/proc/8567/task/8573/stat : 8573 (claspmt) R 8565 8567 8541 0 -1 4202560 68589 0 0 0 477689 85 0 0 25 0 8 0 545372390 2677739520 583072 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=8567/tid=8574] ppid=8565 vsize=2614980 CPUtime=4781.73 cores=0-7
/proc/8567/task/8574/stat : 8574 (claspmt) R 8565 8567 8541 0 -1 4202560 12502 0 0 0 478151 22 0 0 25 0 8 0 545372390 2677739520 583072 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=8567/tid=8575] ppid=8565 vsize=2614980 CPUtime=4780.99 cores=0-7
/proc/8567/task/8575/stat : 8575 (claspmt) R 8565 8567 8541 0 -1 4202560 364549 0 0 0 477818 281 0 0 25 0 8 0 545372390 2677739520 583072 33554432000 4194304 4964704 140737107311472 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38225.8
Current children cumulated vsize (KiB) 2614980

[startup+4842.3 s]
/proc/loadavg: 8.03 8.01 8.00 9/194 8719
/proc/meminfo: memFree=22716264/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=2645172 CPUtime=38705.5 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 601775 0 0 0 3869994 553 0 0 25 0 8 0 545372360 2708656128 592201 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/8567/statm: 661293 592201 466 189 0 655642 0
[pid=8567/tid=8569] ppid=8565 vsize=2645172 CPUtime=4841.04 cores=0-7
/proc/8567/task/8569/stat : 8569 (claspmt) R 8565 8567 8541 0 -1 4202560 10344 0 0 0 484091 13 0 0 25 0 8 0 545372390 2708656128 592201 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668484 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=8567/tid=8570] ppid=8565 vsize=2645172 CPUtime=4841.79 cores=0-7
/proc/8567/task/8570/stat : 8570 (claspmt) R 8565 8567 8541 0 -1 4202560 51914 0 0 0 484128 51 0 0 25 0 8 0 545372390 2708656128 592201 33554432000 4194304 4964704 140737107311472 18446744073709551615 4332681 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=8567/tid=8571] ppid=8565 vsize=2645172 CPUtime=4821.38 cores=0-7
/proc/8567/task/8571/stat : 8571 (claspmt) R 8565 8567 8541 0 -1 4202560 15573 0 0 0 482116 22 0 0 25 0 8 0 545372390 2708656128 592201 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668632 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=8567/tid=8572] ppid=8565 vsize=2645172 CPUtime=4838.72 cores=0-7
/proc/8567/task/8572/stat : 8572 (claspmt) R 8565 8567 8541 0 -1 4202560 44442 0 0 0 483829 43 0 0 25 0 8 0 545372390 2708656128 592201 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=8567/tid=8573] ppid=8565 vsize=2645172 CPUtime=4837.72 cores=0-7
/proc/8567/task/8573/stat : 8573 (claspmt) R 8565 8567 8541 0 -1 4202560 68607 0 0 0 483687 85 0 0 25 0 8 0 545372390 2708656128 592201 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668931 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=8567/tid=8574] ppid=8565 vsize=2645172 CPUtime=4841.73 cores=0-7
/proc/8567/task/8574/stat : 8574 (claspmt) R 8565 8567 8541 0 -1 4202560 12502 0 0 0 484151 22 0 0 25 0 8 0 545372390 2708656128 592201 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=8567/tid=8575] ppid=8565 vsize=2645172 CPUtime=4840.98 cores=0-7
/proc/8567/task/8575/stat : 8575 (claspmt) R 8565 8567 8541 0 -1 4202560 373113 0 0 0 483813 285 0 0 25 0 8 0 545372390 2708656128 592201 33554432000 4194304 4964704 140737107311472 18446744073709551615 4320848 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38705.5
Current children cumulated vsize (KiB) 2645172

[startup+4902.3 s]
/proc/loadavg: 8.04 8.02 8.00 10/194 8721
/proc/meminfo: memFree=22716120/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=2645172 CPUtime=39185.1 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 601775 0 0 0 3917956 553 0 0 25 0 8 0 545372360 2708656128 592201 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/8567/statm: 661293 592201 466 189 0 655642 0
[pid=8567/tid=8569] ppid=8565 vsize=2645172 CPUtime=4901 cores=0-7
/proc/8567/task/8569/stat : 8569 (claspmt) R 8565 8567 8541 0 -1 4202560 10344 0 0 0 490087 13 0 0 25 0 8 0 545372390 2708656128 592201 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668907 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=8567/tid=8570] ppid=8565 vsize=2645172 CPUtime=4901.79 cores=0-7
/proc/8567/task/8570/stat : 8570 (claspmt) R 8565 8567 8541 0 -1 4202560 51914 0 0 0 490128 51 0 0 25 0 8 0 545372390 2708656128 592201 33554432000 4194304 4964704 140737107311472 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=8567/tid=8571] ppid=8565 vsize=2645172 CPUtime=4881.05 cores=0-7
/proc/8567/task/8571/stat : 8571 (claspmt) R 8565 8567 8541 0 -1 4202560 15573 0 0 0 488083 22 0 0 25 0 8 0 545372390 2708656128 592201 33554432000 4194304 4964704 140737107311472 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=8567/tid=8572] ppid=8565 vsize=2645172 CPUtime=4898.73 cores=0-7
/proc/8567/task/8572/stat : 8572 (claspmt) R 8565 8567 8541 0 -1 4202560 44442 0 0 0 489830 43 0 0 25 0 8 0 545372390 2708656128 592201 33554432000 4194304 4964704 140737107311472 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=8567/tid=8573] ppid=8565 vsize=2645172 CPUtime=4897.72 cores=0-7
/proc/8567/task/8573/stat : 8573 (claspmt) R 8565 8567 8541 0 -1 4202560 68607 0 0 0 489687 85 0 0 25 0 8 0 545372390 2708656128 592201 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=8567/tid=8574] ppid=8565 vsize=2645172 CPUtime=4901.72 cores=0-7
/proc/8567/task/8574/stat : 8574 (claspmt) R 8565 8567 8541 0 -1 4202560 12502 0 0 0 490150 22 0 0 25 0 8 0 545372390 2708656128 592201 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=8567/tid=8575] ppid=8565 vsize=2645172 CPUtime=4900.99 cores=0-7
/proc/8567/task/8575/stat : 8575 (claspmt) R 8565 8567 8541 0 -1 4202560 373113 0 0 0 489813 286 0 0 25 0 8 0 545372390 2708656128 592201 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668618 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39185.1
Current children cumulated vsize (KiB) 2645172

[startup+4962.3 s]
/proc/loadavg: 8.01 8.02 8.00 9/194 8722
/proc/meminfo: memFree=22716096/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=2645172 CPUtime=39664.8 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 601819 0 0 0 3965922 554 0 0 25 0 8 0 545372360 2708656128 592245 33554432000 4194304 4964704 140737107311472 18446744073709551615 4321029 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/8567/statm: 661293 592245 466 189 0 655642 0
[pid=8567/tid=8569] ppid=8565 vsize=2645172 CPUtime=4961 cores=0-7
/proc/8567/task/8569/stat : 8569 (claspmt) R 8565 8567 8541 0 -1 4202560 10344 0 0 0 496087 13 0 0 25 0 8 0 545372390 2708656128 592245 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=8567/tid=8570] ppid=8565 vsize=2645172 CPUtime=4961.8 cores=0-7
/proc/8567/task/8570/stat : 8570 (claspmt) R 8565 8567 8541 0 -1 4202560 51914 0 0 0 496128 52 0 0 25 0 8 0 545372390 2708656128 592245 33554432000 4194304 4964704 140737107311472 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=8567/tid=8571] ppid=8565 vsize=2645172 CPUtime=4940.72 cores=0-7
/proc/8567/task/8571/stat : 8571 (claspmt) R 8565 8567 8541 0 -1 4202560 15573 0 0 0 494050 22 0 0 25 0 8 0 545372390 2708656128 592245 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670230 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=8567/tid=8572] ppid=8565 vsize=2645172 CPUtime=4958.73 cores=0-7
/proc/8567/task/8572/stat : 8572 (claspmt) R 8565 8567 8541 0 -1 4202560 44464 0 0 0 495830 43 0 0 25 0 8 0 545372390 2708656128 592245 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=8567/tid=8573] ppid=8565 vsize=2645172 CPUtime=4957.73 cores=0-7
/proc/8567/task/8573/stat : 8573 (claspmt) R 8565 8567 8541 0 -1 4202560 68607 0 0 0 495687 86 0 0 25 0 8 0 545372390 2708656128 592245 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=8567/tid=8574] ppid=8565 vsize=2645172 CPUtime=4961.71 cores=0-7
/proc/8567/task/8574/stat : 8574 (claspmt) R 8565 8567 8541 0 -1 4202560 12513 0 0 0 496149 22 0 0 25 0 8 0 545372390 2708656128 592245 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=8567/tid=8575] ppid=8565 vsize=2645172 CPUtime=4960.99 cores=0-7
/proc/8567/task/8575/stat : 8575 (claspmt) R 8565 8567 8541 0 -1 4202560 373115 0 0 0 495813 286 0 0 25 0 8 0 545372390 2708656128 592245 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39664.8
Current children cumulated vsize (KiB) 2645172



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5004.3 s]
/proc/loadavg: 8.01 8.01 8.00 9/194 8724
/proc/meminfo: memFree=22715948/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=2645172 CPUtime=40000.6 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 601821 0 0 0 3999503 554 0 0 25 0 8 0 545372360 2708656128 592247 33554432000 4194304 4964704 140737107311472 18446744073709551615 4653475 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/8567/statm: 661293 592247 466 189 0 655642 0
[pid=8567/tid=8569] ppid=8565 vsize=2645172 CPUtime=5003.01 cores=0-7
/proc/8567/task/8569/stat : 8569 (claspmt) R 8565 8567 8541 0 -1 4202560 10344 0 0 0 500288 13 0 0 25 0 8 0 545372390 2708656128 592247 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=8567/tid=8570] ppid=8565 vsize=2645172 CPUtime=5003.81 cores=0-7
/proc/8567/task/8570/stat : 8570 (claspmt) R 8565 8567 8541 0 -1 4202560 51914 0 0 0 500329 52 0 0 25 0 8 0 545372390 2708656128 592247 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=8567/tid=8571] ppid=8565 vsize=2645172 CPUtime=4982.49 cores=0-7
/proc/8567/task/8571/stat : 8571 (claspmt) R 8565 8567 8541 0 -1 4202560 15573 0 0 0 498227 22 0 0 25 0 8 0 545372390 2708656128 592247 33554432000 4194304 4964704 140737107311472 18446744073709551615 4668907 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=8567/tid=8572] ppid=8565 vsize=2645172 CPUtime=5000.74 cores=0-7
/proc/8567/task/8572/stat : 8572 (claspmt) R 8565 8567 8541 0 -1 4202560 44464 0 0 0 500031 43 0 0 25 0 8 0 545372390 2708656128 592247 33554432000 4194304 4964704 140737107311472 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=8567/tid=8573] ppid=8565 vsize=2645172 CPUtime=4999.74 cores=0-7
/proc/8567/task/8573/stat : 8573 (claspmt) R 8565 8567 8541 0 -1 4202560 68607 0 0 0 499888 86 0 0 25 0 8 0 545372390 2708656128 592247 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=8567/tid=8574] ppid=8565 vsize=2645172 CPUtime=5003.72 cores=0-7
/proc/8567/task/8574/stat : 8574 (claspmt) R 8565 8567 8541 0 -1 4202560 12515 0 0 0 500350 22 0 0 25 0 8 0 545372390 2708656128 592247 33554432000 4194304 4964704 140737107311472 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=8567/tid=8575] ppid=8565 vsize=2645172 CPUtime=5002.99 cores=0-7
/proc/8567/task/8575/stat : 8575 (claspmt) R 8565 8567 8541 0 -1 4202560 373115 0 0 0 500013 286 0 0 25 0 8 0 545372390 2708656128 592247 33554432000 4194304 4964704 140737107311472 18446744073709551615 4669119 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 2645172

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5004.8 s]
/proc/loadavg: 8.01 8.01 8.00 2/188 8725
/proc/meminfo: memFree=22735092/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=2624572 CPUtime=40001.1 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 601831 0 0 0 3999559 554 0 0 25 0 1 0 545372360 2687561728 587186 33554432000 4194304 4964704 140737107311472 18446744073709551615 4646575 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/8567/statm: 656143 587186 476 189 0 650492 0
Current children cumulated CPU time (s) 40001.1
Current children cumulated vsize (KiB) 2624572

[startup+5005 s]
/proc/loadavg: 8.01 8.01 8.00 2/188 8725
/proc/meminfo: memFree=22735092/32950928 swapFree=67111348/67111528
[pid=8567] ppid=8565 vsize=1848980 CPUtime=40001.3 cores=0-7
/proc/8567/stat : 8567 (claspmt) R 8565 8567 8541 0 -1 4202496 601831 0 0 0 3999575 557 0 0 25 0 1 0 545372360 1893355520 406792 33554432000 4194304 4964704 140737107311472 18446744073709551615 208080275591 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/8567/statm: 462245 406792 476 189 0 456594 0
Current children cumulated CPU time (s) 40001.3
Current children cumulated vsize (KiB) 1848980

Child status: 0
Real time (s): 5005.08
CPU time (s): 40001.4
CPU user time (s): 39995.8
CPU system time (s): 5.65514
CPU usage (%): 799.216
Max. virtual memory (cumulated for all children) (KiB): 2645172

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39995.8
system time used= 5.65514
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 601832
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 73660
involuntary context switches= 191497

runsolver used 7.40487 second user time and 19.909 second system time

The end

Launcher Data 

Begin job on node127 at 2011-05-02 14:02:07
IDJOB=3321383
IDBENCH=70791
IDSOLVER=1711
FILE ID=node127/3321383-1304337727
RUNJOBID= node127-1304337727-8555
PBS_JOBID= 13178044
Free space on /tmp= 73364 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT09/APPLICATIONS/bitverif/smulo/smulo064.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321383-1304337727/watcher-3321383-1304337727 -o /tmp/evaluation-result-3321383-1304337727/solver-3321383-1304337727 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321383-1304337727.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 472e9d10d7fd1a82d06da0809d0002df
RANDOM SEED=776864902

node127.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.60
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      25086416 kB
Buffers:        808576 kB
Cached:        3486908 kB
SwapCached:          8 kB
Active:        3250436 kB
Inactive:      4157020 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      25086416 kB
SwapTotal:    67111528 kB
SwapFree:     67111348 kB
Dirty:            3296 kB
Writeback:           0 kB
AnonPages:     3112248 kB
Mapped:          17836 kB
Slab:           383296 kB
PageTables:      11656 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:  3645468 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73364 MiB
End job on node127 at 2011-05-02 15:25:33