Trace number 3322781

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40001.6 5008.98

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m42-q663-n40-p30-s67876261.cnf
MD5SUMed0227339111fb337104188c725538d0
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.128979
Satisfiable
(Un)Satisfiability was proved
Number of variables1634
Number of clauses212885
Sum of the clauses size427324
Maximum clause length42
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2212845
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 540

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...395352/instance-3322781-1304395352.cnf
1.00/1.06	c Solving...
40000.60/5008.01	
40000.60/5008.01	*** INTERRUPTED! ***
40000.60/5008.01	*** Shutting down threads...
40001.54/5008.97	s UNKNOWN
40001.54/5008.97	
40001.54/5008.97	c INTERRUPTED   : 1
40001.54/5008.97	c Models        : 0+    
40001.54/5008.97	c Time          : 5008.110s (Solving: 5007.05s 1st Model: 0.00s Unsat: 0.00s)
40001.54/5008.97	c CPU Time      : 40000.750s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3322781-1304395352/watcher-3322781-1304395352 -o /tmp/evaluation-result-3322781-1304395352/solver-3322781-1304395352 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322781-1304395352.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.56 7.94 7.82 2/310 29506
/proc/meminfo: memFree=21035540/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=22532 CPUtime=0 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 1020 0 0 0 0 0 0 0 24 0 1 0 551134015 23072768 847 33554432000 4194304 4964704 140735820606608 18446744073709551615 268272349893 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/29506/statm: 5633 848 386 189 0 507 0

[startup+0.053532 s]
/proc/loadavg: 7.56 7.94 7.82 2/310 29506
/proc/meminfo: memFree=21035540/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=31112 CPUtime=0.04 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 4041 0 0 0 4 0 0 0 24 0 1 0 551134015 31858688 2793 33554432000 4194304 4964704 140735820606608 18446744073709551615 4615394 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/29506/statm: 7778 2794 397 189 0 2652 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 31112

[startup+0.100522 s]
/proc/loadavg: 7.56 7.94 7.82 2/310 29506
/proc/meminfo: memFree=21035540/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=35708 CPUtime=0.09 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 5108 0 0 0 9 0 0 0 24 0 1 0 551134015 36564992 3860 33554432000 4194304 4964704 140735820606608 18446744073709551615 4607536 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/29506/statm: 8927 3860 399 189 0 3801 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 35708

[startup+0.300576 s]
/proc/loadavg: 7.56 7.94 7.82 2/310 29506
/proc/meminfo: memFree=21035540/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=35708 CPUtime=0.29 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 5108 0 0 0 29 0 0 0 25 0 1 0 551134015 36564992 3860 33554432000 4194304 4964704 140735820606608 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/29506/statm: 8927 3860 399 189 0 3801 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 35708

[startup+0.700432 s]
/proc/loadavg: 7.56 7.94 7.82 2/310 29506
/proc/meminfo: memFree=21035540/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=35708 CPUtime=0.69 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 5109 0 0 0 69 0 0 0 25 0 1 0 551134015 36564992 3861 33554432000 4194304 4964704 140735820606608 18446744073709551615 4600434 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/29506/statm: 8927 3861 400 189 0 3801 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 35708

[startup+1.50028 s]
/proc/loadavg: 7.56 7.94 7.82 2/311 29507
/proc/meminfo: memFree=21020468/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=328612 CPUtime=3.73 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 11862 0 0 0 369 4 0 0 25 0 8 0 551134015 336498688 10172 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320955 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/29506/statm: 82153 10172 464 189 0 76502 0
Current children cumulated CPU time (s) 3.73
Current children cumulated vsize (KiB) 328612

[startup+3.10099 s]
/proc/loadavg: 7.59 7.94 7.82 9/318 29514
/proc/meminfo: memFree=20967836/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=343276 CPUtime=16.52 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 30575 0 0 0 1639 13 0 0 25 0 8 0 551134015 351514624 28885 33554432000 4194304 4964704 140735820606608 18446744073709551615 4342918 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/29506/statm: 85819 28885 464 189 0 80168 0
[pid=29506/tid=29508] ppid=29504 vsize=343276 CPUtime=1.91 cores=0-7
/proc/29506/task/29508/stat : 29508 (claspmt) R 29504 29506 29272 0 -1 4202560 3017 0 0 0 189 2 0 0 19 0 8 0 551134122 351514624 28885 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320911 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29506/tid=29509] ppid=29504 vsize=343276 CPUtime=1.92 cores=0-7
/proc/29506/task/29509/stat : 29509 (claspmt) R 29504 29506 29272 0 -1 4202560 2804 0 0 0 191 1 0 0 21 0 8 0 551134122 351514624 28885 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320961 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29506/tid=29510] ppid=29504 vsize=343276 CPUtime=1.92 cores=0-7
/proc/29506/task/29510/stat : 29510 (claspmt) R 29504 29506 29272 0 -1 4202560 3128 0 0 0 190 2 0 0 19 0 8 0 551134122 351514624 28886 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=29506/tid=29511] ppid=29504 vsize=343276 CPUtime=1.9 cores=0-7
/proc/29506/task/29511/stat : 29511 (claspmt) R 29504 29506 29272 0 -1 4202560 3471 0 0 0 189 1 0 0 21 0 8 0 551134122 351514624 28887 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29506/tid=29512] ppid=29504 vsize=343276 CPUtime=2.02 cores=0-7
/proc/29506/task/29512/stat : 29512 (claspmt) R 29504 29506 29272 0 -1 4202560 2446 0 0 0 202 0 0 0 21 0 8 0 551134122 351514624 28887 33554432000 4194304 4964704 140735820606608 18446744073709551615 268272349893 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29506/tid=29513] ppid=29504 vsize=343276 CPUtime=1.92 cores=0-7
/proc/29506/task/29513/stat : 29513 (claspmt) R 29504 29506 29272 0 -1 4202560 3342 0 0 0 191 1 0 0 19 0 8 0 551134122 351514624 28887 33554432000 4194304 4964704 140735820606608 18446744073709551615 4342941 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29506/tid=29514] ppid=29504 vsize=343276 CPUtime=1.92 cores=0-7
/proc/29506/task/29514/stat : 29514 (claspmt) R 29504 29506 29272 0 -1 4202560 3785 0 0 0 190 2 0 0 19 0 8 0 551134122 351514624 28887 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 16.52
Current children cumulated vsize (KiB) 343276

[startup+6.30042 s]
/proc/loadavg: 7.59 7.94 7.82 9/318 29514
/proc/meminfo: memFree=20850468/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=422392 CPUtime=42.06 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 52980 0 0 0 4182 24 0 0 25 0 8 0 551134015 432529408 51290 33554432000 4194304 4964704 140735820606608 18446744073709551615 268272732351 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/29506/statm: 105598 51290 466 189 0 99947 0
[pid=29506/tid=29508] ppid=29504 vsize=422392 CPUtime=5.1 cores=0-7
/proc/29506/task/29508/stat : 29508 (claspmt) R 29504 29506 29272 0 -1 4202560 5601 0 0 0 507 3 0 0 25 0 8 0 551134122 432529408 51290 33554432000 4194304 4964704 140735820606608 18446744073709551615 4342941 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29506/tid=29509] ppid=29504 vsize=422392 CPUtime=5.12 cores=0-7
/proc/29506/task/29509/stat : 29509 (claspmt) R 29504 29506 29272 0 -1 4202560 4777 0 0 0 510 2 0 0 25 0 8 0 551134122 432529408 51290 33554432000 4194304 4964704 140735820606608 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29506/tid=29510] ppid=29504 vsize=422392 CPUtime=5.11 cores=0-7
/proc/29506/task/29510/stat : 29510 (claspmt) R 29504 29506 29272 0 -1 4202560 5470 0 0 0 508 3 0 0 25 0 8 0 551134122 432529408 51290 33554432000 4194304 4964704 140735820606608 18446744073709551615 4653290 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=29506/tid=29511] ppid=29504 vsize=422392 CPUtime=5.07 cores=0-7
/proc/29506/task/29511/stat : 29511 (claspmt) R 29504 29506 29272 0 -1 4202560 6444 0 0 0 504 3 0 0 25 0 8 0 551134122 432529408 51291 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670011 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29506/tid=29512] ppid=29504 vsize=422392 CPUtime=5.21 cores=0-7
/proc/29506/task/29512/stat : 29512 (claspmt) R 29504 29506 29272 0 -1 4202560 6191 0 0 0 520 1 0 0 25 0 8 0 551134122 432529408 51291 33554432000 4194304 4964704 140735820606608 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29506/tid=29513] ppid=29504 vsize=422392 CPUtime=5.11 cores=0-7
/proc/29506/task/29513/stat : 29513 (claspmt) R 29504 29506 29272 0 -1 4202560 5618 0 0 0 510 1 0 0 25 0 8 0 551134122 432529408 51291 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29506/tid=29514] ppid=29504 vsize=422392 CPUtime=5.12 cores=0-7
/proc/29506/task/29514/stat : 29514 (claspmt) R 29504 29506 29272 0 -1 4202560 7395 0 0 0 508 4 0 0 25 0 8 0 551134122 432529408 51291 33554432000 4194304 4964704 140735820606608 18446744073709551615 4654517 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 42.06
Current children cumulated vsize (KiB) 422392

[startup+12.7013 s]
/proc/loadavg: 7.65 7.94 7.82 9/318 29514
/proc/meminfo: memFree=20700204/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=443128 CPUtime=93.18 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 87546 0 0 0 9273 45 0 0 25 0 8 0 551134015 453763072 85856 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320981 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/29506/statm: 110782 85856 466 189 0 105131 0
[pid=29506/tid=29508] ppid=29504 vsize=443128 CPUtime=11.51 cores=0-7
/proc/29506/task/29508/stat : 29508 (claspmt) R 29504 29506 29272 0 -1 4202560 11197 0 0 0 1143 8 0 0 25 0 8 0 551134122 453763072 85856 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670281 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29506/tid=29509] ppid=29504 vsize=443128 CPUtime=11.52 cores=0-7
/proc/29506/task/29509/stat : 29509 (claspmt) R 29504 29506 29272 0 -1 4202560 6806 0 0 0 1149 3 0 0 25 0 8 0 551134122 453763072 85856 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29506/tid=29510] ppid=29504 vsize=443128 CPUtime=11.51 cores=0-7
/proc/29506/task/29510/stat : 29510 (claspmt) R 29504 29506 29272 0 -1 4202560 9041 0 0 0 1146 5 0 0 25 0 8 0 551134122 453763072 85856 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=29506/tid=29511] ppid=29504 vsize=443128 CPUtime=11.4 cores=0-7
/proc/29506/task/29511/stat : 29511 (claspmt) R 29504 29506 29272 0 -1 4202560 11149 0 0 0 1135 5 0 0 25 0 8 0 551134122 453763072 85856 33554432000 4194304 4964704 140735820606608 18446744073709551615 4362349 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29506/tid=29512] ppid=29504 vsize=443128 CPUtime=11.61 cores=0-7
/proc/29506/task/29512/stat : 29512 (claspmt) R 29504 29506 29272 0 -1 4202560 10620 0 0 0 1158 3 0 0 25 0 8 0 551134122 453763072 85857 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29506/tid=29513] ppid=29504 vsize=443128 CPUtime=11.52 cores=0-7
/proc/29506/task/29513/stat : 29513 (claspmt) R 29504 29506 29272 0 -1 4202560 8803 0 0 0 1148 4 0 0 25 0 8 0 551134122 453763072 85857 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29506/tid=29514] ppid=29504 vsize=443128 CPUtime=11.52 cores=0-7
/proc/29506/task/29514/stat : 29514 (claspmt) R 29504 29506 29272 0 -1 4202560 13616 0 0 0 1143 9 0 0 25 0 8 0 551134122 453763072 85857 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 93.18
Current children cumulated vsize (KiB) 443128

[startup+25.501 s]
/proc/loadavg: 7.71 7.94 7.82 9/318 29514
/proc/meminfo: memFree=20626976/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=714308 CPUtime=195.41 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 104972 0 0 0 19484 57 0 0 25 0 8 0 551134015 731451392 103282 33554432000 4194304 4964704 140735820606608 18446744073709551615 4627263 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/29506/statm: 178577 103282 466 189 0 172926 0
[pid=29506/tid=29508] ppid=29504 vsize=714308 CPUtime=24.3 cores=0-7
/proc/29506/task/29508/stat : 29508 (claspmt) R 29504 29506 29272 0 -1 4202560 14234 0 0 0 2421 9 0 0 25 0 8 0 551134122 731451392 103282 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29506/tid=29509] ppid=29504 vsize=714308 CPUtime=24.32 cores=0-7
/proc/29506/task/29509/stat : 29509 (claspmt) R 29504 29506 29272 0 -1 4202560 8162 0 0 0 2428 4 0 0 25 0 8 0 551134122 731451392 103282 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29506/tid=29510] ppid=29504 vsize=714308 CPUtime=24.3 cores=0-7
/proc/29506/task/29510/stat : 29510 (claspmt) R 29504 29506 29272 0 -1 4202560 10397 0 0 0 2424 6 0 0 25 0 8 0 551134122 731451392 103282 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=29506/tid=29511] ppid=29504 vsize=714308 CPUtime=24.06 cores=0-7
/proc/29506/task/29511/stat : 29511 (claspmt) R 29504 29506 29272 0 -1 4202560 13169 0 0 0 2400 6 0 0 25 0 8 0 551134122 731451392 103282 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29506/tid=29512] ppid=29504 vsize=714308 CPUtime=24.41 cores=0-7
/proc/29506/task/29512/stat : 29512 (claspmt) R 29504 29506 29272 0 -1 4202560 12559 0 0 0 2437 4 0 0 25 0 8 0 551134122 731451392 103282 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29506/tid=29513] ppid=29504 vsize=714308 CPUtime=24.3 cores=0-7
/proc/29506/task/29513/stat : 29513 (claspmt) R 29504 29506 29272 0 -1 4202560 10511 0 0 0 2425 5 0 0 25 0 8 0 551134122 731451392 103282 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29506/tid=29514] ppid=29504 vsize=714308 CPUtime=24.31 cores=0-7
/proc/29506/task/29514/stat : 29514 (claspmt) R 29504 29506 29272 0 -1 4202560 16257 0 0 0 2420 11 0 0 25 0 8 0 551134122 731451392 103282 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670277 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 195.41
Current children cumulated vsize (KiB) 714308

[startup+51.1005 s]
/proc/loadavg: 7.80 7.95 7.83 9/318 29515
/proc/meminfo: memFree=20540516/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=801316 CPUtime=399.91 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 126332 0 0 0 39922 69 0 0 25 0 8 0 551134015 820547584 124619 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670286 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/29506/statm: 200329 124619 466 189 0 194678 0
[pid=29506/tid=29508] ppid=29504 vsize=801316 CPUtime=49.88 cores=0-7
/proc/29506/task/29508/stat : 29508 (claspmt) R 29504 29506 29272 0 -1 4202560 16597 0 0 0 4978 10 0 0 25 0 8 0 551134122 820547584 124619 33554432000 4194304 4964704 140735820606608 18446744073709551615 4321054 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29506/tid=29509] ppid=29504 vsize=801316 CPUtime=49.91 cores=0-7
/proc/29506/task/29509/stat : 29509 (claspmt) R 29504 29506 29272 0 -1 4202560 10049 0 0 0 4987 4 0 0 25 0 8 0 551134122 820547584 124619 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320934 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29506/tid=29510] ppid=29504 vsize=801316 CPUtime=49.9 cores=0-7
/proc/29506/task/29510/stat : 29510 (claspmt) R 29504 29506 29272 0 -1 4202560 11476 0 0 0 4983 7 0 0 25 0 8 0 551134122 820547584 124619 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=29506/tid=29511] ppid=29504 vsize=801316 CPUtime=49.4 cores=0-7
/proc/29506/task/29511/stat : 29511 (claspmt) R 29504 29506 29272 0 -1 4202560 15385 0 0 0 4933 7 0 0 25 0 8 0 551134122 820547584 124619 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670440 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29506/tid=29512] ppid=29504 vsize=801316 CPUtime=50.01 cores=0-7
/proc/29506/task/29512/stat : 29512 (claspmt) R 29504 29506 29272 0 -1 4202560 14666 0 0 0 4995 6 0 0 25 0 8 0 551134122 820547584 124619 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29506/tid=29513] ppid=29504 vsize=801316 CPUtime=49.9 cores=0-7
/proc/29506/task/29513/stat : 29513 (claspmt) R 29504 29506 29272 0 -1 4202560 13474 0 0 0 4983 7 0 0 25 0 8 0 551134122 820547584 124619 33554432000 4194304 4964704 140735820606608 18446744073709551615 4321020 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29506/tid=29514] ppid=29504 vsize=801316 CPUtime=49.9 cores=0-7
/proc/29506/task/29514/stat : 29514 (claspmt) R 29504 29506 29272 0 -1 4202560 20734 0 0 0 4977 13 0 0 25 0 8 0 551134122 820547584 124619 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 399.91
Current children cumulated vsize (KiB) 801316

[startup+102.312 s]
/proc/loadavg: 8.13 8.00 7.85 10/319 29520
/proc/meminfo: memFree=20324700/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=842844 CPUtime=808.96 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 179062 0 0 0 80793 103 0 0 25 0 8 0 551134015 863072256 177151 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670202 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/29506/statm: 210711 177151 466 189 0 205060 0
[pid=29506/tid=29508] ppid=29504 vsize=842844 CPUtime=101.06 cores=0-7
/proc/29506/task/29508/stat : 29508 (claspmt) R 29504 29506 29272 0 -1 4202560 21772 0 0 0 10090 16 0 0 25 0 8 0 551134122 863072256 177151 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29506/tid=29509] ppid=29504 vsize=842844 CPUtime=101.12 cores=0-7
/proc/29506/task/29509/stat : 29509 (claspmt) R 29504 29506 29272 0 -1 4202560 13466 0 0 0 10106 6 0 0 25 0 8 0 551134122 863072256 177151 33554432000 4194304 4964704 140735820606608 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29506/tid=29510] ppid=29504 vsize=842844 CPUtime=101.1 cores=0-7
/proc/29506/task/29510/stat : 29510 (claspmt) R 29504 29506 29272 0 -1 4202560 16304 0 0 0 10100 10 0 0 25 0 8 0 551134122 863072256 177151 33554432000 4194304 4964704 140735820606608 18446744073709551615 4644868 0 0 4096 18946 0 0 0 -1 4 0 0 0

################
# More data... #
################

[pid=29506] ppid=29504 vsize=4067800 CPUtime=37718.1 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 936758 0 0 0 3770860 946 0 0 25 0 8 0 551134015 4165427200 927792 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670266 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/29506/statm: 1016950 927792 466 189 0 1011299 0
[pid=29506/tid=29508] ppid=29504 vsize=4067800 CPUtime=4699.52 cores=0-7
/proc/29506/task/29508/stat : 29508 (claspmt) R 29504 29506 29272 0 -1 4202560 71404 0 0 0 469876 76 0 0 25 0 8 0 551134122 4165427200 927792 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29506/tid=29509] ppid=29504 vsize=4067800 CPUtime=4721.2 cores=0-7
/proc/29506/task/29509/stat : 29509 (claspmt) R 29504 29506 29272 0 -1 4202560 80319 0 0 0 472087 33 0 0 25 0 8 0 551134122 4165427200 927792 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29506/tid=29510] ppid=29504 vsize=4067800 CPUtime=4720.69 cores=0-7
/proc/29506/task/29510/stat : 29510 (claspmt) R 29504 29506 29272 0 -1 4202560 59302 0 0 0 472001 68 0 0 25 0 8 0 551134122 4165427200 927792 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=29506/tid=29511] ppid=29504 vsize=4067800 CPUtime=4709.42 cores=0-7
/proc/29506/task/29511/stat : 29511 (claspmt) R 29504 29506 29272 0 -1 4202560 85450 0 0 0 470860 82 0 0 25 0 8 0 551134122 4165427200 927792 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=29506/tid=29512] ppid=29504 vsize=4067800 CPUtime=4721.04 cores=0-7
/proc/29506/task/29512/stat : 29512 (claspmt) R 29504 29506 29272 0 -1 4202560 79867 0 0 0 472041 63 0 0 25 0 8 0 551134122 4165427200 927792 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29506/tid=29513] ppid=29504 vsize=4067800 CPUtime=4720.59 cores=0-7
/proc/29506/task/29513/stat : 29513 (claspmt) R 29504 29506 29272 0 -1 4202560 81445 0 0 0 471965 94 0 0 25 0 8 0 551134122 4165427200 927792 33554432000 4194304 4964704 140735820606608 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29506/tid=29514] ppid=29504 vsize=4067800 CPUtime=4720.81 cores=0-7
/proc/29506/task/29514/stat : 29514 (claspmt) R 29504 29506 29272 0 -1 4202560 180899 0 0 0 471899 182 0 0 25 0 8 0 551134122 4165427200 927792 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37718.1
Current children cumulated vsize (KiB) 4067800

[startup+4782.3 s]
/proc/loadavg: 8.02 8.04 8.00 9/318 29656
/proc/meminfo: memFree=17314764/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=4067800 CPUtime=38197.3 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 937739 0 0 0 3818786 948 0 0 25 0 8 0 551134015 4165427200 928773 33554432000 4194304 4964704 140735820606608 18446744073709551615 4332462 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/29506/statm: 1016950 928773 466 189 0 1011299 0
[pid=29506/tid=29508] ppid=29504 vsize=4067800 CPUtime=4758.92 cores=0-7
/proc/29506/task/29508/stat : 29508 (claspmt) R 29504 29506 29272 0 -1 4202560 71412 0 0 0 475816 76 0 0 25 0 8 0 551134122 4165427200 928773 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29506/tid=29509] ppid=29504 vsize=4067800 CPUtime=4781.19 cores=0-7
/proc/29506/task/29509/stat : 29509 (claspmt) R 29504 29506 29272 0 -1 4202560 80995 0 0 0 478086 33 0 0 25 0 8 0 551134122 4165427200 928773 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29506/tid=29510] ppid=29504 vsize=4067800 CPUtime=4780.68 cores=0-7
/proc/29506/task/29510/stat : 29510 (claspmt) R 29504 29506 29272 0 -1 4202560 59302 0 0 0 478000 68 0 0 25 0 8 0 551134122 4165427200 928773 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=29506/tid=29511] ppid=29504 vsize=4067800 CPUtime=4769.41 cores=0-7
/proc/29506/task/29511/stat : 29511 (claspmt) R 29504 29506 29272 0 -1 4202560 85706 0 0 0 476859 82 0 0 25 0 8 0 551134122 4165427200 928773 33554432000 4194304 4964704 140735820606608 18446744073709551615 4660755 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=29506/tid=29512] ppid=29504 vsize=4067800 CPUtime=4781.03 cores=0-7
/proc/29506/task/29512/stat : 29512 (claspmt) R 29504 29506 29272 0 -1 4202560 79867 0 0 0 478040 63 0 0 25 0 8 0 551134122 4165427200 928773 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29506/tid=29513] ppid=29504 vsize=4067800 CPUtime=4780.56 cores=0-7
/proc/29506/task/29513/stat : 29513 (claspmt) R 29504 29506 29272 0 -1 4202560 81478 0 0 0 477961 95 0 0 25 0 8 0 551134122 4165427200 928773 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670454 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29506/tid=29514] ppid=29504 vsize=4067800 CPUtime=4780.8 cores=0-7
/proc/29506/task/29514/stat : 29514 (claspmt) R 29504 29506 29272 0 -1 4202560 180899 0 0 0 477897 183 0 0 25 0 8 0 551134122 4165427200 928773 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38197.3
Current children cumulated vsize (KiB) 4067800

[startup+4842.3 s]
/proc/loadavg: 8.08 8.05 8.00 9/318 29658
/proc/meminfo: memFree=17309804/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=4067800 CPUtime=38676.7 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 939006 0 0 0 3866722 949 0 0 25 0 8 0 551134015 4165427200 930040 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320951 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/29506/statm: 1016950 930040 466 189 0 1011299 0
[pid=29506/tid=29508] ppid=29504 vsize=4067800 CPUtime=4818.32 cores=0-7
/proc/29506/task/29508/stat : 29508 (claspmt) R 29504 29506 29272 0 -1 4202560 71417 0 0 0 481756 76 0 0 25 0 8 0 551134122 4165427200 930040 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29506/tid=29509] ppid=29504 vsize=4067800 CPUtime=4841.19 cores=0-7
/proc/29506/task/29509/stat : 29509 (claspmt) R 29504 29506 29272 0 -1 4202560 82165 0 0 0 484085 34 0 0 25 0 8 0 551134122 4165427200 930040 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29506/tid=29510] ppid=29504 vsize=4067800 CPUtime=4840.66 cores=0-7
/proc/29506/task/29510/stat : 29510 (claspmt) R 29504 29506 29272 0 -1 4202560 59341 0 0 0 483998 68 0 0 25 0 8 0 551134122 4165427200 930040 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=29506/tid=29511] ppid=29504 vsize=4067800 CPUtime=4829.41 cores=0-7
/proc/29506/task/29511/stat : 29511 (claspmt) R 29504 29506 29272 0 -1 4202560 85747 0 0 0 482859 82 0 0 25 0 8 0 551134122 4165427200 930040 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=29506/tid=29512] ppid=29504 vsize=4067800 CPUtime=4841.03 cores=0-7
/proc/29506/task/29512/stat : 29512 (claspmt) R 29504 29506 29272 0 -1 4202560 79867 0 0 0 484040 63 0 0 25 0 8 0 551134122 4165427200 930040 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670357 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29506/tid=29513] ppid=29504 vsize=4067800 CPUtime=4840.56 cores=0-7
/proc/29506/task/29513/stat : 29513 (claspmt) R 29504 29506 29272 0 -1 4202560 81480 0 0 0 483961 95 0 0 25 0 8 0 551134122 4165427200 930040 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29506/tid=29514] ppid=29504 vsize=4067800 CPUtime=4840.8 cores=0-7
/proc/29506/task/29514/stat : 29514 (claspmt) R 29504 29506 29272 0 -1 4202560 180908 0 0 0 483896 184 0 0 25 0 8 0 551134122 4165427200 930040 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38676.7
Current children cumulated vsize (KiB) 4067800

[startup+4902.3 s]
/proc/loadavg: 8.03 8.03 8.00 9/318 29660
/proc/meminfo: memFree=17306676/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=4067800 CPUtime=39156.1 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 939794 0 0 0 3914660 950 0 0 25 0 8 0 551134015 4165427200 930828 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/29506/statm: 1016950 930828 466 189 0 1011299 0
[pid=29506/tid=29508] ppid=29504 vsize=4067800 CPUtime=4877.72 cores=0-7
/proc/29506/task/29508/stat : 29508 (claspmt) R 29504 29506 29272 0 -1 4202560 71419 0 0 0 487696 76 0 0 25 0 8 0 551134122 4165427200 930828 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29506/tid=29509] ppid=29504 vsize=4067800 CPUtime=4901.19 cores=0-7
/proc/29506/task/29509/stat : 29509 (claspmt) R 29504 29506 29272 0 -1 4202560 82871 0 0 0 490085 34 0 0 25 0 8 0 551134122 4165427200 930828 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29506/tid=29510] ppid=29504 vsize=4067800 CPUtime=4900.67 cores=0-7
/proc/29506/task/29510/stat : 29510 (claspmt) R 29504 29506 29272 0 -1 4202560 59352 0 0 0 489999 68 0 0 25 0 8 0 551134122 4165427200 930828 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=29506/tid=29511] ppid=29504 vsize=4067800 CPUtime=4889.41 cores=0-7
/proc/29506/task/29511/stat : 29511 (claspmt) R 29504 29506 29272 0 -1 4202560 85787 0 0 0 488859 82 0 0 25 0 8 0 551134122 4165427200 930828 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=29506/tid=29512] ppid=29504 vsize=4067800 CPUtime=4901.03 cores=0-7
/proc/29506/task/29512/stat : 29512 (claspmt) R 29504 29506 29272 0 -1 4202560 79867 0 0 0 490040 63 0 0 25 0 8 0 551134122 4165427200 930828 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29506/tid=29513] ppid=29504 vsize=4067800 CPUtime=4900.55 cores=0-7
/proc/29506/task/29513/stat : 29513 (claspmt) R 29504 29506 29272 0 -1 4202560 81492 0 0 0 489960 95 0 0 25 0 8 0 551134122 4165427200 930828 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29506/tid=29514] ppid=29504 vsize=4067800 CPUtime=4900.8 cores=0-7
/proc/29506/task/29514/stat : 29514 (claspmt) R 29504 29506 29272 0 -1 4202560 180908 0 0 0 489896 184 0 0 25 0 8 0 551134122 4165427200 930828 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39156.1
Current children cumulated vsize (KiB) 4067800

[startup+4962.3 s]
/proc/loadavg: 8.01 8.03 8.00 9/318 29661
/proc/meminfo: memFree=17304200/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=4067800 CPUtime=39635.4 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 940634 0 0 0 3962586 953 0 0 25 0 8 0 551134015 4165427200 931340 33554432000 4194304 4964704 140735820606608 18446744073709551615 4342939 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/29506/statm: 1016950 931340 466 189 0 1011299 0
[pid=29506/tid=29508] ppid=29504 vsize=4067800 CPUtime=4937.52 cores=0-7
/proc/29506/task/29508/stat : 29508 (claspmt) R 29504 29506 29272 0 -1 4202560 72115 0 0 0 493676 76 0 0 25 0 8 0 551134122 4165427200 931340 33554432000 4194304 4964704 140735820606608 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29506/tid=29509] ppid=29504 vsize=4067800 CPUtime=4961.19 cores=0-7
/proc/29506/task/29509/stat : 29509 (claspmt) R 29504 29506 29272 0 -1 4202560 82878 0 0 0 496085 34 0 0 25 0 8 0 551134122 4165427200 931340 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670458 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29506/tid=29510] ppid=29504 vsize=4067800 CPUtime=4960.66 cores=0-7
/proc/29506/task/29510/stat : 29510 (claspmt) R 29504 29506 29272 0 -1 4202560 59353 0 0 0 495998 68 0 0 25 0 8 0 551134122 4165427200 931340 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=29506/tid=29511] ppid=29504 vsize=4067800 CPUtime=4948.96 cores=0-7
/proc/29506/task/29511/stat : 29511 (claspmt) R 29504 29506 29272 0 -1 4202560 85915 0 0 0 494813 83 0 0 25 0 8 0 551134122 4165427200 931340 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29506/tid=29512] ppid=29504 vsize=4067800 CPUtime=4961.04 cores=0-7
/proc/29506/task/29512/stat : 29512 (claspmt) R 29504 29506 29272 0 -1 4202560 79867 0 0 0 496040 64 0 0 25 0 8 0 551134122 4165427200 931340 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29506/tid=29513] ppid=29504 vsize=4067800 CPUtime=4960.55 cores=0-7
/proc/29506/task/29513/stat : 29513 (claspmt) R 29504 29506 29272 0 -1 4202560 81497 0 0 0 495960 95 0 0 25 0 8 0 551134122 4165427200 931340 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29506/tid=29514] ppid=29504 vsize=4067800 CPUtime=4960.79 cores=0-7
/proc/29506/task/29514/stat : 29514 (claspmt) R 29504 29506 29272 0 -1 4202560 180911 0 0 0 495895 184 0 0 25 0 8 0 551134122 4165427200 931340 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39635.4
Current children cumulated vsize (KiB) 4067800



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5008 s]
/proc/loadavg: 8.08 8.04 8.00 9/318 29662
/proc/meminfo: memFree=17302924/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=4067800 CPUtime=40000.6 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 941016 0 0 0 3999106 954 0 0 25 0 8 0 551134015 4165427200 931722 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320981 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/29506/statm: 1016950 931722 466 189 0 1011299 0
[pid=29506/tid=29508] ppid=29504 vsize=4067800 CPUtime=4983.22 cores=0-7
/proc/29506/task/29508/stat : 29508 (claspmt) R 29504 29506 29272 0 -1 4202560 72185 0 0 0 498246 76 0 0 25 0 8 0 551134122 4165427200 931722 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29506/tid=29509] ppid=29504 vsize=4067800 CPUtime=5006.91 cores=0-7
/proc/29506/task/29509/stat : 29509 (claspmt) R 29504 29506 29272 0 -1 4202560 83181 0 0 0 500656 35 0 0 25 0 8 0 551134122 4165427200 931722 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=29506/tid=29510] ppid=29504 vsize=4067800 CPUtime=5006.36 cores=0-7
/proc/29506/task/29510/stat : 29510 (claspmt) R 29504 29506 29272 0 -1 4202560 59356 0 0 0 500568 68 0 0 25 0 8 0 551134122 4165427200 931722 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=29506/tid=29511] ppid=29504 vsize=4067800 CPUtime=4994.21 cores=0-7
/proc/29506/task/29511/stat : 29511 (claspmt) R 29504 29506 29272 0 -1 4202560 85915 0 0 0 499338 83 0 0 25 0 8 0 551134122 4165427200 931722 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=29506/tid=29512] ppid=29504 vsize=4067800 CPUtime=5006.75 cores=0-7
/proc/29506/task/29512/stat : 29512 (claspmt) R 29504 29506 29272 0 -1 4202560 79867 0 0 0 500611 64 0 0 25 0 8 0 551134122 4165427200 931722 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29506/tid=29513] ppid=29504 vsize=4067800 CPUtime=5006.26 cores=0-7
/proc/29506/task/29513/stat : 29513 (claspmt) R 29504 29506 29272 0 -1 4202560 81503 0 0 0 500531 95 0 0 25 0 8 0 551134122 4165427200 931722 33554432000 4194304 4964704 140735820606608 18446744073709551615 4670230 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=29506/tid=29514] ppid=29504 vsize=4067800 CPUtime=5006.51 cores=0-7
/proc/29506/task/29514/stat : 29514 (claspmt) R 29504 29506 29272 0 -1 4202560 180911 0 0 0 500467 184 0 0 25 0 8 0 551134122 4165427200 931722 33554432000 4194304 4964704 140735820606608 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 4067800

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5008.6 s]
/proc/loadavg: 8.08 8.04 8.00 2/312 29663
/proc/meminfo: memFree=17302972/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=4056648 CPUtime=40001.2 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 941026 0 0 0 3999170 954 0 0 25 0 1 0 551134015 4154007552 923671 33554432000 4194304 4964704 140735820606608 18446744073709551615 268272345659 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/29506/statm: 1014162 923671 476 189 0 1008511 0
Current children cumulated CPU time (s) 40001.2
Current children cumulated vsize (KiB) 4056648

[startup+5008.8 s]
/proc/loadavg: 8.08 8.04 8.00 2/312 29663
/proc/meminfo: memFree=17302972/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=4056648 CPUtime=40001.4 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 941026 0 0 0 3999190 954 0 0 25 0 1 0 551134015 4154007552 923658 33554432000 4194304 4964704 140735820606608 18446744073709551615 268272345659 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/29506/statm: 1014162 923658 476 189 0 1008511 0
Current children cumulated CPU time (s) 40001.4
Current children cumulated vsize (KiB) 4056648

[startup+5008.9 s]
/proc/loadavg: 8.08 8.04 8.00 2/312 29663
/proc/meminfo: memFree=17302972/32950928 swapFree=67111464/67111528
[pid=29506] ppid=29504 vsize=2065372 CPUtime=40001.5 cores=0-7
/proc/29506/stat : 29506 (claspmt) R 29504 29506 29272 0 -1 4202496 941026 0 0 0 3999193 961 0 0 25 0 1 0 551134015 2114940928 454993 33554432000 4194304 4964704 140735820606608 18446744073709551615 268272732295 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/29506/statm: 516343 454976 476 189 0 510692 0
Current children cumulated CPU time (s) 40001.5
Current children cumulated vsize (KiB) 2065372

Child status: 0
Real time (s): 5008.98
CPU time (s): 40001.6
CPU user time (s): 39991.9
CPU system time (s): 9.69353
CPU usage (%): 798.598
Max. virtual memory (cumulated for all children) (KiB): 4067800

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39991.9
system time used= 9.69353
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 941027
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 149481
involuntary context switches= 187746

runsolver used 13.8569 second user time and 36.2445 second system time

The end

Launcher Data 

Begin job on node122 at 2011-05-03 06:02:32
IDJOB=3322781
IDBENCH=88222
IDSOLVER=1711
FILE ID=node122/3322781-1304395352
RUNJOBID= node122-1304391831-29286
PBS_JOBID= 13181848
Free space on /tmp= 73812 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m42-q663-n40-p30-s67876261.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322781-1304395352/watcher-3322781-1304395352 -o /tmp/evaluation-result-3322781-1304395352/solver-3322781-1304395352 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322781-1304395352.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= ed0227339111fb337104188c725538d0
RANDOM SEED=1532518409

node122.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      21035688 kB
Buffers:        397184 kB
Cached:        3168000 kB
SwapCached:         64 kB
Active:        8593808 kB
Inactive:      2935412 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      21035688 kB
SwapTotal:    67111528 kB
SwapFree:     67111464 kB
Dirty:            3544 kB
Writeback:           0 kB
AnonPages:     7964148 kB
Mapped:          25576 kB
Slab:           289364 kB
PageTables:      31648 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 52522896 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73812 MiB
End job on node122 at 2011-05-03 07:26:02