Trace number 3381093

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40000.4 5008.35

General information on the benchmark

Namecrafted/kullmann/VanDerWaerden/
VanderWaerden3k/VanDerWaerden_2-3-18_312.cnf
MD5SUM1144bbd60c180aa5e72d35b74e9676cb
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables312
Number of clauses26889
Sum of the clauses size121302
Maximum clause length18
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 324180
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 52709

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...504063/instance-3381093-1305504063.cnf
0.10/0.12	c Solving...
40000.14/5008.22	
40000.14/5008.22	*** INTERRUPTED! ***
40000.14/5008.22	*** Shutting down threads...
40000.31/5008.35	s UNKNOWN
40000.31/5008.35	
40000.31/5008.35	c INTERRUPTED   : 1
40000.31/5008.35	c Models        : 0+    
40000.31/5008.35	c Time          : 5008.218s (Solving: 5008.10s 1st Model: 0.00s Unsat: 0.00s)
40000.31/5008.35	c CPU Time      : 40000.210s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3381093-1305504063/watcher-3381093-1305504063 -o /tmp/evaluation-result-3381093-1305504063/solver-3381093-1305504063 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381093-1305504063.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.77 7.74 7.86 2/363 4203
/proc/meminfo: memFree=22760316/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=22296 CPUtime=0 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 894 0 0 0 0 0 0 0 25 0 1 0 642821579 22831104 788 33554432000 4194304 4964800 140737454484032 18446744073709551615 254261277381 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/4203/statm: 5574 789 386 189 0 448 0

[startup+0.107554 s]
/proc/loadavg: 6.77 7.74 7.86 2/363 4203
/proc/meminfo: memFree=22760316/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=23556 CPUtime=0.1 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 1221 0 0 0 10 0 0 0 25 0 1 0 642821579 24121344 1113 33554432000 4194304 4964800 140737454484032 18446744073709551615 4607585 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/4203/statm: 5889 1113 399 189 0 763 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 23556

[startup+0.206129 s]
/proc/loadavg: 6.77 7.74 7.86 2/363 4203
/proc/meminfo: memFree=22760316/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=320716 CPUtime=4.29 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 5934 0 0 0 427 2 0 0 18 0 8 0 642821579 328413184 5760 33554432000 4194304 4964800 140737454484032 18446744073709551615 4660236 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4203/statm: 80213 5913 464 189 0 74562 0
Current children cumulated CPU time (s) 4.29
Current children cumulated vsize (KiB) 320716

[startup+1.04835 s]
/proc/loadavg: 6.77 7.74 7.86 2/363 4203
/proc/meminfo: memFree=22760316/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=321248 CPUtime=7.47 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 6679 0 0 0 744 3 0 0 19 0 8 0 642821579 328957952 6505 33554432000 4194304 4964800 140737454484032 18446744073709551615 4627365 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4203/statm: 80312 6505 464 189 0 74661 0
Current children cumulated CPU time (s) 7.47
Current children cumulated vsize (KiB) 321248

[startup+1.14833 s]
/proc/loadavg: 6.77 7.74 7.86 2/363 4203
/proc/meminfo: memFree=22760316/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=321592 CPUtime=8.27 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 7011 0 0 0 824 3 0 0 19 0 8 0 642821579 329310208 6837 33554432000 4194304 4964800 140737454484032 18446744073709551615 4642520 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4203/statm: 80398 6837 464 189 0 74747 0
Current children cumulated CPU time (s) 8.27
Current children cumulated vsize (KiB) 321592

[startup+1.50027 s]
/proc/loadavg: 6.77 7.74 7.86 2/363 4203
/proc/meminfo: memFree=22760316/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=321728 CPUtime=11.08 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 7173 0 0 0 1105 3 0 0 19 0 8 0 642821579 329449472 6999 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4203/statm: 80432 6999 464 189 0 74781 0
Current children cumulated CPU time (s) 11.08
Current children cumulated vsize (KiB) 321728

[startup+3.10099 s]
/proc/loadavg: 6.77 7.74 7.86 9/371 4211
/proc/meminfo: memFree=22726784/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=323772 CPUtime=23.86 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 9323 0 0 0 2382 4 0 0 24 0 8 0 642821579 331542528 9149 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670295 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4203/statm: 80943 9149 466 189 0 75292 0
[pid=4203/tid=4205] ppid=4201 vsize=323772 CPUtime=2.96 cores=0-7
/proc/4203/task/4205/stat : 4205 (claspmt) R 4201 4203 4177 0 -1 4202560 921 0 0 0 296 0 0 0 25 0 8 0 642821591 331542528 9149 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4203/tid=4206] ppid=4201 vsize=323772 CPUtime=2.94 cores=0-7
/proc/4203/task/4206/stat : 4206 (claspmt) R 4201 4203 4177 0 -1 4202560 1303 0 0 0 294 0 0 0 25 0 8 0 642821591 331542528 9149 33554432000 4194304 4964800 140737454484032 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4203/tid=4207] ppid=4201 vsize=323772 CPUtime=2.96 cores=0-7
/proc/4203/task/4207/stat : 4207 (claspmt) R 4201 4203 4177 0 -1 4202560 828 0 0 0 296 0 0 0 25 0 8 0 642821591 331542528 9149 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4203/tid=4208] ppid=4201 vsize=323772 CPUtime=2.97 cores=0-7
/proc/4203/task/4208/stat : 4208 (claspmt) R 4201 4203 4177 0 -1 4202560 906 0 0 0 297 0 0 0 24 0 8 0 642821591 331542528 9149 33554432000 4194304 4964800 140737454484032 18446744073709551615 4362425 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4203/tid=4209] ppid=4201 vsize=323772 CPUtime=2.97 cores=0-7
/proc/4203/task/4209/stat : 4209 (claspmt) R 4201 4203 4177 0 -1 4202560 917 0 0 0 297 0 0 0 25 0 8 0 642821591 331542528 9149 33554432000 4194304 4964800 140737454484032 18446744073709551615 4683657 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4203/tid=4210] ppid=4201 vsize=323772 CPUtime=2.96 cores=0-7
/proc/4203/task/4210/stat : 4210 (claspmt) R 4201 4203 4177 0 -1 4202560 955 0 0 0 296 0 0 0 22 0 8 0 642821591 331542528 9149 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4203/tid=4211] ppid=4201 vsize=323772 CPUtime=2.96 cores=0-7
/proc/4203/task/4211/stat : 4211 (claspmt) R 4201 4203 4177 0 -1 4202560 1369 0 0 0 295 1 0 0 25 0 8 0 642821591 331542528 9149 33554432000 4194304 4964800 140737454484032 18446744073709551615 4627873 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 23.86
Current children cumulated vsize (KiB) 323772

[startup+6.31442 s]
/proc/loadavg: 6.87 7.75 7.86 9/371 4211
/proc/meminfo: memFree=22715012/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=326308 CPUtime=49.52 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 12780 0 0 0 4946 6 0 0 25 0 8 0 642821579 334139392 12606 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4203/statm: 81577 12606 466 189 0 75926 0
[pid=4203/tid=4205] ppid=4201 vsize=326308 CPUtime=6.17 cores=0-7
/proc/4203/task/4205/stat : 4205 (claspmt) R 4201 4203 4177 0 -1 4202560 1298 0 0 0 617 0 0 0 25 0 8 0 642821591 334139392 12606 33554432000 4194304 4964800 140737454484032 18446744073709551615 4669280 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4203/tid=4206] ppid=4201 vsize=326308 CPUtime=6.11 cores=0-7
/proc/4203/task/4206/stat : 4206 (claspmt) R 4201 4203 4177 0 -1 4202560 1936 0 0 0 611 0 0 0 25 0 8 0 642821591 334139392 12606 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4203/tid=4207] ppid=4201 vsize=326308 CPUtime=6.17 cores=0-7
/proc/4203/task/4207/stat : 4207 (claspmt) R 4201 4203 4177 0 -1 4202560 1187 0 0 0 617 0 0 0 25 0 8 0 642821591 334139392 12607 33554432000 4194304 4964800 140737454484032 18446744073709551615 4627309 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4203/tid=4208] ppid=4201 vsize=326308 CPUtime=6.18 cores=0-7
/proc/4203/task/4208/stat : 4208 (claspmt) R 4201 4203 4177 0 -1 4202560 1301 0 0 0 618 0 0 0 25 0 8 0 642821591 334139392 12607 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4203/tid=4209] ppid=4201 vsize=326308 CPUtime=6.18 cores=0-7
/proc/4203/task/4209/stat : 4209 (claspmt) R 4201 4203 4177 0 -1 4202560 1252 0 0 0 618 0 0 0 25 0 8 0 642821591 334139392 12607 33554432000 4194304 4964800 140737454484032 18446744073709551615 4659841 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4203/tid=4210] ppid=4201 vsize=326308 CPUtime=6.17 cores=0-7
/proc/4203/task/4210/stat : 4210 (claspmt) R 4201 4203 4177 0 -1 4202560 1266 0 0 0 617 0 0 0 25 0 8 0 642821591 334139392 12607 33554432000 4194304 4964800 140737454484032 18446744073709551615 4654485 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4203/tid=4211] ppid=4201 vsize=326308 CPUtime=6.17 cores=0-7
/proc/4203/task/4211/stat : 4211 (claspmt) R 4201 4203 4177 0 -1 4202560 2065 0 0 0 616 1 0 0 25 0 8 0 642821591 334139392 12607 33554432000 4194304 4964800 140737454484032 18446744073709551615 254261277729 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 49.52
Current children cumulated vsize (KiB) 326308

[startup+12.7013 s]
/proc/loadavg: 6.96 7.75 7.86 9/371 4211
/proc/meminfo: memFree=22695708/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=329692 CPUtime=100.54 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 17579 0 0 0 10044 10 0 0 25 0 8 0 642821579 337604608 17203 33554432000 4194304 4964800 140737454484032 18446744073709551615 4320951 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4203/statm: 82423 17203 466 189 0 76772 0
[pid=4203/tid=4205] ppid=4201 vsize=329692 CPUtime=12.56 cores=0-7
/proc/4203/task/4205/stat : 4205 (claspmt) R 4201 4203 4177 0 -1 4202560 1927 0 0 0 1255 1 0 0 25 0 8 0 642821591 337604608 17203 33554432000 4194304 4964800 140737454484032 18446744073709551615 4312169 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4203/tid=4206] ppid=4201 vsize=329692 CPUtime=12.43 cores=0-7
/proc/4203/task/4206/stat : 4206 (claspmt) R 4201 4203 4177 0 -1 4202560 3050 0 0 0 1243 0 0 0 25 0 8 0 642821591 337604608 17203 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4203/tid=4207] ppid=4201 vsize=329692 CPUtime=12.56 cores=0-7
/proc/4203/task/4207/stat : 4207 (claspmt) R 4201 4203 4177 0 -1 4202560 1601 0 0 0 1255 1 0 0 25 0 8 0 642821591 337604608 17203 33554432000 4194304 4964800 140737454484032 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4203/tid=4208] ppid=4201 vsize=329692 CPUtime=12.56 cores=0-7
/proc/4203/task/4208/stat : 4208 (claspmt) R 4201 4203 4177 0 -1 4202560 1829 0 0 0 1256 0 0 0 25 0 8 0 642821591 337604608 17203 33554432000 4194304 4964800 140737454484032 18446744073709551615 4627213 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4203/tid=4209] ppid=4201 vsize=329692 CPUtime=12.56 cores=0-7
/proc/4203/task/4209/stat : 4209 (claspmt) R 4201 4203 4177 0 -1 4202560 1680 0 0 0 1256 0 0 0 25 0 8 0 642821591 337604608 17203 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4203/tid=4210] ppid=4201 vsize=329692 CPUtime=12.56 cores=0-7
/proc/4203/task/4210/stat : 4210 (claspmt) R 4201 4203 4177 0 -1 4202560 1784 0 0 0 1255 1 0 0 20 0 8 0 642821591 337604608 17203 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4203/tid=4211] ppid=4201 vsize=329692 CPUtime=12.56 cores=0-7
/proc/4203/task/4211/stat : 4211 (claspmt) R 4201 4203 4177 0 -1 4202560 2840 0 0 0 1254 2 0 0 25 0 8 0 642821591 337604608 17203 33554432000 4194304 4964800 140737454484032 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 100.54
Current children cumulated vsize (KiB) 329692

[startup+25.501 s]
/proc/loadavg: 7.18 7.76 7.86 9/371 4211
/proc/meminfo: memFree=22673808/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=333692 CPUtime=202.76 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 23241 0 0 0 20262 14 0 0 25 0 8 0 642821579 341700608 22861 33554432000 4194304 4964800 140737454484032 18446744073709551615 4627365 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4203/statm: 83423 22861 466 189 0 77772 0
[pid=4203/tid=4205] ppid=4201 vsize=333692 CPUtime=25.35 cores=0-7
/proc/4203/task/4205/stat : 4205 (claspmt) R 4201 4203 4177 0 -1 4202560 2688 0 0 0 2534 1 0 0 25 0 8 0 642821591 341700608 22861 33554432000 4194304 4964800 140737454484032 18446744073709551615 4669184 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4203/tid=4206] ppid=4201 vsize=333692 CPUtime=25.08 cores=0-7
/proc/4203/task/4206/stat : 4206 (claspmt) R 4201 4203 4177 0 -1 4202560 4265 0 0 0 2506 2 0 0 25 0 8 0 642821591 341700608 22861 33554432000 4194304 4964800 140737454484032 18446744073709551615 4668753 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4203/tid=4207] ppid=4201 vsize=333692 CPUtime=25.35 cores=0-7
/proc/4203/task/4207/stat : 4207 (claspmt) R 4201 4203 4177 0 -1 4202560 2130 0 0 0 2534 1 0 0 25 0 8 0 642821591 341700608 22861 33554432000 4194304 4964800 140737454484032 18446744073709551615 4627290 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4203/tid=4208] ppid=4201 vsize=333692 CPUtime=25.36 cores=0-7
/proc/4203/task/4208/stat : 4208 (claspmt) R 4201 4203 4177 0 -1 4202560 2532 0 0 0 2535 1 0 0 25 0 8 0 642821591 341700608 22861 33554432000 4194304 4964800 140737454484032 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4203/tid=4209] ppid=4201 vsize=333692 CPUtime=25.37 cores=0-7
/proc/4203/task/4209/stat : 4209 (claspmt) R 4201 4203 4177 0 -1 4202560 2375 0 0 0 2536 1 0 0 25 0 8 0 642821591 341700608 22861 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670129 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4203/tid=4210] ppid=4201 vsize=333692 CPUtime=25.35 cores=0-7
/proc/4203/task/4210/stat : 4210 (claspmt) R 4201 4203 4177 0 -1 4202560 2254 0 0 0 2534 1 0 0 25 0 8 0 642821591 341700608 22861 33554432000 4194304 4964800 140737454484032 18446744073709551615 4668689 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4203/tid=4211] ppid=4201 vsize=333692 CPUtime=25.36 cores=0-7
/proc/4203/task/4211/stat : 4211 (claspmt) R 4201 4203 4177 0 -1 4202560 3673 0 0 0 2533 3 0 0 25 0 8 0 642821591 341700608 22861 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670264 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 202.76
Current children cumulated vsize (KiB) 333692

[startup+51.1005 s]
/proc/loadavg: 7.46 7.78 7.86 9/371 4212
/proc/meminfo: memFree=22643616/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=342328 CPUtime=407.23 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 31359 0 0 0 40703 20 0 0 25 0 8 0 642821579 350543872 30904 33554432000 4194304 4964800 140737454484032 18446744073709551615 4660390 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4203/statm: 85582 30904 466 189 0 79931 0
[pid=4203/tid=4205] ppid=4201 vsize=342328 CPUtime=50.95 cores=0-7
/proc/4203/task/4205/stat : 4205 (claspmt) R 4201 4203 4177 0 -1 4202560 3670 0 0 0 5093 2 0 0 25 0 8 0 642821591 350543872 30904 33554432000 4194304 4964800 140737454484032 18446744073709551615 4628368 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4203/tid=4206] ppid=4201 vsize=342328 CPUtime=50.37 cores=0-7
/proc/4203/task/4206/stat : 4206 (claspmt) R 4201 4203 4177 0 -1 4202560 6134 0 0 0 5034 3 0 0 25 0 8 0 642821591 350543872 30904 33554432000 4194304 4964800 140737454484032 18446744073709551615 4690465 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4203/tid=4207] ppid=4201 vsize=342328 CPUtime=50.94 cores=0-7
/proc/4203/task/4207/stat : 4207 (claspmt) R 4201 4203 4177 0 -1 4202560 2797 0 0 0 5093 1 0 0 25 0 8 0 642821591 350543872 30904 33554432000 4194304 4964800 140737454484032 18446744073709551615 4659398 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4203/tid=4208] ppid=4201 vsize=342328 CPUtime=50.95 cores=0-7
/proc/4203/task/4208/stat : 4208 (claspmt) R 4201 4203 4177 0 -1 4202560 3746 0 0 0 5094 1 0 0 25 0 8 0 642821591 350543872 30904 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4203/tid=4209] ppid=4201 vsize=342328 CPUtime=50.96 cores=0-7
/proc/4203/task/4209/stat : 4209 (claspmt) R 4201 4203 4177 0 -1 4202560 3217 0 0 0 5095 1 0 0 25 0 8 0 642821591 350543872 30904 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4203/tid=4210] ppid=4201 vsize=342328 CPUtime=50.95 cores=0-7
/proc/4203/task/4210/stat : 4210 (claspmt) R 4201 4203 4177 0 -1 4202560 3003 0 0 0 5093 2 0 0 25 0 8 0 642821591 350543872 30904 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4203/tid=4211] ppid=4201 vsize=342328 CPUtime=50.95 cores=0-7
/proc/4203/task/4211/stat : 4211 (claspmt) R 4201 4203 4177 0 -1 4202560 4848 0 0 0 5091 4 0 0 25 0 8 0 642821591 350543872 30904 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 407.23
Current children cumulated vsize (KiB) 342328

[startup+102.3 s]
/proc/loadavg: 7.76 7.81 7.87 9/371 4213
/proc/meminfo: memFree=22606348/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=351144 CPUtime=816.14 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 40262 0 0 0 81584 30 0 0 25 0 8 0 642821579 359571456 39619 33554432000 4194304 4964800 140737454484032 18446744073709551615 4654535 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4203/statm: 87786 39619 466 189 0 82135 0
[pid=4203/tid=4205] ppid=4201 vsize=351144 CPUtime=102.13 cores=0-7
/proc/4203/task/4205/stat : 4205 (claspmt) R 4201 4203 4177 0 -1 4202560 5173 0 0 0 10209 4 0 0 25 0 8 0 642821591 359571456 39619 33554432000 4194304 4964800 140737454484032 18446744073709551615 4668698 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4203/tid=4206] ppid=4201 vsize=351144 CPUtime=100.95 cores=0-7
/proc/4203/task/4206/stat : 4206 (claspmt) R 4201 4203 4177 0 -1 4202560 7504 0 0 0 10091 4 0 0 25 0 8 0 642821591 359571456 39619 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4203/tid=4207] ppid=4201 vsize=351144 CPUtime=102.13 cores=0-7
/proc/4203/task/4207/stat : 4207 (claspmt) R 4201 4203 4177 0 -1 4202560 3535 0 0 0 10211 2 0 0 20 0 8 0 642821591 359571456 39619 33554432000 4194304 4964800 140737454484032 18446744073709551615 4668689 0 0 4096 18946 0 0 0 -1 2 0 0 0

################
# More data... #
################

[pid=4203/tid=4206] ppid=4201 vsize=676908 CPUtime=4630.55 cores=0-7
/proc/4203/task/4206/stat : 4206 (claspmt) R 4201 4203 4177 0 -1 4202560 23916 0 0 0 463002 53 0 0 25 0 8 0 642821591 693153792 113926 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4203/tid=4207] ppid=4201 vsize=676908 CPUtime=4660.96 cores=0-7
/proc/4203/task/4207/stat : 4207 (claspmt) R 4201 4203 4177 0 -1 4202560 7593 0 0 0 466053 43 0 0 25 0 8 0 642821591 693153792 113926 33554432000 4194304 4964800 140737454484032 18446744073709551615 4668748 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4203/tid=4208] ppid=4201 vsize=676908 CPUtime=4656.43 cores=0-7
/proc/4203/task/4208/stat : 4208 (claspmt) R 4201 4203 4177 0 -1 4202560 14602 0 0 0 465602 41 0 0 25 0 8 0 642821591 693153792 113926 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4203/tid=4209] ppid=4201 vsize=676908 CPUtime=4661.96 cores=0-7
/proc/4203/task/4209/stat : 4209 (claspmt) R 4201 4203 4177 0 -1 4202560 14766 0 0 0 466155 41 0 0 25 0 8 0 642821591 693153792 113926 33554432000 4194304 4964800 140737454484032 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4203/tid=4210] ppid=4201 vsize=676908 CPUtime=4661.44 cores=0-7
/proc/4203/task/4210/stat : 4210 (claspmt) R 4201 4203 4177 0 -1 4202560 10301 0 0 0 466087 57 0 0 25 0 8 0 642821591 693153792 113926 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4203/tid=4211] ppid=4201 vsize=676908 CPUtime=4661.54 cores=0-7
/proc/4203/task/4211/stat : 4211 (claspmt) R 4201 4203 4177 0 -1 4202560 14961 0 0 0 466098 56 0 0 25 0 8 0 642821591 693153792 113926 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37237.3
Current children cumulated vsize (KiB) 676908

[startup+4722.3 s]
/proc/loadavg: 8.09 8.06 8.00 9/371 4365
/proc/meminfo: memFree=22305820/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=676908 CPUtime=37716.5 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 126768 0 0 0 3771253 398 0 0 25 0 8 0 642821579 693153792 114183 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4203/statm: 169227 114183 466 189 0 163576 0
[pid=4203/tid=4205] ppid=4201 vsize=676908 CPUtime=4702.32 cores=0-7
/proc/4203/task/4205/stat : 4205 (claspmt) R 4201 4203 4177 0 -1 4202560 31622 0 0 0 470175 57 0 0 25 0 8 0 642821591 693153792 114183 33554432000 4194304 4964800 140737454484032 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4203/tid=4206] ppid=4201 vsize=676908 CPUtime=4689.83 cores=0-7
/proc/4203/task/4206/stat : 4206 (claspmt) R 4201 4203 4177 0 -1 4202560 23923 0 0 0 468929 54 0 0 25 0 8 0 642821591 693153792 114183 33554432000 4194304 4964800 140737454484032 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4203/tid=4207] ppid=4201 vsize=676908 CPUtime=4720.94 cores=0-7
/proc/4203/task/4207/stat : 4207 (claspmt) R 4201 4203 4177 0 -1 4202560 7650 0 0 0 472051 43 0 0 25 0 8 0 642821591 693153792 114183 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4203/tid=4208] ppid=4201 vsize=676908 CPUtime=4716.42 cores=0-7
/proc/4203/task/4208/stat : 4208 (claspmt) R 4201 4203 4177 0 -1 4202560 14608 0 0 0 471600 42 0 0 25 0 8 0 642821591 693153792 114183 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4203/tid=4209] ppid=4201 vsize=676908 CPUtime=4721.96 cores=0-7
/proc/4203/task/4209/stat : 4209 (claspmt) R 4201 4203 4177 0 -1 4202560 14768 0 0 0 472155 41 0 0 25 0 8 0 642821591 693153792 114183 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4203/tid=4210] ppid=4201 vsize=676908 CPUtime=4721.45 cores=0-7
/proc/4203/task/4210/stat : 4210 (claspmt) R 4201 4203 4177 0 -1 4202560 10350 0 0 0 472086 59 0 0 25 0 8 0 642821591 693153792 114183 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4203/tid=4211] ppid=4201 vsize=676908 CPUtime=4721.53 cores=0-7
/proc/4203/task/4211/stat : 4211 (claspmt) R 4201 4203 4177 0 -1 4202560 14967 0 0 0 472096 57 0 0 25 0 8 0 642821591 693153792 114183 33554432000 4194304 4964800 140737454484032 18446744073709551615 4669221 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37716.5
Current children cumulated vsize (KiB) 676908

[startup+4782.3 s]
/proc/loadavg: 8.03 8.05 8.00 9/371 4366
/proc/meminfo: memFree=22303564/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=677572 CPUtime=38195.8 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 127509 0 0 0 3819174 402 0 0 25 0 8 0 642821579 693833728 114892 33554432000 4194304 4964800 140737454484032 18446744073709551615 4689809 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4203/statm: 169393 114892 466 189 0 163742 0
[pid=4203/tid=4205] ppid=4201 vsize=677572 CPUtime=4762.31 cores=0-7
/proc/4203/task/4205/stat : 4205 (claspmt) R 4201 4203 4177 0 -1 4202560 31789 0 0 0 476174 57 0 0 25 0 8 0 642821591 693833728 114892 33554432000 4194304 4964800 140737454484032 18446744073709551615 4362510 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4203/tid=4206] ppid=4201 vsize=677572 CPUtime=4749.12 cores=0-7
/proc/4203/task/4206/stat : 4206 (claspmt) R 4201 4203 4177 0 -1 4202560 24189 0 0 0 474857 55 0 0 25 0 8 0 642821591 693833728 114892 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670321 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4203/tid=4207] ppid=4201 vsize=677572 CPUtime=4780.94 cores=0-7
/proc/4203/task/4207/stat : 4207 (claspmt) R 4201 4203 4177 0 -1 4202560 7661 0 0 0 478050 44 0 0 25 0 8 0 642821591 693833728 114892 33554432000 4194304 4964800 140737454484032 18446744073709551615 4668728 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4203/tid=4208] ppid=4201 vsize=677572 CPUtime=4776.41 cores=0-7
/proc/4203/task/4208/stat : 4208 (claspmt) R 4201 4203 4177 0 -1 4202560 14702 0 0 0 477599 42 0 0 25 0 8 0 642821591 693833728 114892 33554432000 4194304 4964800 140737454484032 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4203/tid=4209] ppid=4201 vsize=677572 CPUtime=4781.96 cores=0-7
/proc/4203/task/4209/stat : 4209 (claspmt) R 4201 4203 4177 0 -1 4202560 14775 0 0 0 478154 42 0 0 19 0 8 0 642821591 693833728 114892 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4203/tid=4210] ppid=4201 vsize=677572 CPUtime=4781.44 cores=0-7
/proc/4203/task/4210/stat : 4210 (claspmt) R 4201 4203 4177 0 -1 4202560 10473 0 0 0 478085 59 0 0 25 0 8 0 642821591 693833728 114892 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4203/tid=4211] ppid=4201 vsize=677572 CPUtime=4781.51 cores=0-7
/proc/4203/task/4211/stat : 4211 (claspmt) R 4201 4203 4177 0 -1 4202560 15034 0 0 0 478094 57 0 0 25 0 8 0 642821591 693833728 114892 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670357 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38195.8
Current children cumulated vsize (KiB) 677572

[startup+4842.3 s]
/proc/loadavg: 8.07 8.05 8.00 9/371 4368
/proc/meminfo: memFree=22302288/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=677788 CPUtime=38675 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 128033 0 0 0 3867090 407 0 0 25 0 8 0 642821579 694054912 115284 33554432000 4194304 4964800 140737454484032 18446744073709551615 4668801 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4203/statm: 169447 115284 466 189 0 163796 0
[pid=4203/tid=4205] ppid=4201 vsize=677788 CPUtime=4822.3 cores=0-7
/proc/4203/task/4205/stat : 4205 (claspmt) R 4201 4203 4177 0 -1 4202560 32031 0 0 0 482172 58 0 0 25 0 8 0 642821591 694054912 115284 33554432000 4194304 4964800 140737454484032 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4203/tid=4206] ppid=4201 vsize=677788 CPUtime=4808.39 cores=0-7
/proc/4203/task/4206/stat : 4206 (claspmt) R 4201 4203 4177 0 -1 4202560 24192 0 0 0 480784 55 0 0 25 0 8 0 642821591 694054912 115284 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4203/tid=4207] ppid=4201 vsize=677788 CPUtime=4840.91 cores=0-7
/proc/4203/task/4207/stat : 4207 (claspmt) R 4201 4203 4177 0 -1 4202560 7693 0 0 0 484047 44 0 0 25 0 8 0 642821591 694054912 115284 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4203/tid=4208] ppid=4201 vsize=677788 CPUtime=4836.41 cores=0-7
/proc/4203/task/4208/stat : 4208 (claspmt) R 4201 4203 4177 0 -1 4202560 14790 0 0 0 483598 43 0 0 25 0 8 0 642821591 694054912 115284 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4203/tid=4209] ppid=4201 vsize=677788 CPUtime=4841.96 cores=0-7
/proc/4203/task/4209/stat : 4209 (claspmt) R 4201 4203 4177 0 -1 4202560 14829 0 0 0 484153 43 0 0 25 0 8 0 642821591 694054912 115284 33554432000 4194304 4964800 140737454484032 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4203/tid=4210] ppid=4201 vsize=677788 CPUtime=4841.43 cores=0-7
/proc/4203/task/4210/stat : 4210 (claspmt) R 4201 4203 4177 0 -1 4202560 10473 0 0 0 484084 59 0 0 25 0 8 0 642821591 694054912 115284 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4203/tid=4211] ppid=4201 vsize=677788 CPUtime=4841.5 cores=0-7
/proc/4203/task/4211/stat : 4211 (claspmt) R 4201 4203 4177 0 -1 4202560 15139 0 0 0 484093 57 0 0 25 0 8 0 642821591 694054912 115284 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38675
Current children cumulated vsize (KiB) 677788

[startup+4902.3 s]
/proc/loadavg: 8.08 8.06 8.00 9/371 4370
/proc/meminfo: memFree=22301644/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=677980 CPUtime=39154.3 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 128362 0 0 0 3915017 411 0 0 25 0 8 0 642821579 694251520 115449 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4203/statm: 169495 115449 466 189 0 163844 0
[pid=4203/tid=4205] ppid=4201 vsize=677980 CPUtime=4882.31 cores=0-7
/proc/4203/task/4205/stat : 4205 (claspmt) R 4201 4203 4177 0 -1 4202560 32259 0 0 0 488172 59 0 0 25 0 8 0 642821591 694251520 115449 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4203/tid=4206] ppid=4201 vsize=677980 CPUtime=4867.68 cores=0-7
/proc/4203/task/4206/stat : 4206 (claspmt) R 4201 4203 4177 0 -1 4202560 24192 0 0 0 486712 56 0 0 25 0 8 0 642821591 694251520 115449 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4203/tid=4207] ppid=4201 vsize=677980 CPUtime=4900.91 cores=0-7
/proc/4203/task/4207/stat : 4207 (claspmt) R 4201 4203 4177 0 -1 4202560 7736 0 0 0 490046 45 0 0 25 0 8 0 642821591 694251520 115449 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4203/tid=4208] ppid=4201 vsize=677980 CPUtime=4896.42 cores=0-7
/proc/4203/task/4208/stat : 4208 (claspmt) R 4201 4203 4177 0 -1 4202560 14805 0 0 0 489599 43 0 0 25 0 8 0 642821591 694251520 115449 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4203/tid=4209] ppid=4201 vsize=677980 CPUtime=4901.96 cores=0-7
/proc/4203/task/4209/stat : 4209 (claspmt) R 4201 4203 4177 0 -1 4202560 14862 0 0 0 490153 43 0 0 25 0 8 0 642821591 694251520 115449 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4203/tid=4210] ppid=4201 vsize=677980 CPUtime=4901.45 cores=0-7
/proc/4203/task/4210/stat : 4210 (claspmt) R 4201 4203 4177 0 -1 4202560 10474 0 0 0 490085 60 0 0 25 0 8 0 642821591 694251520 115449 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4203/tid=4211] ppid=4201 vsize=677980 CPUtime=4901.51 cores=0-7
/proc/4203/task/4211/stat : 4211 (claspmt) R 4201 4203 4177 0 -1 4202560 15148 0 0 0 490093 58 0 0 25 0 8 0 642821591 694251520 115449 33554432000 4194304 4964800 140737454484032 18446744073709551615 4668709 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39154.3
Current children cumulated vsize (KiB) 677980

[startup+4962.3 s]
/proc/loadavg: 8.08 8.06 8.00 9/371 4372
/proc/meminfo: memFree=22300988/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=678428 CPUtime=39633.5 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 128646 0 0 0 3962931 415 0 0 25 0 8 0 642821579 694710272 115634 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670328 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4203/statm: 169607 115634 466 189 0 163956 0
[pid=4203/tid=4205] ppid=4201 vsize=678428 CPUtime=4941.66 cores=0-7
/proc/4203/task/4205/stat : 4205 (claspmt) R 4201 4203 4177 0 -1 4202560 32423 0 0 0 494107 59 0 0 25 0 8 0 642821591 694710272 115634 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4203/tid=4206] ppid=4201 vsize=678428 CPUtime=4927.58 cores=0-7
/proc/4203/task/4206/stat : 4206 (claspmt) R 4201 4203 4177 0 -1 4202560 24205 0 0 0 492701 57 0 0 25 0 8 0 642821591 694710272 115634 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4203/tid=4207] ppid=4201 vsize=678428 CPUtime=4960.88 cores=0-7
/proc/4203/task/4207/stat : 4207 (claspmt) R 4201 4203 4177 0 -1 4202560 7737 0 0 0 496043 45 0 0 25 0 8 0 642821591 694710272 115634 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4203/tid=4208] ppid=4201 vsize=678428 CPUtime=4956.4 cores=0-7
/proc/4203/task/4208/stat : 4208 (claspmt) R 4201 4203 4177 0 -1 4202560 14805 0 0 0 495597 43 0 0 25 0 8 0 642821591 694710272 115634 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4203/tid=4209] ppid=4201 vsize=678428 CPUtime=4961.94 cores=0-7
/proc/4203/task/4209/stat : 4209 (claspmt) R 4201 4203 4177 0 -1 4202560 14872 0 0 0 496151 43 0 0 25 0 8 0 642821591 694710272 115634 33554432000 4194304 4964800 140737454484032 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4203/tid=4210] ppid=4201 vsize=678428 CPUtime=4961.42 cores=0-7
/proc/4203/task/4210/stat : 4210 (claspmt) R 4201 4203 4177 0 -1 4202560 10517 0 0 0 496082 60 0 0 25 0 8 0 642821591 694710272 115634 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4203/tid=4211] ppid=4201 vsize=678428 CPUtime=4961.5 cores=0-7
/proc/4203/task/4211/stat : 4211 (claspmt) R 4201 4203 4177 0 -1 4202560 15201 0 0 0 496091 59 0 0 25 0 8 0 642821591 694710272 115634 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39633.5
Current children cumulated vsize (KiB) 678428



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5008.2 s]
/proc/loadavg: 8.03 8.05 8.00 9/371 4373
/proc/meminfo: memFree=22299724/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=678428 CPUtime=40000.1 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 129000 0 0 0 3999596 418 0 0 25 0 8 0 642821579 694710272 115889 33554432000 4194304 4964800 140737454484032 18446744073709551615 4659391 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/4203/statm: 169607 115889 466 189 0 163956 0
[pid=4203/tid=4205] ppid=4201 vsize=678428 CPUtime=4987.37 cores=0-7
/proc/4203/task/4205/stat : 4205 (claspmt) R 4201 4203 4177 0 -1 4202560 32522 0 0 0 498678 59 0 0 25 0 8 0 642821591 694710272 115889 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=4203/tid=4206] ppid=4201 vsize=678428 CPUtime=4973.49 cores=0-7
/proc/4203/task/4206/stat : 4206 (claspmt) R 4201 4203 4177 0 -1 4202560 24206 0 0 0 497292 57 0 0 25 0 8 0 642821591 694710272 115889 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4203/tid=4207] ppid=4201 vsize=678428 CPUtime=5006.78 cores=0-7
/proc/4203/task/4207/stat : 4207 (claspmt) R 4201 4203 4177 0 -1 4202560 7737 0 0 0 500633 45 0 0 25 0 8 0 642821591 694710272 115889 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4203/tid=4208] ppid=4201 vsize=678428 CPUtime=5002.31 cores=0-7
/proc/4203/task/4208/stat : 4208 (claspmt) R 4201 4203 4177 0 -1 4202560 14861 0 0 0 500187 44 0 0 25 0 8 0 642821591 694710272 115889 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4203/tid=4209] ppid=4201 vsize=678428 CPUtime=5007.85 cores=0-7
/proc/4203/task/4209/stat : 4209 (claspmt) R 4201 4203 4177 0 -1 4202560 14994 0 0 0 500742 43 0 0 25 0 8 0 642821591 694710272 115889 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4203/tid=4210] ppid=4201 vsize=678428 CPUtime=5006.97 cores=0-7
/proc/4203/task/4210/stat : 4210 (claspmt) R 4201 4203 4177 0 -1 4202560 10522 0 0 0 500637 60 0 0 25 0 8 0 642821591 694710272 115889 33554432000 4194304 4964800 140737454484032 18446744073709551615 4363545 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4203/tid=4211] ppid=4201 vsize=678428 CPUtime=5007.4 cores=0-7
/proc/4203/task/4211/stat : 4211 (claspmt) R 4201 4203 4177 0 -1 4202560 15272 0 0 0 500680 60 0 0 25 0 8 0 642821591 694710272 115889 33554432000 4194304 4964800 140737454484032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 678428

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5008.32 s]
/proc/loadavg: 8.03 8.05 8.00 9/371 4373
/proc/meminfo: memFree=22299724/32951124 swapFree=67057896/67111528
[pid=4203] ppid=4201 vsize=674808 CPUtime=40000.3 cores=0-7
/proc/4203/stat : 4203 (claspmt) R 4201 4203 4177 0 -1 4202496 129010 0 0 0 3999613 418 0 0 25 0 1 0 642821579 691003392 113795 33554432000 4194304 4964800 140737454484032 18446744073709551615 254261272731 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/4203/statm: 168702 113795 476 189 0 163051 0
Current children cumulated CPU time (s) 40000.3
Current children cumulated vsize (KiB) 674808

Child status: 0
Real time (s): 5008.35
CPU time (s): 40000.4
CPU user time (s): 39996.1
CPU system time (s): 4.20736
CPU usage (%): 798.673
Max. virtual memory (cumulated for all children) (KiB): 678428

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39996.1
system time used= 4.20736
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 129011
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 323929
involuntary context switches= 190971

runsolver used 17.0554 second user time and 42.1466 second system time

The end

Launcher Data 

Begin job on node140 at 2011-05-16 02:01:03
IDJOB=3381093
IDBENCH=83335
IDSOLVER=1865
FILE ID=node140/3381093-1305504063
RUNJOBID= node140-1305504062-4191
PBS_JOBID= 13325475
Free space on /tmp= 73456 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/crafted/kullmann/VanDerWaerden/VanderWaerden3k/VanDerWaerden_2-3-18_312.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3381093-1305504063/watcher-3381093-1305504063 -o /tmp/evaluation-result-3381093-1305504063/solver-3381093-1305504063 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381093-1305504063.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 1144bbd60c180aa5e72d35b74e9676cb
RANDOM SEED=352952653

node140.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.14
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      22760472 kB
Buffers:        326036 kB
Cached:        4477008 kB
SwapCached:       8088 kB
Active:        6711152 kB
Inactive:      3147156 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      22760472 kB
SwapTotal:    67111528 kB
SwapFree:     67057896 kB
Dirty:            1716 kB
Writeback:           0 kB
AnonPages:     5052756 kB
Mapped:          29232 kB
Slab:           227948 kB
PageTables:      35992 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 91924188 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73456 MiB
End job on node140 at 2011-05-16 03:24:33