Trace number 3321312

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40001.3 5011.29

General information on the benchmark

Nameindustrial/palacios/hard/
sortnet-7-ipc5-h15-unsat.cnf
MD5SUM0683489092a5eb0c881dd17c2c45d08e
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables105623
Number of clauses374254
Sum of the clauses size911890
Maximum clause length43
Minimum clause length1
Number of clauses of size 1128
Number of clauses of size 2269133
Number of clauses of size 399944
Number of clauses of size 4407
Number of clauses of size 5309
Number of clauses of size over 54333

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...335834/instance-3321312-1304335834.cnf
1.19/1.29	c Solving...
40000.49/5010.52	
40000.49/5010.52	*** INTERRUPTED! ***
40000.49/5010.52	*** Shutting down threads...
40001.24/5011.28	s UNKNOWN
40001.24/5011.28	
40001.24/5011.28	c INTERRUPTED   : 1
40001.24/5011.28	c Models        : 0+    
40001.24/5011.28	c Time          : 5010.527s (Solving: 5009.23s 1st Model: 0.00s Unsat: 0.00s)
40001.24/5011.28	c CPU Time      : 40000.580s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321312-1304335834/watcher-3321312-1304335834 -o /tmp/evaluation-result-3321312-1304335834/solver-3321312-1304335834 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321312-1304335834.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.00 4.55 5.79 2/225 18016
/proc/meminfo: memFree=25106844/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=34384 CPUtime=0 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 1840 0 0 0 0 0 0 0 21 0 1 0 545184358 35209216 1769 33554432000 4194304 4964704 140733239757504 18446744073709551615 4700332 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/18016/statm: 8596 1777 376 189 0 3470 0

[startup+0.108553 s]
/proc/loadavg: 4.00 4.55 5.79 2/225 18016
/proc/meminfo: memFree=25106844/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=57988 CPUtime=0.1 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 12049 0 0 0 9 1 0 0 22 0 1 0 545184358 59379712 9387 33554432000 4194304 4964704 140733239757504 18446744073709551615 270612771525 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/18016/statm: 14497 9389 389 189 0 9371 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 57988

[startup+0.200537 s]
/proc/loadavg: 4.00 4.55 5.79 2/225 18016
/proc/meminfo: memFree=25106844/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=76516 CPUtime=0.19 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 17622 0 0 0 17 2 0 0 22 0 1 0 545184358 78352384 13918 33554432000 4194304 4964704 140733239757504 18446744073709551615 4607585 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/18016/statm: 19129 13918 403 189 0 14003 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 76516

[startup+0.300518 s]
/proc/loadavg: 4.00 4.55 5.79 2/225 18016
/proc/meminfo: memFree=25106844/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=77704 CPUtime=0.29 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 17941 0 0 0 27 2 0 0 22 0 1 0 545184358 79568896 14237 33554432000 4194304 4964704 140733239757504 18446744073709551615 4612110 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/18016/statm: 19426 14238 404 189 0 14300 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 77704

[startup+0.700448 s]
/proc/loadavg: 4.00 4.55 5.79 2/225 18016
/proc/meminfo: memFree=25106844/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=89656 CPUtime=0.69 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 20936 0 0 0 67 2 0 0 24 0 1 0 545184358 91807744 17232 33554432000 4194304 4964704 140733239757504 18446744073709551615 4600112 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/18016/statm: 22414 17232 404 189 0 17288 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 89656

[startup+1.50035 s]
/proc/loadavg: 3.76 4.49 5.77 2/226 18017
/proc/meminfo: memFree=25036412/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=621312 CPUtime=1.67 cores=0-7
/proc/18016/stat : 18016 (claspmt) S 18014 18016 17990 0 -1 4202496 53380 0 0 0 150 17 0 0 25 0 8 0 545184358 636223488 41999 33554432000 4194304 4964704 140733239757504 18446744073709551615 270624926617 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/18016/statm: 155328 41999 438 189 0 149677 0
Current children cumulated CPU time (s) 1.67
Current children cumulated vsize (KiB) 621312

[startup+3.10103 s]
/proc/loadavg: 3.76 4.49 5.77 2/233 18024
/proc/meminfo: memFree=24940396/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=621312 CPUtime=3.27 cores=0-7
/proc/18016/stat : 18016 (claspmt) S 18014 18016 17990 0 -1 4202496 53445 0 0 0 310 17 0 0 25 0 8 0 545184358 636223488 42064 33554432000 4194304 4964704 140733239757504 18446744073709551615 270624926617 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/18016/statm: 155328 42064 438 189 0 149677 0
[pid=18016/tid=18018] ppid=18014 vsize=621312 CPUtime=0.03 cores=0-7
/proc/18016/task/18018/stat : 18018 (claspmt) S 18014 18016 17990 0 -1 4202560 3673 0 0 0 1 2 0 0 18 0 8 0 545184488 636223488 42064 33554432000 4194304 4964704 140733239757504 18446744073709551615 270624926617 0 0 4096 18946 18446604440278207360 0 0 -1 1 0 0 0
[pid=18016/tid=18019] ppid=18014 vsize=621312 CPUtime=0.03 cores=0-7
/proc/18016/task/18019/stat : 18019 (claspmt) S 18014 18016 17990 0 -1 4202560 4800 0 0 0 1 2 0 0 17 0 8 0 545184488 636223488 42064 33554432000 4194304 4964704 140733239757504 18446744073709551615 270624926617 0 0 4096 18946 18446604440278207360 0 0 -1 2 0 0 0
[pid=18016/tid=18020] ppid=18014 vsize=621312 CPUtime=0.02 cores=0-7
/proc/18016/task/18020/stat : 18020 (claspmt) S 18014 18016 17990 0 -1 4202560 4680 0 0 0 1 1 0 0 17 0 8 0 545184488 636223488 42064 33554432000 4194304 4964704 140733239757504 18446744073709551615 270624926617 0 0 4096 18946 18446604440278207360 0 0 -1 4 0 0 0
[pid=18016/tid=18021] ppid=18014 vsize=621312 CPUtime=0.03 cores=0-7
/proc/18016/task/18021/stat : 18021 (claspmt) S 18014 18016 17990 0 -1 4202560 4045 0 0 0 1 2 0 0 17 0 8 0 545184488 636223488 42064 33554432000 4194304 4964704 140733239757504 18446744073709551615 270624926617 0 0 4096 18946 18446604440278207360 0 0 -1 3 0 0 0
[pid=18016/tid=18022] ppid=18014 vsize=621312 CPUtime=1.74 cores=0-7
/proc/18016/task/18022/stat : 18022 (claspmt) R 18014 18016 17990 0 -1 4202560 4490 0 0 0 173 1 0 0 20 0 8 0 545184488 636223488 42064 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18016/tid=18023] ppid=18014 vsize=621312 CPUtime=0.04 cores=0-7
/proc/18016/task/18023/stat : 18023 (claspmt) S 18014 18016 17990 0 -1 4202560 4813 0 0 0 2 2 0 0 18 0 8 0 545184488 636223488 42064 33554432000 4194304 4964704 140733239757504 18446744073709551615 270624926617 0 0 4096 18946 18446604440278207360 0 0 -1 5 0 0 0
[pid=18016/tid=18024] ppid=18014 vsize=621312 CPUtime=0.03 cores=0-7
/proc/18016/task/18024/stat : 18024 (claspmt) S 18014 18016 17990 0 -1 4202560 4406 0 0 0 2 1 0 0 17 0 8 0 545184488 636223488 42064 33554432000 4194304 4964704 140733239757504 18446744073709551615 270624926617 0 0 4096 18946 18446604440278207360 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3.27
Current children cumulated vsize (KiB) 621312

[startup+6.30046 s]
/proc/loadavg: 3.76 4.49 5.77 2/233 18024
/proc/meminfo: memFree=24940628/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=621312 CPUtime=6.47 cores=0-7
/proc/18016/stat : 18016 (claspmt) S 18014 18016 17990 0 -1 4202496 53446 0 0 0 630 17 0 0 25 0 8 0 545184358 636223488 42065 33554432000 4194304 4964704 140733239757504 18446744073709551615 270624926617 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/18016/statm: 155328 42065 438 189 0 149677 0
[pid=18016/tid=18018] ppid=18014 vsize=621312 CPUtime=0.03 cores=0-7
/proc/18016/task/18018/stat : 18018 (claspmt) S 18014 18016 17990 0 -1 4202560 3673 0 0 0 1 2 0 0 18 0 8 0 545184488 636223488 42065 33554432000 4194304 4964704 140733239757504 18446744073709551615 270624926617 0 0 4096 18946 18446604440278207360 0 0 -1 1 0 0 0
[pid=18016/tid=18019] ppid=18014 vsize=621312 CPUtime=0.03 cores=0-7
/proc/18016/task/18019/stat : 18019 (claspmt) S 18014 18016 17990 0 -1 4202560 4800 0 0 0 1 2 0 0 17 0 8 0 545184488 636223488 42065 33554432000 4194304 4964704 140733239757504 18446744073709551615 270624926617 0 0 4096 18946 18446604440278207360 0 0 -1 2 0 0 0
[pid=18016/tid=18020] ppid=18014 vsize=621312 CPUtime=0.02 cores=0-7
/proc/18016/task/18020/stat : 18020 (claspmt) S 18014 18016 17990 0 -1 4202560 4680 0 0 0 1 1 0 0 17 0 8 0 545184488 636223488 42065 33554432000 4194304 4964704 140733239757504 18446744073709551615 270624926617 0 0 4096 18946 18446604440278207360 0 0 -1 4 0 0 0
[pid=18016/tid=18021] ppid=18014 vsize=621312 CPUtime=0.03 cores=0-7
/proc/18016/task/18021/stat : 18021 (claspmt) S 18014 18016 17990 0 -1 4202560 4045 0 0 0 1 2 0 0 17 0 8 0 545184488 636223488 42065 33554432000 4194304 4964704 140733239757504 18446744073709551615 270624926617 0 0 4096 18946 18446604440278207360 0 0 -1 3 0 0 0
[pid=18016/tid=18022] ppid=18014 vsize=621312 CPUtime=4.94 cores=0-7
/proc/18016/task/18022/stat : 18022 (claspmt) R 18014 18016 17990 0 -1 4202560 4491 0 0 0 493 1 0 0 25 0 8 0 545184488 636223488 42065 33554432000 4194304 4964704 140733239757504 18446744073709551615 4627168 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18016/tid=18023] ppid=18014 vsize=621312 CPUtime=0.04 cores=0-7
/proc/18016/task/18023/stat : 18023 (claspmt) S 18014 18016 17990 0 -1 4202560 4813 0 0 0 2 2 0 0 18 0 8 0 545184488 636223488 42065 33554432000 4194304 4964704 140733239757504 18446744073709551615 270624926617 0 0 4096 18946 18446604440278207360 0 0 -1 5 0 0 0
[pid=18016/tid=18024] ppid=18014 vsize=621312 CPUtime=0.03 cores=0-7
/proc/18016/task/18024/stat : 18024 (claspmt) S 18014 18016 17990 0 -1 4202560 4406 0 0 0 2 1 0 0 17 0 8 0 545184488 636223488 42065 33554432000 4194304 4964704 140733239757504 18446744073709551615 270624926617 0 0 4096 18946 18446604440278207360 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 6.47
Current children cumulated vsize (KiB) 621312

[startup+12.7013 s]
/proc/loadavg: 3.90 4.49 5.75 9/233 18025
/proc/meminfo: memFree=24926680/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=621312 CPUtime=55.44 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 57465 0 0 0 5520 24 0 0 25 0 8 0 545184358 636223488 46084 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/18016/statm: 155328 46084 464 189 0 149677 0
[pid=18016/tid=18018] ppid=18014 vsize=621312 CPUtime=6.12 cores=0-7
/proc/18016/task/18018/stat : 18018 (claspmt) R 18014 18016 17990 0 -1 4202560 3951 0 0 0 610 2 0 0 25 0 8 0 545184488 636223488 46084 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18016/tid=18019] ppid=18014 vsize=621312 CPUtime=6.12 cores=0-7
/proc/18016/task/18019/stat : 18019 (claspmt) R 18014 18016 17990 0 -1 4202560 5855 0 0 0 608 4 0 0 25 0 8 0 545184488 636223488 46084 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18016/tid=18020] ppid=18014 vsize=621312 CPUtime=6.11 cores=0-7
/proc/18016/task/18020/stat : 18020 (claspmt) R 18014 18016 17990 0 -1 4202560 5284 0 0 0 609 2 0 0 25 0 8 0 545184488 636223488 46084 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=18016/tid=18021] ppid=18014 vsize=621312 CPUtime=6.12 cores=0-7
/proc/18016/task/18021/stat : 18021 (claspmt) R 18014 18016 17990 0 -1 4202560 4636 0 0 0 609 3 0 0 25 0 8 0 545184488 636223488 46084 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18016/tid=18022] ppid=18014 vsize=621312 CPUtime=11.34 cores=0-7
/proc/18016/task/18022/stat : 18022 (claspmt) R 18014 18016 17990 0 -1 4202560 4504 0 0 0 1133 1 0 0 25 0 8 0 545184488 636223488 46084 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18016/tid=18023] ppid=18014 vsize=621312 CPUtime=6.13 cores=0-7
/proc/18016/task/18023/stat : 18023 (claspmt) R 18014 18016 17990 0 -1 4202560 5321 0 0 0 611 2 0 0 25 0 8 0 545184488 636223488 46084 33554432000 4194304 4964704 140733239757504 18446744073709551615 4361293 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18016/tid=18024] ppid=18014 vsize=621312 CPUtime=6.12 cores=0-7
/proc/18016/task/18024/stat : 18024 (claspmt) R 18014 18016 17990 0 -1 4202560 4779 0 0 0 610 2 0 0 25 0 8 0 545184488 636223488 46084 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668632 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 55.44
Current children cumulated vsize (KiB) 621312

[startup+25.501 s]
/proc/loadavg: 4.53 4.60 5.78 9/233 18025
/proc/meminfo: memFree=24906324/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=621576 CPUtime=157.74 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 62656 0 0 0 15741 33 0 0 25 0 8 0 545184358 636493824 51275 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668942 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/18016/statm: 155394 51275 464 189 0 149743 0
[pid=18016/tid=18018] ppid=18014 vsize=621576 CPUtime=18.92 cores=0-7
/proc/18016/task/18018/stat : 18018 (claspmt) R 18014 18016 17990 0 -1 4202560 4423 0 0 0 1889 3 0 0 25 0 8 0 545184488 636493824 51275 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18016/tid=18019] ppid=18014 vsize=621576 CPUtime=18.92 cores=0-7
/proc/18016/task/18019/stat : 18019 (claspmt) R 18014 18016 17990 0 -1 4202560 7416 0 0 0 1886 6 0 0 25 0 8 0 545184488 636493824 51275 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670114 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18016/tid=18020] ppid=18014 vsize=621576 CPUtime=18.91 cores=0-7
/proc/18016/task/18020/stat : 18020 (claspmt) R 18014 18016 17990 0 -1 4202560 6122 0 0 0 1887 4 0 0 25 0 8 0 545184488 636493824 51275 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=18016/tid=18021] ppid=18014 vsize=621576 CPUtime=18.93 cores=0-7
/proc/18016/task/18021/stat : 18021 (claspmt) R 18014 18016 17990 0 -1 4202560 5778 0 0 0 1888 5 0 0 25 0 8 0 545184488 636493824 51275 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668693 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18016/tid=18022] ppid=18014 vsize=621576 CPUtime=24.14 cores=0-7
/proc/18016/task/18022/stat : 18022 (claspmt) R 18014 18016 17990 0 -1 4202560 4539 0 0 0 2413 1 0 0 25 0 8 0 545184488 636493824 51275 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18016/tid=18023] ppid=18014 vsize=621576 CPUtime=18.94 cores=0-7
/proc/18016/task/18023/stat : 18023 (claspmt) R 18014 18016 17990 0 -1 4202560 5838 0 0 0 1890 4 0 0 25 0 8 0 545184488 636493824 51275 33554432000 4194304 4964704 140733239757504 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18016/tid=18024] ppid=18014 vsize=621576 CPUtime=18.92 cores=0-7
/proc/18016/task/18024/stat : 18024 (claspmt) R 18014 18016 17990 0 -1 4202560 4952 0 0 0 1889 3 0 0 25 0 8 0 545184488 636493824 51275 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 157.74
Current children cumulated vsize (KiB) 621576

[startup+51.1005 s]
/proc/loadavg: 5.71 4.87 5.83 9/233 18026
/proc/meminfo: memFree=24866528/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=622404 CPUtime=362.35 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 72599 0 0 0 36188 47 0 0 25 0 8 0 545184358 637341696 61218 33554432000 4194304 4964704 140733239757504 18446744073709551615 4627263 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/18016/statm: 155601 61218 466 189 0 149950 0
[pid=18016/tid=18018] ppid=18014 vsize=622404 CPUtime=44.52 cores=0-7
/proc/18016/task/18018/stat : 18018 (claspmt) R 18014 18016 17990 0 -1 4202560 4741 0 0 0 4448 4 0 0 25 0 8 0 545184488 637341696 61218 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18016/tid=18019] ppid=18014 vsize=622404 CPUtime=44.52 cores=0-7
/proc/18016/task/18019/stat : 18019 (claspmt) R 18014 18016 17990 0 -1 4202560 11310 0 0 0 4441 11 0 0 25 0 8 0 545184488 637341696 61218 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18016/tid=18020] ppid=18014 vsize=622404 CPUtime=44.51 cores=0-7
/proc/18016/task/18020/stat : 18020 (claspmt) R 18014 18016 17990 0 -1 4202560 6799 0 0 0 4446 5 0 0 25 0 8 0 545184488 637341696 61218 33554432000 4194304 4964704 140733239757504 18446744073709551615 4363056 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=18016/tid=18021] ppid=18014 vsize=622404 CPUtime=44.52 cores=0-7
/proc/18016/task/18021/stat : 18021 (claspmt) R 18014 18016 17990 0 -1 4202560 6786 0 0 0 4446 6 0 0 25 0 8 0 545184488 637341696 61218 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18016/tid=18022] ppid=18014 vsize=622404 CPUtime=49.74 cores=0-7
/proc/18016/task/18022/stat : 18022 (claspmt) R 18014 18016 17990 0 -1 4202560 4605 0 0 0 4973 1 0 0 25 0 8 0 545184488 637341696 61218 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18016/tid=18023] ppid=18014 vsize=622404 CPUtime=44.53 cores=0-7
/proc/18016/task/18023/stat : 18023 (claspmt) R 18014 18016 17990 0 -1 4202560 6788 0 0 0 4448 5 0 0 25 0 8 0 545184488 637341696 61218 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668635 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18016/tid=18024] ppid=18014 vsize=622404 CPUtime=44.52 cores=0-7
/proc/18016/task/18024/stat : 18024 (claspmt) R 18014 18016 17990 0 -1 4202560 7524 0 0 0 4446 6 0 0 25 0 8 0 545184488 637341696 61218 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 362.35
Current children cumulated vsize (KiB) 622404

[startup+102.3 s]
/proc/loadavg: 7.08 5.40 5.95 9/233 18027
/proc/meminfo: memFree=24809228/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=632076 CPUtime=771.63 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 86660 0 0 0 77096 67 0 0 25 0 8 0 545184358 647245824 75279 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668648 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/18016/statm: 158019 75279 466 189 0 152368 0
[pid=18016/tid=18018] ppid=18014 vsize=632076 CPUtime=95.73 cores=0-7
/proc/18016/task/18018/stat : 18018 (claspmt) R 18014 18016 17990 0 -1 4202560 5269 0 0 0 9568 5 0 0 25 0 8 0 545184488 647245824 75279 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668907 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18016/tid=18019] ppid=18014 vsize=632076 CPUtime=95.73 cores=0-7
/proc/18016/task/18019/stat : 18019 (claspmt) R 18014 18016 17990 0 -1 4202560 16728 0 0 0 9554 19 0 0 25 0 8 0 545184488 647245824 75279 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668635 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18016/tid=18020] ppid=18014 vsize=632076 CPUtime=95.72 cores=0-7
/proc/18016/task/18020/stat : 18020 (claspmt) R 18014 18016 17990 0 -1 4202560 7848 0 0 0 9565 7 0 0 25 0 8 0 545184488 647245824 75279 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 4 0 0 0

################
# More data... #
################

[pid=18016] ppid=18014 vsize=2325412 CPUtime=37696.9 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 526564 0 0 0 3769029 665 0 0 25 0 8 0 545184358 2381221888 512056 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670100 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/18016/statm: 581353 512056 466 189 0 575702 0
[pid=18016/tid=18018] ppid=18014 vsize=2325412 CPUtime=4688.87 cores=0-7
/proc/18016/task/18018/stat : 18018 (claspmt) R 18014 18016 17990 0 -1 4202560 14726 0 0 0 468868 19 0 0 25 0 8 0 545184488 2381221888 512056 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18016/tid=18019] ppid=18014 vsize=2325412 CPUtime=4715.8 cores=0-7
/proc/18016/task/18019/stat : 18019 (claspmt) R 18014 18016 17990 0 -1 4202560 120205 0 0 0 471429 151 0 0 25 0 8 0 545184488 2381221888 512056 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670100 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18016/tid=18020] ppid=18014 vsize=2325412 CPUtime=4715.76 cores=0-7
/proc/18016/task/18020/stat : 18020 (claspmt) R 18014 18016 17990 0 -1 4202560 16693 0 0 0 471556 20 0 0 25 0 8 0 545184488 2381221888 512056 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668907 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=18016/tid=18021] ppid=18014 vsize=2325412 CPUtime=4715.8 cores=0-7
/proc/18016/task/18021/stat : 18021 (claspmt) R 18014 18016 17990 0 -1 4202560 63974 0 0 0 471465 115 0 0 25 0 8 0 545184488 2381221888 512056 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18016/tid=18022] ppid=18014 vsize=2325412 CPUtime=4721.03 cores=0-7
/proc/18016/task/18022/stat : 18022 (claspmt) R 18014 18016 17990 0 -1 4202560 36123 0 0 0 472057 46 0 0 25 0 8 0 545184488 2381221888 512056 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18016/tid=18023] ppid=18014 vsize=2325412 CPUtime=4710.75 cores=0-7
/proc/18016/task/18023/stat : 18023 (claspmt) R 18014 18016 17990 0 -1 4202560 14373 0 0 0 471057 18 0 0 25 0 8 0 545184488 2381221888 512056 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18016/tid=18024] ppid=18014 vsize=2325412 CPUtime=4715.13 cores=0-7
/proc/18016/task/18024/stat : 18024 (claspmt) R 18014 18016 17990 0 -1 4202560 229695 0 0 0 471237 276 0 0 25 0 8 0 545184488 2381221888 512056 33554432000 4194304 4964704 140733239757504 18446744073709551615 4669333 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37696.9
Current children cumulated vsize (KiB) 2325412

[startup+4782.3 s]
/proc/loadavg: 8.00 8.01 7.96 9/233 18164
/proc/meminfo: memFree=23035528/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=2345240 CPUtime=38176.4 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 531504 0 0 0 3816977 667 0 0 25 0 8 0 545184358 2401525760 516996 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670100 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/18016/statm: 586310 516996 466 189 0 580659 0
[pid=18016/tid=18018] ppid=18014 vsize=2345240 CPUtime=4748.87 cores=0-7
/proc/18016/task/18018/stat : 18018 (claspmt) R 18014 18016 17990 0 -1 4202560 14727 0 0 0 474868 19 0 0 25 0 8 0 545184488 2401525760 516996 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18016/tid=18019] ppid=18014 vsize=2345240 CPUtime=4775.8 cores=0-7
/proc/18016/task/18019/stat : 18019 (claspmt) R 18014 18016 17990 0 -1 4202560 120206 0 0 0 477429 151 0 0 25 0 8 0 545184488 2401525760 516996 33554432000 4194304 4964704 140733239757504 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18016/tid=18020] ppid=18014 vsize=2345240 CPUtime=4775.76 cores=0-7
/proc/18016/task/18020/stat : 18020 (claspmt) R 18014 18016 17990 0 -1 4202560 16744 0 0 0 477556 20 0 0 25 0 8 0 545184488 2401525760 516996 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668725 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=18016/tid=18021] ppid=18014 vsize=2345240 CPUtime=4775.8 cores=0-7
/proc/18016/task/18021/stat : 18021 (claspmt) R 18014 18016 17990 0 -1 4202560 63975 0 0 0 477465 115 0 0 25 0 8 0 545184488 2401525760 516996 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18016/tid=18022] ppid=18014 vsize=2345240 CPUtime=4781.03 cores=0-7
/proc/18016/task/18022/stat : 18022 (claspmt) R 18014 18016 17990 0 -1 4202560 36123 0 0 0 478057 46 0 0 25 0 8 0 545184488 2401525760 516996 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670100 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18016/tid=18023] ppid=18014 vsize=2345240 CPUtime=4770.31 cores=0-7
/proc/18016/task/18023/stat : 18023 (claspmt) R 18014 18016 17990 0 -1 4202560 14375 0 0 0 477013 18 0 0 25 0 8 0 545184488 2401525760 516996 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668635 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18016/tid=18024] ppid=18014 vsize=2345240 CPUtime=4775.12 cores=0-7
/proc/18016/task/18024/stat : 18024 (claspmt) R 18014 18016 17990 0 -1 4202560 234579 0 0 0 477234 278 0 0 25 0 8 0 545184488 2401525760 516996 33554432000 4194304 4964704 140733239757504 18446744073709551615 4627263 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38176.4
Current children cumulated vsize (KiB) 2345240

[startup+4842.31 s]
/proc/loadavg: 8.00 8.00 7.96 9/233 18165
/proc/meminfo: memFree=23021992/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=2358268 CPUtime=38656.1 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 534785 0 0 0 3864938 670 0 0 25 0 8 0 545184358 2414866432 520277 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/18016/statm: 589567 520277 466 189 0 583916 0
[pid=18016/tid=18018] ppid=18014 vsize=2358268 CPUtime=4808.88 cores=0-7
/proc/18016/task/18018/stat : 18018 (claspmt) R 18014 18016 17990 0 -1 4202560 14727 0 0 0 480869 19 0 0 25 0 8 0 545184488 2414866432 520277 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18016/tid=18019] ppid=18014 vsize=2358268 CPUtime=4835.81 cores=0-7
/proc/18016/task/18019/stat : 18019 (claspmt) R 18014 18016 17990 0 -1 4202560 120207 0 0 0 483430 151 0 0 25 0 8 0 545184488 2414866432 520277 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18016/tid=18020] ppid=18014 vsize=2358268 CPUtime=4835.76 cores=0-7
/proc/18016/task/18020/stat : 18020 (claspmt) R 18014 18016 17990 0 -1 4202560 16765 0 0 0 483556 20 0 0 25 0 8 0 545184488 2414866432 520277 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=18016/tid=18021] ppid=18014 vsize=2358268 CPUtime=4835.81 cores=0-7
/proc/18016/task/18021/stat : 18021 (claspmt) R 18014 18016 17990 0 -1 4202560 63975 0 0 0 483466 115 0 0 25 0 8 0 545184488 2414866432 520277 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18016/tid=18022] ppid=18014 vsize=2358268 CPUtime=4841.04 cores=0-7
/proc/18016/task/18022/stat : 18022 (claspmt) R 18014 18016 17990 0 -1 4202560 36123 0 0 0 484058 46 0 0 25 0 8 0 545184488 2414866432 520277 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18016/tid=18023] ppid=18014 vsize=2358268 CPUtime=4829.88 cores=0-7
/proc/18016/task/18023/stat : 18023 (claspmt) R 18014 18016 17990 0 -1 4202560 14382 0 0 0 482970 18 0 0 25 0 8 0 545184488 2414866432 520277 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18016/tid=18024] ppid=18014 vsize=2358268 CPUtime=4835.11 cores=0-7
/proc/18016/task/18024/stat : 18024 (claspmt) R 18014 18016 17990 0 -1 4202560 237831 0 0 0 483231 280 0 0 25 0 8 0 545184488 2414866432 520277 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38656.1
Current children cumulated vsize (KiB) 2358268

[startup+4902.3 s]
/proc/loadavg: 8.00 8.00 7.96 9/233 18167
/proc/meminfo: memFree=23020736/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=2358268 CPUtime=39135.6 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 535039 0 0 0 3912889 670 0 0 25 0 8 0 545184358 2414866432 520531 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668931 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/18016/statm: 589567 520531 466 189 0 583916 0
[pid=18016/tid=18018] ppid=18014 vsize=2358268 CPUtime=4868.87 cores=0-7
/proc/18016/task/18018/stat : 18018 (claspmt) R 18014 18016 17990 0 -1 4202560 14979 0 0 0 486868 19 0 0 25 0 8 0 545184488 2414866432 520531 33554432000 4194304 4964704 140733239757504 18446744073709551615 4669051 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18016/tid=18019] ppid=18014 vsize=2358268 CPUtime=4895.8 cores=0-7
/proc/18016/task/18019/stat : 18019 (claspmt) R 18014 18016 17990 0 -1 4202560 120207 0 0 0 489429 151 0 0 25 0 8 0 545184488 2414866432 520531 33554432000 4194304 4964704 140733239757504 18446744073709551615 4378499 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18016/tid=18020] ppid=18014 vsize=2358268 CPUtime=4895.76 cores=0-7
/proc/18016/task/18020/stat : 18020 (claspmt) R 18014 18016 17990 0 -1 4202560 16765 0 0 0 489556 20 0 0 25 0 8 0 545184488 2414866432 520531 33554432000 4194304 4964704 140733239757504 18446744073709551615 4669200 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=18016/tid=18021] ppid=18014 vsize=2358268 CPUtime=4895.8 cores=0-7
/proc/18016/task/18021/stat : 18021 (claspmt) R 18014 18016 17990 0 -1 4202560 63976 0 0 0 489465 115 0 0 25 0 8 0 545184488 2414866432 520531 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18016/tid=18022] ppid=18014 vsize=2358268 CPUtime=4901.03 cores=0-7
/proc/18016/task/18022/stat : 18022 (claspmt) R 18014 18016 17990 0 -1 4202560 36123 0 0 0 490057 46 0 0 25 0 8 0 545184488 2414866432 520531 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668560 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18016/tid=18023] ppid=18014 vsize=2358268 CPUtime=4889.45 cores=0-7
/proc/18016/task/18023/stat : 18023 (claspmt) R 18014 18016 17990 0 -1 4202560 14382 0 0 0 488927 18 0 0 25 0 8 0 545184488 2414866432 520531 33554432000 4194304 4964704 140733239757504 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18016/tid=18024] ppid=18014 vsize=2358268 CPUtime=4895.1 cores=0-7
/proc/18016/task/18024/stat : 18024 (claspmt) R 18014 18016 17990 0 -1 4202560 237832 0 0 0 489230 280 0 0 25 0 8 0 545184488 2414866432 520531 33554432000 4194304 4964704 140733239757504 18446744073709551615 4365192 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39135.6
Current children cumulated vsize (KiB) 2358268

[startup+4962.3 s]
/proc/loadavg: 8.00 8.00 7.96 9/233 18169
/proc/meminfo: memFree=23020464/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=2358268 CPUtime=39615.1 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 535098 0 0 0 3960843 670 0 0 25 0 8 0 545184358 2414866432 520590 33554432000 4194304 4964704 140733239757504 18446744073709551615 4669270 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/18016/statm: 589567 520590 466 189 0 583916 0
[pid=18016/tid=18018] ppid=18014 vsize=2358268 CPUtime=4928.87 cores=0-7
/proc/18016/task/18018/stat : 18018 (claspmt) R 18014 18016 17990 0 -1 4202560 14991 0 0 0 492868 19 0 0 25 0 8 0 545184488 2414866432 520590 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18016/tid=18019] ppid=18014 vsize=2358268 CPUtime=4955.8 cores=0-7
/proc/18016/task/18019/stat : 18019 (claspmt) R 18014 18016 17990 0 -1 4202560 120209 0 0 0 495429 151 0 0 25 0 8 0 545184488 2414866432 520590 33554432000 4194304 4964704 140733239757504 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18016/tid=18020] ppid=18014 vsize=2358268 CPUtime=4955.76 cores=0-7
/proc/18016/task/18020/stat : 18020 (claspmt) R 18014 18016 17990 0 -1 4202560 16765 0 0 0 495556 20 0 0 25 0 8 0 545184488 2414866432 520590 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668466 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=18016/tid=18021] ppid=18014 vsize=2358268 CPUtime=4955.8 cores=0-7
/proc/18016/task/18021/stat : 18021 (claspmt) R 18014 18016 17990 0 -1 4202560 64014 0 0 0 495465 115 0 0 25 0 8 0 545184488 2414866432 520590 33554432000 4194304 4964704 140733239757504 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18016/tid=18022] ppid=18014 vsize=2358268 CPUtime=4961.03 cores=0-7
/proc/18016/task/18022/stat : 18022 (claspmt) R 18014 18016 17990 0 -1 4202560 36123 0 0 0 496057 46 0 0 25 0 8 0 545184488 2414866432 520590 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670100 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18016/tid=18023] ppid=18014 vsize=2358268 CPUtime=4949 cores=0-7
/proc/18016/task/18023/stat : 18023 (claspmt) R 18014 18016 17990 0 -1 4202560 14382 0 0 0 494882 18 0 0 25 0 8 0 545184488 2414866432 520590 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668512 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18016/tid=18024] ppid=18014 vsize=2358268 CPUtime=4955.09 cores=0-7
/proc/18016/task/18024/stat : 18024 (claspmt) R 18014 18016 17990 0 -1 4202560 237839 0 0 0 495229 280 0 0 25 0 8 0 545184488 2414866432 520590 33554432000 4194304 4964704 140733239757504 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39615.1
Current children cumulated vsize (KiB) 2358268



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.5 s]
/proc/loadavg: 8.04 8.01 7.97 10/233 18170
/proc/meminfo: memFree=23019820/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=2358268 CPUtime=40000.5 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 535285 0 0 0 3999379 670 0 0 25 0 8 0 545184358 2414866432 520777 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670291 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/18016/statm: 589567 520777 466 189 0 583916 0
[pid=18016/tid=18018] ppid=18014 vsize=2358268 CPUtime=4977.09 cores=0-7
/proc/18016/task/18018/stat : 18018 (claspmt) R 18014 18016 17990 0 -1 4202560 15029 0 0 0 497690 19 0 0 25 0 8 0 545184488 2414866432 520777 33554432000 4194304 4964704 140733239757504 18446744073709551615 4669051 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18016/tid=18019] ppid=18014 vsize=2358268 CPUtime=5004.01 cores=0-7
/proc/18016/task/18019/stat : 18019 (claspmt) R 18014 18016 17990 0 -1 4202560 120295 0 0 0 500250 151 0 0 25 0 8 0 545184488 2414866432 520777 33554432000 4194304 4964704 140733239757504 18446744073709551615 4378499 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18016/tid=18020] ppid=18014 vsize=2358268 CPUtime=5003.97 cores=0-7
/proc/18016/task/18020/stat : 18020 (claspmt) R 18014 18016 17990 0 -1 4202560 16772 0 0 0 500377 20 0 0 25 0 8 0 545184488 2414866432 520777 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668635 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=18016/tid=18021] ppid=18014 vsize=2358268 CPUtime=5004.01 cores=0-7
/proc/18016/task/18021/stat : 18021 (claspmt) R 18014 18016 17990 0 -1 4202560 64038 0 0 0 500286 115 0 0 25 0 8 0 545184488 2414866432 520777 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18016/tid=18022] ppid=18014 vsize=2358268 CPUtime=5009.25 cores=0-7
/proc/18016/task/18022/stat : 18022 (claspmt) R 18014 18016 17990 0 -1 4202560 36123 0 0 0 500879 46 0 0 25 0 8 0 545184488 2414866432 520777 33554432000 4194304 4964704 140733239757504 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18016/tid=18023] ppid=18014 vsize=2358268 CPUtime=4996.86 cores=0-7
/proc/18016/task/18023/stat : 18023 (claspmt) R 18014 18016 17990 0 -1 4202560 14405 0 0 0 499668 18 0 0 25 0 8 0 545184488 2414866432 520777 33554432000 4194304 4964704 140733239757504 18446744073709551615 4660415 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18016/tid=18024] ppid=18014 vsize=2358268 CPUtime=5003.29 cores=0-7
/proc/18016/task/18024/stat : 18024 (claspmt) R 18014 18016 17990 0 -1 4202560 237848 0 0 0 500049 280 0 0 25 0 8 0 545184488 2414866432 520777 33554432000 4194304 4964704 140733239757504 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 2358268

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.7 s]
/proc/loadavg: 8.04 8.01 7.97 10/233 18170
/proc/meminfo: memFree=23019820/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=2334920 CPUtime=40000.7 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 535296 0 0 0 3999403 671 0 0 25 0 1 0 545184358 2390958080 513372 33554432000 4194304 4964704 140733239757504 18446744073709551615 4628834 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/18016/statm: 583730 513372 476 189 0 578079 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 2334920

[startup+5011.1 s]
/proc/loadavg: 8.04 8.01 7.97 10/233 18170
/proc/meminfo: memFree=23019820/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=2303960 CPUtime=40001.1 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 535296 0 0 0 3999443 671 0 0 25 0 1 0 545184358 2359255040 505281 33554432000 4194304 4964704 140733239757504 18446744073709551615 270612768530 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/18016/statm: 575990 505281 476 189 0 570339 0
Current children cumulated CPU time (s) 40001.1
Current children cumulated vsize (KiB) 2303960

[startup+5011.2 s]
/proc/loadavg: 8.04 8.01 7.97 10/233 18170
/proc/meminfo: memFree=23019820/32951124 swapFree=67111516/67111528
[pid=18016] ppid=18014 vsize=2191344 CPUtime=40001.2 cores=0-7
/proc/18016/stat : 18016 (claspmt) R 18014 18016 17990 0 -1 4202496 535296 0 0 0 3999452 672 0 0 25 0 1 0 545184358 2243936256 452862 33554432000 4194304 4964704 140733239757504 18446744073709551615 4646685 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/18016/statm: 547836 452862 476 189 0 542185 0
Current children cumulated CPU time (s) 40001.2
Current children cumulated vsize (KiB) 2191344

Child status: 0
Real time (s): 5011.29
CPU time (s): 40001.3
CPU user time (s): 39994.5
CPU system time (s): 6.81496
CPU usage (%): 798.224
Max. virtual memory (cumulated for all children) (KiB): 2358268

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39994.5
system time used= 6.81496
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 535297
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 20677
involuntary context switches= 188432

runsolver used 9.73052 second user time and 26.7199 second system time

The end

Launcher Data 

Begin job on node108 at 2011-05-02 13:30:34
IDJOB=3321312
IDBENCH=20406
IDSOLVER=1711
FILE ID=node108/3321312-1304335834
RUNJOBID= node108-1304335834-18004
PBS_JOBID= 13178019
Free space on /tmp= 73380 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT07/industrial/palacios/hard/sortnet-7-ipc5-h15-unsat.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321312-1304335834/watcher-3321312-1304335834 -o /tmp/evaluation-result-3321312-1304335834/solver-3321312-1304335834 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3321312-1304335834.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 0683489092a5eb0c881dd17c2c45d08e
RANDOM SEED=962238839

node108.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25107132 kB
Buffers:        660676 kB
Cached:        2084184 kB
SwapCached:         12 kB
Active:        4973608 kB
Inactive:      2455324 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25107132 kB
SwapTotal:    67111528 kB
SwapFree:     67111516 kB
Dirty:            6996 kB
Writeback:           0 kB
AnonPages:     4684480 kB
Mapped:          19908 kB
Slab:           333692 kB
PageTables:      19100 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 20930056 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73380 MiB
End job on node108 at 2011-05-02 14:54:07