Trace number 3373227

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 4800.27 1203.9

General information on the benchmark

NameSAT03/handmade/anton/UNSAT/
SGI_30_70_22_70_7-dir.shuffled-as.sat03-154.cnf
MD5SUMaa9ee2fa26214e80fac6d004d5d0b003
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables660
Number of clauses56654
Sum of the clauses size113924
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 256632
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 522

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...429204/instance-3373227-1305429204.cnf
0.19/0.26	c Solving...
4800.17/1203.83	
4800.17/1203.83	*** INTERRUPTED! ***
4800.17/1203.83	*** Shutting down threads...
4800.17/1203.89	s UNKNOWN
4800.17/1203.89	
4800.17/1203.89	c INTERRUPTED   : 1
4800.17/1203.89	c Models        : 0+    
4800.17/1203.89	c Time          : 1203.830s (Solving: 1203.57s 1st Model: 0.00s Unsat: 0.00s)
4800.17/1203.89	c CPU Time      : 4800.200s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3373227-1305429204/watcher-3373227-1305429204 -o /tmp/evaluation-result-3373227-1305429204/solver-3373227-1305429204 -C 4800 -W 1300 -M 15500 HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373227-1305429204.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.70 5.22 4.94 2/354 29087
/proc/meminfo: memFree=18521980/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=22880 CPUtime=0.01 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 1159 0 0 0 1 0 0 0 22 0 1 0 635336252 23429120 915 33554432000 4194304 4964800 140735423429456 18446744073709551615 233509958341 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/29087/statm: 5720 915 385 189 0 594 0

[startup+0.084996 s]
/proc/loadavg: 3.70 5.22 4.94 2/354 29087
/proc/meminfo: memFree=18521980/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=24408 CPUtime=0.08 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 1575 0 0 0 8 0 0 0 22 0 1 0 635336252 24993792 1325 33554432000 4194304 4964800 140735423429456 18446744073709551615 4607536 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/29087/statm: 6102 1325 398 189 0 976 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 24408

[startup+0.101006 s]
/proc/loadavg: 3.70 5.22 4.94 2/354 29087
/proc/meminfo: memFree=18521980/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=24408 CPUtime=0.09 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 1575 0 0 0 9 0 0 0 22 0 1 0 635336252 24993792 1325 33554432000 4194304 4964800 140735423429456 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/29087/statm: 6102 1325 398 189 0 976 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 24408

[startup+0.308528 s]
/proc/loadavg: 3.70 5.22 4.94 2/354 29087
/proc/meminfo: memFree=18521980/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=236824 CPUtime=0.53 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) D 29085 29087 19731 0 -1 4202496 2581 0 0 0 53 0 0 0 18 0 4 0 635336252 242507776 2218 33554432000 4194304 4964800 140735423429456 18446744073709551615 233510340791 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 59206 2436 458 189 0 53555 0
Current children cumulated CPU time (s) 0.53
Current children cumulated vsize (KiB) 236824

[startup+1.0259 s]
/proc/loadavg: 3.70 5.22 4.94 2/354 29087
/proc/meminfo: memFree=18521980/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=242072 CPUtime=3.28 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 6973 0 0 0 324 4 0 0 19 0 4 0 635336252 247881728 6610 33554432000 4194304 4964800 140735423429456 18446744073709551615 233510340791 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 60518 6610 458 189 0 54867 0
Current children cumulated CPU time (s) 3.28
Current children cumulated vsize (KiB) 242072

[startup+1.50071 s]
/proc/loadavg: 3.70 5.22 4.94 6/361 29097
/proc/meminfo: memFree=18485592/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=243300 CPUtime=5.21 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 8472 0 0 0 516 5 0 0 19 0 4 0 635336252 249139200 8109 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 60825 8109 458 189 0 55174 0
[pid=29087/tid=29095] ppid=29085 vsize=243300 CPUtime=1.23 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 1790 0 0 0 122 1 0 0 19 0 4 0 635336278 249139200 8109 33554432000 4194304 4964800 140735423429456 18446744073709551615 4332418 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=243300 CPUtime=1.23 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 1175 0 0 0 123 0 0 0 19 0 4 0 635336278 249139200 8109 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=243300 CPUtime=1.23 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 1395 0 0 0 123 0 0 0 25 0 4 0 635336278 249139200 8109 33554432000 4194304 4964800 140735423429456 18446744073709551615 4669104 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 5.21
Current children cumulated vsize (KiB) 243300

[startup+3.10043 s]
/proc/loadavg: 3.81 5.21 4.94 6/361 29097
/proc/meminfo: memFree=18476044/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=245436 CPUtime=11.55 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 10964 0 0 0 1149 6 0 0 23 0 4 0 635336252 251326464 10601 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670328 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 61359 10601 460 189 0 55708 0
[pid=29087/tid=29095] ppid=29085 vsize=245436 CPUtime=2.82 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 2621 0 0 0 280 2 0 0 19 0 4 0 635336278 251326464 10601 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670140 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=245436 CPUtime=2.81 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 1803 0 0 0 281 0 0 0 22 0 4 0 635336278 251326464 10601 33554432000 4194304 4964800 140735423429456 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=245436 CPUtime=2.83 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 1736 0 0 0 282 1 0 0 25 0 4 0 635336278 251326464 10601 33554432000 4194304 4964800 140735423429456 18446744073709551615 4325299 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 11.55
Current children cumulated vsize (KiB) 245436

[startup+6.30085 s]
/proc/loadavg: 3.81 5.21 4.94 9/364 29101
/proc/meminfo: memFree=18436408/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=248168 CPUtime=24.31 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 16438 0 0 0 2422 9 0 0 25 0 4 0 635336252 254124032 16075 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320981 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 62042 16075 460 189 0 56391 0
[pid=29087/tid=29095] ppid=29085 vsize=248168 CPUtime=6.01 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 4552 0 0 0 598 3 0 0 25 0 4 0 635336278 254124032 16075 33554432000 4194304 4964800 140735423429456 18446744073709551615 4359840 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=248168 CPUtime=6.02 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 2474 0 0 0 601 1 0 0 25 0 4 0 635336278 254124032 16075 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=248168 CPUtime=6.02 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 3080 0 0 0 601 1 0 0 25 0 4 0 635336278 254124032 16075 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320919 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 24.31
Current children cumulated vsize (KiB) 248168

[startup+12.7007 s]
/proc/loadavg: 4.45 5.31 4.98 9/364 29101
/proc/meminfo: memFree=18395008/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=255072 CPUtime=49.83 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 22620 0 0 0 4972 11 0 0 25 0 4 0 635336252 261193728 22257 33554432000 4194304 4964800 140735423429456 18446744073709551615 4343144 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 63768 22257 460 189 0 58117 0
[pid=29087/tid=29095] ppid=29085 vsize=255072 CPUtime=12.41 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 5228 0 0 0 1238 3 0 0 25 0 4 0 635336278 261193728 22257 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320911 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=255072 CPUtime=12.42 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 3453 0 0 0 1241 1 0 0 25 0 4 0 635336278 261193728 22257 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=255072 CPUtime=12.43 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 3782 0 0 0 1241 2 0 0 25 0 4 0 635336278 261193728 22257 33554432000 4194304 4964800 140735423429456 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 49.83
Current children cumulated vsize (KiB) 255072

[startup+25.5004 s]
/proc/loadavg: 5.00 5.39 5.01 9/364 29101
/proc/meminfo: memFree=18356452/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=268212 CPUtime=100.88 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 30691 0 0 0 10074 14 0 0 25 0 4 0 635336252 274649088 30328 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670346 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 67053 30328 460 189 0 61402 0
[pid=29087/tid=29095] ppid=29085 vsize=268212 CPUtime=25.22 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 6599 0 0 0 2518 4 0 0 25 0 4 0 635336278 274649088 30329 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=268212 CPUtime=25.21 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 4928 0 0 0 2520 1 0 0 25 0 4 0 635336278 274649088 30329 33554432000 4194304 4964800 140735423429456 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=268212 CPUtime=25.22 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 6789 0 0 0 2519 3 0 0 25 0 4 0 635336278 274649088 30329 33554432000 4194304 4964800 140735423429456 18446744073709551615 233509958689 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 100.88
Current children cumulated vsize (KiB) 268212

[startup+51.1138 s]
/proc/loadavg: 6.17 5.63 5.10 9/364 29102
/proc/meminfo: memFree=18297240/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=269648 CPUtime=203.02 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 40274 0 0 0 20281 21 0 0 25 0 4 0 635336252 276119552 39892 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320909 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 67412 39892 460 189 0 61761 0
[pid=29087/tid=29095] ppid=29085 vsize=269648 CPUtime=50.83 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 9687 0 0 0 5077 6 0 0 25 0 4 0 635336278 276119552 39892 33554432000 4194304 4964800 140735423429456 18446744073709551615 233509958689 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=269648 CPUtime=50.83 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 6094 0 0 0 5081 2 0 0 25 0 4 0 635336278 276119552 39892 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=269648 CPUtime=50.84 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 9613 0 0 0 5079 5 0 0 18 0 4 0 635336278 276119552 39892 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 203.02
Current children cumulated vsize (KiB) 269648

[startup+102.301 s]
/proc/loadavg: 7.29 6.01 5.25 9/364 29103
/proc/meminfo: memFree=18215008/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=288088 CPUtime=407.16 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 56465 0 0 0 40682 34 0 0 21 0 4 0 635336252 295002112 55902 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320966 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 72022 55902 460 189 0 66371 0
[pid=29087/tid=29095] ppid=29085 vsize=288088 CPUtime=102.01 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 12403 0 0 0 10192 9 0 0 25 0 4 0 635336278 295002112 55903 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=288088 CPUtime=102.01 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 8021 0 0 0 10199 2 0 0 25 0 4 0 635336278 295002112 55903 33554432000 4194304 4964800 140735423429456 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=288088 CPUtime=102.02 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 17325 0 0 0 10190 12 0 0 25 0 4 0 635336278 295002112 55903 33554432000 4194304 4964800 140735423429456 18446744073709551615 233509958689 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 407.16
Current children cumulated vsize (KiB) 288088

[startup+162.301 s]
/proc/loadavg: 7.89 6.41 5.43 9/364 29105
/proc/meminfo: memFree=18130524/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=497032 CPUtime=646.44 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 75198 0 0 0 64591 53 0 0 25 0 4 0 635336252 508960768 74252 33554432000 4194304 4964800 140735423429456 18446744073709551615 4321102 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 124258 74252 460 189 0 118607 0
[pid=29087/tid=29095] ppid=29085 vsize=497032 CPUtime=162 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 15741 0 0 0 16187 13 0 0 25 0 4 0 635336278 508960768 74252 33554432000 4194304 4964800 140735423429456 18446744073709551615 4659391 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=497032 CPUtime=162.02 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 10960 0 0 0 16198 4 0 0 25 0 4 0 635336278 508960768 74252 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=497032 CPUtime=162.02 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 26510 0 0 0 16179 23 0 0 25 0 4 0 635336278 508960768 74252 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 646.44
Current children cumulated vsize (KiB) 497032

[startup+222.301 s]
/proc/loadavg: 8.03 6.71 5.60 9/364 29106
/proc/meminfo: memFree=18054728/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=545460 CPUtime=885.74 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 93019 0 0 0 88512 62 0 0 25 0 4 0 635336252 558551040 91728 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 136365 91728 460 189 0 130714 0
[pid=29087/tid=29095] ppid=29085 vsize=545460 CPUtime=221.99 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 23544 0 0 0 22182 17 0 0 25 0 4 0 635336278 558551040 91728 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=545460 CPUtime=222.03 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 14328 0 0 0 22198 5 0 0 25 0 4 0 635336278 558551040 91728 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=545460 CPUtime=222.02 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 33081 0 0 0 22175 27 0 0 25 0 4 0 635336278 558551040 91728 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 885.74
Current children cumulated vsize (KiB) 545460

[startup+282.3 s]

################
# More data... #
################

[pid=29087] ppid=29085 vsize=581132 CPUtime=2800.09 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 122436 0 0 0 279917 92 0 0 25 0 4 0 635336252 595079168 120753 33554432000 4194304 4964800 140735423429456 18446744073709551615 4321102 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 145283 120753 460 189 0 139632 0
[pid=29087/tid=29095] ppid=29085 vsize=581132 CPUtime=701.98 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 32491 0 0 0 70173 25 0 0 25 0 4 0 635336278 595079168 120753 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320921 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=581132 CPUtime=702.04 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 20649 0 0 0 70196 8 0 0 25 0 4 0 635336278 595079168 120753 33554432000 4194304 4964800 140735423429456 18446744073709551615 4332201 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=581132 CPUtime=702.03 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 43077 0 0 0 70161 42 0 0 25 0 4 0 635336278 595079168 120753 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2800.09
Current children cumulated vsize (KiB) 581132

[startup+762.301 s]
/proc/loadavg: 8.33 7.98 6.73 9/364 29124
/proc/meminfo: memFree=17890064/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=581132 CPUtime=3039.34 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 123300 0 0 0 303840 94 0 0 25 0 4 0 635336252 595079168 121617 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 145283 121617 460 189 0 139632 0
[pid=29087/tid=29095] ppid=29085 vsize=581132 CPUtime=761.97 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 32640 0 0 0 76172 25 0 0 25 0 4 0 635336278 595079168 121617 33554432000 4194304 4964800 140735423429456 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=581132 CPUtime=762.03 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 20680 0 0 0 76195 8 0 0 25 0 4 0 635336278 595079168 121617 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=581132 CPUtime=762.03 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 43761 0 0 0 76159 44 0 0 25 0 4 0 635336278 595079168 121617 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3039.34
Current children cumulated vsize (KiB) 581132

[startup+822.3 s]
/proc/loadavg: 8.15 8.00 6.81 9/364 29125
/proc/meminfo: memFree=17883096/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=581504 CPUtime=3278.62 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 123421 0 0 0 327768 94 0 0 25 0 4 0 635336252 595460096 121738 33554432000 4194304 4964800 140735423429456 18446744073709551615 4321079 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 145376 121738 460 189 0 139725 0
[pid=29087/tid=29095] ppid=29085 vsize=581504 CPUtime=821.96 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 32729 0 0 0 82171 25 0 0 25 0 4 0 635336278 595460096 121738 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=581504 CPUtime=822.03 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 20702 0 0 0 82195 8 0 0 25 0 4 0 635336278 595460096 121738 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=581504 CPUtime=822.03 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 43761 0 0 0 82159 44 0 0 25 0 4 0 635336278 595460096 121738 33554432000 4194304 4964800 140735423429456 18446744073709551615 4627272 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3278.62
Current children cumulated vsize (KiB) 581504

[startup+882.301 s]
/proc/loadavg: 8.17 8.02 6.89 9/364 29127
/proc/meminfo: memFree=17875504/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=585096 CPUtime=3517.92 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 124471 0 0 0 351697 95 0 0 25 0 4 0 635336252 599138304 122788 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 146274 122788 460 189 0 140623 0
[pid=29087/tid=29095] ppid=29085 vsize=585096 CPUtime=881.95 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 32909 0 0 0 88170 25 0 0 25 0 4 0 635336278 599138304 122788 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=585096 CPUtime=882.03 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 21557 0 0 0 88195 8 0 0 25 0 4 0 635336278 599138304 122788 33554432000 4194304 4964800 140735423429456 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=585096 CPUtime=882.03 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 43775 0 0 0 88159 44 0 0 25 0 4 0 635336278 599138304 122788 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3517.92
Current children cumulated vsize (KiB) 585096

[startup+942.301 s]
/proc/loadavg: 8.09 8.03 6.97 9/364 29129
/proc/meminfo: memFree=17862944/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=585096 CPUtime=3757.19 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 126424 0 0 0 375620 99 0 0 25 0 4 0 635336252 599138304 124741 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 146274 124741 460 189 0 140623 0
[pid=29087/tid=29095] ppid=29085 vsize=585096 CPUtime=941.95 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 33033 0 0 0 94169 26 0 0 25 0 4 0 635336278 599138304 124741 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=585096 CPUtime=942.03 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 21565 0 0 0 94195 8 0 0 25 0 4 0 635336278 599138304 124741 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=585096 CPUtime=942.02 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 45596 0 0 0 94154 48 0 0 25 0 4 0 635336278 599138304 124741 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3757.19
Current children cumulated vsize (KiB) 585096

[startup+1002.3 s]
/proc/loadavg: 8.15 8.05 7.04 9/364 29130
/proc/meminfo: memFree=17840704/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=585096 CPUtime=3996.5 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 131795 0 0 0 399543 107 0 0 25 0 4 0 635336252 599138304 130016 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 146274 130016 460 189 0 140623 0
[pid=29087/tid=29095] ppid=29085 vsize=585096 CPUtime=1001.94 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 33168 0 0 0 100168 26 0 0 25 0 4 0 635336278 599138304 130016 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=585096 CPUtime=1002.03 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 21597 0 0 0 100195 8 0 0 25 0 4 0 635336278 599138304 130016 33554432000 4194304 4964800 140735423429456 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=585096 CPUtime=1002.03 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 45971 0 0 0 100154 49 0 0 25 0 4 0 635336278 599138304 130016 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3996.5
Current children cumulated vsize (KiB) 585096

[startup+1062.3 s]
/proc/loadavg: 8.13 8.06 7.10 9/364 29132
/proc/meminfo: memFree=17838560/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=585096 CPUtime=4235.77 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 131880 0 0 0 423470 107 0 0 25 0 4 0 635336252 599138304 130069 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670129 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 146274 130069 460 189 0 140623 0
[pid=29087/tid=29095] ppid=29085 vsize=585096 CPUtime=1061.94 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 33169 0 0 0 106168 26 0 0 25 0 4 0 635336278 599138304 130069 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=585096 CPUtime=1062.03 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 21665 0 0 0 106195 8 0 0 25 0 4 0 635336278 599138304 130069 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=585096 CPUtime=1062.03 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 45978 0 0 0 106154 49 0 0 25 0 4 0 635336278 599138304 130069 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320953 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4235.77
Current children cumulated vsize (KiB) 585096

[startup+1122.3 s]
/proc/loadavg: 8.27 8.09 7.17 9/364 29133
/proc/meminfo: memFree=17835432/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=585608 CPUtime=4475.05 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 132020 0 0 0 447398 107 0 0 25 0 4 0 635336252 599662592 130209 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 146402 130209 460 189 0 140751 0
[pid=29087/tid=29095] ppid=29085 vsize=585608 CPUtime=1121.92 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 33303 0 0 0 112166 26 0 0 25 0 4 0 635336278 599662592 130209 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=585608 CPUtime=1122.03 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 21667 0 0 0 112195 8 0 0 25 0 4 0 635336278 599662592 130209 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=585608 CPUtime=1122.03 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 45982 0 0 0 112154 49 0 0 25 0 4 0 635336278 599662592 130209 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4475.05
Current children cumulated vsize (KiB) 585608

[startup+1182.3 s]
/proc/loadavg: 8.16 8.09 7.22 9/364 29135
/proc/meminfo: memFree=17816556/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=651144 CPUtime=4714.34 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 136258 0 0 0 471319 115 0 0 25 0 4 0 635336252 666771456 134351 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320928 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 162786 134351 460 189 0 157135 0
[pid=29087/tid=29095] ppid=29085 vsize=651144 CPUtime=1181.92 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 37475 0 0 0 118158 34 0 0 25 0 4 0 635336278 666771456 134351 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=651144 CPUtime=1182.03 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 21716 0 0 0 118195 8 0 0 25 0 4 0 635336278 666771456 134351 33554432000 4194304 4964800 140735423429456 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=651144 CPUtime=1182.03 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 45998 0 0 0 118154 49 0 0 25 0 4 0 635336278 666771456 134351 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4714.34
Current children cumulated vsize (KiB) 651144



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.8 s]
/proc/loadavg: 8.11 8.08 7.24 9/364 29136
/proc/meminfo: memFree=17815068/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=651144 CPUtime=4800.17 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 136489 0 0 0 479901 116 0 0 25 0 4 0 635336252 666771456 134519 33554432000 4194304 4964800 140735423429456 18446744073709551615 4321102 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 162786 134519 460 189 0 157135 0
[pid=29087/tid=29095] ppid=29085 vsize=651144 CPUtime=1203.45 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 37483 0 0 0 120311 34 0 0 25 0 4 0 635336278 666771456 134519 33554432000 4194304 4964800 140735423429456 18446744073709551615 4659391 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=651144 CPUtime=1203.56 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 21856 0 0 0 120347 9 0 0 25 0 4 0 635336278 666771456 134519 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320905 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=651144 CPUtime=1203.56 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 46069 0 0 0 120307 49 0 0 25 0 4 0 635336278 666771456 134519 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.17
Current children cumulated vsize (KiB) 651144

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.8 s]
/proc/loadavg: 8.11 8.08 7.24 9/364 29136
/proc/meminfo: memFree=17815068/32951124 swapFree=67111348/67111528
[pid=29087] ppid=29085 vsize=651144 CPUtime=4800.17 cores=1,3,5,7
/proc/29087/stat : 29087 (claspmt) R 29085 29087 19731 0 -1 4202496 136489 0 0 0 479901 116 0 0 25 0 4 0 635336252 666771456 134519 33554432000 4194304 4964800 140735423429456 18446744073709551615 4321102 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/29087/statm: 162786 134519 460 189 0 157135 0
[pid=29087/tid=29095] ppid=29085 vsize=651144 CPUtime=1203.45 cores=1,3,5,7
/proc/29087/task/29095/stat : 29095 (claspmt) R 29085 29087 19731 0 -1 4202560 37483 0 0 0 120311 34 0 0 25 0 4 0 635336278 666771456 134519 33554432000 4194304 4964800 140735423429456 18446744073709551615 4659391 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=29087/tid=29096] ppid=29085 vsize=651144 CPUtime=1203.56 cores=1,3,5,7
/proc/29087/task/29096/stat : 29096 (claspmt) R 29085 29087 19731 0 -1 4202560 21856 0 0 0 120347 9 0 0 25 0 4 0 635336278 666771456 134519 33554432000 4194304 4964800 140735423429456 18446744073709551615 4320905 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=29087/tid=29097] ppid=29085 vsize=651144 CPUtime=1203.56 cores=1,3,5,7
/proc/29087/task/29097/stat : 29097 (claspmt) R 29085 29087 19731 0 -1 4202560 46069 0 0 0 120307 49 0 0 25 0 4 0 635336278 666771456 134519 33554432000 4194304 4964800 140735423429456 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.17
Current children cumulated vsize (KiB) 651144

Child status: 0
Real time (s): 1203.9
CPU time (s): 4800.27
CPU user time (s): 4799.08
CPU system time (s): 1.18582
CPU usage (%): 398.726
Max. virtual memory (cumulated for all children) (KiB): 651144

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4799.08
system time used= 1.18582
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 136501
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 11391
involuntary context switches= 31715

runsolver used 4.00639 second user time and 10.2424 second system time

The end

Launcher Data 

Begin job on node145 at 2011-05-15 05:13:24
IDJOB=3373227
IDBENCH=83976
IDSOLVER=1853
FILE ID=node145/3373227-1305429204
RUNJOBID= node145-1305429204-29074
PBS_JOBID= 13324541
Free space on /tmp= 73672 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT03/handmade/anton/UNSAT/SGI_30_70_22_70_7-dir.shuffled-as.sat03-154.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3373227-1305429204/watcher-3373227-1305429204 -o /tmp/evaluation-result-3373227-1305429204/solver-3373227-1305429204 -C 4800 -W 1300 -M 15500  HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373227-1305429204.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= aa9ee2fa26214e80fac6d004d5d0b003
RANDOM SEED=1234178077

node145.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      18522260 kB
Buffers:        305728 kB
Cached:        8292244 kB
SwapCached:          0 kB
Active:        7359528 kB
Inactive:      6715304 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      18522260 kB
SwapTotal:    67111528 kB
SwapFree:     67111348 kB
Dirty:            2016 kB
Writeback:           0 kB
AnonPages:     5476908 kB
Mapped:          27080 kB
Slab:           255588 kB
PageTables:      33828 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 79341808 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73672 MiB
End job on node145 at 2011-05-15 05:33:30