Trace number 3372996

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 4800.4 1203.17

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m37-q505.75-n35-p15-s48276711.cnf
MD5SUM1bac7b96ea6eb843b1de7405a721cd87
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.861868
Satisfiable
(Un)Satisfiability was proved
Number of variables1244
Number of clauses150604
Sum of the clauses size302382
Maximum clause length37
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2150569
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 535

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...432371/instance-3372996-1305432371.cnf
0.59/0.64	c Solving...
4800.21/1203.01	
4800.21/1203.01	*** INTERRUPTED! ***
4800.21/1203.01	*** Shutting down threads...
4800.33/1203.16	s UNKNOWN
4800.33/1203.16	
4800.33/1203.16	c INTERRUPTED   : 1
4800.33/1203.16	c Models        : 0+    
4800.33/1203.16	c Time          : 1203.018s (Solving: 1202.38s 1st Model: 0.00s Unsat: 0.00s)
4800.33/1203.16	c CPU Time      : 4800.240s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372996-1305432371/watcher-3372996-1305432371 -o /tmp/evaluation-result-3372996-1305432371/solver-3372996-1305432371 -C 4800 -W 1300 -M 15500 HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3372996-1305432371.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.02 6.95 4.20 5/253 21741
/proc/meminfo: memFree=21154124/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=21736 CPUtime=0 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 733 0 0 0 0 0 0 0 25 0 1 0 654834967 22257664 655 33554432000 4194304 4964800 140736099779936 18446744073709551615 271550198469 0 0 4096 18946 0 0 0 17 3 0 0 0
/proc/21741/statm: 5434 656 385 189 0 308 0

[startup+0.102411 s]
/proc/loadavg: 8.02 6.95 4.20 5/253 21741
/proc/meminfo: memFree=21154124/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=31192 CPUtime=0.09 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 3686 0 0 0 9 0 0 0 25 0 1 0 654834967 31940608 2853 33554432000 4194304 4964800 140736099779936 18446744073709551615 4607562 0 0 4096 18946 0 0 0 17 3 0 0 0
/proc/21741/statm: 7798 2853 398 189 0 2672 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 31192

[startup+0.200391 s]
/proc/loadavg: 8.02 6.95 4.20 5/253 21741
/proc/meminfo: memFree=21154124/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=31192 CPUtime=0.19 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 3686 0 0 0 19 0 0 0 25 0 1 0 654834967 31940608 2853 33554432000 4194304 4964800 140736099779936 18446744073709551615 4607549 0 0 4096 18946 0 0 0 17 3 0 0 0
/proc/21741/statm: 7798 2853 398 189 0 2672 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 31192

[startup+0.30037 s]
/proc/loadavg: 8.02 6.95 4.20 5/253 21741
/proc/meminfo: memFree=21154124/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=31192 CPUtime=0.29 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 3686 0 0 0 29 0 0 0 25 0 1 0 654834967 31940608 2853 33554432000 4194304 4964800 140736099779936 18446744073709551615 4607585 0 0 4096 18946 0 0 0 17 3 0 0 0
/proc/21741/statm: 7798 2853 398 189 0 2672 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 31192

[startup+0.700262 s]
/proc/loadavg: 8.02 6.95 4.20 5/253 21741
/proc/meminfo: memFree=21154124/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=242704 CPUtime=0.84 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 4688 0 0 0 84 0 0 0 25 0 4 0 654834967 248528896 3559 33554432000 4194304 4964800 140736099779936 18446744073709551615 4627204 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 60676 3560 458 189 0 55025 0
Current children cumulated CPU time (s) 0.84
Current children cumulated vsize (KiB) 242704

[startup+1.50112 s]
/proc/loadavg: 8.02 6.95 4.20 9/257 21745
/proc/meminfo: memFree=21124960/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=243816 CPUtime=4.04 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 9625 0 0 0 401 3 0 0 25 0 4 0 654834967 249667584 8496 33554432000 4194304 4964800 140736099779936 18446744073709551615 4660157 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 60954 8496 458 189 0 55303 0
[pid=21741/tid=21743] ppid=21739 vsize=243816 CPUtime=0.83 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 1153 0 0 0 83 0 0 0 25 0 4 0 654835032 249667584 8496 33554432000 4194304 4964800 140736099779936 18446744073709551615 4627281 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=243816 CPUtime=0.85 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 1204 0 0 0 85 0 0 0 25 0 4 0 654835032 249667584 8496 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320848 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=243816 CPUtime=0.84 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 2124 0 0 0 83 1 0 0 25 0 4 0 654835032 249667584 8496 33554432000 4194304 4964800 140736099779936 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4.04
Current children cumulated vsize (KiB) 243816

[startup+3.10083 s]
/proc/loadavg: 8.02 6.95 4.20 9/257 21745
/proc/meminfo: memFree=21099044/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=248164 CPUtime=10.42 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 14977 0 0 0 1037 5 0 0 25 0 4 0 654834967 254119936 13848 33554432000 4194304 4964800 140736099779936 18446744073709551615 4321102 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 62041 13848 458 189 0 56390 0
[pid=21741/tid=21743] ppid=21739 vsize=248164 CPUtime=2.42 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 2608 0 0 0 240 2 0 0 25 0 4 0 654835032 254119936 13848 33554432000 4194304 4964800 140736099779936 18446744073709551615 4332630 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=248164 CPUtime=2.45 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 1758 0 0 0 245 0 0 0 25 0 4 0 654835032 254119936 13848 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=248164 CPUtime=2.43 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 3518 0 0 0 242 1 0 0 20 0 4 0 654835032 254119936 13848 33554432000 4194304 4964800 140736099779936 18446744073709551615 4627211 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 10.42
Current children cumulated vsize (KiB) 248164

[startup+6.30025 s]
/proc/loadavg: 8.09 6.99 4.23 9/257 21745
/proc/meminfo: memFree=21049188/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=255796 CPUtime=23.18 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 23069 0 0 0 2310 8 0 0 25 0 4 0 654834967 261935104 21940 33554432000 4194304 4964800 140736099779936 18446744073709551615 271550198817 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 63949 21940 460 189 0 58298 0
[pid=21741/tid=21743] ppid=21739 vsize=255796 CPUtime=5.59 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 5191 0 0 0 556 3 0 0 25 0 4 0 654835032 261935104 21940 33554432000 4194304 4964800 140736099779936 18446744073709551615 4627248 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=255796 CPUtime=5.65 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 3383 0 0 0 564 1 0 0 25 0 4 0 654835032 261935104 21940 33554432000 4194304 4964800 140736099779936 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=255796 CPUtime=5.64 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 4288 0 0 0 562 2 0 0 25 0 4 0 654835032 261935104 21940 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 23.18
Current children cumulated vsize (KiB) 255796

[startup+12.7011 s]
/proc/loadavg: 8.09 7.00 4.25 9/257 21746
/proc/meminfo: memFree=21000204/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=264628 CPUtime=48.74 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 31365 0 0 0 4861 13 0 0 25 0 4 0 654834967 270979072 30236 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 66157 30236 460 189 0 60506 0
[pid=21741/tid=21743] ppid=21739 vsize=264628 CPUtime=11.94 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 5962 0 0 0 1190 4 0 0 25 0 4 0 654835032 270979072 30236 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=264628 CPUtime=12.05 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 4310 0 0 0 1204 1 0 0 25 0 4 0 654835032 270979072 30236 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=264628 CPUtime=12.04 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 7021 0 0 0 1201 3 0 0 25 0 4 0 654835032 270979072 30236 33554432000 4194304 4964800 140736099779936 18446744073709551615 4361097 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 48.74
Current children cumulated vsize (KiB) 264628

[startup+25.5008 s]
/proc/loadavg: 8.21 7.08 4.32 9/257 21746
/proc/meminfo: memFree=20947636/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=271864 CPUtime=99.83 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 39593 0 0 0 9964 19 0 0 25 0 4 0 654834967 278388736 38464 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 67966 38464 460 189 0 62315 0
[pid=21741/tid=21743] ppid=21739 vsize=271864 CPUtime=24.63 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 5988 0 0 0 2459 4 0 0 25 0 4 0 654835032 278388736 38464 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670392 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=271864 CPUtime=24.85 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 6084 0 0 0 2484 1 0 0 25 0 4 0 654835032 278388736 38464 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320864 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=271864 CPUtime=24.83 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 10797 0 0 0 2477 6 0 0 25 0 4 0 654835032 278388736 38464 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 99.83
Current children cumulated vsize (KiB) 271864

[startup+51.1011 s]
/proc/loadavg: 8.14 7.16 4.41 9/257 21747
/proc/meminfo: memFree=20852240/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=286416 CPUtime=202.02 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 56561 0 0 0 20171 31 0 0 25 0 4 0 654834967 293289984 55315 33554432000 4194304 4964800 140736099779936 18446744073709551615 4669104 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 71604 55315 460 189 0 65953 0
[pid=21741/tid=21743] ppid=21739 vsize=286416 CPUtime=50.01 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 16430 0 0 0 4991 10 0 0 25 0 4 0 654835032 293289984 55315 33554432000 4194304 4964800 140736099779936 18446744073709551615 4669136 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=286416 CPUtime=50.45 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 6647 0 0 0 5043 2 0 0 25 0 4 0 654835032 293289984 55315 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=286416 CPUtime=50.44 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 16576 0 0 0 5033 11 0 0 25 0 4 0 654835032 293289984 55315 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 202.02
Current children cumulated vsize (KiB) 286416

[startup+102.301 s]
/proc/loadavg: 8.22 7.33 4.62 9/257 21748
/proc/meminfo: memFree=20785136/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=374020 CPUtime=406.41 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 68249 0 0 0 40602 39 0 0 25 0 4 0 654834967 382996480 66888 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320981 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 93505 66888 460 189 0 87854 0
[pid=21741/tid=21743] ppid=21739 vsize=374020 CPUtime=100.8 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 22123 0 0 0 10067 13 0 0 25 0 4 0 654835032 382996480 66888 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=374020 CPUtime=101.65 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 7277 0 0 0 10163 2 0 0 25 0 4 0 654835032 382996480 66888 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=374020 CPUtime=101.65 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 20340 0 0 0 10150 15 0 0 25 0 4 0 654835032 382996480 66888 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 406.41
Current children cumulated vsize (KiB) 374020

[startup+162.301 s]
/proc/loadavg: 8.29 7.51 4.85 9/257 21750
/proc/meminfo: memFree=20612236/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=451188 CPUtime=645.9 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 87965 0 0 0 64533 57 0 0 25 0 4 0 654834967 462016512 86312 33554432000 4194304 4964800 140736099779936 18446744073709551615 4332418 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 112797 86312 460 189 0 107146 0
[pid=21741/tid=21743] ppid=21739 vsize=451188 CPUtime=160.29 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 27971 0 0 0 16012 17 0 0 25 0 4 0 654835032 462016512 86312 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=451188 CPUtime=161.65 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 10782 0 0 0 16161 4 0 0 25 0 4 0 654835032 462016512 86312 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=451188 CPUtime=161.64 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 28872 0 0 0 16138 26 0 0 25 0 4 0 654835032 462016512 86312 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 645.9
Current children cumulated vsize (KiB) 451188

[startup+222.31 s]
/proc/loadavg: 8.15 7.61 5.05 9/257 21751
/proc/meminfo: memFree=20476920/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=538664 CPUtime=885.45 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 100838 0 0 0 88477 68 0 0 25 0 4 0 654834967 551591936 99104 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320966 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 134666 99104 460 189 0 129015 0
[pid=21741/tid=21743] ppid=21739 vsize=538664 CPUtime=219.81 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 31295 0 0 0 21959 22 0 0 25 0 4 0 654835032 551591936 99104 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=538664 CPUtime=221.66 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 14640 0 0 0 22160 6 0 0 25 0 4 0 654835032 551591936 99104 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=538664 CPUtime=221.65 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 33581 0 0 0 22136 29 0 0 25 0 4 0 654835032 551591936 99104 33554432000 4194304 4964800 140736099779936 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 885.45
Current children cumulated vsize (KiB) 538664

[startup+282.301 s]

################
# More data... #
################

Current children cumulated CPU time (s) 2561.92
Current children cumulated vsize (KiB) 786612

[startup+702.301 s]
/proc/loadavg: 8.40 8.13 6.32 9/257 21764
/proc/meminfo: memFree=19945544/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=789576 CPUtime=2801.43 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 173603 0 0 0 279997 146 0 0 25 0 4 0 654834967 808525824 171223 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670346 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 197394 171223 460 189 0 191743 0
[pid=21741/tid=21743] ppid=21739 vsize=789576 CPUtime=695.82 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 57129 0 0 0 69527 55 0 0 25 0 4 0 654835032 808525824 171223 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=789576 CPUtime=701.66 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 29045 0 0 0 70155 11 0 0 25 0 4 0 654835032 808525824 171223 33554432000 4194304 4964800 140736099779936 18446744073709551615 4321042 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=789576 CPUtime=701.63 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 59826 0 0 0 70100 63 0 0 25 0 4 0 654835032 808525824 171223 33554432000 4194304 4964800 140736099779936 18446744073709551615 4321002 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2801.43
Current children cumulated vsize (KiB) 789576

[startup+762.301 s]
/proc/loadavg: 8.24 8.14 6.44 9/257 21766
/proc/meminfo: memFree=19915244/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=857288 CPUtime=3040.96 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 181224 0 0 0 303941 155 0 0 25 0 4 0 654834967 877862912 178026 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320909 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 214322 178026 460 189 0 208671 0
[pid=21741/tid=21743] ppid=21739 vsize=857288 CPUtime=755.32 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 58172 0 0 0 75477 55 0 0 25 0 4 0 654835032 877862912 178026 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=857288 CPUtime=761.67 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 30141 0 0 0 76155 12 0 0 25 0 4 0 654835032 877862912 178026 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=857288 CPUtime=761.64 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 59835 0 0 0 76101 63 0 0 25 0 4 0 654835032 877862912 178026 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3040.96
Current children cumulated vsize (KiB) 857288

[startup+822.301 s]
/proc/loadavg: 8.23 8.15 6.55 9/257 21767
/proc/meminfo: memFree=19867224/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=859528 CPUtime=3280.45 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 183061 0 0 0 327889 156 0 0 25 0 4 0 654834967 880156672 179863 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 214882 179863 460 189 0 209231 0
[pid=21741/tid=21743] ppid=21739 vsize=859528 CPUtime=814.83 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 58998 0 0 0 81427 56 0 0 25 0 4 0 654835032 880156672 179863 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=859528 CPUtime=821.66 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 31132 0 0 0 82153 13 0 0 25 0 4 0 654835032 880156672 179863 33554432000 4194304 4964800 140736099779936 18446744073709551615 4321029 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=859528 CPUtime=821.63 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 59843 0 0 0 82100 63 0 0 25 0 4 0 654835032 880156672 179863 33554432000 4194304 4964800 140736099779936 18446744073709551615 4660163 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3280.45
Current children cumulated vsize (KiB) 859528

[startup+882.31 s]
/proc/loadavg: 8.20 8.15 6.64 9/257 21769
/proc/meminfo: memFree=19675492/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=925492 CPUtime=3520 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 195488 0 0 0 351829 171 0 0 25 0 4 0 654834967 947703808 192290 33554432000 4194304 4964800 140736099779936 18446744073709551615 4332496 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 231373 192290 460 189 0 225722 0
[pid=21741/tid=21743] ppid=21739 vsize=925492 CPUtime=874.34 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 59906 0 0 0 87377 57 0 0 25 0 4 0 654835032 947703808 192290 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=925492 CPUtime=881.68 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 34718 0 0 0 88153 15 0 0 25 0 4 0 654835032 947703808 192290 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=925492 CPUtime=881.64 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 67774 0 0 0 88089 75 0 0 25 0 4 0 654835032 947703808 192290 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3520
Current children cumulated vsize (KiB) 925492

[startup+942.301 s]
/proc/loadavg: 8.30 8.18 6.75 9/257 21770
/proc/meminfo: memFree=19596720/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=925492 CPUtime=3759.45 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 198307 0 0 0 375772 173 0 0 25 0 4 0 654834967 947703808 195109 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320966 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 231373 195109 460 189 0 225722 0
[pid=21741/tid=21743] ppid=21739 vsize=925492 CPUtime=933.82 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 60471 0 0 0 93324 58 0 0 25 0 4 0 654835032 947703808 195109 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670520 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=925492 CPUtime=941.66 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 36527 0 0 0 94151 15 0 0 25 0 4 0 654835032 947703808 195109 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=925492 CPUtime=941.64 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 68061 0 0 0 94088 76 0 0 25 0 4 0 654835032 947703808 195109 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3759.45
Current children cumulated vsize (KiB) 925492

[startup+1002.3 s]
/proc/loadavg: 8.17 8.16 6.83 9/257 21772
/proc/meminfo: memFree=19577220/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=931696 CPUtime=3998.95 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 201223 0 0 0 399721 174 0 0 25 0 4 0 654834967 954056704 198025 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320909 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 232924 198025 460 189 0 227273 0
[pid=21741/tid=21743] ppid=21739 vsize=931696 CPUtime=993.33 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 60730 0 0 0 99275 58 0 0 25 0 4 0 654835032 954056704 198025 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=931696 CPUtime=1001.67 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 38351 0 0 0 100151 16 0 0 25 0 4 0 654835032 954056704 198025 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=931696 CPUtime=1001.64 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 68894 0 0 0 100087 77 0 0 25 0 4 0 654835032 954056704 198025 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3998.95
Current children cumulated vsize (KiB) 931696

[startup+1062.3 s]
/proc/loadavg: 8.35 8.21 6.92 9/257 21774
/proc/meminfo: memFree=19553248/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=937028 CPUtime=4238.46 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 203416 0 0 0 423670 176 0 0 25 0 4 0 654834967 959516672 200218 33554432000 4194304 4964800 140736099779936 18446744073709551615 4627536 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 234257 200218 460 189 0 228606 0
[pid=21741/tid=21743] ppid=21739 vsize=937028 CPUtime=1052.82 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 61216 0 0 0 105224 58 0 0 25 0 4 0 654835032 959516672 200218 33554432000 4194304 4964800 140736099779936 18446744073709551615 4627259 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=937028 CPUtime=1061.67 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 40043 0 0 0 106151 16 0 0 25 0 4 0 654835032 959516672 200218 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670330 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=937028 CPUtime=1061.63 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 68909 0 0 0 106086 77 0 0 25 0 4 0 654835032 959516672 200218 33554432000 4194304 4964800 140736099779936 18446744073709551615 4332656 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4238.46
Current children cumulated vsize (KiB) 937028

[startup+1122.3 s]
/proc/loadavg: 7.03 7.92 6.91 5/248 21793
/proc/meminfo: memFree=20749828/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=1015936 CPUtime=4477.97 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 208019 0 0 0 447618 179 0 0 25 0 4 0 654834967 1040318464 204821 33554432000 4194304 4964800 140736099779936 18446744073709551615 4627378 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 253984 204821 460 189 0 248333 0
[pid=21741/tid=21743] ppid=21739 vsize=1015936 CPUtime=1112.34 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 62138 0 0 0 111175 59 0 0 25 0 4 0 654835032 1040318464 204821 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=1015936 CPUtime=1121.67 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 41360 0 0 0 112150 17 0 0 25 0 4 0 654835032 1040318464 204821 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=1015936 CPUtime=1121.64 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 71262 0 0 0 112085 79 0 0 25 0 4 0 654835032 1040318464 204821 33554432000 4194304 4964800 140736099779936 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4477.97
Current children cumulated vsize (KiB) 1015936

[startup+1182.3 s]
/proc/loadavg: 5.17 7.22 6.73 5/248 21795
/proc/meminfo: memFree=20702560/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=1058640 CPUtime=4717.55 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 220551 0 0 0 471567 188 0 0 25 0 4 0 654834967 1084047360 216537 33554432000 4194304 4964800 140736099779936 18446744073709551615 4627204 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 264660 216537 460 189 0 259009 0
[pid=21741/tid=21743] ppid=21739 vsize=1058640 CPUtime=1171.91 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 67580 0 0 0 117128 63 0 0 25 0 4 0 654835032 1084047360 216537 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=1058640 CPUtime=1181.67 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 43159 0 0 0 118150 17 0 0 25 0 4 0 654835032 1084047360 216537 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=1058640 CPUtime=1181.64 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 76551 0 0 0 118081 83 0 0 25 0 4 0 654835032 1084047360 216537 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670313 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4717.55
Current children cumulated vsize (KiB) 1058640



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203 s]
/proc/loadavg: 4.84 7.01 6.67 5/248 21795
/proc/meminfo: memFree=20668444/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=1088856 CPUtime=4800.21 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 229047 0 0 0 479828 193 0 0 25 0 4 0 654834967 1114988544 225033 33554432000 4194304 4964800 140736099779936 18446744073709551615 4325321 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/21741/statm: 272214 225033 460 189 0 266563 0
[pid=21741/tid=21743] ppid=21739 vsize=1088856 CPUtime=1192.45 cores=1,3,5,7
/proc/21741/task/21743/stat : 21743 (claspmt) R 21739 21741 14333 0 -1 4202560 68306 0 0 0 119182 63 0 0 25 0 4 0 654835032 1114988544 225034 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670295 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21741/tid=21744] ppid=21739 vsize=1088856 CPUtime=1202.39 cores=1,3,5,7
/proc/21741/task/21744/stat : 21744 (claspmt) R 21739 21741 14333 0 -1 4202560 43660 0 0 0 120221 18 0 0 25 0 4 0 654835032 1114988544 225034 33554432000 4194304 4964800 140736099779936 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21741/tid=21745] ppid=21739 vsize=1088856 CPUtime=1202.35 cores=1,3,5,7
/proc/21741/task/21745/stat : 21745 (claspmt) R 21739 21741 14333 0 -1 4202560 77542 0 0 0 120151 84 0 0 25 0 4 0 654835032 1114988544 225034 33554432000 4194304 4964800 140736099779936 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 1088856

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.11 s]
/proc/loadavg: 4.84 7.01 6.67 5/248 21795
/proc/meminfo: memFree=20668444/32951124 swapFree=67056788/67111528
[pid=21741] ppid=21739 vsize=1019516 CPUtime=4800.33 cores=1,3,5,7
/proc/21741/stat : 21741 (claspmt) R 21739 21741 14333 0 -1 4202496 229061 0 0 0 479840 193 0 0 25 0 1 0 654834967 1043984384 221550 33554432000 4194304 4964800 140736099779936 18446744073709551615 271550192843 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/21741/statm: 254879 221550 470 189 0 249228 0
Current children cumulated CPU time (s) 4800.33
Current children cumulated vsize (KiB) 1019516

Child status: 0
Real time (s): 1203.17
CPU time (s): 4800.4
CPU user time (s): 4798.43
CPU system time (s): 1.9737
CPU usage (%): 398.98
Max. virtual memory (cumulated for all children) (KiB): 1088856

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.43
system time used= 1.9737
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 229063
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 9032
involuntary context switches= 22495

runsolver used 2.78058 second user time and 6.99094 second system time

The end

Launcher Data 

Begin job on node133 at 2011-05-15 06:06:11
IDJOB=3372996
IDBENCH=88212
IDSOLVER=1853
FILE ID=node133/3372996-1305432371
RUNJOBID= node133-1305431802-21325
PBS_JOBID= 13324568
Free space on /tmp= 73664 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m37-q505.75-n35-p15-s48276711.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372996-1305432371/watcher-3372996-1305432371 -o /tmp/evaluation-result-3372996-1305432371/solver-3372996-1305432371 -C 4800 -W 1300 -M 15500  HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3372996-1305432371.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 1bac7b96ea6eb843b1de7405a721cd87
RANDOM SEED=592354662

node133.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6741.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21154404 kB
Buffers:        284576 kB
Cached:        3400136 kB
SwapCached:       8724 kB
Active:        8982076 kB
Inactive:      2534820 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21154404 kB
SwapTotal:    67111528 kB
SwapFree:     67056788 kB
Dirty:            2348 kB
Writeback:           0 kB
AnonPages:     7828576 kB
Mapped:          21808 kB
Slab:           194844 kB
PageTables:      24268 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 19638888 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73672 MiB
End job on node133 at 2011-05-15 06:26:16