Trace number 3373464

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 4800.38 1203.28

General information on the benchmark

Nameindustrial/IBM_FV_2002_13_rule_1/
SAT_dat.k100.cnf
MD5SUMcf1b8194ff6d413d396ac9e26bb031ef
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark202.908
Satisfiable
(Un)Satisfiability was proved
Number of variables50797
Number of clauses177980
Sum of the clauses size470252
Maximum clause length37
Minimum clause length1
Number of clauses of size 11318
Number of clauses of size 295086
Number of clauses of size 374860
Number of clauses of size 4486
Number of clauses of size 5945
Number of clauses of size over 55285

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...418207/instance-3373464-1305418207.cnf
0.29/0.37	c Solving...
4800.17/1203.11	
4800.17/1203.11	*** INTERRUPTED! ***
4800.17/1203.11	*** Shutting down threads...
4800.31/1203.27	s UNKNOWN
4800.31/1203.27	
4800.31/1203.27	c INTERRUPTED   : 1
4800.31/1203.27	c Models        : 0+    
4800.31/1203.27	c Time          : 1203.113s (Solving: 1202.75s 1st Model: 0.00s Unsat: 0.00s)
4800.31/1203.27	c CPU Time      : 4800.210s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3373464-1305418207/watcher-3373464-1305418207 -o /tmp/evaluation-result-3373464-1305418207/solver-3373464-1305418207 -C 4800 -W 1300 -M 15500 HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373464-1305418207.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.44 6.52 7.48 2/292 3680
/proc/meminfo: memFree=13116480/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=20876 CPUtime=0 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 504 0 1 0 0 0 0 0 21 0 1 0 653420360 21377024 436 33554432000 4194304 4964800 140733264785600 18446744073709551615 208080232192 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/3680/statm: 5219 436 372 189 0 93 0

[startup+0.080217 s]
/proc/loadavg: 4.44 6.52 7.48 2/292 3680
/proc/meminfo: memFree=13116480/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=42716 CPUtime=0.07 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 7485 0 1 0 6 1 0 0 21 0 1 0 653420360 43741184 5780 33554432000 4194304 4964800 140733264785600 18446744073709551615 4618467 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/3680/statm: 10679 5781 398 189 0 5553 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 42716

[startup+0.100214 s]
/proc/loadavg: 4.44 6.52 7.48 2/292 3680
/proc/meminfo: memFree=13116480/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=45624 CPUtime=0.09 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 8154 0 1 0 8 1 0 0 21 0 1 0 653420360 46718976 6449 33554432000 4194304 4964800 140733264785600 18446744073709551615 208079888274 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/3680/statm: 11406 6449 399 189 0 6280 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 45624

[startup+0.300177 s]
/proc/loadavg: 4.44 6.52 7.48 2/292 3680
/proc/meminfo: memFree=13116480/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=51088 CPUtime=0.29 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 9532 0 1 0 28 1 0 0 22 0 1 0 653420360 52314112 7827 33554432000 4194304 4964800 140733264785600 18446744073709551615 4621762 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/3680/statm: 12772 7827 403 189 0 7646 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 51088

[startup+0.701103 s]
/proc/loadavg: 4.44 6.52 7.48 2/292 3680
/proc/meminfo: memFree=13116480/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=275556 CPUtime=1.63 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 17978 0 2 0 160 3 0 0 25 0 4 0 653420360 282169344 14230 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 68889 14230 458 189 0 63238 0
Current children cumulated CPU time (s) 1.63
Current children cumulated vsize (KiB) 275556

[startup+1.50095 s]
/proc/loadavg: 4.24 6.44 7.45 6/299 3690
/proc/meminfo: memFree=12956160/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=275556 CPUtime=4.82 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 19216 0 2 0 478 4 0 0 25 0 4 0 653420360 282169344 15468 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 68889 15468 458 189 0 63238 0
[pid=3680/tid=3682] ppid=3678 vsize=275556 CPUtime=1.11 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 2364 0 0 0 111 0 0 0 19 0 4 0 653420398 282169344 15468 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670658 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=275556 CPUtime=1.11 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 3382 0 0 0 110 1 0 0 25 0 4 0 653420398 282169344 15468 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=275556 CPUtime=1.12 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 2921 0 0 0 111 1 0 0 19 0 4 0 653420398 282169344 15468 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4.82
Current children cumulated vsize (KiB) 275556

[startup+3.10064 s]
/proc/loadavg: 4.24 6.44 7.45 6/299 3690
/proc/meminfo: memFree=12927020/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=275556 CPUtime=11.22 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 21468 0 2 0 1115 7 0 0 25 0 4 0 653420360 282169344 17720 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670499 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 68889 17720 458 189 0 63238 0
[pid=3680/tid=3682] ppid=3678 vsize=275556 CPUtime=2.71 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 2389 0 0 0 271 0 0 0 23 0 4 0 653420398 282169344 17720 33554432000 4194304 4964800 140733264785600 18446744073709551615 4370224 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=275556 CPUtime=2.7 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 4149 0 0 0 268 2 0 0 25 0 4 0 653420398 282169344 17720 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=275556 CPUtime=2.72 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 3971 0 0 0 270 2 0 0 23 0 4 0 653420398 282169344 17720 33554432000 4194304 4964800 140733264785600 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.22
Current children cumulated vsize (KiB) 275556

[startup+6.30103 s]
/proc/loadavg: 4.54 6.47 7.46 9/302 3693
/proc/meminfo: memFree=12843464/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=276612 CPUtime=23.99 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 24039 0 2 0 2390 9 0 0 25 0 4 0 653420360 283250688 20291 33554432000 4194304 4964800 140733264785600 18446744073709551615 4374864 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 69153 20291 458 189 0 63502 0
[pid=3680/tid=3682] ppid=3678 vsize=276612 CPUtime=5.91 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 2651 0 0 0 591 0 0 0 25 0 4 0 653420398 283250688 20291 33554432000 4194304 4964800 140733264785600 18446744073709551615 4653507 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=276612 CPUtime=5.87 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 5353 0 0 0 584 3 0 0 25 0 4 0 653420398 283250688 20291 33554432000 4194304 4964800 140733264785600 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=276612 CPUtime=5.92 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 4798 0 0 0 589 3 0 0 25 0 4 0 653420398 283250688 20291 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 23.99
Current children cumulated vsize (KiB) 276612

[startup+12.7008 s]
/proc/loadavg: 4.82 6.49 7.46 9/302 3693
/proc/meminfo: memFree=12809916/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=279364 CPUtime=49.52 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 28993 0 2 0 4939 13 0 0 25 0 4 0 653420360 286068736 25245 33554432000 4194304 4964800 140733264785600 18446744073709551615 4668568 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 69841 25245 458 189 0 64190 0
[pid=3680/tid=3682] ppid=3678 vsize=279364 CPUtime=12.31 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 3330 0 0 0 1231 0 0 0 25 0 4 0 653420398 286068736 25245 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=279364 CPUtime=12.21 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 7253 0 0 0 1216 5 0 0 25 0 4 0 653420398 286068736 25245 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=279364 CPUtime=12.31 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 6383 0 0 0 1228 3 0 0 25 0 4 0 653420398 286068736 25245 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 49.52
Current children cumulated vsize (KiB) 279364

[startup+25.5004 s]
/proc/loadavg: 5.52 6.57 7.47 9/302 3694
/proc/meminfo: memFree=12755836/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=291336 CPUtime=100.6 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 36186 0 2 0 10042 18 0 0 25 0 4 0 653420360 298328064 32438 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670321 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 72834 32438 460 189 0 67183 0
[pid=3680/tid=3682] ppid=3678 vsize=291336 CPUtime=25.11 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 3602 0 0 0 2510 1 0 0 25 0 4 0 653420398 298328064 32438 33554432000 4194304 4964800 140733264785600 18446744073709551615 4370259 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=291336 CPUtime=24.88 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 9947 0 0 0 2482 6 0 0 25 0 4 0 653420398 298328064 32438 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=291336 CPUtime=25.11 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 9615 0 0 0 2506 5 0 0 25 0 4 0 653420398 298328064 32438 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 100.6
Current children cumulated vsize (KiB) 291336

[startup+51.1005 s]
/proc/loadavg: 6.43 6.69 7.48 9/302 3694
/proc/meminfo: memFree=12683580/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=300312 CPUtime=202.77 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 46473 0 2 0 20249 28 0 0 25 0 4 0 653420360 307519488 42725 33554432000 4194304 4964800 140733264785600 18446744073709551615 4668568 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 75078 42725 460 189 0 69427 0
[pid=3680/tid=3682] ppid=3678 vsize=300312 CPUtime=50.71 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 5769 0 0 0 5068 3 0 0 25 0 4 0 653420398 307519488 42725 33554432000 4194304 4964800 140733264785600 18446744073709551615 4669563 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=300312 CPUtime=50.26 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 12641 0 0 0 5017 9 0 0 25 0 4 0 653420398 307519488 42725 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=300312 CPUtime=50.71 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 12994 0 0 0 5063 8 0 0 25 0 4 0 653420398 307519488 42725 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 202.77
Current children cumulated vsize (KiB) 300312

[startup+102.311 s]
/proc/loadavg: 7.32 6.89 7.51 9/302 3696
/proc/meminfo: memFree=12543428/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=378296 CPUtime=407.11 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 65844 0 2 0 40664 47 0 0 25 0 4 0 653420360 387375104 62096 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670328 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 94574 62096 460 189 0 88923 0
[pid=3680/tid=3682] ppid=3678 vsize=378296 CPUtime=101.92 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 8198 0 0 0 10187 5 0 0 25 0 4 0 653420398 387375104 62096 33554432000 4194304 4964800 140733264785600 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=378296 CPUtime=101.08 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 19230 0 0 0 10091 17 0 0 25 0 4 0 653420398 387375104 62096 33554432000 4194304 4964800 140733264785600 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=378296 CPUtime=101.84 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 19074 0 0 0 10170 14 0 0 25 0 4 0 653420398 387375104 62096 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 407.11
Current children cumulated vsize (KiB) 378296

[startup+162.3 s]
/proc/loadavg: 7.81 7.11 7.54 9/302 3697
/proc/meminfo: memFree=12451012/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=390840 CPUtime=646.52 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 70978 0 2 0 64601 51 0 0 25 0 4 0 653420360 400220160 67230 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670180 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 97710 67230 460 189 0 92059 0
[pid=3680/tid=3682] ppid=3678 vsize=390840 CPUtime=161.9 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 10663 0 0 0 16183 7 0 0 25 0 4 0 653420398 400220160 67230 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=390840 CPUtime=161.08 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 20379 0 0 0 16090 18 0 0 25 0 4 0 653420398 400220160 67230 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=390840 CPUtime=161.28 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 19960 0 0 0 16114 14 0 0 25 0 4 0 653420398 400220160 67230 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 646.52
Current children cumulated vsize (KiB) 390840

[startup+222.301 s]
/proc/loadavg: 7.98 7.28 7.57 9/302 3699
/proc/meminfo: memFree=12351656/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=542404 CPUtime=885.97 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 83802 0 2 0 88534 63 0 0 25 0 4 0 653420360 555421696 80011 33554432000 4194304 4964800 140733264785600 18446744073709551615 4321079 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 135601 80011 460 189 0 129950 0
[pid=3680/tid=3682] ppid=3678 vsize=542404 CPUtime=221.91 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 11982 0 0 0 22182 9 0 0 25 0 4 0 653420398 555421696 80011 33554432000 4194304 4964800 140733264785600 18446744073709551615 4683980 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=542404 CPUtime=221.07 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 23403 0 0 0 22088 19 0 0 25 0 4 0 653420398 555421696 80011 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670437 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=542404 CPUtime=220.74 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 25260 0 0 0 22056 18 0 0 25 0 4 0 653420398 555421696 80011 33554432000 4194304 4964800 140733264785600 18446744073709551615 4669140 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 885.97
Current children cumulated vsize (KiB) 542404

[startup+282.3 s]

################
# More data... #
################

Current children cumulated CPU time (s) 2562.12
Current children cumulated vsize (KiB) 619412

[startup+702.3 s]
/proc/loadavg: 8.18 7.96 7.78 9/302 3712
/proc/meminfo: memFree=11725204/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=684948 CPUtime=2801.56 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 147861 0 2 0 280034 122 0 0 25 0 4 0 653420360 701386752 143284 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 171237 143284 460 189 0 165586 0
[pid=3680/tid=3682] ppid=3678 vsize=684948 CPUtime=700.37 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 31709 0 0 0 70013 24 0 0 25 0 4 0 653420398 701386752 143284 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=684948 CPUtime=701.09 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 36340 0 0 0 70082 27 0 0 25 0 4 0 653420398 701386752 143284 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=684948 CPUtime=697.9 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 48375 0 0 0 69745 45 0 0 25 0 4 0 653420398 701386752 143284 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2801.56
Current children cumulated vsize (KiB) 684948

[startup+762.311 s]
/proc/loadavg: 7.89 7.92 7.78 9/302 3743
/proc/meminfo: memFree=12283492/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=750484 CPUtime=3041.03 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 149008 0 2 0 303980 123 0 0 25 0 4 0 653420360 768495616 144431 33554432000 4194304 4964800 140733264785600 18446744073709551615 4660835 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 187621 144431 460 189 0 181970 0
[pid=3680/tid=3682] ppid=3678 vsize=750484 CPUtime=759.83 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 32711 0 0 0 75958 25 0 0 25 0 4 0 653420398 768495616 144431 33554432000 4194304 4964800 140733264785600 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=750484 CPUtime=761.1 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 36340 0 0 0 76082 28 0 0 25 0 4 0 653420398 768495616 144431 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=750484 CPUtime=757.91 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 48520 0 0 0 75746 45 0 0 25 0 4 0 653420398 768495616 144431 33554432000 4194304 4964800 140733264785600 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3041.03
Current children cumulated vsize (KiB) 750484

[startup+822.301 s]
/proc/loadavg: 8.00 7.95 7.80 9/302 3745
/proc/meminfo: memFree=12198028/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=750484 CPUtime=3280.45 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 155006 0 2 0 327912 133 0 0 25 0 4 0 653420360 768495616 150429 33554432000 4194304 4964800 140733264785600 18446744073709551615 4668568 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 187621 150429 460 189 0 181970 0
[pid=3680/tid=3682] ppid=3678 vsize=750484 CPUtime=819.28 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 32712 0 0 0 81903 25 0 0 25 0 4 0 653420398 768495616 150429 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=750484 CPUtime=821.09 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 36341 0 0 0 82081 28 0 0 25 0 4 0 653420398 768495616 150429 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670542 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=750484 CPUtime=817.9 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 53748 0 0 0 81737 53 0 0 25 0 4 0 653420398 768495616 150429 33554432000 4194304 4964800 140733264785600 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3280.45
Current children cumulated vsize (KiB) 750484

[startup+882.301 s]
/proc/loadavg: 8.04 7.97 7.81 9/302 3747
/proc/meminfo: memFree=12082868/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=750484 CPUtime=3519.89 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 160097 0 2 0 351850 139 0 0 25 0 4 0 653420360 768495616 155520 33554432000 4194304 4964800 140733264785600 18446744073709551615 4669479 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 187621 155520 460 189 0 181970 0
[pid=3680/tid=3682] ppid=3678 vsize=750484 CPUtime=878.73 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 32748 0 0 0 87847 26 0 0 25 0 4 0 653420398 768495616 155520 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=750484 CPUtime=881.09 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 36341 0 0 0 88081 28 0 0 25 0 4 0 653420398 768495616 155520 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=750484 CPUtime=877.91 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 53821 0 0 0 87738 53 0 0 25 0 4 0 653420398 768495616 155520 33554432000 4194304 4964800 140733264785600 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3519.89
Current children cumulated vsize (KiB) 750484

[startup+942.301 s]
/proc/loadavg: 8.05 7.99 7.83 9/302 3748
/proc/meminfo: memFree=11975080/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=816020 CPUtime=3759.34 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 165948 0 2 0 375788 146 0 0 25 0 4 0 653420360 835604480 161371 33554432000 4194304 4964800 140733264785600 18446744073709551615 4669350 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 204005 161371 460 189 0 198354 0
[pid=3680/tid=3682] ppid=3678 vsize=816020 CPUtime=938.19 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 32773 0 0 0 93793 26 0 0 25 0 4 0 653420398 835604480 161371 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=816020 CPUtime=941.09 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 36433 0 0 0 94081 28 0 0 25 0 4 0 653420398 835604480 161371 33554432000 4194304 4964800 140733264785600 18446744073709551615 4668756 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=816020 CPUtime=937.9 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 55873 0 0 0 93735 55 0 0 25 0 4 0 653420398 835604480 161371 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3759.34
Current children cumulated vsize (KiB) 816020

[startup+1002.3 s]
/proc/loadavg: 8.12 8.02 7.84 9/302 3750
/proc/meminfo: memFree=11865060/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=818448 CPUtime=3998.78 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 168802 0 2 0 399729 149 0 0 25 0 4 0 653420360 838090752 164225 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670678 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 204612 164225 460 189 0 198961 0
[pid=3680/tid=3682] ppid=3678 vsize=818448 CPUtime=997.65 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 33218 0 0 0 99739 26 0 0 25 0 4 0 653420398 838090752 164225 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=818448 CPUtime=1001.07 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 36433 0 0 0 100079 28 0 0 25 0 4 0 653420398 838090752 164225 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=818448 CPUtime=997.9 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 55875 0 0 0 99735 55 0 0 25 0 4 0 653420398 838090752 164225 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3998.78
Current children cumulated vsize (KiB) 818448

[startup+1062.3 s]
/proc/loadavg: 8.10 8.03 7.85 9/302 3751
/proc/meminfo: memFree=11772768/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=818448 CPUtime=4238.24 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 168804 0 2 0 423674 150 0 0 25 0 4 0 653420360 838090752 164227 33554432000 4194304 4964800 140733264785600 18446744073709551615 4668900 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 204612 164227 460 189 0 198961 0
[pid=3680/tid=3682] ppid=3678 vsize=818448 CPUtime=1057.66 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 33220 0 0 0 105740 26 0 0 25 0 4 0 653420398 838090752 164227 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=818448 CPUtime=1060.54 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 36433 0 0 0 106026 28 0 0 25 0 4 0 653420398 838090752 164227 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=818448 CPUtime=1057.9 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 55875 0 0 0 105734 56 0 0 25 0 4 0 653420398 838090752 164227 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4238.24
Current children cumulated vsize (KiB) 818448

[startup+1122.3 s]
/proc/loadavg: 8.09 8.04 7.87 9/302 3753
/proc/meminfo: memFree=11681720/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=818448 CPUtime=4477.69 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 168820 0 2 0 447619 150 0 0 25 0 4 0 653420360 838090752 164243 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670264 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 204612 164243 460 189 0 198961 0
[pid=3680/tid=3682] ppid=3678 vsize=818448 CPUtime=1117.66 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 33220 0 0 0 111740 26 0 0 25 0 4 0 653420398 838090752 164243 33554432000 4194304 4964800 140733264785600 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=818448 CPUtime=1119.99 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 36436 0 0 0 111971 28 0 0 25 0 4 0 653420398 838090752 164243 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670313 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=818448 CPUtime=1117.9 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 55888 0 0 0 111734 56 0 0 25 0 4 0 653420398 838090752 164243 33554432000 4194304 4964800 140733264785600 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4477.69
Current children cumulated vsize (KiB) 818448

[startup+1182.3 s]
/proc/loadavg: 8.22 8.07 7.89 9/302 3755
/proc/meminfo: memFree=11554088/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=818448 CPUtime=4717.14 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 179802 0 2 0 471550 164 0 0 25 0 4 0 653420360 838090752 175225 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670180 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 204612 175225 460 189 0 198961 0
[pid=3680/tid=3682] ppid=3678 vsize=818448 CPUtime=1177.66 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 33265 0 0 0 117740 26 0 0 25 0 4 0 653420398 838090752 175225 33554432000 4194304 4964800 140733264785600 18446744073709551615 4669507 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=818448 CPUtime=1179.45 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 38118 0 0 0 117914 31 0 0 25 0 4 0 653420398 838090752 175225 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=818448 CPUtime=1177.9 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 59653 0 0 0 117730 60 0 0 25 0 4 0 653420398 838090752 175225 33554432000 4194304 4964800 140733264785600 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4717.14
Current children cumulated vsize (KiB) 818448



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.1 s]
/proc/loadavg: 8.16 8.07 7.89 9/302 3755
/proc/meminfo: memFree=11503232/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=818448 CPUtime=4800.17 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 179851 0 2 0 479853 164 0 0 25 0 4 0 653420360 838090752 175274 33554432000 4194304 4964800 140733264785600 18446744073709551615 4669376 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/3680/statm: 204612 175274 460 189 0 198961 0
[pid=3680/tid=3682] ppid=3678 vsize=818448 CPUtime=1198.47 cores=0,2,4,6
/proc/3680/task/3682/stat : 3682 (claspmt) R 3678 3680 3297 0 -1 4202560 33265 0 0 0 119821 26 0 0 25 0 4 0 653420398 838090752 175274 33554432000 4194304 4964800 140733264785600 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3680/tid=3683] ppid=3678 vsize=818448 CPUtime=1200.07 cores=0,2,4,6
/proc/3680/task/3683/stat : 3683 (claspmt) R 3678 3680 3297 0 -1 4202560 38127 0 0 0 119976 31 0 0 25 0 4 0 653420398 838090752 175274 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670330 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3680/tid=3684] ppid=3678 vsize=818448 CPUtime=1198.71 cores=0,2,4,6
/proc/3680/task/3684/stat : 3684 (claspmt) R 3678 3680 3297 0 -1 4202560 59655 0 0 0 119811 60 0 0 25 0 4 0 653420398 838090752 175274 33554432000 4194304 4964800 140733264785600 18446744073709551615 4670264 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.17
Current children cumulated vsize (KiB) 818448

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.22 s]
/proc/loadavg: 8.16 8.07 7.89 9/302 3755
/proc/meminfo: memFree=11503232/32950928 swapFree=67083212/67111528
[pid=3680] ppid=3678 vsize=803680 CPUtime=4800.31 cores=0,2,4,6
/proc/3680/stat : 3680 (claspmt) R 3678 3680 3297 0 -1 4202496 179861 0 2 0 479867 164 0 0 25 0 1 0 653420360 822968320 171706 33554432000 4194304 4964800 140733264785600 18446744073709551615 208079889386 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/3680/statm: 200920 171706 470 189 0 195269 0
Current children cumulated CPU time (s) 4800.31
Current children cumulated vsize (KiB) 803680

Child status: 0
Real time (s): 1203.28
CPU time (s): 4800.38
CPU user time (s): 4798.7
CPU system time (s): 1.68074
CPU usage (%): 398.942
Max. virtual memory (cumulated for all children) (KiB): 818448

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.7
system time used= 1.68074
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 179863
page faults= 2
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3063
involuntary context switches= 22628

runsolver used 2.98355 second user time and 7.97579 second system time

The end

Launcher Data 

Begin job on node127 at 2011-05-15 02:10:07
IDJOB=3373464
IDBENCH=20536
IDSOLVER=1853
FILE ID=node127/3373464-1305418207
RUNJOBID= node127-1305418207-3666
PBS_JOBID= 13324545
Free space on /tmp= 73468 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT07/industrial/IBM_FV_2002_13_rule_1/SAT_dat.k100.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3373464-1305418207/watcher-3373464-1305418207 -o /tmp/evaluation-result-3373464-1305418207/solver-3373464-1305418207 -C 4800 -W 1300 -M 15500  HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373464-1305418207.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= cf1b8194ff6d413d396ac9e26bb031ef
RANDOM SEED=1012147228

node127.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.60
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      13116644 kB
Buffers:        292416 kB
Cached:        3368380 kB
SwapCached:       1620 kB
Active:       16827192 kB
Inactive:      2671152 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      13116644 kB
SwapTotal:    67111528 kB
SwapFree:     67083212 kB
Dirty:           10108 kB
Writeback:           0 kB
AnonPages:    15836436 kB
Mapped:          25732 kB
Slab:           226556 kB
PageTables:      47148 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 44322612 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73448 MiB
End job on node127 at 2011-05-15 02:30:12