Trace number 3381262

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40000.7 5011.16

General information on the benchmark

NameSAT05/crafted/sabharwal05/counting/fphp/unsat/
harder/fphp-020-019.shuffled-as.sat05-1232.cnf
MD5SUMabb32bff5eb954e790612a6209864b5d
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark2.08668
Satisfiable
(Un)Satisfiability was proved
Number of variables380
Number of clauses7050
Sum of the clauses size14440
Maximum clause length19
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 27030
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 520

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...510971/instance-3381262-1305510971.cnf
0.00/0.02	c Solving...
40000.58/5011.11	
40000.58/5011.11	*** INTERRUPTED! ***
40000.58/5011.11	*** Shutting down threads...
40000.58/5011.15	s UNKNOWN
40000.58/5011.15	
40000.58/5011.15	c INTERRUPTED   : 1
40000.58/5011.15	c Models        : 0+    
40000.58/5011.15	c Time          : 5011.118s (Solving: 5011.10s 1st Model: 0.00s Unsat: 0.00s)
40000.58/5011.15	c CPU Time      : 40000.670s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3381262-1305510971/watcher-3381262-1305510971 -o /tmp/evaluation-result-3381262-1305510971/solver-3381262-1305510971 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381262-1305510971.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.02 8.04 8.00 2/331 3045
/proc/meminfo: memFree=24287996/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=21180 CPUtime=0 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 593 0 0 0 0 0 0 0 25 0 1 0 741327818 21688320 524 33554432000 4194304 4964800 140736071991888 18446744073709551615 231895490304 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/3045/statm: 5295 525 386 189 0 169 0

[startup+0.0243931 s]
/proc/loadavg: 8.02 8.04 8.00 2/331 3045
/proc/meminfo: memFree=24287996/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=184828 CPUtime=0.02 cores=0-7
/proc/3045/stat : 3045 (claspmt) S 3043 3045 2749 0 -1 4202496 916 0 0 0 2 0 0 0 25 0 8 0 741327818 189263872 834 33554432000 4194304 4964800 140736071991888 18446744073709551615 231907306393 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/3045/statm: 46207 836 435 189 0 40556 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 184828

[startup+0.100817 s]
/proc/loadavg: 8.02 8.04 8.00 2/331 3045
/proc/meminfo: memFree=24287996/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=382500 CPUtime=0.53 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 2506 0 0 0 52 1 0 0 18 0 8 0 741327818 391680000 2424 33554432000 4194304 4964800 140736071991888 18446744073709551615 4627335 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/3045/statm: 95625 2424 464 189 0 89974 0
Current children cumulated CPU time (s) 0.53
Current children cumulated vsize (KiB) 382500

[startup+0.300257 s]
/proc/loadavg: 8.02 8.04 8.00 2/331 3045
/proc/meminfo: memFree=24287996/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=383160 CPUtime=2.09 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 3834 0 0 0 206 3 0 0 18 0 8 0 741327818 392355840 3752 33554432000 4194304 4964800 140736071991888 18446744073709551615 4342186 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/3045/statm: 95790 3752 464 189 0 90139 0
Current children cumulated CPU time (s) 2.09
Current children cumulated vsize (KiB) 383160

[startup+0.700183 s]
/proc/loadavg: 8.02 8.04 8.00 2/331 3045
/proc/meminfo: memFree=24287996/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=385144 CPUtime=5.28 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 5910 0 0 0 524 4 0 0 18 0 8 0 741327818 394387456 5828 33554432000 4194304 4964800 140736071991888 18446744073709551615 4658520 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/3045/statm: 96286 5828 464 189 0 90635 0
Current children cumulated CPU time (s) 5.28
Current children cumulated vsize (KiB) 385144

[startup+1.50103 s]
/proc/loadavg: 8.02 8.04 8.00 9/339 3053
/proc/meminfo: memFree=24264752/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=385548 CPUtime=11.66 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 7908 0 0 0 1160 6 0 0 18 0 8 0 741327818 394801152 7826 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670534 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/3045/statm: 96387 7826 464 189 0 90736 0
[pid=3045/tid=3047] ppid=3043 vsize=385548 CPUtime=1.46 cores=0-7
/proc/3045/task/3047/stat : 3047 (claspmt) R 3043 3045 2749 0 -1 4202560 874 0 0 0 146 0 0 0 19 0 8 0 741327820 394801152 7826 33554432000 4194304 4964800 140736071991888 18446744073709551615 4653382 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=3045/tid=3048] ppid=3043 vsize=385548 CPUtime=1.45 cores=0-7
/proc/3045/task/3048/stat : 3048 (claspmt) R 3043 3045 2749 0 -1 4202560 1214 0 0 0 145 0 0 0 19 0 8 0 741327820 394801152 7826 33554432000 4194304 4964800 140736071991888 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3045/tid=3049] ppid=3043 vsize=385548 CPUtime=1.41 cores=0-7
/proc/3045/task/3049/stat : 3049 (claspmt) R 3043 3045 2749 0 -1 4202560 888 0 0 0 141 0 0 0 25 0 8 0 741327820 394801152 7826 33554432000 4194304 4964800 140736071991888 18446744073709551615 4627536 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3045/tid=3050] ppid=3043 vsize=385548 CPUtime=1.45 cores=0-7
/proc/3045/task/3050/stat : 3050 (claspmt) R 3043 3045 2749 0 -1 4202560 854 0 0 0 145 0 0 0 20 0 8 0 741327820 394801152 7826 33554432000 4194304 4964800 140736071991888 18446744073709551615 4321065 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=3045/tid=3051] ppid=3043 vsize=385548 CPUtime=1.45 cores=0-7
/proc/3045/task/3051/stat : 3051 (claspmt) R 3043 3045 2749 0 -1 4202560 778 0 0 0 144 1 0 0 18 0 8 0 741327820 394801152 7826 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3045/tid=3052] ppid=3043 vsize=385548 CPUtime=1.46 cores=0-7
/proc/3045/task/3052/stat : 3052 (claspmt) R 3043 3045 2749 0 -1 4202560 943 0 0 0 145 1 0 0 18 0 8 0 741327820 394801152 7826 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=3045/tid=3053] ppid=3043 vsize=385548 CPUtime=1.44 cores=0-7
/proc/3045/task/3053/stat : 3053 (claspmt) R 3043 3045 2749 0 -1 4202560 803 0 0 0 144 0 0 0 18 0 8 0 741327820 394801152 7826 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.66
Current children cumulated vsize (KiB) 385548

[startup+3.10073 s]
/proc/loadavg: 8.02 8.04 8.00 9/339 3054
/proc/meminfo: memFree=24250988/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=388644 CPUtime=24.43 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 12091 0 0 0 2435 8 0 0 18 0 8 0 741327818 397971456 12009 33554432000 4194304 4964800 140736071991888 18446744073709551615 4321102 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/3045/statm: 97161 12009 464 189 0 91510 0
[pid=3045/tid=3047] ppid=3043 vsize=388644 CPUtime=3.05 cores=0-7
/proc/3045/task/3047/stat : 3047 (claspmt) R 3043 3045 2749 0 -1 4202560 1184 0 0 0 305 0 0 0 25 0 8 0 741327820 397971456 12009 33554432000 4194304 4964800 140736071991888 18446744073709551615 4364560 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=3045/tid=3048] ppid=3043 vsize=388644 CPUtime=3.05 cores=0-7
/proc/3045/task/3048/stat : 3048 (claspmt) R 3043 3045 2749 0 -1 4202560 1775 0 0 0 305 0 0 0 25 0 8 0 741327820 397971456 12009 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3045/tid=3049] ppid=3043 vsize=388644 CPUtime=3 cores=0-7
/proc/3045/task/3049/stat : 3049 (claspmt) R 3043 3045 2749 0 -1 4202560 1659 0 0 0 300 0 0 0 25 0 8 0 741327820 397971456 12009 33554432000 4194304 4964800 140736071991888 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3045/tid=3050] ppid=3043 vsize=388644 CPUtime=3.05 cores=0-7
/proc/3045/task/3050/stat : 3050 (claspmt) R 3043 3045 2749 0 -1 4202560 1333 0 0 0 305 0 0 0 25 0 8 0 741327820 397971456 12010 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=3045/tid=3051] ppid=3043 vsize=388644 CPUtime=3.06 cores=0-7
/proc/3045/task/3051/stat : 3051 (claspmt) R 3043 3045 2749 0 -1 4202560 1323 0 0 0 304 2 0 0 21 0 8 0 741327820 397971456 12010 33554432000 4194304 4964800 140736071991888 18446744073709551615 231895188094 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3045/tid=3052] ppid=3043 vsize=388644 CPUtime=3.05 cores=0-7
/proc/3045/task/3052/stat : 3052 (claspmt) R 3043 3045 2749 0 -1 4202560 1466 0 0 0 304 1 0 0 21 0 8 0 741327820 397971456 12010 33554432000 4194304 4964800 140736071991888 18446744073709551615 4378499 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=3045/tid=3053] ppid=3043 vsize=388644 CPUtime=3.02 cores=0-7
/proc/3045/task/3053/stat : 3053 (claspmt) R 3043 3045 2749 0 -1 4202560 1309 0 0 0 302 0 0 0 18 0 8 0 741327820 397971456 12010 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670361 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.43
Current children cumulated vsize (KiB) 388644

[startup+6.30114 s]
/proc/loadavg: 8.02 8.04 8.00 9/339 3054
/proc/meminfo: memFree=24235480/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=390412 CPUtime=49.96 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 14124 0 0 0 4986 10 0 0 23 0 8 0 741327818 399781888 14036 33554432000 4194304 4964800 140736071991888 18446744073709551615 4342213 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/3045/statm: 97603 14036 464 189 0 91952 0
[pid=3045/tid=3047] ppid=3043 vsize=390412 CPUtime=6.25 cores=0-7
/proc/3045/task/3047/stat : 3047 (claspmt) R 3043 3045 2749 0 -1 4202560 1395 0 0 0 624 1 0 0 20 0 8 0 741327820 399781888 14036 33554432000 4194304 4964800 140736071991888 18446744073709551615 4627290 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=3045/tid=3048] ppid=3043 vsize=390412 CPUtime=6.24 cores=0-7
/proc/3045/task/3048/stat : 3048 (claspmt) R 3043 3045 2749 0 -1 4202560 2169 0 0 0 624 0 0 0 25 0 8 0 741327820 399781888 14036 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3045/tid=3049] ppid=3043 vsize=390412 CPUtime=6.21 cores=0-7
/proc/3045/task/3049/stat : 3049 (claspmt) R 3043 3045 2749 0 -1 4202560 1946 0 0 0 620 1 0 0 22 0 8 0 741327820 399781888 14036 33554432000 4194304 4964800 140736071991888 18446744073709551615 231895595551 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3045/tid=3050] ppid=3043 vsize=390412 CPUtime=6.24 cores=0-7
/proc/3045/task/3050/stat : 3050 (claspmt) R 3043 3045 2749 0 -1 4202560 1551 0 0 0 624 0 0 0 25 0 8 0 741327820 399781888 14036 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=3045/tid=3051] ppid=3043 vsize=390412 CPUtime=6.25 cores=0-7
/proc/3045/task/3051/stat : 3051 (claspmt) R 3043 3045 2749 0 -1 4202560 1555 0 0 0 623 2 0 0 25 0 8 0 741327820 399781888 14036 33554432000 4194304 4964800 140736071991888 18446744073709551615 4627290 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3045/tid=3052] ppid=3043 vsize=390412 CPUtime=6.26 cores=0-7
/proc/3045/task/3052/stat : 3052 (claspmt) R 3043 3045 2749 0 -1 4202560 1863 0 0 0 624 2 0 0 20 0 8 0 741327820 399781888 14036 33554432000 4194304 4964800 140736071991888 18446744073709551615 4642509 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=3045/tid=3053] ppid=3043 vsize=390412 CPUtime=6.19 cores=0-7
/proc/3045/task/3053/stat : 3053 (claspmt) R 3043 3045 2749 0 -1 4202560 1510 0 0 0 619 0 0 0 25 0 8 0 741327820 399781888 14036 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 49.96
Current children cumulated vsize (KiB) 390412

[startup+12.7009 s]
/proc/loadavg: 8.01 8.04 8.00 9/339 3054
/proc/meminfo: memFree=24217504/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=393732 CPUtime=101.05 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 18177 0 0 0 10089 16 0 0 18 0 8 0 741327818 403181568 18089 33554432000 4194304 4964800 140736071991888 18446744073709551615 4660123 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/3045/statm: 98433 18089 464 189 0 92782 0
[pid=3045/tid=3047] ppid=3043 vsize=393732 CPUtime=12.63 cores=0-7
/proc/3045/task/3047/stat : 3047 (claspmt) R 3043 3045 2749 0 -1 4202560 1813 0 0 0 1261 2 0 0 20 0 8 0 741327820 403181568 18089 33554432000 4194304 4964800 140736071991888 18446744073709551615 4654543 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=3045/tid=3048] ppid=3043 vsize=393732 CPUtime=12.65 cores=0-7
/proc/3045/task/3048/stat : 3048 (claspmt) R 3043 3045 2749 0 -1 4202560 2975 0 0 0 1264 1 0 0 25 0 8 0 741327820 403181568 18089 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3045/tid=3049] ppid=3043 vsize=393732 CPUtime=12.6 cores=0-7
/proc/3045/task/3049/stat : 3049 (claspmt) R 3043 3045 2749 0 -1 4202560 2421 0 0 0 1259 1 0 0 25 0 8 0 741327820 403181568 18089 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3045/tid=3050] ppid=3043 vsize=393732 CPUtime=12.64 cores=0-7
/proc/3045/task/3050/stat : 3050 (claspmt) R 3043 3045 2749 0 -1 4202560 2162 0 0 0 1263 1 0 0 19 0 8 0 741327820 403181568 18089 33554432000 4194304 4964800 140736071991888 18446744073709551615 4669017 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=3045/tid=3051] ppid=3043 vsize=393732 CPUtime=12.64 cores=0-7
/proc/3045/task/3051/stat : 3051 (claspmt) R 3043 3045 2749 0 -1 4202560 1927 0 0 0 1262 2 0 0 20 0 8 0 741327820 403181568 18089 33554432000 4194304 4964800 140736071991888 18446744073709551615 4684107 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3045/tid=3052] ppid=3043 vsize=393732 CPUtime=12.66 cores=0-7
/proc/3045/task/3052/stat : 3052 (claspmt) R 3043 3045 2749 0 -1 4202560 2550 0 0 0 1263 3 0 0 25 0 8 0 741327820 403181568 18089 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=3045/tid=3053] ppid=3043 vsize=393732 CPUtime=12.51 cores=0-7
/proc/3045/task/3053/stat : 3053 (claspmt) R 3043 3045 2749 0 -1 4202560 1830 0 0 0 1251 0 0 0 19 0 8 0 741327820 403181568 18089 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670520 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.05
Current children cumulated vsize (KiB) 393732

[startup+25.5005 s]
/proc/loadavg: 8.01 8.03 8.00 9/339 3054
/proc/meminfo: memFree=24195968/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=397640 CPUtime=203.21 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 23690 0 0 0 20297 24 0 0 19 0 8 0 741327818 407183360 23533 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320955 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/3045/statm: 99410 23533 464 189 0 93759 0
[pid=3045/tid=3047] ppid=3043 vsize=397640 CPUtime=25.4 cores=0-7
/proc/3045/task/3047/stat : 3047 (claspmt) R 3043 3045 2749 0 -1 4202560 2542 0 0 0 2537 3 0 0 25 0 8 0 741327820 407183360 23533 33554432000 4194304 4964800 140736071991888 18446744073709551615 4654552 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=3045/tid=3048] ppid=3043 vsize=397640 CPUtime=25.43 cores=0-7
/proc/3045/task/3048/stat : 3048 (claspmt) R 3043 3045 2749 0 -1 4202560 3941 0 0 0 2542 1 0 0 20 0 8 0 741327820 407183360 23533 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3045/tid=3049] ppid=3043 vsize=397640 CPUtime=25.39 cores=0-7
/proc/3045/task/3049/stat : 3049 (claspmt) R 3043 3045 2749 0 -1 4202560 3036 0 0 0 2537 2 0 0 25 0 8 0 741327820 407183360 23533 33554432000 4194304 4964800 140736071991888 18446744073709551615 4658556 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3045/tid=3050] ppid=3043 vsize=397640 CPUtime=25.43 cores=0-7
/proc/3045/task/3050/stat : 3050 (claspmt) R 3043 3045 2749 0 -1 4202560 3012 0 0 0 2541 2 0 0 25 0 8 0 741327820 407183360 23533 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=3045/tid=3051] ppid=3043 vsize=397640 CPUtime=25.43 cores=0-7
/proc/3045/task/3051/stat : 3051 (claspmt) R 3043 3045 2749 0 -1 4202560 2875 0 0 0 2540 3 0 0 25 0 8 0 741327820 407183360 23533 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670313 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3045/tid=3052] ppid=3043 vsize=397640 CPUtime=25.44 cores=0-7
/proc/3045/task/3052/stat : 3052 (claspmt) R 3043 3045 2749 0 -1 4202560 3217 0 0 0 2540 4 0 0 25 0 8 0 741327820 407183360 23533 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320993 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=3045/tid=3053] ppid=3043 vsize=397640 CPUtime=25.16 cores=0-7
/proc/3045/task/3053/stat : 3053 (claspmt) R 3043 3045 2749 0 -1 4202560 2082 0 0 0 2515 1 0 0 25 0 8 0 741327820 407183360 23533 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.21
Current children cumulated vsize (KiB) 397640

[startup+51.1008 s]
/proc/loadavg: 8.07 8.04 8.00 9/339 3055
/proc/meminfo: memFree=24168184/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=402504 CPUtime=407.56 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 30731 0 0 0 40718 38 0 0 25 0 8 0 741327818 412164096 30574 33554432000 4194304 4964800 140736071991888 18446744073709551615 4312171 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/3045/statm: 100626 30574 464 189 0 94975 0
[pid=3045/tid=3047] ppid=3043 vsize=402504 CPUtime=50.96 cores=0-7
/proc/3045/task/3047/stat : 3047 (claspmt) R 3043 3045 2749 0 -1 4202560 3243 0 0 0 5090 6 0 0 25 0 8 0 741327820 412164096 30574 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=3045/tid=3048] ppid=3043 vsize=402504 CPUtime=51.01 cores=0-7
/proc/3045/task/3048/stat : 3048 (claspmt) R 3043 3045 2749 0 -1 4202560 5096 0 0 0 5098 3 0 0 25 0 8 0 741327820 412164096 30574 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320947 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3045/tid=3049] ppid=3043 vsize=402504 CPUtime=50.98 cores=0-7
/proc/3045/task/3049/stat : 3049 (claspmt) R 3043 3045 2749 0 -1 4202560 3738 0 0 0 5094 4 0 0 21 0 8 0 741327820 412164096 30574 33554432000 4194304 4964800 140736071991888 18446744073709551615 4325305 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3045/tid=3050] ppid=3043 vsize=402504 CPUtime=51 cores=0-7
/proc/3045/task/3050/stat : 3050 (claspmt) R 3043 3045 2749 0 -1 4202560 3919 0 0 0 5096 4 0 0 25 0 8 0 741327820 412164096 30574 33554432000 4194304 4964800 140736071991888 18446744073709551615 4321097 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=3045/tid=3051] ppid=3043 vsize=402504 CPUtime=51.01 cores=0-7
/proc/3045/task/3051/stat : 3051 (claspmt) R 3043 3045 2749 0 -1 4202560 4040 0 0 0 5096 5 0 0 25 0 8 0 741327820 412164096 30574 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3045/tid=3052] ppid=3043 vsize=402504 CPUtime=51.02 cores=0-7
/proc/3045/task/3052/stat : 3052 (claspmt) R 3043 3045 2749 0 -1 4202560 4159 0 0 0 5096 6 0 0 25 0 8 0 741327820 412164096 30574 33554432000 4194304 4964800 140736071991888 18446744073709551615 4342199 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=3045/tid=3053] ppid=3043 vsize=402504 CPUtime=50.47 cores=0-7
/proc/3045/task/3053/stat : 3053 (claspmt) R 3043 3045 2749 0 -1 4202560 2645 0 0 0 5045 2 0 0 25 0 8 0 741327820 412164096 30574 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 407.56

################
# More data... #
################


[startup+4722.3 s]
/proc/loadavg: 8.01 8.03 8.05 9/339 3714
/proc/meminfo: memFree=23949500/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=430992 CPUtime=37694.9 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 92076 0 0 0 3767795 1698 0 0 22 0 8 0 741327818 441335808 74723 33554432000 4194304 4964800 140736071991888 18446744073709551615 4653304 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/3045/statm: 107748 74723 464 189 0 102097 0
[pid=3045/tid=3047] ppid=3043 vsize=430992 CPUtime=4705.85 cores=0-7
/proc/3045/task/3047/stat : 3047 (claspmt) R 3043 3045 2749 0 -1 4202560 12040 0 0 0 470271 314 0 0 25 0 8 0 741327820 441335808 74723 33554432000 4194304 4964800 140736071991888 18446744073709551615 4327267 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=3045/tid=3048] ppid=3043 vsize=430992 CPUtime=4703.12 cores=0-7
/proc/3045/task/3048/stat : 3048 (claspmt) R 3043 3045 2749 0 -1 4202560 16184 0 0 0 470094 218 0 0 25 0 8 0 741327820 441335808 74723 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3045/tid=3049] ppid=3043 vsize=430992 CPUtime=4713.2 cores=0-7
/proc/3045/task/3049/stat : 3049 (claspmt) R 3043 3045 2749 0 -1 4202560 12427 0 0 0 471108 212 0 0 25 0 8 0 741327820 441335808 74723 33554432000 4194304 4964800 140736071991888 18446744073709551615 4660118 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=3045/tid=3050] ppid=3043 vsize=430992 CPUtime=4711.38 cores=0-7
/proc/3045/task/3050/stat : 3050 (claspmt) R 3043 3045 2749 0 -1 4202560 11065 0 0 0 470931 207 0 0 19 0 8 0 741327820 441335808 74723 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=3045/tid=3051] ppid=3043 vsize=430992 CPUtime=4718.01 cores=0-7
/proc/3045/task/3051/stat : 3051 (claspmt) R 3043 3045 2749 0 -1 4202560 11190 0 0 0 471640 161 0 0 25 0 8 0 741327820 441335808 74723 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670357 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3045/tid=3052] ppid=3043 vsize=430992 CPUtime=4715.49 cores=0-7
/proc/3045/task/3052/stat : 3052 (claspmt) R 3043 3045 2749 0 -1 4202560 12117 0 0 0 471347 202 0 0 25 0 8 0 741327820 441335808 74723 33554432000 4194304 4964800 140736071991888 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3045/tid=3053] ppid=3043 vsize=430992 CPUtime=4712.45 cores=0-7
/proc/3045/task/3053/stat : 3053 (claspmt) R 3043 3045 2749 0 -1 4202560 7772 0 0 0 471048 197 0 0 25 0 8 0 741327820 441335808 74723 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37694.9
Current children cumulated vsize (KiB) 430992

[startup+4782.3 s]
/proc/loadavg: 8.04 8.03 8.05 9/339 3716
/proc/meminfo: memFree=23948820/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=430968 CPUtime=38173.9 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 92330 0 0 0 3815670 1719 0 0 25 0 8 0 741327818 441311232 74897 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670273 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/3045/statm: 107742 74897 464 189 0 102091 0
[pid=3045/tid=3047] ppid=3043 vsize=430968 CPUtime=4765.1 cores=0-7
/proc/3045/task/3047/stat : 3047 (claspmt) R 3043 3045 2749 0 -1 4202560 12051 0 0 0 476192 318 0 0 18 0 8 0 741327820 441311232 74897 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=3045/tid=3048] ppid=3043 vsize=430968 CPUtime=4763.06 cores=0-7
/proc/3045/task/3048/stat : 3048 (claspmt) R 3043 3045 2749 0 -1 4202560 16190 0 0 0 476085 221 0 0 25 0 8 0 741327820 441311232 74897 33554432000 4194304 4964800 140736071991888 18446744073709551615 4660236 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3045/tid=3049] ppid=3043 vsize=430968 CPUtime=4773.16 cores=0-7
/proc/3045/task/3049/stat : 3049 (claspmt) R 3043 3045 2749 0 -1 4202560 12427 0 0 0 477102 214 0 0 25 0 8 0 741327820 441311232 74897 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=3045/tid=3050] ppid=3043 vsize=430968 CPUtime=4771.32 cores=0-7
/proc/3045/task/3050/stat : 3050 (claspmt) R 3043 3045 2749 0 -1 4202560 11232 0 0 0 476923 209 0 0 25 0 8 0 741327820 441311232 74897 33554432000 4194304 4964800 140736071991888 18446744073709551615 4362339 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=3045/tid=3051] ppid=3043 vsize=430968 CPUtime=4777.98 cores=0-7
/proc/3045/task/3051/stat : 3051 (claspmt) R 3043 3045 2749 0 -1 4202560 11190 0 0 0 477635 163 0 0 25 0 8 0 741327820 441311232 74897 33554432000 4194304 4964800 140736071991888 18446744073709551615 4627248 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3045/tid=3052] ppid=3043 vsize=430968 CPUtime=4775.45 cores=0-7
/proc/3045/task/3052/stat : 3052 (claspmt) R 3043 3045 2749 0 -1 4202560 12133 0 0 0 477339 206 0 0 22 0 8 0 741327820 441311232 74897 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3045/tid=3053] ppid=3043 vsize=430968 CPUtime=4772.42 cores=0-7
/proc/3045/task/3053/stat : 3053 (claspmt) R 3043 3045 2749 0 -1 4202560 7772 0 0 0 477043 199 0 0 25 0 8 0 741327820 441311232 74897 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38173.9
Current children cumulated vsize (KiB) 430968

[startup+4842.3 s]
/proc/loadavg: 8.21 8.08 8.07 9/339 3718
/proc/meminfo: memFree=23949028/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=431436 CPUtime=38652.9 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 92451 0 0 0 3863551 1740 0 0 25 0 8 0 741327818 441790464 74874 33554432000 4194304 4964800 140736071991888 18446744073709551615 4332393 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/3045/statm: 107859 74874 464 189 0 102208 0
[pid=3045/tid=3047] ppid=3043 vsize=431436 CPUtime=4824.4 cores=0-7
/proc/3045/task/3047/stat : 3047 (claspmt) R 3043 3045 2749 0 -1 4202560 12051 0 0 0 482118 322 0 0 25 0 8 0 741327820 441790464 74874 33554432000 4194304 4964800 140736071991888 18446744073709551615 4321042 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=3045/tid=3048] ppid=3043 vsize=431436 CPUtime=4823.01 cores=0-7
/proc/3045/task/3048/stat : 3048 (claspmt) R 3043 3045 2749 0 -1 4202560 16193 0 0 0 482077 224 0 0 25 0 8 0 741327820 441790464 74874 33554432000 4194304 4964800 140736071991888 18446744073709551615 4654537 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3045/tid=3049] ppid=3043 vsize=431436 CPUtime=4833.13 cores=0-7
/proc/3045/task/3049/stat : 3049 (claspmt) R 3043 3045 2749 0 -1 4202560 12489 0 0 0 483097 216 0 0 22 0 8 0 741327820 441790464 74874 33554432000 4194304 4964800 140736071991888 18446744073709551615 4342157 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=3045/tid=3050] ppid=3043 vsize=431436 CPUtime=4831.26 cores=0-7
/proc/3045/task/3050/stat : 3050 (claspmt) R 3043 3045 2749 0 -1 4202560 11233 0 0 0 482915 211 0 0 25 0 8 0 741327820 441790464 74874 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=3045/tid=3051] ppid=3043 vsize=431436 CPUtime=4837.95 cores=0-7
/proc/3045/task/3051/stat : 3051 (claspmt) R 3043 3045 2749 0 -1 4202560 11209 0 0 0 483630 165 0 0 25 0 8 0 741327820 441790464 74874 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320993 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3045/tid=3052] ppid=3043 vsize=431436 CPUtime=4835.4 cores=0-7
/proc/3045/task/3052/stat : 3052 (claspmt) R 3043 3045 2749 0 -1 4202560 12169 0 0 0 483331 209 0 0 25 0 8 0 741327820 441790464 74874 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670343 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3045/tid=3053] ppid=3043 vsize=431436 CPUtime=4832.38 cores=0-7
/proc/3045/task/3053/stat : 3053 (claspmt) R 3043 3045 2749 0 -1 4202560 7772 0 0 0 483037 201 0 0 22 0 8 0 741327820 441790464 74874 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38652.9
Current children cumulated vsize (KiB) 431436

[startup+4902.3 s]
/proc/loadavg: 8.22 8.10 8.07 9/339 3719
/proc/meminfo: memFree=23948632/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=431600 CPUtime=39131.9 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 92668 0 0 0 3911436 1759 0 0 25 0 8 0 741327818 441958400 75045 33554432000 4194304 4964800 140736071991888 18446744073709551615 4654506 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/3045/statm: 107900 75045 464 189 0 102249 0
[pid=3045/tid=3047] ppid=3043 vsize=431600 CPUtime=4883.71 cores=0-7
/proc/3045/task/3047/stat : 3047 (claspmt) R 3043 3045 2749 0 -1 4202560 12125 0 0 0 488046 325 0 0 25 0 8 0 741327820 441958400 75045 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670378 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=3045/tid=3048] ppid=3043 vsize=431600 CPUtime=4882.96 cores=0-7
/proc/3045/task/3048/stat : 3048 (claspmt) R 3043 3045 2749 0 -1 4202560 16198 0 0 0 488068 228 0 0 25 0 8 0 741327820 441958400 75045 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3045/tid=3049] ppid=3043 vsize=431600 CPUtime=4893.1 cores=0-7
/proc/3045/task/3049/stat : 3049 (claspmt) R 3043 3045 2749 0 -1 4202560 12489 0 0 0 489091 219 0 0 25 0 8 0 741327820 441958400 75045 33554432000 4194304 4964800 140736071991888 18446744073709551615 4332387 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=3045/tid=3050] ppid=3043 vsize=431600 CPUtime=4891.21 cores=0-7
/proc/3045/task/3050/stat : 3050 (claspmt) R 3043 3045 2749 0 -1 4202560 11245 0 0 0 488907 214 0 0 22 0 8 0 741327820 441958400 75045 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=3045/tid=3051] ppid=3043 vsize=431600 CPUtime=4897.93 cores=0-7
/proc/3045/task/3051/stat : 3051 (claspmt) R 3043 3045 2749 0 -1 4202560 11209 0 0 0 489626 167 0 0 25 0 8 0 741327820 441958400 75045 33554432000 4194304 4964800 140736071991888 18446744073709551615 4338784 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3045/tid=3052] ppid=3043 vsize=431600 CPUtime=4895.37 cores=0-7
/proc/3045/task/3052/stat : 3052 (claspmt) R 3043 3045 2749 0 -1 4202560 12204 0 0 0 489326 211 0 0 25 0 8 0 741327820 441958400 75045 33554432000 4194304 4964800 140736071991888 18446744073709551615 4654540 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3045/tid=3053] ppid=3043 vsize=431600 CPUtime=4892.34 cores=0-7
/proc/3045/task/3053/stat : 3053 (claspmt) R 3043 3045 2749 0 -1 4202560 7842 0 0 0 489031 203 0 0 25 0 8 0 741327820 441958400 75045 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39131.9
Current children cumulated vsize (KiB) 431600

[startup+4962.3 s]
/proc/loadavg: 8.10 8.08 8.07 9/339 3721
/proc/meminfo: memFree=23948140/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=431604 CPUtime=39611 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 93063 0 0 0 3959318 1779 0 0 25 0 8 0 741327818 441962496 75232 33554432000 4194304 4964800 140736071991888 18446744073709551615 4321020 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/3045/statm: 107901 75232 464 189 0 102250 0
[pid=3045/tid=3047] ppid=3043 vsize=431604 CPUtime=4943.2 cores=0-7
/proc/3045/task/3047/stat : 3047 (claspmt) R 3043 3045 2749 0 -1 4202560 12125 0 0 0 493992 328 0 0 25 0 8 0 741327820 441962496 75232 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=3045/tid=3048] ppid=3043 vsize=431604 CPUtime=4942.9 cores=0-7
/proc/3045/task/3048/stat : 3048 (claspmt) R 3043 3045 2749 0 -1 4202560 16270 0 0 0 494060 230 0 0 18 0 8 0 741327820 441962496 75232 33554432000 4194304 4964800 140736071991888 18446744073709551615 4654543 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3045/tid=3049] ppid=3043 vsize=431604 CPUtime=4953.06 cores=0-7
/proc/3045/task/3049/stat : 3049 (claspmt) R 3043 3045 2749 0 -1 4202560 12650 0 0 0 495085 221 0 0 25 0 8 0 741327820 441962496 75232 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=3045/tid=3050] ppid=3043 vsize=431604 CPUtime=4950.95 cores=0-7
/proc/3045/task/3050/stat : 3050 (claspmt) R 3043 3045 2749 0 -1 4202560 11245 0 0 0 494879 216 0 0 25 0 8 0 741327820 441962496 75232 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=3045/tid=3051] ppid=3043 vsize=431604 CPUtime=4957.89 cores=0-7
/proc/3045/task/3051/stat : 3051 (claspmt) R 3043 3045 2749 0 -1 4202560 11326 0 0 0 495620 169 0 0 25 0 8 0 741327820 441962496 75232 33554432000 4194304 4964800 140736071991888 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3045/tid=3052] ppid=3043 vsize=431604 CPUtime=4955.32 cores=0-7
/proc/3045/task/3052/stat : 3052 (claspmt) R 3043 3045 2749 0 -1 4202560 12249 0 0 0 495319 213 0 0 25 0 8 0 741327820 441962496 75232 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3045/tid=3053] ppid=3043 vsize=431604 CPUtime=4952.31 cores=0-7
/proc/3045/task/3053/stat : 3053 (claspmt) R 3043 3045 2749 0 -1 4202560 7842 0 0 0 495025 206 0 0 25 0 8 0 741327820 441962496 75232 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39611
Current children cumulated vsize (KiB) 431604



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5011.1 s]
/proc/loadavg: 8.04 8.06 8.06 9/339 3722
/proc/meminfo: memFree=23947780/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=431952 CPUtime=40000.6 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 93439 0 0 0 3998258 1800 0 0 25 0 8 0 741327818 442318848 75398 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/3045/statm: 107988 75398 464 189 0 102337 0
[pid=3045/tid=3047] ppid=3043 vsize=431952 CPUtime=4991.94 cores=0-7
/proc/3045/task/3047/stat : 3047 (claspmt) R 3043 3045 2749 0 -1 4202560 12163 0 0 0 498864 330 0 0 25 0 8 0 741327820 442318848 75398 33554432000 4194304 4964800 140736071991888 18446744073709551615 231895595551 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=3045/tid=3048] ppid=3043 vsize=431952 CPUtime=4991.68 cores=0-7
/proc/3045/task/3048/stat : 3048 (claspmt) R 3043 3045 2749 0 -1 4202560 16286 0 0 0 498935 233 0 0 25 0 8 0 741327820 442318848 75398 33554432000 4194304 4964800 140736071991888 18446744073709551615 231895595551 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3045/tid=3049] ppid=3043 vsize=431952 CPUtime=5001.85 cores=0-7
/proc/3045/task/3049/stat : 3049 (claspmt) R 3043 3045 2749 0 -1 4202560 12696 0 0 0 499961 224 0 0 19 0 8 0 741327820 442318848 75398 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=3045/tid=3050] ppid=3043 vsize=431952 CPUtime=4999.33 cores=0-7
/proc/3045/task/3050/stat : 3050 (claspmt) R 3043 3045 2749 0 -1 4202560 11307 0 0 0 499713 220 0 0 25 0 8 0 741327820 442318848 75398 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=3045/tid=3051] ppid=3043 vsize=431952 CPUtime=5006.68 cores=0-7
/proc/3045/task/3051/stat : 3051 (claspmt) R 3043 3045 2749 0 -1 4202560 11363 0 0 0 500497 171 0 0 25 0 8 0 741327820 442318848 75398 33554432000 4194304 4964800 140736071991888 18446744073709551615 4627338 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3045/tid=3052] ppid=3043 vsize=431952 CPUtime=5004.11 cores=0-7
/proc/3045/task/3052/stat : 3052 (claspmt) R 3043 3045 2749 0 -1 4202560 12250 0 0 0 500195 216 0 0 25 0 8 0 741327820 442318848 75398 33554432000 4194304 4964800 140736071991888 18446744073709551615 231895595551 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3045/tid=3053] ppid=3043 vsize=431952 CPUtime=5000.95 cores=0-7
/proc/3045/task/3053/stat : 3053 (claspmt) R 3043 3045 2749 0 -1 4202560 8000 0 0 0 499887 208 0 0 18 0 8 0 741327820 442318848 75398 33554432000 4194304 4964800 140736071991888 18446744073709551615 4365711 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 431952

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5011.1 s]
/proc/loadavg: 8.04 8.06 8.06 9/339 3722
/proc/meminfo: memFree=23947780/32950928 swapFree=60044312/67111528
[pid=3045] ppid=3043 vsize=431952 CPUtime=40000.6 cores=0-7
/proc/3045/stat : 3045 (claspmt) R 3043 3045 2749 0 -1 4202496 93439 0 0 0 3998258 1800 0 0 25 0 8 0 741327818 442318848 75398 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/3045/statm: 107988 75398 464 189 0 102337 0
[pid=3045/tid=3047] ppid=3043 vsize=431952 CPUtime=4991.94 cores=0-7
/proc/3045/task/3047/stat : 3047 (claspmt) R 3043 3045 2749 0 -1 4202560 12163 0 0 0 498864 330 0 0 25 0 8 0 741327820 442318848 75398 33554432000 4194304 4964800 140736071991888 18446744073709551615 231895595551 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=3045/tid=3048] ppid=3043 vsize=431952 CPUtime=4991.68 cores=0-7
/proc/3045/task/3048/stat : 3048 (claspmt) R 3043 3045 2749 0 -1 4202560 16286 0 0 0 498935 233 0 0 25 0 8 0 741327820 442318848 75398 33554432000 4194304 4964800 140736071991888 18446744073709551615 231895595551 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=3045/tid=3049] ppid=3043 vsize=431952 CPUtime=5001.85 cores=0-7
/proc/3045/task/3049/stat : 3049 (claspmt) R 3043 3045 2749 0 -1 4202560 12696 0 0 0 499961 224 0 0 19 0 8 0 741327820 442318848 75398 33554432000 4194304 4964800 140736071991888 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=3045/tid=3050] ppid=3043 vsize=431952 CPUtime=4999.33 cores=0-7
/proc/3045/task/3050/stat : 3050 (claspmt) R 3043 3045 2749 0 -1 4202560 11307 0 0 0 499713 220 0 0 25 0 8 0 741327820 442318848 75398 33554432000 4194304 4964800 140736071991888 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=3045/tid=3051] ppid=3043 vsize=431952 CPUtime=5006.68 cores=0-7
/proc/3045/task/3051/stat : 3051 (claspmt) R 3043 3045 2749 0 -1 4202560 11363 0 0 0 500497 171 0 0 25 0 8 0 741327820 442318848 75398 33554432000 4194304 4964800 140736071991888 18446744073709551615 4627338 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=3045/tid=3052] ppid=3043 vsize=431952 CPUtime=5004.11 cores=0-7
/proc/3045/task/3052/stat : 3052 (claspmt) R 3043 3045 2749 0 -1 4202560 12250 0 0 0 500195 216 0 0 25 0 8 0 741327820 442318848 75398 33554432000 4194304 4964800 140736071991888 18446744073709551615 231895595551 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=3045/tid=3053] ppid=3043 vsize=431952 CPUtime=5000.95 cores=0-7
/proc/3045/task/3053/stat : 3053 (claspmt) R 3043 3045 2749 0 -1 4202560 8000 0 0 0 499887 208 0 0 18 0 8 0 741327820 442318848 75398 33554432000 4194304 4964800 140736071991888 18446744073709551615 4365711 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 431952

Child status: 0
Real time (s): 5011.16
CPU time (s): 40000.7
CPU user time (s): 39982.7
CPU system time (s): 18.0233
CPU usage (%): 798.233
Max. virtual memory (cumulated for all children) (KiB): 432196

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39982.7
system time used= 18.0233
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 93452
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3778476
involuntary context switches= 208003

runsolver used 16.0316 second user time and 37.0794 second system time

The end

Launcher Data 

Begin job on node114 at 2011-05-16 03:56:11
IDJOB=3381262
IDBENCH=85542
IDSOLVER=1865
FILE ID=node114/3381262-1305510971
RUNJOBID= node114-1305508089-2763
PBS_JOBID= 13325513
Free space on /tmp= 72996 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT05/crafted/sabharwal05/counting/fphp/unsat/harder/fphp-020-019.shuffled-as.sat05-1232.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3381262-1305510971/watcher-3381262-1305510971 -o /tmp/evaluation-result-3381262-1305510971/solver-3381262-1305510971 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381262-1305510971.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= abb32bff5eb954e790612a6209864b5d
RANDOM SEED=2027233523

node114.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      24288524 kB
Buffers:         18144 kB
Cached:         187256 kB
SwapCached:      48132 kB
Active:         534700 kB
Inactive:      7950760 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      24288524 kB
SwapTotal:    67111528 kB
SwapFree:     60044312 kB
Dirty:             788 kB
Writeback:           4 kB
AnonPages:     8238464 kB
Mapped:          20176 kB
Slab:            64232 kB
PageTables:      47128 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 57419008 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73016 MiB
End job on node114 at 2011-05-16 05:19:45