Trace number 3381261

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40001.1 5009.2

General information on the benchmark

NameSAT05/crafted/sabharwal05/counting/fphp/unsat/
harder/fphp-030-029.shuffled-as.sat05-1234.cnf
MD5SUMc12aa6a0ac1bca448bef51c167f3bbbb
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark17.0914
Satisfiable
(Un)Satisfiability was proved
Number of variables870
Number of clauses24825
Sum of the clauses size50460
Maximum clause length29
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 224795
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 530

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...510930/instance-3381261-1305510930.cnf
0.00/0.05	c Solving...
40000.79/5009.01	
40000.79/5009.01	*** INTERRUPTED! ***
40000.79/5009.01	*** Shutting down threads...
40000.97/5009.19	s UNKNOWN
40000.97/5009.19	
40000.97/5009.19	c INTERRUPTED   : 1
40000.97/5009.19	c Models        : 0+    
40000.97/5009.19	c Time          : 5009.019s (Solving: 5008.96s 1st Model: 0.00s Unsat: 0.00s)
40000.97/5009.19	c CPU Time      : 40000.870s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3381261-1305510930/watcher-3381261-1305510930 -o /tmp/evaluation-result-3381261-1305510930/solver-3381261-1305510930 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381261-1305510930.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.40 7.90 7.96 2/348 11441
/proc/meminfo: memFree=19349740/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=22000 CPUtime=0 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 841 0 0 0 0 0 0 0 25 0 1 0 662692657 22528000 731 33554432000 4194304 4964800 140733926027488 18446744073709551615 274473628357 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/11441/statm: 5500 736 392 189 0 374 0

[startup+0.101256 s]
/proc/loadavg: 7.40 7.90 7.96 2/348 11441
/proc/meminfo: memFree=19349740/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=383112 CPUtime=0.29 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 2727 0 0 0 29 0 0 0 25 0 8 0 662692657 392306688 2530 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320944 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/11441/statm: 95778 2531 464 189 0 90127 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 383112

[startup+0.201076 s]
/proc/loadavg: 7.40 7.90 7.96 2/348 11441
/proc/meminfo: memFree=19349740/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=451592 CPUtime=1.07 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 7186 0 0 0 105 2 0 0 25 0 8 0 662692657 462430208 6989 33554432000 4194304 4964800 140733926027488 18446744073709551615 274473634698 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11441/statm: 112898 6990 464 189 0 107247 0
Current children cumulated CPU time (s) 1.07
Current children cumulated vsize (KiB) 451592

[startup+0.301059 s]
/proc/loadavg: 7.40 7.90 7.96 2/348 11441
/proc/meminfo: memFree=19349740/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=453220 CPUtime=1.84 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 10199 0 0 0 181 3 0 0 25 0 8 0 662692657 464097280 10002 33554432000 4194304 4964800 140733926027488 18446744073709551615 4362287 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11441/statm: 113305 10002 464 189 0 107654 0
Current children cumulated CPU time (s) 1.84
Current children cumulated vsize (KiB) 453220

[startup+0.700989 s]
/proc/loadavg: 7.40 7.90 7.96 2/348 11441
/proc/meminfo: memFree=19349740/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=454016 CPUtime=5.03 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 13095 0 0 0 498 5 0 0 25 0 8 0 662692657 464912384 12898 33554432000 4194304 4964800 140733926027488 18446744073709551615 4325321 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11441/statm: 113504 12898 464 189 0 107853 0
Current children cumulated CPU time (s) 5.03
Current children cumulated vsize (KiB) 454016

[startup+1.50085 s]
/proc/loadavg: 7.40 7.90 7.96 9/356 11449
/proc/meminfo: memFree=19282476/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=521956 CPUtime=11.4 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 19910 0 0 0 1131 9 0 0 25 0 8 0 662692657 534482944 19713 33554432000 4194304 4964800 140733926027488 18446744073709551615 274474072607 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11441/statm: 130489 19713 466 189 0 124838 0
[pid=11441/tid=11443] ppid=11439 vsize=521956 CPUtime=1.42 cores=0-7
/proc/11441/task/11443/stat : 11443 (claspmt) R 11439 11441 11415 0 -1 4202560 2081 0 0 0 141 1 0 0 19 0 8 0 662692663 534482944 19713 33554432000 4194304 4964800 140733926027488 18446744073709551615 4660144 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11441/tid=11444] ppid=11439 vsize=521956 CPUtime=1.41 cores=0-7
/proc/11441/task/11444/stat : 11444 (claspmt) R 11439 11441 11415 0 -1 4202560 3952 0 0 0 140 1 0 0 19 0 8 0 662692663 534482944 19713 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11441/tid=11445] ppid=11439 vsize=521956 CPUtime=1.4 cores=0-7
/proc/11441/task/11445/stat : 11445 (claspmt) R 11439 11441 11415 0 -1 4202560 1834 0 0 0 140 0 0 0 25 0 8 0 662692663 534482944 19713 33554432000 4194304 4964800 140733926027488 18446744073709551615 4642509 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11441/tid=11446] ppid=11439 vsize=521956 CPUtime=1.41 cores=0-7
/proc/11441/task/11446/stat : 11446 (claspmt) R 11439 11441 11415 0 -1 4202560 2614 0 0 0 141 0 0 0 25 0 8 0 662692663 534482944 19713 33554432000 4194304 4964800 140733926027488 18446744073709551615 274474072607 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11441/tid=11447] ppid=11439 vsize=521956 CPUtime=1.42 cores=0-7
/proc/11441/task/11447/stat : 11447 (claspmt) R 11439 11441 11415 0 -1 4202560 1629 0 0 0 141 1 0 0 25 0 8 0 662692663 534482944 19713 33554432000 4194304 4964800 140733926027488 18446744073709551615 274474072607 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11441/tid=11448] ppid=11439 vsize=521956 CPUtime=1.41 cores=0-7
/proc/11441/task/11448/stat : 11448 (claspmt) R 11439 11441 11415 0 -1 4202560 2131 0 0 0 140 1 0 0 25 0 8 0 662692663 534482944 19713 33554432000 4194304 4964800 140733926027488 18446744073709551615 4342144 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11441/tid=11449] ppid=11439 vsize=521956 CPUtime=1.4 cores=0-7
/proc/11441/task/11449/stat : 11449 (claspmt) R 11439 11441 11415 0 -1 4202560 2599 0 0 0 140 0 0 0 25 0 8 0 662692663 534482944 19713 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 11.4
Current children cumulated vsize (KiB) 521956

[startup+3.10057 s]
/proc/loadavg: 7.52 7.92 7.96 9/356 11449
/proc/meminfo: memFree=19259296/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=525584 CPUtime=24.18 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 26010 0 0 0 2406 12 0 0 25 0 8 0 662692657 538198016 25813 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320953 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11441/statm: 131396 25813 466 189 0 125745 0
[pid=11441/tid=11443] ppid=11439 vsize=525584 CPUtime=3.01 cores=0-7
/proc/11441/task/11443/stat : 11443 (claspmt) R 11439 11441 11415 0 -1 4202560 2963 0 0 0 300 1 0 0 23 0 8 0 662692663 538198016 25813 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11441/tid=11444] ppid=11439 vsize=525584 CPUtime=3.01 cores=0-7
/proc/11441/task/11444/stat : 11444 (claspmt) R 11439 11441 11415 0 -1 4202560 5768 0 0 0 299 2 0 0 25 0 8 0 662692663 538198016 25813 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11441/tid=11445] ppid=11439 vsize=525584 CPUtime=3 cores=0-7
/proc/11441/task/11445/stat : 11445 (claspmt) R 11439 11441 11415 0 -1 4202560 2668 0 0 0 300 0 0 0 25 0 8 0 662692663 538198016 25813 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11441/tid=11446] ppid=11439 vsize=525584 CPUtime=3.01 cores=0-7
/proc/11441/task/11446/stat : 11446 (claspmt) R 11439 11441 11415 0 -1 4202560 3319 0 0 0 301 0 0 0 25 0 8 0 662692663 538198016 25813 33554432000 4194304 4964800 140733926027488 18446744073709551615 4334283 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11441/tid=11447] ppid=11439 vsize=525584 CPUtime=3.02 cores=0-7
/proc/11441/task/11447/stat : 11447 (claspmt) R 11439 11441 11415 0 -1 4202560 2173 0 0 0 301 1 0 0 18 0 8 0 662692663 538198016 25813 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11441/tid=11448] ppid=11439 vsize=525584 CPUtime=2.99 cores=0-7
/proc/11441/task/11448/stat : 11448 (claspmt) R 11439 11441 11415 0 -1 4202560 2648 0 0 0 298 1 0 0 25 0 8 0 662692663 538198016 25813 33554432000 4194304 4964800 140733926027488 18446744073709551615 4332351 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11441/tid=11449] ppid=11439 vsize=525584 CPUtime=3.01 cores=0-7
/proc/11441/task/11449/stat : 11449 (claspmt) R 11439 11441 11415 0 -1 4202560 3056 0 0 0 300 1 0 0 25 0 8 0 662692663 538198016 25813 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 24.18
Current children cumulated vsize (KiB) 525584

[startup+6.301 s]
/proc/loadavg: 7.52 7.92 7.96 9/356 11450
/proc/meminfo: memFree=19200364/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=534648 CPUtime=49.73 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 42683 0 0 0 4952 21 0 0 25 0 8 0 662692657 547479552 42405 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320909 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11441/statm: 133662 42405 466 189 0 128011 0
[pid=11441/tid=11443] ppid=11439 vsize=534648 CPUtime=6.21 cores=0-7
/proc/11441/task/11443/stat : 11443 (claspmt) R 11439 11441 11415 0 -1 4202560 5357 0 0 0 618 3 0 0 25 0 8 0 662692663 547479552 42405 33554432000 4194304 4964800 140733926027488 18446744073709551615 4683952 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11441/tid=11444] ppid=11439 vsize=534648 CPUtime=6.22 cores=0-7
/proc/11441/task/11444/stat : 11444 (claspmt) R 11439 11441 11415 0 -1 4202560 9354 0 0 0 618 4 0 0 25 0 8 0 662692663 547479552 42405 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11441/tid=11445] ppid=11439 vsize=534648 CPUtime=6.21 cores=0-7
/proc/11441/task/11445/stat : 11445 (claspmt) R 11439 11441 11415 0 -1 4202560 4071 0 0 0 619 2 0 0 25 0 8 0 662692663 547479552 42405 33554432000 4194304 4964800 140733926027488 18446744073709551615 4332624 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11441/tid=11446] ppid=11439 vsize=534648 CPUtime=6.21 cores=0-7
/proc/11441/task/11446/stat : 11446 (claspmt) R 11439 11441 11415 0 -1 4202560 5668 0 0 0 620 1 0 0 25 0 8 0 662692663 547479552 42405 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11441/tid=11447] ppid=11439 vsize=534648 CPUtime=6.21 cores=0-7
/proc/11441/task/11447/stat : 11447 (claspmt) R 11439 11441 11415 0 -1 4202560 3580 0 0 0 620 1 0 0 25 0 8 0 662692663 547479552 42405 33554432000 4194304 4964800 140733926027488 18446744073709551615 4644876 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11441/tid=11448] ppid=11439 vsize=534648 CPUtime=6.15 cores=0-7
/proc/11441/task/11448/stat : 11448 (claspmt) R 11439 11441 11415 0 -1 4202560 4420 0 0 0 613 2 0 0 25 0 8 0 662692663 547479552 42405 33554432000 4194304 4964800 140733926027488 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11441/tid=11449] ppid=11439 vsize=534648 CPUtime=6.2 cores=0-7
/proc/11441/task/11449/stat : 11449 (claspmt) R 11439 11441 11415 0 -1 4202560 5478 0 0 0 618 2 0 0 25 0 8 0 662692663 547479552 42405 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 49.73
Current children cumulated vsize (KiB) 534648

[startup+12.7009 s]
/proc/loadavg: 7.67 7.94 7.97 9/356 11450
/proc/meminfo: memFree=19117264/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=544180 CPUtime=100.84 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 59630 0 0 0 10054 30 0 0 25 0 8 0 662692657 557240320 59150 33554432000 4194304 4964800 140733926027488 18446744073709551615 4321042 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11441/statm: 136045 59150 466 189 0 130394 0
[pid=11441/tid=11443] ppid=11439 vsize=544180 CPUtime=12.61 cores=0-7
/proc/11441/task/11443/stat : 11443 (claspmt) R 11439 11441 11415 0 -1 4202560 8905 0 0 0 1256 5 0 0 25 0 8 0 662692663 557240320 59150 33554432000 4194304 4964800 140733926027488 18446744073709551615 4332375 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11441/tid=11444] ppid=11439 vsize=544180 CPUtime=12.62 cores=0-7
/proc/11441/task/11444/stat : 11444 (claspmt) R 11439 11441 11415 0 -1 4202560 11967 0 0 0 1257 5 0 0 21 0 8 0 662692663 557240320 59150 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11441/tid=11445] ppid=11439 vsize=544180 CPUtime=12.6 cores=0-7
/proc/11441/task/11445/stat : 11445 (claspmt) R 11439 11441 11415 0 -1 4202560 5905 0 0 0 1256 4 0 0 25 0 8 0 662692663 557240320 59150 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11441/tid=11446] ppid=11439 vsize=544180 CPUtime=12.6 cores=0-7
/proc/11441/task/11446/stat : 11446 (claspmt) R 11439 11441 11415 0 -1 4202560 8241 0 0 0 1258 2 0 0 25 0 8 0 662692663 557240320 59150 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11441/tid=11447] ppid=11439 vsize=544180 CPUtime=12.62 cores=0-7
/proc/11441/task/11447/stat : 11447 (claspmt) R 11439 11441 11415 0 -1 4202560 5068 0 0 0 1260 2 0 0 25 0 8 0 662692663 557240320 59150 33554432000 4194304 4964800 140733926027488 18446744073709551615 4660384 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11441/tid=11448] ppid=11439 vsize=544180 CPUtime=12.48 cores=0-7
/proc/11441/task/11448/stat : 11448 (claspmt) R 11439 11441 11415 0 -1 4202560 5675 0 0 0 1245 3 0 0 18 0 8 0 662692663 557240320 59150 33554432000 4194304 4964800 140733926027488 18446744073709551615 4321020 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11441/tid=11449] ppid=11439 vsize=544180 CPUtime=12.6 cores=0-7
/proc/11441/task/11449/stat : 11449 (claspmt) R 11439 11441 11415 0 -1 4202560 7317 0 0 0 1257 3 0 0 25 0 8 0 662692663 557240320 59150 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 100.84
Current children cumulated vsize (KiB) 544180

[startup+25.5006 s]
/proc/loadavg: 7.79 7.95 7.97 9/356 11450
/proc/meminfo: memFree=19051104/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=550768 CPUtime=203.06 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 76113 0 0 0 20265 41 0 0 25 0 8 0 662692657 563986432 75625 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11441/statm: 137692 75625 466 189 0 132041 0
[pid=11441/tid=11443] ppid=11439 vsize=550768 CPUtime=25.4 cores=0-7
/proc/11441/task/11443/stat : 11443 (claspmt) R 11439 11441 11415 0 -1 4202560 11228 0 0 0 2533 7 0 0 25 0 8 0 662692663 563986432 75625 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670343 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11441/tid=11444] ppid=11439 vsize=550768 CPUtime=25.41 cores=0-7
/proc/11441/task/11444/stat : 11444 (claspmt) R 11439 11441 11415 0 -1 4202560 14087 0 0 0 2535 6 0 0 25 0 8 0 662692663 563986432 75625 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11441/tid=11445] ppid=11439 vsize=550768 CPUtime=25.39 cores=0-7
/proc/11441/task/11445/stat : 11445 (claspmt) R 11439 11441 11415 0 -1 4202560 6676 0 0 0 2535 4 0 0 25 0 8 0 662692663 563986432 75625 33554432000 4194304 4964800 140733926027488 18446744073709551615 4654052 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11441/tid=11446] ppid=11439 vsize=550768 CPUtime=25.4 cores=0-7
/proc/11441/task/11446/stat : 11446 (claspmt) R 11439 11441 11415 0 -1 4202560 8989 0 0 0 2537 3 0 0 25 0 8 0 662692663 563986432 75625 33554432000 4194304 4964800 140733926027488 18446744073709551615 4379998 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11441/tid=11447] ppid=11439 vsize=550768 CPUtime=25.41 cores=0-7
/proc/11441/task/11447/stat : 11447 (claspmt) R 11439 11441 11415 0 -1 4202560 7198 0 0 0 2538 3 0 0 25 0 8 0 662692663 563986432 75625 33554432000 4194304 4964800 140733926027488 18446744073709551615 4660130 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11441/tid=11448] ppid=11439 vsize=550768 CPUtime=25.12 cores=0-7
/proc/11441/task/11448/stat : 11448 (claspmt) R 11439 11441 11415 0 -1 4202560 7835 0 0 0 2508 4 0 0 25 0 8 0 662692663 563986432 75625 33554432000 4194304 4964800 140733926027488 18446744073709551615 4654476 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11441/tid=11449] ppid=11439 vsize=550768 CPUtime=25.4 cores=0-7
/proc/11441/task/11449/stat : 11449 (claspmt) R 11439 11441 11415 0 -1 4202560 11482 0 0 0 2534 6 0 0 25 0 8 0 662692663 563986432 75625 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670335 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 203.06
Current children cumulated vsize (KiB) 550768

[startup+51.1011 s]
/proc/loadavg: 7.86 7.96 7.97 9/356 11451
/proc/meminfo: memFree=18955188/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=569968 CPUtime=407.52 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 99968 0 0 0 40694 58 0 0 25 0 8 0 662692657 583647232 99480 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320981 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11441/statm: 142492 99480 466 189 0 136841 0
[pid=11441/tid=11443] ppid=11439 vsize=569968 CPUtime=50.98 cores=0-7
/proc/11441/task/11443/stat : 11443 (claspmt) R 11439 11441 11415 0 -1 4202560 15457 0 0 0 5088 10 0 0 25 0 8 0 662692663 583647232 99480 33554432000 4194304 4964800 140733926027488 18446744073709551615 4684145 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11441/tid=11444] ppid=11439 vsize=569968 CPUtime=50.99 cores=0-7
/proc/11441/task/11444/stat : 11444 (claspmt) R 11439 11441 11415 0 -1 4202560 18141 0 0 0 5090 9 0 0 25 0 8 0 662692663 583647232 99480 33554432000 4194304 4964800 140733926027488 18446744073709551615 4324798 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11441/tid=11445] ppid=11439 vsize=569968 CPUtime=50.99 cores=0-7
/proc/11441/task/11445/stat : 11445 (claspmt) R 11439 11441 11415 0 -1 4202560 7560 0 0 0 5094 5 0 0 25 0 8 0 662692663 583647232 99480 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11441/tid=11446] ppid=11439 vsize=569968 CPUtime=50.99 cores=0-7
/proc/11441/task/11446/stat : 11446 (claspmt) R 11439 11441 11415 0 -1 4202560 9970 0 0 0 5096 3 0 0 25 0 8 0 662692663 583647232 99480 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11441/tid=11447] ppid=11439 vsize=569968 CPUtime=50.99 cores=0-7
/proc/11441/task/11447/stat : 11447 (claspmt) R 11439 11441 11415 0 -1 4202560 8801 0 0 0 5094 5 0 0 25 0 8 0 662692663 583647232 99480 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11441/tid=11448] ppid=11439 vsize=569968 CPUtime=50.45 cores=0-7
/proc/11441/task/11448/stat : 11448 (claspmt) R 11439 11441 11415 0 -1 4202560 11469 0 0 0 5038 7 0 0 25 0 8 0 662692663 583647232 99480 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670361 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11441/tid=11449] ppid=11439 vsize=569968 CPUtime=50.99 cores=0-7
/proc/11441/task/11449/stat : 11449 (claspmt) R 11439 11441 11415 0 -1 4202560 17323 0 0 0 5090 9 0 0 25 0 8 0 662692663 583647232 99480 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 407.52

################
# More data... #
################

[pid=11441/tid=11444] ppid=11439 vsize=2102972 CPUtime=4661.52 cores=0-7
/proc/11441/task/11444/stat : 11444 (claspmt) R 11439 11441 11415 0 -1 4202560 99733 0 0 0 466063 89 0 0 25 0 8 0 662692663 2153443328 415376 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11441/tid=11445] ppid=11439 vsize=2102972 CPUtime=4659.47 cores=0-7
/proc/11441/task/11445/stat : 11445 (claspmt) R 11439 11441 11415 0 -1 4202560 39433 0 0 0 465879 68 0 0 25 0 8 0 662692663 2153443328 415376 33554432000 4194304 4964800 140733926027488 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11441/tid=11446] ppid=11439 vsize=2102972 CPUtime=4660.57 cores=0-7
/proc/11441/task/11446/stat : 11446 (claspmt) R 11439 11441 11415 0 -1 4202560 30433 0 0 0 466009 48 0 0 25 0 8 0 662692663 2153443328 415376 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11441/tid=11447] ppid=11439 vsize=2102972 CPUtime=4655.76 cores=0-7
/proc/11441/task/11447/stat : 11447 (claspmt) R 11439 11441 11415 0 -1 4202560 34771 0 0 0 465520 56 0 0 25 0 8 0 662692663 2153443328 415376 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11441/tid=11448] ppid=11439 vsize=2102972 CPUtime=4646.64 cores=0-7
/proc/11441/task/11448/stat : 11448 (claspmt) R 11439 11441 11415 0 -1 4202560 49704 0 0 0 464589 75 0 0 25 0 8 0 662692663 2153443328 415376 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11441/tid=11449] ppid=11439 vsize=2102972 CPUtime=4660.1 cores=0-7
/proc/11441/task/11449/stat : 11449 (claspmt) R 11439 11441 11415 0 -1 4202560 45704 0 0 0 465946 64 0 0 25 0 8 0 662692663 2153443328 415376 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 37231.7
Current children cumulated vsize (KiB) 2102972

[startup+4722.3 s]
/proc/loadavg: 8.10 8.07 8.01 9/356 12071
/proc/meminfo: memFree=17757572/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=2102972 CPUtime=37711 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 436296 0 0 0 3770488 609 0 0 25 0 8 0 662692657 2153443328 415927 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320909 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/11441/statm: 525743 415927 466 189 0 520092 0
[pid=11441/tid=11443] ppid=11439 vsize=2102972 CPUtime=4695.45 cores=0-7
/proc/11441/task/11443/stat : 11443 (claspmt) R 11439 11441 11415 0 -1 4202560 94344 0 0 0 469415 130 0 0 22 0 8 0 662692663 2153443328 415927 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11441/tid=11444] ppid=11439 vsize=2102972 CPUtime=4721.53 cores=0-7
/proc/11441/task/11444/stat : 11444 (claspmt) R 11439 11441 11415 0 -1 4202560 99771 0 0 0 472064 89 0 0 25 0 8 0 662692663 2153443328 415927 33554432000 4194304 4964800 140733926027488 18446744073709551615 4644156 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11441/tid=11445] ppid=11439 vsize=2102972 CPUtime=4719.47 cores=0-7
/proc/11441/task/11445/stat : 11445 (claspmt) R 11439 11441 11415 0 -1 4202560 39433 0 0 0 471879 68 0 0 25 0 8 0 662692663 2153443328 415927 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11441/tid=11446] ppid=11439 vsize=2102972 CPUtime=4720.58 cores=0-7
/proc/11441/task/11446/stat : 11446 (claspmt) R 11439 11441 11415 0 -1 4202560 30435 0 0 0 472008 50 0 0 25 0 8 0 662692663 2153443328 415927 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11441/tid=11447] ppid=11439 vsize=2102972 CPUtime=4715.76 cores=0-7
/proc/11441/task/11447/stat : 11447 (claspmt) R 11439 11441 11415 0 -1 4202560 34793 0 0 0 471520 56 0 0 25 0 8 0 662692663 2153443328 415927 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670321 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11441/tid=11448] ppid=11439 vsize=2102972 CPUtime=4706.4 cores=0-7
/proc/11441/task/11448/stat : 11448 (claspmt) R 11439 11441 11415 0 -1 4202560 49704 0 0 0 470564 76 0 0 25 0 8 0 662692663 2153443328 415927 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11441/tid=11449] ppid=11439 vsize=2102972 CPUtime=4720.08 cores=0-7
/proc/11441/task/11449/stat : 11449 (claspmt) R 11439 11441 11415 0 -1 4202560 45704 0 0 0 471944 64 0 0 25 0 8 0 662692663 2153443328 415927 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320886 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 37711
Current children cumulated vsize (KiB) 2102972

[startup+4782.3 s]
/proc/loadavg: 8.03 8.05 8.00 9/356 12073
/proc/meminfo: memFree=17757640/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=2102972 CPUtime=38190.1 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 436307 0 0 0 3818399 614 0 0 25 0 8 0 662692657 2153443328 415937 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320955 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/11441/statm: 525743 415937 466 189 0 520092 0
[pid=11441/tid=11443] ppid=11439 vsize=2102972 CPUtime=4754.74 cores=0-7
/proc/11441/task/11443/stat : 11443 (claspmt) R 11439 11441 11415 0 -1 4202560 94351 0 0 0 475342 132 0 0 25 0 8 0 662692663 2153443328 415937 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11441/tid=11444] ppid=11439 vsize=2102972 CPUtime=4781.51 cores=0-7
/proc/11441/task/11444/stat : 11444 (claspmt) R 11439 11441 11415 0 -1 4202560 99771 0 0 0 478061 90 0 0 25 0 8 0 662692663 2153443328 415937 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11441/tid=11445] ppid=11439 vsize=2102972 CPUtime=4779.46 cores=0-7
/proc/11441/task/11445/stat : 11445 (claspmt) R 11439 11441 11415 0 -1 4202560 39433 0 0 0 477877 69 0 0 25 0 8 0 662692663 2153443328 415937 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11441/tid=11446] ppid=11439 vsize=2102972 CPUtime=4780.55 cores=0-7
/proc/11441/task/11446/stat : 11446 (claspmt) R 11439 11441 11415 0 -1 4202560 30435 0 0 0 478005 50 0 0 25 0 8 0 662692663 2153443328 415937 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320947 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11441/tid=11447] ppid=11439 vsize=2102972 CPUtime=4775.75 cores=0-7
/proc/11441/task/11447/stat : 11447 (claspmt) R 11439 11441 11415 0 -1 4202560 34793 0 0 0 477518 57 0 0 25 0 8 0 662692663 2153443328 415937 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11441/tid=11448] ppid=11439 vsize=2102972 CPUtime=4766.37 cores=0-7
/proc/11441/task/11448/stat : 11448 (claspmt) R 11439 11441 11415 0 -1 4202560 49708 0 0 0 476561 76 0 0 25 0 8 0 662692663 2153443328 415937 33554432000 4194304 4964800 140733926027488 18446744073709551615 4321116 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11441/tid=11449] ppid=11439 vsize=2102972 CPUtime=4780.05 cores=0-7
/proc/11441/task/11449/stat : 11449 (claspmt) R 11439 11441 11415 0 -1 4202560 45704 0 0 0 477941 64 0 0 25 0 8 0 662692663 2153443328 415937 33554432000 4194304 4964800 140733926027488 18446744073709551615 4327267 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38190.1
Current children cumulated vsize (KiB) 2102972

[startup+4842.31 s]
/proc/loadavg: 8.01 8.04 8.00 9/356 12075
/proc/meminfo: memFree=17757472/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=2102972 CPUtime=38669.4 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 436310 0 0 0 3866327 617 0 0 25 0 8 0 662692657 2153443328 415938 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320981 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/11441/statm: 525743 415938 466 189 0 520092 0
[pid=11441/tid=11443] ppid=11439 vsize=2102972 CPUtime=4814.02 cores=0-7
/proc/11441/task/11443/stat : 11443 (claspmt) R 11439 11441 11415 0 -1 4202560 94354 0 0 0 481269 133 0 0 25 0 8 0 662692663 2153443328 415938 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11441/tid=11444] ppid=11439 vsize=2102972 CPUtime=4841.51 cores=0-7
/proc/11441/task/11444/stat : 11444 (claspmt) R 11439 11441 11415 0 -1 4202560 99771 0 0 0 484061 90 0 0 25 0 8 0 662692663 2153443328 415938 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11441/tid=11445] ppid=11439 vsize=2102972 CPUtime=4839.46 cores=0-7
/proc/11441/task/11445/stat : 11445 (claspmt) R 11439 11441 11415 0 -1 4202560 39433 0 0 0 483876 70 0 0 25 0 8 0 662692663 2153443328 415938 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11441/tid=11446] ppid=11439 vsize=2102972 CPUtime=4840.55 cores=0-7
/proc/11441/task/11446/stat : 11446 (claspmt) R 11439 11441 11415 0 -1 4202560 30435 0 0 0 484005 50 0 0 25 0 8 0 662692663 2153443328 415938 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11441/tid=11447] ppid=11439 vsize=2102972 CPUtime=4835.75 cores=0-7
/proc/11441/task/11447/stat : 11447 (claspmt) R 11439 11441 11415 0 -1 4202560 34793 0 0 0 483518 57 0 0 25 0 8 0 662692663 2153443328 415938 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11441/tid=11448] ppid=11439 vsize=2102972 CPUtime=4826.39 cores=0-7
/proc/11441/task/11448/stat : 11448 (claspmt) R 11439 11441 11415 0 -1 4202560 49708 0 0 0 482562 77 0 0 25 0 8 0 662692663 2153443328 415938 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11441/tid=11449] ppid=11439 vsize=2102972 CPUtime=4840.06 cores=0-7
/proc/11441/task/11449/stat : 11449 (claspmt) R 11439 11441 11415 0 -1 4202560 45704 0 0 0 483941 65 0 0 25 0 8 0 662692663 2153443328 415938 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38669.4
Current children cumulated vsize (KiB) 2102972

[startup+4902.3 s]
/proc/loadavg: 8.00 8.03 8.00 9/356 12076
/proc/meminfo: memFree=17757304/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=2102972 CPUtime=39148.6 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 436345 0 0 0 3914238 619 0 0 25 0 8 0 662692657 2153443328 415970 33554432000 4194304 4964800 140733926027488 18446744073709551615 4659391 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/11441/statm: 525743 415970 466 189 0 520092 0
[pid=11441/tid=11443] ppid=11439 vsize=2102972 CPUtime=4873.29 cores=0-7
/proc/11441/task/11443/stat : 11443 (claspmt) R 11439 11441 11415 0 -1 4202560 94354 0 0 0 487196 133 0 0 25 0 8 0 662692663 2153443328 415970 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11441/tid=11444] ppid=11439 vsize=2102972 CPUtime=4901.49 cores=0-7
/proc/11441/task/11444/stat : 11444 (claspmt) R 11439 11441 11415 0 -1 4202560 99778 0 0 0 490059 90 0 0 25 0 8 0 662692663 2153443328 415970 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11441/tid=11445] ppid=11439 vsize=2102972 CPUtime=4899.45 cores=0-7
/proc/11441/task/11445/stat : 11445 (claspmt) R 11439 11441 11415 0 -1 4202560 39433 0 0 0 489874 71 0 0 25 0 8 0 662692663 2153443328 415970 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11441/tid=11446] ppid=11439 vsize=2102972 CPUtime=4900.53 cores=0-7
/proc/11441/task/11446/stat : 11446 (claspmt) R 11439 11441 11415 0 -1 4202560 30454 0 0 0 490003 50 0 0 25 0 8 0 662692663 2153443328 415970 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11441/tid=11447] ppid=11439 vsize=2102972 CPUtime=4895.73 cores=0-7
/proc/11441/task/11447/stat : 11447 (claspmt) R 11439 11441 11415 0 -1 4202560 34802 0 0 0 489516 57 0 0 25 0 8 0 662692663 2153443328 415970 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11441/tid=11448] ppid=11439 vsize=2102972 CPUtime=4886.37 cores=0-7
/proc/11441/task/11448/stat : 11448 (claspmt) R 11439 11441 11415 0 -1 4202560 49708 0 0 0 488560 77 0 0 25 0 8 0 662692663 2153443328 415970 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11441/tid=11449] ppid=11439 vsize=2102972 CPUtime=4900.02 cores=0-7
/proc/11441/task/11449/stat : 11449 (claspmt) R 11439 11441 11415 0 -1 4202560 45704 0 0 0 489937 65 0 0 25 0 8 0 662692663 2153443328 415970 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39148.6
Current children cumulated vsize (KiB) 2102972

[startup+4962.3 s]
/proc/loadavg: 8.05 8.04 8.00 9/356 12078
/proc/meminfo: memFree=17757564/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=2102972 CPUtime=39627.8 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 436359 0 0 0 3962153 623 0 0 25 0 8 0 662692657 2153443328 415974 33554432000 4194304 4964800 140733926027488 18446744073709551615 4659391 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/11441/statm: 525743 415974 466 189 0 520092 0
[pid=11441/tid=11443] ppid=11439 vsize=2102972 CPUtime=4932.55 cores=0-7
/proc/11441/task/11443/stat : 11443 (claspmt) R 11439 11441 11415 0 -1 4202560 94359 0 0 0 493121 134 0 0 25 0 8 0 662692663 2153443328 415974 33554432000 4194304 4964800 140733926027488 18446744073709551615 4359851 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11441/tid=11444] ppid=11439 vsize=2102972 CPUtime=4961.49 cores=0-7
/proc/11441/task/11444/stat : 11444 (claspmt) R 11439 11441 11415 0 -1 4202560 99778 0 0 0 496058 91 0 0 25 0 8 0 662692663 2153443328 415974 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11441/tid=11445] ppid=11439 vsize=2102972 CPUtime=4959.44 cores=0-7
/proc/11441/task/11445/stat : 11445 (claspmt) R 11439 11441 11415 0 -1 4202560 39433 0 0 0 495873 71 0 0 25 0 8 0 662692663 2153443328 415974 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11441/tid=11446] ppid=11439 vsize=2102972 CPUtime=4960.53 cores=0-7
/proc/11441/task/11446/stat : 11446 (claspmt) R 11439 11441 11415 0 -1 4202560 30454 0 0 0 496002 51 0 0 25 0 8 0 662692663 2153443328 415974 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11441/tid=11447] ppid=11439 vsize=2102972 CPUtime=4955.72 cores=0-7
/proc/11441/task/11447/stat : 11447 (claspmt) R 11439 11441 11415 0 -1 4202560 34810 0 0 0 495515 57 0 0 25 0 8 0 662692663 2153443328 415974 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11441/tid=11448] ppid=11439 vsize=2102972 CPUtime=4946.36 cores=0-7
/proc/11441/task/11448/stat : 11448 (claspmt) R 11439 11441 11415 0 -1 4202560 49708 0 0 0 494558 78 0 0 25 0 8 0 662692663 2153443328 415974 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11441/tid=11449] ppid=11439 vsize=2102972 CPUtime=4960 cores=0-7
/proc/11441/task/11449/stat : 11449 (claspmt) R 11439 11441 11415 0 -1 4202560 45705 0 0 0 495935 65 0 0 25 0 8 0 662692663 2153443328 415974 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39627.8
Current children cumulated vsize (KiB) 2102972



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5009 s]
/proc/loadavg: 8.08 8.05 8.00 9/356 12079
/proc/meminfo: memFree=17757520/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=2102972 CPUtime=40000.8 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 436368 0 0 0 3999454 625 0 0 25 0 8 0 662692657 2153443328 415983 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/11441/statm: 525743 415983 466 189 0 520092 0
[pid=11441/tid=11443] ppid=11439 vsize=2102972 CPUtime=4978.67 cores=0-7
/proc/11441/task/11443/stat : 11443 (claspmt) R 11439 11441 11415 0 -1 4202560 94360 0 0 0 497733 134 0 0 25 0 8 0 662692663 2153443328 415983 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11441/tid=11444] ppid=11439 vsize=2102972 CPUtime=5008.19 cores=0-7
/proc/11441/task/11444/stat : 11444 (claspmt) R 11439 11441 11415 0 -1 4202560 99786 0 0 0 500728 91 0 0 25 0 8 0 662692663 2153443328 415983 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11441/tid=11445] ppid=11439 vsize=2102972 CPUtime=5006.15 cores=0-7
/proc/11441/task/11445/stat : 11445 (claspmt) R 11439 11441 11415 0 -1 4202560 39433 0 0 0 500544 71 0 0 25 0 8 0 662692663 2153443328 415983 33554432000 4194304 4964800 140733926027488 18446744073709551615 4659391 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11441/tid=11446] ppid=11439 vsize=2102972 CPUtime=5007.23 cores=0-7
/proc/11441/task/11446/stat : 11446 (claspmt) R 11439 11441 11415 0 -1 4202560 30454 0 0 0 500672 51 0 0 25 0 8 0 662692663 2153443328 415983 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11441/tid=11447] ppid=11439 vsize=2102972 CPUtime=5002.43 cores=0-7
/proc/11441/task/11447/stat : 11447 (claspmt) R 11439 11441 11415 0 -1 4202560 34810 0 0 0 500186 57 0 0 25 0 8 0 662692663 2153443328 415983 33554432000 4194304 4964800 140733926027488 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11441/tid=11448] ppid=11439 vsize=2102972 CPUtime=4993.07 cores=0-7
/proc/11441/task/11448/stat : 11448 (claspmt) R 11439 11441 11415 0 -1 4202560 49708 0 0 0 499229 78 0 0 25 0 8 0 662692663 2153443328 415983 33554432000 4194304 4964800 140733926027488 18446744073709551615 4684120 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11441/tid=11449] ppid=11439 vsize=2102972 CPUtime=5006.7 cores=0-7
/proc/11441/task/11449/stat : 11449 (claspmt) R 11439 11441 11415 0 -1 4202560 45705 0 0 0 500604 66 0 0 25 0 8 0 662692663 2153443328 415983 33554432000 4194304 4964800 140733926027488 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 2102972

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5009.12 s]
/proc/loadavg: 8.08 8.05 8.00 9/356 12079
/proc/meminfo: memFree=17757520/32950928 swapFree=67042944/67111528
[pid=11441] ppid=11439 vsize=1587720 CPUtime=40001 cores=0-7
/proc/11441/stat : 11441 (claspmt) R 11439 11441 11415 0 -1 4202496 436378 0 0 0 3999470 627 0 0 25 0 1 0 662692657 1625825280 339064 33554432000 4194304 4964800 140733926027488 18446744073709551615 274474010983 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/11441/statm: 396930 339064 476 189 0 391279 0
Current children cumulated CPU time (s) 40001
Current children cumulated vsize (KiB) 1587720

Child status: 0
Real time (s): 5009.2
CPU time (s): 40001.1
CPU user time (s): 39994.7
CPU system time (s): 6.34903
CPU usage (%): 798.552
Max. virtual memory (cumulated for all children) (KiB): 2109308

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39994.7
system time used= 6.34903
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 436379
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 467832
involuntary context switches= 196158

runsolver used 16.9274 second user time and 41.8796 second system time

The end

Launcher Data 

Begin job on node125 at 2011-05-16 03:55:30
IDJOB=3381261
IDBENCH=85538
IDSOLVER=1865
FILE ID=node125/3381261-1305510930
RUNJOBID= node125-1305510930-11429
PBS_JOBID= 13325528
Free space on /tmp= 73188 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT05/crafted/sabharwal05/counting/fphp/unsat/harder/fphp-030-029.shuffled-as.sat05-1234.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3381261-1305510930/watcher-3381261-1305510930 -o /tmp/evaluation-result-3381261-1305510930/solver-3381261-1305510930 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381261-1305510930.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= c12aa6a0ac1bca448bef51c167f3bbbb
RANDOM SEED=22564520

node125.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5266.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      19350144 kB
Buffers:        189832 kB
Cached:        1908112 kB
SwapCached:      55652 kB
Active:        6133372 kB
Inactive:      7138716 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      19350144 kB
SwapTotal:    67111528 kB
SwapFree:     67042944 kB
Dirty:            1240 kB
Writeback:           0 kB
AnonPages:    11118780 kB
Mapped:          34284 kB
Slab:           217988 kB
PageTables:      43320 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 79237140 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73264 MiB
End job on node125 at 2011-05-16 05:19:01