Trace number 3311249

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
STRUCTure 0.1 (hbr=2)? (TO) 4800.37 1205.74

General information on the benchmark

Namerandom/large/
unif-k5-r20-v750-c15000-S64285713-004-UNKNOWN.cnf
MD5SUM57320bb4a9b2a7589463bdcd0ecf61a4
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark4.54731
Satisfiable
(Un)Satisfiability was proved
Number of variables750
Number of clauses15000
Sum of the clauses size75000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 515000
Number of clauses of size over 50

Solver Data 

0.10/0.12	28:55 INFO : Reading from input HOME/instance-3311249-1304116135.cnf
0.51/0.37	Executor set job limit to 1000000
0.51/0.37	Executor set job limit to 1000000
0.51/0.37	Executor set job limit to 1000000
0.51/0.37	Executor set job limit to 1000000
0.51/0.37	Starting Executor!
0.51/0.37	Starting Executor!
0.51/0.37	Starting Executor!
0.51/0.37	Starting Executor!
1.71/0.89	28:56 INFO : 750 variables remaining out of 750
1.71/0.89	28:56 INFO : 90000 literals remaining out of 90000

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3311249-1304116135/watcher-3311249-1304116135 -o /tmp/evaluation-result-3311249-1304116135/solver-3311249-1304116135 -C 4800 -W 1300 -M 15500 run 15000 4 HOME HOME HOME/instance-3311249-1304116135.cnf -hbr 2 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.10 5.34 5.68 3/198 9862
/proc/meminfo: memFree=12741688/32951124 swapFree=67111524/67111528
[pid=9862] ppid=9860 vsize=20404 CPUtime=0 cores=0,2,4,6
/proc/9862/stat : 9862 (java) R 9860 9862 9441 0 -1 4202496 670 0 0 0 0 0 0 0 25 0 1 0 523213945 20893696 158 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230804242103 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/9862/statm: 5101 160 128 9 0 124 0

[startup+0.011737 s]
/proc/loadavg: 5.10 5.34 5.68 3/198 9862
/proc/meminfo: memFree=12741688/32951124 swapFree=67111524/67111528
[pid=9862] ppid=9860 vsize=36924 CPUtime=0 cores=0,2,4,6
/proc/9862/stat : 9862 (java) S 9860 9862 9441 0 -1 4202496 1625 0 1 0 0 0 0 0 25 0 2 0 523213945 37810176 1071 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820968421 0 0 0 16784584 18446744073709551615 0 0 17 2 0 0 0
/proc/9862/statm: 9231 1071 832 9 0 387 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 36924

[startup+0.100761 s]
/proc/loadavg: 5.10 5.34 5.68 3/198 9862
/proc/meminfo: memFree=12741688/32951124 swapFree=67111524/67111528
[pid=9862] ppid=9860 vsize=15676904 CPUtime=0.1 cores=0,2,4,6
/proc/9862/stat : 9862 (java) S 9860 9862 9441 0 -1 4202496 5561 0 1 0 9 1 0 0 25 0 18 0 523213945 16053149696 4973 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 2 0 0 0
/proc/9862/statm: 3919226 4973 2116 9 0 3909945 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 15676904

[startup+0.300707 s]
/proc/loadavg: 5.10 5.34 5.68 3/198 9862
/proc/meminfo: memFree=12741688/32951124 swapFree=67111524/67111528
[pid=9862] ppid=9860 vsize=15683012 CPUtime=0.51 cores=0,2,4,6
/proc/9862/stat : 9862 (java) S 9860 9862 9441 0 -1 4202496 14391 0 1 0 49 2 0 0 25 0 18 0 523213945 16059404288 12642 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 2 0 0 0
/proc/9862/statm: 3920753 12646 2207 9 0 3911472 0
Current children cumulated CPU time (s) 0.51
Current children cumulated vsize (KiB) 15683012

[startup+0.700607 s]
/proc/loadavg: 5.10 5.34 5.68 3/198 9862
/proc/meminfo: memFree=12741688/32951124 swapFree=67111524/67111528
[pid=9862] ppid=9860 vsize=15698048 CPUtime=1.41 cores=0,2,4,6
/proc/9862/stat : 9862 (java) S 9860 9862 9441 0 -1 4202496 37046 0 1 0 136 5 0 0 25 0 22 0 523213945 16074801152 33078 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 2 0 0 0
/proc/9862/statm: 3924512 33078 2242 9 0 3915231 0
Current children cumulated CPU time (s) 1.41
Current children cumulated vsize (KiB) 15698048

[startup+1.50626 s]
/proc/loadavg: 5.02 5.32 5.67 5/220 9884
/proc/meminfo: memFree=12608592/32951124 swapFree=67111524/67111528
[pid=9862] ppid=9860 vsize=15699852 CPUtime=3.78 cores=0,2,4,6
/proc/9862/stat : 9862 (java) S 9860 9862 9441 0 -1 4202496 57663 0 1 0 370 8 0 0 25 0 22 0 523213945 16076648448 49890 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 2 0 0 0
/proc/9862/statm: 3924963 49890 2274 9 0 3915682 0
[pid=9862/tid=9864] ppid=9860 vsize=15699852 CPUtime=0.86 cores=0,2,4,6
/proc/9862/task/9864/stat : 9864 (java) S 9860 9862 9441 0 -1 4202560 21908 0 1 0 84 2 0 0 20 0 22 0 523213946 16076648448 49890 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9865] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9865/stat : 9865 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 17 0 22 0 523213946 16076648448 49890 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9866] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9866/stat : 9866 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 17 0 22 0 523213946 16076648448 49890 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9867] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9867/stat : 9867 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 17 0 22 0 523213946 16076648448 49890 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9868] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9868/stat : 9868 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 17 0 22 0 523213946 16076648448 49890 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9869] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9869/stat : 9869 (java) S 9860 9862 9441 0 -1 4202560 224 0 0 0 0 0 0 0 17 0 22 0 523213946 16076648448 49890 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9870] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9870/stat : 9870 (java) R 9860 9862 9441 0 -1 4202560 591 0 0 0 0 0 0 0 15 0 22 0 523213946 16076648448 49890 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820982726 0 4 0 16800975 0 0 0 -1 4 0 0 0
[pid=9862/tid=9871] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9871/stat : 9871 (java) R 9860 9862 9441 0 -1 4202560 938 0 0 0 0 0 0 0 15 0 22 0 523213946 16076648448 49891 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 0 0 0 -1 4 0 0 0
[pid=9862/tid=9872] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9872/stat : 9872 (java) S 9860 9862 9441 0 -1 4202560 451 0 0 0 0 0 0 0 17 0 22 0 523213946 16076648448 49893 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9873] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9873/stat : 9873 (java) S 9860 9862 9441 0 -1 4202560 11 0 0 0 0 0 0 0 15 0 22 0 523213947 16076648448 49894 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 0 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9874] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9874/stat : 9874 (java) S 9860 9862 9441 0 -1 4202560 17 0 0 0 0 0 0 0 18 0 22 0 523213948 16076648448 49896 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9875] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9875/stat : 9875 (java) S 9860 9862 9441 0 -1 4202560 8 0 0 0 0 0 0 0 18 0 22 0 523213948 16076648448 49898 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9876] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9876/stat : 9876 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 523213950 16076648448 49901 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820989361 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9877] ppid=9860 vsize=15699852 CPUtime=0.86 cores=0,2,4,6
/proc/9862/task/9877/stat : 9877 (java) S 9860 9862 9441 0 -1 4202560 5753 0 0 0 85 1 0 0 15 0 22 0 523213950 16076648448 49904 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9878] ppid=9860 vsize=15699852 CPUtime=0.81 cores=0,2,4,6
/proc/9862/task/9878/stat : 9878 (java) S 9860 9862 9441 0 -1 4202560 12623 0 0 0 79 2 0 0 15 0 22 0 523213950 16076648448 49905 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9879] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9879/stat : 9879 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 523213950 16076648448 49907 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9880] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9880/stat : 9880 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 22 0 523213950 16076648448 49909 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820982272 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9881] ppid=9860 vsize=15699852 CPUtime=0.2 cores=0,2,4,6
/proc/9862/task/9881/stat : 9881 (java) S 9860 9862 9441 0 -1 4202560 1055 0 0 0 20 0 0 0 16 0 22 0 523213982 16076648448 49913 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9882] ppid=9860 vsize=15699852 CPUtime=0.23 cores=0,2,4,6
/proc/9862/task/9882/stat : 9882 (java) S 9860 9862 9441 0 -1 4202560 1297 0 0 0 23 0 0 0 16 0 22 0 523213982 16076648448 49919 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9883] ppid=9860 vsize=15699852 CPUtime=0.57 cores=0,2,4,6
/proc/9862/task/9883/stat : 9883 (java) S 9860 9862 9441 0 -1 4202560 9814 0 0 0 56 1 0 0 16 0 22 0 523213983 16076648448 49920 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9884] ppid=9860 vsize=15699852 CPUtime=0.18 cores=0,2,4,6
/proc/9862/task/9884/stat : 9884 (java) S 9860 9862 9441 0 -1 4202560 1609 0 0 0 18 0 0 0 16 0 22 0 523213983 16076648448 49921 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3.78
Current children cumulated vsize (KiB) 15699852

[startup+3.10114 s]
/proc/loadavg: 5.02 5.32 5.67 6/220 9884
/proc/meminfo: memFree=12480252/32951124 swapFree=67111524/67111528
[pid=9862] ppid=9860 vsize=15699852 CPUtime=10.14 cores=0,2,4,6
/proc/9862/stat : 9862 (java) S 9860 9862 9441 0 -1 4202496 119720 0 1 0 995 19 0 0 25 0 22 0 523213945 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 2 0 0 0
/proc/9862/statm: 3924963 111832 2274 9 0 3915682 0
[pid=9862/tid=9864] ppid=9860 vsize=15699852 CPUtime=0.86 cores=0,2,4,6
/proc/9862/task/9864/stat : 9864 (java) S 9860 9862 9441 0 -1 4202560 21908 0 1 0 84 2 0 0 20 0 22 0 523213946 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9865] ppid=9860 vsize=15699852 CPUtime=0.01 cores=0,2,4,6
/proc/9862/task/9865/stat : 9865 (java) S 9860 9862 9441 0 -1 4202560 5955 0 0 0 0 1 0 0 15 0 22 0 523213946 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9866] ppid=9860 vsize=15699852 CPUtime=0.01 cores=0,2,4,6
/proc/9862/task/9866/stat : 9866 (java) S 9860 9862 9441 0 -1 4202560 5409 0 0 0 1 0 0 0 15 0 22 0 523213946 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9867] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9867/stat : 9867 (java) S 9860 9862 9441 0 -1 4202560 3163 0 0 0 0 0 0 0 15 0 22 0 523213946 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 6 0 0 0
[pid=9862/tid=9868] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9868/stat : 9868 (java) S 9860 9862 9441 0 -1 4202560 4006 0 0 0 0 0 0 0 16 0 22 0 523213946 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9869] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9869/stat : 9869 (java) S 9860 9862 9441 0 -1 4202560 4642 0 0 0 0 0 0 0 16 0 22 0 523213946 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9870] ppid=9860 vsize=15699852 CPUtime=0.02 cores=0,2,4,6
/proc/9862/task/9870/stat : 9870 (java) S 9860 9862 9441 0 -1 4202560 7778 0 0 0 1 1 0 0 16 0 22 0 523213946 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 6 0 0 0
[pid=9862/tid=9871] ppid=9860 vsize=15699852 CPUtime=0.02 cores=0,2,4,6
/proc/9862/task/9871/stat : 9871 (java) S 9860 9862 9441 0 -1 4202560 6753 0 0 0 1 1 0 0 16 0 22 0 523213946 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9872] ppid=9860 vsize=15699852 CPUtime=0.01 cores=0,2,4,6
/proc/9862/task/9872/stat : 9872 (java) S 9860 9862 9441 0 -1 4202560 4268 0 0 0 1 0 0 0 15 0 22 0 523213946 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 6 0 0 0
[pid=9862/tid=9873] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9873/stat : 9873 (java) S 9860 9862 9441 0 -1 4202560 236 0 0 0 0 0 0 0 15 0 22 0 523213947 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820982272 0 0 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9874] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9874/stat : 9874 (java) S 9860 9862 9441 0 -1 4202560 17 0 0 0 0 0 0 0 18 0 22 0 523213948 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9875] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9875/stat : 9875 (java) S 9860 9862 9441 0 -1 4202560 8 0 0 0 0 0 0 0 18 0 22 0 523213948 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9876] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9876/stat : 9876 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 523213950 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820989361 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9877] ppid=9860 vsize=15699852 CPUtime=0.89 cores=0,2,4,6
/proc/9862/task/9877/stat : 9877 (java) S 9860 9862 9441 0 -1 4202560 5756 0 0 0 88 1 0 0 15 0 22 0 523213950 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9878] ppid=9860 vsize=15699852 CPUtime=0.81 cores=0,2,4,6
/proc/9862/task/9878/stat : 9878 (java) S 9860 9862 9441 0 -1 4202560 12623 0 0 0 79 2 0 0 15 0 22 0 523213950 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9879] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9879/stat : 9879 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 523213950 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9880] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9880/stat : 9880 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 22 0 523213950 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820982272 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9881] ppid=9860 vsize=15699852 CPUtime=1.74 cores=0,2,4,6
/proc/9862/task/9881/stat : 9881 (java) R 9860 9862 9441 0 -1 4202560 6237 0 0 0 173 1 0 0 17 0 22 0 523213982 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912505904360 0 4 0 16800975 0 0 0 -1 6 0 0 0
[pid=9862/tid=9882] ppid=9860 vsize=15699852 CPUtime=1.77 cores=0,2,4,6
/proc/9862/task/9882/stat : 9882 (java) R 9860 9862 9441 0 -1 4202560 7289 0 0 0 177 0 0 0 17 0 22 0 523213982 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912505868515 0 4 0 16800975 0 0 0 -1 2 0 0 0
[pid=9862/tid=9883] ppid=9860 vsize=15699852 CPUtime=2.12 cores=0,2,4,6
/proc/9862/task/9883/stat : 9883 (java) R 9860 9862 9441 0 -1 4202560 14979 0 0 0 211 1 0 0 17 0 22 0 523213983 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912505718580 0 4 0 16800975 0 0 0 -1 4 0 0 0
[pid=9862/tid=9884] ppid=9860 vsize=15699852 CPUtime=1.7 cores=0,2,4,6
/proc/9862/task/9884/stat : 9884 (java) R 9860 9862 9441 0 -1 4202560 7333 0 0 0 170 0 0 0 17 0 22 0 523213983 16076648448 111832 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912505939047 0 4 0 16800975 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 10.14
Current children cumulated vsize (KiB) 15699852

[startup+6.30155 s]
/proc/loadavg: 5.02 5.32 5.67 6/220 9884
/proc/meminfo: memFree=11903776/32951124 swapFree=67111524/67111528
[pid=9862] ppid=9860 vsize=15699852 CPUtime=22.92 cores=0,2,4,6
/proc/9862/stat : 9862 (java) S 9860 9862 9441 0 -1 4202496 295692 0 1 0 2247 45 0 0 25 0 22 0 523213945 16076648448 287268 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 2 0 0 0
/proc/9862/statm: 3924963 287280 2277 9 0 3915682 0
[pid=9862/tid=9864] ppid=9860 vsize=15699852 CPUtime=0.86 cores=0,2,4,6
/proc/9862/task/9864/stat : 9864 (java) S 9860 9862 9441 0 -1 4202560 21908 0 1 0 84 2 0 0 20 0 22 0 523213946 16076648448 287284 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9865] ppid=9860 vsize=15699852 CPUtime=0.05 cores=0,2,4,6
/proc/9862/task/9865/stat : 9865 (java) S 9860 9862 9441 0 -1 4202560 12027 0 0 0 2 3 0 0 16 0 22 0 523213946 16076648448 287285 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9866] ppid=9860 vsize=15699852 CPUtime=0.06 cores=0,2,4,6
/proc/9862/task/9866/stat : 9866 (java) S 9860 9862 9441 0 -1 4202560 17928 0 0 0 3 3 0 0 16 0 22 0 523213946 16076648448 287286 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 6 0 0 0
[pid=9862/tid=9867] ppid=9860 vsize=15699852 CPUtime=0.03 cores=0,2,4,6
/proc/9862/task/9867/stat : 9867 (java) S 9860 9862 9441 0 -1 4202560 8511 0 0 0 2 1 0 0 15 0 22 0 523213946 16076648448 287287 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 6 0 0 0
[pid=9862/tid=9868] ppid=9860 vsize=15699852 CPUtime=0.06 cores=0,2,4,6
/proc/9862/task/9868/stat : 9868 (java) S 9860 9862 9441 0 -1 4202560 16006 0 0 0 3 3 0 0 16 0 22 0 523213946 16076648448 287288 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9869] ppid=9860 vsize=15699852 CPUtime=0.04 cores=0,2,4,6
/proc/9862/task/9869/stat : 9869 (java) S 9860 9862 9441 0 -1 4202560 10649 0 0 0 2 2 0 0 16 0 22 0 523213946 16076648448 287290 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9870] ppid=9860 vsize=15699852 CPUtime=0.02 cores=0,2,4,6
/proc/9862/task/9870/stat : 9870 (java) S 9860 9862 9441 0 -1 4202560 7778 0 0 0 1 1 0 0 15 0 22 0 523213946 16076648448 287291 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 6 0 0 0
[pid=9862/tid=9871] ppid=9860 vsize=15699852 CPUtime=0.04 cores=0,2,4,6
/proc/9862/task/9871/stat : 9871 (java) S 9860 9862 9441 0 -1 4202560 12424 0 0 0 3 1 0 0 16 0 22 0 523213946 16076648448 287293 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9872] ppid=9860 vsize=15699852 CPUtime=0.01 cores=0,2,4,6
/proc/9862/task/9872/stat : 9872 (java) S 9860 9862 9441 0 -1 4202560 4268 0 0 0 1 0 0 0 15 0 22 0 523213946 16076648448 287294 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 6 0 0 0
[pid=9862/tid=9873] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9873/stat : 9873 (java) S 9860 9862 9441 0 -1 4202560 527 0 0 0 0 0 0 0 15 0 22 0 523213947 16076648448 287296 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820982272 0 0 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9874] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9874/stat : 9874 (java) S 9860 9862 9441 0 -1 4202560 17 0 0 0 0 0 0 0 18 0 22 0 523213948 16076648448 287298 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9875] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9875/stat : 9875 (java) S 9860 9862 9441 0 -1 4202560 8 0 0 0 0 0 0 0 18 0 22 0 523213948 16076648448 287299 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9876] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9876/stat : 9876 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 523213950 16076648448 287301 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820989361 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9877] ppid=9860 vsize=15699852 CPUtime=1.02 cores=0,2,4,6
/proc/9862/task/9877/stat : 9877 (java) S 9860 9862 9441 0 -1 4202560 5776 0 0 0 101 1 0 0 15 0 22 0 523213950 16076648448 287302 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0

################
# More data... #
################

[pid=9862/tid=9882] ppid=9860 vsize=15699852 CPUtime=1108.87 cores=0,2,4,6
/proc/9862/task/9882/stat : 9882 (java) R 9860 9862 9441 0 -1 4202560 353142 0 0 0 110843 44 0 0 25 0 22 0 523213982 16076648448 129911 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46979615471915 0 4 0 16800975 0 0 0 -1 2 0 0 0
[pid=9862/tid=9883] ppid=9860 vsize=15699852 CPUtime=1109.09 cores=0,2,4,6
/proc/9862/task/9883/stat : 9883 (java) R 9860 9862 9441 0 -1 4202560 366571 0 0 0 110859 50 0 0 25 0 22 0 523213983 16076648448 129911 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 18446744073699065949 0 4 0 16800975 0 0 0 -1 0 0 0 0
[pid=9862/tid=9884] ppid=9860 vsize=15699852 CPUtime=1108.3 cores=0,2,4,6
/proc/9862/task/9884/stat : 9884 (java) R 9860 9862 9441 0 -1 4202560 354140 0 0 0 110782 48 0 0 25 0 22 0 523213983 16076648448 129911 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912505608445 0 4 0 16800975 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4468.17
Current children cumulated vsize (KiB) 15699852

[startup+1182.3 s]
/proc/loadavg: 8.10 8.02 7.31 9/220 9966
/proc/meminfo: memFree=22220664/32951124 swapFree=67111524/67111528
[pid=9862] ppid=9860 vsize=15699852 CPUtime=4707.06 cores=0,2,4,6
/proc/9862/stat : 9862 (java) S 9860 9862 9441 0 -1 4202496 2912295 0 1 0 470182 524 0 0 25 0 22 0 523213945 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 2 0 0 0
/proc/9862/statm: 3924963 148147 2298 9 0 3915682 0
[pid=9862/tid=9864] ppid=9860 vsize=15699852 CPUtime=0.86 cores=0,2,4,6
/proc/9862/task/9864/stat : 9864 (java) S 9860 9862 9441 0 -1 4202560 21908 0 1 0 84 2 0 0 20 0 22 0 523213946 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9865] ppid=9860 vsize=15699852 CPUtime=3.69 cores=0,2,4,6
/proc/9862/task/9865/stat : 9865 (java) S 9860 9862 9441 0 -1 4202560 172335 0 0 0 337 32 0 0 16 0 22 0 523213946 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9866] ppid=9860 vsize=15699852 CPUtime=3.81 cores=0,2,4,6
/proc/9862/task/9866/stat : 9866 (java) S 9860 9862 9441 0 -1 4202560 168410 0 0 0 351 30 0 0 15 0 22 0 523213946 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 6 0 0 0
[pid=9862/tid=9867] ppid=9860 vsize=15699852 CPUtime=3.69 cores=0,2,4,6
/proc/9862/task/9867/stat : 9867 (java) S 9860 9862 9441 0 -1 4202560 163635 0 0 0 340 29 0 0 16 0 22 0 523213946 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9868] ppid=9860 vsize=15699852 CPUtime=3.76 cores=0,2,4,6
/proc/9862/task/9868/stat : 9868 (java) S 9860 9862 9441 0 -1 4202560 176354 0 0 0 343 33 0 0 16 0 22 0 523213946 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 6 0 0 0
[pid=9862/tid=9869] ppid=9860 vsize=15699852 CPUtime=3.69 cores=0,2,4,6
/proc/9862/task/9869/stat : 9869 (java) S 9860 9862 9441 0 -1 4202560 174777 0 0 0 339 30 0 0 16 0 22 0 523213946 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9870] ppid=9860 vsize=15699852 CPUtime=3.69 cores=0,2,4,6
/proc/9862/task/9870/stat : 9870 (java) S 9860 9862 9441 0 -1 4202560 167492 0 0 0 338 31 0 0 15 0 22 0 523213946 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9871] ppid=9860 vsize=15699852 CPUtime=3.82 cores=0,2,4,6
/proc/9862/task/9871/stat : 9871 (java) S 9860 9862 9441 0 -1 4202560 178406 0 0 0 350 32 0 0 15 0 22 0 523213946 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 6 0 0 0
[pid=9862/tid=9872] ppid=9860 vsize=15699852 CPUtime=3.77 cores=0,2,4,6
/proc/9862/task/9872/stat : 9872 (java) S 9860 9862 9441 0 -1 4202560 165355 0 0 0 351 26 0 0 16 0 22 0 523213946 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 6 0 0 0
[pid=9862/tid=9873] ppid=9860 vsize=15699852 CPUtime=1.74 cores=0,2,4,6
/proc/9862/task/9873/stat : 9873 (java) S 9860 9862 9441 0 -1 4202560 7713 0 0 0 105 69 0 0 16 0 22 0 523213947 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820982272 0 0 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9874] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9874/stat : 9874 (java) S 9860 9862 9441 0 -1 4202560 17 0 0 0 0 0 0 0 18 0 22 0 523213948 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9875] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9875/stat : 9875 (java) S 9860 9862 9441 0 -1 4202560 8 0 0 0 0 0 0 0 18 0 22 0 523213948 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9876] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9876/stat : 9876 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 523213950 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820989361 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9877] ppid=9860 vsize=15699852 CPUtime=1.28 cores=0,2,4,6
/proc/9862/task/9877/stat : 9877 (java) S 9860 9862 9441 0 -1 4202560 5814 0 0 0 127 1 0 0 15 0 22 0 523213950 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9878] ppid=9860 vsize=15699852 CPUtime=1.04 cores=0,2,4,6
/proc/9862/task/9878/stat : 9878 (java) S 9860 9862 9441 0 -1 4202560 12666 0 0 0 102 2 0 0 15 0 22 0 523213950 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9879] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9879/stat : 9879 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 523213950 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9880] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9880/stat : 9880 (java) S 9860 9862 9441 0 -1 4202560 67 0 0 0 0 0 0 0 15 0 22 0 523213950 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820982272 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9881] ppid=9860 vsize=15699852 CPUtime=1167.99 cores=0,2,4,6
/proc/9862/task/9881/stat : 9881 (java) R 9860 9862 9441 0 -1 4202560 371643 0 0 0 116750 49 0 0 25 0 22 0 523213982 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912505849295 0 4 0 16800975 0 0 0 -1 0 0 0 0
[pid=9862/tid=9882] ppid=9860 vsize=15699852 CPUtime=1168.13 cores=0,2,4,6
/proc/9862/task/9882/stat : 9882 (java) R 9860 9862 9441 0 -1 4202560 368294 0 0 0 116767 46 0 0 25 0 22 0 523213982 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912505523903 0 4 0 16800975 0 0 0 -1 4 0 0 0
[pid=9862/tid=9883] ppid=9860 vsize=15699852 CPUtime=1168.38 cores=0,2,4,6
/proc/9862/task/9883/stat : 9883 (java) R 9860 9862 9441 0 -1 4202560 384287 0 0 0 116785 53 0 0 25 0 22 0 523213983 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912505928929 0 4 0 16800975 0 0 0 -1 2 0 0 0
[pid=9862/tid=9884] ppid=9860 vsize=15699852 CPUtime=1167.57 cores=0,2,4,6
/proc/9862/task/9884/stat : 9884 (java) R 9860 9862 9441 0 -1 4202560 371755 0 0 0 116706 51 0 0 25 0 22 0 523213983 16076648448 148147 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912506198557 0 4 0 16800975 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4707.06
Current children cumulated vsize (KiB) 15699852



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1205.7 s]
/proc/loadavg: 8.07 8.02 7.33 9/220 9967
/proc/meminfo: memFree=22207736/32951124 swapFree=67111524/67111528
[pid=9862] ppid=9860 vsize=15699852 CPUtime=4800.31 cores=0,2,4,6
/proc/9862/stat : 9862 (java) S 9860 9862 9441 0 -1 4202496 2950761 0 1 0 479499 532 0 0 25 0 22 0 523213945 16076648448 130941 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 2 0 0 0
/proc/9862/statm: 3924963 130941 2298 9 0 3915682 0
[pid=9862/tid=9864] ppid=9860 vsize=15699852 CPUtime=0.86 cores=0,2,4,6
/proc/9862/task/9864/stat : 9864 (java) S 9860 9862 9441 0 -1 4202560 21908 0 1 0 84 2 0 0 20 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9865] ppid=9860 vsize=15699852 CPUtime=3.77 cores=0,2,4,6
/proc/9862/task/9865/stat : 9865 (java) S 9860 9862 9441 0 -1 4202560 174444 0 0 0 345 32 0 0 15 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9866] ppid=9860 vsize=15699852 CPUtime=3.89 cores=0,2,4,6
/proc/9862/task/9866/stat : 9866 (java) S 9860 9862 9441 0 -1 4202560 171996 0 0 0 358 31 0 0 16 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9867] ppid=9860 vsize=15699852 CPUtime=3.77 cores=0,2,4,6
/proc/9862/task/9867/stat : 9867 (java) S 9860 9862 9441 0 -1 4202560 166454 0 0 0 347 30 0 0 15 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9868] ppid=9860 vsize=15699852 CPUtime=3.83 cores=0,2,4,6
/proc/9862/task/9868/stat : 9868 (java) S 9860 9862 9441 0 -1 4202560 178755 0 0 0 349 34 0 0 15 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9869] ppid=9860 vsize=15699852 CPUtime=3.77 cores=0,2,4,6
/proc/9862/task/9869/stat : 9869 (java) S 9860 9862 9441 0 -1 4202560 178182 0 0 0 347 30 0 0 16 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9870] ppid=9860 vsize=15699852 CPUtime=3.78 cores=0,2,4,6
/proc/9862/task/9870/stat : 9870 (java) S 9860 9862 9441 0 -1 4202560 170030 0 0 0 346 32 0 0 16 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9871] ppid=9860 vsize=15699852 CPUtime=3.91 cores=0,2,4,6
/proc/9862/task/9871/stat : 9871 (java) S 9860 9862 9441 0 -1 4202560 182031 0 0 0 358 33 0 0 15 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9872] ppid=9860 vsize=15699852 CPUtime=3.84 cores=0,2,4,6
/proc/9862/task/9872/stat : 9872 (java) S 9860 9862 9441 0 -1 4202560 167407 0 0 0 358 26 0 0 16 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 6 0 0 0
[pid=9862/tid=9873] ppid=9860 vsize=15699852 CPUtime=1.76 cores=0,2,4,6
/proc/9862/task/9873/stat : 9873 (java) S 9860 9862 9441 0 -1 4202560 7816 0 0 0 106 70 0 0 15 0 22 0 523213947 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820982272 0 0 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9874] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9874/stat : 9874 (java) S 9860 9862 9441 0 -1 4202560 17 0 0 0 0 0 0 0 18 0 22 0 523213948 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9875] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9875/stat : 9875 (java) S 9860 9862 9441 0 -1 4202560 8 0 0 0 0 0 0 0 18 0 22 0 523213948 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9876] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9876/stat : 9876 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 523213950 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820989361 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9877] ppid=9860 vsize=15699852 CPUtime=1.28 cores=0,2,4,6
/proc/9862/task/9877/stat : 9877 (java) S 9860 9862 9441 0 -1 4202560 5814 0 0 0 127 1 0 0 15 0 22 0 523213950 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9878] ppid=9860 vsize=15699852 CPUtime=1.04 cores=0,2,4,6
/proc/9862/task/9878/stat : 9878 (java) S 9860 9862 9441 0 -1 4202560 12666 0 0 0 102 2 0 0 15 0 22 0 523213950 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9879] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9879/stat : 9879 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 523213950 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9880] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9880/stat : 9880 (java) S 9860 9862 9441 0 -1 4202560 69 0 0 0 0 0 0 0 15 0 22 0 523213950 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820982272 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9881] ppid=9860 vsize=15699852 CPUtime=1191.17 cores=0,2,4,6
/proc/9862/task/9881/stat : 9881 (java) R 9860 9862 9441 0 -1 4202560 376316 0 0 0 119067 50 0 0 25 0 22 0 523213982 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912506442226 0 4 0 16800975 0 0 0 -1 0 0 0 0
[pid=9862/tid=9882] ppid=9860 vsize=15699852 CPUtime=1191.31 cores=0,2,4,6
/proc/9862/task/9882/stat : 9882 (java) R 9860 9862 9441 0 -1 4202560 371741 0 0 0 119084 47 0 0 25 0 22 0 523213982 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912506098609 0 4 0 16800975 0 0 0 -1 4 0 0 0
[pid=9862/tid=9883] ppid=9860 vsize=15699852 CPUtime=1191.57 cores=0,2,4,6
/proc/9862/task/9883/stat : 9883 (java) R 9860 9862 9441 0 -1 4202560 387520 0 0 0 119103 54 0 0 25 0 22 0 523213983 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912505600808 0 4 0 16800975 0 0 0 -1 2 0 0 0
[pid=9862/tid=9884] ppid=9860 vsize=15699852 CPUtime=1190.61 cores=0,2,4,6
/proc/9862/task/9884/stat : 9884 (java) R 9860 9862 9441 0 -1 4202560 376232 0 0 0 119010 51 0 0 25 0 22 0 523213983 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912505848879 0 4 0 16800975 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.31
Current children cumulated vsize (KiB) 15699852

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1205.7 s]
/proc/loadavg: 8.07 8.02 7.33 9/220 9967
/proc/meminfo: memFree=22207736/32951124 swapFree=67111524/67111528
[pid=9862] ppid=9860 vsize=15699852 CPUtime=4800.31 cores=0,2,4,6
/proc/9862/stat : 9862 (java) S 9860 9862 9441 0 -1 4202496 2950761 0 1 0 479499 532 0 0 25 0 22 0 523213945 16076648448 130941 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 2 0 0 0
/proc/9862/statm: 3924963 130941 2298 9 0 3915682 0
[pid=9862/tid=9864] ppid=9860 vsize=15699852 CPUtime=0.86 cores=0,2,4,6
/proc/9862/task/9864/stat : 9864 (java) S 9860 9862 9441 0 -1 4202560 21908 0 1 0 84 2 0 0 20 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9865] ppid=9860 vsize=15699852 CPUtime=3.77 cores=0,2,4,6
/proc/9862/task/9865/stat : 9865 (java) S 9860 9862 9441 0 -1 4202560 174444 0 0 0 345 32 0 0 15 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9866] ppid=9860 vsize=15699852 CPUtime=3.89 cores=0,2,4,6
/proc/9862/task/9866/stat : 9866 (java) S 9860 9862 9441 0 -1 4202560 171996 0 0 0 358 31 0 0 16 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9867] ppid=9860 vsize=15699852 CPUtime=3.77 cores=0,2,4,6
/proc/9862/task/9867/stat : 9867 (java) S 9860 9862 9441 0 -1 4202560 166454 0 0 0 347 30 0 0 15 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9868] ppid=9860 vsize=15699852 CPUtime=3.83 cores=0,2,4,6
/proc/9862/task/9868/stat : 9868 (java) S 9860 9862 9441 0 -1 4202560 178755 0 0 0 349 34 0 0 15 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9869] ppid=9860 vsize=15699852 CPUtime=3.77 cores=0,2,4,6
/proc/9862/task/9869/stat : 9869 (java) S 9860 9862 9441 0 -1 4202560 178182 0 0 0 347 30 0 0 16 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9870] ppid=9860 vsize=15699852 CPUtime=3.78 cores=0,2,4,6
/proc/9862/task/9870/stat : 9870 (java) S 9860 9862 9441 0 -1 4202560 170030 0 0 0 346 32 0 0 16 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9871] ppid=9860 vsize=15699852 CPUtime=3.91 cores=0,2,4,6
/proc/9862/task/9871/stat : 9871 (java) S 9860 9862 9441 0 -1 4202560 182031 0 0 0 358 33 0 0 15 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9872] ppid=9860 vsize=15699852 CPUtime=3.84 cores=0,2,4,6
/proc/9862/task/9872/stat : 9872 (java) S 9860 9862 9441 0 -1 4202560 167407 0 0 0 358 26 0 0 16 0 22 0 523213946 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 6 0 0 0
[pid=9862/tid=9873] ppid=9860 vsize=15699852 CPUtime=1.76 cores=0,2,4,6
/proc/9862/task/9873/stat : 9873 (java) S 9860 9862 9441 0 -1 4202560 7816 0 0 0 106 70 0 0 15 0 22 0 523213947 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820982272 0 0 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9874] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9874/stat : 9874 (java) S 9860 9862 9441 0 -1 4202560 17 0 0 0 0 0 0 0 18 0 22 0 523213948 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9875] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9875/stat : 9875 (java) S 9860 9862 9441 0 -1 4202560 8 0 0 0 0 0 0 0 18 0 22 0 523213948 16076648448 130944 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9876] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9876/stat : 9876 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 523213950 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820989361 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9877] ppid=9860 vsize=15699852 CPUtime=1.28 cores=0,2,4,6
/proc/9862/task/9877/stat : 9877 (java) S 9860 9862 9441 0 -1 4202560 5814 0 0 0 127 1 0 0 15 0 22 0 523213950 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9878] ppid=9860 vsize=15699852 CPUtime=1.04 cores=0,2,4,6
/proc/9862/task/9878/stat : 9878 (java) S 9860 9862 9441 0 -1 4202560 12666 0 0 0 102 2 0 0 15 0 22 0 523213950 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 4 0 0 0
[pid=9862/tid=9879] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9879/stat : 9879 (java) S 9860 9862 9441 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 523213950 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820981657 0 4 0 16800975 18446604451947262080 0 0 -1 2 0 0 0
[pid=9862/tid=9880] ppid=9860 vsize=15699852 CPUtime=0 cores=0,2,4,6
/proc/9862/task/9880/stat : 9880 (java) S 9860 9862 9441 0 -1 4202560 69 0 0 0 0 0 0 0 15 0 22 0 523213950 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 230820982272 0 4 0 16800975 18446604451947262080 0 0 -1 0 0 0 0
[pid=9862/tid=9881] ppid=9860 vsize=15699852 CPUtime=1191.17 cores=0,2,4,6
/proc/9862/task/9881/stat : 9881 (java) R 9860 9862 9441 0 -1 4202560 376316 0 0 0 119067 50 0 0 25 0 22 0 523213982 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912506442226 0 4 0 16800975 0 0 0 -1 0 0 0 0
[pid=9862/tid=9882] ppid=9860 vsize=15699852 CPUtime=1191.31 cores=0,2,4,6
/proc/9862/task/9882/stat : 9882 (java) R 9860 9862 9441 0 -1 4202560 371741 0 0 0 119084 47 0 0 25 0 22 0 523213982 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912506098609 0 4 0 16800975 0 0 0 -1 4 0 0 0
[pid=9862/tid=9883] ppid=9860 vsize=15699852 CPUtime=1191.57 cores=0,2,4,6
/proc/9862/task/9883/stat : 9883 (java) R 9860 9862 9441 0 -1 4202560 387520 0 0 0 119103 54 0 0 25 0 22 0 523213983 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912505600808 0 4 0 16800975 0 0 0 -1 2 0 0 0
[pid=9862/tid=9884] ppid=9860 vsize=15699852 CPUtime=1190.61 cores=0,2,4,6
/proc/9862/task/9884/stat : 9884 (java) R 9860 9862 9441 0 -1 4202560 376232 0 0 0 119010 51 0 0 25 0 22 0 523213983 16076648448 130945 33554432000 1073741824 1073778376 140737446556960 18446744073709551615 46912505848879 0 4 0 16800975 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.31
Current children cumulated vsize (KiB) 15699852

Child status: 143
Real time (s): 1205.74
CPU time (s): 4800.37
CPU user time (s): 4795.02
CPU system time (s): 5.35319
CPU usage (%): 398.127
Max. virtual memory (cumulated for all children) (KiB): 15699852

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4795.02
system time used= 5.35319
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2951601
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 133808
involuntary context switches= 95526

runsolver used 1.9767 second user time and 4.67229 second system time

The end

Launcher Data 

Begin job on node104 at 2011-04-30 00:28:55
IDJOB=3311249
IDBENCH=82634
IDSOLVER=1698
FILE ID=node104/3311249-1304116135
RUNJOBID= node104-1304112416-9459
PBS_JOBID= 13170692
Free space on /tmp= 71544 MiB

SOLVER NAME= STRUCTure 0.1 (hbr=2)
BENCH NAME= SAT11/random/large/unif-k5-r20-v750-c15000-S64285713-004-UNKNOWN.cnf
COMMAND LINE= run 15000 NBCORE TMPDIR DIR BENCHNAME -hbr 2
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3311249-1304116135/watcher-3311249-1304116135 -o /tmp/evaluation-result-3311249-1304116135/solver-3311249-1304116135 -C 4800 -W 1300 -M 15500  run 15000 4 HOME HOME HOME/instance-3311249-1304116135.cnf -hbr 2

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 57320bb4a9b2a7589463bdcd0ecf61a4
RANDOM SEED=1496356666

node104.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5293.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      12741960 kB
Buffers:       1251028 kB
Cached:        2898028 kB
SwapCached:          4 kB
Active:       15603796 kB
Inactive:      4069800 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      12741960 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:            2028 kB
Writeback:           0 kB
AnonPages:    15524120 kB
Mapped:          23460 kB
Slab:           442340 kB
PageTables:      34640 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 15712508 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71544 MiB
End job on node104 at 2011-04-30 00:49:03