Trace number 3311625

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
STRUCTure 0.1 (hbr=2)? (TO) 4800.21 1206.73

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v700-c2982-S1617172756-030.SATISFIABLE.cnf
MD5SUM47049d4705301f4ac565d8b47d11d0a5
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.006998
Satisfiable
(Un)Satisfiability was proved
Number of variables700
Number of clauses2982
Sum of the clauses size8946
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 32982
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.10/0.12	34:03 INFO : Reading from input HOME/instance-3311625-1304177643.cnf
0.26/0.26	Executor set job limit to 1000000
0.26/0.27	Executor set job limit to 1000000
0.26/0.27	Executor set job limit to 1000000
0.26/0.27	Executor set job limit to 1000000
0.26/0.27	Starting Executor!
0.26/0.27	Starting Executor!
0.26/0.27	Starting Executor!
0.26/0.27	Starting Executor!
0.77/0.42	34:03 INFO : 695 variables remaining out of 700
0.77/0.42	34:03 INFO : 11800 literals remaining out of 11928

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3311625-1304177643/watcher-3311625-1304177643 -o /tmp/evaluation-result-3311625-1304177643/solver-3311625-1304177643 -C 4800 -W 1300 -M 15500 run 15000 4 HOME HOME HOME/instance-3311625-1304177643.cnf -hbr 2 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.15 8.12 8.05 2/174 15507
/proc/meminfo: memFree=30919424/32951124 swapFree=67111448/67111528
[pid=15507] ppid=15505 vsize=23028 CPUtime=0 cores=0,2,4,6
/proc/15507/stat : 15507 (java) R 15505 15507 15356 0 -1 4202496 910 0 0 0 0 0 0 0 25 0 1 0 529364725 23580672 398 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230804203264 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/15507/statm: 5757 410 318 9 0 124 0

[startup+0.012234 s]
/proc/loadavg: 8.15 8.12 8.05 2/174 15507
/proc/meminfo: memFree=30919424/32951124 swapFree=67111448/67111528
[pid=15507] ppid=15505 vsize=15560980 CPUtime=0 cores=0,2,4,6
/proc/15507/stat : 15507 (java) S 15505 15507 15356 0 -1 4202496 1751 0 1 0 0 0 0 0 25 0 2 0 529364725 15934443520 1197 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820968421 0 0 0 16784584 18446744073709551615 0 0 17 4 0 0 0
/proc/15507/statm: 3890245 1217 906 9 0 3881401 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15560980

[startup+0.100241 s]
/proc/loadavg: 8.15 8.12 8.05 2/174 15507
/proc/meminfo: memFree=30919424/32951124 swapFree=67111448/67111528
[pid=15507] ppid=15505 vsize=15676872 CPUtime=0.1 cores=0,2,4,6
/proc/15507/stat : 15507 (java) S 15505 15507 15356 0 -1 4202496 5532 0 1 0 9 1 0 0 25 0 18 0 529364725 16053116928 4966 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 4 0 0 0
/proc/15507/statm: 3919218 4967 2116 9 0 3909937 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 15676872

[startup+0.301228 s]
/proc/loadavg: 8.15 8.12 8.05 2/174 15507
/proc/meminfo: memFree=30919424/32951124 swapFree=67111448/67111528
[pid=15507] ppid=15505 vsize=15687656 CPUtime=0.5 cores=0,2,4,6
/proc/15507/stat : 15507 (java) S 15505 15507 15356 0 -1 4202496 11422 0 1 0 48 2 0 0 25 0 22 0 529364725 16064159744 9950 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 4 0 0 0
/proc/15507/statm: 3921914 9950 2213 9 0 3912633 0
Current children cumulated CPU time (s) 0.5
Current children cumulated vsize (KiB) 15687656

[startup+1.01591 s]
/proc/loadavg: 8.15 8.12 8.05 2/174 15507
/proc/meminfo: memFree=30919424/32951124 swapFree=67111448/67111528
[pid=15507] ppid=15505 vsize=15711828 CPUtime=3.02 cores=0,2,4,6
/proc/15507/stat : 15507 (java) S 15505 15507 15356 0 -1 4202496 48906 0 1 0 294 8 0 0 25 0 22 0 529364725 16088911872 40869 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 4 0 0 0
/proc/15507/statm: 3927957 40869 2261 9 0 3918676 0
Current children cumulated CPU time (s) 3.02
Current children cumulated vsize (KiB) 15711828

[startup+1.50692 s]
/proc/loadavg: 8.15 8.12 8.05 9/220 15571
/proc/meminfo: memFree=30530796/32951124 swapFree=67111448/67111528
[pid=15507] ppid=15505 vsize=15709128 CPUtime=4.98 cores=0,2,4,6
/proc/15507/stat : 15507 (java) S 15505 15507 15356 0 -1 4202496 65559 0 1 0 487 11 0 0 25 0 22 0 529364725 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 4 0 0 0
/proc/15507/statm: 3927282 54681 2273 9 0 3918001 0
[pid=15507/tid=15509] ppid=15505 vsize=15709128 CPUtime=0.39 cores=0,2,4,6
/proc/15507/task/15509/stat : 15509 (java) S 15505 15507 15356 0 -1 4202560 7037 0 1 0 38 1 0 0 20 0 22 0 529364726 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15510] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15510/stat : 15510 (java) S 15505 15507 15356 0 -1 4202560 1554 0 0 0 0 0 0 0 16 0 22 0 529364726 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15511] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15511/stat : 15511 (java) S 15505 15507 15356 0 -1 4202560 1115 0 0 0 0 0 0 0 16 0 22 0 529364726 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 6 0 0 0
[pid=15507/tid=15512] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15512/stat : 15512 (java) S 15505 15507 15356 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 22 0 529364726 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 6 0 0 0
[pid=15507/tid=15513] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15513/stat : 15513 (java) S 15505 15507 15356 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 22 0 529364726 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 6 0 0 0
[pid=15507/tid=15514] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15514/stat : 15514 (java) S 15505 15507 15356 0 -1 4202560 1156 0 0 0 0 0 0 0 16 0 22 0 529364726 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15515] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15515/stat : 15515 (java) S 15505 15507 15356 0 -1 4202560 620 0 0 0 0 0 0 0 15 0 22 0 529364726 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15516] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15516/stat : 15516 (java) S 15505 15507 15356 0 -1 4202560 1191 0 0 0 0 0 0 0 16 0 22 0 529364726 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15517] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15517/stat : 15517 (java) S 15505 15507 15356 0 -1 4202560 968 0 0 0 0 0 0 0 15 0 22 0 529364726 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15518] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15518/stat : 15518 (java) S 15505 15507 15356 0 -1 4202560 75 0 0 0 0 0 0 0 15 0 22 0 529364727 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820982272 0 0 0 16800975 18446604445976933696 0 0 -1 6 0 0 0
[pid=15507/tid=15519] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15519/stat : 15519 (java) S 15505 15507 15356 0 -1 4202560 17 0 0 0 0 0 0 0 18 0 22 0 529364728 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15520] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15520/stat : 15520 (java) S 15505 15507 15356 0 -1 4202560 9 0 0 0 0 0 0 0 18 0 22 0 529364728 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15521] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15521/stat : 15521 (java) S 15505 15507 15356 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 529364730 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820989361 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15522] ppid=15505 vsize=15709128 CPUtime=0.95 cores=0,2,4,6
/proc/15507/task/15522/stat : 15522 (java) S 15505 15507 15356 0 -1 4202560 6807 0 0 0 94 1 0 0 17 0 22 0 529364730 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15523] ppid=15505 vsize=15709128 CPUtime=1.07 cores=0,2,4,6
/proc/15507/task/15523/stat : 15523 (java) S 15505 15507 15356 0 -1 4202560 14916 0 0 0 105 2 0 0 17 0 22 0 529364730 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15524] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15524/stat : 15524 (java) S 15505 15507 15356 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 529364730 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15525] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15525/stat : 15525 (java) S 15505 15507 15356 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 22 0 529364730 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820982272 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15564] ppid=15505 vsize=15709128 CPUtime=0.9 cores=0,2,4,6
/proc/15507/task/15564/stat : 15564 (java) R 15505 15507 15356 0 -1 4202560 12821 0 0 0 89 1 0 0 17 0 22 0 529364752 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505610984 0 4 0 16800975 0 0 0 -1 0 0 0 0
[pid=15507/tid=15565] ppid=15505 vsize=15709128 CPUtime=0.3 cores=0,2,4,6
/proc/15507/task/15565/stat : 15565 (java) R 15505 15507 15356 0 -1 4202560 823 0 0 0 30 0 0 0 19 0 22 0 529364752 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505746566 0 4 0 16800975 0 0 0 -1 4 0 0 0
[pid=15507/tid=15566] ppid=15505 vsize=15709128 CPUtime=0.82 cores=0,2,4,6
/proc/15507/task/15566/stat : 15566 (java) R 15505 15507 15356 0 -1 4202560 13424 0 0 0 81 1 0 0 17 0 22 0 529364752 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505798142 0 4 0 16800975 0 0 0 -1 6 0 0 0
[pid=15507/tid=15567] ppid=15505 vsize=15709128 CPUtime=0.43 cores=0,2,4,6
/proc/15507/task/15567/stat : 15567 (java) R 15505 15507 15356 0 -1 4202560 1665 0 0 0 43 0 0 0 18 0 22 0 529364752 16086147072 54681 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505878085 0 4 0 16800975 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4.98
Current children cumulated vsize (KiB) 15709128

[startup+3.10162 s]
/proc/loadavg: 8.14 8.12 8.05 9/220 15571
/proc/meminfo: memFree=30234624/32951124 swapFree=67111448/67111528
[pid=15507] ppid=15505 vsize=15709128 CPUtime=11.34 cores=0,2,4,6
/proc/15507/stat : 15507 (java) S 15505 15507 15356 0 -1 4202496 161275 0 1 0 1111 23 0 0 25 0 22 0 529364725 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 4 0 0 0
/proc/15507/statm: 3927282 150330 2277 9 0 3918001 0
[pid=15507/tid=15509] ppid=15505 vsize=15709128 CPUtime=0.39 cores=0,2,4,6
/proc/15507/task/15509/stat : 15509 (java) S 15505 15507 15356 0 -1 4202560 7037 0 1 0 38 1 0 0 20 0 22 0 529364726 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15510] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15510/stat : 15510 (java) S 15505 15507 15356 0 -1 4202560 2767 0 0 0 0 0 0 0 15 0 22 0 529364726 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15511] ppid=15505 vsize=15709128 CPUtime=0.01 cores=0,2,4,6
/proc/15507/task/15511/stat : 15511 (java) S 15505 15507 15356 0 -1 4202560 2550 0 0 0 1 0 0 0 16 0 22 0 529364726 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 6 0 0 0
[pid=15507/tid=15512] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15512/stat : 15512 (java) S 15505 15507 15356 0 -1 4202560 1129 0 0 0 0 0 0 0 15 0 22 0 529364726 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15513] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15513/stat : 15513 (java) S 15505 15507 15356 0 -1 4202560 2056 0 0 0 0 0 0 0 15 0 22 0 529364726 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15514] ppid=15505 vsize=15709128 CPUtime=0.01 cores=0,2,4,6
/proc/15507/task/15514/stat : 15514 (java) S 15505 15507 15356 0 -1 4202560 3810 0 0 0 1 0 0 0 16 0 22 0 529364726 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15515] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15515/stat : 15515 (java) S 15505 15507 15356 0 -1 4202560 1574 0 0 0 0 0 0 0 16 0 22 0 529364726 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15516] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15516/stat : 15516 (java) S 15505 15507 15356 0 -1 4202560 3307 0 0 0 0 0 0 0 15 0 22 0 529364726 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 6 0 0 0
[pid=15507/tid=15517] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15517/stat : 15517 (java) S 15505 15507 15356 0 -1 4202560 2592 0 0 0 0 0 0 0 15 0 22 0 529364726 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15518] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15518/stat : 15518 (java) S 15505 15507 15356 0 -1 4202560 449 0 0 0 0 0 0 0 15 0 22 0 529364727 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820982272 0 0 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15519] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15519/stat : 15519 (java) S 15505 15507 15356 0 -1 4202560 17 0 0 0 0 0 0 0 18 0 22 0 529364728 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15520] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15520/stat : 15520 (java) S 15505 15507 15356 0 -1 4202560 9 0 0 0 0 0 0 0 18 0 22 0 529364728 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15521] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15521/stat : 15521 (java) S 15505 15507 15356 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 529364730 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820989361 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15522] ppid=15505 vsize=15709128 CPUtime=0.96 cores=0,2,4,6
/proc/15507/task/15522/stat : 15522 (java) S 15505 15507 15356 0 -1 4202560 6807 0 0 0 95 1 0 0 15 0 22 0 529364730 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15523] ppid=15505 vsize=15709128 CPUtime=1.15 cores=0,2,4,6
/proc/15507/task/15523/stat : 15523 (java) S 15505 15507 15356 0 -1 4202560 14920 0 0 0 113 2 0 0 16 0 22 0 529364730 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15524] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15524/stat : 15524 (java) S 15505 15507 15356 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 529364730 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15525] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15525/stat : 15525 (java) S 15505 15507 15356 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 22 0 529364730 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820982272 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15564] ppid=15505 vsize=15709128 CPUtime=2.48 cores=0,2,4,6
/proc/15507/task/15564/stat : 15564 (java) R 15505 15507 15356 0 -1 4202560 33527 0 0 0 244 4 0 0 19 0 22 0 529364752 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 18446744073699066094 0 4 0 16800975 0 0 0 -1 0 0 0 0
[pid=15507/tid=15565] ppid=15505 vsize=15709128 CPUtime=1.8 cores=0,2,4,6
/proc/15507/task/15565/stat : 15565 (java) R 15505 15507 15356 0 -1 4202560 21549 0 0 0 177 3 0 0 21 0 22 0 529364752 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505656192 0 4 0 16800975 0 0 0 -1 4 0 0 0
[pid=15507/tid=15566] ppid=15505 vsize=15709128 CPUtime=2.39 cores=0,2,4,6
/proc/15507/task/15566/stat : 15566 (java) R 15505 15507 15356 0 -1 4202560 33796 0 0 0 236 3 0 0 20 0 22 0 529364752 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 18446744073699065939 0 4 0 16800975 0 0 0 -1 6 0 0 0
[pid=15507/tid=15567] ppid=15505 vsize=15709128 CPUtime=1.99 cores=0,2,4,6
/proc/15507/task/15567/stat : 15567 (java) R 15505 15507 15356 0 -1 4202560 22020 0 0 0 197 2 0 0 21 0 22 0 529364752 16086147072 150330 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505797996 0 4 0 16800975 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 11.34
Current children cumulated vsize (KiB) 15709128

[startup+6.30104 s]
/proc/loadavg: 8.14 8.12 8.05 9/220 15571
/proc/meminfo: memFree=29237100/32951124 swapFree=67111448/67111528
[pid=15507] ppid=15505 vsize=15709128 CPUtime=24.09 cores=0,2,4,6
/proc/15507/stat : 15507 (java) S 15505 15507 15356 0 -1 4202496 289134 0 1 0 2371 38 0 0 25 0 22 0 529364725 16086147072 276653 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 4 0 0 0
/proc/15507/statm: 3927282 276653 2277 9 0 3918001 0
[pid=15507/tid=15509] ppid=15505 vsize=15709128 CPUtime=0.39 cores=0,2,4,6
/proc/15507/task/15509/stat : 15509 (java) S 15505 15507 15356 0 -1 4202560 7037 0 1 0 38 1 0 0 20 0 22 0 529364726 16086147072 276653 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15510] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15510/stat : 15510 (java) S 15505 15507 15356 0 -1 4202560 3298 0 0 0 0 0 0 0 16 0 22 0 529364726 16086147072 276653 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15511] ppid=15505 vsize=15709128 CPUtime=0.01 cores=0,2,4,6
/proc/15507/task/15511/stat : 15511 (java) S 15505 15507 15356 0 -1 4202560 2924 0 0 0 1 0 0 0 16 0 22 0 529364726 16086147072 276653 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15512] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15512/stat : 15512 (java) S 15505 15507 15356 0 -1 4202560 1480 0 0 0 0 0 0 0 15 0 22 0 529364726 16086147072 276653 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15513] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15513/stat : 15513 (java) S 15505 15507 15356 0 -1 4202560 3470 0 0 0 0 0 0 0 16 0 22 0 529364726 16086147072 276653 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15514] ppid=15505 vsize=15709128 CPUtime=0.01 cores=0,2,4,6
/proc/15507/task/15514/stat : 15514 (java) S 15505 15507 15356 0 -1 4202560 3846 0 0 0 1 0 0 0 15 0 22 0 529364726 16086147072 276653 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15515] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15515/stat : 15515 (java) S 15505 15507 15356 0 -1 4202560 1610 0 0 0 0 0 0 0 15 0 22 0 529364726 16086147072 276653 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15516] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15516/stat : 15516 (java) S 15505 15507 15356 0 -1 4202560 3777 0 0 0 0 0 0 0 16 0 22 0 529364726 16086147072 276653 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 6 0 0 0
[pid=15507/tid=15517] ppid=15505 vsize=15709128 CPUtime=0.01 cores=0,2,4,6
/proc/15507/task/15517/stat : 15517 (java) S 15505 15507 15356 0 -1 4202560 2761 0 0 0 1 0 0 0 15 0 22 0 529364726 16086147072 276653 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15518] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15518/stat : 15518 (java) S 15505 15507 15356 0 -1 4202560 450 0 0 0 0 0 0 0 15 0 22 0 529364727 16086147072 276653 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820982272 0 0 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15519] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15519/stat : 15519 (java) S 15505 15507 15356 0 -1 4202560 17 0 0 0 0 0 0 0 18 0 22 0 529364728 16086147072 276653 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15520] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15520/stat : 15520 (java) S 15505 15507 15356 0 -1 4202560 9 0 0 0 0 0 0 0 18 0 22 0 529364728 16086147072 276653 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15521] ppid=15505 vsize=15709128 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15521/stat : 15521 (java) S 15505 15507 15356 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 529364730 16086147072 276653 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820989361 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15522] ppid=15505 vsize=15709128 CPUtime=1.04 cores=0,2,4,6
/proc/15507/task/15522/stat : 15522 (java) S 15505 15507 15356 0 -1 4202560 7243 0 0 0 103 1 0 0 16 0 22 0 529364730 16086147072 276653 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0

################
# More data... #
################

[pid=15507/tid=15565] ppid=15505 vsize=15696648 CPUtime=1107.74 cores=0,2,4,6
/proc/15507/task/15565/stat : 15565 (java) R 15505 15507 15356 0 -1 4202560 393618 0 0 0 110720 54 0 0 25 0 22 0 529364752 16073367552 108084 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505702712 0 4 0 16800975 0 0 0 -1 0 0 0 0
[pid=15507/tid=15566] ppid=15505 vsize=15696648 CPUtime=1108.68 cores=0,2,4,6
/proc/15507/task/15566/stat : 15566 (java) R 15505 15507 15356 0 -1 4202560 410221 0 0 0 110811 57 0 0 25 0 22 0 529364752 16073367552 108084 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505747085 0 4 0 16800975 0 0 0 -1 2 0 0 0
[pid=15507/tid=15567] ppid=15505 vsize=15696648 CPUtime=1107.21 cores=0,2,4,6
/proc/15507/task/15567/stat : 15567 (java) R 15505 15507 15356 0 -1 4202560 392525 0 0 0 110669 52 0 0 25 0 22 0 529364752 16073367552 108084 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505702808 0 4 0 16800975 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4464.41
Current children cumulated vsize (KiB) 15696648

[startup+1182.3 s]
/proc/loadavg: 8.09 8.06 8.04 9/220 15603
/proc/meminfo: memFree=30176108/32951124 swapFree=67111448/67111528
[pid=15507] ppid=15505 vsize=15696648 CPUtime=4703.03 cores=0,2,4,6
/proc/15507/stat : 15507 (java) S 15505 15507 15356 0 -1 4202496 2915445 0 1 0 469759 544 0 0 25 0 22 0 529364725 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 4 0 0 0
/proc/15507/statm: 3924162 104042 2298 9 0 3914881 0
[pid=15507/tid=15509] ppid=15505 vsize=15696648 CPUtime=0.39 cores=0,2,4,6
/proc/15507/task/15509/stat : 15509 (java) S 15505 15507 15356 0 -1 4202560 7037 0 1 0 38 1 0 0 20 0 22 0 529364726 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15510] ppid=15505 vsize=15696648 CPUtime=3.6 cores=0,2,4,6
/proc/15507/task/15510/stat : 15510 (java) S 15505 15507 15356 0 -1 4202560 153148 0 0 0 331 29 0 0 15 0 22 0 529364726 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15511] ppid=15505 vsize=15696648 CPUtime=3.57 cores=0,2,4,6
/proc/15507/task/15511/stat : 15511 (java) S 15505 15507 15356 0 -1 4202560 143619 0 0 0 332 25 0 0 15 0 22 0 529364726 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15512] ppid=15505 vsize=15696648 CPUtime=3.55 cores=0,2,4,6
/proc/15507/task/15512/stat : 15512 (java) S 15505 15507 15356 0 -1 4202560 147270 0 0 0 329 26 0 0 15 0 22 0 529364726 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 6 0 0 0
[pid=15507/tid=15513] ppid=15505 vsize=15696648 CPUtime=3.64 cores=0,2,4,6
/proc/15507/task/15513/stat : 15513 (java) S 15505 15507 15356 0 -1 4202560 151180 0 0 0 337 27 0 0 15 0 22 0 529364726 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15514] ppid=15505 vsize=15696648 CPUtime=3.7 cores=0,2,4,6
/proc/15507/task/15514/stat : 15514 (java) S 15505 15507 15356 0 -1 4202560 153339 0 0 0 344 26 0 0 16 0 22 0 529364726 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15515] ppid=15505 vsize=15696648 CPUtime=3.63 cores=0,2,4,6
/proc/15507/task/15515/stat : 15515 (java) S 15505 15507 15356 0 -1 4202560 149343 0 0 0 336 27 0 0 15 0 22 0 529364726 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 6 0 0 0
[pid=15507/tid=15516] ppid=15505 vsize=15696648 CPUtime=3.73 cores=0,2,4,6
/proc/15507/task/15516/stat : 15516 (java) S 15505 15507 15356 0 -1 4202560 157527 0 0 0 345 28 0 0 15 0 22 0 529364726 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 6 0 0 0
[pid=15507/tid=15517] ppid=15505 vsize=15696648 CPUtime=3.64 cores=0,2,4,6
/proc/15507/task/15517/stat : 15517 (java) S 15505 15507 15356 0 -1 4202560 148980 0 0 0 336 28 0 0 15 0 22 0 529364726 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15518] ppid=15505 vsize=15696648 CPUtime=2.18 cores=0,2,4,6
/proc/15507/task/15518/stat : 15518 (java) S 15505 15507 15356 0 -1 4202560 7608 0 0 0 132 86 0 0 15 0 22 0 529364727 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820982272 0 0 0 16800975 18446604445976933696 0 0 -1 6 0 0 0
[pid=15507/tid=15519] ppid=15505 vsize=15696648 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15519/stat : 15519 (java) S 15505 15507 15356 0 -1 4202560 17 0 0 0 0 0 0 0 18 0 22 0 529364728 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15520] ppid=15505 vsize=15696648 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15520/stat : 15520 (java) S 15505 15507 15356 0 -1 4202560 9 0 0 0 0 0 0 0 18 0 22 0 529364728 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15521] ppid=15505 vsize=15696648 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15521/stat : 15521 (java) S 15505 15507 15356 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 529364730 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820989361 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15522] ppid=15505 vsize=15696648 CPUtime=1.13 cores=0,2,4,6
/proc/15507/task/15522/stat : 15522 (java) S 15505 15507 15356 0 -1 4202560 7261 0 0 0 112 1 0 0 15 0 22 0 529364730 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15523] ppid=15505 vsize=15696648 CPUtime=1.32 cores=0,2,4,6
/proc/15507/task/15523/stat : 15523 (java) S 15505 15507 15356 0 -1 4202560 14962 0 0 0 130 2 0 0 16 0 22 0 529364730 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15524] ppid=15505 vsize=15696648 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15524/stat : 15524 (java) S 15505 15507 15356 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 529364730 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15525] ppid=15505 vsize=15696648 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15525/stat : 15525 (java) S 15505 15507 15356 0 -1 4202560 65 0 0 0 0 0 0 0 15 0 22 0 529364730 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820982272 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15564] ppid=15505 vsize=15696648 CPUtime=1167.16 cores=0,2,4,6
/proc/15507/task/15564/stat : 15564 (java) R 15505 15507 15356 0 -1 4202560 421823 0 0 0 116659 57 0 0 25 0 22 0 529364752 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505798074 0 4 0 16800975 0 0 0 -1 6 0 0 0
[pid=15507/tid=15565] ppid=15505 vsize=15696648 CPUtime=1167.07 cores=0,2,4,6
/proc/15507/task/15565/stat : 15565 (java) R 15505 15507 15356 0 -1 4202560 411343 0 0 0 116650 57 0 0 25 0 22 0 529364752 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505728363 0 4 0 16800975 0 0 0 -1 4 0 0 0
[pid=15507/tid=15566] ppid=15505 vsize=15696648 CPUtime=1168 cores=0,2,4,6
/proc/15507/task/15566/stat : 15566 (java) R 15505 15507 15356 0 -1 4202560 428700 0 0 0 116740 60 0 0 25 0 22 0 529364752 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505811571 0 4 0 16800975 0 0 0 -1 0 0 0 0
[pid=15507/tid=15567] ppid=15505 vsize=15696648 CPUtime=1166.55 cores=0,2,4,6
/proc/15507/task/15567/stat : 15567 (java) R 15505 15507 15356 0 -1 4202560 410856 0 0 0 116600 55 0 0 25 0 22 0 529364752 16073367552 104042 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505727647 0 4 0 16800975 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4703.03
Current children cumulated vsize (KiB) 15696648



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1206.7 s]
/proc/loadavg: 8.06 8.05 8.04 9/220 15603
/proc/meminfo: memFree=30132568/32951124 swapFree=67111448/67111528
[pid=15507] ppid=15505 vsize=15696648 CPUtime=4800.15 cores=0,2,4,6
/proc/15507/stat : 15507 (java) S 15505 15507 15356 0 -1 4202496 2961836 0 1 0 479461 554 0 0 25 0 22 0 529364725 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 4 0 0 0
/proc/15507/statm: 3924162 100151 2298 9 0 3914881 0
[pid=15507/tid=15509] ppid=15505 vsize=15696648 CPUtime=0.39 cores=0,2,4,6
/proc/15507/task/15509/stat : 15509 (java) S 15505 15507 15356 0 -1 4202560 7037 0 1 0 38 1 0 0 20 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15510] ppid=15505 vsize=15696648 CPUtime=3.69 cores=0,2,4,6
/proc/15507/task/15510/stat : 15510 (java) S 15505 15507 15356 0 -1 4202560 159122 0 0 0 339 30 0 0 15 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15511] ppid=15505 vsize=15696648 CPUtime=3.66 cores=0,2,4,6
/proc/15507/task/15511/stat : 15511 (java) S 15505 15507 15356 0 -1 4202560 149206 0 0 0 340 26 0 0 15 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15512] ppid=15505 vsize=15696648 CPUtime=3.6 cores=0,2,4,6
/proc/15507/task/15512/stat : 15512 (java) S 15505 15507 15356 0 -1 4202560 150708 0 0 0 333 27 0 0 15 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 6 0 0 0
[pid=15507/tid=15513] ppid=15505 vsize=15696648 CPUtime=3.71 cores=0,2,4,6
/proc/15507/task/15513/stat : 15513 (java) S 15505 15507 15356 0 -1 4202560 155270 0 0 0 344 27 0 0 16 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15514] ppid=15505 vsize=15696648 CPUtime=3.78 cores=0,2,4,6
/proc/15507/task/15514/stat : 15514 (java) S 15505 15507 15356 0 -1 4202560 159158 0 0 0 351 27 0 0 15 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15515] ppid=15505 vsize=15696648 CPUtime=3.7 cores=0,2,4,6
/proc/15507/task/15515/stat : 15515 (java) S 15505 15507 15356 0 -1 4202560 154015 0 0 0 342 28 0 0 16 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 6 0 0 0
[pid=15507/tid=15516] ppid=15505 vsize=15696648 CPUtime=3.79 cores=0,2,4,6
/proc/15507/task/15516/stat : 15516 (java) S 15505 15507 15356 0 -1 4202560 162568 0 0 0 351 28 0 0 15 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15517] ppid=15505 vsize=15696648 CPUtime=3.71 cores=0,2,4,6
/proc/15507/task/15517/stat : 15517 (java) S 15505 15507 15356 0 -1 4202560 153799 0 0 0 343 28 0 0 15 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15518] ppid=15505 vsize=15696648 CPUtime=2.21 cores=0,2,4,6
/proc/15507/task/15518/stat : 15518 (java) S 15505 15507 15356 0 -1 4202560 7717 0 0 0 133 88 0 0 15 0 22 0 529364727 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820982272 0 0 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15519] ppid=15505 vsize=15696648 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15519/stat : 15519 (java) S 15505 15507 15356 0 -1 4202560 17 0 0 0 0 0 0 0 18 0 22 0 529364728 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15520] ppid=15505 vsize=15696648 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15520/stat : 15520 (java) S 15505 15507 15356 0 -1 4202560 9 0 0 0 0 0 0 0 18 0 22 0 529364728 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15521] ppid=15505 vsize=15696648 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15521/stat : 15521 (java) S 15505 15507 15356 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 529364730 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820989361 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15522] ppid=15505 vsize=15696648 CPUtime=1.13 cores=0,2,4,6
/proc/15507/task/15522/stat : 15522 (java) S 15505 15507 15356 0 -1 4202560 7261 0 0 0 112 1 0 0 15 0 22 0 529364730 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15523] ppid=15505 vsize=15696648 CPUtime=1.32 cores=0,2,4,6
/proc/15507/task/15523/stat : 15523 (java) S 15505 15507 15356 0 -1 4202560 14962 0 0 0 130 2 0 0 16 0 22 0 529364730 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15524] ppid=15505 vsize=15696648 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15524/stat : 15524 (java) S 15505 15507 15356 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 529364730 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15525] ppid=15505 vsize=15696648 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15525/stat : 15525 (java) S 15505 15507 15356 0 -1 4202560 67 0 0 0 0 0 0 0 15 0 22 0 529364730 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820982272 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15564] ppid=15505 vsize=15696648 CPUtime=1191.23 cores=0,2,4,6
/proc/15507/task/15564/stat : 15564 (java) R 15505 15507 15356 0 -1 4202560 423327 0 0 0 119065 58 0 0 25 0 22 0 529364752 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505963930 0 4 0 16800975 0 0 0 -1 0 0 0 0
[pid=15507/tid=15565] ppid=15505 vsize=15696648 CPUtime=1191.16 cores=0,2,4,6
/proc/15507/task/15565/stat : 15565 (java) R 15505 15507 15356 0 -1 4202560 413218 0 0 0 119059 57 0 0 25 0 22 0 529364752 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505798169 0 4 0 16800975 0 0 0 -1 6 0 0 0
[pid=15507/tid=15566] ppid=15505 vsize=15696648 CPUtime=1192.15 cores=0,2,4,6
/proc/15507/task/15566/stat : 15566 (java) R 15505 15507 15356 0 -1 4202560 430232 0 0 0 119155 60 0 0 25 0 22 0 529364752 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505702712 0 4 0 16800975 0 0 0 -1 4 0 0 0
[pid=15507/tid=15567] ppid=15505 vsize=15696648 CPUtime=1190.7 cores=0,2,4,6
/proc/15507/task/15567/stat : 15567 (java) R 15505 15507 15356 0 -1 4202560 412785 0 0 0 119015 55 0 0 25 0 22 0 529364752 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505702808 0 4 0 16800975 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.15
Current children cumulated vsize (KiB) 15696648

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1206.7 s]
/proc/loadavg: 8.06 8.05 8.04 9/220 15603
/proc/meminfo: memFree=30132568/32951124 swapFree=67111448/67111528
[pid=15507] ppid=15505 vsize=15696648 CPUtime=4800.15 cores=0,2,4,6
/proc/15507/stat : 15507 (java) S 15505 15507 15356 0 -1 4202496 2961836 0 1 0 479461 554 0 0 25 0 22 0 529364725 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 4 0 0 0
/proc/15507/statm: 3924162 100151 2298 9 0 3914881 0
[pid=15507/tid=15509] ppid=15505 vsize=15696648 CPUtime=0.39 cores=0,2,4,6
/proc/15507/task/15509/stat : 15509 (java) S 15505 15507 15356 0 -1 4202560 7037 0 1 0 38 1 0 0 20 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15510] ppid=15505 vsize=15696648 CPUtime=3.69 cores=0,2,4,6
/proc/15507/task/15510/stat : 15510 (java) S 15505 15507 15356 0 -1 4202560 159122 0 0 0 339 30 0 0 15 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15511] ppid=15505 vsize=15696648 CPUtime=3.66 cores=0,2,4,6
/proc/15507/task/15511/stat : 15511 (java) S 15505 15507 15356 0 -1 4202560 149206 0 0 0 340 26 0 0 15 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15512] ppid=15505 vsize=15696648 CPUtime=3.6 cores=0,2,4,6
/proc/15507/task/15512/stat : 15512 (java) S 15505 15507 15356 0 -1 4202560 150708 0 0 0 333 27 0 0 15 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 6 0 0 0
[pid=15507/tid=15513] ppid=15505 vsize=15696648 CPUtime=3.71 cores=0,2,4,6
/proc/15507/task/15513/stat : 15513 (java) S 15505 15507 15356 0 -1 4202560 155270 0 0 0 344 27 0 0 16 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15514] ppid=15505 vsize=15696648 CPUtime=3.78 cores=0,2,4,6
/proc/15507/task/15514/stat : 15514 (java) S 15505 15507 15356 0 -1 4202560 159158 0 0 0 351 27 0 0 15 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15515] ppid=15505 vsize=15696648 CPUtime=3.7 cores=0,2,4,6
/proc/15507/task/15515/stat : 15515 (java) S 15505 15507 15356 0 -1 4202560 154015 0 0 0 342 28 0 0 16 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 6 0 0 0
[pid=15507/tid=15516] ppid=15505 vsize=15696648 CPUtime=3.79 cores=0,2,4,6
/proc/15507/task/15516/stat : 15516 (java) S 15505 15507 15356 0 -1 4202560 162568 0 0 0 351 28 0 0 15 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15517] ppid=15505 vsize=15696648 CPUtime=3.71 cores=0,2,4,6
/proc/15507/task/15517/stat : 15517 (java) S 15505 15507 15356 0 -1 4202560 153799 0 0 0 343 28 0 0 15 0 22 0 529364726 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 0 0 0 0
[pid=15507/tid=15518] ppid=15505 vsize=15696648 CPUtime=2.21 cores=0,2,4,6
/proc/15507/task/15518/stat : 15518 (java) S 15505 15507 15356 0 -1 4202560 7717 0 0 0 133 88 0 0 15 0 22 0 529364727 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820982272 0 0 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15519] ppid=15505 vsize=15696648 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15519/stat : 15519 (java) S 15505 15507 15356 0 -1 4202560 17 0 0 0 0 0 0 0 18 0 22 0 529364728 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15520] ppid=15505 vsize=15696648 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15520/stat : 15520 (java) S 15505 15507 15356 0 -1 4202560 9 0 0 0 0 0 0 0 18 0 22 0 529364728 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15521] ppid=15505 vsize=15696648 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15521/stat : 15521 (java) S 15505 15507 15356 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 529364730 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820989361 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15522] ppid=15505 vsize=15696648 CPUtime=1.13 cores=0,2,4,6
/proc/15507/task/15522/stat : 15522 (java) S 15505 15507 15356 0 -1 4202560 7261 0 0 0 112 1 0 0 15 0 22 0 529364730 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15523] ppid=15505 vsize=15696648 CPUtime=1.32 cores=0,2,4,6
/proc/15507/task/15523/stat : 15523 (java) S 15505 15507 15356 0 -1 4202560 14962 0 0 0 130 2 0 0 16 0 22 0 529364730 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15524] ppid=15505 vsize=15696648 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15524/stat : 15524 (java) S 15505 15507 15356 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 22 0 529364730 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820981657 0 4 0 16800975 18446604445976933696 0 0 -1 4 0 0 0
[pid=15507/tid=15525] ppid=15505 vsize=15696648 CPUtime=0 cores=0,2,4,6
/proc/15507/task/15525/stat : 15525 (java) S 15505 15507 15356 0 -1 4202560 67 0 0 0 0 0 0 0 15 0 22 0 529364730 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 230820982272 0 4 0 16800975 18446604445976933696 0 0 -1 2 0 0 0
[pid=15507/tid=15564] ppid=15505 vsize=15696648 CPUtime=1191.23 cores=0,2,4,6
/proc/15507/task/15564/stat : 15564 (java) R 15505 15507 15356 0 -1 4202560 423327 0 0 0 119065 58 0 0 25 0 22 0 529364752 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505963930 0 4 0 16800975 0 0 0 -1 0 0 0 0
[pid=15507/tid=15565] ppid=15505 vsize=15696648 CPUtime=1191.16 cores=0,2,4,6
/proc/15507/task/15565/stat : 15565 (java) R 15505 15507 15356 0 -1 4202560 413218 0 0 0 119059 57 0 0 25 0 22 0 529364752 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505798169 0 4 0 16800975 0 0 0 -1 6 0 0 0
[pid=15507/tid=15566] ppid=15505 vsize=15696648 CPUtime=1192.15 cores=0,2,4,6
/proc/15507/task/15566/stat : 15566 (java) R 15505 15507 15356 0 -1 4202560 430232 0 0 0 119155 60 0 0 25 0 22 0 529364752 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505702712 0 4 0 16800975 0 0 0 -1 4 0 0 0
[pid=15507/tid=15567] ppid=15505 vsize=15696648 CPUtime=1190.7 cores=0,2,4,6
/proc/15507/task/15567/stat : 15567 (java) R 15505 15507 15356 0 -1 4202560 412785 0 0 0 119015 55 0 0 25 0 22 0 529364752 16073367552 100151 33554432000 1073741824 1073778376 140735541683984 18446744073709551615 46912505702808 0 4 0 16800975 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.15
Current children cumulated vsize (KiB) 15696648

Child status: 143
Real time (s): 1206.73
CPU time (s): 4800.21
CPU user time (s): 4794.64
CPU system time (s): 5.56315
CPU usage (%): 397.785
Max. virtual memory (cumulated for all children) (KiB): 15711844

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4794.64
system time used= 5.56315
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2961862
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 207966
involuntary context switches= 118018

runsolver used 2.02669 second user time and 4.6213 second system time

The end

Launcher Data 

Begin job on node104 at 2011-04-30 17:34:03
IDJOB=3311625
IDBENCH=83010
IDSOLVER=1698
FILE ID=node104/3311625-1304177643
RUNJOBID= node104-1304176434-15374
PBS_JOBID= 13172420
Free space on /tmp= 72540 MiB

SOLVER NAME= STRUCTure 0.1 (hbr=2)
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v700-c2982-S1617172756-030.SATISFIABLE.cnf
COMMAND LINE= run 15000 NBCORE TMPDIR DIR BENCHNAME -hbr 2
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3311625-1304177643/watcher-3311625-1304177643 -o /tmp/evaluation-result-3311625-1304177643/solver-3311625-1304177643 -C 4800 -W 1300 -M 15500  run 15000 4 HOME HOME HOME/instance-3311625-1304177643.cnf -hbr 2

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 47049d4705301f4ac565d8b47d11d0a5
RANDOM SEED=821491441

node104.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5293.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30920820 kB
Buffers:        418804 kB
Cached:        1137844 kB
SwapCached:         80 kB
Active:         170764 kB
Inactive:      1451012 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30920820 kB
SwapTotal:    67111528 kB
SwapFree:     67111448 kB
Dirty:            1628 kB
Writeback:           0 kB
AnonPages:       64252 kB
Mapped:          14684 kB
Slab:           345068 kB
PageTables:       4256 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   194332 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72540 MiB
End job on node104 at 2011-04-30 17:54:12