friedkiwi/ibm-information-center
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14ed2849c6
ibm-information-center/dist/eclipse/plugins/i5OS.ic.rzaig_5.4.0.1/rzaiginvestigateclusterinfo.htm

415 lines
21 KiB
HTML
Raw Blame History

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html
PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html lang="en-us" xml:lang="en-us">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="security" content="public" />
<meta name="Robots" content="index,follow" />
<meta http-equiv="PICS-Label" content='(PICS-1.1 "http://www.icra.org/ratingsv02.html" l gen true r (cz 1 lz 1 nz 1 oz 1 vz 1) "http://www.rsac.org/ratingsv01.html" l gen true r (n 0 s 0 v 0 l 0) "http://www.classify.org/safesurf/" l gen true r (SS~~000 1))' />
<meta name="DC.Type" content="concept" />
<meta name="DC.Title" content="Investigate a problem with CLUSTERINFO macro" />
<meta name="abstract" content="The CLUSTERINFO macro displays the information that cluster resource services contains about nodes, CRGs and active cluster IP addresses." />
<meta name="description" content="The CLUSTERINFO macro displays the information that cluster resource services contains about nodes, CRGs and active cluster IP addresses." />
<meta name="DC.Relation" scheme="URI" content="rzaigtroubleshoot.htm" />
<meta name="copyright" content="(C) Copyright IBM Corporation 1998, 2006" />
<meta name="DC.Rights.Owner" content="(C) Copyright IBM Corporation 1998, 2006" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="rzaiginvestigateclusterinfo" />
<meta name="DC.Language" content="en-us" />
<!-- All rights reserved. Licensed Materials Property of IBM -->
<!-- US Government Users Restricted Rights -->
<!-- Use, duplication or disclosure restricted by -->
<!-- GSA ADP Schedule Contract with IBM Corp. -->
<link rel="stylesheet" type="text/css" href="./ibmdita.css" />
<link rel="stylesheet" type="text/css" href="./ic.css" />
<title>Investigate a problem with CLUSTERINFO macro</title>
</head>
<body id="rzaiginvestigateclusterinfo"><a name="rzaiginvestigateclusterinfo"><!-- --></a>
<!-- Java sync-link --><script language="Javascript" src="../rzahg/synch.js" type="text/javascript"></script>
<h1 class="topictitle1">Investigate a problem with CLUSTERINFO macro</h1>
<div><p>The CLUSTERINFO macro displays the information that cluster resource
services contains about nodes, CRGs and active cluster IP addresses.</p>
<p>The CLUSTERINFO macro creates a snapshot of information about the current
cluster. The command navigates through clustering objects and creates a description
of the cluster on the local node. The CLUSTERINFO macro provides a flight
recorder for different cluster objects and can aid in determining the source
of a problem within the cluster. To access the CLUSTERINFO macro, complete
the following steps:</p>
<div class="p"><ol><li>On a character-based interface, enter <span class="cmdname">STRSST</span>.</li>
<li>Sign on with your Service Tools user profile.</li>
<li>On the Start Service Tool, select Option 1 (Start a service tool).</li>
<li>Select Option 4 (Display/Alter/Dump).</li>
<li>Select Option 2 (Dump to Printer).</li>
<li>Select Option 2 (Licensed Internal Code (LIC) data).</li>
<li>Select Option 14 (Advanced Analysis).</li>
<li>Enter a 1 in front of CLUSTERINFO macro option. Press Enter.</li>
</ol>
Once the CLUSTERINFO macro displays, use the -H option to display help
for all the options that are available with this macro. The following usage
diagram describes each of the options that are available for the CLUSTERINFO
macro:</div>
<div class="p">
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" frame="border" border="1" rules="all"><caption>Table 1. Options for CLUSTERINFO macro</caption><thead align="left"><tr valign="bottom"><th valign="bottom" id="d0e46">Option</th>
<th valign="bottom" id="d0e48">Description</th>
</tr>
</thead>
<tbody><tr><td valign="top" headers="d0e46 ">-H</td>
<td valign="top" headers="d0e48 ">Displays help screen for options</td>
</tr>
<tr><td valign="top" headers="d0e46 ">-A </td>
<td valign="top" headers="d0e48 ">Displays all information</td>
</tr>
<tr><td valign="top" headers="d0e46 ">-FR </td>
<td valign="top" headers="d0e48 ">Displays flight recorder entries</td>
</tr>
<tr><td valign="top" headers="d0e46 ">-HB</td>
<td valign="top" headers="d0e48 ">Displays heartbeat information</td>
</tr>
<tr><td valign="top" headers="d0e46 ">-PERF</td>
<td valign="top" headers="d0e48 ">Displays performance counters </td>
</tr>
<tr><td valign="top" headers="d0e46 "> -Q </td>
<td valign="top" headers="d0e48 ">Displays outgoing message queue status</td>
</tr>
<tr><td valign="top" headers="d0e46 ">-G </td>
<td valign="top" headers="d0e48 ">Unsuppresses display of all CC broadcast groups</td>
</tr>
<tr><td valign="top" headers="d0e46 ">-T </td>
<td valign="top" headers="d0e48 ">Displays tuning parameters</td>
</tr>
<tr><td valign="top" headers="d0e46 ">-M</td>
<td valign="top" headers="d0e48 ">Displays matrices for all DAs</td>
</tr>
<tr><td valign="top" headers="d0e46 "> -DP</td>
<td valign="top" headers="d0e48 "> Display dataPort information</td>
</tr>
</tbody>
</table>
</div>
</div>
<p> </p>
<div class="section"><h4 class="sectiontitle">Interpreting CLUSTERINFO macro results</h4><div class="p">In this example,
the -A option is specified so all fields are dumped. The flight recorders
are the main tools for debug. Note that these flight recorders are deleted
when the cluster node is ended or deleted. For problem analysis the CLUSTERINFO
macro must be run before the cluster is ended or deleted. In some cases the
flight recorders may be written to a vlog when the cluster is deleted or ended.
The flight recorders record various events that affect the structure and
performance of the cluster. It is beyond the scope of this information to
provide detailed interpretation of flight recorder data.<div class="note"><span class="notetitle">Note:</span> <img src="./delta.gif" alt="Start of change" />Vertical
ellipses indicate that a portion of the results was removed and is not displayed
in the output.<img src="./deltaend.gif" alt="End of change" /></div>
</div>
</div>
<div class="section"><h4 class="sectiontitle">Section 1 of CLUSTERINFO macro results</h4><pre>DISPLAY/ALTER/DUMP CLUSTERINFO -NEW2 08/23/05 13:36:37 PAGE 1
Running macro: CLUSTERINFO -A
Use -H for command information
Cluster Name : MYCLUSTER
Local Node Name: SYSTEM1
CC/CTS Version : 5
Macro Timestamp: 08/23/05 13:36:37.079</pre>
<p>Section 1 contains generic information about the cluster, such
as the cluster name, cluster version, and the timestamp when the report was
generated. In this example, the cluster name is MYCLUSTER and the local node
name is SYSTEM1.</p>
</div>
<div class="section"><h4 class="sectiontitle">Section 2 of CLUSTERINFO macro results</h4><pre>Cluster Object Addresses
CstcClusterServices Address: DBF08681C9161580
Cluster Address : FC5B04B0D4001000
Cluster Task Address : B00010000E932000
Cluster Task Q Address : DBF08681C9169A00
Clue Group Services Address: CDAB6D0339001000
CC Services Address : FC5B04B0D4008000</pre>
<p>Section 2 provides pointers to the locations of key cluster
objects. </p>
</div>
<div class="section"><h4 class="sectiontitle">Section 3 of CLUSTERINFO macro results</h4><pre>Message Statistics
Number of non-fragmented messages: 250
Number of fragmented messages : 1
Number of fragments : 7
Number of acks : 148</pre>
<p>Section 3 contains messaging statistics for the cluster, such
as the number of fragments and the number of acknowledgements (acks).</p>
</div>
<div class="section"><h4 class="sectiontitle">Section 4 of CLUSTERINFO macro results</h4><pre>Node Map
Node ID : SYSTEM1
GenesisSubnetId : 9.5.251.0
CCNode * : FC5B04B0D4007000
CCSrvNode *: FC5B04B0D404F000
Adapter 1 : 9.5.251.46 Primary
Status : 0x01 Reachable
Line Type : 0x09 Ethernet
Node ID : SYSTEM2
GenesisSubnetId : 9.5.251.0
CCNode * : FC5B04B0D4060000
CCSrvNode *: FC5B04B0D4061000
Adapter 1 : 9.5.251.47 Primary
Status : 0x01 Reachable
Line Type : 0x09 Ethernet</pre>
<p>Section 4 lists all the current active cluster nodes in a node
map. In this example there are two active nodes, SYSTEM1 and SYSTEM2. T</p>
</div>
<div class="section"><h4 class="sectiontitle">Section 5 of CLUSTERINFO macro results</h4><pre>Subnet Map
Subnet ID: 127.0.0.0
CCSubnet * : FC5B04B0D4006000
CCSrvSubnet* : FC5B04B0D400C000
Retries : 0
Msg Timeouts : 0
Bad Msg Counter : 0
Failed Default Address: 0
Subnet ID: 226.5.5.5
CCSubnet * : FC5B04B0D405B100
CCSrvSubnet* : FC5B04B0D405C000
Retries : 0
Msg Timeouts : 0
Bad Msg Counter : 0
Failed Default Address: 0</pre>
<p>Section 5 contains a list of all subnet objects that are in
the cluster.</p>
</div>
<div class="section"><h4 class="sectiontitle">Section 6 of CLUSTERINFO macro results</h4><pre>Group Map
Group ID: 0x0000000000000001
Name : CTS
CCGroup * : FC5B04B0D405FF00
CCSrvGroup *: FC5B04B0D4064B00
Member Nodes
SYSTEM1
SYSTEM2
Group ID: 0x0000000000000002
Name : CTS
CCGroup * : FC5B04B0D4055100
CCSrvGroup *: FC5B04B0D4055200
Member Nodes
SYSTEM1
SYSTEM2
.
.
.</pre>
<p>Section 6 list of all the current cluster groups. Each group
is called a distributed activity group. These groups are used for communication
between the groups on each active node in the cluster. Majority of the groups
deal are for licensed internal code (LIC). These are identifiable by a group
name of CTS and BADA. You will also see a group for CCTL (QCSTCTL job in
the operating system), CRGM (QCSTCRGM job in the operating system) and each
cluster resource group (CRG) job. Groups for CRG jobs will not have a group
name. Each group has member nodes. Member nodes are the nodes which are
communicating together for this group. </p>
</div>
<div class="section"><h4 class="sectiontitle">Section 7 of CLUSTERINFO macro results</h4><pre>Partition Map
Partition Map is empty</pre>
<div class="p">Section 7 contains a list of all the nodes in the SLIC partition
list. <div class="note"><span class="notetitle">Note:</span> This is not the same concept as XPF partitioned nodes.</div>
</div>
</div>
<div class="section"><h4 class="sectiontitle">Section 8 of CLUSTERINFO macro results</h4><pre>CTS Client List
CTS Client List is empty</pre>
<p>Section 8 contains a list of all the registered cluster clients,
such as data ports.</p>
</div>
<div class="section"><h4 class="sectiontitle">Section 9 of CLUSTERINFO macro results</h4><pre>Flight Recorder : CSTCSVFR
Flight Recorder Address: DBF08681C9161620</pre>
<p>Section 9 contains the cluster services flight recorder (CSTCSVFR)
which remains on the system until an IPL. </p>
</div>
<div class="section"><h4 class="sectiontitle">Section 10 of CLUSTERINFO macro results</h4><pre>Message Statistics
Number of non-fragmented messages: 250
Number of fragmented messages : 1
Number of fragments : 7
Number of acks : 148
Time Stamp: 08/18/05 14:00:15.329
Trace Point: 0x0010 CstcClusterServicesTracePtCreatedFlightRecorder
C3D9C5C1E3C5C6D9 &lt;CREATEFR&gt;
Time Stamp: 08/22/05 16:43:28.912
Trace Point: 0x0020 CstcClusterServicesTracePtCreatedClusterObject
D4D6D9C5E8404040 4040C5F8D3770500 &lt;MYCLUSTER E8L...&gt;
1000 &lt;.. &gt;
Time Stamp: 08/23/05 13:33:40.935
Trace Point: 0x0030 CstcClusterServicesTracePtDeletedClusterObject
D4D6D9C5E8404040 404040E2E3 &lt;MYCLUSTER ST &gt;
Time Stamp: 08/23/05 13:33:41.204
Trace Point: 0x0030 CstcClusterServicesTracePtDeletedClusterObject
C3D4D7E3 &lt;CMPT &gt;
Time Stamp: 08/23/05 13:33:55.122
Trace Point: 0x0020 CstcClusterServicesTracePtCreatedClusterObject
D4D6D9C5E8404040 4040FC5B04B0D400 &lt;MYCLUSTER ....M.&gt;
1000 &lt;.. &gt;
</pre>
<p>Section 10 contains the flight recorder for
CSTCCCFR. This cluster flight recorder remains on the system until clustering
is ended on this node.</p>
</div>
<div class="section"><h4 class="sectiontitle">Section 11 of CLUSTERINFO macro results</h4><pre>Flight Recorder : CSTCCLFR
Flight Recorder Address: FC5B04B0D4001E80
-------------------------------------------------------------------------------
Time Stamp: 08/23/05 13:33:54.944
Trace Point: 0x1010 CstcClusterTracePtCreatedSubnetObject
7F000000FC5B04B0 D4006000 &lt;........M.-. &gt;
Time Stamp: 08/23/05 13:33:55.062
Trace Point: 0x1000 CstcClusterTracePtCreatedNodeObject
C3E2E3D9D9C3C8C3 E2E3F5F2FC5B04B0 &lt;CSTRSYSTEM1....&gt;
D4007000 &lt;M... &gt;
Time Stamp: 08/23/05 13:33:55.122
Trace Point: 0x1020 CstcClusterTracePtCreatedMCGroupObject
0000000000000001 00000000D9C3C8C3 &lt;............RCHC&gt;
E2E3F5F2 &lt;ST52 &gt;
.
.
.</pre>
<p>Section 11 contains cluster communication flight recorder
(CSTECLFR). This cluster flight recorder remains on the system until clustering
is ended on this node.</p>
</div>
<div class="section"><h4 class="sectiontitle">Section 12 of CLUSTERINFO macro results</h4><pre>Flight Recorder : CSTCCCFR
Flight Recorder Address: FC5B04B0D4006380
-------------------------------------------------------------------------------
Time Stamp: 08/23/05 13:33:55.080
Trace Point: 0x3000 CstcCCScamTracePtScamOpen
FC5B04B0D400E480 0000000000000000 &lt;....M.U.........&gt;
Time Stamp: 08/23/05 13:33:55.097
Trace Point: 0x3010 CstcCCScamTracePtScamBind
FC5B04B0D400E480 0000000000000000 &lt;....M.U.........&gt;
Time Stamp: 08/23/05 13:33:55.100
Trace Point: 0x3000 CstcCCScamTracePtScamOpen
FC5B04B0D400E480 0000000000000000 &lt;....M.U.........&gt;
D6E4E3 &lt;OUT &gt;
Time Stamp: 08/23/05 13:33:55.100
Trace Point: 0x3010 CstcCCScamTracePtScamBind
FC5B04B0D400E480 0000000000000000 &lt;....M.U.........&gt;
.
.
.</pre>
<p>Section 12 contains the clue flight recorder (CSTCCCFR), which
remains on the system until clustering is ended on this node.</p>
</div>
<div class="section"><h4 class="sectiontitle">Section 13 of CLUSTERINFO macro results</h4><pre>Time Stamp: 08/23/05 13:33:55.201
C3A2A385C7E27A7A C3A2A385C7E24082 &lt;CsteGS::CsteGS b&gt;
85878995A2 &lt;egins &gt;
Time Stamp: 08/23/05 13:33:55.201
C3A2A385C4C14083 9695A2A399A483A3 &lt;CsteDA construct&gt;
85847A40C2C1C4C1 404040404040 &lt;ed: BADA &gt;
Time Stamp: 08/23/05 13:33:55.201
C3A2A385C7E27A7A C3A2A385C7E24081 &lt;CsteGS::CsteGS a&gt;
8484408281848140 A39640C4C16D9389 &lt;dd bada to DA_li&gt;
A2A3 &lt;st &gt;
.
.
.</pre>
<p>Section 13 displays the contents of the send queues and active
message queues. If this section is not empty for an extended period of time
it indicates a problem in the cluster. </p>
</div>
<div class="section"><h4 class="sectiontitle">Section 14 of CLUSTERINFO macro results</h4><pre>Flight Recorder : CSTECLF2
Flight Recorder Address: CDAB6D0339001300
-------------------------------------------------------------------------------
Time Stamp: 08/23/05 13:33:55.201
C3A2A385C4C17A7A C3A2A385C4C16B40 &lt;CsteDA::CsteDA, &gt;
83998581A385D4C3 C79996A49740C9C4 &lt;createMCGroup ID&gt;
407E40F0A7F1F5 &lt; = 0x15 &gt;
Time Stamp: 08/23/05 13:33:55.209
C3A2A385E2C3D985 977A7A84859389A5 &lt;CsteSCRep::deliv&gt;
85994094A287E3A8 97857EF0A7F140A2 &lt;er msgType=0x1 s&gt;
A482E3A897857EF0 A7F240C4C17EC2C1 &lt;ubType=0x2 DA=BA&gt;
C4C1404040404040 &lt;DA &gt;
Time Stamp: 08/23/05 13:33:55.209
C3A2A385C4C17A7A A58985A66B409985 &lt;CsteDA::view, re&gt;
9496A585D4C3C799 96A497D485948285 &lt;moveMCGroupMembe&gt;
99A240C9C4407E40 F0A7F1F540969384 &lt;rs ID = 0x15 old&gt;
6D959684856D9389 A2A340A289A98540 &lt;_node_list size &gt;
7E40F0A7F0 &lt;= 0x0 &gt;
.
.
.
</pre>
<p>Section 14 contains flight recorder information.</p>
</div>
<div class="section"><h4 class="sectiontitle">Section 15 of CLUSTERINFO macro results</h4><pre>Message Queues
Send Queues:
Send Queue: 00 Messages: 00 MessageQueue*: FC5B04B0D400BF80
Send Queue: 01 Messages: 00 MessageQueue*: FC5B04B0D400DF80
Send Queue: 02 Messages: 00 MessageQueue*: FC5B04B0D400E600
Send Queue: 03 Messages: 00 MessageQueue*: FC5B04B0D400E680
Send Queue: 04 Messages: 00 MessageQueue*: FC5B04B0D400E700
Send Queue: 05 Messages: 00 MessageQueue*: FC5B04B0D400E780
Send Queue: 06 Messages: 00 MessageQueue*: FC5B04B0D400E800
Send Queue: 07 Messages: 00 MessageQueue*: FC5B04B0D400E880
Send Queue: 08 Messages: 00 MessageQueue*: FC5B04B0D400E900
Send Queue: 09 Messages: 00 MessageQueue*: FC5B04B0D400E980
Send Queue: 10 Messages: 00 MessageQueue*: FC5B04B0D400EA00
Send Queue: 11 Messages: 00 MessageQueue*: FC5B04B0D400EA80
Send Queue: 12 Messages: 00 MessageQueue*: FC5B04B0D400EB00
Send Queue: 13 Messages: 00 MessageQueue*: FC5B04B0D400EB80
Send Queue: 14 Messages: 00 MessageQueue*: FC5B04B0D400EC00
Send Queue: 15 Messages: 00 MessageQueue*: FC5B04B0D400EC80
Send Queue: 16 Messages: 00 MessageQueue*: FC5B04B0D400ED00
Send Queue: 17 Messages: 00 MessageQueue*: FC5B04B0D400ED80
Send Queue: 18 Messages: 00 MessageQueue*: FC5B04B0D400EE00
Send Queue: 19 Messages: 00 MessageQueue*: FC5B04B0D400EE80
Active Message Queues:
Active Message Queue: 00 Messages: 00 MessageQueue*: FC5B04B0D4008570
Active Message Queue: 01 Messages: 00 MessageQueue*: FC5B04B0D4008640
Active Message Queue: 02 Messages: 00 MessageQueue*: FC5B04B0D4008710
Active Message Queue: 03 Messages: 00 MessageQueue*: FC5B04B0D40087E0
Active Message Queue: 04 Messages: 00 MessageQueue*: FC5B04B0D40088B0
Active Message Queue: 05 Messages: 00 MessageQueue*: FC5B04B0D4008980
Active Message Queue: 06 Messages: 00 MessageQueue*: FC5B04B0D4008A50
Active Message Queue: 07 Messages: 00 MessageQueue*: FC5B04B0D4008B20
Active Message Queue: 08 Messages: 00 MessageQueue*: FC5B04B0D4008BF0
Active Message Queue: 09 Messages: 00 MessageQueue*: FC5B04B0D4008CC0
Active Message Queue: 10 Messages: 00 MessageQueue*: FC5B04B0D4008D90
Active Message Queue: 11 Messages: 00 MessageQueue*: FC5B04B0D4008E60
Active Message Queue: 12 Messages: 00 MessageQueue*: FC5B04B0D4008F30
Active Message Queue: 13 Messages: 00 MessageQueue*: FC5B04B0D4009000
Active Message Queue: 14 Messages: 00 MessageQueue*: FC5B04B0D40090D0
Active Message Queue: 15 Messages: 00 MessageQueue*: FC5B04B0D40091A0
Active Message Queue: 16 Messages: 00 MessageQueue*: FC5B04B0D4009270
Active Message Queue: 17 Messages: 00 MessageQueue*: FC5B04B0D4009340
Active Message Queue: 18 Messages: 00 MessageQueue*: FC5B04B0D4009410
Active Message Queue: 19 Messages: 00 MessageQueue*: FC5B04B0D40094E0
-------------------------------------------------------------------------------
Tuning Values
cstcRcvSendTimerRatio : 2 Default: 2
cstcMcastRelayTimerRatio : 8 Default: 8
cstcMcastRelayHBTimerRatio : 4 Default: 4
cstcHeartbeatBaseTimer : 12288000000 Default: 12288000000
cstcHeartbeatBasePrecision : 4096000000 Default: 4096000000
cstcRetryPrecision : 4096000000 Default: 4096000000
cstcRetryTimerVal : 4096000000 Default: 4096000000
cstcCDATBaseTimer : 491520000000 Default: 491520000000
cstcCDATBasePrecision : 40960000000 Default: 40960000000
cstcRecoveryBaseTimer : 3686400000000 Default: 3686400000000
cstcRecoveryBasePrecision : 491520000000 Default: 491520000000
cstcMaxRetryTime : 32768000000 Default: 32768000000
cstcCCFragmentSize : 1464 Default: 1464
cstcCCSendQOverflow : 1024 Default: 1024
cstcBadMsgCtrThreshold : 3 Default: 3
cstcUnreachableHBAckThreshold: 1 Default: 1
cstcReachableHBAckThreshold : 3 Default: 3
cstcUnreachableHBThreshold : 4 Default: 4
cstcReachableHBThreshold : 4 Default: 4
cstcMaxHBThreshold : 16 Default: 16
cstcDisableMsgTimer : 0 Default: 0
cstcRepeatAckThreshold : 10 Default: 10
DISPLAY/ALTER/DUMP CLUSTERINFO -NEW2 08/23/05 13:36:37 PAGE 87
cstcDelayedAckTimer : 409600000 Default: 409600000
cstcDelayedAckPrecision : 40960000 Default: 40960000
cstcCCSendWindow : 2 Default: 2
cstcCCEnableMcast : 1 Default: 1
cstcCCPerfClass : 2 Default: 2
-------------------------------------------------------------------------------
****** END OF DUMP ******</pre>
<p>Section 15 contains tuning values. Tuning values are equivilent
to the cluster performance and configuration information that is described
in the <a href="../apis/clcntrtcrs.htm">Retrieve Cluster
Resource Services Information (QcstRetrieveCRSInfo) API</a>. Section 14
contains the current value and the default for these fields. </p>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="rzaigtroubleshoot.htm" title="Find error recovery solutions for problems that are specific to clusters.">Troubleshoot clusters</a></div>
</div>
</div>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink