Powerscale Hardware Installation: Participant Guide
Powerscale Hardware Installation: Participant Guide
HARDWARE
INSTALLATION
PARTICIPANT GUIDE
PARTICIPANT GUIDE
Table of Contents
Appendix ................................................................................................. 89
The graphic shows the PowerScale Solutions Expert certification track. You can
leverage the Dell Technologies Proven Professional program to realize your full
potential. A combination of technology-focused and role-based training and exams
to cover concepts and principles as well as the full range of Dell Technologies'
hardware, software, and solutions. You can accelerate your career and your
organization’s capabilities.
PowerScale Solutions
(C) - Classroom
Prerequisite Skills
To understand the content and successfully complete this course, a student must
have a suitable knowledge base or skill set. The student must have an
understanding of:
• Current PowerScale hardware portfolio and the OneFS operating system
• PowerScale Concepts
• Isilon InfiniBand to Ethernet Backend Conversion
Course Objectives
Module Objectives
Installation References
The Isilon Info Hubs page has links to specific OneFS software, hardware, and
troubleshooting pages. The OneFS in hubs have documents for installation-related
areas.
When working with PowerScale equipment, it is critical to ensure you adhere to the
following precautions.
6
1
2
5
3
1: The AC supply circuit for PowerScale nodes must supply the total current that is
specified on the label of the node. All AC power supply connections must be
properly grounded. Connections that are not directly connected to the branch
circuit, such as nodes that are connected to a power strip, must also be properly
grounded. Do not overload the branch circuit of the AC supply that provides power
to the rack holding PowerScale nodes. The total rack load should not exceed 80%
of the branch circuit rating. For high availability, the left and right sides of any rack
must receive power from separate branch feed circuits. To help protect the system
from sudden increases or decreases in electrical power, use a surge suppressor,
line conditioner, or uninterruptible power supply or UPS.
2: To avoid personal injury or damage to the hardware, always use two people to
lift or move a node or chassis. A Gen 6 chassis can weigh more than 200 lbs. It is
recommended to use a lift to install the components into the rack. If a lift is not
available, you must remove all drive sleds and compute modules from the chassis
before lifting. Even when lifting an empty chassis, never attempt to lift and install
with fewer than two people.
3:
Electrostatic Discharge
4: If you install PowerScale nodes in a rack that is not bolted to the floor, use both
front and side stabilizers. Installing PowerScale nodes in an unbolted rack without
these stabilizers could cause the rack to tip over, potentially resulting in bodily
injury. Use only approved replacement parts and equipment.
6: You can install racks in raised or nonraised floor data centers capable of
supporting that system. It is your responsibility to ensure that data center floor can
support the weight of the system. A fully populated rack with A2000 chassis’
weighs about 3,500 lbs (1,590 kg). If the floor is rated at less than 3,500 lbs, then
additional care and planning must be taken. Some data center floors have different
static load vs. dynamic (rolling) load specifications, and sectional weight and load
point limits. This becomes important while moving preracked solutions around the
data center.
1Failure to heed these warnings may also void the product warranty. Only trained
and qualified personnel should install or replace equipment. Select the button
options for specific information. Always refer to the current Site Preparation and
Planning Guide for proper procedures and environmental information.
SolVe
Before you arrive at a client site, remember to read the call notes and follow the
processes that are detailed in them. Check if there are any special instructions from
PowerScale Technical Support that you must follow.
1: Download SolVe Desktop application on the system. Go to the Tools and Sites
section, choose SolVe. And select SolVe Desktop Executable. Depending on the
browser used, you may be presented with security dialogue boxes. Take the
needed actions to launch the executable.
2:
Click through the Setup wizard and then select Install. Clicking Finish launches
the SolVe Desktop. SolVe must be authorized for use. Select OK. A few general
items2.
2 Notice the dialog in the lower left showing the version. This area also shows the
progress when upgrading and downloading content. Also notice in the lower right
the service topics. Once connected, many of articles that are shown may not be
3: From the menu, select Authorize and download the list of available products.
Adhere to the instructions shown, that is to leave SolVe open, enter credentials,
this is using SSO, and open the keychain file. Select OK. And then go to
downloads and open the keychain file.
4:
Next are the Release Notes. Review and then close this window. Bring back the
SolVe. Notice the dialog3 in the lower left indicating the keychain is loaded, that
means you are authorized, and content is updated. Now, scroll down, and click
PowerScale to gather the PowrScale content.
5: Click OK. Again, note the progress in the lower left. Once the download is
complete, you see that the PowerScale image has changed. Tools that are
downloaded appear in the upper left corner of the screen without the green arrow
present.
6: Now you can click PowerScale and view the available procedures. If updates are
available for download, you see an information icon, click the icon, and approve the
updated content download.
3The icons with a green arrow indicate that the user must click the icon in order to
download the tool.
4The view is dependent upon Partner Type. A service partner sees what an
employee sees, a direct sales partner sees what a customer sees, and an
ASP/ASN partner sees products depending upon credentials.
SolVe Online
Select each panel for a description for using SolVe Online and generating a
procedure.
Home
The graphic shows the SolVe Online home page. The home page list the latest
advisories and shows the products. The product folders contain any procedure
routes you accessed.
Procedure Routes
SolVe Online retains a history of the procedure routes you accessed. The graphic
shows route. You can download the document from the route.
The example shows the routes to replacing an A200 node and setting up a 40U-P
rack.
Procedure routes.
New Route
When you click on the top level product on the home page, SolVe Online starts the
menu for a new route.
As an example, you want the procedure to replace a drive in an H400 node. Here,
you select Replacement Procedures.
Product Topics
From the Isilon platform page, open the Replacement Procedures topic. The
example shows both the PowerScale and the Gen 6 topics expanded. Select the
H400 topic.
Procedure Generator
The generator launches and choose the Replace Drive procedure, and then Next.
Generator Steps
Step 2 asks for the usage information and step 3 shows the procedure to generate.
Procedure Document
Generating the document may take a few minutes. Once complete, the PDF is
downloaded, and you return to the home page. On the home page, the new route is
listed.
Script:
Let us see how easy it is to generate a document. For this exercise, the SolVe
Desktop is installed, and you step through the process to get the Isilon Site
Preparation and Planning Guide. Shown is the shortcut on the desktop. Click the
shortcut to launch the tool.
License Agreement:
SolVe:
The SolVe Desktop launches. Now click the Isilon image. Here we get the
procedures. Expand Node Procedures.
Node Procedures:
Select X210. Select Site Planning and Preparation Guide, then select Next.
Selection:
We keep the destination as shown. Select Generate. Note the information and then
click OK.
Generated:
A word doc is shown that gives some general information. Scroll down. The image
of the document that is shown here is a link to that document. Click the image. The
PDF is opened.
The installation process starts before you arrive at the customer site. When
planning an installation, there are a few items that you must know before you
arrive.
IP Address Verification
The Customer Engineer (CE) typically works with the Implementation Specialist
(IS), who completes any custom configurations after the initial configuration is
done. CEs must verify that all internal networks configured for a PowerScale cluster
use IP version 4 addresses.
Use the pre-engagement questionnaire (PEQ) throughout the installation and initial
configuration process.
Make sure you discuss with the customer the plan for the spacing of the Gen 5
nodes or Gen 6 chassis5.
5 PowerScale hardware can take from 1U to 4U. All Gen 6 chassis are 4U and the
recommended maximum number that can be installed in a standard 19" rack is 12,
with the additional space needed for switches or cabling. Up to 10 chassis can fit in
a 42U rack. Keep in mind that Gen 6 nodes based on the High and Ultra compute
modules will contain high line only (240V) power supplies. When used in an
environment where only low-line power (120V) is available, two rack-mounted step-
up transformers will be required per Gen6 chassis, each of which will consume 2U
additional rack space per chassis and thus reduce the amount of space available
for nodes. Racks should be populated from bottom to top to improve rack stability
and decrease the risk of tipping.
Heavier nodes6
Pairs of step-up transformers7
Ethernet backend8
1U accelerator nodes and SRS gateway servers9
7If needed, should be placed adjacent to their associated chassis, one above the
chassis and the other below (to prevent cabling issues).
8 A new Gen 6 or Gen 6.5 cluster will have Ethernet switches for the BE network.
Gen 5 Node processor Gen 6 Node processor Gen 6.5 Node processor
hierarchy hierarchy hierarchy
- HD-Series - A2000
The node on which you initially run the Configuration Wizard is designated with a
node ID of 112. If selecting between two node types, such as an H500 and F800,
9 1U accelerator nodes and SRS gateway servers should go at the top of the rack.
10 Any gaps between nodes or switches require a filler panel to ensure proper
airflow.
11The cable management tray option requires 1U of rack space per node, so you
will need to plan for the additional space.
with equal OneFS versions, choose the node type with the greater processor power
as node 1.
The processor speeds of each Gen 5, Gen 6, and Gen 6.5 node type are shown in
descending order.
Serial Number
Serial Number
Movie:
The web version of this content contains a movie.
Script:
For A100 accelerator nodes, you must remove the face plate from the front of the
node. Press both latches in simultaneously until they click. Then remove the face
plate. Locate the black plastic tab on the upper left of the node and slide it out. The
serial number is printed on the blue label. When you are done, slide the tab back in
and replace the face plate.
Serial Number
The service tag is in the front of the system or may be on a sticker on the chassis
of the system. The mini Enterprise Service Tag (EST) is found on the back of the
system. Dell uses the information to route support calls to the appropriate
personnel.
1
2 3 4
5 6
1: Pull out the information tag in front of the system to view the Express Service
Code and Service Tag.
5: Service Tag.
Prepare Site
Module Objectives
You can use a standard ANSI/EIA RS310D 19-inch rack system14 in a PowerScale
cluster installation. The rack cabinet must have full earth ground to provide reliable
grounding. The rack should have dual power supplies with a power distribution that
allows load balancing and switches between the two if there is an outage. Use a
rack cabinet that has dual power distribution units, one on each side.
14The default PDU configuration includes 4 PDUs. You receive power cables to
support the maximum configurations and are likely to have extra cords as part of
your shipment.
Generate the appropriate guide using SolVe. The guide explains the tools that are
required, unpacking, and building the rack.
The next step is to attach power cables to the power distribution units on each side
of the cabinet.
15You must connect power cords to unpopulated P1 and P2 connectors on the four
power distribution units within the cabinet.
17The default PDU configuration includes 4 PDUs. You receive power cords to
support the maximum configurations and are likely to have extra cords as part of
your shipment.
40U-P cabinet PDUs do not include a power on/Off switch. Ensure that the circuit
breakers on each PDU are UP, and OFF, position until ready to supply AC power 18
to the unit.
18Equipment that is correctly installed within the rack cabinet is grounded through
the AC power cables and connectors. In general, supplemental grounding is not
required.
19If your site requires external grounding (for example, to a common grounding
network beneath the site floor), you can use the grounding lugs that are provided
on each of the lower power distribution panels of the cabinet.
The customer is responsible for bolting the rack to the floor. If a third-party rack is
used, be sure to secure the rack to the subfloor with a stabilizer bracket or some
other method. Use both front and side stabilizers when installing nodes in a rack
that is not bolted to the floor. Installing in an unbolted rack without these stabilizers
could cause the rack to tip over, potentially resulting in bodily injury. Use only
approved replacement parts and equipment.
1 2 3
5 4
1: If the rack is not bolted to the floor, it is recommended that you secure your rack
cabinet to the floor with stabilizing brackets or use anti-tip devices. The stabilizing
brackets are also important if you are installing or removing a server in the upper
half of the cabinet when the lower half is empty.
2: Sometimes, the trim strips are unattached from the cabinet, or was removed to
detach the shipping brackets. Once the cabinet is secured to the floor stabilization
brackets, install the lower trim strip to the bottom front of the cabinet.
3: The seismic bracket is installed on the front, back, and sides of the cabinet.
5: The anti-move bracket is installed on the front and back of the cabinet.
Once the rack is built, you are ready to unpack the components. First, check the
ShockWatch meter to ensure it has not been activated. If the ShockWatch meter
has been activated, be sure to contact support before continuing. Ensure that you
have sufficient clearance around the rack cabinet. If components are still packed
onto a pallet, you need approximately eight ft of clearance to position the pallet.
Open the shipping container and remove the PowerScale components.
More information20.
20To avoid personal injury or component damage, use two people to lift and move
a Gen 6 chassis. Inspect each component for any sign of damage. If the
components appear damaged in any way, notify PowerScale technical support. Do
not use a damaged component.
After you have built the rack, you will want to repack and return the shipping
material as it is reused for cost and environmental reasons. Leave the job site of
the customer as clean as or cleaner than it was when you arrived.
1: The first step is to lay each of the outer sleeves flat, then fold it lengthwise
before collapsing the three sections.
2: The second step is to stack the shipping brackets on the pallet. After that, place
ramp section B upside-down on the pallet, then stack section A, right-side up. Then
stack the two folded outer sleeves above the ramp sections.
3: The third step is to place ramp section B upside-down on the pallet, then stack
section A, right-side up. Then stack the two folded outer sleeves above the ramp
sections. Next, apply the two collar sections to either side of the pallet, hold them in
place with the package hood.
4: Finally band the unit with shipping straps and return it to Dell EMC.
Module Objectives
Sliding Rails
Once you have built your rack cabinet, which is unpacked the node and verified its
contents, and repacked the shipping material, you can start installing the hardware.
Click each tab to learn more.
Sliding Rails
The first step is to attach the sliding rails. A sliding rail system is used to attach the
node or chassis to the rack or cabinet and allow you easy access to the
components. The sliding rail kit is compatible with rack cabinets with the following
hole types:
Adjustable Rails
1U, 2U, and 4U rails for nodes and chassis adjust in length from 24 inches to 36
inches to accommodate a variety of cabinet depths. The rails are not left-specific or
right-specific and can be installed on either side of the rack. The video
demonstrates installing the rails for a Gen 6 chassis.
Movie:
The web version of this content contains a movie.
Pre-Threaded Holes
Movie:
The web version of this content contains a movie.
Considerations
Pre-Threaded Holes Script: To install the rail kit in a rack with prethreaded holes,
replace the preassembled step alignment pins in the ends of the outer rails with the
proper shoulder alignment pins. Adjust the outer slide rail rear slide bracket to fit
the depth of the rack cabinet, ensuring that the alignment pins protrude through the
rack mounting rails. Secure each outer rail to the rack cabinet using two 8, 32 x
0.75” knurled thumbscrews in the center holes of the slide rail.
Installing 1U Nodes
Before you begin, verify that the rail kit contains all the necessary components.
Each 1U slide rail consists of a slide bracket, an outer rail, an intermediate rail, and
an inner rail. Change the rail alignment posts if needed. Download and view the
SolVe Desktop procedure for the most recent instruction details.
1: Remove the inner rails from the rail assemblies on both rails.
3: Install both the slide rails in the rack. Each assembly contains a right and a left
slide rail.
4: You are now ready to install the node on the rails in the rack, pushing in until the
slide rails are locked. Remember, the enclosure is heavy and should be installed
into or removed from a rack by two people.
5: To further secure the rail assembly and server in the cabinet, insert and tighten a
small stabilizer screw directly behind each bezel latch.
Installing 2U Nodes
Movie:
Installing 2U Nodes Transcript: First, verify that the rail kit contains all the
necessary components. The 2U node rail kit includes:
Each slide rail consists of a slide bracket, an outer rail, an intermediate rail, and an
inner rail. To remove the inner rails from the rail assemblies, fully extend the slide
rails to the open and locked positions. Press the release button on the inner slide
rail and pull it from the intermediate slide rail until they separate.
Next, you attach the inner rails to the sides of the node. Align the large end of the
keyhole slots of the inner rail over the shoulder screws on the side of the node
chassis. It may be necessary to loosen the shoulder screws to slide the rails on.
Push the inner slide rail until it locks into place. Tighten the shoulder screws. Then,
attach the second slide rail on the other side the same way. To attach the rail to the
rack, first determine the rack cabinet that you are using. See the Rail Kit Installation
Guide for specific instructions for different types of rack cabinets. Here, we are
using a rack cabinet with 3/8” square holes. (Note: If using a rack with prethreaded
holes, replace the preassembled step alignment pins in the ends of the outer rails
with the proper shoulder alignment pins before proceeding to the next step.)
First, place the rear slide bracket into the back of the rack. Adjust the outer slide rail
to fit the depth of the rack cabinet, ensuring that the alignment pins protrude
through holes in the rack mounting rails and that the outer and rear slide brackets
are level. Attach to the same holes on both ends of the rack. Place the node
retaining bracket on the front of the right-side rail so that the middle hole covers the
top alignment pin. Then secure the bracket with one of the 8-32 x 0.75-inch knurled
thumbscrews. Secure all other ends of the rail to the rack by inserting the 8-32 x
0.75-inch knurled thumbscrews. Tighten the screws. Repeat the process for the
second rail.
Next, install the node in the rack. However, Do Not Continue until you confirm that
both rails are secured to the rack and all mounting screws are in place and
tightened. Failing to do so could result in injury or damage to the node, and to avoid
injury, use two people to lift and move the node.
Fully extend each intermediate slide rail until the rail is fully open and locked into
place. With a person on each side of the node, keep the node level with the slide
rails and align the ends of the inner slide rails with the ends of the intermediate
slide rails. Slide the inner slide rails on each side of the node into the intermediate
slide rails, and then retract the slide rails until the node is fully inserted in the rack.
There is an audible click when the node is secure. Confirm that the inner slide rails
attached to the node are inserted correctly and firmly secured to the intermediate
slide rails that are attached to the rack.
Secure the node to the rack cabinet using either the 6-32 x 0.25-inch knurled
chassis retaining screws or chassis retaining screws and retainer bracket
depending on the node type. Tighten the screws.
Installing 4U Nodes
Before you begin, unpack and verify that the rail kit contains all the necessary
components. Each slide rail consists of an inner rail, an outer rail, and an
intermediate rail. To attach the rail to the rack, first determine the rack cabinet that
you are using. See the Rail Kit Installation Guide for specific instructions for
different types of rack cabinets. The graphic shows the installation for a rack
cabinet with 3/8” square holes.
1: Place the rear slide bracket into the back of the rack. Adjust the outer slide rail to
fit the depth of the rack cabinet, ensuring that the alignment pins protrude through
holes in the rack mounting rails and that the outer and rear slide brackets are level
and attached to the same holes on both ends of the rack. Secure all other ends of
the rail to the rack by inserting the 8-32 x 0.75-inch knurled thumbscrews. Tighten
the screws. Repeat the process for the second rail.
2: Attach a clip-on nut retainer to the third hole above the slide rail on each side of
the rack. The next step is to install the node in the rack. However, do not continue
until you confirm that both rails are secured to the rack and all mounting screws are
in place and tightened. Failing to do so could result in injury or damage to the node.
Remember to always use two people to lift a node. If a 4U node is shipped with
hard drives already installed, it can weight over 200 lbs. Use of a mechanical lift is
recommended. To use the cable management tray, be sure to leave 1U space
under the node for the cable management tray. For ease of installation, install the
Cable Management Tray before installing the main chassis.
3: Once the node is installed in the rack, insert the drives into the node.
4: Once the drives are inserted into the node, insert the front panel.
Important: 4U nodes ship with the inner slide rails pre-installed on the
node. If using a rack with prethreaded holes, replace the
preassembled step alignment pins in the ends of the outer rails with
the proper shoulder alignment pins before proceeding to the next
step.
A chassis containing drives and nodes can weigh more than 200 pounds. You
should attach the chassis to a lift to install it in a rack. If a lift is not available, you
must remove all drive sleds and compute modules from the chassis before
attempting to lift it. Even when the chassis is empty, never attempt to lift and install
the chassis with fewer than two people.
1. The first step in installing the chassis is to align the chassis with the rails that
are attached to the rack.
2. Slide the first few inches of the back of the chassis onto the supporting ledge of
the rails.
3. Release the lift casters and carefully slide the chassis into the cabinet as far as
the lift allows.
4. Secure the lift casters on the floor.
5. Carefully push the node off the lift arms and into the rack. Ensure to leave the
lift under the chassis until the chassis is safely balanced and secured within the
cabinet.
6. Install two mounting screws at the top and bottom of each rail to secure the
chassis to the rack.
1: Back-end switches
• Provided by PowerScale.
• Switch ports face rear21.
• Connect each switch to separate power circuit.
• For multiple power connectors22, connect each cable to different power circuit.
• Each pair of switch supports a single cluster23.
21 Install switches into the rack with the ports facing to the rear of the rack. In a
redundant switch deployment, each switch should connect to a separate power
circuit for redundancy.
22If the switch has multiple power connectors, connect each of the switch’s power
cables to a different power circuit. Some switches have additional power
connectors to support optional redundant power supplies that may not be installed.
Ensure that each power cable is plugged into a connector that actually provides
power to the switch.
• Switches24:
• QLogic 72-port
• Mellanox 36-port
• High-speed, unmanaged fabric.
• Greater range of sizes.
• Identical functionality for Gen5 and Gen 6 nodes.
• Switches25:
• Celestica
• Arista
• High-speed, managed fabric.
• Limited monitoring.
23 Each pair of back-end switches supports a single cluster. Only one cluster is
allowed to be connected to the same pair of back-end switches. Connecting cables
to the switch is covered later.
24
InfiniBand switches are still supported for Gen 5 nodes and for mixed Gen 5 and
Gen 6 node clusters.
25 A pure Gen 6 cluster uses Ethernet switches (Celestica and Arista initially) for its
back-end network. When adding Gen 6 nodes to an existing Gen 5 cluster, the Gen
6 nodes connect to the InfiniBand switches. Follow the manufacturer’s instructions
for mounting the switches in the rack. If the customer is building a new cluster of all
Gen 6 nodes, the back-end switches must be Ethernet.
• No small switches:
• 24-port minimum26
• 252 max27 (at launch).
26 The new Ethernet back-end is only supported on the Gen 6 nodes. The minimum
allowable switch size is a 24-port switch.
Legend:
• S - 1:1 supported
• LC - Line card required
• BO - Breakout cable (1:4)
• AN - Autonegotiate to lower speed
• NS - Not supported
• XS - Supported, but not recommended
Dell Z9100 BO S XS BO XS S
Dell Z9264 BO S XS BO XS S
Dell S4148 S NS S AN NS NS
Dell S4112 S NS S AN NS NS
Celestica BO S BO BO/AN AN NS
D4040
Celestica S NS S AN NS NS
D2060
Installing Components
Module Objectives
Installation Guides
You can generate installation guides using SolVe. The example shows the first step
in generating an F200 node install guide.
You may need to install chassis components after the chassis is racked. For
instance, if no lift is available to install the chassis into the rack, then remove the
compute modules and drive sleds of the node. Once removed, install the chassis,
using two people, and then reinstall the compute modules and drive sleds of the
node. Select each tab to view a short demonstration on installing the drives, drive
sleds, and compute modules.
This short video demonstrates installing drives into a drive sled and installing the
sled into a chassis. See the student guide for the video transcript.
Movie:
The web version of this content contains a movie.
Compute Module
This video demonstrates how to install the compute module of a node into a
chassis. Refer the student guide for the video transcript.
Movie:
The web version of this content contains a movie.
Install Drive and Sled Script: For 3.5" drives, push the drive and paddle card
straight down into the empty connector. Insert the tabs on the drive sled cover into
the slots in the drive sled. For 2.5” drives, with the top cover at an angle, insert the
cover's three tabs into the slots on the sled. Next lower the open side of the cover
until it is flat against the drive sled. Slide the cover forward to lock it onto the sled.
Swing the blue sled cover latches back into place and press until you feel the
latches click. Ensure that the drive sled handle is open. With two hands, slide the
drive sled into the same bay you removed it from. Push the drive sled handle back
into the face of the sled to secure the drive sled in the bay. Swing the display
assembly back against the chassis until you feel it click into place. Align the front
bezel with the front of the chassis, then push until you feel the bezel snap into
place.
Install Compute Module Script: Keep the lever in the open position until the
compute module is pushed all the way into the bay. Support the node with both
hands and slide it into the node bay. Push the release lever in against the compute
module back panel. You can feel the lever pull the module into place in the bay. If
the lever does not pull the compute module into the bay, pull the lever back into the
open position. Next, ensure that the module is pushed all the way into the bay, then
push the lever in against the compute module again. Tighten the thumbscrew on
the release lever to secure the lever in place.
The F200 has four SAS SSDs numbered from 0 to 3 and uses no SAS expander.
The F600 drive bays 0 and 1 are empty due to the internal cabling layout of the
node. The F600 has eight NVMe SSDs that use bays 2 to 9.
Bay 0 and bay 1 - Bay 2 and Bay 4 and Bay 6 and Bay 8 and
NOT USED bay 3 bay 5 bay 7 bay 9
1
3
8
4
5 9
6 10
7
11
1: CPU2 is available on the F600. On the F200, this area is the location of the
NVDIMM battery.
5: Disk drives. The F200 uses SAS SSDs, and the F600 uses NVMe SSDs. Hard
drives are supplied in hot swappable hard drive carriers that fit in the hard drive
slots. You can check the status indicator on the carrier to identify a faulted hard
drive.
7: Backplane. Label each hard drive before removing from the node to ensure that
the drives are returned to the same drive bay.
8: Power supply units (PSU). The node needs one PSU for normal operation.
Remove and replace only one PSU at a time in a node that is powered on. PSUs
must be of the same type and have the same maximum output power.
9: Internal Dual SD Module (IDSDM). The F200 and F600 nodes use one of the
two micro SD slots. The micro SD shows up in OneFS as a USB device (da0).
isi_hwmon monitors IDSDM. IDSDM in OneFS is used for secondary backup for
PSI information and temporary storage for FW upgrade binaries.
10: Frontend NIC. The F600 uses PCIe slot 3 for front-end connectivity, and the
F200 uses the rack network daughter card (rNDC).
F200 rNDC.
11: Backend. The backend NIC is installed in slot 1 for both the F200 and F600.
The F200 has a 25 GbE NIC while the F600 uses a 100 GbE NIC.
Bezels in A-Series devices are accelerator node specific and may not appear as
shown. Bezels may include a keylock. All bezels include tabs on either side that
you press in to release the bezel and its latches. Push the ends, not the middle, of
the bezel, press the bezel onto the latch brackets until it snaps into place. The
video demonstrates installing the bezel on a Gen 6 chassis.
Movie:
Module Objectives
Once the system is racked and stacked, move on to connecting the node to the
internal and external networks.
Network
A PowerScale cluster uses separate internal and external networks for backend
and front-end connectivity.
Internal Network28
External Network29
28The internal network is dedicated for intracluster data and messages, providing
point-to-point connectivity with microsecond latency. Nodes connect to the internal
network of a cluster to communicate with other cluster nodes. Nodes connect to the
external client network to communicate with the rest of the world.
29 Gen 6 nodes can use either InfiniBand or Ethernet switches for back-end
connectivity. You cannot mix Ethernet and InfiniBand back-ends within one cluster.
If Gen 6 nodes are added to a cluster with Gen 5 nodes, the backend uses
InfiniBand switches only.
InfiniBand30
Hybrid Cables31
The graphic shows a Gen 6, four node cluster with an InfiniBand backend. There
are no special considerations when connecting to an Ethernet back-end. The two
steps are:
30All current Gen 5 and Gen 6 nodes and InfiniBand switches use 40 Gb/sec Quad
Data Rate (QDR) InfiniBand, which uses QSFP type cables. Legacy nodes and
switches use 10 Gb/sec Single Data Rate (SDR) or 20 Gb/sec Double Data Rate
(DDR) InfiniBand which uses CX4-type cables.
31 Hybrid cables are available to connect Legacy nodes to current IB switches and
the opposite way. Reference the PowerScale Supportability and Compatibility
Guide if working with legacy switches and nodes to determine the cable type
needed. Nodes with QDR InfiniBand adapters are incompatible with SDR
InfiniBand switches.
1. Connect an InfiniBand or Ethernet cable between the int-a port and the switch
(switch 1) for the Internal A network.
2. If the network topology supports a second internal network, connect the int-b
port to a separate network switch (switch 2) for the int-b network.
Gen 6.5 only supports Ethernet. Each Gen F200 and F600 node provides two ports
for backend connectivity. The PCIe slot 1 is used. F600 nodes use 100GbE or
25GbE ports. F200 nodes use 10GbE or 25GbE ports.
Breakout Cables
The 40 GbE and 100 GbE connections contain 4 individual lines of 10 GbE and 25
GbE.
Most switches support breaking out a QSFP port into four SFP ports using a 1:4
breakout cable.
The backend is managed automatically when the switch detects the cable type as a
breakout cable.
Each cable and the cabling should adhere to the cabling recommendations listed:
• Color code the cables to depict their type and function within the environment.
• Both ends of the cables should be clearly labeled depicting cable usage and
connection points in accordance to the labeling convention.
• Labeling should be done with correct type of label to sustain environmental
stress such as heat and humidity.
• Attached labels should not constrict the cable.
• The minimum bend radius of an InfiniBand cable is 20 cm.
• Avoid placing cables along sharp edges.
• Do not zip and constrict cables.
• Avoid using zip ties for the cable bundling. Instead use Velcro ties.
• Bundle different cable types separately and keep a good distance between
power and InfiniBand cables.
• Ensure that your cabling support is well anchored and able to withstand cable
weight.
• Restrict cables from dangling between rack spaces.
• Retain a sufficient loop on the cable in order to facilitate maintenance work, for
example, when sliding out a node.
• Align cable placements in order to facilitate future addition of cables along the
path.
External networks connect the cluster to the outside world. Subnets are used in
external networks to manage connections more efficiently. Specifying external
network subnets depends on the topology of the network32. The Ethernet cable
connects the node to the external network of cluster so the node can communicate
with external clients. Use an Ethernet cable to connect the ext-1 port on the node to
the switch for the external network.
Gen 6 nodes33 have two ports, ext-1 and ext-2, as the graphic depicts.
Gen 6.5
Protocols: NFS, SMB, S3,
2x10 GbE, 2x25 GbE, 2x 40
HTTP, FTP, HDFS, SWIFT
GbE, 2x100 GbE
32In a basic network topology, one in which all nodes reside on a single subnet,
only one external subnet is required.
33
The Gen 6 node can connect to one or more external Ethernet networks using 10
GbE or 40 GbE connections.
The graphic shows a closer look at the external and internal connectivity.
Slot 1 is used for backend communication on both the F200 and F600.
Slot 3 is used for the F600 2x 25 GbE or 2x 100 GbE front-end network
connections.
The rack network daughter card (rNDC) is used for the F200 2x 25 GbE front-end
network connections.
PCIe slot 1 - used for all BE PCIe slot 3 - used for F600
communication FE
FE FE
The graphic shows the R640 and does not represent the F200 and F600 PCIe and rNDC
configuration.
Connecting Power
Nodes and chassis contain redundant power supplies to ensure that the node
remains powered in case a power supply fails. Connect the power supply cables to
the node power connectors, and then connect each power supply cable to a
separate power source.
• Gen 6
o Compute has single power
supply.34
o Node-pair provides redundant
power.35
o Nodes automatically powers
on.36
• Gen 6.5
The video demonstrates connecting An LED shows the current state of each power
the power cord to a Gen 6 node. supply.
Movie:
The web version of this content contains a movie.
35If the power supply fails, the node receives power from the power supply of the
adjacent node.
36Gen 6 nodes automatically power on when they are connected to power. When
setting up new nodes, if you are not yet ready for them to be powered on, do not
connect the power cables.
Connecting Power Cord Transcript: Connect the power cable to the power
supply. Rotate the metal bail down over the power cable to hold the cord in place.
Configuration Manager
For initial configuration, access the CLI by establishing a serial connection to the
node designated as node 1. The serial console gives you serial access when you
cannot or do not want to use the network. Other reasons for accessing using a
serial connection may be for troubleshooting, site rules, a network outage, so on.
Serial Port37
37 The serial port is usually a male DB9 connector. This port is called the service
port. Connect a serial null modem cable between a serial port of a local client, such
as a laptop, and the node service port. Connect to the node designated as node 1.
As most laptops today no longer have serial ports, you might need to use a USB-
to-serial converter. On the local client, launch a serial terminal emulator.
• Data bits = 8
• Parity = none
• Stop bits = 1
• Flow control = hardware
Course Summary
Course Summary
Now that you have completed this course, you should be able to:
→ Plan an installation.
→ Prepare the site.
→ Install the node in the rack.
→ Install components.
→ Cable the node to a network.
Electrostatic Discharge
Electrostatic Discharge is a major cause of damage to electronic components and
potentially dangerous to the installer. To avoid ESD damage, review ESD
procedures before arriving at the customer site and adhere to the precautions when
onsite.
Antistatic Packaging:
Leave components in
antistatic packaging until
time to install.