EMC Isilon Customer Troubleshooting Guide - Troubleshoot Your SmartConnect Configuration-Docu63149
EMC Isilon Customer Troubleshooting Guide - Troubleshoot Your SmartConnect Configuration-Docu63149
TROUBLESHOOT YOUR
SMARTCONNECT
CONFIGURATION
Abstract
This guide will help you to troubleshoot the following scenarios:
-- You are unable to connect to the SmartConnect name.
-- The SmartConnect name resolves to an incorrect IP address.
September 3, 2019
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
Contents and overview
Note
Follow all of these steps, in order, until you reach a resolution.
2. Perform
Start troubleshooting
troubleshooting Page 4
steps in order.
SmartConnect setup
Page 13
Packet capture
Page 17
3. Appendixes
Appendix A
If you need further assistance
Appendix B
How to use this flowchart
Appendix C
Example output
Appendix D
Example output
Appendix E
Example output
Appendix F
Example output
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
Before you begin
CAUTION!
If the node, subnet, or pool that you are working on goes down during the course of
troubleshooting and you do not have any other way to connect to the cluster, you could
experience data unavailability.
Therefore, make sure that you have more than one way to connect to the cluster before
you start this troubleshooting process. The best method is to have a serial console
connection available. This way, if you are unable to connect through the network, you
will still be able to connect to the cluster physically.
For specific requirements and instructions for making a physical connection to the
cluster, see article 304071 on the Online Support site.
Before you begin troubleshooting, confirm that you can connect through either another
subnet or pool, or that you have physical access to the cluster.
1. Open an SSH connection to the cluster and log in by using the root account.
Note: If the cluster is in compliance mode, use the compadmin account to log in. All compadmin commands must be
preceded by the sudo prefix.
2. Change the directory to /ifs/dat a/Isilon_Support by running the following command:
cd /ifs/dat a/Isilon_Support
3. Run the following command to capture all input and output from the session:
screen -L
This will create a file named screenlog.0 that will be appended to during your session.
4. Perform troubleshooting.
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
Start troubleshooting
Introduction
Start troubleshooting here. For an overview of
the conventions used in this flowchart, see
Appendix B: How to use this flowchart. Start
Verify that the SmartConnect processes are running by running the following command for your
version of OneFS. Run the command from the node where the SmartConnect service IP (SSIP)
address resides:
See Appendix
_________G for example output.
Are the
processes Yes Go to Page 6
running?
No
Go to Page 5
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
SmartConnect configuration
Page
5
Yes
Go to Page 6
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
SmartConnect configuration (2)
- ________________________
Page 4 - Start troubleshooting
- ______________________________
Page 5 - SmartConnect configuration
Page
6
Identify the SSIP addresses for the specific subnet you are using by running the following
command:
OneFS 8.2
isi net work subnet s list -v | egrep -i " ^ [ [ :space:] ] * (ID|SC Service Addrs)"
OneFS 7.2
isi net works list subnet s
See__________
Appendix C for example output. Copy the output to a text file for reference later in the
troubleshooting process.
OneFS 8.x
isi net work pools list -v
OneFS 7.2
isi net works list pools -v
See __________
Appendix D for example output.
Go to Page 7
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
SmartConnect configuration (3)
Page
7
See _________
Appendix E for example output and explanation of what to look for.
See _________
Appendix E for example output and explanation of what to look for.
Do the
commands return an
IP address as per the No Go to Page 12
examples in
Note _________
Appendix E?
These two commands
should return similar
output, but both should
return an IP address that
resides on the cluster. Yes
Go to Page 8
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
SmartConnect configuration (4)
Does the
returned IP address
reside on the
No Go to Page 9
cluster?
Yes
Did the
nslookup query
resolve to the SSIP Yes Go to Page 13
address?
No
Go to Page 11
Example output
Cl ust er - 1# i si _f or _ar r ay - l " i f conf i g | gr ep 192. 168. 228. 10"
Cl ust er - 1: i net 192. 168. 228. 10 net mask 0xf f f f f f 00 br oadcast 192. 168. 228. 255 zone 1
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
SmartConnect configuration (5)
From the node where the SSIP address resides, attempt to resolve
the cluster name by running the following command, where
< SC_FQDN> is the fully qualified domain name of the SmartConnect
zone, and < ssip> is the SmartConnect service IP address:
Yes
Go to Page 10
Example output
Cl ust er - 1# i si _f or _ar r ay - s " i f conf i g | gr ep 192. 168. 228. 3"
cl ust er - 1: i net 192. 168. 228. 3 net mask 0xf f f f e000 br oadcast 192. 168. 228. 255 zone 1
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
SmartConnect configuration (6)
Page
10
ipconfig /all
See __________
Appendix F for example output.
No Yes
Go to Page 13 Go to Page 17
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
SmartConnect configuration (7)
- _________________________________
Page 8 - SmartConnect configuration (4)
- _________________________________
Page 10 - SmartConnect configuration (6)
Page
11
Did your
networking team
make any changes to
Yes Go to Page 13
your network?
No
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
SmartConnect configuration (8)
Page
12
Refusal
Go to Page 13
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
SmartConnect setup
- ________________________________
Page 8 - SmartConnect configuration (4)
- ________________________________
Page 9 - SmartConnect configuration (5)
- _________________________________
Page 10 - SmartConnect configuration (6)
- _________________________________
Page 11 - SmartConnect configuration (7)
- _________________________________
Page 12 - SmartConnect configuration (8) Page
13
Did fixing
the configured
SmartConnect name Yes End troubleshooting
resolve the
issue?
No
Go to Page 14
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
SmartConnect setup (2)
Page
14
Go to Page 15
Example output
Notes:
- A delegation record (or NS record) should exist, and it should match the zone name.
- An A record (IPv4 address) or AAAA record (IPv6 address), shown in the output as i nt er net addr ess, should
show as the SmartConnect Service IP address.
cl ust er - 1# nsl ookup - nor ecur se - t ype=ns exampl e. i si l on. l ocal 192. 168. 38. 101
Ser ver : 192. 168. 38. 101
Addr ess: 192. 168. 38. 101#53
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
SmartConnect setup (3)
Page
15
The A or AAAA record should point to a specific SSIP address for the
zone and the subnet where the SmartConnect name resides. If this is
not the case, you need to adjust your A or AAAA record so that it points
to the correct zone and subnet.
Go to Page 16
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
SmartConnect setup (4)
Depending on which DNS you are using, see one of the following articles about
configuring SmartConnect:
No
Has access
Go to Page 17 No been
restored?
Yes
End troubleshooting
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
Packet capture
- _________________________________
Page 10 - SmartConnect configuration (6) Note
- _________________________________
Page 12 - SmartConnect configuration (8) If no spaces exist between the
- ___________________________
Page 15 - SmartConnect setup (3) arguments, be aware that isi_netlogger
- ___________________________
Page 16 - SmartConnect setup (4) will parse flags improperly. The result is
that the isi_netlogger execution is
ineffective and captures nothing.
Page Ensure you leave a space between any
17 argument and the specified value.
Go to Page 18
Example output
vl an0: f l ags=8843<UP, BROADCAST, RUNNI NG, SI MPLEX, MULTI CAST> met r i c 0 mt u 1500
opt i ons=3<RXCSUM, TXCSUM>
et her 00: 25: 90: c7: 06: ee
i net 192. 168. 10. 1 net mask 0xf f f f f f 00 br oadcast 192. 168. 255. 255 zone 1
medi a: Et her net aut osel ect ( 1000baseTX <f ul l - dupl ex>)
st at us: act i ve
vl an: 1897 par ent i nt er f ace: em0
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
Packet capture (2)
Page
You could have arrived here from: 18
To stop the PCAP, press CTRL+C. The compressed PCAP file is saved in the
/ i f s/ net l og/ bundl ed directory and is named using the following time stamp format:
Review the results of the PCAP with your network and security
teams to ensure that the DNS queries from your DNS server are
reaching the node where the SSIP address resides.
No
Gather a full log set and PCAP output by running the following two commands, in succession:
isi_gat her_info
isi_gat her_info --local-only --nologs -f /ifs/net log
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
Appendix A: If you need further assistance
Upload node log files and the screen log file to Isilon Technical Support
1. When troubleshooting is complete, type exi t to end your screen session.
2. Gather and upload the node log set and include the SSH screen log file by using the command appropriate for
your method of uploading files. If you are not sure which method to use, use FTP.
ESRS:
isi_gat her_info --esrs --local-only -f /ifs/dat a/Isilon_Support /screenlog.0
FTP:
isi_gat her_info --ft p --local-only -f /ifs/dat a/Isilon_Support /screenlog.0
HTTP:
isi_gat her_info --ht t p --local-only -f /ifs/dat a/Isilon_Support /screenlog.0
SupportIQ:
Copy and paste the following command.
Note: When you copy and paste the command into the command-line interface, it will appear on multiple lines
(exactly as it appears on the page), but when you press Enter, the command will run as it should.
3. If you receive a message that the upload was unsuccessful, refer to ____________
article 304567 for directions on how to
upload files over FTP.
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
Appendix B: How to use this flowchart
Introduction
Describes what the section helps you to
accomplish.
End point
Document Shape
Calls out supporting documentation
for a process step. When possible,
these shapes contain links to the
reference document.
Sometimes linked to a process step
with a colored dot.
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
Appendix C: Example output
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
Appendix D: Example output
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
Appendix D: Example output (2)
- Previous page
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
Appendix E: Example output
A successful result will show the IP address. In this example, the returned IP address is
192.168.228.14, shown in bold in the following code example. If no IP address is listed, the
command failed. If you receive other failures (for example a timeout or an error message),
contact ___________________
Isilon Technical Support.
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
Appendix F: Example output
Host Name . . . . . . . . . . . . : dc
Pr i mar y Dns Suf f i x . . . . . . . : exampl e. l ocal
Node Type . . . . . . . . . . . . : Hybr i d
I P Rout i ng Enabl ed. . . . . . . . : No
WI NS Pr oxy Enabl ed. . . . . . . . : No
DNS Suf f i x Sear ch Li st . . . . . . : exampl e. l ocal
Et her net adapt er Local Ar ea Connect i on:
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
Appendix G: Example output
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.
Copyright © 2019 Dell Inc. or its subsidiaries. All rights reserved.
Dell believes the information in this publication is accurate as of its publication date. The
information is subject to change without notice.
Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks
may be the property of their respective owners.
EMC Corporation
Hopkinton, Massachusetts 01748-9103
1-508-435-1000 in North America 1-866-464-7381
www.EMC.com
___________________________
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
___________________
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isilon-docfeedback.