UPTIME BULLETIN

A Newsletter from SPE, a Division of Dell EMC For Unity//SC/VNX/VNXe Q1

KB 539664 – background info & best practices on VOLUME 32
March, 2020
the push to install new firmware on some SSDs
Reviewing new SSD 1
Dell EMC called attention to KB 539664 in last quarter’s Unity Uptime Bulletin and there are firmware discussed
plans to proactively push out drive firmware upgrade recommendations even more aggressively in KB 539664
in the form of a DTA or even an FCO in the near future. In this article, we will explore the under-
MS Windows hosts 2
lying issue we are attempting to fix, and we will address best practices and known issues asso- may lose access to
ciated with deployment of the firmware fix. volumes that were
Live Migrated
Knowledge base article 539664 discusses a deficiency in the firmware for certain SSDs com-
monly used by Unity storage systems. The complete list of affected part numbers can be found FAST Cache users 2
in the KB article. Dell EMC’s current drive firmware “big bundle” contains a fix for this issue, so should upgrade to
if you are not sure if you have any of the affected SSDs, you can simply use Unity Unisphere to Unity OE 5.x
install the latest “big bundle” of drive firmware. The Unity Unisphere tool will determine which
drives, if any, are downrev on firmware and will prompt the user to upgrade. Users will also typi-
Review of Unity 2
cally get prompted to upgrade their drive firmware after a successful OE upgrading using Unity Target OE 5.0.2
Unisphere. Reference KB solution 490700 for guidance on how to install drive firmware.
The issue that is being addressed by the new firmware: The deficiency which we fixing in Did you know? 2
the new firmware involves a premature invoking of the drive’s EOL flag. This can result in a
higher replacement rate than expected. It is important for customers to understand that when Latest code 3
and if an SSD’s EOL flag gets set, this does not automatically fail the drive. Rather, the drive releases and
will get marked for proactive sparing when and if hot spares are available to the system. When targets.
a drive proactively spares, the underlying RAID group is not degraded, so the process is safer in CloudIQ update. 4
comparison to a full rebuild. For this reason, Dell EMC does not consider this a critical issue,
although we do believe that it is important enough that customers should install the new firm-
ware for these drives at their earliest convenience. It is likely that Dell EMC may issue a DTA
or FCO soon that strongly advises that customers take this new disk firmware.
There is an alternative workaround discussed in KB 539664 for customers who cannot immedi-
ately apply the new firmware. Dell EMC has a script available that can be run against your stor-
age system that will set a bit on each affected drive that turns off reporting of correctable errors.
It is this particular reporting which is being incorrectly interpreted as a more serious error which
would warrant the EOL flag being set. This workaround isn’t meant to be a long term solution
as it doesn’t survive a reboot, and disabling recoverable error logging isn’t an ideal solution.
Special consideration: If a customer has a D@RE enabled storage system, in the unlike-
ly event that a system drive in slots 0_0_0 through 0_0_3 fails during a firmware upgrade, don’t
try to replace the drive without opening a case with support and referencing the above KB.
A small percentage of drive firmware upgrades for this fix may hang: Dell EMC is aware
that a small percentage of systems that have applied this particular drive firmware fix (about
3%) have encountered a single drive failure which in turn causes the drive firmware upgrade
process to hang. While a solution is being investigated, however, guidance remains un-
changed: the value of the fix outweighs the risk that a small percentage of upgrades may
hang. A new KB solution, 541103, is available which may be referenced if a customer opens a
case with technical support in the event that such a problem is encountered during the firmware
upgrade. The new solution presents the safest, fastest, easiest ways to recover from the hang
and resume the upgrade process.

Search the web for the Up-

Documentation time Bulletin here:
WE’D LIKE YOUR FEEDBACK
https://support.emc.com/docu947
ABOUT THE UPTIME  https://mydocuments.emc.com/ 01_VNX2,-VNXe,-Unity,-SC-
BULLETIN. SEND US YOUR VNX Uptime-Bulletin---Current-
IDEAS FOR FUTURE TOPICS Issue.pdf
AT : [email protected]
Customer
 UPTIME BULLETIN Page 2

Microsoft Windows hosts may lose access to volumes which were

Live Migrated following a host reboot
Microsoft Windows hosts that are connected to a Storage Center running versions 7.3 or 7.4 may lose access to vol-
umes on the SC following a host reboot if the volumes were part of a Live Migrate from one system to another. A cus-
tomer available workaround is detailed in article SLN320590 which you can get to by clicking on the link, or by searching
for the article number on dell.com/support. An alternate workaround is available by contacting Dell technical support.

Unity hybrid arrays using FAST Cache should be upgraded to any

Unity OE 5.0.x or higher
Dell EMC recently released the following DTA which discusses a rare yet very serious data integrity issue that could im-
pact FAST Cache of hybrid systems running Unity OE versions below 5.x.
DTA 540332: Dell EMC Unity: FAST Cache over provisioning problem causes computational error (User Correctable)
This issue is extremely rare, and it requires that FAST Cache disks invoke a “dynamic over-provisioning” branch of code
which only runs if the system determines that FAST Cache SSDs are wearing out much too quickly. Customers should
be advised that Dell EMC may issue an FCO targeting impacted systems as soon as Q2 of 2020 in an attempt to remove
exposure to our entire install base. It is possible that this FCO may include recommended critical disk firmware versions
as well. There are many other benefits to being on Dell EMC suggested “target” code levels as a matter of best practice.

Reviewing Unity OE 5.0.2.0.5.009 which was recently promoted to

the status of “Target”
There are many benefits to running the version of OE that Dell EMC declares to be the official “Target” revision for the
Unity storage system. An OE version must meet a rigorous set of criteria before it can be named target. This criteria
includes factors such as meeting a minimum number of installations, as well as minimum runtime hours, no unaddressed
regressions, and a very strict limit on how many high severity issues may exist. In addition to the security of knowing
that Unity OE 5.0.2 has met these rigorous thresholds, this particular upgrade has the following additional things to offer:
 51 bug fixes including 4 “blocked thread” fixes
 Security enhancements
 Serviceability enhancements, most notably including “phase 2” of a three phase set of improvements targeting dump
reliability, size reduction, and ease/speed of retrieval
 Includes the full fix for the .tmp file issue partially addressed in 5.0.1. See KB 537331 for more details

Did you know? KB 538181. This issue is in SC-Uptime-Bulletin---Current-

RHEL 7.6 itself and isn’t limited to Issue.pdf
 Hosts running RHEL 7.6 connect- Isilon storage systems.
ed to storage systems over NFS
may hang during node reboots or  The most recent officially pub-
IP address failovers. RHEL may lished version of the Unity/VNX/SC
have knowledge base articles of Uptime Bulletin may now always
their own addressing this issue. A be found at the following link:
KB has been written for this issue https://support.emc.com/
for Isilon which can be found here: docu94701_VNX2,-VNXe,-Unity,-
 UPTIME BULLETIN Page 3

Dell EMC Unity/SC/VNX/VNXe target versions

DELL EMC has established target revisions for each product to ensure stable and reliable environments. As a best practice, DELL
EMC recommends that you operate at target code levels or above to benefit from the latest enhancements and fixes available.
Search using the term “adoption rates” in http://support.emc.com for current Dell EMC Unity/VNX/VNXe target code adoption rates.

VNXe2 OE VERSION RELEASE DATE STATUS

3.1.10.9946299 08/26/19 Target
3.1.11.10003441 01/22/2020 Latest Release
VNXe1600 OE VERSION RELEASE DATE STATUS
3.1.9.9570228 11/30/17 Target
3.1.9.9570228 11/30/17 Latest Release
Dell EMC UNITY OE VERSION RELEASE DATE STATUS
5.0.2.0.5.009 01/16/2020 Target
5.0.2.0.5.009 01/16/2020 Latest Release
UNIFIED VNX2 OE VERSIONS (8.1 & R33) RELEASE DATE STATUS
8.1.9.236 (VNX for File) 01/29/19 Target
8.1.9.236 (VNX for File) 01/29/19 Latest Release
05.33.009.5.238 (VNX for Block) 05/02/19 Target
05.33.009.5.238 (VNX for Block) 05/02/19 Latest Release
SCv20x0, SCv30x0, SC4020, SC5020,
RELEASE DATE STATUS
SC7020,SC8000, SC9000 SCOS VERSION
07.03.20 08/21/19 Target
07.04.10 02/28/20 Latest

See Product Release Notes for a full list of

enhancements per new code release.
Dell SCOS OE enhancements Dell SCOS OE enhancements
in release 07.04.10 in release 07.04.02
 Enhancement: Added Mac OS X 10.x All SC models started shipping from the factory
Dell EMC Unity OE enhance- MPIO and VMware ESXi 7.0 to the Server in mid-December 2019 with SCOS 07.04.02.
ments in release 5.0.2.0.5.009 operating system list in Storage Center  Features expanded CloudIQ support
 Provides full/final relief to an issue intro-  Enhancement: Space reporting improve-  Offers Veeam Snapshot integration
duced in 5.0.0 that could cause some Mi- ments in the Storage Manager Client and  Supports Microsoft Windows server 2019
crosoft Office application file saves to de- Unisphere web interface
 Supports vSphere 6.7 server
lete the original save file and result in an  Fixed: After updating to Storage Center
earlier copy of the file with a .tmp extension 07.04.02, an SCv3000 series or SC5020  Offers HTML 5 WebUI initial configuration
 This release focuses on key serviceability storage system with 1378 W low-line PSUs
fixes and enhancements such as phase 2 might see an alert stating that the PSUs are Dell SCOS Disk FW updates
of a series of improvements designed for reporting an incorrect wattage
more reliability and speed in getting dumps  Fixed: A storage system with SEDs
scheduled for release
and reducing dump sizes Disk firmware updates for SCOS 07.03.20 and
might fail to initialize after a complete sys- 07.04.10 are scheduled for release by the end of
 Adds support for new SB-327 compliance tem power cycle March 2020. Please refer to the Storage Center
(California) as well as Lot 9 EU+ require-  Fixed: When new drives are installed in Drive Firmware Updates 2020 release notes on
ments a Storage Center, the drives might report dell.com/support for specific disk FW updates.
 Resolves an issue where a file system an incorrect Power On time of 136 years
could go offline due to a stale file handle  Fixed: A controller might reset because
of a null pointer error that occurs during
replication after updating to Storage Center
7.4.2
 UPTIME BULLETIN Page 4

Recent enhancements in CloudIQ

Converged System Support
We have added support for Converged
Infrastructure platforms (Vblock and
VxBlock Systems). You can now view
the Converged System inventory details
in CloudIQ. VxBlock Central Version 2.5
(or later) needs to be installed. SRS and
CDP must be enabled for collecting and
sending data.
Consolidated Advisor Access
CloudIQ has consolidated Dell EMC Ad-
visors and Partner Advisors to a single
grouping, which as a side benefit means
Partners need only be invited by Cus-
tomers entering their email address.
Mute Host Health checks
We've added the ability to temporarily
pause, and then resume, the Host
Health Checks for hosts that do not re-
quire high availability, or hosts that are in
maintenance. Use the new Customiza-
tion page under the Admin Tab to pause
and resume Host Health Checks. When
hosts are paused, a system's Host and
Host Details pages identify the hosts
whose health checks are in a paused
state. This feature applies to Unity and
SC Series Hosts only.
View a Summary of Reclaimable Stor-
age from the Overview page
We've added a new tile to the bottom of
the Overview page that summarizes the
reclaimable capacity of your systems.
The tile will show Used, Reclaimable
and Free capacities of a system in a bar
chart. The summaries are sorted by the
reclaimable capacity.

Dell Inc. believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” DELL INC. MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN
THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Use, copying, and distribution of any Dell EMC software described in this publication requires an applicable software license. Dell EMC and other trademarks are trademarks of Dell, Inc. or its subsidiar-
ies. Other trademarks may be trademarks of their respective owners. Copyright © 2019 Dell EMC Corporation. All rights reserved. Published in the USA, March, 2020.