LSI SAS Utilities - Locating a Disk
The Context:
So, I had a failed disk in my ZFS array…
sarahkat@srv:sas2ircu_linux_x86_rel $ zpool status pool: wolf
state: DEGRADED
status: One or more devices are faulted in response to persistent errors.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Replace the faulted device, or use 'zpool clear' to mark the device
repaired.
scan: none requested
config:
NAME STATE READ WRITE CKSUM
wolf DEGRADED 0 0 0
raidz2-0 DEGRADED 0 0 0
sdb ONLINE 0 0 0
sdc ONLINE 0 0 0
sde ONLINE 0 0 0
sdf FAULTED 0 12 0 too many errors
sdg ONLINE 0 0 0
sdh ONLINE 0 0 0
sdi ONLINE 0 0 0
errors: No known data errors
The Problem:
Of course, spotting which device pointer in software has the failure is easy, but here’s the problem:
Which dang physical drive bay corresponds to /dev/sdf??? This led me down a small rabbit hole.
I found that lsscsi, lsblk, etc don’t really show any details about physical slot details. Though they do come in use later.
Finding the Utility:
I ended up finding this utility linked through supermicro that promises to show some useful information about the disk setup…
Interestingly, this isn’t a well-known utility, it’s not available via software repositories, and this is basically the one single place I could find a download. (I would like to host the .zip out of fear of the page going 404, but, without knowing the licensing details about distribution, I’d rather just keep it in my private archive for now)
I was able to download it, unzip it, and mark the sas2ircu_linux_x86_rel/sas2ircu file as exectuable and run it. Note: This utility relies on being run as a privileged user, i.e. under sudo or as root.
Finding which disks are in which slots:
First, let’s get the controller information just to be sure:
sarahkat@srv:sas2ircu_linux_x86_rel $ sudo ./sas2ircu LIST
LSI Corporation SAS2 IR Configuration Utility.
Version 5.00.00.00 (2010.02.09)
Copyright (c) 2009 LSI Corporation. All rights reserved.
Adapter Vendor Device SubSys SubSys
Index Type ID ID Pci Address Ven ID Dev ID
----- ------------ ------ ------ ----------------- ------ ------
0 SAS2008 1000h 72h 00h:02h:00h:00h 1028h 1f51h
SAS2IRCU: Utility Completed Successfully.
Alright, so we can see under the Index column, our controller number is ‘0’, we can use this in the syntax of the next command: ./sas2ircu <controller number> DISPLAY to get info on each drive attached to the controller.
sarahkat@srv:sas2ircu_linux_x86_rel $ sudo ./sas2ircu 0 DISPLAY
LSI Corporation SAS2 IR Configuration Utility.
Version 5.00.00.00 (2010.02.09)
Copyright (c) 2009 LSI Corporation. All rights reserved.
Read configuration has been initiated for controller 0
------------------------------------------------------------------------
Controller information
------------------------------------------------------------------------
Controller type : SAS2008
BIOS version : 7.27.01.01
Firmware version : 20.00.07.00
Channel description : 1 Serial Attached SCSI
Initiator ID : 0
Maximum physical devices : 255
Concurrent commands supported : 3432
Slot : Unknown
Segment : 0
Bus : 2
Device : 0
Function : 0
RAID Support : No
------------------------------------------------------------------------
IR Volume information
------------------------------------------------------------------------
------------------------------------------------------------------------
Physical device information
------------------------------------------------------------------------
Initiator at ID #0
### INFORMATION CUT FOR BREVITY
Device is a Hard disk
Enclosure # : 2
Slot # : 3
State : Ready (RDY)
Size (in MB)/(in sectors) : 1907762/3907096671
Manufacturer : SEAGATE
Model Number : DKS2C-H2R0SS
Firmware Revision : 5C05
Serial No : 9WM6WCX9
Protocol : SAS
Drive Type : SAS_HDD
Device is a Hard disk
Enclosure # : 2
Slot # : 4
State : Ready (RDY)
Size (in MB)/(in sectors) : 1907762/3907096671
Manufacturer : SEAGATE
Model Number : DKS2C-H2R0SS
Firmware Revision : 5C05
Serial No : 9WM4GB2Y
Protocol : SAS
Drive Type : SAS_HDD
Device is a Hard disk
Enclosure # : 2
Slot # : 5
State : Ready (RDY)
Size (in MB)/(in sectors) : 1907762/3907096671
Manufacturer : SEAGATE
Model Number : DKS2C-H2R0SS
Firmware Revision : 5C05
Serial No : 9WM4GAWE
Protocol : SAS
Drive Type : SAS_HDD
### INFORMATION CUT FOR BREVITY
------------------------------------------------------------------------
Enclosure information
------------------------------------------------------------------------
Enclosure# : 1
Logical ID : 5848f690:ef42fc00
Numslots : 8
StartSlot : 0
Enclosure# : 2
Logical ID : 500056b3:6789abff
Numslots : 38
StartSlot : 0
------------------------------------------------------------------------
SAS2IRCU: Command DISPLAY Completed Successfully.
SAS2IRCU: Utility Completed Successfully.
So, with this command, we can see that it tells us about the ‘Enclosure #’ and ‘Slot #’ for each drive. However, this doesn’t correlate for us the device pointer in use by linux, unfortunately. We don’t know which one is /dev/sdf, our ‘FAULTED’ drive as shown by ZFS. Well, the next command helps with that:
Finding the Serial Number for /dev/sdf:
sarahkat@srv:sas2ircu_linux_x86_rel $ lsblk --nodeps -o name,vendor,model,serial,type,tran
NAME VENDOR MODEL SERIAL TYPE TRAN
sda ATA ST9500620NS 9XF09LAJ disk sas
sdb SEAGATE DKS2C-H2R0SS 9WM6SCN40000C1506ZHH disk sas
sdc SEAGATE DKS2C-H2R0SS 9WM4GAWE0000C132EEPP disk sas
sdd ATA ST9500620NS 9XF06PDV disk sas
sde SEAGATE DKS2C-H2R0SS 9WM6WDDC0000C1364X9Y disk sas
sdf SEAGATE DKS2C-H2R0SS 9WM4GB2Y0000C1197W0L disk sas
sdg SEAGATE DKS2C-H2R0SS 9WM6SDD30000C14785RW disk sas
sdh SEAGATE DKS2C-H2R0SS 9WM6WCX900009150J0BU disk sas
sdi JMicron JMicron DB987654322DD disk usb
This command prints us out the ‘SERIAL’ column, which contains the full serial number of our disk. This correlates with the serial number our sas2ircu utility shows us. However, one quirk: sas2ircu only prints the first 8 characters. So we can match up sdf’s 9WM4GB2Y with sas2ircu to find that the serial number belongs to Enclosure 2, Slot 4.
I did try using sas2ircu’s LOCATE feature, which should toggle the trouble LED on the drive bay, however I only got a failure message in return. However, knowing which slot the device is in should allow us to live-pull and hot-swap the disk reliably without pulling the wrong disk and further degrading the zfs pool.
Some Final Notes:
Yes, I know these drives are old as all hell, and are the model used in the semi-proprietary NetApp SAN shelves and such. But, they’re cheap and I haven’t the cash to grab a 6+ drive array of brand new 6tb disks, so I’m letting these hold me over until then.
And secondly: The reason I’m having so much issue identifying the physical drive is because I’m choosing to use a generic LSI2008 firmware flashed onto the dell H310 for the pass-through HBA functionality. If this were the nicely-integrated Dell firmware, I’m sure it’d be easier, but their controllers don’t monitor disks S.M.A.R.T. data and such, as compared with the capabilities of ZFS. Thus, the troubles of dealing with a non-supported configuration.