Michael - Question on SAS backplane (camcontrol?)

New About Yours API Help
8.4 KB, Plain text
Hey Allan and Benedict,

I've got a SuperMicro Storage Bridge Bay server (two nodes connected to a shared SAS backplane) and I just popped in a couple SATA SSDs.  Since it's a dual-path SAS backplane, I grabbed a couple SAS interposer cards so that I could use them via either server.  I'm not planning on doing any sort of clustered filesystem or similar (since from all accounts, the interposers aren't super reliable/designed for that), but I wanted the flexibility to address them from whichever node was best, depending on what I ultimately decide to use them for.  
Unfortunately, it looks like the SSDs only registered with one of the two servers upon being plugged in.  I'm fairly sure that I can "deregister" one and have the other server pick it up, but I'm a little confused as to how.  I've looked at the camcontrol man pages, which I think are the right place, but honestly they're a little over my head and I don't want to accidentally mess up my server by running the wrong thing.  You guys have been asking for questions, so I figured I'd float it over, maybe you can give me a pointer.

As usual, wonderful show & thanks for doing it every week.

Thanks,

~Michael

Here is the output of dmesg on drive(s) plug-in from server 1 (the one that can't seem to talk to the disk, since it only shows the ses0 device, not the 
----------------------
[770] ses0: pass14,da11: Element descriptor: 'Slot 15'
[770] ses0: pass14,da11: SAS Device Slot Element: 1 Phys at Slot 14
[770] ses0:  phy 0: SATA device
[770] ses0:  phy 0: parent 500304800359f87f addr 500304800359f85a
[770] da11 at mps0 bus 0 scbus0 target 24 lun 0
da11: <ATA C300-CTFDDAC256M 0006> Fixed Direct Access SPC-4 SCSI device
da11: Serial Number 00000000110703034DEC
da11: 600.000MB/s transfers
da11: Command Queueing enabled
da11: 244198MB (500118192 512 byte sectors)
[1062] (da12:mps0:0:12:0): SERVICE ACTION IN(16). CDB: 9e 10 00 00 00 00 00 00 00 00 00 00 00 20 00 00
(da12:mps0:0:12:0): SCSI sense: ABORTED COMMAND asc:44,0 (Internal target failure)
(da12:mps0:0:12:0): Field Replaceable Unit: 60
[1062] (da12:mps0:0:12:0): fatal error, failed to attach to device
[1062] ses0: pass15,da12: Element descriptor: 'Slot 03'
[1062] g_access(958): provider da12 has error 6 set
[1062] ses0: pass15,da12: SAS Device Slot Element: 1 Phys at Slot 2
[1062] ses0:  phy 0: SAS device type 1 id 0
[1062] ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
[1062] ses0:  phy 0: parent 500304800359f87f addr 500605ba00e2a29d
[1062] g_access(958): provider da12 has error 6 set
[1303] mps0: mpssas_prepare_remove: Sending reset for target ID 24
[1303] da11 at mps0 bus 0 scbus0 target 24 lun 0
da11: <ATA C300-CTFDDAC256M 0006>  s/n 00000000110703034DEC detached
[1303] mps0: Unfreezing devq for target ID 24
[1303] (da11:mps0:0:24:0): Periph destroyed
[1303] xpt_release_devq(): requested 1 > present 0
[1350] (da11:mps0:0:24:0): SERVICE ACTION IN(16). CDB: 9e 10 00 00 00 00 00 00 00 00 00 00 00 20 00 00
(da11:mps0:0:24:0): SCSI sense: ABORTED COMMAND asc:44,0 (Internal target failure)
(da11:mps0:0:24:0): Field Replaceable Unit: 60
[1350] (da11:mps0:0:24:0): fatal error, failed to attach to device
[1350] g_access(958): provider da11 has error 6 set
[1350] ses0: pass14,da11: Element descriptor: 'Slot 15'
[1350] g_access(958): provider da11 has error 6 set
[1350] ses0: pass14,da11: SAS Device Slot Element: 1 Phys at Slot 14
[1350] ses0:  phy 0: SAS device type 1 id 0
[1350] ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
[1350] ses0:  phy 0: parent 500304800359f87f addr 500605ba00e08e65
---------------------------------------------------------

`camcontrol devlist` from server 1, showing the drives as /dev/pass14 and /dev/pass15 :
--------------
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 11 lun 0 (da0,pass0)
<C300-CTF DDAC256MAG    SM 4321>   at scbus0 target 24 lun 0 (pass14)
<C300-CTF DDAC256MAG    SM 4321>   at scbus0 target 12 lun 0 (pass15)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 13 lun 0 (da1,pass1)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 14 lun 0 (da2,pass2)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 15 lun 0 (da3,pass3)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 17 lun 0 (da4,pass4)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 19 lun 0 (da5,pass5)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 21 lun 0 (da6,pass6)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 22 lun 0 (pass7,da7)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 23 lun 0 (da8,pass8)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 25 lun 0 (da9,pass9)
<LSI CORP SAS2X36 0424>            at scbus0 target 26 lun 0 (ses0,pass10)
<SATADOM-ML 3SE S130710K>          at scbus1 target 0 lun 0 (ada0,pass11)
<AHCI SGPIO Enclosure 1.00 0001>   at scbus7 target 0 lun 0 (ses1,pass12)
<FLASH Drive SM_USB20 1100>        at scbus8 target 0 lun 0 (da10,pass13)
---------------
And dmesg from server 2 (the one that seems to have registered the drives correctly, seeing as they have /dev/daX entries
----------------------
da11 at mps0 bus 0 scbus0 target 19 lun 0
da11: <HITACHI HUA723020ALA64SM NA04> Fixed Direct Access SPC-3 SCSI device
da11: Serial Number 00000000110703034CEB
da11: 600.000MB/s transfers
da11: Command Queueing enabled
da11: Attempt to query device size failed: ABORTED COMMAND, Internal target failure
ses0: da11,pass14: Element descriptor: 'Slot 03'
ses0: da11,pass14: SAS Device Slot Element: 1 Phys at Slot 2
ses0:  phy 0: SAS device type 1 id 1
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 50030480003d6cff addr 500605ba00e2a29e
        (probe0:mps0:0:20:0): TEST UNIT READY. CDB: 00 00 00 00 00 00 length 0 SMID 971 terminated ioc 804b loginfo 31120101 scsi 0 state c xfer 0
da12 at mps0 bus 0 scbus0 target 20 lun 0
da12: <HITACHI HUA723020ALA64SM NA04> Fixed Direct Access SPC-3 SCSI device
da12: Serial Number 00000000110703034DEC
da12: 600.000MB/s transfers
da12: Command Queueing enabled
da12: Attempt to query device size failed: ABORTED COMMAND, Internal target failure
ses0: da12,pass15: Element descriptor: 'Slot 15'
ses0: da12,pass15: SAS Device Slot Element: 1 Phys at Slot 14
ses0:  phy 0: SAS device type 1 id 1
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 50030480003d6cff addr 500605ba00e08e66
-----------------
And `camcontrol devlist` from server 2, showing the drives as /dev/da11 and /dev/da12
-------------
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 8 lun 0 (pass0,da0)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 9 lun 0 (pass1,da1)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 10 lun 0 (pass2,da2)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 11 lun 0 (pass3,da3)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 12 lun 0 (pass4,da4)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 13 lun 0 (pass5,da5)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 14 lun 0 (pass6,da6)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 15 lun 0 (da7,pass7)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 16 lun 0 (pass8,da8)
<WD WD4001FYYG-01SL3 VR08>         at scbus0 target 17 lun 0 (pass9,da9)
<LSI CORP SAS2X36 0417>            at scbus0 target 18 lun 0 (pass10,ses0)
<HITACHI HUA723020ALA64SM NA04>    at scbus0 target 19 lun 0 (da11,pass14)
<HITACHI HUA723020ALA64SM NA04>    at scbus0 target 20 lun 0 (da12,pass15)
<SATADOM-ML 3SE S130710K>          at scbus1 target 0 lun 0 (pass11,ada0)
<AHCI SGPIO Enclosure 1.00 0001>   at scbus7 target 0 lun 0 (pass12,ses1)
<FLASH Drive SM_USB20 1100>        at scbus8 target 0 lun 0 (pass13,da10)
-----------------------
I checked server 2's `smartctl -a /dev/pass14` since they show as Hitachi HUAXXX here, but C300-CTF on server 1, and it correctly shows that they're the same thing, but the naming difference was odd to me.
----
smartctl 7.0 2018-12-30 r4883 [FreeBSD 11.2-RELEASE-p3 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               C300-CTF
Product:              DDAC256MAG    SM
Revision:             4321
Compliance:           SPC-3
Rotation Rate:        Solid State Device
Form Factor:          2.5 inches
Logical Unit id:      0x500605ba00e2a29c
Serial number:        00000000110703034CEB
Device type:          disk
Transport protocol:   SAS (SPL-3)
Local Time is:        Fri Mar 29 18:49:13 2019 MDT
device Test Unit Ready  [aborted command]
A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.
Pasted 6 months, 2 weeks ago — Expires in 169 days
URL: http://dpaste.com/2VSKEGW