Nexenta / Solaris / Comstar / illumos one port fiber channel target and one initiator
Post date: Jan 6, 2014 5:00:28 AM
So you have a 2 or 4 port HBA or multiple Q-Logic HBAs in the same host and you want to make some ports targets and some ports initiators.. Fear not, it is relatively simple.
In my example, I am using Q-Logic cards which use either the qlc driver (default) for initiator ports, or the qlt driver for the target ports.
To make an FC initiator port into a target port, we simply change the driver. In order to change which driver for a specific port, instead of changing the driver for all of a specific type of hardware, we first must know where the card is on host. We will call this the identify-name...
We can use the modular debugger to list the current devices claimed by the driver and map it back to the node ID.
root@vnexenta:/volumes# mdb -k
Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc pcplusmp scsi_vhci zfs mpt sd sockfs ip hook neti sctp arp usba qlc stmf_sbd stmf lofs idm cpc random crypto smbsrv nfs logindmux ptm nsctl sdbc ufs nsmb sv rdc ii ]
> ::devbindings -q qlc
ffffff018abd7040 pciex1077,2432, instance #0 (driver name: qlc)
ffffff018abd4cf0 pciex1077,2432, instance #1 (driver name: qlc)
> $q
Okay so we see that QLC has claimed two ports. Now if these are currently FC initiator ports, you can use luxadm to quickly hint to us the identify-name / node path. (If they are not FC initiator ports, read to the bottom)
root@vnexenta:/volumes# luxadm -e port
/devices/pci@0,0/pci15ad,7a0@15/pci1077,138@0/fp@0,0:devctl NOT CONNECTED
/devices/pci@0,0/pci15ad,7a0@15/pci1077,138@0,1/fp@0,0:devctl NOT CONNECTED
Now we can update one of those ports to use the qlt driver as follows using the update_drv command and the devices identify-name, and reboot.
root@vnexenta:/volumes# update_drv -i '/pci@0,0/pci15ad,7a0@15/pci1077,138@0,1' -a qlt
root@vnexenta:/volumes# init 6
So what does update_drv actually do and how does the change persist across reboots?!? It attempts to do its job to the running kernel, then it updates /etc/driver_aliases. In this case it simply appends to the bottom:
qlt "/pci@0,0/pci15ad,7a0@15/pci1077,138@0,1"
Once the host boots back up, we can check who claimed the drivers...
root@vnexenta:/volumes# mdb -k
Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc pcplusmp scsi_vhci zfs mpt sd sockfs ip hook neti sctp arp usba qlc stmf_sbd stmf lofs idm cpc random crypto smbsrv nfs logindmux ptm nsctl sdbc ufs nsmb sv rdc ii ]
> ::devbindings -q qlt
ffffff018abd4cf0 /pci@0,0/pci15ad,7a0@15/pci1077,138@0,1, instance #0 (driver name: qlt)
> ::devbindings -q qlc
ffffff018abd7040 pciex1077,2432, instance #0 (driver name: qlc)
>
And there you have it!
Okay so what if you don't have luxadm or luxadm doesn't show your hardware?? Well fear not, all the info you need to compose the identify name is available in mdb... get the address of your device from loaded driver, figure out how it is connected, then walk the tree for node names that make up the entirety of identify name. Note that the root node in the identify name is assumed and does not need to be looked up. Since we want to replace qlc, the identify-name is a path of the node name from after the root to qlc separated by / .
root@vnexenta:/volumes# mdb -k
Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc pcplusmp scsi_vhci zfs mpt
sd sockfs ip hook neti sctp arp usba qlc stmf_sbd stmf lofs idm cpc random crypto smbsrv nfs
logindmux ptm nsctl sdbc ufs nsmb sv rdc ii ]
> ::devbindings -q qlc
ffffff018abd7040 pciex1077,2432, instance #0 (driver name: qlc)
> ffffff018abd7040::prtconf
ffffff018ab78cb0 i86pc (driver name: rootnex)
ffffff018ab787a0 pciex_root_complex, instance #0 (driver name: npe)
ffffff018ab6b7b0 pciexclass,060400, instance #0 (driver name: pcieb)
ffffff018abd7040 pciex1077,2432, instance #0 (driver name: qlc)
ffffff018ab78a28 fp, instance #0 (driver name: fp)
> ffffff018abd7040$<devinfo_brief
DEVINFO MAJ REFCNT NODENAME NODESTATE
INST CIRCULAR BINDNAME STATE
ffffff018abd7040 224 3 pci1077,138@0 DS_READY
0 0 pciex1077,2432 <S_EVADD,S_NEED_RESET>
0 <MADE_CHILDREN>
> ffffff018ab6b7b0$<devinfo_brief
DEVINFO MAJ REFCNT NODENAME NODESTATE
INST CIRCULAR BINDNAME STATE
ffffff018ab6b7b0 184 5 pci15ad,7a0@15 DS_READY
0 0 pciexclass,060400 <S_EVADD,S_NEED_RESET>
0 <MADE_CHILDREN>
> ffffff018ab787a0$<devinfo_brief
DEVINFO MAJ REFCNT NODENAME NODESTATE
INST CIRCULAR BINDNAME STATE
ffffff018ab787a0 183 42 pci@0,0 DS_READY
0 0 pciex_root_complex <S_EVADD,S_NEED_RESET>
0 <MADE_CHILDREN>
So in this example, the identify name for update_drv is /pci@0,0/pci15ad,7a0@15/pci1077,138@0
This information can also be handy when looking to replace other drivers in Solaris derived environments.
DEVINFO NAME
THREAD FLAGS
THREAD FLAGS
THREAD FLAGS
>$q