Dynamic Tracking of Fibre Channel devices in AIX - PID change in Fibre Channel Fabric

Port identifiers (called PIDs) are used by the routing and zoning services in Fibre Channel fabrics to
identify ports in the network. All devices in a fabric must use the same PID format, so when you add
new equipment to the SAN, you might need to change the PID format on legacy equipment.

Many scenarios cause a device to receive a new PID; for example, unplugging the device from one
port and plugging it into a different port as part of fabric maintenance, or changing the domain ID
of a switch, which might be necessary when merging fabrics, or changing compatibility mode
settings.


AIX® 6.1 and higher supports dynamic tracking of Fibre Channel (FC) devices.

Previous releases of AIX required a user to unconfigure FC storage device and adapter device instances before making changes on the system area network (SAN) that might result in an N_Port ID (SCSI ID) change of any remote storage ports.

If dynamic tracking of FC devices is enabled, the FC adapter driver detects when the Fibre Channel N_Port ID of a device changes. The FC adapter driver then reroutes traffic destined for that device to the new address while the devices are still online. Events that can cause an N_Port ID to change include moving a cable between a switch and storage device from one switch port to another, connecting two separate switches using an inter-switch link (ISL), and possibly rebooting a switch.

Dynamic tracking of FC devices is controlled by a new fscsi device attribute, dyntrk. The default setting for this attribute is no. To enable dynamic tracking of FC devices, set this attribute to dyntrk=yes, as shown in the example:

chdev -l fscsi0 -a dyntrk=yes  

In this example, the fscsi device instance is fscsi0. Dynamic tracking logic is called when the adapter driver receives an indication from the switch that there has been a link event involving a remote storage device port.

Dynamic tracking support requires the following:

• A switched environment. It is not supported in arbitrated loop environments, including public loop.
• FC 6227 adapter firmware, level 3.22A1 or higher.
• FC 6228 adapter firmware, level 3.82A1 or higher.
• FC 6239 adapter firmware, all levels.
• All subsequent FC adapter releases support Fast I/O Failure.
• The World Wide Name (Port Name) and Node Names devices must remain constant, and the World Wide Name device must be unique. Changing the World Wide Name or Node Name of an available or online device can result in I/O failures. In addition, each FC storage device instance must have world_wide_name and node_name attributes. Updated filesets that contain the sn_location attribute (see the following bullet) must also be updated to contain both of these attributes.
• The storage device must provide a reliable method to extract a unique serial number for each LUN. The AIX FC device drivers do not autodetect serial number location, so the method for serial number extraction must be explicitly provided by any storage vendor in order to support dynamic tracking for their devices. This information is conveyed to the drivers using the sn_location ODM attribute for each storage device. If the disk or tape driver detects that the sn_location ODM attribute is missing, an error log of type INFO is generated and dynamic tracking is not enabled.
  Note: The sn_location attribute might not be displayed, so running the lsattr command on an hdisk, for example, might not show the attribute even though it could be present in ODM.
• The FC device drivers can track devices on a SAN fabric, which is a fabric as seen from a single host bus adapter, if the N_Port IDs on the fabric stabilize within about 15 seconds. If cables are not reseated or N_Port IDs continue to change after the initial 15 seconds, I/O failures could result.
• Devices are not tracked across host bus adapters. Devices only track if they remain visible from the same HBA that they were originally connected to.
  For example, if device A is moved from one location to another on fabric A attached to host bus adapter A (in other words, its N_Port on fabric A changes), the device is seamlessly tracked without any user intervention, and I/O to this device can continue.
  However, if a device A is visible from HBA A but not from HBA B, and device A is moved from the fabric attached to HBA A to the fabric attached to HBA B, device A is not accessible on fabric A nor on fabric B. User intervention would be required to make it available on fabric B by running the cfgmgr command. The AIX device instance on fabric A would no longer be usable, and a new device instance on fabric B would be created. This device would have to be added manually to volume groups, multipath device instances, and so on. This is essentially the same as removing a device from fabric A and adding a new device to fabric B.
• No dynamic tracking can be performed for FC dump devices while an AIX system dump is in progress. In addition, dynamic tracking is not supported when booting or running the cfgmgr command. SAN changes should not be made while any of these operations are in progress.
• After devices are tracked, ODM might contain stale information because Small Computer System Interface (SCSI) IDs in ODM no longer reflect actual SCSI IDs on the SAN. ODM remains in this state until cfgmgr is run manually or the system is rebooted (provided all drivers, including any third party FC SCSI target drivers, are dynamic-tracking capable). If cfgmgr is run manually, cfgmgr must be run on all affected fscsi devices. This can be accomplished by running cfgmgr without any options, or by running cfgmgr on each fscsi device individually.
  Note: Running cfgmgr at run time to recalibrate the SCSI IDs might not update the SCSI ID in ODM for a storage device if the storage device is currently opened, such as when volume groups are varied on. The cfgmgr command must be run on devices that are not opened or the system must be rebooted to recalibrate the SCSI IDs. Stale SCSI IDs in ODM have no adverse affect on the FC drivers, and recalibration of SCSI IDs in ODM is not necessary for the FC drivers to function properly. Any applications that communicate with the adapter driver directly using ioctl calls and that use the SCSI ID values from ODM, however, must be updated (see the next bullet) to avoid using potentially stale SCSI IDs.
• All applications and kernel extensions that communicate with the FC adapter driver, either through ioctl calls or directly to the FC driver's entry points, must support the version 1 ioctl and scsi_buf APIs of the FC adapter driver to work properly with FC dynamic tracking. Noncompliant applications or kernel extensions might not function properly or might even fail after a dynamic tracking event. If the FC adapter driver detects an application or kernel extension that is not adhering to the new version 1 ioctl and scsi_buf API, an error log of type INFO is generated and dynamic tracking might not be enabled for the device that this application or kernel extension is trying to communicate with.
  For ISVs developing kernel extensions or applications that communicate with the AIX Fibre Channel Driver stack, refer to the Required FCP, iSCSI, and Virtual SCSI Client Adapter Device Driver ioctl Commands and Understanding the scsi_buf Structure for changes necessary to support dynamic tracking.
• Even with dynamic tracking enabled, users should make SAN changes, such as moving or swapping cables and establishing ISL links, during maintenance windows. Making SAN changes during full production runs is discouraged because the interval of time to perform any SAN changes is too short. Cables that are not reseated correctly, for example, could result in I/O failures. Performing these operations during periods of little or no traffic minimizes the impact of I/O failures.

The base AIX FC SCSI Disk and FC SCSI Tape and FastT device drivers support dynamic tracking. The IBM® ESS, EMC Symmetrix, and HDS storage devices support dynamic tracking provided that the vendor provides the ODM filesets with the necessary sn_location and node_name attributes. Contact the storage vendor if you are not sure if your current level of ODM fileset supports dynamic tracking.

If vendor-specific ODM entries are not being used for the storage device, but the ESS, Symmetrix, or HDS storage subsystem is configured with the MPIO Other FC SCSI Disk message, dynamic tracking is supported for the devices in this configuration. This supersedes the need for the sn_location attribute. All current AIX Path Control Modules (PCM) shipped with the AIX base support dynamic tracking.

The STK tape device using the standard AIX device driver also supports dynamic tracking provided the STK fileset contains the necessary sn_location and node_name attributes.

Note: SAN changes involving tape devices should be made with no active I/O. Because of the serial nature of tape devices, a single I/O failure can cause an application to fail, including tape backups.

Devices configured with the Other FC SCSI Disk or Other FC SCSI Tape messages do not support dynamic tracking.

Source:
IBM:
http://publib.boulder.ibm.com/infocenter/aix/v6r1/index.jsp?topic=/com.ibm.aix.prftungd/doc/prftungd/dynamic_tracking.htm

Brocade:
Fabric OS - Administrator’s Guide