CodingBox Documentation

FC port diagnostics: counters, slow drain, SFP checks

A SAN switch counts everything that goes wrong on a port, and the counters map cleanly onto physical causes — if you know which counter means "dirty connector" and which means "the host is not draining its buffers". This page lists the counters on the two dominant platforms, what each one points to, how slow-drain congestion shows up, and how to read the SFP itself from the switch.

The counters

Counter (Brocade porterrshow)Cisco MDS equivalent (show interface fc… counters)MeaningPoints to
enc_out — encoding errors outside framesinvalid transmission wordsbad symbols between framesoptics, cable, connector, speed mismatch — the classic physical-layer counter
enc_in — encoding errors inside framesinvalid transmission words / CRCcorrupted symbols inside framessame causes, more severe
crc_errCRC errorsframe failed CRCmarginal link: dirty end-face, low Rx, failing SFP; on ISLs check both ends
crc_g_eofCRC with good EOFCRC error but frame was terminated correctly — introduced upstream of this portlook at the previous hop
too_shrt / too_long / bad_eofframe too short / too long / bad EOFmalformed framesusually a consequence of enc_in, or a faulty device
link_faillink failureslink went downcable pull, SFP fault, power, far-end reset
loss_sync / loss_sigsync loss / signal losssignal or word sync lostlow Rx, LOS, flapping — Link flapping
frjt / fbsyF_RJT / F_BSYfabric rejected/busied framesfabric or zoning, not optics
disc_c3 — class 3 discardstimeout discardsframes dropped after timing out in the switchslow drain / congestion
c3timeout tx/rxdirection of the timeoutstx timeouts: the attached device is slow; rx: an upstream problem
pcs_err (16G+)64B/66B PCS block errorsphysical layer at 16/32GFC
uncor_err (16G+ with FEC)FEC uncorrectedFEC could not fix the blocklink at the margin — VDM & FEC metrics
credit loss (portstatsshow: tim_txcrd_z)credit loss / tx credit not availabletime spent with zero transmit creditscongestion or lost R_RDYs on a dirty link

Rule of thumb: enc_out, crc_err, loss_sync, pcs_err grow with a physical problem; disc_c3, c3timeout, tim_txcrd_z grow with a congestion problem; frjt/fbsy with a fabric problem. Clear the counters, wait, and look at rates, not totals.

Slow drain

An end device that returns credits slowly (an overloaded host, a failing HBA, a mismatch of speeds along a path) holds frames in switch buffers; those frames time out and are discarded, and the congestion spreads backwards through ISLs to unrelated devices. It is the most damaging SAN problem and it looks, at first, like a random performance issue.

SignWhere
tim_txcrd_z climbing on an F_Portthe device on that port is slow to return credits
disc_c3 / c3timeout tx on the same portframes to that device are timing out
disc_c3 on ISLs and other F_Portscongestion has spread
Bottleneck/MAPS "latency" alerts (Brocade), show logging onboard flow-control request-timeout, congestion-drop / slow-drain detection (Cisco)platform tooling

Mitigation: fix or isolate the slow device (port fencing, quarantine to a low-priority virtual channel), shorten congestion-drop timeouts on edge ports, avoid speed steps along a path, keep ISL oversubscription sane (SAN design). Optics are almost never the cause of slow drain — but a marginal link with lost R_RDYs can mimic it, so check enc_out/crc first.

SFP checks from the switch

TaskBrocade FOSCisco MDS / NX-OS
SFP identity and DDMsfpshow <port>show interface fc1/1 transceiver details
Port state and speedportshow <port>, switchshowshow interface fc1/1, show interface brief
Error countersporterrshow, portstatsshow <port>show interface fc1/1 counters [detailed]
Clear countersportstatsclear / statsclearclear counters interface fc1/1
Link testportloopbacktest, D_Port diagnostics (portcfgdport, portdporttest)show interface fc1/1 transceiver details + loopback via diagnostic tools
Health policyMAPS (thresholds on CRC, ITW, credit loss, SFP power/temperature)port-monitor policies (RX/TX power, CRC, ITW, credit loss)

sfpshow and the transceiver details show vendor, part number, serial, speeds, and DDM values with thresholds — the same bytes CodingBox reads on the bench. Both platforms flag unsupported optics; a port stuck in Mod_Inv, No_Module or "unsupported transceiver" is a policy rejection, not a fault (Vendor lock, FC optics).

Diagnostic sequence for a bad FC port

  1. porterrshow / counters: physical (enc_out, crc, sync) vs congestion (disc_c3, credits) vs fabric (rjt/bsy).
  2. SFP DDM: Rx power on both ends against class windows (Typical values); Tx bias trend for ageing.
  3. Clean and inspect connectors; swap the patch cord; recheck counters at rate.
  4. Speed: force the negotiated speed to the lower generation to see if errors stop (a marginal 32G link may be clean at 16G) — Speed & rate.
  5. D_Port / loopback test to separate switch, SFP and cable.
  6. If congestion: find the slow device by tim_txcrd_z, not by the port that complains.

In CodingBox

A suspect FC SFP pulled from a switch can be read on the bench: identity, FC speed/media codes, checksums and live DDM on Check transceiver and DDM. A module that is fine on the bench and errors in the port shifts the suspicion to the cable, the connector or the far end.