CodingBox Documentation

Forwarding basics: VLANs, LAG, spanning tree, routing

A transceiver problem never announces itself as a transceiver problem. It appears as a spanning-tree topology change, a LAG member suspended, an OSPF neighbour bouncing, a VXLAN tunnel flapping or "half the users are slow". To read those symptoms back to the physical layer you need the forwarding vocabulary: how a switch learns MAC addresses, what VLANs and trunks are, how link aggregation hashes flows, why spanning tree blocks ports and what routing protocols do when a link blinks. This page is that vocabulary, kept to what matters when the root cause may be a dirty connector.

Layer 2: MAC learning and VLANs

ConceptWhat happensOptics angle
MAC learningsource MAC of each frame is recorded against the ingress port; unknown destinations are floodeda flapping port flushes and relearns entries — brief floods after every flap
Ageingentries expire (300 s default)
VLAN (802.1Q)a broadcast domain; frames carry a 12-bit tag (1–4094) on trunks
Access portone untagged VLAN
Trunk portmany tagged VLANs, one native untaggednative-VLAN mismatch across an uplink shows as partial connectivity, not a link fault
MTU / jumbo1 500 bytes default, 9 000–9 216 jumboMTU mismatch across an uplink passes small packets, drops large — looks like intermittent optics
Storm control, BPDU guard, port securityprotective features that shut portsan errdisabled port from any of these looks like a dead module (Symptom index)

Spanning tree

VariantConvergenceNotes
STP (802.1D)30–50 slegacy
RSTP (802.1w)< 1–3 sdefault on most
MSTP (802.1s)as RSTP, per instanceVLAN groups
PVST+/Rapid-PVST+per VLANCisco

Spanning tree blocks redundant L2 paths. What a bad optic does to it:

  • a flapping uplink generates topology change notifications (TCN); each TCN flushes MAC tables network-wide and causes seconds of flooding — users see stalls every flap (Link flapping);
  • a unidirectional link (one fibre broken, Tx works, Rx does not) can create a loop if BPDUs stop arriving — UDLD, loop guard and bridge assurance exist for exactly this (No link);
  • blocked port LEDs (amber) are normal on redundant links, not a fault (Port naming & LEDs).

Leaf–spine fabrics avoid STP by routing every link.

ItemRule
ProtocolLACP (802.1AX / 802.3ad): members exchange PDUs, negotiate membership; static LAG has no protection
Memberssame speed, duplex, MTU; most NOSes require same FEC and media type — 4 × 25G optics or 4 × 25G DAC, not mixed
Hashingflows distributed by hash of MAC/IP/port fields; one flow never exceeds one member's speed
Member with errorsLACP keeps it in the bundle while PDUs pass; CRC errors on one member corrupt the flows hashed to it — "some sessions slow, others fine" is the signature (VDM & FEC metrics)
Min-linksbundle goes down if fewer than N members are up — protects against running on one degraded link
NamesPort-channel, ae, Eth-Trunk, bond, PortChannel per NOS
Across two switchesMLAG/vPC/VLT (Stacking & MLAG)

Practical: when a LAG "works but is slow", check per-member counters and DDM, not the bundle.

Layer 3: routing on the switch

ConceptWhat it isOptics angle
SVI / routed portIP interface on a VLAN or on a physical portrouted uplinks have no STP; a flap is a routing event
Static / OSPF / IS-IS / BGProuting protocols; DC fabrics use BGP (eBGP per link) or OSPFneighbour up/down logs pinpoint which link flapped and when
ECMPequal-cost multipath over N uplinks; flows hashed like LAGone bad uplink degrades 1/N of flows; BFD (50–300 ms) detects failure faster than protocol timers
Dampeningsuppresses a flapping interface from advertisinga flapping optic ends up dampened — link "up" but traffic not routed
VRFseparate routing tables (management VRF!)pinging the switch over the wrong VRF looks like a link fault
First-hop redundancy (HSRP/VRRP)virtual gateway between two switchesflaps cause gateway switchover

Overlays

OverlayWhereOptics angle
VXLAN / EVPNDC fabrics: L2 over L3tunnels ride ECMP; a failing spine link degrades tunnels randomly, not deterministically
MPLS / SRcarrierfast reroute (< 50 ms) masks flaps — check the underlay counters
EVPN multihoming (ESI-LAG)replaces MLAGmember links to different leaves, same rules as LAG members

Flow control and lossless classes

MechanismPurposeOptics angle
802.3x pausestop the neighbourrarely used; global pause hides congestion
PFC (802.1Qbb)per-priority pause for RoCE/FCoEa link with CRC errors triggers retransmits, not pause — but pause storms from a flapping port can stall a lossless class fabric-wide (Lossless Ethernet)
ECNmark congestion

Reading forwarding symptoms back to the physical layer

Forwarding symptomPhysical cause to check
STP TCNs every few minutes, MAC flaps in logsone uplink flapping — DDM Rx power near sensitivity, dirty connector
LACP member "suspended" / "individual"FEC or speed mismatch on that member; module not recognised
OSPF/BGP neighbour resets on one linkflapping optic, or module at temperature limit
ECMP: a subset of flows slow or lossyone uplink with FEC uncorrectables / CRC
Jumbo frames fail, pings workMTU — not optics
Loop or broadcast storm after a fibre repairunidirectional link, polarity swap on MPO, missing UDLD
Traffic stops after a software upgradethird-party optics rejected on the new release (Compatibility matrices & firmware)
Server sees link up, no trafficwrong VLAN/native mismatch, or LAG on switch and none on server

The physical checks behind every row: Symptom index, Verifying optics on a switch.

In CodingBox

Forwarding-layer symptoms point at a port; the port points at a module; the module's DDM, identity and thresholds — read on the switch or on the bench — close the case. Keeping a per-module record in the code database turns "the LAG was slow again" into "member 3, serial X, Rx −17 dBm, replaced".