IB cabling by generation: DAC, AOC, transceivers, split cables

An InfiniBand cluster is cabled by reach class: copper where the switch is within arm's length, active copper a little further, active optical cables to the next row, transceivers with structured fibre for the rest. Each generation moved the boundaries inward and changed the connector, and NDR added twin-port modules and split cables that confuse first-time installers. This page is the cabling reference — reaches, connectors, split configurations, mechanical and identity rules.

Connectors by generation

Generation	Per-lane rate	4x port	Connector	Notes
SDR / DDR	2.5 / 5 Gb/s	10 / 20 Gb/s	CX4 (copper), later QSFP	legacy
QDR	10 Gb/s	40 Gb/s	QSFP	SFF-8436 management
FDR	14.06 Gb/s	56 Gb/s	QSFP+ (FDR-rated)	64B/66B; FEC optional
EDR	25.78 Gb/s	100 Gb/s	QSFP28	same optics as 100G Ethernet, different coding
HDR	53.1 Gb/s PAM4	200 Gb/s	QSFP56	1:2 splitters to HDR100
NDR	106 Gb/s PAM4	400 Gb/s	OSFP twin-port (2 × 400G per module) on switches; OSFP or QSFP112 on HCAs	800G per switch cage
XDR	212 Gb/s PAM4	800 Gb/s	OSFP / OSFP-XD	1.6T per switch cage

Form-factor details: QSFP28, QSFP56, OSFP, QSFP112.

Reach classes

Medium	QDR	FDR	EDR	HDR	NDR	Notes
Passive DAC	≤ 7 m	≤ 3–5 m	≤ 3 m	≤ 2 m	≤ 1.5–2 m	reach shrinks each generation; thicker gauge for longer runs
Active copper (ACC/AEC)	—	≤ 5 m	≤ 5 m	≤ 3–4 m	≤ 3 m	powered plugs; AEC with retimers for NDR
AOC	≤ 100 m (some 300)	≤ 100 m	≤ 100 m	≤ 100 m	≤ 30–50 m typical	850 nm VCSEL, MMF ribbon inside
Transceiver SR-class (MMF)	≤ 300 m	≤ 100–150 m	≤ 70–100 m	≤ 70–100 m	≤ 50 m (VR) / 100 m (SR)	OM3/OM4, MPO
Transceiver DR/FR-class (SMF)	—	—	500 m – 2 km (PSM4, CWDM4)	500 m – 2 km	500 m (DR4) / 2 km (FR4)	MPO-12 APC (DR4), duplex LC (FR4)

What is inside each cable type: Cable internals.

NDR twin-port and split cables

An NDR switch cage holds one twin-port OSFP module = two independent 400G ports behind two MPO-12/APC (or two LC-duplex) receptacles. Cabling options from one module:

Configuration	Cable	Far end
2 × 400G straight	two MPO-12 APC trunks	two 400G NDR HCA ports (OSFP or QSFP112) or another switch
1:2 split (800G → 2 × 400G)	Y-cable, one OSFP end, two OSFP/QSFP112 ends	two HCAs — the standard server connection
1:4 split (800G → 4 × 200G)	one OSFP end, four QSFP56/QSFP112 ends	HDR-speed HCAs (NDR200)
AOC twin-port	one OSFP twin-port end, two OSFP/QSFP112 ends	up to ~30–50 m
DAC twin-port	one OSFP end, two ends	≤ 1.5–2 m; heavy — in-rack only

Each split leg is a separate link with its own negotiation; the switch sees two (or four) ports. Identity coding must describe the whole assembly consistently at every end (Cable internals).

Finned vs flat OSFP

Finned-top OSFP — integrated heat sink for switch cages with open airflow above the module. Does not fit a flat cage.
Flat-top OSFP — for HCAs and cages with a riding heat sink. Fits either cage, but in an open switch cage it may run hot.
QSFP112 on HCAs shares the NDR electrical interface at 4 × 106 Gb/s in a QSFP footprint; a QSFP112 HCA port pairs with a split cable from an OSFP switch.

Thermal consequences: Power & thermal.

Mechanical rules

Rule	Why
Respect bend radius (DAC ≥ 10× diameter; AOC/fibre ≥ 30 mm)	crushed twinax → lane errors; bent fibre → Rx loss
Support cable weight — twin-port DACs are heavy	pulls on the cage, intermittent contact, cracked paddle cards
Pinned/unpinned and APC on every MPO	Breakout & MPO cabling
Clean and inspect every MPO before insertion	one particle costs a lane 1–3 dB
Keep dust caps on unused cages and connectors	contamination is the top cause of lane failures
Route away from hot exhaust	AOC engines and AEC plugs age with temperature

Identity and vendor policy

InfiniBand switches and HCAs from the dominant vendor validate cables and transceivers against a supported list and read their cable info (vendor, PN, SN, length, technology) into the fabric manager. An unvalidated cable may link at reduced speed or be flagged; a cable coded with the wrong length or rate class may fail negotiation. The identity bytes are the standard SFF-8636/CMIS fields plus the InfiniBand rate declarations (Compliance codes, Vendor lock).

Inventory practice

Label both ends with the link (switch/port ↔ host/port); split cables need per-leg labels.
Record vendor, PN, SN and length per cable in the fabric manager or CMDB; pull the data from mlxcables or from CodingBox at incoming inspection.
Keep spares by reach class and connector; NDR twin-port and split variants are not interchangeable with plain 400G.
Burn in new optics before deployment (Link reliability & monitoring).

In CodingBox

Cable ends and transceivers are read like any module: identity with cable length, technology and rate class, per-lane DDM for AOC and optics, CMIS state for NDR modules (Check transceiver, DDM). Reading every leg of a split cable before installation catches the mismatched end that would otherwise link at the wrong speed.

The XDR-generation modules in detail: OSFP224 and OSFP-XD.

How HCAs and NIC drivers validate these cables, and the tools that show the result from the host: Transceiver compatibility on NICs, NIC tools and diagnostics.