Ethernet & Switching
From a shared coaxial wire in 1973 to dedicated 800 Gbps ports — how Ethernet became the universal LAN standard and how switches replaced hubs, eliminated collisions, and scale to millions of ports.
// CHAPTER 01
The Wire That Became the World
Ethernet's 50-year journey from a shared coax to a global standard
In 1973, Robert Metcalfe and David Boggs at Xerox PARC connected computers to a single thick coaxial cable and called the protocol Ethernet — named after the luminiferous ether that 19th century physicists believed carried light through space. The original speed was 2.94 Mbps. Today, 800 Gbps Ethernet links carry data inside the world's largest datacenters. The frame format defined in those first experiments is still recognizable in every packet on your network right now.
Ethernet's longevity is the result of a remarkable design philosophy: a minimal, extensible standard that separates the physical layer from the logical framing. When the physics improved (coax → twisted pair → fiber → 400G optical), the frame format stayed the same. When speeds increased by a factor of 270,000, the switching logic stayed the same. This is what a well-designed protocol looks like.
// REAL-WORLD SCENARIO
IEEE 802.3 — The Standard That Formalized Ethernet
In 1980, Digital Equipment Corporation (DEC), Intel, and Xerox published the "DIX Ethernet" specification (named for their initials). In 1983, IEEE formalized it as IEEE 802.3. The key difference: IEEE 802.3 replaced the EtherType field with a length field, creating two subtly incompatible frame formats. Modern Ethernet uses values ≥ 0x0600 (1536) as EtherType and values ≤ 1500 as length — a backward-compatible coexistence.
Today, IEEE 802.3 is one of the most active standards in IEEE. New amendments are published constantly: 802.3bz (2.5G/5GBASE-T), 802.3cd (50G/100G/200G), 802.3ck (100G/200G/400G), 802.3cu (100G/400G over SMF), 802.3db (100G/400G/800G). Each new generation preserves the same frame format while redefining only the physical layer encoding.
// CHAPTER 02
The Ethernet Frame — A 50-Year Standard
Every field, every byte, every edge case
The Ethernet frame is the Protocol Data Unit (PDU) at Layer 2. Every frame has the same structure — from a 64-byte ARP request to a 9000-byte jumbo frame containing a TLS record. Understanding every field is not academic: network engineers read frames in Wireshark, write code that parses them, and configure hardware that processes them at line rate.
Preamble and Start Frame Delimiter (SFD)
Before the actual frame, 8 bytes are prepended by the NIC's hardware: 7 bytes of preamble (alternating 1s and 0s: 10101010 repeated) followed by 1 byte SFD (10101011 — the two consecutive 1s signal "frame starting now"). The preamble's purpose is clock synchronization: the receiver's clock recovery circuit locks to the alternating pattern, synchronizing its sampling with the transmitter. The SFD is stripped before the frame is passed up the stack — it never appears in Wireshark captures.
Destination and Source MAC Addresses (6 bytes each)
MAC (Media Access Control) addresses are 48-bit (6-byte) globally unique hardware identifiers assigned to every NIC at manufacture. Format: 6 hex bytes separated by colons or hyphens (AA:BB:CC:DD:EE:FF). The first 3 bytes are the OUI (Organizationally Unique Identifier) — assigned by IEEE to manufacturers. The last 3 bytes are assigned by the manufacturer to individual devices.
First byte bit 0 (LSB): multicast bit. If 1, the address is a multicast/broadcast address. FF:FF:FF:FF:FF:FF is the L2 broadcast (all bits 1). 01:00:5E:xx:xx:xx is IPv4 multicast (first 25 bits fixed). 33:33:xx:xx:xx:xx is IPv6 multicast.
First byte bit 1: locally administered bit. If 1, the address was assigned locally (not from IEEE registry). VMs often use locally administered MACs. x2, x6, xA, xE in the second hex digit = locally administered.
EtherType / Length Field (2 bytes)
Values ≥ 0x0600 (1536) identify the encapsulated Layer 3 protocol (EtherType). Values ≤ 1500 represent the frame payload length (IEEE 802.3 format). Common EtherType values:
802.1Q VLAN Tag (optional 4 bytes)
When a frame is VLAN-tagged, a 4-byte 802.1Q tag is inserted after the Source MAC: 2 bytes TPID (0x8100) + 2 bytes TCI. The TCI contains: 3-bit PCP (Priority Code Point, values 0–7 for QoS), 1-bit DEI (Drop Eligible Indicator, formerly CFI), and 12-bit VLAN ID (0–4095, but 0 and 4095 reserved, so 1–4094 usable). This optionally increases the minimum frame size to 68 bytes and maximum to 1522 bytes for tagged frames.
Payload (46–1500 bytes)
The payload carries the upper-layer PDU (IPv4 packet, ARP message, IPv6 packet). Minimum payload: 46 bytes. If the actual data is shorter (e.g., a 28-byte ARP message), the NIC pads with zeros to reach 46 bytes. This ensures minimum frame size of 64 bytes — required for CSMA/CD collision detection (a transmitter must still be transmitting when a collision's jam signal arrives from the far end of a 500m 10BASE-5 segment).
Maximum payload: 1500 bytes (standard MTU). This is the MTU (Maximum Transmission Unit) of Ethernet. IP packets larger than 1500 bytes must be fragmented at Layer 3. Jumbo frames extend the payload to 9000 bytes for datacenter use — requires configuration on all devices end-to-end (switch ports must support jumbo MTU, NIC drivers must be configured).
FCS — Frame Check Sequence (4 bytes)
The 4-byte CRC-32 computed over the frame content (Destination MAC through end of payload). The transmitter computes it and appends it. The receiver recomputes the CRC and compares. A mismatch means the frame was corrupted and it is silently dropped — no error is returned to the sender. This is a silent discard. Upper layers (TCP) detect the missing data via retransmission timeout and retransmit. UDP applications must handle loss themselves.
// CHAPTER 03
Hubs vs Switches — The Paradigm Shift
Why the collision domain was the enemy
Before switches became affordable (~1995), networks used hubs. A hub is a physical-layer repeater: it receives a signal on one port and repeats it out all other ports. Every device connected to a hub shares a single collision domain — they must take turns, just like the original coaxial Ethernet. Switches changed this completely.
Why Hubs Were Abandoned
The Ethernet CSMA/CD protocol was designed for shared media. It works reasonably well at low utilization (under 40% of link capacity). Above 40%, collision rates grow non-linearly — the probability of collision increases with each additional transmitter. At 70% utilization, collision backoffs dominate, and effective throughput can actually decrease with higher offered load. A 100-device hub network is essentially unusable for any significant traffic.
Switches solve this by giving each device its own dedicated path. Each switch port is its own collision domain — collision detection only applies within that port. A modern full-duplex switch port has zero collisions: the switch uses separate transmit and receive paths (or virtual full-duplex with CSMA/CD disabled), and each port's buffer absorbs any timing conflicts.
// CHAPTER 04
The CAM Table — How Switches Learn and Forward
The data structure that eliminates flooding
A switch's forwarding intelligence lives in the CAM table (Content Addressable Memory table), also called the MAC address table. The CAM table maps each learned MAC address to the switch port it was last seen on, enabling unicast forwarding — sending frames only to the port where the destination device resides.
The Three Switch Behaviors
Every frame a switch processes triggers exactly one of three actions:
1. Learn: When a frame arrives, the switch reads the source MAC and records (source MAC, ingress port, timestamp) in the CAM table. This happens on every frame regardless of what action is taken for forwarding. The switch is always learning.
2. Forward (unicast): If the destination MAC is in the CAM table, send the frame only to the associated port. No other ports see the frame.
3. Flood: If the destination MAC is NOT in the CAM table (unknown unicast), or if the destination is a broadcast (FF:FF:FF:FF:FF:FF), or if it is a multicast address without multicast snooping configured — the frame is flooded to all ports in the same VLAN except the ingress port. Flooding is the fallback, not the normal case.
CAM Table Aging
CAM table entries have a timer (default: 300 seconds on most switches). If no frame is received from a MAC address for 300 seconds, the entry is removed. This handles: devices that have been powered off, devices that have moved to a different port, and VMs that have migrated. When an entry ages out, the next frame to that MAC is flooded (unknown unicast) until the device transmits again and is re-learned.
CAM table overflow attack (MAC flooding): An attacker sends frames with thousands of random source MACs, filling the CAM table. When full, new entries cannot be added — all traffic (including known unicasts) is flooded to all ports. The attacker's port now receives all traffic on the VLAN — effectively performing a passive wiretap. Defense: port security (maximum MAC addresses per port), 802.1X port authentication.
CAM Table Size Limits
CAM tables are implemented in TCAM (Ternary Content Addressable Memory) — extremely fast but very expensive silicon. Typical sizes: 8,000–16,000 entries on access layer switches, 64,000–256,000 on core/datacenter switches. Large campus networks can exhaust access switch CAM tables if too many devices are on the same VLAN — another reason to segment with VLANs.
// CHAPTER 05
CSMA/CD — The Collision Resolution Protocol
The algorithm that made shared Ethernet work
CSMA/CD (Carrier Sense Multiple Access with Collision Detection) is the Layer 2 access control protocol that governed all pre-switch Ethernet. Understanding it matters even today because: it explains why minimum frame size is 64 bytes, it is still relevant for half-duplex links (legacy equipment, some DOCSIS upstream channels), and understanding it is required for networking certifications and interviews.
The CSMA/CD Algorithm
Why 64 Bytes? The Slot Time Relationship
The minimum frame size of 64 bytes directly results from CSMA/CD physics. Consider: a device at one end of a maximum-length 10BASE-5 segment (500 m) starts transmitting. A device at the other end starts transmitting 0.1 µs before the first device's signal arrives (just missed the carrier sense). A collision occurs near the far end. The jam signal must propagate back to the first device — total worst-case round-trip propagation: ~51.2 µs for 10 Mbps. At 10 Mbps, 51.2 µs × 10 Mbps = 512 bits = 64 bytes. If the frame is shorter than 64 bytes, the first device might finish transmitting before the collision signal returns — it would never know the frame was destroyed.
Full-duplex Ethernet disables CSMA/CD entirely because there is no shared medium — the switch port and NIC have dedicated TX and RX pairs. The minimum frame size of 64 bytes remains to maintain backward compatibility with legacy frame parsing code, not for collision detection.
IFG — Inter-Frame Gap
Between consecutive frames, the standard requires a mandatory idle period: 9.6 µs at 10 Mbps, 0.96 µs at 100 Mbps, 0.096 µs at 1 Gbps. The IFG allows receiving NICs to process the previous frame (update CRC check, move data to buffer) before the next frame arrives. At Gigabit speeds and above, hardware pipelining handles frames arriving nearly back-to-back — the IFG is 96 nanoseconds (12 bytes at 1 Gbps).
// CHAPTER 06
Ethernet Evolution — 10 Mbps to 800 Gbps
Five decades of speed improvements on the same frame format
Each generation of Ethernet redefined the physical layer while preserving the same frame structure. The naming convention tells you everything: speed + BASE + medium code. "BASE" means baseband (the full bandwidth is used for one signal — not divided into frequency bands like cable TV). The medium code indicates the cable type or distance.
Auto-Negotiation (IEEE 802.3u)
Introduced with Fast Ethernet (100BASE-TX), auto-negotiation lets two devices automatically agree on the highest common speed and duplex mode. During link setup, devices exchange Fast Link Pulses (FLPs) — a burst of 33 pulses encoded as a 16-bit data word advertising capabilities: 10HD, 10FD, 100HD, 100FD, 1000FD, pause frames, asymmetric pause. Both sides select the highest common capability.
1000BASE-T requires auto-negotiation — there is no forced-gigabit mode. The 4D-PAM5 coding used by 1000BASE-T requires the master/slave relationship negotiated during auto-neg to synchronize the echo cancellation coefficients. Forcing to 1000/full without auto-neg simply doesn't work (the link won't come up).
Jumbo Frames
Standard Ethernet MTU is 1500 bytes. Jumbo frames extend this to 9000 bytes (sometimes 9216 bytes to accommodate VXLAN/MPLS encapsulation overhead). Benefits: fewer frames per data transfer → less CPU overhead per byte → higher throughput for large transfers (storage, backup, bulk data movement). The 9000-byte size reduces CPU interrupts by 6× compared to standard MTU for the same data. Jumbo frames must be configured consistently: NIC, switch port, router interface, and destination NIC must all support the same MTU, or fragmentation/drops occur.
// CHAPTER 07
Switch Architecture — How Hardware Makes It Fast
Store-and-forward, cut-through, TCAM, and switching fabric
A modern enterprise switch forwards millions of frames per second while simultaneously learning MACs, enforcing ACLs, applying QoS, and updating counters. This performance is possible only because of dedicated hardware — custom ASICs purpose-built for switching.
Forwarding Modes
Store-and-forward: The switch receives the entire frame, verifies the CRC, then forwards it. Latency = frame size / link rate (for a 1500-byte frame on 1 Gbps: 12 µs). Advantage: no error frames propagate (a corrupted frame is discarded before forwarding). This is the standard mode for all production switches and is required when input and output ports run at different speeds (rate matching requires buffering the entire frame anyway).
Cut-through: The switch begins forwarding as soon as it reads the destination MAC (after the first 14 bytes). Latency: ~1 µs (just the header read time). Disadvantage: error frames propagate — a frame with a bad CRC is already halfway forwarded before the FCS is even received. Most modern high-end datacenter switches (Broadcom Tomahawk ASICs) support cut-through mode on same-speed port pairs as an optional performance optimization.
Fragment-free (modified cut-through): Waits for the first 64 bytes before forwarding. This filters collision fragments (which are always <64 bytes in CSMA/CD networks) while keeping latency lower than store-and-forward. Used in older switches and rarely today.
TCAM — Ternary Content Addressable Memory
The CAM table uses TCAM hardware. Unlike standard RAM (look up a value by address), TCAM lets you supply a value and find the address in a single clock cycle — a hardware parallel search of all entries simultaneously. Each TCAM cell stores ternary values: 0, 1, or X (don't care). This enables: exact MAC address lookup (used for CAM/forwarding table), prefix matching (for IP routing tables with masks), ACL evaluation (match packets with specific source IP ranges and port ranges).
TCAM is extremely expensive — each bit of TCAM requires 4 transistors vs 1 for SRAM and 1 for DRAM. This is why switch CAM tables have hard limits (8K–256K entries) and why expanding routing table capacity requires buying a higher-end switch. TCAM can't be upgraded after purchase.
Switching Fabric and Port ASICs
A switch's switching fabric is the internal high-speed crossbar that connects all port ASICs. Each port ASIC handles reception, transmission, and per-port logic for a group of ports. The switching fabric must have enough bandwidth to allow all ports to transmit simultaneously without blocking — this is a non-blocking switch. A 48-port 1G switch requires a 96 Gbps switching fabric (48 ports × 2 directions × 1 Gbps). Budget switches may have a fabric that is oversubscribed — if all ports transmit simultaneously, some must wait. Datacenter switches are non-blocking; access closet switches may be 4:1 oversubscribed (acceptable because not all ports are simultaneously saturated).
// CHAPTER 08
Broadcast Domains and the Limits of Layer 2
Why Layer 2 alone cannot scale
A switch forwards broadcasts to every port in the same VLAN. This defines a broadcast domain — the set of all devices that receive each other's broadcast frames. Every ARP request, every DHCP discover, every spanning tree BPDU is flooded to every device in the broadcast domain.
As broadcast domains grow, so does broadcast overhead. Consider a /16 subnet with 65,000 devices: every ARP request reaches every device. Every DHCP discover reaches every device. At a certain scale, broadcast traffic alone consumes significant bandwidth and CPU on every host. The rule of thumb: keep broadcast domains under 500 devices; 250 is safer for networks with chatty protocols. VLANs provide the segmentation — each VLAN is its own broadcast domain.
Unknown Unicast Flooding
Beyond broadcasts, unknown unicast flooding is a silent performance problem. Every frame destined for a MAC not in the CAM table is flooded to all ports. In a network with many devices or high turnover (cloud VMs spawning and dying), a significant percentage of traffic can be unknown unicast floods. Symptoms: unexpectedly high traffic on ports that shouldn't be seeing that traffic; CPU spikes on devices caused by processing discarded frames.
Layer 3 as the Solution
Routers (Layer 3 devices) do not forward Layer 2 broadcasts — they are broadcast domain boundaries. When a router receives a broadcast frame, it processes it locally and never forwards it to other interfaces. This is why large networks use VLANs (each VLAN is a broadcast domain) with a router or Layer 3 switch providing inter-VLAN routing. The design principle: use Layer 2 within a broadcast domain, use Layer 3 to connect broadcast domains.
// CHAPTER 09
Port Security and MAC Address Management
Controlling which devices can connect
Switches can restrict which MAC addresses are allowed on each port, preventing unauthorized devices from connecting or limiting the impact of MAC flooding attacks.
Port Security
Cisco's port security feature limits the number of MAC addresses learned on a port. When the limit is reached:
Sticky MAC: The switch dynamically learns the first N MAC addresses and saves them to the running config as static secure MAC addresses. On reboot, these addresses are restored — no need to manually configure each MAC. Useful for locking a port to the device currently plugged in.
802.1X Port-Based Network Access Control
Port security with MAC addresses is easy to bypass (just spoof the allowed MAC). IEEE 802.1X is the proper solution: authenticate the user/device before granting network access. The switch port acts as an Authenticator — it blocks all traffic except EAP (Extensible Authentication Protocol) exchanges until the connecting device (Supplicant) authenticates with the RADIUS server (Authentication Server). Once authenticated, the switch places the port in the correct VLAN and grants access.
// CHAPTER 10
Link Aggregation — LACP and EtherChannel
Bonding multiple physical links into one logical link
A single Ethernet link provides limited bandwidth and no redundancy. Link Aggregation (LAG) bonds multiple physical links between two devices into a single logical interface, providing both bandwidth multiplication and link redundancy.
LACP — IEEE 802.3ad / 802.1AX
LACP (Link Aggregation Control Protocol) is the IEEE standard for dynamic LAG negotiation. Both ends exchange LACP PDUs (LACPDUs) advertising their system ID, port priorities, and state. Compatible ports that agree on parameters form an aggregation group automatically. LACP modes:
Load Balancing in LAG
A LAG bundles N links but does not distribute a single flow across all links — a single TCP connection always travels on one physical link. LAG load balances at the flow level: different flows are hashed to different links. Common hash inputs: source+destination MAC (L2), source+destination IP (L3), source+destination IP+port (L4). This means a single large flow (one TCP connection) can only use one link — for a single iSCSI transfer, LAG provides no speed improvement. Multiple simultaneous flows do distribute.
Compatibility Requirements
All member links in a LAG must have identical: speed, duplex, VLAN configuration, and spanning tree port settings. A mismatch causes the link to be excluded from the bundle. Common mistake: adding a port with different native VLAN or trunk configuration — LACP rejects the port and it operates as a standalone link without error.
// CHAPTER 11
Flow Control and Storm Control
Preventing packet loss and broadcast storms
IEEE 802.3x Flow Control (PAUSE Frames)
When a switch's input buffer is near full, it can signal the connected device to pause transmission temporarily. The switch sends an Ethernet PAUSE frame (EtherType 0x8808) containing a pause timer value (0–65535 × 512 bit-times). The receiving NIC stops transmitting for the specified duration. This prevents buffer overflow at the cost of temporary transmission suspension.
PAUSE frames are problematic in multi-hop networks: a pause from one congested link can propagate back to all upstream senders, head-of-line blocking traffic that doesn't need to pause. Priority Flow Control (PFC, IEEE 802.1Qbb) — used in lossless Ethernet for RoCE (RDMA over Converged Ethernet) — operates per-priority class, pausing only the congested priority without affecting other classes.
Storm Control
A broadcast storm occurs when broadcast traffic regenerates itself in a loop — a switch receives a broadcast, floods it, another switch receives and floods, and so on. Without Spanning Tree Protocol, a single broadcast frame loops forever, doubling with each retransmission until the network is completely saturated. Storm control is a per-port rate limiter for broadcast, multicast, and unknown unicast traffic:
// CHAPTER 12
LLDP and CDP — Switch Discovery Protocols
How network devices map the physical topology
Network devices announce themselves to directly connected neighbors using link-layer discovery protocols. This data powers network management systems, automated inventory, and troubleshooting tools.
CDP — Cisco Discovery Protocol
CDP is Cisco-proprietary, Layer 2, multicast (01:00:0C:CC:CC:CC). Sent every 60 seconds. Advertises: device ID (hostname), platform, capabilities (router/switch/phone), software version, native VLAN, duplex, IP address, and port ID. CDP is enabled by default on all Cisco interfaces. Significant security risk: an attacker on the same segment receives full device inventory. Disable on external-facing ports: no cdp enable.
LLDP — IEEE 802.1AB
LLDP (Link Layer Discovery Protocol) is the open standard equivalent of CDP. TLV (Type-Length-Value) based — extensible. Core TLVs: Chassis ID, Port ID, TTL (30–120 seconds). Optional TLVs: system name, description, capabilities, management address. LLDP-MED extends LLDP for IP phones: negotiates VLAN, DSCP, PoE power requirements, emergency location information. Supported by all major vendors (Cisco, Juniper, Arista, HP, etc.).
// CHAPTER 13
Port Mirroring and SPAN
Capturing traffic without tapping the wire
SPAN (Switched Port Analyzer) — called "port mirroring" on most non-Cisco vendors — copies traffic from one or more ports (source) to a designated monitor port (destination). Used for: IDS/IPS sensors, packet capture analysis, network performance monitoring, passive wiretapping for forensics.
The SPAN destination port receives a copy of all source traffic and must have sufficient bandwidth to handle the aggregate. If 10 × 1 Gbps ports are spanned to a single 1 Gbps destination, only 10% of traffic is captured (the rest is dropped by the SPAN engine). Use a higher-bandwidth destination port, or limit source to a single port, or use a traffic tap instead.
// CHAPTER 14
Modern Ethernet — Datacenter and Beyond
Where Ethernet is going at 400G, 800G, and RoCE
Modern datacenter networking extends Ethernet far beyond its original LAN purpose. Two key developments: ultra-high-speed Ethernet (400G, 800G for spine-leaf fabric) and lossless Ethernet (for RDMA storage and HPC workloads).
400G and 800G Ethernet in Datacenters
A modern hyperscale datacenter switch (Broadcom Tomahawk 4: 25.6 Tbps, Broadcom Tomahawk 5: 51.2 Tbps) connects tens or hundreds of servers via high-density QSFP-DD 400G or 800G ports. These switches handle 10+ billion packets per second in hardware with single-digit microsecond latency. The switching ASIC processes every frame through a programmable pipeline: parse headers, look up forwarding table, apply ACL, decrement TTL, recompute CRC, output to correct port — all in hardware, at line rate, simultaneously on every port.
RoCE — RDMA over Converged Ethernet
RDMA (Remote Direct Memory Access) allows one server to write directly into another server's memory without CPU involvement — bypassing the OS kernel entirely. Originally, RDMA required InfiniBand. RoCEv2 (RDMA over Converged Ethernet v2) runs RDMA over standard 25G/100G Ethernet with UDP/IP encapsulation.
RoCE requires lossless Ethernet: RDMA is extremely sensitive to packet loss — a single dropped packet forces retransmission of large amounts of data (the RDMA window). Lossless Ethernet uses Priority Flow Control (PFC) to pause the sending port before buffer overflow occurs, preventing drops. The entire network path must be configured for PFC: NIC, switch, and QoS policies.
TSN — Time-Sensitive Networking
IEEE 802.1 TSN is a set of standards that make Ethernet deterministic — guaranteed maximum latency for time-critical traffic. Used in: industrial automation (replacing proprietary fieldbuses), automotive in-vehicle networking (replacing CAN bus), audio/video production (AES67, SMPTE ST 2110). TSN standards include: 802.1AS (timing synchronization to 1 µs accuracy), 802.1Qbv (time-aware shaper — scheduled transmission windows), 802.1Qbu (frame preemption — interrupt low-priority frames mid-transmission for time-critical ones).
// CHAPTER 15
Troubleshooting Ethernet and Switching
A systematic approach to Layer 2 problems
Common Ethernet Problems and Symptoms
Interpreting Interface Error Counters
// CHAPTER 16
Interview Questions
From beginner to PhD
What happens when a switch receives a frame for a MAC address not in its CAM table?
Why is the minimum Ethernet frame size 64 bytes? What happens to shorter frames?
What is the difference between a collision domain and a broadcast domain? How do switches and routers affect each?
A switch's CAM table is full. What happens to new traffic? How would you detect and prevent this?
Explain LACP negotiation in detail: what TLVs are exchanged, what constitutes a compatible LAG pair, and what happens when a member link fails?
A datacenter switch is showing increasing 'late collision' errors on a port connected to a server. The link shows as 1Gbps/full-duplex on both sides. What are the possible causes and how do you definitively diagnose?
Late collisions on a full-duplex link are theoretically impossible — collisions require a shared medium and CSMA/CD, which full-duplex disables. If late collisions appear on a reported full-duplex link, the actual link state is inconsistent:
1. Duplex mismatch despite "showing" full-duplex: The switch port is forced to 1G/full while the server NIC is set to auto-negotiate. Per IEEE 802.3, when the forced side sends at full-duplex but the auto-negotiating side receives no FLPs, it defaults to half-duplex (parallel detection). The server NIC is in half-duplex, running CSMA/CD, detecting "collisions" when the full-duplex switch keeps transmitting while the NIC tries to backoff. The switch counter shows "late collisions" because the NIC's backoff causes it to start a new transmission mid-frame from the switch's perspective. Diagnosis: show interfaces on BOTH sides — if server shows half-duplex and switch shows full-duplex, this is the cause. Fix: set both to autonegotiate.
2. Faulty cable or NIC causing signal corruption: Severe signal integrity issues (marginal cable, damaged connector, EMI) can cause the NIC's carrier sense circuitry to misfire, misinterpreting received frames as idle. The NIC transmits when the switch is also transmitting, and the resulting noise is interpreted as a late collision. Diagnosis: replace cable, test with known-good SFP, check DOM signal levels.
3. NIC driver/firmware bug: Rare, but some NIC firmware incorrectly reports duplex to the OS while operating in half-duplex mode internally. Check NIC vendor firmware version, compare against known-bad firmware list. Workaround: force NIC to 1G/full via ethtool or driver parameters.
🎯 Key Takeaways
- ✓Ethernet was invented in 1973 at Xerox PARC. IEEE 802.3 (1983) standardized it. The same frame format has survived 50+ years while speeds increased 270,000-fold from 10 Mbps to 2.7 Tbps.
- ✓An Ethernet frame: Preamble(7B)+SFD(1B) | Dst MAC(6B) | Src MAC(6B) | [802.1Q tag(4B)] | EtherType(2B) | Payload(46–1500B) | FCS(4B). Minimum 64 bytes (CSMA/CD), maximum 1522 bytes (standard), 9022 bytes (jumbo).
- ✓The FCS (CRC-32) detects bit errors — a corrupted frame is silently dropped at Layer 2 with no error notification to the sender. Upper layers (TCP retransmit, UDP application) must detect and recover.
- ✓Hubs are shared collision domains — CSMA/CD required. Switches give each port its own collision domain. Full-duplex switch ports have zero collisions and disable CSMA/CD entirely.
- ✓The CAM table maps (MAC address, VLAN, port, timer). On unknown destination: flood. On known: unicast. Source MAC is always learned. Default aging: 300 seconds. TCAM limits entries to 8K–256K.
- ✓CSMA/CD: Carrier Sense → transmit → detect collision → jam → exponential backoff. 64-byte minimum frame guarantees the sender is still transmitting when the collision signal arrives from the furthest point.
- ✓Store-and-forward: full frame buffered, CRC checked, then forwarded (12 µs for 1500B at 1G). Cut-through: forward after reading 14-byte header (~1 µs). Cut-through propagates errors; store-and-forward does not.
- ✓LACP (802.3ad) bonds multiple links into one logical interface. Load balancing hashes flows — a single TCP connection only uses one physical member. All members must match speed/duplex/VLAN config.
- ✓Port security limits MAC addresses per port (maximum + sticky). 802.1X is the proper solution: authenticate the user/device via RADIUS before granting any network access.
- ✓LLDP (IEEE 802.1AB) and CDP (Cisco-proprietary) discover directly connected neighbors — device ID, port ID, capabilities, VLANs, PoE. Disable on external-facing ports (security risk).
- ✓SPAN/port mirroring copies traffic to a monitor port for IDS/packet capture. RSPAN spans across switches via VLAN. ERSPAN encapsulates in GRE for remote IP-based collection.
- ✓Late collisions on a full-duplex port = duplex mismatch. CRC errors = cable/fiber/SFP issues. High unknown unicast flood rate = CAM table near capacity or MAC flood attack.
Discussion
0Have a better approach? Found something outdated? Share it — your knowledge helps everyone learning here.