Loop Detection & Prevention SOP

Modified on Fri, 12 Jun at 3:56 PM

NETWORK LOOP DETECTION & PREVENTION - STANDARD OPERATING PROCEDURE

Knowledge Base Article

Document ID: KB-LOOP-001
Version: 1.0
Last Updated: June 12, 2026
Platform: Juniper Networks (EX, QFX, SRX, vMX)
Status: Production

1. OVERVIEW

Network loops in Layer 2/3 environments can cause catastrophic failures including broadcast storms, MAC address table corruption, and complete network outage. This SOP provides procedures for detecting, preventing, and recovering from network loops.

1.1 Objective

Quickly detect active network loops
Prevent loops through configuration
Resolve loops with minimal downtime
Implement monitoring to prevent recurrence

1.2 Scope

This document covers: - Layer 2 (Ethernet) loops - Layer 3 routing loops - Broadcast storms - Loop prevention mechanisms (STP, RSTP, BFD) - Loop detection techniques and tools - Post-incident recovery procedures

2. UNDERSTANDING NETWORK LOOPS

2.1 Types of Loops

Layer 2 (Switching) Loops: - Caused by redundant cable connections without STP - Results in frames forwarding infinitely between ports - Leads to broadcast storms and MAC table saturation

Layer 3 (Routing) Loops: - Packets forwarded between routers without path convergence - Often caused by: static route misconfiguration, protocol convergence lag - Results in packets discarded or cycling indefinitely

Mixed Layer 2/3 Loops: - Occur in complex topologies mixing switching and routing - Difficult to detect and resolve

2.2 Loop Indicators

Broadcast storm: Excessive broadcast/multicast traffic
High CPU on switches: Processing flooded frames
MAC address flapping: Same MAC appearing on multiple ports
Network unavailability: Complete connectivity loss
Link saturation: Ports showing 100% utilization continuously
High packet drops: Excessive discarded frames

3. PREREQUISITES FOR TROUBLESHOOTING

Before starting loop detection/recovery:

Network access to affected switches/routers
Ability to shut down ports (lab environment preferred)
Network monitoring tools (sflow, packet capture capability)
Baseline topology diagram showing expected connections
Physical access to patch panels if needed
Maintenance window scheduled for reconnection
Backup configuration for quick rollback
Serial console access for unresponsive devices

4. DETECTING ACTIVE LOOPS

Procedure 4.1: Quick Loop Detection Signs

Immediate Indicators (Check First):

# Check CPU utilization
show chassis routing-engine

# Check interface statistics for unusual activity
show interfaces statistics | match "packets dropped\|errors"

# Check for MAC flapping
show route forwarding-table family inet | head
show ethernet-switching mac-table learning

# Monitor interface activity
show interfaces statistics | match "bytes.*in\|bytes.*out"

What to Look For: - ✗ CPU > 80% without traffic burst - ✗ Interface showing simultaneous RX and TX of same frames - ✗ High packet drop rates (>1%) - ✗ MAC address appearing on multiple ports - ✗ Port showing 100% utilization but low throughput

Procedure 4.2: Identify Source VLAN

Command:

show ethernet-switching mac-table
show ethernet-switching mac-table interface ge-0/0/0.0
show interfaces ge-0/0/0 family ethernet-switching

Analyze Output: - Determine which VLAN(s) affected - Check if all VLANs looping or isolated ones - Look for MAC addresses appearing multiple times

Collect VLAN Information:

show ethernet-switching vlan summary
show configuration vlans | grep -i "vlan-id\|members"

Procedure 4.3: Trace Loop Path

Method 1: Port Analysis

# Check port connections
show interfaces descriptions | grep -v "DISABLED\|disable"

# For each port, note what's connected:
# ge-0/0/0 -> Switch-A Port 1
# ge-0/0/1 -> Switch-B Port 1

Method 2: MAC Learning Analysis

# Check where packets entering:
show ethernet-switching mac-table | grep "XXX.YYY.ZZZ"

# Monitor real-time MAC movement:
show ethernet-switching mac-table | match "MAC"
show ethernet-switching mac-table continuous

Method 3: BPDU Analysis

# Check if STP active on switches
show spanning-tree bridge

# If STP enabled, check blocked ports:
show spanning-tree port | grep Blocking

Procedure 4.4: Packet Capture for Loop Confirmation

Steps:

# Start capture on suspected looped port
request packet-capture interface ge-0/0/0.0 file /var/tmp/loop_capture.pcap count 100 timeout 10

# Alternative: Capture with specific filter
request packet-capture interface ge-0/0/0.0 file /var/tmp/loop_bcast.pcap count 500 filter "dst ff:ff:ff:ff:ff:ff or dst 01:00:5e:00:00:00"

# Save capture
file copy /var/tmp/loop_capture.pcap /var/tmp/loop_capture_backup.pcap

Analysis with Wireshark: - Look for identical frames repeating - Check TTL/Hop Count decrements - Identify source and destination MACs - Look for broadcast/multicast frames

5. EMERGENCY LOOP MITIGATION

Procedure 5.1: Immediate Actions (When Loop Detected)

IMPORTANT: These steps must be done in coordinated manner

Step 1: Isolate Affected VLANs

# Identify VLAN in loop:
show ethernet-switching vlan summary

# If loop in VLAN 100:
# Option A: Disable VLAN on affected port
delete interfaces ge-0/0/0 unit 0 family ethernet-switching vlan members

# Option B: Shutdown port completely (fastest)
set interfaces ge-0/0/0 disable
commit

# Verify impact:
show interfaces ge-0/0/0.0 terse

Step 2: Verify No Broadcast Storm

# Check CPU dropped back to normal
show chassis routing-engine

# Verify traffic patterns returned to normal
show interfaces statistics | head -20

Step 3: Document Affected Ports

Procedure 5.2: Controlled Loop Isolation

If Broadcast Storm Still Active:

# Disable entire VLAN (careful - may impact users)
set vlans VLAN-NAME disable
commit

# Or disable all non-root bridge ports:
show spanning-tree port | grep "Blocking"
# Shutdown those interfaces

# Monitor recovery:
show interfaces statistics | grep "packets dropped"

6. LOOP ROOT CAUSE ANALYSIS

Procedure 6.1: Physical Topology Verification

Step 1: Document Expected Connections

# Create connectivity map:
# Expected:
# Switch1-Port1 <-> Switch2-Port1 (Link 1)
# Switch2-Port2 <-> Switch3-Port1 (Link 2)
# Switch3-Port2 <-> Core-Port1 (Link 3)

Step 2: Identify Unexpected Connections

# Physical inspection of cabling:
# Check patch panels for duplicate connections
# Look for:
# - Same switch connected to itself
# - Redundant connections without STP
# - Device connected to multiple ports
# - Broken cabling causing connection to wrong port

Step 3: Document All Redundant Connections

# List all potential loop-creating paths:
# If loop found between Switch A and B, document:
show lldp neighbors interface ge-0/0/0.0
show lldp neighbors detail

Procedure 6.2: STP Configuration Check

Step 1: Verify STP is Running

show spanning-tree bridge
show spanning-tree port | grep -E "Blocking|Forwarding"

Step 2: Check Root Bridge Election

show spanning-tree bridge | grep "Bridge ID"
# Root bridge should have lowest priority
show configuration protocols rstp | grep priority

Step 3: Verify Port States

show spanning-tree port
# Look for:
# ✓ One root port per non-root bridge
# ✓ Designated ports in Forwarding
# ✓ Alternate ports in Blocking

If STP Not Protecting Loop:

# STP may have failed - verify:
show spanning-tree statistics | grep -i "protocol"

# Enable RSTP if not active:
set protocols rstp
commit

Procedure 6.3: Routing Loop Analysis

Step 1: Check for Multiple Routes to Same Destination

show route <destination-ip>
show route <destination-ip> all

# If multiple routes with same cost:
show route summary

Step 2: Analyze Route Convergence

show route receiving-protocol bgp
show route receiving-protocol ospf

# Check for redistributed routes:
show configuration protocols bgp | grep -i "export\|import"

Step 3: Check Static Route Configuration

show configuration routing-options static
show route static

# Look for overlapping static routes
show route | grep "Static\|Direct"

7. LOOP PREVENTION CONFIGURATION

Procedure 7.1: Enable Spanning Tree Protocol

For Layer 2 Loop Prevention:

# Enable RSTP (preferred over STP)
set protocols rstp
set protocols rstp bridge-priority 32768 # Set appropriate priority
set protocols rstp max-age 20
set protocols rstp forward-delay 15
set protocols rstp hello-time 2

# Enable on interfaces
set interfaces ge-0/0/0 unit 0 family bridge
set interfaces ge-0/0/0 unit 0 family bridge port-mode access
set interfaces ge-0/0/0 unit 0 family bridge vlan members VLAN-NAME

commit

Verify:

show spanning-tree bridge
show spanning-tree port
show spanning-tree brief

Procedure 7.2: Enable Loop Detection/Prevention Mechanisms

BFD for Routing Loop Detection:

# Enable BFD on BGP
set protocols bgp group peers bfd-liveness-detection minimum-interval 300
set protocols bgp group peers bfd-liveness-detection multiplier 3

# Enable BFD on OSPF
set protocols ospf bfd-liveness-detection minimum-interval 300
set protocols ospf bfd-liveness-detection multiplier 3

commit

Storm Control (Broadcast Suppression):

# Limit broadcast/multicast traffic
set interfaces ge-0/0/0 unit 0 family ethernet-switching storm-control broadcast 40
set interfaces ge-0/0/0 unit 0 family ethernet-switching storm-control multicast 40
set interfaces ge-0/0/0 unit 0 family ethernet-switching storm-control unknown-unicast 40

commit

Procedure 7.3: Implement Loop Guard and BPDU Guard

Loop Guard (Prevents STP loops):

# Enable on non-root bridge ports
set interfaces ge-0/0/0 unit 0 family bridge loop-guard action block

commit

BPDU Guard (Protects access ports):

# Enable on edge ports
set interfaces ge-0/0/24 unit 0 family bridge bpdu-guard-action shutdown

commit

Procedure 7.4: Configure Root Guard

Prevents Other Devices Becoming Root Bridge:

# Enable on switches that should not be root
set interfaces ge-0/0/0 unit 0 family bridge root-guard action block

commit

8. LOG COLLECTION FOR LOOP ISSUES

8.1 Immediate Loop Detection Logs

Execute immediately upon detecting loop:

# Capture current state
show spanning-tree bridge > loop_stp_bridge.txt
show spanning-tree port > loop_stp_ports.txt
show ethernet-switching mac-table > loop_mac_table.txt
show interfaces statistics > loop_interface_stats.txt

# Capture history
show log messages | last 500 > loop_system_log.txt
show log messages | match "loop\|LOOP\|flap\|broadcast" | last 100 > loop_specific.txt

# Capture traffic patterns
show interfaces ge-0/0/0.0 | grep -i "input\|output" > loop_traffic.txt

# Capture CPU/Memory
show chassis routing-engine > loop_cpu_mem.txt

8.2 Packet Capture for Loop Analysis

Capture looped frames:

# Capture broadcast/multicast traffic
request packet-capture interface ge-0/0/0.0 file /var/tmp/loop_broadcast.pcap count 500 filter "dst ff:ff:ff:ff:ff:ff or dst 01:00:5e:00:00:00" timeout 30

# Capture all traffic on interface
request packet-capture interface ge-0/0/0.0 file /var/tmp/loop_all_traffic.pcap count 1000 timeout 30

# Save files
file copy /var/tmp/loop_*.pcap /var/tmp/backup/

Analysis: - Open in Wireshark - Look for frame duplication - Identify cyclic forwarding patterns - Note frame counts and intervals

8.3 Topology and Configuration Logs

Document affected topology:

# Get complete topology
show lldp neighbors > loop_topology_lldp.txt
show lldp neighbors detail >> loop_topology_lldp.txt

# Get interface configuration
show configuration interfaces > loop_interface_config.txt

# Get STP configuration
show configuration protocols rstp > loop_stp_config.txt
show configuration protocols stp >> loop_stp_config.txt

# Get bridge configuration
show configuration bridge-domains >> loop_interface_config.txt

9. LOOP RESOLUTION STEPS

Scenario A: Layer 2 Loop (Physical Cabling Error)

Step 1: Identify Physical Connection

show lldp neighbors detail | grep -A 5 "ge-0/0/0"

Step 2: Verify with Network Team - Contact team responsible for cabling - Confirm which connection is correct - Document accidental/redundant connection

Step 3: Remove Duplicate Connection - Physically disconnect wrong cable - Document location and date - Create change ticket

Step 4: Re-Enable STP if Disabled

set protocols rstp
commit
show spanning-tree brief (verify state)

Step 5: Verify No Loop Exists

# Monitor traffic for 5-10 minutes
show interfaces statistics | grep "packets dropped"
show chassis routing-engine (verify CPU normal)

Scenario B: STP Not Blocking Loop

Step 1: Verify STP Running

show spanning-tree bridge | grep -i "enabled"

Step 2: Check for Root Bridge Election

show spanning-tree bridge
show spanning-tree port | grep -E "Root|Designated"

Step 3: If STP Not Running

# Enable RSTP
set protocols rstp bridge-priority 32768
set interfaces ge-0/0/0 unit 0 family bridge
commit

# Wait for convergence (3-6 seconds for RSTP)
show spanning-tree brief (verify blocking port)

Step 4: If STP Running but Not Blocking

# Check port configuration
show spanning-tree port ge-0/0/0.0 detail

# Check for BPDU reception issues
show spanning-tree port ge-0/0/0.0 statistics | grep BPDU

# If no BPDUs, verify interface up:
show interfaces ge-0/0/0.0 detail | grep "Admin\|Link"

Scenario C: Routing Loop (BGP/OSPF)

Step 1: Identify Looping Route

show route <destination>
traceroute <destination> # Look for repeated addresses

Step 2: Check BGP Configuration

show configuration protocols bgp | grep -i "export\|import\|redistribute"
show route receiving-protocol bgp | grep <destination>

Step 3: Check OSPF Configuration

show configuration protocols ospf | grep -i "export\|import\|redistribute"
show route receiving-protocol ospf | grep <destination>

Step 4: Fix Configuration Issue

# If wrong redistribution:
delete protocols bgp group peers export POLICY-NAME
commit

# If wrong static route:
delete routing-options static route <bad-route>
commit

Step 5: Verify Route Converges

show route <destination>
traceroute <destination> # Verify path is acyclic

10. POST-INCIDENT RECOVERY

10.1 Verify Full Recovery

Step 1: Confirm No Active Loop

show interfaces statistics | grep "packets dropped"
show chassis routing-engine (CPU should be normal)
show spanning-tree brief (no flapping)

Step 2: Re-enable Disabled Ports

# If ports were shutdown:
delete interfaces ge-0/0/0 disable
commit

# Verify port comes up
show interfaces ge-0/0/0.0 terse

Step 3: Test Affected Services

ping <critical-destinations>
traceroute <critical-destinations>

10.2 Implement Prevention

Step 1: Deploy STP/RSTP

set protocols rstp
commit

Step 2: Configure Storm Control

set interfaces ge-0/0/0 unit 0 family ethernet-switching storm-control broadcast 40
commit

Step 3: Document Topology - Create network diagram - Document all switch connections - Mark redundant vs. critical links

Step 4: Test Loop Detection

# In lab environment, intentionally create loop
# Verify STP blocks it automatically
# Verify monitoring alerts on loop detection

11. MONITORING FOR LOOP PREVENTION

11.1 Recommended Alerts

Set up monitoring for: - High broadcast traffic: > 10% of link capacity - MAC address flapping: Same MAC on different ports - STP topology changes: More than 2 per hour - CPU spike: > 80% without known traffic burst - Interface discards: Sustained packet loss > 1%

11.2 Monitoring Commands

# Regular health check
show interfaces statistics | grep "packets dropped\|errors"
show spanning-tree statistics | grep "topology"
show ethernet-switching mac-table | wc -l # Shouldn't exceed 48K

# Trend analysis
monitor interface ge-0/0/0.0
show interfaces ge-0/0/0.0 statistics | grep "input rate\|output rate"

12. ESCALATION CRITERIA

Escalate to JTAC if:

Loop detected but STP not blocking it
Loop causes complete network outage
Recovery requires device restart
Persistent loops despite mitigation
Multiple simultaneous loops detected
Inability to identify loop source after 30 minutes
STP causing high CPU usage
Routing loops not resolved by config correction

Provide JTAC with: - All logs from Section 8 - PCAP files showing looped traffic - Complete topology diagram - Configuration files from all affected switches - Timeline of loop detection and actions taken - Hardware/software version information

13. REFERENCE CONFIGURATION

Minimal Loop Prevention Config

# Enable RSTP
set protocols rstp bridge-priority 32768
set protocols rstp max-age 20

# Enable on all interfaces
set interfaces ge-0/0/0 unit 0 family bridge

# Enable storm control
set interfaces ge-0/0/0 unit 0 family ethernet-switching storm-control broadcast 40
set interfaces ge-0/0/0 unit 0 family ethernet-switching storm-control multicast 40

# Save and commit
commit

14. DOCUMENT HISTORY

Version	Date	Author	Changes
1.0	2026-06-12	Network Engineering	Initial creation

For Support: Contact Network Operations Center or JTAC
Document Owner: Network Engineering Team
Last Reviewed: June 12, 2026