HomeIncident briefing
Back to the library
INCIDENT #124 · NETWORK OPERATIONS

Data center: Only the upper half responds

Hard25 minutesPro lab

The incident

Application workers connect to database and connection-pool subnets through separate access and service aggregation layers. Four routed leaves share two spines; clients and services use separate access handoffs while partner and monitoring traffic terminate on other leaves. Addresses in the lower half of the primary service subnet fail, while .200 responds. The secondary service is unaffected. Investigate database connectivity from R1: 10.166.96.0/24 to 10.166.110.0/24 (TCP/5432), and 10.166.97.0/24 to 10.166.111.0/24 (TCP/6432).

Your objective

Restore both authorized service transports and return reachability, including clients and service hosts in both halves of the primary subnet. Keep R5 blocked from both service transports, keep SSH blocked to both service subnets, and preserve monitoring between R1 and R6. Do not change physical topology or addressing. These are network forwarding checks, not application-level transactions.

The network

Network topology8 devices 10 links

Data center: Only the upper half responds — network topologyR1: Client edge. R2: Client leaf. R3: Spine A. R4: Spine B. R5: Partner leaf. R6: Monitoring leaf. R7: Service leaf. R8: Service gateway. R1 connects to R2. R2 connects to R3. R2 connects to R4. R5 connects to R3. R5 connects to R4. R6 connects to R3. R6 connects to R4. R7 connects to R3. R7 connects to R4. R7 connects to R8. Connections show physical links, not service health.R1 ↔ R2 · 10.166.112.0/30R2 ↔ R3 · 10.166.113.0/30R2 ↔ R4 · 10.166.114.0/30R5 ↔ R3 · 10.166.115.0/30R5 ↔ R4 · 10.166.116.0/30R6 ↔ R3 · 10.166.117.0/30R6 ↔ R4 · 10.166.118.0/30R7 ↔ R3 · 10.166.119.0/30R7 ↔ R4 · 10.166.120.0/30R7 ↔ R8 · 10.166.121.0/30R1Client edgeR1 · Client edge · Starting deviceR2Client leafR2 · Client leafR3Spine AR3 · Spine AR4Spine BR4 · Spine BR5Partner leafR5 · Partner leafR6MonitoringleafR6 · Monitoring leafR7Service leafR7 · Service leafR8ServicegatewayR8 · Service gateway
Physical links · Service health not shown
Link addressing (10)
  • R1 R210.166.112.0/30
  • R2 R310.166.113.0/30
  • R2 R410.166.114.0/30
  • R5 R310.166.115.0/30
  • R5 R410.166.116.0/30
  • R6 R310.166.117.0/30
  • R6 R410.166.118.0/30
  • R7 R310.166.119.0/30
  • R7 R410.166.120.0/30
  • R7 R810.166.121.0/30
8 devices · Cisco IOS syntax · Simulated network

What you’ll practice

longest prefix matchsubnet boundariesforwarding loops

Interview exercises are inspired by common industry competencies. They are not proprietary interview questions.