300-640 DCAI — Cisco Implementing Data Center AI Infrastructure Cheat Sheet

Use this as an independent compact review for Cisco Cisco Implementing Data Center AI Infrastructure (300-640 DCAI). Focus on implementation decisions: AI fabric design, RoCEv2 behavior, lossless Ethernet, Cisco Nexus operations, Cisco UCS/Intersight compute lifecycle, storage paths, automation, observability, and troubleshooting.

The exam identity is:

This page is IT Mastery review support. It is designed to pair naturally with an IT Mastery practice plan using original practice questions, a question bank, targeted topic drills, and detailed explanations.

Training vs inference distinctions

AI workload patterns to recognize

Training versus inference

Typical AI fabric planes

    flowchart LR
	    User[Users / APIs] --> FE[Frontend service fabric]
	    FE --> App[Inference / scheduler / apps]
	    App --> Storage[Dataset / model storage]
	    GPU[GPU servers] <--> BE[Backend AI fabric: RoCEv2 / RDMA]
	    GPU --> Storage
	    Mgmt[OOB management] --> GPU
	    Mgmt --> Nexus[Cisco Nexus switches]
	    Ops[Cisco Intersight / Nexus Dashboard / NDFC] --> Mgmt

PFC, ECN, and drops

RoCEv2 implementation checklist

• Use a dedicated QoS class for RDMA traffic.
• Map RDMA traffic consistently: application/NIC marking → DSCP/CoS → switch qos-group → queue.
• Enable PFC only for the intended no-drop priority.
• Configure ECN/WRED behavior for early congestion signaling where supported.
• Keep MTU consistent across server NICs, switch interfaces, port channels, routed links, and overlays where applicable.
• Verify the trust boundary: do switches trust server markings, rewrite them, or classify by ACL?
• Avoid mixing RDMA with storage bursts, backup, checkpoint, or general TCP traffic in the same no-drop queue.
• Monitor PFC pause counters and ECN marks continuously; “no drops” is not proof of a healthy fabric.

RoCEv2, RDMA, and lossless Ethernet

RDMA allows applications to move data directly between memory regions with lower CPU overhead and lower latency. In Ethernet AI fabrics, RoCEv2 is commonly associated with RDMA over UDP/IP, which means the IP fabric, QoS policy, and host NIC configuration all matter.

What to remember

PFC versus ECN versus congestion control

RDMA implementation checklist

Use this checklist when reviewing a configuration or troubleshooting scenario:

Common RoCE/RDMA traps

• “Lossless” does not mean “no congestion.” It means the design tries to avoid drops for selected traffic while still controlling congestion.
• PFC is not a substitute for good fabric design. A congested or oversubscribed design can still perform poorly.
• PFC can spread congestion. Pause behavior can create head-of-line blocking if applied carelessly.
• ECN requires end-host participation. Switch marking alone does not reduce sender rate.
• MTU mismatch can look intermittent. Small tests may pass while real workloads fail.
• One bad link can affect a whole job. Distributed training often waits for the slowest participant.
• QoS markings must be trusted and preserved intentionally. Do not assume DSCP, CoS, or priority values survive every boundary.

Illustrative NX-OS QoS skeleton

Platform syntax and feature availability vary by Cisco Nexus model and NX-OS release. Treat this as a pattern, not a copy-paste answer.

class-map type qos match-any RDMA-QOS
  match dscp <rdma-dscp>

policy-map type qos AI-CLASSIFY
  class RDMA-QOS
    set qos-group <rdma-qos-group>

class-map type network-qos RDMA-NO-DROP
  match qos-group <rdma-qos-group>

policy-map type network-qos AI-NETWORK-QOS
  class type network-qos RDMA-NO-DROP
    mtu <jumbo-mtu>
    pause pfc-cos <rdma-cos>

policy-map type queuing AI-QUEUING
  class type queuing <rdma-queue-class>
    bandwidth percent <reserved-percent>
    random-detect ecn

system qos
  service-policy type qos input AI-CLASSIFY
  service-policy type network-qos AI-NETWORK-QOS
  service-policy type queuing output AI-QUEUING

Useful verification commands

show interface ethernet <slot/port>
show interface ethernet <slot/port> counters
show interface ethernet <slot/port> priority-flow-control
show queuing interface ethernet <slot/port>
show policy-map interface ethernet <slot/port>
show class-map type qos
show policy-map type network-qos
show policy-map type queuing
show lldp neighbors
show port-channel summary
show ip route
show bgp ipv4 unicast summary
show logging last <lines>

High-yield mental model

AI infrastructure is not “just fast networking.” It is an end-to-end system where GPU utilization depends on the combined behavior of compute, NICs, fabric, storage, automation, and observability.

A strong 300-640 DCAI review mindset is: What is the bottleneck, where is it measured, and which control plane or data plane mechanism is responsible?

Underlay vs overlay

Cisco Intersight vs UCS Manager vs Nexus Dashboard

Storage vs backend RDMA

Storage and data pipeline review

AI systems are often starved by storage before the network fabric is fully used. Review how data enters, moves through, and leaves the training or inference pipeline.

Storage troubleshooting decision points

• If all nodes slow down during checkpointing, check storage throughput, network queues, and QoS isolation.
• If only one node is slow, check local mount, path, NIC, driver, and cabling.
• If small-file workloads are slow, metadata performance may be the bottleneck.
• If large sequential reads are slow, check path bandwidth and storage backend limits.
• If storage and RDMA share links, verify classification and congestion behavior.

AI-specific security considerations

• Separate management, storage, frontend, and backend fabric access.
• Restrict who can change QoS, PFC, ECN, and fabric templates; mistakes can affect the whole cluster.
• Protect datasets, model artifacts, checkpoints, and container registries.
• Use change control for firmware, driver, and CUDA-related stack changes.
• Monitor for configuration drift between rails, leaves, and server NICs.
• Keep automation credentials scoped and rotated.

Security, isolation, and governance

AI infrastructure frequently carries sensitive datasets, model artifacts, credentials, and multi-tenant workloads.

Common security mistakes

• Reusing broad admin credentials in automation.
• Mixing management, storage, and workload traffic without clear policy.
• Allowing route leaking without an explicit purpose.
• Ignoring logging until after an incident.
• Treating AI lab environments as exempt from production controls.

Automation safeguards

• Maintain a source of truth for fabric topology, addressing, VRFs, QoS classes, and cabling.
• Validate generated config before deployment.
• Use staged rollout for QoS, PFC, ECN, and MTU changes.
• Capture pre-change counters and control-plane state.
• Confirm rollback steps before changing fabric-wide policies.
• Test one rail or failure domain when possible before broad rollout.

Cisco operations, management, and automation

For Cisco data center AI infrastructure, be comfortable with how implementation and operations tools fit together. You do not need to treat tools as magic boxes; understand what each tool controls or observes.

Automation review checklist

Observability signals to correlate

Do not troubleshoot with a single counter. Correlate:

• Interface errors and discards
• Queue drops and queue depth
• PFC pause frames
• ECN marks
• Link utilization and microburst indicators
• Routing adjacency state
• EVPN/VTEP state if overlays are used
• Host NIC counters
• GPU utilization
• Storage latency and throughput
• Application logs and job timing

Symptom-to-cause matrix

RoCEv2 / QoS

• RDMA traffic is classified consistently at ingress.
• DSCP/CoS/qos-group mapping is consistent across the fabric.
• PFC is enabled only on the intended priority.
• ECN marking is configured for the intended queue where supported.
• MTU is consistent across server NICs, switchports, port channels, routed links, and overlays.
• Queue counters are monitored before and after changes.
• Storage and checkpoint traffic do not pollute the RDMA no-drop queue.

Compute

• UCS/server firmware, NIC firmware, GPU drivers, and OS drivers are compatible.
• BIOS/performance settings match workload requirements.
• GPU-to-NIC locality is understood.
• Power and cooling are sufficient under sustained load.
• Intersight/UCS profiles reflect desired identity and firmware policy.
• Host NIC settings align with switch QoS, PFC, ECN, and MTU.

Storage

• Dataset path throughput matches expected GPU consumption.
• Checkpoint traffic is planned and monitored.
• Storage traffic has its own class or policy when needed.
• Metadata bottlenecks are considered for small-file workloads.
• Registry/model artifact pulls are cached or staged for scale-out events.

Operations

• NDFC/Intersight templates are version controlled or otherwise governed.
• Telemetry covers interfaces, queues, PFC, ECN, routes, server health, and job metrics.
• AAA/RBAC and management VRFs are in place.
• Pre-change and post-change validations are defined.
• Rollback steps are documented for QoS, MTU, routing, and firmware changes.

QoS review for AI fabrics

QoS in AI infrastructure is about protecting latency-sensitive and loss-sensitive traffic without starving other traffic.

Decision rule

When a scenario asks what to fix first, prioritize in this order:

1. Correct classification and marking
2. Correct queue and PFC/ECN policy
3. Consistent MTU
4. Host NIC congestion-control behavior
5. Fabric capacity and ECMP distribution
6. Workload-level validation

If the traffic is not classified correctly, every downstream QoS mechanism may be irrelevant.

Quick design checks

Ask these questions when reviewing a scenario:

1. Is traffic mostly east-west or north-south? Distributed AI training usually stresses east-west paths.

2. Is the fabric nonblocking enough for the workload? A fabric that is acceptable for general virtualization may be insufficient for GPU clusters.

3. Are paths symmetric and predictable? Asymmetry can complicate troubleshooting, hashing, and telemetry interpretation.

4. Is ECMP actually distributing traffic? A single elephant flow, poor hashing inputs, or polarization can overload one path while others sit idle.

5. Are MTU, QoS, and RDMA settings consistent from host to switch to host? Partial configuration is a common cause of intermittent failures.

Common EVPN/VXLAN review traps

• Underlay reachability must work before overlay troubleshooting is meaningful.
• A VTEP loopback issue can look like an endpoint issue.
• VNI/VRF mismatches can isolate workloads even when VLANs appear correct locally.
• MTU must account for encapsulation overhead.
• Control-plane reachability and data-plane forwarding are related but not the same.
• Route import/export mistakes can create either black holes or unintended reachability.

Host-side troubleshooting clues

Good troubleshooting order

1. Define the failure. Is it loss, latency, low throughput, failed adjacency, or workload timeout?
2. Scope the blast radius. One host, one rack, one fabric path, one tenant, or all workloads?
3. Separate host, fabric, and storage. Use counters and tests that isolate each layer.
4. Verify the control plane. Routing, EVPN, management reachability, and policy distribution.
5. Verify the data plane. Interfaces, queues, drops, MTU, encapsulation, and path utilization.
6. Check end-host settings. NIC mode, drivers, firmware, container/device access, congestion control.
7. Validate with representative traffic. Basic ping is not enough for AI workloads.
8. Change one variable at a time. Then compare pre/post telemetry.

Candidate mistakes to avoid

Recommended practice loop

1. Start with topic drills on RDMA, QoS, fabric design, and troubleshooting.
2. Review every missed question using the detailed explanations, not just the correct answer.
3. Build a short error log with three columns: concept missed, clue ignored, rule to remember.
4. Move to mixed question bank sets to practice switching topics.
5. Finish with timed mock exams only after your topic-level accuracy is stable.