doca-urom

DOCA UROM

Examples

• "Push UCX traffic onto the BlueField." — this is in scope. Stay on host-side doca-urom; do not deploy the DPU-side UROM Service here.
• "First doca_urom call returns NOT_PERMITTED." — check host/service version alignment and the RDMA substrate before changing the offload call.

Instructions

Follow Loading order below. Open TASKS.md for configure / offload / debug. Open CAPABILITIES.md for service vs worker contexts. Do not open every companion file.

Where to start: This skill assumes DOCA is already installed on both the host and the BlueField, the DOCA UROM Service is deployed and running on the BlueField side, and the user is doing hands-on UROM work from the host side — i.e. using doca-urom from an HPC / UCX / MPI stack on the host to enqueue remote memory operations (puts, gets, atomics, active messages, collective primitives) that the BlueField DPU will execute on the host's behalf. Open the tasks companion if the user wants to do something (configure / build / modify / run / test / debug); open the capabilities companion when the question is what can the host-side UROM API express on this version + this BlueField + this UROM Service version. If the user has not installed DOCA yet, route to doca-setup first; if the user is asking about the DPU-side UROM Service itself (deployment, container, operation lifecycle on the DPU side), that is a DIFFERENT artifact — route via doca-public-knowledge-map ## DOCA services to the public DOCA UROM Service guide. This skill is the host-side library; the UROM Service is the DPU-side executor, and they are a paired contract.

How to use this skill. Read this file, then search the companion file for the ## <name> heading the question needs and read that whole section. Answer from it, using only its symbols and commands. Present every command to the user as a suggestion to review and run on their own DOCA host; the agent does not execute it on their behalf. Do not browse docs yourself; use the entry points in doca-public-knowledge-map. Documentation links listed in this skill are references for the user to open. When the bundle is ambiguous or the installed version differs, point the user to the official DOCA UROM library guide linked under Related skills and to the installed headers and shipped samples as the authority. Give environment checks as read-only commands for the user's DOCA host first. Present any elevated privilege use, permission or group change, or service or container restart on the host or DPU only as a suggestion that needs explicit user approval, for the user to run. For out-of-scope requests, name the skill that owns them and stop; if that skill is not available, name the public DOCA UROM library guide or the public DOCA UROM Service guide instead and stop. Linked sibling skills and external guides are reference material only; this skill's safety rules, stop conditions, and propose-to-the-user rule for commands still apply when following them.

Before recommending UROM for a generic MPI or UCX request, confirm the user has a BlueField DPU host running DOCA; if not, apply the path-selection rule in CAPABILITIES.md and route the user accordingly.

Example questions this skill answers well

The CLASSES of UROM questions this skill is built to answer, each with one worked example. The agent should treat the class as the load-bearing piece — the worked example is a single instance.

• "How do I offload my MPI / UCX remote memory operations from the host CPU to the BlueField DPU?" — worked example: "my MPI all-reduce is consuming host CPU cycles I'd rather use for compute — how do I push that work onto the BlueField via UROM?". Answered by the host-library-plus-DPU-service paired-contract model in the capabilities companion
  • the host-side bring-up workflow in the tasks companion.
• "Is the DOCA UROM Service even running on my BlueField, and why does that matter before I write any doca_urom_* code?" — worked example: "my first doca_urom_* call returns DOCA_ERROR_NOT_PERMITTED on a host where DOCA is otherwise healthy". Answered by the env-precondition matrix in the capabilities companion
  • the service-deployed-and-running check in the tasks companion step 1, which routes service-side env questions to doca-public-knowledge-map ## DOCA services.
• "Is this UROM operation type / atomic / collective supported on my device + this DOCA install + this UROM Service version?" — worked example: "does my BlueField support remote atomic Fetch-and-Add for an MPI window?". Answered by the plugin-discovery rule (doca_urom_service_get_plugins_list on a started Service — UROM operations are plugin-defined Command tasks, so the supported-plugins list is the capability surface) in the capabilities companion
  • the discovery step in the tasks companion.
• "How does doca-urom relate to doca-rdma — am I replacing it, layering on top, or something else?" — worked example: "I already have raw doca-rdma working; should I rewrite to use UROM, or is that the wrong tool?". Answered by the path-selection rule in the capabilities companion (UROM uses the RDMA transport substrate underneath but adds the DPU-offload contract on top, and is the right tool only when host CPU is the bottleneck due to communication overhead — small / simple point-to-point cases stay on doca-rdma).
• "Is this UROM API on my installed DOCA version?" — worked example: "is the collective-ops plugin discoverable via doca_urom_service_get_plugins_list on DOCA 3.x". Answered by the version-compatibility overlay in the capabilities companion, which cross-links the canonical detection chain in doca-version and adds the UROM-specific host library and DPU service versions must match overlay.
• "What does this DOCA_ERROR_* from a doca_urom_* call mean and which layer caused it?" — worked example: "DOCA_ERROR_NOT_PERMITTED on the first doca_urom_* enqueue after doca_ctx_start() succeeded". Answered by the UROM overlay on the cross-library taxonomy in the capabilities companion
  • the layered ladder in the tasks companion that escalates to doca-debug.

Audience

This skill serves external developers building HPC / UCX / MPI applications that consume the DOCA UROM library from the host side — i.e., users whose code calls doca_urom_* (directly in C / C++, or through FFI / bindings from another language, or through a UCX-based stack such as OpenMPI / MPICH that has been wired to use DOCA UROM as a UCX transport) to push remote memory operations onto the BlueField DPU instead of executing them on the host CPU. It is not for NVIDIA developers contributing to DOCA UROM itself, nor is it the place to learn how to deploy / operate the DOCA UROM Service on the DPU side — that goes through the public DOCA UROM Service guide via doca-public-knowledge-map ## DOCA services.

Language scope. DOCA UROM ships as a host-side C library with pkg-config module name doca-urom. The shipped samples under /opt/mellanox/doca/samples/doca_urom/ are written in C (NVIDIA's choice). C and C++ consumers — including UCX-based stacks that wrap the library — are the canonical case and the worked examples in the tasks companion assume that path. Other-language consumers (Rust, Go, Python, …) consume the same *.so through FFI or language-specific bindings; the skill's contribution in that case is to keep the lifecycle, capability-discovery, service-deployed-and-running, error-taxonomy, and RDMA-substrate guidance language-neutral, and to route the agent to the public C ABI as the authoritative surface that any wrapper will eventually call.

When to load this skill

Load this skill when the user is doing hands-on DOCA UROM work from the host side, in any language. Concretely:

• Initializing a UROM Service context (doca_urom_service_*) on a doca_dev that maps to the BlueField the user wants to offload to, then creating Worker contexts (doca_urom_worker_*) attached to that Service, and confirming the matching DPU-side UROM Service is reachable before the first enqueue.
• Enqueueing remote memory operations (puts, gets, atomics, active messages, collective primitives) through the host-side doca_urom_* API and progressing the DOCA progress engine for completions.
• Reading or setting library properties via the host-side UROM API and calling doca_urom_service_get_plugins_list on a started Service to discover which plugins (and therefore which operation types / atomics / collectives, since these are plugin-defined) this device + this DOCA install + this DPU-side UROM Service version actually supports.
• Applying the host-side UROM lifecycle, capability-discovery, and error rules to a UCX-based HPC stack (OpenMPI, MPICH, custom UCX consumer) that is already wired to use UROM. Designing the UCX transport integration itself remains upstream-stack work.
• Debugging a DOCA_ERROR_* returned from a doca_urom_* call — in particular disambiguating DPU-side UROM Service not reachable from operation type not supported on this device from standard doca_dev access denied from underlying RDMA transport failure.
• Designing or extending non-C bindings (Rust, Go, Python, …) that wrap the UROM C ABI — for the lifecycle, service-deployed-and-running, capability-discovery, and error-taxonomy rules the wrapper must honor.

Do not load this skill for general DOCA orientation, install of DOCA itself, deployment / operation of the DOCA UROM Service on the DPU side (a separate artifact, with its own public guide reachable via doca-public-knowledge-map ## DOCA services), or non-UROM library questions. For those, use doca-public-knowledge-map.

What this skill provides

This is a thin loader. The body keeps only the orientation needed to pick the right next file. The substantive UROM-specific material lives in two companion files:

• CAPABILITIES.md — what the host-side UROM API can express on this version + this BlueField + this UROM Service version: the paired-contract model (host library enqueues; DPU service executes); the two host-side context types — the doca_urom_service (one per BlueField, bound to its doca_dev) and the doca_urom_worker contexts attached to it; the enqueue-side operation surface (puts, gets, atomics, active messages, collective primitives) delivered as plugin-defined Worker Command tasks and named generically because exact symbol shapes are plugin- and install-bound; the plugin-discovery surface (doca_urom_service_get_plugins_list); the UROM error taxonomy mapped onto the cross-library DOCA_ERROR_* set; the observability surface (completion events on the DOCA progress engine, capability snapshots, infrastructure-side RDMA counters); and the safety policy that gates env preconditions (DOCA UROM Service deployed and running on the DPU side; host library and DPU service versions agreeing; an RDMA-capable BlueField + DOCA install).
• TASKS.md — step-by-step workflows for the six in-scope UROM verbs: configure, build, modify, run, test, debug. Plus a Deferred task verbs block that points out-of-scope questions at the right next skill.

The skill assumes a host + BlueField pair where DOCA is already installed at the standard location, the DOCA UROM Service is already deployed and running on the BlueField, the underlying RDMA transport between host and BlueField is healthy, and the user already has at least a sketch of the HPC / UCX / MPI stack they want to offload. It does not cover installing DOCA, deploying the UROM Service container on the BlueField, or bringing up RDMA / RoCE / IB transport between host and BlueField — those paths go through doca-setup, the public DOCA UROM Service guide, and doca-rdma respectively.

What this skill deliberately does not ship

This skill is agent guidance, not a samples or templates bundle. To keep the boundary clean, it deliberately does not contain — and pull requests should not add:

• Pre-written DOCA UROM application source code, in any language. The verified UROM source code is the shipped C samples at /opt/mellanox/doca/samples/doca_urom/. The agent's job is to route the user to those files and prescribe a minimum-diff modification on a copy of the chosen sample in the user's project directory (edit only that copy and leave the shared install tree unchanged) via the universal modify-a-sample workflow in doca-programming-guide, layered with the UROM-specific overrides in the tasks companion.
• An MPI / UCX algorithm or collective implementation. This library offloads remote memory operations; the algorithm selecting which puts / gets / atomics / collectives to issue for a given MPI primitive is the user's HPC stack (OpenMPI, MPICH, custom UCX consumer) or the user's domain expertise. The agent must refuse to invent collective algorithms and must route any "what algorithm should my all-reduce use" question to the upstream MPI / UCX documentation — that is a research / stack-design question, not a UROM API question.
• Standalone build manifests (meson.build, CMakeLists.txt, Cargo.toml, …) parked inside the skill. The agent constructs the build manifest in the user's project directory against the user's installed DOCA, where pkg-config --modversion doca-urom is the source of truth.
• A samples/, bindings/, or reference/ subtree of any kind. A mock or incomplete artifact in this skill's tree, even one labeled "reference", is misleading: users will read it as buildable.
• The DOCA UROM Service surface. That service is a separate artifact with its own public guide; routing for its deployment / operation lives in doca-public-knowledge-map ## DOCA services. Conflating the library (this skill, host-side enqueue) with the service (DPU-side executor) is the single most common UROM first-app design error.

Loading order

1. Read this skill first to confirm the user's question is in scope (host-side library work; not DPU-side service deployment, and not MPI / UCX algorithm design).
2. For the UROM capability matrix, the paired-contract model (host library + DPU service), the Service + Worker context model, the enqueue-side operation surface, the plugin-discovery rule, the env-precondition policy, the error taxonomy, the observability surface, and the safety policy, see the capabilities companion.
3. For step-by-step workflows — configure, build, modify, run, test, debug — see the tasks companion.

Both companion files cross-link to each other, doca-version for the canonical DOCA version-handling rules (with the UROM overlay that the host library version and the DPU service version must match), doca-rdma for the underlying RDMA transport substrate UROM offloads ride on top of, and doca-public-knowledge-map whenever the right answer is "look it up in the public DOCA UROM library guide, the public DOCA UROM Service guide, or in the on-disk install layout" rather than "UROM host-side-specific guidance".

Related skills

• doca-public-knowledge-map — the routing table for every public DOCA documentation source and the on-disk layout of an installed DOCA package. The DOCA UROM library public guide is at https://docs.nvidia.com/doca/sdk/DOCA-UROM/index.html; the DOCA UROM Service is a different artifact documented separately under the DOCA services section of that map.
• doca-rdma — the underlying RDMA transport substrate the BlueField uses to actually move bytes between nodes once UROM has offloaded a remote memory operation. UROM does NOT replace RDMA; it adds the DPU-offload contract on top so the host CPU does not have to post the verbs itself. When the user's intent is simple point-to-point RDMA and the host CPU is not the bottleneck, the right tool is doca-rdma directly — UROM adds the service-side contract that isn't worth its overhead for that case.
• doca-setup — env preparation, install verification, BlueField setup, and the I have no install yet path with the public NGC DOCA container. This skill assumes its preconditions are satisfied AND that the DPU-side UROM Service is deployed and running on the BlueField.
• doca-version — canonical DOCA version-handling rules. This skill's ## Version compatibility cross-links the four-way match rule and adds the UROM-specific host library and DPU service must agree on version overlay.
• doca-structured-tools-contract — the bundle's structured-tools precedence rule (detect / prefer / fall back / report). The Command appendix in the tasks companion honors this contract.
• doca-programming-guide — general DOCA programming patterns shared by every library: the canonical pkg-config + meson build pattern, the universal modify-a-shipped-sample first-app workflow, the universal Core-context lifecycle, the cross-library DOCA_ERROR_* taxonomy, and the program-side debug order. This skill layers UROM specifics on top.
• doca-debug — the cross-cutting debug ladder (install / version / build / link / runtime / program / driver). UROM-specific debug (DPU-side UROM Service not running / not reachable, host library + DPU service version skew, RDMA transport failure underneath the UROM offload, operation type not in this device + firmware + service version) overlays on top of that ladder.

The DOCA UROM Service that runs on the DPU side is deliberately not in scope for this skill — it is a separate artifact with its own public guide, reachable through doca-public-knowledge-map ## DOCA services. Conflating the library and the service is the single most common UROM first-app design error: the agent must surface the library / service split explicitly whenever the question straddles the two.