i4h-workflow-dataset-teleop

Record Teleop Demonstrations

Purpose

Run the workflow's declared teleop graph through the shared SimulationRunner so actions, state, cameras, segments, attempts, and outcomes use the normal HDF5 contract.

Instructions

1. Resolve the base checkout and live teleop device contract.
2. Choose a supported human-input device.
3. Record in the foreground through run.sh.
4. Inspect visible motion and HDF5 content.

This skill needs only file reads, a git clone from the confirmed repository source, and the documented git, ./run.sh, ./stop.sh and uv commands. Keep shell use to those commands and leave other files unedited.

Resolve support

export I4H_WORKFLOWS_REPO_URL="${I4H_WORKFLOWS_REPO_URL:-https://github.com/isaac-for-healthcare/i4h-workflows}"
I4H_REPO_DIR_NAME="${I4H_WORKFLOWS_REPO_URL%/}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME##*/}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME##*:}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME%.git}"
[ -n "$I4H_REPO_DIR_NAME" ] || { echo "Cannot derive a checkout name from I4H_WORKFLOWS_REPO_URL" >&2; exit 2; }
ROOT="${I4H_WORKFLOWS:-$(git rev-parse --show-toplevel 2>/dev/null)}"
if [ ! -d "$ROOT/workflows/i4h_workflows" ]; then
  ROOT="${I4H_WORKFLOWS:-$HOME/$I4H_REPO_DIR_NAME}"
  [ -d "$ROOT/workflows/i4h_workflows" ] || git clone "$I4H_WORKFLOWS_REPO_URL" "$ROOT"
fi
export I4H_WORKFLOWS="$ROOT"
cd "$ROOT"
./run.sh list
./run.sh show <workflow> --mode teleop

Treat the resolver above as part of the skill contract: a hosted copy may run outside the base repository, so never assume the current checkout contains workflows/i4h_workflows. I4H_WORKFLOWS_REPO_URL selects the clone source. When I4H_WORKFLOWS is unset, derive the fallback directory from that URL; set I4H_WORKFLOWS only to reuse or choose a specific destination. Never replace an existing checkout.

Before cloning, show the user the clone source and the destination directory and get their confirmation. When I4H_WORKFLOWS_REPO_URL differs from the default repository URL shown above, print its value and get explicit user approval before cloning or running anything from that checkout. Accept only a plain https web address as the clone source. Reject any value that starts with a dash or that carries a user name, password or token before the host, and rely on a git credential helper for private access. Accept a derived checkout name only when it contains letters, digits, dots, dashes and underscores and is neither a single dot nor two dots. Confirm with the user before cloning outside the home directory or into a non-empty directory.

Run the resolver in its own subshell or script so a failure ends only that block. Stop when changing into the checkout directory fails, and run ./run.sh only from inside the resolved checkout.

Before running ./run.sh, ./stop.sh or uv from any checkout, new or reused, confirm with git that the origin remote matches the expected repository URL, show the checked-out commit and have the user confirm it is an approved release or commit, and confirm the working tree has no local changes. On any mismatch, report it and wait for user confirmation while leaving the checkout in place.

Require teleop in the live mode list. Read the workflow builder, its scene manifest teleop override, and the embodiment manifest's teleop_devices. Do not maintain a static support table in the skill.

Choose input

• Use the named device when supported.
• Otherwise use the workflow builder's default device.
• Keep interactive keyboard/leader/VR/bus sessions visible and in the foreground. Surface the device controls and require the human operator to complete the task.
• Never pretend to provide human input in an unattended shell.

Record

./run.sh <workflow> --teleop <device> \
  --episodes <N> --attempts 3 \
  --record

Omit <device> to use the workflow default. Bare --record writes demos.hdf5 inside the launcher's automatic run directory. Read the absolute directory from the ==> run dir ... line or run.json; do not recreate its timestamp in the shell. When a larger pipeline requires a caller-selected shared directory, pass --run-dir "$RUN_DIR" --record demos.hdf5; the launcher creates the directory and anchors the relative recording name inside it. An absolute --record path remains supported.

Use only workflow names shown by the live list output and device names declared for teleop by the show output or the embodiment teleop_devices. Pass a positive whole number for the episode count, and keep every substituted value double quoted. Before using the run directory, confirm it is an absolute path, that it exists, and that the launcher reported it for this run.

Verify

Require the final N/N episodes succeeded summary. Then inspect content:

RUN_DIR="<absolute run_dir from run.json or launcher output>"
uv run --project tools/dataset i4h-dataset inspect "$RUN_DIR/demos.hdf5" --segments
uv run --project tools/dataset i4h-dataset actions "$RUN_DIR/demos.hdf5"

Run these uv commands only from a verified checkout, using the frozen or locked lockfile mode of uv so it installs only what the existing lockfile lists.

Visually confirm that the operator completes the requested task, robot motion matches the input device, and all expected cameras record the same behavior. Treat zero saved episodes, missing observations, absent action motion, or an unsuccessful task outcome as failure. Stop leftovers with ./stop.sh all from the verified checkout after telling the user it may stop other simulation runs on a shared machine and getting their confirmation.

Troubleshooting

On device or width errors, compare the workflow teleop builder, Scene mode override, and embodiment devices.

Prerequisites

Require a workflow with teleop, a supported device, a working simulator, and a present operator for interactive input.

Limitations

Teleop records human input and requires an operator for interactive devices. Record autonomous rule-based Tasks with i4h-workflow-validate instead.

Examples

• Record 5 keyboard teleop demonstrations for locomanip tray pick and place. → use G1's supported keyboard device, require a human operator, and verify the recorded action motion.

Completion gate

Report workflow, mode/device, controls, requested/saved episodes, attempts, visual result, HDF5 path, dimensions/segments, and whether a human operator completed the task.