Visualization¶

The rd visualize command opens a converted episode in the Rerun viewer. It logs RGB images, plasma-colorized depth maps, camera frustums, colored 3-D point clouds, and end-effector action scalars.

Data must be converted first with rd convert before it can be visualized.

Usage¶

Running rd visualize with no arguments launches an interactive fzf selector:

rd visualize

You will be prompted to choose a task, then an episode:

  Visualize task>
  pick_purrito

  Select episode>
  0000  (pick_purrito_20260303_115816)
  0001  (pick_purrito_20260303_120053)

Options can be passed alongside the interactive flow:

# Every other frame
rd visualize --stride 2

# Full resolution (slower)
rd visualize --image-scale 1.0

Browser viewer (SSH tunnel)¶

Pass --web to serve the viewer over HTTP instead of spawning the native desktop app. This is useful when working over an SSH connection:

rd visualize --web

The command prints a URL and the SSH tunnel command to use:

Open in browser: http://localhost:9090?url=rerun%2Bhttp%3A%2F%2Flocalhost%3A9091%2Fproxy
SSH tunnel:      ssh -L 9090:localhost:9090 -L 9091:localhost:9091 <host>

On your local machine, run the printed ssh command (or add the -L flags to your existing SSH session), then open the URL in a browser. The process keeps running until you press Ctrl-C.

The default web port is 9090; change it with --web-port:

rd visualize --web --web-port 8080

Options¶

Option	Default	Description
`--stride`	`1`	Log every N-th frame
`--image-scale`	`0.25`	Downsample factor applied to images and point clouds
`--web`	`false`	Serve viewer over HTTP for browser access
`--web-port`	`9090`	HTTP port for the web viewer

What is logged¶

Rerun path	Content
`world/cameras/<name>`	Camera pose (`Transform3D`) + intrinsics (`Pinhole`) + RGB texture
`world/cameras/<name>/rgb`	RGB image in the 2-D Spatial View
`world/cameras/<name>/depth`	Plasma-colorized inverse-depth image
`world/points/<name>`	Colored world-space point cloud unprojected from depth
`action/right/<label>`	Right end-effector pose + gripper scalar (`pos(3) + rot_mat_flat(9) + gripper(1)`)
`action/left/<label>`	Left end-effector pose + gripper scalar
`trajectory/right`	Full right-arm EE trajectory as a static 3-D line (red)
`trajectory/left`	Full left-arm EE trajectory as a static 3-D line (blue)
`trajectory/right/current`	Current right-arm EE position (moves with timeline)
`trajectory/left/current`	Current left-arm EE position (moves with timeline)
`info`	Recording metadata text

Depth colorization¶

Depth maps are visualized using an inverse-depth plasma colormap (closer objects appear brighter). The colormap is normalized to the 95th percentile of valid depth values, and invalid pixels (zero depth) are shown as black.

Point clouds¶

World-space point clouds are reconstructed by unprojecting the downsampled depth map through the per-frame camera intrinsics and extrinsics. Points are colored by the corresponding RGB pixel. Invalid points (non-finite or at the world origin) are discarded.

Action scalars¶

The action key in each lowdim file contains the end-effector 6-D pose and gripper state for both arms, expressed in the left-arm base frame (the global coordinate origin - see Arm configuration):

right: pos(3)  +  rot_mat_flat(9)  +  gripper(1)   # indices  0–12
left:  pos(3)  +  rot_mat_flat(9)  +  gripper(1)   # indices 13–25

The rotation is a 3×3 rotation matrix flattened row-major (9 values) — no gimbal lock or wrap-around discontinuities.

Use the Rerun timeline to scrub through frames. The frame timeline corresponds directly to the frame index in the converted episode.