dgp.utils.torch_extension package¶

dgp.utils.torch_extension.camera module¶

class dgp.utils.torch_extension.camera.Camera(K, D=None, p_cw=None)¶

Bases: Module

Fully-differentiable camera class whose extrinsics operate on the appropriate pose manifold. Supports fully-differentiable 3D-to-2D, 2D-to-3D projection/back-projections, scaled camera and inverse warp functionality.

Note: This class implements the batched camera class, where a batch of camera intrinsics (K) and extrinsics (Tcw) are used for camera projection, back-projection.

K: torch.FloatTensor (B33): Camera calibration matrix.
p_cw: dgp.utils.torch_extension.pose.Pose or dgp.utils.torch_extension.pose.QuaternionPose: Pose from world to camera frame.

property Kinv¶

Analytic inverse camera intrinsic (K^-1)

Kinv: torch.FloatTensor (B33): Batch of inverse camera matrices K^-1.

property P¶

Projection matrix.

P: torch.Tensor (B34): Projection matrix

property cx¶

property cy¶

classmethod from_params(fx, fy, cx, cy, p_cw=None, B=1)¶

Create camera batch from calibration parameters.

fx: float: Camera focal length along x-axis.
fy: float: Camera focal length along y-axis.
cx: float: Camera x-axis principal point.
cy: float: Camera y-axis principal point.
p_cw: Pose: Pose from world to camera frame, with a batch size of 1.
B: int: Batch size for p_cw and K

Camera: Camera object with relevant intrinsics and batch size of B.

property fx¶

property fy¶

project(X, frame='w', shape=None)¶

Project 3D points from specified reference frame onto image plane TODO: Support sparse point cloud projection.

X: torch.FloatTensor: Batch of 3D spatial coordinates for each pixel in the specified reference frame. Shape must be B3HW or B3N.
frame: str, optional: Reference frame in which the input points (X) are specified. Default: ‘w’.
shape: tuple, optional: Optional image shape. Default: None.

x: torch.FloatTensor (B2HW or B2N): Batch of normalized 2D image coordinates for each pixel in the specified reference frame. Normalized points range from (-1,1).

ValueError: Raised if the shape of X is unsupported or if the frame type is unknown.

reconstruct(depth, frame='c')¶

Back-project to 3D in specified reference frame, given depth map

depth: torch.FloatTensor: Depth image. Shape must be B1HW.
frame: str, optional: Reference frame in which the output 3-D points are specified. Default: ‘c’.

X: torch.FloatTensor: Batch of 3D spatial coordinates for each pixel in the specified reference frame. Shape will be B3HW.

scaled(x_scale, y_scale=None)¶

Scale the camera by specified factor.

x_scale: float: x-axis scale factor.
y_scale: float: y-axis scale factor.

Camera: Scaled camera object.

to(*args, **kwargs)¶

Move camera object to specified device.

*args: tuple: Positional arguments to a to() call to move a tensor-like object to a particular device.
**kwargs: dict: Keyword arguments to a to() call to move a tensor-like object to a particular device.

training: bool¶

transform(X, frame)¶

Transform 3D points into the camera reference frame.

X: torch.FloatTensor: Points reference in the frame reference frame. Shape must be B3*.
frame: str: Reference frame in which the output 3-D points are specified. Options are ‘c’ and ‘w’ that correspond to camera and world reference frames.

Xc: torch.FloatTensor (B3*): Transformed 3D points into the camera reference frame.

ValueError: Raised if frame is an unsupported reference frame.

unproject(uv)¶

Project points from 2D into 3D given known camera intrinsics

uv: torch.FloatTensor: Input un-normalized points in 2D (B, 2, N)

rays: torch.FloatTensor (B, 3, N): Rays projected out into 3D

dgp.utils.torch_extension.camera.construct_K(fx, fy, cx, cy, dtype=torch.float32, device=None)¶

Create camera intrinsics from focal length and focal center

fx: float: Camera focal length along x-axis.
fy: float: Camera focal length along y-axis.
cx: float: Camera x-axis principal point.
cy: float: Camera y-axis principal point.
dtype: str: Tensor dtype
device: str: Tensor device

torch.FloatTensor: Camera intrinsic matrix (33)

dgp.utils.torch_extension.camera.scale_intrinsics(K, x_scale, y_scale)¶

Scale intrinsics matrix given x and y-axes scales. Note: This function works for both torch and numpy.

K: torch.FloatTensor, np.ndarray: An intrinsics matrix to scale. Shape is B33 or 33.
x_scale: float: x-axis scale factor.
y_scale: float: y-axis scale factor.

torch.FloatTensor, np.ndarray: Scaled camera intrinsic matrix. Shape is B33 or 33.

dgp.utils.torch_extension.pose module¶

Torch utilities for rigid-body pose manipulation.

Some of the rotation/quaternion utilities have been borrowed from: https://github.com/facebookresearch/QuaterNet/blob/master/common/quaternion.py https://github.com/arraiyopensource/kornia/blob/master/kornia/geometry/conversions.py

class dgp.utils.torch_extension.pose.Pose(value)¶

Bases: object

Generic rigid-body transformation class that operates on the appropriately defined manifold.

value: torch.FloatTensor (44, B44): Input transformation tensors either batched (B44) or as a single value (44).

value: torch.FloatTensor (B44): Input transformation tensor batched (B44)

copy()¶

classmethod identity(B=1, device=None, dtype=torch.float32)¶

Batch of identity matrices.

B: int, optional: Batch size. Default: 1.
device: str, optional: A device for a tensor-like object; ex: “cpu”. Default: None.
dtype: optional: A data type for a tensor-like object. Default: torch.float.

Pose: Batch of identity transformation poses.

inverse()¶

Invert homogeneous matrix as a rigid transformation.

Pose: Pose batch inverted on the appropriate manifold.

property matrix¶

Returns the batched homogeneous matrix as a tensor

result: torch.FloatTensor (B44): Bx4x4 homogeneous matrix

repeat(*args, **kwargs)¶

Repeat the Pose tensor

*args: tuple: Positional arguments to a repeat() call to repeat a tensor-like object along particular dimensions.
**kwargs: dict: Keyword arguments to a repeat() call to repeat a tensor-like object along particular dimension.

property rotation_matrix¶

Returns the 3x3 rotation matrix (R)

result: torch.FloatTensor (B33): Bx3x3 rotation matrix

to(*args, **kwargs)¶

Move object to specified device

*args: tuple: Positional arguments to a to() call to move a tensor-like object to a particular device.
**kwargs: dict: Keyword arguments to a to() call to move a tensor-like object to a particular device.

transform_points(X0)¶

Transform 3-D points from one frame to another via rigid-body transformation.

X0: torch.FloatTensor: 3-D points in torch.FloatTensor (shaped either B3N or B3HW).

torch.FloatTensor (B3N or B3HW): Transformed 3-D points with the same shape as X0.

transform_pose(other)¶

Left-multiply (oplus) rigid-body transformation.

other: Pose: Pose to left-multiply with (self * other)

Pose: Transformed Pose via rigid-transform on the manifold.

property translation¶

Return the translation component of the pose as a torch.Tensor.

tvec: torch.FloatTensor (B3): Translation component of the Pose object.

class dgp.utils.torch_extension.pose.QuaternionPose(wxyz, tvec)¶

Bases: object

Derived Pose class that operates on the quaternion manifold instead.

wxyz: torch.FloatTensor (4, B4): Input quaternion tensors either batched (B4) or as a single value (4,).
tvec: torch.FloatTensor (3, B3): Input translation tensors either batched (B3) or as a single value (3,).

classmethod from_matrix(value)¶

Create a batched QuaternionPose from a batched homogeneous matrix.

value: torch.FloatTensor: Batched homogeneous matrix. Shape is B44.

pose: QuaternionPosec: QuaternionPose batch. Batch dimension is shape B.

classmethod identity(B=1, device=None, dtype=torch.float32)¶

Batch of identity matrices.

B: int, optional: Batch size. Default: 1.
device: str: A device to send a tensor-like object to. Ex: “cpu”.
dtype: optional: A data type for a tensor-like object. Default: torch.float.

Pose: Batch of identity transformation poses.

inverse()¶: Invert T=[trans, quaternion] as a rigid transformation. Returns:

QuaternionPose: Pose batch inverted on the appropriate manifold.

property matrix¶

Returns the batched homogeneous matrix as a tensor

result: torch.FloatTensor (B44): Bx4x4 homogeneous matrix

repeat(B)¶

Repeat the QuaternionPose tensor

B: int: The size of the batch dimension.

property rotation_matrix¶

Returns the 3x3 rotation matrix (R)

result: torch.FloatTensor (B33): Bx3x3 rotation matrix

to(*args, **kwargs)¶

Move object to specified device

*args: tuple: Positional arguments to a to() call to move a tensor-like object to a particular device.
**kwargs: dict: Keyword arguments to a to() call to move a tensor-like object to a particular device.

transform_points(X0)¶

Transform 3-D points from one frame to another via rigid-body transformation.

Note: This function can be modified to do batched rotation operation with quaternions directly.

X0: torch.FloatTensor: 3-D points in torch.FloatTensor (shaped either B3N or B3HW).

torch.FloatTensor (B3N or B3HW): Transformed 3-D points with the same shape as X0.

transform_pose(other)¶

Left-multiply (oplus) rigid-body transformation.

other: QuaternionPose: Pose to left-multiply with (self * other)

QuaternionPose: Transformed Pose via rigid-transform on the manifold.

property translation¶

Return the translation component of the pose as a torch.Tensor.

tvec: torch.FloatTensor (B3): Translation component of the Pose object.

dgp.utils.torch_extension.pose.invert_pose(T01)¶

Invert homogeneous matrix as a rigid transformation: T^-1 = [R^T | -R^T * t]

T01: torch.FloatTensor: Input batch of transformation tensors. Shape is B44.

T10: torch.FloatTensor (B44): Inverted batch of transformation tensors.

dgp.utils.torch_extension.pose.normalize_quaternion(quaternion, eps=1e-12)¶

Normalizes a quaternion. The quaternion should be in (w, x, y, z) format.

quaternion: torch.FloatTensor: Input quaternion to normalize. Shape is B4.
eps: float, optional: Epsilon value to avoid zero division. Default: 1e-12.

normalized_quaternion: torch.FloatTensor (B4): Normalized quaternion.

TypeError: Raised if quaternion is not a torch.Tensor.
ValueError: Raised if the shape of quaternion is not supported.

dgp.utils.torch_extension.pose.qinv(q)¶

Returns the quaternion conjugate. (w, x, y, z)

q: torch.FloatTensor: Input quaternion to invert. Shape is B4.

quaternion: torch.FloatTensor (B4): Inverted quaternion.

dgp.utils.torch_extension.pose.qmul(q, r)¶

Multiply quaternion(s) q with quaternion(s) r. Expects two equally-sized tensors of shape (, 4), where * denotes any number of dimensions. Returns q*r as a tensor of shape (, 4).

q: torch.FloatTensor: Input quaternion to use for rotation. Shape is B4.
r: torch.FloatTensor: Second quaternion to use for rotation composition. Shape is B4.

rotated_q: torch.FloatTensor (B4): Composed quaternion rotation.

dgp.utils.torch_extension.pose.qrot(q, v)¶

Rotate vector(s) v about the rotation described by quaternion(s) q. Expects a tensor of shape (, 4) for q and a tensor of shape (, 3) for v, where * denotes any number of dimensions. Returns a tensor of shape (*, 3).

q: torch.FloatTensor: Input quaternion to use for rotation. Shape is B4.
v: torch.FloatTensor: Input vector to rotate with. Shape is B3.

vector: torch.FloatTensor (B3): Rotated vector.

dgp.utils.torch_extension.pose.quaternion_to_rotation_matrix(quaternion)¶

Converts a quaternion to a rotation matrix. The quaternion should be in (w, x, y, z) format.

quaternion: torch.FloatTensor: Input quaternion to convert. Shape is B4.

rotation_matrix: torch.FloatTensor (B33): Batched rotation matrix.

TypeError: Raised if quaternion is not a torch.Tensor.
ValueError: Raised if the shape of quaternion is not supported.

dgp.utils.torch_extension.pose.rotation_matrix_to_quaternion(rotation_matrix, eps=1e-08)¶

Convert 3x3 rotation matrix to 4d quaternion vector. The quaternion vector has components in (w, x, y, z) format.

rotation_matrix: torch.FloatTensor: Input rotation matrix to convert. Shape is B33.
eps: float, optional: Epsilon value to avoid zero division. Default: 1e-8.

quaternion: torch.FloatTensor (B4): Batched rotation in quaternion.

TypeError: Raised if rotation_matrix is not a torch.Tensor.
ValueError: Raised if the shape of rotation_matrix is unsupported.

dgp.utils.torch_extension.stn module¶

dgp.utils.torch_extension.stn.image_grid(B, H, W, dtype, device, normalized=False)¶

Create an image mesh grid with shape B3HW given image shape BHW

B: int: Batch size
H: int: Grid Height
W: int: Batch size
dtype: str: Tensor dtype
device: str: Tensor device
normalized: bool: Normalized image coordinates or integer-grid.

grid: torch.Tensor: Mesh-grid for the corresponding image shape (B3HW)

dgp.utils.torch_extension.stn.meshgrid(B, H, W, dtype, device, normalized=False)¶

Create mesh-grid given batch size, height and width dimensions.

B: int: Batch size
H: int: Grid Height
W: int: Batch size
dtype: torch.dtype: Tensor dtype
device: str: Tensor device
normalized: bool: Normalized image coordinates or integer-grid.

xs: torch.Tensor: Batched mesh-grid x-coordinates (BHW).
ys: torch.Tensor: Batched mesh-grid y-coordinates (BHW).

Quick Links

Table of Contents

Previous topic

dgp.utils.torch_extension package¶

dgp.utils.torch_extension.camera module¶

dgp.utils.torch_extension.pose module¶

dgp.utils.torch_extension.stn module¶

Module contents¶