dgp.utils package

Subpackages

dgp.utils.accumulate module

Useful utilities for accumulating sensory information over time

dgp.utils.accumulate.accumulate_points(point_datums, target_datum, transform_boxes=False)

Accumulates lidar or radar points by transforming all datums in point_datums to the frame used in target_datum.

point_datums: list[dict]

List of datum dictionaries to accumulate

target_datum: dict

A single datum to use as the reference frame and reference time.

transform_boxes: bool, optional

Flag to denote if cuboid annotations and instance ids should be used to warp points to the target frame. Only valid for Lidar. Default: False.

p_target: dict

A new datum with accumulated points and an additional field ‘accumulation_offset_t’ that indicates the delta in microseconds between the target_datum and the given point.

dgp.utils.accumulate.points_in_cuboid(query_points, cuboid)

Tests if a point is contained by a cuboid. Assumes points are in the same frame as the cuboid, i.e, cuboid.pose translates points in the cuboid local frame to the query_point frame.

query_points: np.ndarray

Numpy array shape (N,3) of points.

cuboid: dgp.utils.structures.bounding_box_3d.BoundingBox3D

Cuboid in same reference frame as query points

in_view: np.ndarray

Numpy boolean array shape (N,) where a True entry denotes a point insided the cuboid

dgp.utils.annotation module

dgp.utils.cache module

dgp.utils.cache.clear_cache(directory='/home/runner/.dgp/cache')

Clear DGP cache to avoid growing disk-usage.

directory: str, optional

A pathname to the directory used by DGP for caching. Default: DGP_CACHE_DIR.

dgp.utils.cache.diskcache(protocol='npz', cache_dir=None)

Disk-caching method/function decorator that caches results into dgp cache for arbitrary pickle-able / numpy objects.

protocol: str, optional

Serialization protocol. Choices: {npz, pkl}. Default: “npz” (numpy).

cache_dir: str, optional

Directory to cache instead of the default DGP cache. Default: None.

dgp.utils.camera module

dgp.utils.cli_utils module

Various utility functions for Click options

dgp.utils.cli_utils.add_options(**kwargs)

Decorator function to add a list of Click options to a Click subcommand.

**kwargs: dict

The options keyword argument shall be a list of string options to extend a Click CLI.

dgp.utils.colors module

This file contains definitions of predefined RGB colors, and color-related utility functions.

dgp.utils.colors.adjust_lightness(color, factor=1.0)

Adjust lightness of an RGB color.

color: Tuple[int]

RGB color

factor: float, optional

Factor of lightness adjustment. Default: 1.0.

Tuple[int]

Adjusted RGB color.

dgp.utils.colors.color_borders(img, color, thickness=10)

Draw a frame (i.e. the 4 sides) of image.

img: np.ndarray

Input image with shape of (H, W, 3) and unit8 type.

color: Tuple[int]

Color to draw the frame.

thickness: int, optional

Thickness of the frame. Default: 10.

img_w_frame: np.ndarray

Image with colored frame.

dgp.utils.colors.get_unique_colors(num_colors, in_bgr=False, cmap='tab20')

Use matplotlib color maps (‘tab10’ or ‘tab20’) to get given number of unique colors.

CAVEAT: matplotlib only supports default qualitative color palletes of fixed number, up to 20.

num_colors: int

Number of colors. Must be less than 20.

in_bgr: bool, optional

Whether or not to return the color in BGR order. Default: False.

cmap: str, optional

matplotlib colormap name (https://matplotlib.org/tutorials/colors/colormaps.html) Must be a qualitative color map. Default: “tab20”.

List[Tuple[int]]

List of colors in 3-tuple.

dgp.utils.dataset_conversion module

dgp.utils.datasets module

dgp.utils.pose module

dgp.utils.protobuf module

dgp.utils.statistics module

dgp.utils.testing module

exception dgp.utils.testing.SkipTest

Bases: Exception

Raise this exception in a test to skip it.

Usually you can use TestCase.skipTest() or one of the skipping decorators instead of raising this directly.

dgp.utils.testing.assert_allclose(actual, desired, rtol=1e-07, atol=0, equal_nan=True, err_msg='', verbose=True, *, strict=False)

Raises an AssertionError if two objects are not equal up to desired tolerance.

Given two array_like objects, check that their shapes and all elements are equal (but see the Notes for the special handling of a scalar). An exception is raised if the shapes mismatch or any values conflict. In contrast to the standard usage in numpy, NaNs are compared like numbers, no assertion is raised if both objects have NaNs in the same positions.

The test is equivalent to allclose(actual, desired, rtol, atol) (note that allclose has different default values). It compares the difference between actual and desired to atol + rtol * abs(desired).

New in version 1.5.0.

actualarray_like

Array obtained.

desiredarray_like

Array desired.

rtolfloat, optional

Relative tolerance.

atolfloat, optional

Absolute tolerance.

equal_nanbool, optional.

If True, NaNs will compare equal.

err_msgstr, optional

The error message to be printed in case of failure.

verbosebool, optional

If True, the conflicting values are appended to the error message.

strictbool, optional

If True, raise an AssertionError when either the shape or the data type of the arguments does not match. The special handling of scalars mentioned in the Notes section is disabled.

New in version 2.0.0.

AssertionError

If actual and desired are not equal up to specified precision.

assert_array_almost_equal_nulp, assert_array_max_ulp

When one of actual and desired is a scalar and the other is array_like, the function performs the comparison as if the scalar were broadcasted to the shape of the array. This behaviour can be disabled with the strict parameter.

>>> x = [1e-5, 1e-3, 1e-1]
>>> y = np.arccos(np.cos(x))
>>> np.testing.assert_allclose(x, y, rtol=1e-5, atol=0)

As mentioned in the Notes section, assert_allclose has special handling for scalars. Here, the test checks that the value of numpy.sin is nearly zero at integer multiples of π.

>>> x = np.arange(3) * np.pi
>>> np.testing.assert_allclose(np.sin(x), 0, atol=1e-15)

Use strict to raise an AssertionError when comparing an array with one or more dimensions against a scalar.

>>> np.testing.assert_allclose(np.sin(x), 0, atol=1e-15, strict=True)
Traceback (most recent call last):
    ...
AssertionError:
Not equal to tolerance rtol=1e-07, atol=1e-15

(shapes (3,), () mismatch)
 ACTUAL: array([ 0.000000e+00,  1.224647e-16, -2.449294e-16])
 DESIRED: array(0)

The strict parameter also ensures that the array data types match:

>>> y = np.zeros(3, dtype=np.float32)
>>> np.testing.assert_allclose(np.sin(x), y, atol=1e-15, strict=True)
Traceback (most recent call last):
    ...
AssertionError:
Not equal to tolerance rtol=1e-07, atol=1e-15

(dtypes float64, float32 mismatch)
 ACTUAL: array([ 0.000000e+00,  1.224647e-16, -2.449294e-16])
 DESIRED: array([0., 0., 0.], dtype=float32)
dgp.utils.testing.assert_almost_equal(actual, desired, decimal=7, err_msg='', verbose=True)

Raises an AssertionError if two items are not equal up to desired precision.

Note

It is recommended to use one of assert_allclose, assert_array_almost_equal_nulp or assert_array_max_ulp instead of this function for more consistent floating point comparisons.

The test verifies that the elements of actual and desired satisfy:

abs(desired-actual) < float64(1.5 * 10**(-decimal))

That is a looser test than originally documented, but agrees with what the actual implementation in assert_array_almost_equal did up to rounding vagaries. An exception is raised at conflicting values. For ndarrays this delegates to assert_array_almost_equal

actualarray_like

The object to check.

desiredarray_like

The expected object.

decimalint, optional

Desired precision, default is 7.

err_msgstr, optional

The error message to be printed in case of failure.

verbosebool, optional

If True, the conflicting values are appended to the error message.

AssertionError

If actual and desired are not equal up to specified precision.

assert_allclose: Compare two array_like objects for equality with desired

relative and/or absolute precision.

assert_array_almost_equal_nulp, assert_array_max_ulp, assert_equal

>>> from numpy.testing import assert_almost_equal
>>> assert_almost_equal(2.3333333333333, 2.33333334)
>>> assert_almost_equal(2.3333333333333, 2.33333334, decimal=10)
Traceback (most recent call last):
    ...
AssertionError:
Arrays are not almost equal to 10 decimals
 ACTUAL: 2.3333333333333
 DESIRED: 2.33333334

>>> assert_almost_equal(np.array([1.0,2.3333333333333]),
...                     np.array([1.0,2.33333334]), decimal=9)
Traceback (most recent call last):
    ...
AssertionError:
Arrays are not almost equal to 9 decimals

Mismatched elements: 1 / 2 (50%)
Max absolute difference among violations: 6.66669964e-09
Max relative difference among violations: 2.85715698e-09
 ACTUAL: array([1.         , 2.333333333])
 DESIRED: array([1.        , 2.33333334])
dgp.utils.testing.assert_approx_equal(actual, desired, significant=7, err_msg='', verbose=True)

Raises an AssertionError if two items are not equal up to significant digits.

Note

It is recommended to use one of assert_allclose, assert_array_almost_equal_nulp or assert_array_max_ulp instead of this function for more consistent floating point comparisons.

Given two numbers, check that they are approximately equal. Approximately equal is defined as the number of significant digits that agree.

actualscalar

The object to check.

desiredscalar

The expected object.

significantint, optional

Desired precision, default is 7.

err_msgstr, optional

The error message to be printed in case of failure.

verbosebool, optional

If True, the conflicting values are appended to the error message.

AssertionError

If actual and desired are not equal up to specified precision.

assert_allclose: Compare two array_like objects for equality with desired

relative and/or absolute precision.

assert_array_almost_equal_nulp, assert_array_max_ulp, assert_equal

>>> np.testing.assert_approx_equal(0.12345677777777e-20, 0.1234567e-20)
>>> np.testing.assert_approx_equal(0.12345670e-20, 0.12345671e-20,
...                                significant=8)
>>> np.testing.assert_approx_equal(0.12345670e-20, 0.12345672e-20,
...                                significant=8)
Traceback (most recent call last):
    ...
AssertionError:
Items are not equal to 8 significant digits:
 ACTUAL: 1.234567e-21
 DESIRED: 1.2345672e-21

the evaluated condition that raises the exception is

>>> abs(0.12345670e-20/1e-21 - 0.12345672e-20/1e-21) >= 10**-(8-1)
True
dgp.utils.testing.assert_array_almost_equal(actual, desired, decimal=6, err_msg='', verbose=True)

Raises an AssertionError if two objects are not equal up to desired precision.

Note

It is recommended to use one of assert_allclose, assert_array_almost_equal_nulp or assert_array_max_ulp instead of this function for more consistent floating point comparisons.

The test verifies identical shapes and that the elements of actual and desired satisfy:

abs(desired-actual) < 1.5 * 10**(-decimal)

That is a looser test than originally documented, but agrees with what the actual implementation did up to rounding vagaries. An exception is raised at shape mismatch or conflicting values. In contrast to the standard usage in numpy, NaNs are compared like numbers, no assertion is raised if both objects have NaNs in the same positions.

actualarray_like

The actual object to check.

desiredarray_like

The desired, expected object.

decimalint, optional

Desired precision, default is 6.

err_msgstr, optional

The error message to be printed in case of failure.

verbosebool, optional

If True, the conflicting values are appended to the error message.

AssertionError

If actual and desired are not equal up to specified precision.

assert_allclose: Compare two array_like objects for equality with desired

relative and/or absolute precision.

assert_array_almost_equal_nulp, assert_array_max_ulp, assert_equal

the first assert does not raise an exception

>>> np.testing.assert_array_almost_equal([1.0,2.333,np.nan],
...                                      [1.0,2.333,np.nan])

>>> np.testing.assert_array_almost_equal([1.0,2.33333,np.nan],
...                                      [1.0,2.33339,np.nan], decimal=5)
Traceback (most recent call last):
    ...
AssertionError:
Arrays are not almost equal to 5 decimals

Mismatched elements: 1 / 3 (33.3%)
Max absolute difference among violations: 6.e-05
Max relative difference among violations: 2.57136612e-05
 ACTUAL: array([1.     , 2.33333,     nan])
 DESIRED: array([1.     , 2.33339,     nan])

>>> np.testing.assert_array_almost_equal([1.0,2.33333,np.nan],
...                                      [1.0,2.33333, 5], decimal=5)
Traceback (most recent call last):
    ...
AssertionError:
Arrays are not almost equal to 5 decimals

nan location mismatch:
 ACTUAL: array([1.     , 2.33333,     nan])
 DESIRED: array([1.     , 2.33333, 5.     ])
dgp.utils.testing.assert_array_equal(actual, desired, err_msg='', verbose=True, *, strict=False)

Raises an AssertionError if two array_like objects are not equal.

Given two array_like objects, check that the shape is equal and all elements of these objects are equal (but see the Notes for the special handling of a scalar). An exception is raised at shape mismatch or conflicting values. In contrast to the standard usage in numpy, NaNs are compared like numbers, no assertion is raised if both objects have NaNs in the same positions.

The usual caution for verifying equality with floating point numbers is advised.

Note

When either actual or desired is already an instance of numpy.ndarray and desired is not a dict, the behavior of assert_equal(actual, desired) is identical to the behavior of this function. Otherwise, this function performs np.asanyarray on the inputs before comparison, whereas assert_equal defines special comparison rules for common Python types. For example, only assert_equal can be used to compare nested Python lists. In new code, consider using only assert_equal, explicitly converting either actual or desired to arrays if the behavior of assert_array_equal is desired.

actualarray_like

The actual object to check.

desiredarray_like

The desired, expected object.

err_msgstr, optional

The error message to be printed in case of failure.

verbosebool, optional

If True, the conflicting values are appended to the error message.

strictbool, optional

If True, raise an AssertionError when either the shape or the data type of the array_like objects does not match. The special handling for scalars mentioned in the Notes section is disabled.

New in version 1.24.0.

AssertionError

If actual and desired objects are not equal.

assert_allclose: Compare two array_like objects for equality with desired

relative and/or absolute precision.

assert_array_almost_equal_nulp, assert_array_max_ulp, assert_equal

When one of actual and desired is a scalar and the other is array_like, the function checks that each element of the array_like object is equal to the scalar. This behaviour can be disabled with the strict parameter.

The first assert does not raise an exception:

>>> np.testing.assert_array_equal([1.0,2.33333,np.nan],
...                               [np.exp(0),2.33333, np.nan])

Assert fails with numerical imprecision with floats:

>>> np.testing.assert_array_equal([1.0,np.pi,np.nan],
...                               [1, np.sqrt(np.pi)**2, np.nan])
Traceback (most recent call last):
    ...
AssertionError:
Arrays are not equal

Mismatched elements: 1 / 3 (33.3%)
Max absolute difference among violations: 4.4408921e-16
Max relative difference among violations: 1.41357986e-16
 ACTUAL: array([1.      , 3.141593,      nan])
 DESIRED: array([1.      , 3.141593,      nan])

Use assert_allclose or one of the nulp (number of floating point values) functions for these cases instead:

>>> np.testing.assert_allclose([1.0,np.pi,np.nan],
...                            [1, np.sqrt(np.pi)**2, np.nan],
...                            rtol=1e-10, atol=0)

As mentioned in the Notes section, assert_array_equal has special handling for scalars. Here the test checks that each value in x is 3:

>>> x = np.full((2, 5), fill_value=3)
>>> np.testing.assert_array_equal(x, 3)

Use strict to raise an AssertionError when comparing a scalar with an array:

>>> np.testing.assert_array_equal(x, 3, strict=True)
Traceback (most recent call last):
    ...
AssertionError:
Arrays are not equal

(shapes (2, 5), () mismatch)
 ACTUAL: array([[3, 3, 3, 3, 3],
       [3, 3, 3, 3, 3]])
 DESIRED: array(3)

The strict parameter also ensures that the array data types match:

>>> x = np.array([2, 2, 2])
>>> y = np.array([2., 2., 2.], dtype=np.float32)
>>> np.testing.assert_array_equal(x, y, strict=True)
Traceback (most recent call last):
    ...
AssertionError:
Arrays are not equal

(dtypes int64, float32 mismatch)
 ACTUAL: array([2, 2, 2])
 DESIRED: array([2., 2., 2.], dtype=float32)
dgp.utils.testing.assert_array_less(x, y, err_msg='', verbose=True, *, strict=False)

Raises an AssertionError if two array_like objects are not ordered by less than.

Given two array_like objects x and y, check that the shape is equal and all elements of x are strictly less than the corresponding elements of y (but see the Notes for the special handling of a scalar). An exception is raised at shape mismatch or values that are not correctly ordered. In contrast to the standard usage in NumPy, no assertion is raised if both objects have NaNs in the same positions.

xarray_like

The smaller object to check.

yarray_like

The larger object to compare.

err_msgstring

The error message to be printed in case of failure.

verbosebool

If True, the conflicting values are appended to the error message.

strictbool, optional

If True, raise an AssertionError when either the shape or the data type of the array_like objects does not match. The special handling for scalars mentioned in the Notes section is disabled.

New in version 2.0.0.

AssertionError

If x is not strictly smaller than y, element-wise.

assert_array_equal: tests objects for equality assert_array_almost_equal: test objects for equality up to precision

When one of x and y is a scalar and the other is array_like, the function performs the comparison as though the scalar were broadcasted to the shape of the array. This behaviour can be disabled with the strict parameter.

The following assertion passes because each finite element of x is strictly less than the corresponding element of y, and the NaNs are in corresponding locations.

>>> x = [1.0, 1.0, np.nan]
>>> y = [1.1, 2.0, np.nan]
>>> np.testing.assert_array_less(x, y)

The following assertion fails because the zeroth element of x is no longer strictly less than the zeroth element of y.

>>> y[0] = 1
>>> np.testing.assert_array_less(x, y)
Traceback (most recent call last):
    ...
AssertionError:
Arrays are not strictly ordered `x < y`

Mismatched elements: 1 / 3 (33.3%)
Max absolute difference among violations: 0.
Max relative difference among violations: 0.
 x: array([ 1.,  1., nan])
 y: array([ 1.,  2., nan])

Here, y is a scalar, so each element of x is compared to y, and the assertion passes.

>>> x = [1.0, 4.0]
>>> y = 5.0
>>> np.testing.assert_array_less(x, y)

However, with strict=True, the assertion will fail because the shapes do not match.

>>> np.testing.assert_array_less(x, y, strict=True)
Traceback (most recent call last):
    ...
AssertionError:
Arrays are not strictly ordered `x < y`

(shapes (2,), () mismatch)
 x: array([1., 4.])
 y: array(5.)

With strict=True, the assertion also fails if the dtypes of the two arrays do not match.

>>> y = [5, 5]
>>> np.testing.assert_array_less(x, y, strict=True)
Traceback (most recent call last):
    ...
AssertionError:
Arrays are not strictly ordered `x < y`

(dtypes float64, int64 mismatch)
 x: array([1., 4.])
 y: array([5, 5])
dgp.utils.testing.assert_between(value, low, high, low_inclusive=True, high_inclusive=True)

Assert value is inside the specified range. Parameters ———- value : comparable

Value to be asserted.

lowcomparable

Range lower bound.

highcomparable

Range upper bound.

low_inclusivebool, optional

Allow case when value == low. Default: True.

high_inclusivebool, optional

Allow case when value == high. Default: True.

dgp.utils.testing.assert_equal(first, second, msg=None)

Fail if the two objects are unequal as determined by the ‘==’ operator.

dgp.utils.testing.assert_false(expr, msg=None)

Check that the expression is false.

dgp.utils.testing.assert_greater(a, b, msg=None)

Just like self.assertTrue(a > b), but with a nicer default message.

dgp.utils.testing.assert_greater_equal(a, b, msg=None)

Just like self.assertTrue(a >= b), but with a nicer default message.

dgp.utils.testing.assert_less(a, b, msg=None)

Just like self.assertTrue(a < b), but with a nicer default message.

dgp.utils.testing.assert_less_equal(a, b, msg=None)

Just like self.assertTrue(a <= b), but with a nicer default message.

dgp.utils.testing.assert_not_equal(first, second, msg=None)

Fail if the two objects are equal as determined by the ‘!=’ operator.

dgp.utils.testing.assert_raises(expected_exception, *args, **kwargs)

Fail unless an exception of class expected_exception is raised by the callable when invoked with specified positional and keyword arguments. If a different type of exception is raised, it will not be caught, and the test case will be deemed to have suffered an error, exactly as for an unexpected exception.

If called with the callable and arguments omitted, will return a context object used like this:

with self.assertRaises(SomeException):
    do_something()

An optional keyword argument ‘msg’ can be provided when assertRaises is used as a context object.

The context manager keeps a reference to the exception as the ‘exception’ attribute. This allows you to inspect the exception after the assertion:

with self.assertRaises(SomeException) as cm:
    do_something()
the_exception = cm.exception
self.assertEqual(the_exception.error_code, 3)
dgp.utils.testing.assert_true(expr, msg=None)

Check that the expression is true.

dgp.utils.visualization_engine module

dgp.utils.visualization_utils module

Module contents

dgp.utils.tqdm(*args, **kwargs)