owl.mcam_data¶

owl.mcam_data.new_dataset(N_cameras: (<class 'int'>, <class 'int'>)=(9, 6), image_shape: (<class 'int'>, <class 'int'>)=(3120, 4096), dtype=<class 'numpy.uint8'>, dims=('image_y', 'image_x', 'y', 'x'), coords=None, *, array=None, add_ml_index=True)[source]¶

Create a new xarray.Dataset for MCAM data.

Returns an xarray object that contains MCAM data along with the bare minimum metadata. The dataset holds an mcam_data xr.DataArray of size (*N_cameras, *image_shape) with dimensions dims and coordinates coords.

If a coordinate is not provided, a new coordinate is set with np.arange.

Parameters:

N_cameras (tuple of length 2) – Number of cameras that exist in the MCAM.
image_shape (tuple) – The size of the images returned by the individual cameras.
dtype – The dtype of the array that should be allocated.
dims – A list or tuple containing the names of the dimensions.
coords – A dictionary containing the information for the coordinates.
array – if provided, N_cameras, image_shape and dtype will be overwritten to match array. The array will be used as the data in the returned xarray object.
add_ml_index (bool) – If True, a random machine learning index is added to the dataset.

Returns:

mcam_dataset – The dataset containing an 'images' array with associated dimensions.

Return type:

xr.Dataset

owl.mcam_data.new_rgb_dataset(N_cameras=(6, 4), image_shape=(2432, 4320, 3), dtype=<class 'numpy.uint8'>, dims=('image_y', 'image_x', 'y', 'x', 'rgb'), coords=None, array=None)[source]¶

Create a xarray.Dataset containing RGB MCAM data.

By default, this calls new, but passes the parameters

dims=(‘image_y’, ‘image_x’, ‘y’, ‘x’, ‘rgb’) and coords={‘rgb’: [‘r’, ‘g’, ‘b’]}

Returns:: mcam_dataset – An xarray.Dataset with the last dimension having coordinates of 'rgb'.
Return type:: xr.Dataset

directory

owl.mcam_data.load_timelapse_directory(directory, metadata_filename=None, timelapse_tqdm=None, tqdm=None, sort_reverse=None, engine='ramona')[source]¶

owl.mcam_data.to_rgb(mcam_dataarray, bayer_pattern=None)[source]¶

Convert raw data obtained from the mcam to RGB data.

Parameters:

mcam_dataarray – The raw data acquired from the MCAM. Should be an xarray DataArray object that was ideally created with mcam_data.new.
bayer_pattern ([None, 'rggb', 'bggr', 'grbg', 'gbrg']) – If None is provided, then the bayer_pattern is inferred from the bayer_pattern coordinate of the raw_data. In recent versions, this coordinate should be automatically populated if the data was acquired with an MCAM. If bayer_pattern is specified, then the coordinate of the raw_data is ignored and this parameter is passed to bayer2rgb as the assumed color ordering.

Returns:

rgb_dataarray – Deep copy of mcam_dataarray with rgb color mcam_data.

Return type:

xarray DataArray

owl.mcam_data.to_rgba(mcam_dataarray, bayer_pattern=None)[source]¶

Converts raw data obtained from the mcam to RGBA data.

Parameters:

raw_data – The raw data acquired from the MCAM. Should be an xarray object that was ideally created with mcam_data.new.
bayer_pattern ([None, 'rggb', 'bggr', 'grbg', 'gbrg']) – If None is provided, then the bayer_pattern is inferred from the bayer_pattern coordinate of the raw_data. In recent versions, this coordinate should be automatically populated if the data was acquired with an MCAM. If bayer_pattern is specified, then the coordinate of the raw_data is ignored and this parameter is passed to bayer2rgba as the assumed color ordering.

Returns:

rgba_dataarray – Deep copy of mcam_dataarray with rgba color mcam_data.

Return type:

xarray DataArray

owl.mcam_data.resize_as_square(images, axes=(2, 3))[source]¶

Slice your image retaining the center square.

Parameters:: images (array-like) – numpy-like array containing the following leading dimensions [‘image_y’, ‘image_x’, ‘y’, ‘x’].
Returns:: images – With equal dimensions on the ‘y’ and ‘x’ dimensions.
Return type:: array-like

owl.mcam_data.dataset_from_large_image(image, N_cameras=(6, 4), *, tile_shape=None, contiguous=True, trim_edge=False)[source]¶

Creates an mcam_data like Dataset from a large image.

Parameters:

image (array-like, grayscale [N, M], or rgb [N, M, 3], or rgba [N, M, 4]) – The image you wish to chunk up into smaller pieces.
N_cameras ((int, int)) – Tuple of int for the number of image tiles to create in in each dimension.
tile_shape ((int, int)) – Tuple of int for the shape of each tile. If not provided, the tile shape is inferred from the image shape and N_cameras.
contiguous (bool, optional) – If true, will ensure that the resulting array is contiguous in memory.
trim_edge (bool, optional) – If true, will trim away edges to make a smaller image in the case that the chunks do not line up with the original image size.

Returns:

m – mcam_data like xarray.Dataset structure that holds the image data as the variable images.

Return type:

mcam_data

Example

>>> from owl import mcam_data
>>> from imageio import imread
>>> image = imread('large_image_example.tif')
>>> dataset = mcam_data.from_large_image(image, trim_edge=True)
>>> dataset
<xarray.Dataset>
Dimensions:           (image_y: 6, image_x: 4, y: 256, x: 128)
Coordinates:
* image_y           (image_y) int64 0 1 2 3 4 5
* image_x           (image_x) int64 0 1 2 3
* y                 (y) int64 0 1 2 3 4 5 6 7 ... 249 250 251 252 253 254 255
* x                 (x) int64 0 1 2 3 4 5 6 7 ... 121 122 123 124 125 126 127
    __owl_version__   <U29 '0.18.123.dev6+ga658ddd8.dirty'
    __sys_version__   <U80 '3.9.13 | packaged by Ramona Optics | (main, Aug 3...
    __owl_sys_info__  <U674 "{'python': '3.9.13 | packaged by Ramona Optics |...
Data variables:
    images            (image_y, image_x, y, x) float64 0.0 0.0 0.0 ... 0.0 0.0

>>> mcam_data.save(mcam_data, "large_image_chunked", include_timestamp=False)

owl.mcam_data.save_netcdf(mcam_dataset, filename: Path, *, mode='w', engine='ramona', include_timestamp: bool = True, disk_space_tolerance=100000000.0, tqdm=None)¶

Save data as a hdf5 using netcdf4 API.

Default extension is nc if none is given

Returns the saved path name as a Path object.

Parameters:

mcam_dataset (xarray Dataset) – The mcam_data you wish to save.
filename (Path-like) – The filename where the data should be saved. If the filename has no extension, then an '.nc' extension is added.
mode (str) – The mode to open the file in. Valid options are 'w' (write), or `x` exclusive creation, failing if the file already exists.
engine – Parameter passed to xarray.Dataset.to_netcdf to select the backend used for writing data to disk.
include_timestamp (bool) – If set to True, this will append a timestamp to the provided path name. If set to false, no timestamp will be appended to the path name possibly overwriting any files currently within the previous path name.
disk_space_tolerance (float) – The difference between the free disk space in the given directory and the dataset to be save in bytes. If the directory has fewer free bytes than this tolerance plus the dataset size an error will be raised.
single_file –

Changed in version 0.14.0: In version 0.14.0 this parameter is ignored, and the behavior is always that of single_file=True

Changed in version 0.18.9: In version 0.18.9 this parameter will emit a deprecation warning indicating that it will be removed in version 0.20.0

Returns:

filename – The filename where the the data has been stored. It includes the optional timestamp, and the new suffix.

Return type:

Path-like

owl.mcam_data.save_video_netcdf(mcam_dataset, filename: Path, *, include_timestamp: bool = True, engine='ramona', disk_space_tolerance=100000000.0, tqdm=None, mode='w')¶

Save datasets that contain stacks and provide progress information.

This method provides an optimized way to save datasets for speed of writing and reading. It also provides the user with feedback during data writing through the form of an optional progress bar.

Parameters:

mcam_dataset (xarray Dataset) – The mcam_data you wish to save.
filename (Path-like) – The filename where the data should be saved. If the filename has no extension, then an '.nc' extension is added.
include_timestamp (bool) – If set to True, this will append a timestamp to the provided path name. If set to false, no timestamp will be appended to the path name possibly overwriting any files currently within the previous path name.
disk_space_tolerance (float) – The difference between the free disk space in the given directory and the dataset to be save in bytes. If the directory has fewer free bytes than this tolerance plus the dataset size an error will be raised.
mode (str) – The mode to open the file in. Valid options are 'w' (write), or `x` exclusive creation, failing if the file already exists.

Returns:

The path where the data was saved.

Return type:

save_filepath

owl.mcam_data.load_netcdf(filename: Path, *, delayed=True, scheduler='threading', progress=True, update=True, engine='ramona', chunks=None)¶

Load mcam_data from a hdf5 file using netcdf4 and convert it to whatever we want.

Parameters:

filename (Path-like) – The netcdf4 file where the data is stored.
delayed (bool, optional) – If True, the computation will return a lazy object. As the user of this library, you will have to explicitly force the computation of the lazy object. This function will attempt to automatically determine if the data should be loaded lazily or eagerly. If a particular behavior is desired in your application, the delayed parameter should be specified.
progress – If True, then a progress bar is shown during loading operations. This is only valid if delayed=True.
scheduler – Parameter passed to dask.compute to select which schedule is used to load the data in parallel.
update – Set to False if you wish to load the raw un-updated data. Setting this parameter to False keeps the raw metadata as is in the .nc file but means that the loaded dataset is unsupported by the remainder of the owl analysis functions.
engine – xarray engine used to open the dataset.
chunks – A dictionary of the image axes to chunk and the size of the chunks along the axes. Axes should be referenced by their coordinate name. This is best used in conjunction with delayed=True

Returns:

mcam_dataset – Return the MCAM data in an xarray DataArray with all the metadata.

Return type:

DataArray

owl.mcam_data.append_netcdf(filename, ds_to_append, unlimited_dims, *, engine=None)¶

Append dataset to netCDF4 file.

Append the provided dataset to the file along the unlimited_dim.

Parameters:

filename (Path-like or File-like object) –
The path to the file where the data should be written or alternatively, a file object from netCDF4 or h5netcdf. If filename is a File-like handle, the engine parameter is ignored.

Changed in version 0.18.15: The filename parameter can now accept a file object.
ds_to_append (xarray.Dataset) – xarray dataset to append to the filename.
ulimited_dims (str or List[str]) – Dimension over which to append the dataset to the file. Currently, only one dimension is supported.
engine (str) –
The name of the backend to use to write the file. Should be one of "netcdf4", "h5netcdf", or "ramona".

Added in version 0.18.15: The engine parameter.

owl.mcam_data.get_photometric_corrections(mcam_dataset, illumination_type)[source]¶

Get all all color corrections available for a calibrated system.

Parameters:

mcam_dataset (xarray.Dataset) – An xarray.Dataset of mcam_data.
illumination_type (string ('reflection', 'transmission')) – Which illumination board to use to illuminate the sensors.

Returns:

response_matrix (numpy array) – The matrix that describes the sensor’s response to the leds. It is a M x N x 4 x 4 array of float where M and N are the shape of the sensor array.
coefficient_corrections (numpy array) – Array of pixel correction coefficients of shape M x N x image_shape.
offset_corrections (numpy array) – Array of pixel correction offsets of shape M x N x image_shape.

owl.mcam_data.get_photometric_sensor_corrections(mcam_dataset, illumination_type)[source]¶

Get the matrix used to create the average value of the individual images.

Parameters:

mcam_dataset (xarray.Dataset) – An xarray.Dataset of mcam_data.
illumination_type (string ('reflection', 'transmission')) – Which illumination board to use to illuminate the sensors.

Raises:

ValueError – Unable to get correction if photometric calibration data is not in mcam_dataset.

Returns:

response_matrix – The matrix that describes the sensor’s response to the leds. It is a M x N x 4 x 4 array of float where M and N are the shape of the sensor array.

Return type:

numpy array

owl.mcam_data.bayer_dataset_to_single_channel(dataset, color)[source]¶

Extract a fixed color pixel from data acquired with CFA sensors.

Given data acquired with a sensor contained a color filter array, this function extracts a single monochrome pixel from the array.

This function can help reduce the data analysis load and speed up algorithms that work will with monochromatic images.

While a standard Bayer pattern has one red pixel, and one blue pixel, it contains two green pixels. We denote the green pixel on the first row as the 'green0' pixel, and the green pixel on the second row as the 'green1' pixel. Here 'green' is shorthand for 'green0'.

Parameters:

dataset (mcam_dataset) – Dataset containing MCAM images. The dataset must contain the bayer_pattern that describes the pixel ordering in the color filter array.
color – Color to extract. Must be one of: ['red', 'green', 'blue', 'green0', 'green1']

Returns:

The returned dataset no longer contains the bayer_pattern variable to indicate that it has been converted to that of a monochromatic image.

Return type:

dataset

Notes

The images in the dataset must all have been acquired with the same pattern for the color filter array.

owl.mcam_data.bayer_dataset_to_rgb(dataset)[source]¶

Convert a dataset acquired with a bayer sensor to an RGB dataset.

Given data acquired with a sensor contained a color filter array, this converts the raw data to that of an RGB image.

Parameters:: dataset (mcam_dataset) – Dataset containing MCAM images. The dataset must contain the bayer_pattern that describes the pixel ordering in the color filter array. The images variable will contain a new dimension labeled with ‘rgb’ with dimension of 3.
Returns:: The returned dataset no longer contains the bayer_pattern variable to indicate that it has been converted to that of a monochromatic image.
Return type:: dataset

owl.mcam_data.bayer_dataset_to_grayscale(dataset, *, gray_vector=None)[source]¶

Convert a dataset acquired with a bayer sensor to a grayscale dataset.

Given data acquired with a sensor contained a color filter array, this converts the raw data to that of an RGB image.

Parameters:: dataset (mcam_dataset) – Dataset containing MCAM images. The dataset must contain the bayer_pattern that describes the pixel ordering in the color filter array. The images variable will contain the new grayscale dataset. All other metadata will be retained.
Returns:: The returned dataset no longer contains the bayer_pattern variable to indicate that it has been converted to that of a monochromatic image.
Return type:: dataset

owl.mcam_data.rgb_dataset_to_grayscale(dataset, *, gray_vector=None)[source]¶

Convert a debayered rgb dataset to a grayscale dataset.

Parameters:

dataset (mcam_dataset) – Dataset containing MCAM images. The dataset must contain the dimension rgb. The images variable will contain the new grayscale dataset. All other metadata will be retained.
gray_vector (tuple) – A 3-tuple of floats that describes the weights of the red, green, and blue channels in the conversion to grayscale. The default values are the ITU-R BT.709-1 standard.

Returns:

The returned dataset no longer contains the rgb dimension indicating that it has been converted to that of a grayscale image.

Return type:

dataset

owl.mcam_data.get_analysis_outputs(dataset)[source]¶

owl.mcam_data.get_software_frame_rate(dataset, *, frame_number_coordinate='frame_number')[source]¶

Compute the average frame rate of the data acquired in the dataset.

Using the software timestamp, compute the frame rate of the acquired dataset.

Parameters:

dataset (mcam_dataset) – Dataset containing mcam_data and software_timestamp.
frame_number_coordinate (str) – Coordinate over which to compute the frame rate. The result is averaged over all other coordinates.

Returns:

frame_rate – Average frame rate over all cameras in seconds.

Return type:

float

owl.mcam_data.get_software_frame_time_difference(dataset, *, frame_number_coordinate='frame_number')[source]¶

Compute the average time difference between subsequent frames.

Using the software timestamp, compute the frame rate of the acquired dataset.

Parameters:

dataset (mcam_dataset) – Dataset containing mcam_data and software_timestamp.
frame_number_coordinate (str) – Coordinate over which to compute the frame rate. The result is averaged over all other coordinates.

Returns:

time_difference – Average time difference over all cameras in seconds between each subsequent frame.

Return type:

float

owl.mcam_data.get_bayer_pattern(dataset, default=None)[source]¶

Return the bayer pattern of the underlying sensors.

Parameters:

dataset (mcam_data) – A dataset containing MCAM data. The bayer pattern is expected to be found in a key called bayer_pattern.
default – The default value in the case that the bayer_pattern key is not found in the dataset.

Returns:

bayer_pattern – The bayer pattern as a string. Typical bayer patterns include "rggb", "bggr", "grbg", "gbrg".

Return type:

str

owl.mcam_data.get_bin_mode(dataset)[source]¶

Return the binning mode of the dataset

Parameters:: dataset (mcam_data) – A dataset containing MCAM data. The pixel information is expected to be contained in the coordinates 'y' and 'x'.
Returns:: bin_mode – A single integer for the bin mode.
Return type:: int

Note

The binning mode is assumed to be symmetric in both the row (y) and column (x) dimension. This function uses the information in the y coordinate to extract the bin mode.

owl.mcam_data.get_chroma(dataset)[source]¶

owl.mcam_data.get_groupings(dataset, key, trim_invalid_data=True)[source]¶

Extracts groupings of images based on a given key from a dataset.

Parameters:

dataset (xarray) – The MCAM dataset containing image data.
key (str) – The key in the dataset to use for grouping images.
trim_invalid_data (bool) – Flag to indicate whether to exclude invalid data when generating groupings.

Returns:

groupings – A dictionary where keys are group identifiers and values are lists of tuples containing image indices within the dataset.

Return type:

dict

owl.mcam_data.get_offset(dataset)[source]¶

Return the offset of the dataset

Parameters:: dataset (mcam_data) – A dataset containing MCAM data. The pixel information is expected to be contained in the coordinates 'y' and 'x'.
Returns:: bin_mode – A integer for the offset in the y and x direction.
Return type:: tuple(int, int)

owl.mcam_data.get_stack_dims(dataset, *, key='images')[source]¶: Can return one or more dimensions in which the stack was acquired.

owl.mcam_data.get_stack_dimension(dataset)[source]¶

owl.mcam_data.get_valid_data(dataset)[source]¶

Get the valid data from the dataset.

If the dataset contains a variable named valid_data, then the variable is used to determine the valid data. If not, then a boolean array of the same shape as the images is returned.

owl.mcam_data.is_stack_dataset(dataset)[source]¶

owl.mcam_data.is_well_dataset(dataset)[source]¶

owl.mcam_data.group_data(dataset, key, groupings, func=<function mean>)[source]¶

This function groups data from a dataset based on specified groupings and calculates a summary statistic for each group.

Parameters:

dataset (xarray) – The MCAM dataset containing the data to be grouped.
key (str) – The key column in the dataset based on which the data will be grouped.
groupings (dict) – A dictionary where the keys represent group IDs and the values are lists of image indices for each group.
func (function) – The function to apply to the grouped data to calculate the summary statistic.

Returns:

grouping_values – A dictionary where the keys are group IDs and the values are the calculated summary statistics for each group.

Return type:

dict

owl.mcam_data.using_field_id(dataset)[source]¶

Check if a dataset is using field IDs (i.e., has non-zero field IDs).

Parameters:: dataset (xarray.Dataset) – The dataset to check.
Returns:: True if the dataset has field IDs and at least one non-zero field ID. False otherwise.
Return type:: bool

owl.mcam_data.trim_to_valid(dataset)[source]¶

Trim the dataset to the valid data.

If valid_data is contained within the dataset, then the dataset is sliced such that the removed data contains only invalid data.

Basic slicing is supported, such as contiguous slices, or slices with steps. The slicing is done using the image_y and image_x dimensions of the dataset.