Vireo lighting photometric correction

Similar to the calibrations described in RGB lighting color correction and IR (850 nm) lighting color correction, calibration is also done for the Vireo’s illumination. The Vireo pairs monochrome sensors with a single-wavelength emitter, so each camera records one channel and the illumination is set by a single intensity. This again allows consistency so that a pixel value in one location of the array can be compared to another section of the array in confidence.

The Vireo’s illumination provides multiple illumination modes. Each mode places a different distribution of light on the sample, so the procedure described below is followed once for every mode, producing a correction unique to that mode.

We begin by denoting each camera in the array with a 2-dimensional index, \((i, j)\).

During calibration a sensor response matrix is created by collecting the average pixel response of the sensors to varying illumination intensities. For each measurement, an intensity of the emitter, defined as value, \(l\), is applied and one image is acquired from each of the cameras in the array. The average pixel value is recorded as a single value \(p\) for each sensor \((i, j)\). The measurement is repeated \(n\) times, each time varying the intensity of the light \(l\) and recording \(p\). We give each component above a superscript to denote the index in the acquisition when it was acquired.

Together, these \(n\) measurements can be grouped in a matrix equation written as:

\[\begin{split}\begin{bmatrix} p^0 & p^1 & p^2 & \dots & p^n \end{bmatrix}_{i,j} = \begin{bmatrix} r & b \end{bmatrix}_{i,j} \begin{bmatrix} l^0 & l^1 & l^2 & \ldots & l^n \\ 1 & 1 & 1 & \ldots & 1 \end{bmatrix}\end{split}\]

Where \(p\) is sensor (i, j) average pixel value for \(n\) measurements. \(l\) is the emitter’s power.

The response matrix’s elements are the following:

\(r\) - the sensor’s average response to the illumination.

\(b\) - the offset term of the fit.

The illumination is not driven below the minimum power of its LED driver. Below that point the driver’s output is undefined, so the light reaching the sample is no longer a predictable function of the requested power. This region is avoided during acquisition as well as during calibration. It is also why the offset term is not taken from the fit: when the power that saturates the sensors is low, part of the sweep falls near this minimum, and an intercept fit through those measurements describes the driver’s behavior rather than the sensors’. The offset is set to zero instead.

\[b = 0\]

This can be abstracted to:

\[P_{i,j} = R_{i,j}L\]

Corrections for the sensor responses are created by finding the element by element average of the responses and multiplying by the individual response matrix’s pseudo-inverse.

\[C_{i,j} = R' R_{i,j}^{-1}\]

Where \(C_{i,j}\) is the correction matrix. R' is the average response matrix across sensors, and \(R_{i,j}^{-1}\) is the pseudo-inverse of \(R_{i,j}\). With a single response term this amounts to a gain per camera that brings each sensor to the average response of the array.

Along with this, each sensor’s response is not uniform across all pixels. To account for this surfaces are defined to model the variable corrections of the sensors to the illumination source.

These surfaces are measured with a second intensity sweep. The sensor corrections described above are applied to the data as it is collected, so that what remains to be modeled is the variation within each sensor rather than the differences between them. At every intensity one image is acquired from each camera and divided into a grid of square tiles, and the mean pixel value of each tile is recorded. Each tile is then fit against the illumination intensity in the same manner as the sensor response above, giving a response coefficient and an offset for every tile.

A surface is fit to the grid of tile coefficients and a second surface to the grid of tile offsets. These are modeled by the following polynomial:

\[S_{i,j}(x, y) = a + by + cy^{2} + dx + exy + fx^{2}\]

Where a, b, c, d, e and f are coefficients and x and y define the pixel’s position in the sensor

The coefficient surface is normalized by its own mean and the offset surface has its mean subtracted, so that the surfaces describe how the response varies across a sensor rather than its overall level, which is already set by \(C_{i,j}\).

Applying the sensor corrections and the pixel corrections follow the following formula:

\[P'_{i,j}(x, y) = S_{(i,j),coefficient}(x, y)((C_{i,j} P_{i,j}(x, y)) + S_{(i,j),offset}(x, y))\]

Where \(P'_{i,j}(x, y)\) is the corrected pixel at position (x, y) on sensor (i, j). \(S_{i,j,coefficient}(x, y)\) is the multiplicative correction for sensor (i, j) at pixel position (x, y). \(P_{i,j}(x, y)\) is the original pixel value at position (x, y) on sensor (i, j), and \(S_{i,j,offset}(x, y)\) is the additive correction for sensor (i, j) at pixel position (x, y).

Because of this correction pixel value \(P'_{i,j}(x, y)\) can be compared to \(P'_{k,l}(w, z)\) with confidence.