Abstract

Plankton interact with the environment according to their size and three-dimensional (3D) structure. To study them outdoors, these translucent specimens are imaged in situ. Light projects through a specimen in each image. The specimen has a random scale, drawn from the population’s size distribution and random unknown pose. The specimen appears only once before drifting away. We achieve 3D tomography using such a random ensemble to statistically estimate an average volumetric distribution of the plankton type and specimen size. To counter errors due to non-rigid deformations, we weight the data, drawing from advanced models developed for cryo-electron microscopy. The weights convey the confidence in the quality of each datum. This confidence relies on a statistical error model. We demonstrate the approach on live plankton using an underwater field microscope.

© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. INTRODUCTION

It is important to study plankton for their role in both local and global ecosystems. These organisms are vital to the ecology of the planet [1], as they play a critical role in producing breathable oxygen (50% due to marine phytoplankton) and serve as the base of the marine food web. More recently, an increase in harmful algal blooms has been devastating to marine and freshwater supplies [2,3], while having a negative impact on coastal ecosystems.

Over decades, the basic taxonomy of plankton has been established both by morphological observations and by DNA studies that allow unique attribution. Modern oceanographers increasingly rely on in situ imaging systems to study populations in their natural habitat. However, almost all available inferences about plankton morphology derived from these instruments rely on two-dimensional (2D) projections [48] with the third dimension being conjectured. The determination of three-dimensional (3D) structure is important as it defines the surface-area-to-volume ratio, which is fundamental to the absorption and diffusion of nutrients and waste products. Also, flow is linked to this ratio, through the Reynolds number [9]. In addition to the 3D structure, it is important to assess the size distribution of the population (Fig. 1).

 figure: Fig. 1.

Fig. 1. Specimens viewed in nature have random sizes and poses. Light passes through, yielding a single 2D image per specimen. The goal is to tomographically recover the 3D volumetric distribution of the plankton (on average) and estimate the population size distribution.

Download Full Size | PPT Slide | PDF

Recent advances in 3D imaging focus on lab systems [1012]. However, many plankton are easily deformed by collection and are, thus, difficult to study in the lab. There are, therefore, a host of ocean and lake systems [1315] to observe these organisms in situ. These systems look outward from their housing into the surrounding water. In the majority of systems, imaging is accomplished via the presence of organisms that simply pass through their field of view.

In order to image in relatively high resolution small plankton ($\lt\!{100}\;{\rm micrometers}$), there are four kinds of common in situ imaging systems that can be submerged and operated [16,17]. (i) The Imaging FlowCytobot (IFCB) [15] uses either hydrodynamic or acoustic particle focusing. This results in particles being positioned in the volume in focus, flowing through a chamber. In cases of more spherical particles, IFCB captures a single shot of a randomly sized organism [18] at a random pose. Other systems of relevance to our study are based either on (ii) holography [1921], (iii) shadowgraph illumination [7,22], or (iv) dark-field microscopy.

 figure: Fig. 2.

Fig. 2. (a) Illustration of the Scripps Plankton Camera (SPC). (b) An image of the SPC in situ. The dark-field microscope’s sample volume is between the ports.

Download Full Size | PPT Slide | PDF

Holography and shadowgraph imaging are viable modalities. However, we found several advantages to dark-field microscopy [13]: (a) Dark-field imaging is common, low cost, and robust. (b) Our experience with on-axis holography indicates that the point spread function is range-dependent (a quadratic phase effect) that generally yields lower contrast than dark-field. We also found, in the case of holography, difficulty in yielding viable depth information at the thicknesses of the organisms that we study. A dark-field image is a simple projection rather than a phase measurement. Projected images can provide information that can yield 3D structure using tomography, as we describe in this paper. (c) Small scattering particles in turbid waters, as are commonly found in coastal regions, yield random localized specks in dark-field images. This randomness can be filtered out by tomography (which is based on multiple images). On-axis holography (the practical basis for digital holography) responds to scatterers by non-local speckled patterns, which reduce overall contrast. (d) Dark-field illumination enhances edges of objects whose refractive index is close to that of the surrounding medium. This leads to better resolved appendages and internal structures of plankton. We, thus, chose to work with dark-field optical microscopes to maximize taxonomic resolution, enhance edge contrast, and ensure high performance in scattering-rich environments.

Underwater computer vision has benefited from a dramatic increase in computing capabilities, addressing image quality [2327] and underwater robotics [2833]. It is fitting that the benefits of computational imaging should be applied to in situ study of plankton. Tomography is a computational method that we use. Usually, tomography is performed around a single, temporally fixed object [3438]. Here we are interested in drawing structure and statistics about a population. The data are an ensemble of random views of random objects within a population, each having a different random and unknown scale, orientation, and location (see Fig. 1). A basis to our solution is single particle reconstruction (SPR) [39], which is used in cryo-electron microscopy (Cryo-EM) [40,41]. SPR handles random rotations but not scale variation in a population. An attempt to generalize SPR to handle variable scales was done in Ref. [42], where the approach termed 3D-POP is defined. It is based on pairwise constraints on rotations and scales. However, the work in Ref. [42] is prone to significant estimation errors. Hence, our paper employs a more robust SPR approach [43] and generalizes it to analyze data subject to scale variations. The proposed method uses a set of triplets of images to assess how indicative are constraints derived from image pairs. Then, this assessment is used to weight each constraint. The constraints are then efficiently and optimally fused using linear algebra.

We show the effectiveness of the method on simulated data and publically available bright-field images from the IFCB. We also show it using dark-field images of plankton acquired by our in situ system: the Scripps Plankton Camera (SPC). SPC is an in situ dark-field microscope developed to continuously monitor plankton populations [44] (Fig. 2). It is a free-space imaging system, with the light and camera focused on a plane in an open volume of fluid. The SPC uses a white LED illumination source with the central rays blocked by an opaque disk. Objects in the focal plane, thus, scatter light toward the image plane while the direct light continues past the lens, thus creating the dark-field effect. The paper uses images from two test SPC datasets. Both were taken by a version of the SPC outfitted with a ${0.5} \times$ microscope objective, yielding an object sampling interval of 7.4 µm/pixel with adequate contrast at 30 µm. All presented SPC plankton samples are about 0.5–2.5 mm long. The object sampling interval of the IFCB is 0.29 µm/pixel, enabling imaging of objects in an order of magnitude smaller than the SPC. All plankton samples were manually classified by an expert taxonomist.

2. BACKGROUND ON SINGLE PARTICLE RECONSTRUCTION

A. Tomography Using Random Projections

We denote 3D spatial coordinates by ${\textbf x} = ({x_1},{x_2},{x_3}{)^{T}}$, where ${(\cdot)^{T}}$ denotes transposition. A specimen has a volumetric optical extinction distribution $\beta ({\textbf x})$. In dark-field imaging, $\beta ({\textbf x})$ expresses the density of scattering particles in the object. In bright-field imaging, $\beta ({\textbf x})$ represents the optical density of attenuating media.

Computational tomography (CT) estimates $\beta ({\textbf x})$ using a set of $N$ multi-view orthographic projections. Relative to the camera, the sample orientation in the $n$th image ${{\textbf y}_n}$ is expressed by a 3D rotation matrix ${{\textbf R}_n}$. Orthographic projection through the translucent sample is expressed by linear integration along the ${x_3}$ axis,

$${{\textbf y}_n} = {\cal P}_{{x_1},{x_2}}^{(n)}\{\beta\} = \int \beta \left({{\textbf R}_n^{- 1}{\textbf x}} \right){\rm d}{x_3},$$
where ${x_1}$, ${x_2}$ are image axes. Organisms have a range of scattering characteristics. For the sake of our analysis, let the objects be weak scatterers, as commonly assumed in microscopy restoration problems [45,46]. Images are, thus, approximately compliant with the linear model of Eq. (1).

There are strong and widely used methods for estimating a volumetric distribution $\beta ({\textbf x})$ using tomography based on such a set of projections $\{{{\textbf y}_n}\} _{n = 1}^N$. This paper, thus, does not focus on estimating $\beta ({\textbf x})$. We note, however, that common tomographic methods [4754] assume that the set of $N$ orientation matrices $\{{{{\textbf R}_1}, \ldots ,{{\textbf R}_N}} \}$ is known. Moreover, common tomography assumes that the specimen itself does not change between projections.

A random biological microscopic specimen collected in situ does not comply with these assumptions. Its orientation is random and unknown. Furthermore, each projection image corresponds to a different specimen, hence having random, unknown scale and lateral displacement. This paper focuses on recovering the unknown scale, displacement, and rotation of each specimen, in order to facilitate tomographic recovery of the 3D structure.

B. 3D-POP

There is a prior method for estimating the unknown scale, displacement, and rotation of each specimen: 3D-POP [42]. Our paper presents methods that yield better efficiency, accuracy, and robustness. We briefly describe the main elements of 3D-POP, drawing significantly on its notations.

Analysis is done in the 3D spatial frequency domain ${\textbf k} = ({k_1},{k_2},{k_3})$. The 3D Fourier transform of the volumetric object is ${{\cal F}_{\textbf k}}\{\beta \}$. A central slice is a frequency-plane in the 3D Fourier space that passes through the origin, ${\textbf k} = 0$. Any central slice is a 2D domain, which can be written in terms of a rotated horizontal slice,

$${\textbf R}_n^{- 1}{[{k_1}, {k_2}, 0]^{T}},$$
where ${{\textbf R}_n}$ is a 3D rotation matrix indexed $n$, as in Eq. (1). Figure 3 illustrates 3D Fourier space (3D k-space) and two central slices $n$, $m$ in this space. These slices, respectively, represent the planar domains ${\textbf R}_n^{- 1}{[{k_1}, {k_2}, 0]^{T}}$ and ${\textbf R}_m^{- 1}{[{k_1}, {k_2}, 0]^{T}}$ in 3D k-space.
 figure: Fig. 3.

Fig. 3. 3D Fourier domain and two central slices ${{{\tilde {\textbf y}}}_n},{{{\tilde {\textbf y}}}_m}$. Each central slice is perpendicular to the direction-vector of the respective projection in the microscope. The two slices intersect at a common tilt axis (common line) in the 3D Fourier domain.

Download Full Size | PPT Slide | PDF

Now let us consider the 2D Fourier transform of projection image ${{\textbf y}_n}$. Denote the resulting transformed representation by ${{{\tilde {\textbf y}}}_n}(\rho ,\psi)$, where $\rho$, $\psi$ are polar coordinates in the 2D spatial frequency domain. According the Central Slice Theorem, the 2D function ${{{\tilde {\textbf y}}}_n}$ is equivalent to the central slice ${\textbf R}_n^{- 1}{[{k_1}, {k_2}, 0]^{T}}$ in 3D $ k $-space Eq. (2).

Any two central slices $ n $, $ m $ intersect at a common line in 3D k-space (Fig. 3). Correspondingly, this line in 3D $ k $-space is equivalent to a radial line in the 2D function ${{{\tilde {\textbf y}}}_n}$ at an angle denoted by $\psi = {\psi _{n \to m}}$. It is also equivalent to a radial line in the 2D function ${{{\tilde {\textbf y}}}_m}$, at an angle denoted by $\psi ^\prime = {\psi _{m \to n}}$.

Within these 2D functions, finding the angle pair ${\psi _{n \to m}}$, ${\psi _{m \to n}}$, which corresponds to a common line, yields constraints [41] about the 3D rotations ${{\textbf R}_n}$, ${{\textbf R}_m}$. The constraints are

$$\left[{\begin{array}{* {20}{c}}{\cos {\psi _{n \to m}}}\\{\sin {\psi _{n \to m}}}\\0\end{array}} \right] = {\textbf R}_n^{- 1}{{\textbf R}_m}\left[{\begin{array}{* {20}{c}}{\cos {\psi _{m \to n}}}\\{\sin {\psi _{m \to n}}}\\0\end{array}} \right].$$
Then there are two challenges:
  • (a) Estimating all projection rotations $\{{{\textbf R}_n}\} _{n = 1}^N$ using constraints as in Eq. (3) is a key for subsequent tomographic recovery of $\beta ({\textbf x})$.
  • (b) To yield constraint as Eq. (3), the angle pair ${\psi _{n \to m}}$, ${\psi _{m \to n}}$ must be estimated reliably and efficiently for many real-data image-pairs. This challenge is exacerbated by the fact that $\beta ({\textbf x})$ is randomly shifted and scaled between images of different organisms collected in situ.

Let us start with challenge (b). In the 2D Fourier domain, the one-dimensional (1D) function on a radial line ${{{\tilde {\textbf y}}}_n}(\rho ,{\psi _{n \to m}})$ is expected to highly correlate with the radial function ${{{\tilde {\textbf y}}}_m}(\rho ,{\psi _{m \to n}})$ since both correspond to the same common line. The correlation should generalize to account for a relative scale ${M_{n \to m}}$ between the specimens viewed in images $n$ and $m$. Relative spatial scale is expressed as an inverse-scale in the Fourier domain. The correlation should also generalize to account for a lateral shift ${\tau _{n \to m}}$ of the specimen along the common line direction. This shift yields a linear phase $2\pi \rho {\tau _{n \to m}}$ in the Fourier domain. Consequently, in Ref. [42] the Fourier domain radial correlation is defined by an inner product of scale and phase-shifted 1D functions sampled along two directions, $\psi ,\psi ^\prime $, using variable scale $M$ and lateral shift $\tau$:

$${\Sigma _{nm}}(\psi ,\psi ^\prime, M,\tau) = \int\limits_{- {\rho _{\max}}}^{{\rho _{\max}}} {{\tilde {\textbf y}}}_n^*\left({\frac{\rho}{M},\psi} \right){{{\tilde {\textbf y}}}_m}(\rho ,\psi ^\prime){e^{2\pi i\rho \tau}}{\rm d}\rho .$$
Here $[- {\rho _{\max}},{\rho _{\max}}]$ is the effective radial domain of the functions, and $i = \sqrt {- 1}$. Reference [42] estimates the angle pair, relative scale, and relative shift by
$$\begin{split}&\{{{\hat \psi}_{n \to m}},{{\hat \psi}_{m \to n}},{{\hat M}_{n \to m}},{{\hat \tau}_{n \to m}}\} \\ &\quad = \mathop {{\rm argmax}}\limits_{\psi ,\psi ^\prime, M,\tau} \frac{{{\Sigma _{nm}}(\psi ,\psi ^\prime, M,\tau)}}{{\sqrt {{\Sigma _{nn}}(\psi ,\psi, M,\tau)} \sqrt {{\Sigma _{mm}}(\psi ^\prime ,\psi ^\prime, M,\tau)}}}.\end{split}$$
We now describe challenge (a). Eqs. (3) and (5) yield pairwise relations on relative rotations, scales, and translations. Using a large number of images, these relations are then compounded to yield overall solutions to pose and scale of each specimen in a global coordinate system. Reference [42] found ${\hat M_{n \to m}}$ empirically in the pairwise optimization [Eq. (5)]. The unknown scale of specimen $n$ is ${M_n}$. Denote ${\bar M_n} = \log ({M_n})$. Then, ${\bar M_n} - {\bar M_m} = \log ({\hat M_{n \to m}})$.

Consequently, a data term that penalizes scale inconsistencies is

$${{\cal D}^{{\rm scale}}}\left({\{{{\bar M}_n}\} _{n = 1}^N} \right) = \sum\limits_{n \ne m} {\left| {{{\bar M}_n} - {{\bar M}_m} - \log {{\hat M}_{n \to m}}} \right|^2}.$$
Adding a simple regularization term ${{\cal R}^{{\rm scale}}}$, Ref. [42] then solves linearly the optimization problem,
$$\{{\hat {\bar M}}_n\} _{n = 1}^N = \arg \mathop {\min}\limits_{{{\bar M}_n}} \left[{{{\cal D}^{{\rm scale}}}\left({\{{{\bar M}_n}\}} \right) + {{\cal R}^{{\rm scale}}}\left({\{{{\bar M}_n}\}} \right)} \right].$$
The scale of specimen $n$ is ${\hat M_n} = \exp {\hat {\bar M}}_n$. A similar approach optimizes the translations of all specimens [42], using ${\hat \tau _{n \to m}}$ obtained empirically in the pairwise optimization [Eq. (5)].

1. Problems That Persist

There are several problems with Eqs. (4) and (5) and their implementation. In Ref. [42], Eq. (5) was done by exhaustive search over a four-dimensional (4D) domain of variables. To make the search space manageable, Ref. [42] used a coarse-to-fine approach starting from a sparse sampling of the domain of variables. Equation (5) is performed on this small sparse set. This results in a narrower 4D domain to be searched in finer resolution. However, noise and other factors may lead to errors in Eq. (5). An error occurring in a coarse-sampling stage can lead to intolerable errors that cannot be solved in later, finer-scale sampling. In effect, robustness and accuracy are sacrificed for numerical tractability.

The model in Eqs. (4) and (5) assumes a similarity relation between specimens. It does not account for non-rigid deformations, which are common in natural non-rigid objects in a population. A non-rigid deformation is a perturbation that may lead to deviation from the estimated pose variables. It may also lead to an inaccurate solution of Eq. (5): by chance, correlation of an arbitrary orientation scale and translation can yield better correlation [Eq. (4)] than the correct pose due to the deformation. Hence, a major error in pose propagates to major errors in tomographic recovery. In this paper, we use a more robust approach.

The biggest challenge is that of recovering the orientation. Based on the pairwise relations as in Eqs. (3) and (5), all rotations were estimated in Ref. [42] using a computationally complex, non-convex, constrained optimization as follows:

$$\begin{split}\{{{{\hat {\textbf R}}}_n}\} _{n = 1}^N &= \mathop {\rm argmin}\limits_{{{\textbf R}_1}, \ldots,{{\textbf R}_N}}\sum\limits_{n \ne m} \left\| {{{\textbf R}_n}\left[{\begin{array}{* {20}{c}}{\cos {\psi _{n \to m}}}\\{\sin {\psi _{n \to m}}}\\0\end{array}} \right] - {{\textbf R}_m}\left[{\begin{array}{* {20}{c}}{\cos {\psi _{m \to n}}}\\{\sin {\psi _{m \to n}}}\\0\end{array}} \right]} \right\|_2^2\\&\quad {\rm s.t.}\;{{\textbf R}_n}{\textbf R}_n^{T} = {\textbf I},\det ({{\textbf R}_n}) = 1,n = 1, \ldots, N.\end{split}$$
Here ${\textbf I}$ is a $3 \times 3$ identity matrix [41].

3. STATISTICAL MODEL FOR ROBUST PLANKTON RECONSTRUCTION

A. Estimating Rotations

To mitigate this complexity, we describe a numerically efficient method that solves $\{{{{\hat {\textbf R}}}_n}\} _{n = 1}^N$ given pairwise relations. Contrary to Eq. (8), the method here is not formulated as a non-convex problem or using approximate relaxation. Instead, it uses a linear algebra approach [43] to achieve accurate results quickly. Define

$${{\textbf R}_{m \to n}} \equiv {{\textbf R}_n}{\textbf R}_m^{- 1}.$$
Use the angle pair ${\hat \psi _{n \to m}}$, ${\hat \psi _{m \to n}}$ and a matrix ${{\textbf R}_{m \to n}}$, which satisfies Eqs. (3) and (9) as found as in Ref. [55]. Then, define matrices
$${\textbf S} = \left[{\begin{array}{* {20}{l}}{\textbf I}&\quad{{{\textbf R}_{1 \to 2}}}& \quad\ldots &\quad{{{\textbf R}_{1 \to n}}}& \quad\ldots &\quad{{{\textbf R}_{1 \to N}}}\\{{{\textbf R}_{2 \to 1}}}&\quad{\textbf I}& \quad\ldots &\quad{{{\textbf R}_{2 \to n}}}& \quad\ldots &\quad{{{\textbf R}_{2 \to N}}}\\{\vdots}&\quad{\vdots}& \quad\cdots &\quad{\vdots}& \quad\cdots &\quad{\vdots}\\{{{\textbf R}_{m \to 1}}}&\quad{{{\textbf R}_{m \to 2}}}& \quad\ldots &\quad{{{\textbf R}_{m \to n}}}& \quad\ldots &{{{\textbf R}_{m \to N}}}\\{\vdots}&\quad{\vdots}& \quad\cdots &\quad{\vdots}& \quad\cdots &\quad{\vdots}\\{{{\textbf R}_{N \to 1}}}&\quad{{{\textbf R}_{N \to 2}}}& \quad\ldots &\quad{{{\textbf R}_{N \to n}}}& \quad\ldots &\quad{\textbf I}\end{array}} \right]$$
and
$${\textbf H} = \left[{\begin{array}{* {20}{c}}{{{\textbf R}_1}}\\ \vdots \\{{{\textbf R}_N}}\end{array}} \right].$$
Denote the $n$th block of a $3N \times 3$ matrix by ${(\cdot)_n}$ (i.e., rows numbered $3n - 2$, $3n - 1$, $3n$ in the matrix). Then from Eq. (9),
$${\left({{\textbf{SH}}} \right)_n} = \sum _{m = 1}^N{{\textbf R}_{m \to n}}{{\textbf R}_m} = N{{\textbf R}_n} = N{\left({\textbf H} \right)_n},$$
leading to
$${\textbf{SH}} = N{\textbf H}.$$
Therefore, the three columns of ${\textbf H}$ are eigenvectors of ${\textbf S}$. For each of these three eigenvectors, the corresponding eigenvalue is $N$. Recall that ${\textbf R}_m^{- 1} = {\textbf R}_m^{T}$ and, thus, that ${\textbf{S = H}}{{\textbf H}^{T}}$, which means that ${\textbf S}$ has rank 3. Therefore, $N$ is the only non-zero eigenvalue of ${\textbf S}$.

Using this formulation, we propose an algorithm for empirically estimating the rotations matrices $\{{{{\hat {\textbf R}}}_n}\} _{n = 1}^N$.

  • 1. Construct the empirical matrix ${\textbf S}$, using the estimated angle pair ${\hat \psi _{n \to m}},{\hat \psi _{m \to n}}$ and Eqs. (3), (9), and (10).
  • 2. Perform singular value decomposition (SVD) on ${\textbf S}$. Select the three largest singular values and their corresponding eigenvectors. Denote these column vectors as ${{\textbf h}_1}$, ${{\textbf h}_2}$, ${{\textbf h}_3}$.
  • 3. Construct the estimated rotations by concatenating these column vectors into a matrix:
$$\left[{\begin{array}{* {20}{c}}{{{{{\hat {\textbf R}}}}_1}}\\ \vdots \\{{{{{\hat {\textbf R}}}}_N}}\end{array}} \right] = \left[{{{\textbf h}_1}, {{\textbf h}_2}, {{\textbf h}_3}} \right].$$
This approach generalizes to yield more robust results in the presence of errors in the estimated angle pair ${\hat \psi _{n \to m}}$, ${\hat \psi _{m \to n}}$.

We describe in Sections 3.B and 3.C an approach to yield a confidence weight ${w_{n,m}}$ to each ${\hat \psi _{n \to m}}$, ${\hat \psi _{m \to n}}$ pair. A low value of ${w_{n,m}}$ indicates a low confidence in the correctness of this pair of angles. The weights are normalized so as to satisfy

$$\sum\limits_m {w_{n,m}} = N,\quad {w_{n,m}} \ge 0,\quad \forall n,m.$$
These weights propagate to the empirical matrix ${\textbf S}$: each block ${{\textbf R}_{m \to n}}$ in ${\textbf S}$ is replaced by ${w_{n,m}}{{\textbf R}_{m \to n}}$. This yields an improved matrix $\widetilde {\textbf S}$. Similarly to Eq. (12),
$${\left({\widetilde {\textbf S}{\textbf H}} \right)_n} = \sum _{m = 1}^N{w_{n,m}}{{\textbf R}_{m \to n}}{{\textbf R}_m} = {{\textbf R}_n}\sum _{m = 1}^N{w_{n,m}} = N{\left({\textbf H} \right)_n}.$$
Analogously to Eq. (13), the three columns of ${\textbf H}$ are eigenvectors of $\widetilde {\textbf S}$ with the corresponding eigenvalue $N$. To estimate the rotations matrices $\{{{{\hat {\textbf R}}}_n}\} _{n = 1}^N$, the algorithm above is run using $\widetilde {\textbf S}$ instead of ${\textbf S}$. The advantage of this method is that common lines having a lower confidence have a smaller effect on the estimation of $\{{{{\hat {\textbf R}}}_n}\} _{n = 1}^N$.

B. Statistical Model of Common Lines

To derive weights ${w_{n,m}}$, we need to have a measure of confidence or uncertainty for each common line. The uncertainty depends on a probabilistic model of common lines, the noise sources that affect this model and finite sampling effects. From our experience, errors in ${\hat \psi _{n \to m}}$, ${\hat \psi _{m \to n}}$ cause problems that are more severe than problems stemming from errors in scale or shift. Hence, we focus our attention on angular errors.

Consider the 2D Fourier representation ${{{\tilde {\textbf y}}}_n}$ (Fig. 4). The representation of the true common line with ${{{\tilde {\textbf y}}}_m}$ is a line at true angle ${\psi _{n \to m}}$. Its empirical estimation is ${\hat \psi _{n \to m}}$. The corresponding angular error is

$$\Delta {\psi _{nm}} = {\psi _{n \to m}} - {\hat \psi _{n \to m}}.$$
The direction of any common line in each 2D Fourier domain is estimated only modulo 180°. The reason is that the correlation [Eq. (4)] between any two such Fourier domain lines is invariant to reflection around the zero-frequency. Thus, $\Delta {\psi _{nm}} \in [- {90^ \circ}{,90^ \circ})$.
 figure: Fig. 4.

Fig. 4. Central slice in 3D Fourier domain (seen in Fig. 3) is equivalent to a 2D Fourier transform of an acquired image. A common line (also seen in Fig. 3) is at respective corresponding angles ${\psi _{n \to m}},{\psi _{m \to n}}$ in the 2D Fourier domains. Noisy data (e.g., due to non-rigid deformation of specimens) leads to erroneous estimation of these angles, ${\hat \psi _{n \to m}},{\hat \psi _{m \to n}}$.

Download Full Size | PPT Slide | PDF

In Cryo-EM literature [43], a parametric model had been suggested to express the distribution of $\Delta {\psi _{nm}}$. The model dictates the following:

  • • There is probability $ P $ that a pair ${\hat \psi _{n \to m}}$, ${\hat \psi _{m \to n}}$ is indicative: the angles are random though roughly correct. The probability is assumed to be Gaussian, in the form $\Delta {\psi _{nm}}$, $\Delta {\psi _{mn}} \sim {\cal N}(0,{\sigma ^2})$. Here $\sigma$ is the standard deviation of angular errors in indicative common lines.
  • • There is probability $1 - {P}$ that a pair ${\hat \psi _{n \to m}}$, ${\hat \psi _{m \to n}}$ is arbitrary: each of these angles is random, distributed uniformly around a circle.

Therefore, the probability density function (PDF) of $\Delta {\psi _{nm}}$ is modeled by

$$f\!\left({\Delta {\psi _{nm}}} \right) = \frac{{P}}{{\sigma \sqrt {2\pi}}}\exp \left[{- \frac{{{{\left({\Delta {\psi _{nm}}} \right)}^2}}}{{2{\sigma ^2}}}} \right] + \frac{{(1 - {P})}}{{{{180}^ \circ}}}.$$
The values of $ P $ and $\sigma$ depend on noise sources in the data. In Cryo-EM, the radiometric signal-to-noise ratio (SNR) is very low, and there are no concerns of scale variations or non-rigid deformations. In our case, however, the opposite holds. Hence, we tested in simulations whether this model can be relevant.

1. Effect of Radiometric Noise

We simulated a synthetic volume and projected it using $N = 100$ random orientations. The projections included photon (Poisson) noise, typical to optical cameras. The best quality had photon noise that corresponded to a full-well depth of 10,000 photoelectrons, i.e., ${\rm SNR} = {100}$ near camera saturation. The lower quality corresponded to a full-well depth of 100 photoelectrons, i.e., ${\rm SNR} = {10}$ near camera saturation. Equations (4), (5), and (17) were then run, after which the model of Eq. (18) was fitted to the histogram of errors. The fit was good yet insensitive to the radiometric SNR, yielding ${P} = 0.897 \pm 0.007$ and $\sigma = {1.25^\circ} \pm {0.01^ \circ}$ across the range. Similar results were obtained when introducing random lateral displacements and scales to the imaged samples. For this reason, we do not see radiometric noise as a major source of uncertainty in common line estimation, in our type of imaging.

2. Specimen Deformation as Noise

The objects we consider always have variance in sub-structures, i.e., non-rigid deformations. Such random morphological noise disrupts proper detection and matching of common line pairs. To induce such noise in a simulation, we followed a recipe by Ref. [42]. Let the longest dimension of a volumetric distribution $\beta ({\textbf x})$ be bounded by $G$ voxels. The domain size of $\beta ({\textbf x})$ is $G \times G \times G$ voxels. This distribution is deformed to $\beta ({\textbf x} + {{\textbf v}^{(n)}}[{\textbf x}])$. Here ${{\textbf v}^{(n)}} = [{v_1},{v_2},{v_3}]$ represents a random vector (deformation) field of the specimen corresponding to image $n$. There is a 3D vector of deformation at every point in the object.

The deformation direction ${{{\hat{\textbf v}}}^{(n)}} = [\hat v_1^{(n)},\hat v_2^{(n)},\hat v_3^{(n)}]$ is randomly sampled on the unit sphere. The deformation magnitude varies spatially as a sinusoid having a random amplitude ${V^{(n)}} \sim {\cal U}(0,{V_{\max}}G)$. Each spatial component $c \in \{1,2,3\}$ has random phase $\phi _c^{(n)} \sim {\cal U}(- \pi ,\pi)$. The distortion of component $c$ is

$$v_c^{(n)} = \hat v_c^{(n)}{V^{(n)}}\sin \left[{2\pi {x_c}/(G - 1) + \phi _c^{(n)}} \right].$$
This deformation field model represents a basic Fourier component of distortion. We set $G = 100$. For each value of ${V_{\max}}$, we computed $N = 100$ random projections and deformations. Equations (4), (5), and (17) were run, and then the model of Eq. (18) was fitted to the histogram of angular errors (Fig. 5). Across a range of ${V_{{\max}}}$ values, the fit is good. The values of $ P $ and $\sigma$ are sensitive to the magnitude of distortion amplitude ${V_{{\max}}}$ (Fig. 6).
 figure: Fig. 5.

Fig. 5. Histogram of the angular error $\Delta {\psi _{nm}}$ (in degrees) with ${V_{{\max}}} = 9\%$, when data is affected by simulated random non-rigid deformations. The red curve is a fit of the histogram to Eq. (18). The fit has a coefficient of correlation 0.98.

Download Full Size | PPT Slide | PDF

 figure: Fig. 6.

Fig. 6. Influence of the deformation coefficient ${V_{{\max}}}$ on the parameters of the statistical model [Eq. (18)]. The stronger the deformations, the lower is the probability that correlation-based angle pairs are indicative. Pairs that are indicative have a larger uncertainty as ${V_{{\max}}}$ increases.

Download Full Size | PPT Slide | PDF

C. Common Line Triplets

To establish the weights ${w_{n,m}}$, we need a statistical model. Section 3.B presented a generic model parameterized by $ P $ and $\sigma$. When using real-data, we do not know the values of these statistical parameters. Thus, we derive these parameters based on the actual real-data, which includes many (noisy) estimates of angles ${\{{\hat \psi _{n \to m}}\} _{n,m}}$.

We denote the $ n $, $ m $ pair of images as $\overline {n,m}$. For any $\overline {n,m}$, we should assess the likelihood that the estimated angle pair is indicative or arbitrary. This can be done by relying on common line triplets. Denote a different triplet of projection images (indexed $n,m,l$) of the same object as $\overline {n,m,l}$. It corresponds to true rotations ${{\textbf R}_n} ,{{\textbf R}_m} ,{{\textbf R}_l}$, and consequently to true three matrices ${{\textbf R}_{n \to m}}$, ${{\textbf R}_{m \to l}}$, ${{\textbf R}_{l \to n}}$, following Eq. (9). To be correct rotation matrices, they must satisfy

$${{\textbf R}_{n \to m}}{{\textbf R}_{m \to l}}{{\textbf R}_{l \to n}} - {\textbf I} = 0.$$
When processing real-data, we do not know the true rotations. Instead, we only have noisy estimates ${{{\hat {\textbf R}}}_{n \to m}}$, ${{{\hat {\textbf R}}}_{m \to l}}$, ${{{\hat {\textbf R}}}_{l \to n}}$, due to the noisy estimated set ${\{{\hat \psi _{n \to m}}\} _{\forall m \ne n}}$. Generally Eq. (20) is not satisfied exactly when based on estimated angles and matrices.

1. Triplet Score and Statistics

Reference [43] proposed a score to quantify how closely Eq. (20) is approximated using estimated matrices,

$${s_{n,m,l}} \equiv s\left({{{{{\hat {\textbf R}}}}_{n \to m}}{{{{\hat {\textbf R}}}}_{m \to l}}{{{{\hat {\textbf R}}}}_{l \to n}}} \right),$$
whose range is ${s_{n,m,l}} \in [0,1]$. As the score decreases, the further Eq. (20) is from being satisfied.

Because the empirical values of ${\hat \psi _{n \to m}},{\hat \psi _{m \to l}},{\hat \psi _{l \to n}}$ are random, then ${s_{n,m,l}}$ is random. If all three angles ${\hat \psi _{n \to m}},{\hat \psi _{m \to l}},{\hat \psi _{l \to n}}$ are indicative, then $\overline {n,m,l}$ is said to be an indicative triplet, and ${s_{n,m,l}}$ is likely to be high. However, if any of these angles are arbitrary, then $\overline {n,m,l}$ is said to be an arbitrary triplet, and ${s_{n,m,l}}$ is likely to be low.

These likelihoods are expressed by PDFs [43]. For arbitrary triplets, the PDF of $s$ is denoted by $f_{{\rm triplet}}^{{\rm arbitrary}}(s)$, which can be pre-calculated using a simulation. This simulation does not rely on any image data. Rather, it tests relations between random rotation matrices:

  • (a) Draw a set of a large number of random uniformly distributed common line angles ${\{\hat \psi _{n \to m}^{{\rm simul}}\} _{m,n}}$.
  • (b) Calculate $s \in [0,1]$ for triplets drawn from this set using Eq. (21).
  • (c) Calculate a normalized histogram of $s$ and fit a simple continuous curve [43] to it, in the domain $s \in [0,1]$.

This yields $f_{{\rm triplet}}^{{\rm arbitrary}}(s)$, and it is independent of the particular mechanism underlying any arbitrary orientation errors. The score is maximal for $s = 0$ and decays to zero at $s = 1$ (Fig. 7).

 figure: Fig. 7.

Fig. 7. Top left, the pre-computed PDF of a triplet score $s$, if the triplet of estimated relative rotations is arbitrary. Top right, the pre-computed PDF of a triplet score $s$, if the triplet of estimated relative rotations is indicative. Here estimated relative rotations have errors whose standard deviation is $\sigma = {6^\circ}$. Bottom, empirical histogram ${\cal H}(s)$ of triplet scores. It is modeled as a convex sum of the above PDFs, controlled by $P$. Fitting this parametric model to ${\cal H}(s)$ yields an estimate of ${{\hat P}},\hat \sigma$.

Download Full Size | PPT Slide | PDF

For indicative triplets, the PDF of $s$ is analogously denoted by $f_{{\rm triplet}}^{{\rm indicative}}(s|\sigma)$. Here $\sigma$ is the standard deviation of angular errors in indicative common lines, as used in Section 3.B.

The PDF $f_{{\rm triplet}}^{{\rm indicative}}(s|\sigma)$ is likewise pre-calculated in simulations, irrespective of the underlying mechanism causing orientation errors:

  • (a) Draw a set of a large number of arbitrary orientations ${\{{{\textbf R}_n}\} _n}$, randomly uniformly distributed.
  • (b) Calculate the set of corresponding correct common line angles ${\{\psi _{n \to m}^{{\rm simul}}\} _{m,n}}$.
  • (c) Perturb these angles by random Gaussian errors having standard deviation $\sigma$ [as in Eq. (18)] for ${P} = 1$. This yields a set of indicative ${\{\hat \psi _{n \to m}^{{\rm simul}}\} _{m,n}}$.
  • (d) Calculate $s \in [0,1]$ for triplets drawn from this set, using Eq. (21).
  • (e) Calculate a normalized histogram of $s$ and fit a simple continuous curve [43] to it, in the domain $s \in [0,1]$. This fitted curve is parameterized by $\sigma$.

For the special case $\sigma = 6$, $f_{{\rm triplet}}^{{\rm indicative}}(s|\sigma)$ is indeed maximal close to $s = 1$ and decays to zero at $s = 0$ (Fig. 7).

2. Estimating the Statistical Model Parameters

To estimate $P$ and $\sigma$, we use the pre-calculated PDFs $f_{{\rm triplet}}^{{\rm arbitrary}}(s)$, $f_{{\rm triplet}}^{{\rm indicative}}(s|\sigma)$ together with the set ${\{{\hat \psi _{n \to m}}\} _{m,n}}$ empirically derived from the real-data. Consequently the weights ${w_{n,m}}$ are derived as in Section 3.C.3. Recall that for each $ m $, $ n $, the probability that $\{{\hat \psi _{n \to m}}\}$ is indicative is $ P $. Assume that the estimation of each common line is approximately independent of the estimation of the other lines. Hence, the probability that a triplet is indicative is $\approx {{P}^3}$, and the probability that a triplet is arbitrary is approximately $1 - {{P}^3}$.

From the data-based estimation of ${\{{\hat \psi _{n \to m}}\} _{m,n}}$, the matrices ${\{{{{\hat {\textbf R}}}_{n \to m}}\} _{m,n}}$ are derived. Then Eq. (21) yields a set of scores $\{{s_{n,m,l}}\} _{n \ne m \ne l = 1}^N$. The values in the set of scores yield a normalized histogram ${\cal H}(s)$. According to the model,

$${\cal H}(s) \approx {{P}^3}f_{{\rm triplet}}^{{\rm indicative}}(s|\sigma) + (1 - {{P}^3})f_{{\rm triplet}}^{{\rm arbitrary}}(s).$$
The values of $ P $ and $\sigma$ are then searched such that ${\cal H}(s)$ fits the model: a convex sum of pre-computed PDFs. This yields estimation of the statistical model parameters, ${{\hat P}},\hat \sigma$.

3. Confidence Weights

To derive the weights ${w_{n,m}}$, we can now rely on the estimated ${{\hat P}},\hat \sigma$ and the set of empirical scores $\{{s_{n,m,l}}\} _{n \ne m \ne l = 1}^N$. The confidence in the angles of  $\overline {n,m}$, ${\hat \psi _{n \to m}},{\hat \psi _{m \to n}}$ is assessed by the set of triplet scores associated with $\overline {n,m}$:

$${{\cal S}_{n,m}} \equiv \left\{{{s_{n,m,l}} \forall l \notin \{n,m\}} \right\}.$$
The likelihoods that this set is drawn from indicative or arbitrary common lines are $\ell _{n,m}^{{\rm indicative}}(\hat \sigma)$ and $\ell _{n,m}^{{\rm arbitrary}}$, respectively. Details of estimates of $\ell _{n,m}^{{\rm indicative}}(\hat \sigma)$ and $\ell _{n,m}^{{\rm arbitrary}}$ are given in Appendix A. Given $\ell _{n,m}^{{\rm indicative}}(\hat \sigma)$ and $\ell _{n,m}^{{\rm arbitrary}}$, the likelihood of the scores ${{\cal S}_{n,m}}$ is ${{\hat P}}\ell _{n,m}^{{\rm indicative}}(\hat \sigma) + (1 - {{\hat P}})\ell _{n,m}^{{\rm arbitrary}}$. The probability density of $ n $, $ m $ being indicative, given the set ${{\cal S}_{n,m}}$, is, thus [43],
$${p_{n,m}} = \frac{{{{\hat P}}\ell _{n,m}^{{\rm indicative}}(\hat \sigma)}}{{{{\hat P}}\ell _{n,m}^{{\rm indicative}}(\hat \sigma) + (1 - {{\hat P}})\ell _{n,m}^{{\rm arbitrary}}}}.$$
The confidence weights are set to be
$${w_{n,m}} = \gamma {p_{n,m}},$$
where $\gamma$ is a normalization constant to ensure that Eq. (15) is satisfied.

From this result, a higher weight is given to a common line pair that yields a higher likelihood of indicative triplet scores. This means that the particular common line being weighted has high support across the entire set of common lines. A common line pair, which has little support from the other common lines for being indicative, has a low probability for being indicative and thus has a lower weight. The weight [Eq. (25)] is used for constructing the matrix $\widetilde {\textbf S}$ (Section 3.A). In turn, this enhances the estimation of the rotation matrices $\{{{{\hat {\textbf R}}}_n}\} _{n = 1}^N$.

D. Constraint for Scale Estimation

Recall from Eq. (5) that pairwise rotation angles, scales, and translations are searched in a 4D domain. This is a computationally expensive process that is prone to errors in estimated pairwise rotation angles, which can in turn induce errors in scale. Our procedure (Sections 3.A3.C) reduces these problems, yielding a robust estimation of the rotations, despite uncertainties and significant outliers. The estimated rotations $\{{{{\hat {\textbf R}}}_n}\} _{n = 1}^N$ consequently constrain the pairwise angles. The constrained angles then lead to a simpler search, whose results proved to be more accurate. We follow these steps:

  • (a) Run Eq. (5) for initial values of ${\{{\hat \psi _{n \to m}}\} _{m,n}}$, as well as initial estimates of pairwise scales and shifts. The search is executed on a coarse 4D grid for the sake of speed. The results can have many outliers with large uncertainties. Discard the initial estimates of pairwise scales and shifts.
  • (b) Run the process described in Sections 3.A3.C to estimate the rotations $\{{{{\hat {\textbf R}}}_n}\} _{n = 1}^N$.
  • (c) Use $\{{{{\hat {\textbf R}}}_n}\} _{n = 1}^N$ to derive the common line between any pair of central slices $\overline {n,m}$. This is done in closed-form and yields a new set of common line pairs, denoted by ${\{{\tilde \psi _{n \to m}}\} _{m,n}}$. The new set of common lines ${\{{\tilde \psi _{n \to m}}\} _{m,n}}$ is, thus, a result of robust estimation, which accounts for rotations of all projections in the image set. This is contrary to the set ${\{{\hat \psi _{n \to m}}\} _{m,n}}$, which was crudely sampled based on pairs of images.
  • (d) Constrain Eq. (5) to run on the common line pair ${\tilde \psi _{n \to m}},{\tilde \psi _{m \to n}}$:
$$\{{\hat M_{n \to m}},{\hat \tau _{n \to m}}\} = \arg \mathop {\max}\limits_{M,\tau} \frac{{{\Sigma _{nm}}(M,\tau)}}{{\sqrt {{\Sigma _{nn}}(M,\tau)} \sqrt {{\Sigma _{mm}}(M,\tau)}}},$$
where ${\Sigma _{nm}}(M,\tau) \equiv {\Sigma _{nm}}({\tilde \psi _{n \to m}},{\tilde \psi _{m \to n}},M,\tau)$. This is a 2D search, contrary to Eq. (5).

The resulting pairwise scales are then used in Eqs. (6) and (7), to yield all scales, globally. Similarly, pairwise translations are fed into a linear optimization process. This yields all translations in a global frame, akin to [42]. Once rotations, scales, and translations are estimated, the pairwise search is repeated. In the new iteration, the search domain is narrowed and sampled more finely. This step is integrated into a broader coarse-to-fine approach that follows [42].

Eventually, an estimate of all orientation, scales, and shifts is obtained. Consequently, tomographic 3D recovery of the objects is performed, yielding $\hat \beta ({\textbf x})$. We used an algebraic reconstruction technique [56,57] for this purpose.

4. RESULTS

A. Quantitative Criteria

We performed simulations as well as experiments using real-data. For quantitative criteria, we follow Ref. [42] by noting that all rotations can only be estimated up to an unknown global rotation ${\textbf O}$. Similarly, all scales are defined up to a global unknown ${M_{{\rm rel}}}$. Hence [42], when ground-truth is available during simulations, error measures for rotations and scales are, respectively,

$${\epsilon_{{\rm rot}}} = \frac{1}{N}\mathop {\min}\limits_{\textbf O} \sum\limits_{n = 1}^N \parallel {{\textbf R}_n} - {\textbf O}{{{\hat {\textbf R}}}_n}\parallel _2^2,$$
$${\epsilon_{{\rm scale}}} = \frac{1}{N}\mathop {\min}\limits_{{M_{{\rm rel}}}} \sum\limits_{n = 1}^N {\left| {{M_n} - {M_{{\rm rel}}}{{\hat M}_n}} \right|^2}.$$
The relative error of the estimated object density $\hat \beta ({\textbf x})$ is
$${\epsilon_{{\rm dens}}} = \frac{{\int \left| {\hat \beta ({\textbf x}) - \beta ({\textbf x})} \right|{\rm d}{\textbf x}}}{{\int \beta ({\textbf x}){\rm d}{\textbf x}}}.$$
Since our experiments on real in situ plankton images have no ground-truth, quality is assessed via cross-validation. Rotations, scales, and translations are estimated using the entire ensemble. Then a single image $\tilde n$ is excluded from the $N$-sample ensemble. The 3D estimation $\hat \beta$, thus, uses $N - 1$ images. Then, $\hat \beta$ is re-projected using ${{{\hat {\textbf R}}}_{\tilde n}},{\hat M_{\tilde n}},{\hat \tau _{\tilde n}}$, yielding a synthetic image ${{{\hat {\textbf y}}}_{\tilde n}}$. It is compared to the true image ${{\textbf y}_{\tilde n}}$ using relative root mean square error. This is repeated $\forall \tilde n \in [1,N]$. There are ${N_{{\rm pixel}}}$ pixels. The re-projection error is
$${\epsilon_{{\rm re} {\text -} {\rm proj}}} = \frac{1}{{N\sqrt {{N_{{\rm pixel}}}}}}\sum\limits_{\tilde n = 1}^N \frac{{\parallel {{\textbf y}_{\tilde n}} - {{{{\hat {\textbf y}}}}_{\tilde n}}\parallel _2^1}}{{\max \left({{{\textbf y}_{\tilde n}}} \right)}}.$$
Plankton ecologists usually model population size distribution as a log-normal PDF [58]. Let an object be of size (e.g., length) $\rho$. Then the PDF model is
$${f^{{\rm size}}}(\varrho) = \frac{1}{{\varrho \sqrt {2\pi {\nu ^2}}}}\exp \left\{{- \frac{{{{[\ln (\varrho) - \xi]}^2}}}{{2{\nu ^2}}}} \right\}.$$
Here $\xi ,\nu$ are the mean and standard deviation in the logarithm domain. We can then assess the quality of the estimated scales by fitting Eq. (31) to the histogram of ${\hat M_{\tilde n}}$.

B. Simulation Based on Empirical Data

The simulation is based on a real object whose 3D volumetric distribution is empirically known [59]. In our simulations, this object underwent random deformations as described in Section 3.B.2, with ${V_{{\max}}} = 6\%$. It also underwent random scaling distributed as $\mathop {\log}\nolimits_{e} ({\rm scale}) \sim {\cal U}({- 0.7,0.7})$. This results in a scale factor of $\times 4$ between the lengths of the largest and smallest specimen. The object was projected to $N = 100$ random directions, each yielding an image of size $126 \times 126$ pixels. The projection then underwent random lateral shifts in the image domain, distributed as ${\rm shift} \sim {\cal U}({- 10,10}) [{\rm pixels}]$ in both image axes. The prototype object and sample projections are shown in Fig. 8.

 figure: Fig. 8.

Fig. 8. (a) 3D volumetric object is randomly scaled, deformed shifted, and projected onto 100 random orientations. (b) Here we show several image samples from this simulation.

Download Full Size | PPT Slide | PDF

The initial search domain in Eq. (5) used sampled angles with one-degree resolution. The domain had 10 relative scale samples uniformly spaced in the logarithm domain $[- \log (2) \log (2)]$ and 10 samples of relative offsets in a $[- 15 15]\; {\rm pixel}$ domain. As in Section 3.D, the domains become narrow through an iterative coarse-to-fine search. The initial pairwise matches were processed using the approach of this paper and compared to the result of Ref. [42]. Results are shown in Fig. 9 and Table 1.

 figure: Fig. 9.

Fig. 9. Recovery using data described in Fig. 8. Top, visualization of the 3D object density estimated by (a) our method versus (b) that of Ref. [42]. (c) and (d) The respective scatter plots of $\hat \beta$ versus true $\beta$ (values drawn from 1% random voxels). The red line represents a perfect fit. Our approach clearly yields a better fit to ground-truth than [42]. (e) The statistical model [Eq. (18)] fitted to the empirical histogram of angular errors, similarly to Fig. 6. In the process described in Section 3.D, estimated angle pairs are better constrained. After a single iteration, our error model (red plot) has a narrower standard deviation and a higher indicative probability, compared to errors (blue plot) obtained in the method of Ref. [42].

Download Full Size | PPT Slide | PDF

Tables Icon

Table 1. Results Corresponding to Data Described in Fig. 8a

C. Microscopy Experiments

1. Dark-Field Data

The first dataset contains images of Daphnia sp., a species of freshwater zooplankton (Fig. 10). Outliers are first pruned out [42]: images whose estimated common lines pairs have low correlation values (under a manually chosen threshold) are considered outliers. For this fixed threshold, pruning resulted in a cluster of $N = 147$ samples for [42]. Correlations on the constrained lines described in Section 3.D yielded $N = 145$ inlier samples for our approach. The initial search domain in Eq. (5) is as in the simulations. Comparing the resulting re-projections on sample frames from both approaches indicates that our approach is able to resolve finer details (Fig. 10). Quantitatively, our approach yields ${\epsilon_{{\rm re} {\text -} {\rm proj}}} = 1.38 \pm 0.002$. In comparison, the method of Ref. [42] yields ${\epsilon_{{\rm re} {\text -} {\rm proj}}} = 3.065 \pm 0.004$.

 figure: Fig. 10.

Fig. 10. Experiment on a population of Daphnia sp., imaged using our system. Top, a few raw dark-field images. Middle, corresponding cross-validation re-projections, as described in Section 3.A, using our tomographically estimated object. Bottom, corresponding cross-validation re-projections, using the estimation resulting from the method of Ref. [42].

Download Full Size | PPT Slide | PDF

The second dataset consists of 1000 images of the cosmopolitan zooplankton Oithona sp. (Fig. 11). To keep runtime short in this large database, we used 50 equally spaced angular samples in the search domain of Eq. (5). We used the other domain sample parameters as those in the simulations. Pruning yielded $N = 921$ inlier samples for our approach and $N = 877$ for the method of Ref. [42]. Figure 11 compares re-projections while Fig. 12 compares 3D recoveries.

 figure: Fig. 11.

Fig. 11. Experiment on a population of Oithona sp., imaged using our system. Top, a few raw dark-field images. Middle, corresponding cross-validation re-projections of our tomographically estimated object. Bottom, corresponding cross-validation re-projections of the object resulting from the method of Ref. [42].

Download Full Size | PPT Slide | PDF

 figure: Fig. 12.

Fig. 12. Volumetric density recovery: Oithona sp.

Download Full Size | PPT Slide | PDF

The proposed method yields ${\epsilon_{{\rm re} {\text -} {\rm proj}}} = 4.174 \pm 8 \cdot {10^{- 4}}$. In comparison, the method of Ref. [42] yields ${\epsilon_{{\rm re} {\text -} {\rm proj}}} = 6.559 \pm 0.002$.

Recall that our method yields an estimation of the size distribution. Figures 13(a) and 13(b) show the estimated size histograms of Oithona sp. and Daphnia sp. and the corresponding PDFs fitted using Eq. (31). The estimated size histograms and fitted PDF by Ref. [42] are very similar to ours.

 figure: Fig. 13.

Fig. 13. Resulting size distribution histograms (blue) and fitted [Eq. (31)] density (red). (a) Oithona sp. The fit 95% confidence intervals are $\xi \in [6.48\;6.5]$, $\nu \in [0.138\;0.151]$. (b) Daphnia sp. The fit 95% confidence intervals are $\xi \in [1.12\;1.21]$, $\nu \in [0.244\;0.308]$.

Download Full Size | PPT Slide | PDF

2. Bright-Field Data

For direct comparison to the results of Ref. [42], we used the publicly available plankton image database provided by Ref. [60], specifically focusing on Pyramimonas longicauda. We note that this algal species is an order of magnitude smaller than the zooplankton considered in the other datasets. Pruning of poorly correlated data yielded a set of $N = 136$ images for our analysis and $N = 132$ for running [42]. Figure 14 visualizes the respective 3D recoveries.

 figure: Fig. 14.

Fig. 14. Volumetric density recovery: Pyramimonas longicauda.

Download Full Size | PPT Slide | PDF

Figure 15 shows several re-projections and compares them to the real images. Both figures indicate that the proposed approach yields more complete results, of higher contrast.

 figure: Fig. 15.

Fig. 15. Experiment on a population of Pyramimonas longicauda, from dataset [4]. Top, a logarithm operation on raw bright-field images yields a linear model as in Eq. (1). A few samples are shown. Middle, corresponding cross-validation re-projections of our tomographically estimated object. Bottom, corresponding cross-validation re-projections of the object resulting from the method of Ref. [42].

Download Full Size | PPT Slide | PDF

Quantitatively, our approach yielded ${\epsilon_{{\rm re} {\text -} {\rm proj}}} = 3.446 \pm 0.004$. In comparison, the method of Ref. [42] yielded ${\epsilon_{{\rm re} {\text -} {\rm proj}}} = 3.942 \pm 0.005$.

5. DISCUSSION

Using our method, massive 2D image databases collected globally by ocean and lake in situ observation systems can lead to 3D structure and population statistics estimates. Tackling this problem at scale necessitates more efficient processing. For example, using 3D volumetric models of plankton, learning-based methods can be trained to infer species and distributions, based on image projections taken at random poses.

Our experiments indicate that the method is robust to differences in organism size: the SPC images are of plankton in the 0.5–2.0 mm range, while those captured by the IFCB are tens of microns across. The method seems to be robust to the body plan: Daphnia and Oithona are both zooplankton while Pyramimonas is a photosynthetic algae. We believe the approach will yield reasonable reconstructions for a variety of plankton, though domain experts should be consulted when approaching new taxa. Gelatinous organisms will be particularly challenging since much of their bodies are transparent in seawater.

The approach presented here is not limited to plankton. We believe it can be used in population analysis of a wide variety of small, translucent suspended objects. Beyond other organism classes, the method might also be applied to non-biological targets. These may include submerged suspended sediments, airborne aerosols and hydrosols, ice crystals in clouds, and possibly small meteoroids.

APPENDIX A

We now describe how estimates of $\ell _{n,m}^{{\rm indicative}}(\hat \sigma)$ and $\ell _{n,m}^{{\rm arbitrary}}$ are obtained, for use in Eq. (24).

  • • Assume for a moment that the $ n $, $ m $ common line is arbitrary. Then, by definition, all elements in ${{\cal S}_{n,m}}$ are scores of arbitrary triplets. Then, the likelihood of this set of scores being drawn from arbitrary common lines is
    $$\ell _{n,m}^{{\rm arbitrary}} \equiv {\ell ^{{\rm arbitrary}}}({{\cal S}_{n,m}}) = \prod\limits_{l \notin \{n,m\}} f_{{\rm triplet}}^{{\rm arbitrary}}({s_{n,m,l}}).$$
    Here, it is assumed that ${s_{n,m,l}},{s_{n,m,l^\prime}}$ are approximately statistically independent $\forall l \ne l^\prime $, given that the common line of $\overline {n,m}$ is arbitrary.
  • • Assume for a moment that the common line of $\overline {n,m}$ is indicative. Then, $\overline {n,m,l}$ is indicative only if both the common lines of $\overline {n,l}$ and $\overline {m,l}$ are indicative as well. For such an event, the likelihood of the score ${s_{n,m,l}}$ is, thus, ${{{\hat P}}^2}f_{{\rm triplet}}^{{\rm indicative}}({s_{n,m,l}}|\hat \sigma)$.

The complementing case is that either or both common lines of $\overline {n,l}$ and $\overline {m,l}$ are arbitrary. Then, the triplet $\overline {n,m,l}$ is arbitrary, despite the common line of $\overline {n,m}$ being indicative. In this case, the likelihood of the score ${s_{n,m,l}}$ is, thus, $(1 - {{{\hat P}}^2})f_{{\rm triplet}}^{{\rm arbitrary}}({s_{n,m,l}})$. Consequently, in total, if the common line of $\overline {n,m}$ is indicative, the likelihood of the score ${s_{n,m,l}}$ is ${{{\hat P}}^2}f_{{\rm triplet}}^{{\rm indicative}}({s_{n,m,l}}|\hat \sigma) + (1 - {{{\hat P}}^2})f_{{\rm triplet}}^{{\rm arbitrary}}({s_{n,m,l}})$. Therefore, compounding all scores in ${{\cal S}_{n,m}}$, the likelihood of this set being drawn while the common line of $\overline {n,m}$ is indicative is

$$\begin{split}\ell _{n,m}^{{\rm indicative}}(\hat \sigma) &\equiv {\ell ^{{\rm indicative}}}({{\cal S}_{n,m}}|\hat \sigma)\\ &= \prod\limits_{l \notin \{n,m\}} \left[{{{{{\hat P}}}^2}f_{{\rm triplet}}^{{\rm indicative}}({s_{n,m,l}}|\hat \sigma)} \right.\\&\quad+\left. { (1 - {{{{\hat P}}}^2})f_{{\rm triplet}}^{{\rm arbitrary}}({s_{n,m,l}})} \right].\end{split}$$
Aggregating both cases, recall that $\overline {n,m}$ has prior probabilities ${{\hat P}},(1 - {{\hat P}})$ for being indicative or arbitrary, respectively.

Funding

European Research Council (810370).

Acknowledgment

We thank Aviad Levis and Tali Treibitz for their advice, and Johanan Erez, Ina Talmon, and Daniel Yagodin for technical support. Yoav Schechner is the Mark and Diane Seiden Chair in Science at the Technion. He is a Landau Fellow—supported by the Taub Foundation. His work was conducted in the Ollendorff Minerva Center. Minvera is funded through the BMBF. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 810370-ERC-CloudC.

Disclosures

The authors declare no conflicts of interest.

Data Availability

Data underlying the results presented in this paper are available in Refs. [4,60].

REFERENCES

1. G. C. Hays, A. J. Richardson, and C. Robinson, “Climate change and marine plankton,” Trends Ecol. Evol. 20, 337–344 (2005). [CrossRef]  

2. J. Ryan, R. Kudela, J. Birch, M. Blum, H. Bowers, F. Chavez, G. Doucette, K. Hayashi, R. Marin, C. Mikulski, and J. Pennington, “Causality of an extreme harmful algal bloom in Monterey Bay, California, during the 2014–2016 northeast Pacific warm anomaly,” Geophys. Res. Lett. 44, 5571–5579 (2017). [CrossRef]  

3. C. Anderson, E. Berdalet, R. Kudela, C. Cusack, J. Silke, E. O’Rourke, D. Dugan, M. McCammon, J. Newton, S. Moore, and K. Paige, “Scaling up from regional case studies to a global harmful algal bloom observing system,” Front. Mar. Sci. 6, 250 (2019). [CrossRef]  

4. “Scripps plankton camera website,” http://spc.ucsd.edu.

5. G. Gorsky, M. Picheral, and L. Stemmann, “Use of the underwater video profiler for the study of aggregate dynamics in the North Mediterranean,” Estuarine Coastal Shelf Sci. 50, 121–128 (2000). [CrossRef]  

6. C. S. Davis, S. M. Gallager, M. Marra, and W. K. Stewart, “Rapid visualization of plankton abundance and taxonomic composition using the video plankton recorder,” Deep Sea Res. II 43, 1947–1970 (1996). [CrossRef]  

7. M. D. Ohman, R. E. Davis, J. T. Sherman, K. R. Grindley, B. M. Whitmore, C. F. Nickels, and J. S. Ellen, “Zooglider: an autonomous vehicle for optical and acoustic sensing of zooplankton,” Limnol. Oceanogr. Methods 17, 69–86 (2019). [CrossRef]  

8. R. K. Cowen and C. M. Guigand, “In situ ichthyoplankton imaging system (I SIIS): system design and preliminary results,” Limnol. Oceanogr. Methods 6, 126–132 (2008). [CrossRef]  

9. S. Vogel, Life in Moving Fluids: The Physical Biology of Flow-Revised and Expanded Second Edition (Princeton University, 2020).

10. S. Lee, K. Kim, A. Mubarok, A. Panduwirawan, K. Lee, S. Lee, H. Park, and Y. Park, “High-resolution 3D refractive index tomography and 2D synthetic aperture imaging of live phytoplankton,” J. Opt. Soc. Korea 18, 691–697 (2014). [CrossRef]  

11. S. Johnsen, The Optics of Life: A Biologist’s Guide to Light in Nature (Princeton University, 2012).

12. S. Colin, L. P. Coelho, S. Sunagawa, C. Bowler, E. Karsenti, P. Bork, R. Pepperkok, and C. De Vargas, “Quantitative 3D-imaging for cell biology and ecology of environmental microbial eukaryotes,” Elife 6, e26066 (2017). [CrossRef]  

13. F. Lombard, E. Boss, A. M. Waite, J. Uitz, L. Stemmann, H. M. Sosik, J. Schulz, J.-B. Romagnan, M. Picheral, and J. Pearlman, “Globally consistent quantitative observations of planktonic ecosystems,” Front. Mar. Sci. 6, 196 (2019). [CrossRef]  

14. M. Picheral, L. Guidi, L. Stemmann, D. M. Karl, G. Iddaoud, and G. Gorsky, “The underwater vision profiler 5: an advanced instrument for high spatial resolution studies of particle size spectra and zooplankton,” Limnol. Oceanogr. Methods 8, 462–473 (2010). [CrossRef]  

15. R. J. Olson and H. M. Sosik, “A submersible imaging-in-flow instrument to analyze nano-and microplankton: imaging FlowCytobot,” Limnol. Oceanogr. Methods 5, 195–203 (2007). [CrossRef]  

16. J. S. Jaffe, “Underwater optical imaging: the past, the present, and the prospects,” IEEE J. Ocean. Eng. 40, 683–700 (2014). [CrossRef]  

17. A. D. Mullen, T. Treibitz, P. L. Roberts, and J. S. Jaffe, “An underwater microscope for in situ imaging of seafloor organism,” in Novel Techniques in Microscopy (OSA, 2017), paper NTu1C–1.

18. E. Acevedo-Trejos, G. Brandt, J. Bruggeman, and A. Merico, “Mechanisms shaping size structure and functional diversity of phytoplankton communities in the ocean,” Sci. Rep. 5, 8918 (2015). [CrossRef]  

19. H. Sun, P. W. Benzie, N. Burns, D. C. Hendry, M. A. Player, and J. Watson, “Underwater digital holography for studies of marine plankton,” Philos. Trans. R. Soc. A 366, 1789–1806 (2008). [CrossRef]  

20. E. J. Davies and R. Nepstad, “In situ characterisation of complex suspended particulates surrounding an active submarine tailings placement site in a Norwegian fjord,” Regional Stud. Mar. Sci. 16, 198–207 (2017). [CrossRef]  

21. J. Sheng, E. Malkiel, and J. Katz, “Digital holographic microscope for measuring three-dimensional particle distributions and motions,” Appl. Opt. 45, 3893–3901 (2006). [CrossRef]  

22. S. Samson, T. Hopkins, A. Remsen, L. Langebrake, T. Sutton, and J. Patten, “A system for high-resolution zooplankton imaging,” IEEE J. Ocean. Eng. 26, 671–676 (2001). [CrossRef]  

23. A. Zweifler, D. Akkaynak, and T. Mass, and T. Treibitz, “In situ analysis of coral recruits using fluorescence imaging,” Front. Mar. Sci. 4, 273 (2017). [CrossRef]  

24. D. Akkaynak, T. Treibitz, B. Xiao, U. A. Gürkan, J. J. Allen, U. Demirci, and R. T. Hanlon, “Use of commercial off-the-shelf digital cameras for scientific data acquisition and scene-specific color calibration,” J. Opt. Soc. Am. A 31, 312–321 (2014). [CrossRef]  

25. D. M. Kocak, F. R. Dalgleish, F. M. Caimi, and Y. Y. Schechner, “A focus on recent developments and trends in underwater imaging,” Mar. Technol. Soc. J. 42, 52–67 (2008). [CrossRef]  

26. A. Jordt-Sedlazeck and R. Koch, “Refractive structure-from-motion on underwater images,” in IEEE International Conference on Computer Vision (2013), pp. 57–64.

27. C. Ancuti, C. O. Ancuti, C. De Vleeschouwer, R. Garcia, and A. C. Bovik, “Multi-scale underwater descattering,” in IEEE International Conference on Pattern Recognition (IEEE, 2016), pp. 4202–4207.

28. M. Sheinin and Y. Y. Schechner, “The next best underwater view,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 3764–3773.

29. H. Singh, J. G. Bellingham, F. Hover, S. Lemer, B. A. Moran, K. Von der Heydt, and D. Yoerger, “Docking for an autonomous ocean sampling network,” IEEE J. Ocean. Eng. 26, 498–514 (2001). [CrossRef]  

30. Y. Wang, S. Negahdaripour, and M. D. Aykin, “Calibration and 3D reconstruction of underwater objects with non-single-view projection model by structured light stereo imaging,” Appl. Opt. 55, 6564–6575 (2016). [CrossRef]  

31. T. Nakatani, S. Li, T. Ura, A. Bodenmann, and T. Sakamaki, “3D visual modeling of hydrothermal chimneys using a rotary laser scanning system,” in IEEE Symposium on Underwater Technology and Workshop on Scientific Use of Submarine Cables and Related Technologies (IEEE, 2011), pp. 1–5.

32. M. Leonardi, A. Stahl, M. Gazzea, M. Ludvigsen, I. Rist-Christensen, and S. M. Nornes, “Vision based obstacle avoidance and motion tracking for autonomous behaviors in underwater vehicles,” in IEEE OCEANS (IEEE, 2017), pp. 1–10.

33. E. Kelasidi, K. Y. Pettersen, J. T. Gravdahl, and P. Liljebäck, “Modeling of underwater snake robots,” in IEEE International Conference on Robotics and Automation (IEEE, 2014), pp. 4540–4547.

34. J. Gregson, M. Krimerman, M. B. Hullin, and W. Heidrich, “Stochastic tomography and its applications in 3D imaging of mixing fluids,” ACM Trans. Graph. 31, 52 (2012). [CrossRef]  

35. A. Vainiger, Y. Y. Schechner, T. Treibitz, A. Avni, and D. S. Timor, “Optical wide-field tomography of sediment resuspension,” Opt. Express 27, A766–A778 (2019). [CrossRef]  

36. A. Aides, A. Levis, V. Holodovsky, Y. Y. Schechner, D. Althausen, and A. Vainiger, “Distributed sky imaging radiometry and tomography,” in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2020), pp. 1–12.

37. A. Levis, Y. Y. Schechner, A. B. Davis, and J. Loveridge, “Multi-view polarimetric scattering cloud tomography and retrieval of droplet size,” Remote Sens. 12, 2831 (2020). [CrossRef]  

38. A. Aides, Y. Y. Schechner, V. Holodovsky, M. J. Garay, and A. B. Davis, “Multi sky-view 3D aerosol distribution recovery,” Opt. Express 21, 25820–25833 (2013). [CrossRef]  

39. J. Frank, “Single-particle reconstruction of biological macromolecules in electron microscopy–30 years,” Q. Rev. Biophys. 42, 139–158 (2009). [CrossRef]  

40. R. R. Lederman and A. Singer, “A representation theory perspective on simultaneous alignment and classification,” Appl. Comput. Harmon. Anal. (2019). [CrossRef]  

41. L. Wang, A. Singer, and Z. Wen, “Orientation determination of Cryo-EM images using least unsquared deviations,” SIAM J. Imaging Sci. 6, 2450–2483 (2013). [CrossRef]  

42. A. Levis, Y. Y. Schechner, and R. Talmon, “Statistical tomography of microscopic life,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018), pp. 6411–6420.

43. I. Greenberg and Y. Shkolnisky, “Common lines modeling for reference free Ab-initio reconstruction in cryo-EM,” J. Struct. Biol. 200, 106–117 (2017). [CrossRef]  

44. E. C. Orenstein, D. Ratelle, C. Briseño-Avena, M. L. Carter, P. J. Franks, J. S. Jaffe, and P. L. Roberts, “The Scripps Plankton Camera system: a framework and platform for in situ microscopy,” Limnol. Oceanogr. Methods 18, 681–695 (2020). [CrossRef]  

45. D. A. Agard, Y. Hiraoka, P. Shaw, and J. W. Sedat, “Fluorescence microscopy in three dimensions,” Methods Cell Biol. 30, 353–377 (1989). [CrossRef]  

46. D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Ann. Rev. Biophys. Bioeng. 13, 191–219 (1984). [CrossRef]  

47. M. Alterman, Y. Y. Schechner, M. Vo, and S. G. Narasimhan, “Passive tomography of turbulence strength,” in European Conference on Computer Vision (ECCV) (2014), pp. 47–60.

48. V. Holodovsky, Y. Y. Schechner, A. Levin, A. Levis, and A. Aides, “In-situ multi-view multi-scattering stochastic tomography,” in IEEE International Conference on Computational Photography (ICCP) (2016), pp. 1–12.

49. D. Gürsoy, Y. P. Hong, K. He, K. Hujsak, S. Yoo, S. Chen, Y. Li, M. Ge, L. M. Miller, Y. S. Chu, and V. De Andrade, “Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection,” Sci. Rep. 7, 11818 (2017). [CrossRef]  

50. I. Ihrke and M. Magnor, “Image-based tomographic reconstruction of flames,” in ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Eurographics Association, 2004), pp. 365–373.

51. A. Levis, Y. Y. Schechner, and A. B. Davis, “Multiple-scattering microphysics tomography,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 6740–6749.

52. D. Ren, C. Ophus, M. Chen, and L. Waller, “A multiple scattering algorithm for three dimensional phase contrast atomic electron tomography,” Ultramicroscopy 208, 112860 (2020). [CrossRef]  

53. B. D. Trifonov, “Tomographic reconstruction of transparent objects,” Ph.D. thesis (University of British Columbia, 2006).

54. A. Veeraraghavan, A. V. Genkin, S. Vitaladevuni, L. Scheffer, S. Xu, H. Hess, R. Fetter, M. Cantoni, G. Knott, and D. Chklovskii, “Increasing depth resolution of electron microscopy of neural circuits using sparse tomographic reconstruction,” in IEEE Conference on Computer Vision and Pattern Recognition (2010), pp. 1767–1774.

55. Y. Shkolnisky and A. Singer, “Viewing direction estimation in cryo-EM using synchronization,” SIAM J. Imaging Sci. 5, 1088–1110 (2012). [CrossRef]  

56. R. Gordon, R. Bender, and G. T. Herman, “Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography,” J. Theor. Biol. 29, 471–481 (1970). [CrossRef]  

57. W. van Aarle, W. J. Palenstijn, J. Cant, E. Janssens, F. Bleichrodt, A. Dabravolski, J. De Beenhouwer, K. J. Batenburg, and J. Sijbers, “Fast and flexible X-ray tomography using the ASTRA toolbox,” Opt. Express 24, 25129–25147 (2016). [CrossRef]  

58. Z. V. Finkel, J. Beardall, K. J. Flynn, A. Quigg, T. A. V. Rees, and J. A. Raven, “Phytoplankton in a changing world: cell size and elemental stoichiometry,” J. Plankton Res. 32, 119–137 (2009). [CrossRef]  

59. C. L. Lawson, M. L. Baker, C. Best, C. Bi, M. Dougherty, P. Feng, G. Van Ginkel, B. Devkota, I. Lagerstedt, S. J. Ludtke, and R. Newman, “EMDataBank.org: unified data resource for CryoEM,” Nucleic Acids Res. 39, D456–D464 (2010). [CrossRef]  

60. H. Sosik, E. E. Peacock, and E. Brownlee, “Annotated plankton images data set for developing and evaluating classification methods,” WHOAS2015, http://darchive.mblwhoilibrary.org/handle/1912/7341.

References

  • View by:

  1. G. C. Hays, A. J. Richardson, and C. Robinson, “Climate change and marine plankton,” Trends Ecol. Evol. 20, 337–344 (2005).
    [Crossref]
  2. J. Ryan, R. Kudela, J. Birch, M. Blum, H. Bowers, F. Chavez, G. Doucette, K. Hayashi, R. Marin, C. Mikulski, and J. Pennington, “Causality of an extreme harmful algal bloom in Monterey Bay, California, during the 2014–2016 northeast Pacific warm anomaly,” Geophys. Res. Lett. 44, 5571–5579 (2017).
    [Crossref]
  3. C. Anderson, E. Berdalet, R. Kudela, C. Cusack, J. Silke, E. O’Rourke, D. Dugan, M. McCammon, J. Newton, S. Moore, and K. Paige, “Scaling up from regional case studies to a global harmful algal bloom observing system,” Front. Mar. Sci. 6, 250 (2019).
    [Crossref]
  4. “Scripps plankton camera website,” http://spc.ucsd.edu .
  5. G. Gorsky, M. Picheral, and L. Stemmann, “Use of the underwater video profiler for the study of aggregate dynamics in the North Mediterranean,” Estuarine Coastal Shelf Sci. 50, 121–128 (2000).
    [Crossref]
  6. C. S. Davis, S. M. Gallager, M. Marra, and W. K. Stewart, “Rapid visualization of plankton abundance and taxonomic composition using the video plankton recorder,” Deep Sea Res. II 43, 1947–1970 (1996).
    [Crossref]
  7. M. D. Ohman, R. E. Davis, J. T. Sherman, K. R. Grindley, B. M. Whitmore, C. F. Nickels, and J. S. Ellen, “Zooglider: an autonomous vehicle for optical and acoustic sensing of zooplankton,” Limnol. Oceanogr. Methods 17, 69–86 (2019).
    [Crossref]
  8. R. K. Cowen and C. M. Guigand, “In situ ichthyoplankton imaging system (I SIIS): system design and preliminary results,” Limnol. Oceanogr. Methods 6, 126–132 (2008).
    [Crossref]
  9. S. Vogel, Life in Moving Fluids: The Physical Biology of Flow-Revised and Expanded Second Edition (Princeton University, 2020).
  10. S. Lee, K. Kim, A. Mubarok, A. Panduwirawan, K. Lee, S. Lee, H. Park, and Y. Park, “High-resolution 3D refractive index tomography and 2D synthetic aperture imaging of live phytoplankton,” J. Opt. Soc. Korea 18, 691–697 (2014).
    [Crossref]
  11. S. Johnsen, The Optics of Life: A Biologist’s Guide to Light in Nature (Princeton University, 2012).
  12. S. Colin, L. P. Coelho, S. Sunagawa, C. Bowler, E. Karsenti, P. Bork, R. Pepperkok, and C. De Vargas, “Quantitative 3D-imaging for cell biology and ecology of environmental microbial eukaryotes,” Elife 6, e26066 (2017).
    [Crossref]
  13. F. Lombard, E. Boss, A. M. Waite, J. Uitz, L. Stemmann, H. M. Sosik, J. Schulz, J.-B. Romagnan, M. Picheral, and J. Pearlman, “Globally consistent quantitative observations of planktonic ecosystems,” Front. Mar. Sci. 6, 196 (2019).
    [Crossref]
  14. M. Picheral, L. Guidi, L. Stemmann, D. M. Karl, G. Iddaoud, and G. Gorsky, “The underwater vision profiler 5: an advanced instrument for high spatial resolution studies of particle size spectra and zooplankton,” Limnol. Oceanogr. Methods 8, 462–473 (2010).
    [Crossref]
  15. R. J. Olson and H. M. Sosik, “A submersible imaging-in-flow instrument to analyze nano-and microplankton: imaging FlowCytobot,” Limnol. Oceanogr. Methods 5, 195–203 (2007).
    [Crossref]
  16. J. S. Jaffe, “Underwater optical imaging: the past, the present, and the prospects,” IEEE J. Ocean. Eng. 40, 683–700 (2014).
    [Crossref]
  17. A. D. Mullen, T. Treibitz, P. L. Roberts, and J. S. Jaffe, “An underwater microscope for in situ imaging of seafloor organism,” in Novel Techniques in Microscopy (OSA, 2017), paper NTu1C–1.
  18. E. Acevedo-Trejos, G. Brandt, J. Bruggeman, and A. Merico, “Mechanisms shaping size structure and functional diversity of phytoplankton communities in the ocean,” Sci. Rep. 5, 8918 (2015).
    [Crossref]
  19. H. Sun, P. W. Benzie, N. Burns, D. C. Hendry, M. A. Player, and J. Watson, “Underwater digital holography for studies of marine plankton,” Philos. Trans. R. Soc. A 366, 1789–1806 (2008).
    [Crossref]
  20. E. J. Davies and R. Nepstad, “In situ characterisation of complex suspended particulates surrounding an active submarine tailings placement site in a Norwegian fjord,” Regional Stud. Mar. Sci. 16, 198–207 (2017).
    [Crossref]
  21. J. Sheng, E. Malkiel, and J. Katz, “Digital holographic microscope for measuring three-dimensional particle distributions and motions,” Appl. Opt. 45, 3893–3901 (2006).
    [Crossref]
  22. S. Samson, T. Hopkins, A. Remsen, L. Langebrake, T. Sutton, and J. Patten, “A system for high-resolution zooplankton imaging,” IEEE J. Ocean. Eng. 26, 671–676 (2001).
    [Crossref]
  23. A. Zweifler, D. Akkaynak, and T. Mass, and T. Treibitz, “In situ analysis of coral recruits using fluorescence imaging,” Front. Mar. Sci. 4, 273 (2017).
    [Crossref]
  24. D. Akkaynak, T. Treibitz, B. Xiao, U. A. Gürkan, J. J. Allen, U. Demirci, and R. T. Hanlon, “Use of commercial off-the-shelf digital cameras for scientific data acquisition and scene-specific color calibration,” J. Opt. Soc. Am. A 31, 312–321 (2014).
    [Crossref]
  25. D. M. Kocak, F. R. Dalgleish, F. M. Caimi, and Y. Y. Schechner, “A focus on recent developments and trends in underwater imaging,” Mar. Technol. Soc. J. 42, 52–67 (2008).
    [Crossref]
  26. A. Jordt-Sedlazeck and R. Koch, “Refractive structure-from-motion on underwater images,” in IEEE International Conference on Computer Vision (2013), pp. 57–64.
  27. C. Ancuti, C. O. Ancuti, C. De Vleeschouwer, R. Garcia, and A. C. Bovik, “Multi-scale underwater descattering,” in IEEE International Conference on Pattern Recognition (IEEE, 2016), pp. 4202–4207.
  28. M. Sheinin and Y. Y. Schechner, “The next best underwater view,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 3764–3773.
  29. H. Singh, J. G. Bellingham, F. Hover, S. Lemer, B. A. Moran, K. Von der Heydt, and D. Yoerger, “Docking for an autonomous ocean sampling network,” IEEE J. Ocean. Eng. 26, 498–514 (2001).
    [Crossref]
  30. Y. Wang, S. Negahdaripour, and M. D. Aykin, “Calibration and 3D reconstruction of underwater objects with non-single-view projection model by structured light stereo imaging,” Appl. Opt. 55, 6564–6575 (2016).
    [Crossref]
  31. T. Nakatani, S. Li, T. Ura, A. Bodenmann, and T. Sakamaki, “3D visual modeling of hydrothermal chimneys using a rotary laser scanning system,” in IEEE Symposium on Underwater Technology and Workshop on Scientific Use of Submarine Cables and Related Technologies (IEEE, 2011), pp. 1–5.
  32. M. Leonardi, A. Stahl, M. Gazzea, M. Ludvigsen, I. Rist-Christensen, and S. M. Nornes, “Vision based obstacle avoidance and motion tracking for autonomous behaviors in underwater vehicles,” in IEEE OCEANS (IEEE, 2017), pp. 1–10.
  33. E. Kelasidi, K. Y. Pettersen, J. T. Gravdahl, and P. Liljebäck, “Modeling of underwater snake robots,” in IEEE International Conference on Robotics and Automation (IEEE, 2014), pp. 4540–4547.
  34. J. Gregson, M. Krimerman, M. B. Hullin, and W. Heidrich, “Stochastic tomography and its applications in 3D imaging of mixing fluids,” ACM Trans. Graph. 31, 52 (2012).
    [Crossref]
  35. A. Vainiger, Y. Y. Schechner, T. Treibitz, A. Avni, and D. S. Timor, “Optical wide-field tomography of sediment resuspension,” Opt. Express 27, A766–A778 (2019).
    [Crossref]
  36. A. Aides, A. Levis, V. Holodovsky, Y. Y. Schechner, D. Althausen, and A. Vainiger, “Distributed sky imaging radiometry and tomography,” in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2020), pp. 1–12.
  37. A. Levis, Y. Y. Schechner, A. B. Davis, and J. Loveridge, “Multi-view polarimetric scattering cloud tomography and retrieval of droplet size,” Remote Sens. 12, 2831 (2020).
    [Crossref]
  38. A. Aides, Y. Y. Schechner, V. Holodovsky, M. J. Garay, and A. B. Davis, “Multi sky-view 3D aerosol distribution recovery,” Opt. Express 21, 25820–25833 (2013).
    [Crossref]
  39. J. Frank, “Single-particle reconstruction of biological macromolecules in electron microscopy–30 years,” Q. Rev. Biophys. 42, 139–158 (2009).
    [Crossref]
  40. R. R. Lederman and A. Singer, “A representation theory perspective on simultaneous alignment and classification,” Appl. Comput. Harmon. Anal. (2019).
    [Crossref]
  41. L. Wang, A. Singer, and Z. Wen, “Orientation determination of Cryo-EM images using least unsquared deviations,” SIAM J. Imaging Sci. 6, 2450–2483 (2013).
    [Crossref]
  42. A. Levis, Y. Y. Schechner, and R. Talmon, “Statistical tomography of microscopic life,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018), pp. 6411–6420.
  43. I. Greenberg and Y. Shkolnisky, “Common lines modeling for reference free Ab-initio reconstruction in cryo-EM,” J. Struct. Biol. 200, 106–117 (2017).
    [Crossref]
  44. E. C. Orenstein, D. Ratelle, C. Briseño-Avena, M. L. Carter, P. J. Franks, J. S. Jaffe, and P. L. Roberts, “The Scripps Plankton Camera system: a framework and platform for in situ microscopy,” Limnol. Oceanogr. Methods 18, 681–695 (2020).
    [Crossref]
  45. D. A. Agard, Y. Hiraoka, P. Shaw, and J. W. Sedat, “Fluorescence microscopy in three dimensions,” Methods Cell Biol. 30, 353–377 (1989).
    [Crossref]
  46. D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Ann. Rev. Biophys. Bioeng. 13, 191–219 (1984).
    [Crossref]
  47. M. Alterman, Y. Y. Schechner, M. Vo, and S. G. Narasimhan, “Passive tomography of turbulence strength,” in European Conference on Computer Vision (ECCV) (2014), pp. 47–60.
  48. V. Holodovsky, Y. Y. Schechner, A. Levin, A. Levis, and A. Aides, “In-situ multi-view multi-scattering stochastic tomography,” in IEEE International Conference on Computational Photography (ICCP) (2016), pp. 1–12.
  49. D. Gürsoy, Y. P. Hong, K. He, K. Hujsak, S. Yoo, S. Chen, Y. Li, M. Ge, L. M. Miller, Y. S. Chu, and V. De Andrade, “Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection,” Sci. Rep. 7, 11818 (2017).
    [Crossref]
  50. I. Ihrke and M. Magnor, “Image-based tomographic reconstruction of flames,” in ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Eurographics Association, 2004), pp. 365–373.
  51. A. Levis, Y. Y. Schechner, and A. B. Davis, “Multiple-scattering microphysics tomography,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 6740–6749.
  52. D. Ren, C. Ophus, M. Chen, and L. Waller, “A multiple scattering algorithm for three dimensional phase contrast atomic electron tomography,” Ultramicroscopy 208, 112860 (2020).
    [Crossref]
  53. B. D. Trifonov, “Tomographic reconstruction of transparent objects,” Ph.D. thesis (University of British Columbia, 2006).
  54. A. Veeraraghavan, A. V. Genkin, S. Vitaladevuni, L. Scheffer, S. Xu, H. Hess, R. Fetter, M. Cantoni, G. Knott, and D. Chklovskii, “Increasing depth resolution of electron microscopy of neural circuits using sparse tomographic reconstruction,” in IEEE Conference on Computer Vision and Pattern Recognition (2010), pp. 1767–1774.
  55. Y. Shkolnisky and A. Singer, “Viewing direction estimation in cryo-EM using synchronization,” SIAM J. Imaging Sci. 5, 1088–1110 (2012).
    [Crossref]
  56. R. Gordon, R. Bender, and G. T. Herman, “Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography,” J. Theor. Biol. 29, 471–481 (1970).
    [Crossref]
  57. W. van Aarle, W. J. Palenstijn, J. Cant, E. Janssens, F. Bleichrodt, A. Dabravolski, J. De Beenhouwer, K. J. Batenburg, and J. Sijbers, “Fast and flexible X-ray tomography using the ASTRA toolbox,” Opt. Express 24, 25129–25147 (2016).
    [Crossref]
  58. Z. V. Finkel, J. Beardall, K. J. Flynn, A. Quigg, T. A. V. Rees, and J. A. Raven, “Phytoplankton in a changing world: cell size and elemental stoichiometry,” J. Plankton Res. 32, 119–137 (2009).
    [Crossref]
  59. C. L. Lawson, M. L. Baker, C. Best, C. Bi, M. Dougherty, P. Feng, G. Van Ginkel, B. Devkota, I. Lagerstedt, S. J. Ludtke, and R. Newman, “EMDataBank.org: unified data resource for CryoEM,” Nucleic Acids Res. 39, D456–D464 (2010).
    [Crossref]
  60. H. Sosik, E. E. Peacock, and E. Brownlee, “Annotated plankton images data set for developing and evaluating classification methods,” WHOAS2015, http://darchive.mblwhoilibrary.org/handle/1912/7341 .

2020 (3)

A. Levis, Y. Y. Schechner, A. B. Davis, and J. Loveridge, “Multi-view polarimetric scattering cloud tomography and retrieval of droplet size,” Remote Sens. 12, 2831 (2020).
[Crossref]

E. C. Orenstein, D. Ratelle, C. Briseño-Avena, M. L. Carter, P. J. Franks, J. S. Jaffe, and P. L. Roberts, “The Scripps Plankton Camera system: a framework and platform for in situ microscopy,” Limnol. Oceanogr. Methods 18, 681–695 (2020).
[Crossref]

D. Ren, C. Ophus, M. Chen, and L. Waller, “A multiple scattering algorithm for three dimensional phase contrast atomic electron tomography,” Ultramicroscopy 208, 112860 (2020).
[Crossref]

2019 (4)

A. Vainiger, Y. Y. Schechner, T. Treibitz, A. Avni, and D. S. Timor, “Optical wide-field tomography of sediment resuspension,” Opt. Express 27, A766–A778 (2019).
[Crossref]

C. Anderson, E. Berdalet, R. Kudela, C. Cusack, J. Silke, E. O’Rourke, D. Dugan, M. McCammon, J. Newton, S. Moore, and K. Paige, “Scaling up from regional case studies to a global harmful algal bloom observing system,” Front. Mar. Sci. 6, 250 (2019).
[Crossref]

M. D. Ohman, R. E. Davis, J. T. Sherman, K. R. Grindley, B. M. Whitmore, C. F. Nickels, and J. S. Ellen, “Zooglider: an autonomous vehicle for optical and acoustic sensing of zooplankton,” Limnol. Oceanogr. Methods 17, 69–86 (2019).
[Crossref]

F. Lombard, E. Boss, A. M. Waite, J. Uitz, L. Stemmann, H. M. Sosik, J. Schulz, J.-B. Romagnan, M. Picheral, and J. Pearlman, “Globally consistent quantitative observations of planktonic ecosystems,” Front. Mar. Sci. 6, 196 (2019).
[Crossref]

2017 (6)

A. Zweifler, D. Akkaynak, and T. Mass, and T. Treibitz, “In situ analysis of coral recruits using fluorescence imaging,” Front. Mar. Sci. 4, 273 (2017).
[Crossref]

A. Zweifler, D. Akkaynak, and T. Mass, and T. Treibitz, “In situ analysis of coral recruits using fluorescence imaging,” Front. Mar. Sci. 4, 273 (2017).
[Crossref]

S. Colin, L. P. Coelho, S. Sunagawa, C. Bowler, E. Karsenti, P. Bork, R. Pepperkok, and C. De Vargas, “Quantitative 3D-imaging for cell biology and ecology of environmental microbial eukaryotes,” Elife 6, e26066 (2017).
[Crossref]

J. Ryan, R. Kudela, J. Birch, M. Blum, H. Bowers, F. Chavez, G. Doucette, K. Hayashi, R. Marin, C. Mikulski, and J. Pennington, “Causality of an extreme harmful algal bloom in Monterey Bay, California, during the 2014–2016 northeast Pacific warm anomaly,” Geophys. Res. Lett. 44, 5571–5579 (2017).
[Crossref]

E. J. Davies and R. Nepstad, “In situ characterisation of complex suspended particulates surrounding an active submarine tailings placement site in a Norwegian fjord,” Regional Stud. Mar. Sci. 16, 198–207 (2017).
[Crossref]

I. Greenberg and Y. Shkolnisky, “Common lines modeling for reference free Ab-initio reconstruction in cryo-EM,” J. Struct. Biol. 200, 106–117 (2017).
[Crossref]

D. Gürsoy, Y. P. Hong, K. He, K. Hujsak, S. Yoo, S. Chen, Y. Li, M. Ge, L. M. Miller, Y. S. Chu, and V. De Andrade, “Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection,” Sci. Rep. 7, 11818 (2017).
[Crossref]

2016 (2)

2015 (1)

E. Acevedo-Trejos, G. Brandt, J. Bruggeman, and A. Merico, “Mechanisms shaping size structure and functional diversity of phytoplankton communities in the ocean,” Sci. Rep. 5, 8918 (2015).
[Crossref]

2014 (3)

2013 (2)

A. Aides, Y. Y. Schechner, V. Holodovsky, M. J. Garay, and A. B. Davis, “Multi sky-view 3D aerosol distribution recovery,” Opt. Express 21, 25820–25833 (2013).
[Crossref]

L. Wang, A. Singer, and Z. Wen, “Orientation determination of Cryo-EM images using least unsquared deviations,” SIAM J. Imaging Sci. 6, 2450–2483 (2013).
[Crossref]

2012 (2)

Y. Shkolnisky and A. Singer, “Viewing direction estimation in cryo-EM using synchronization,” SIAM J. Imaging Sci. 5, 1088–1110 (2012).
[Crossref]

J. Gregson, M. Krimerman, M. B. Hullin, and W. Heidrich, “Stochastic tomography and its applications in 3D imaging of mixing fluids,” ACM Trans. Graph. 31, 52 (2012).
[Crossref]

2010 (2)

M. Picheral, L. Guidi, L. Stemmann, D. M. Karl, G. Iddaoud, and G. Gorsky, “The underwater vision profiler 5: an advanced instrument for high spatial resolution studies of particle size spectra and zooplankton,” Limnol. Oceanogr. Methods 8, 462–473 (2010).
[Crossref]

C. L. Lawson, M. L. Baker, C. Best, C. Bi, M. Dougherty, P. Feng, G. Van Ginkel, B. Devkota, I. Lagerstedt, S. J. Ludtke, and R. Newman, “EMDataBank.org: unified data resource for CryoEM,” Nucleic Acids Res. 39, D456–D464 (2010).
[Crossref]

2009 (2)

J. Frank, “Single-particle reconstruction of biological macromolecules in electron microscopy–30 years,” Q. Rev. Biophys. 42, 139–158 (2009).
[Crossref]

Z. V. Finkel, J. Beardall, K. J. Flynn, A. Quigg, T. A. V. Rees, and J. A. Raven, “Phytoplankton in a changing world: cell size and elemental stoichiometry,” J. Plankton Res. 32, 119–137 (2009).
[Crossref]

2008 (3)

H. Sun, P. W. Benzie, N. Burns, D. C. Hendry, M. A. Player, and J. Watson, “Underwater digital holography for studies of marine plankton,” Philos. Trans. R. Soc. A 366, 1789–1806 (2008).
[Crossref]

D. M. Kocak, F. R. Dalgleish, F. M. Caimi, and Y. Y. Schechner, “A focus on recent developments and trends in underwater imaging,” Mar. Technol. Soc. J. 42, 52–67 (2008).
[Crossref]

R. K. Cowen and C. M. Guigand, “In situ ichthyoplankton imaging system (I SIIS): system design and preliminary results,” Limnol. Oceanogr. Methods 6, 126–132 (2008).
[Crossref]

2007 (1)

R. J. Olson and H. M. Sosik, “A submersible imaging-in-flow instrument to analyze nano-and microplankton: imaging FlowCytobot,” Limnol. Oceanogr. Methods 5, 195–203 (2007).
[Crossref]

2006 (1)

2005 (1)

G. C. Hays, A. J. Richardson, and C. Robinson, “Climate change and marine plankton,” Trends Ecol. Evol. 20, 337–344 (2005).
[Crossref]

2001 (2)

H. Singh, J. G. Bellingham, F. Hover, S. Lemer, B. A. Moran, K. Von der Heydt, and D. Yoerger, “Docking for an autonomous ocean sampling network,” IEEE J. Ocean. Eng. 26, 498–514 (2001).
[Crossref]

S. Samson, T. Hopkins, A. Remsen, L. Langebrake, T. Sutton, and J. Patten, “A system for high-resolution zooplankton imaging,” IEEE J. Ocean. Eng. 26, 671–676 (2001).
[Crossref]

2000 (1)

G. Gorsky, M. Picheral, and L. Stemmann, “Use of the underwater video profiler for the study of aggregate dynamics in the North Mediterranean,” Estuarine Coastal Shelf Sci. 50, 121–128 (2000).
[Crossref]

1996 (1)

C. S. Davis, S. M. Gallager, M. Marra, and W. K. Stewart, “Rapid visualization of plankton abundance and taxonomic composition using the video plankton recorder,” Deep Sea Res. II 43, 1947–1970 (1996).
[Crossref]

1989 (1)

D. A. Agard, Y. Hiraoka, P. Shaw, and J. W. Sedat, “Fluorescence microscopy in three dimensions,” Methods Cell Biol. 30, 353–377 (1989).
[Crossref]

1984 (1)

D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Ann. Rev. Biophys. Bioeng. 13, 191–219 (1984).
[Crossref]

1970 (1)

R. Gordon, R. Bender, and G. T. Herman, “Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography,” J. Theor. Biol. 29, 471–481 (1970).
[Crossref]

Acevedo-Trejos, E.

E. Acevedo-Trejos, G. Brandt, J. Bruggeman, and A. Merico, “Mechanisms shaping size structure and functional diversity of phytoplankton communities in the ocean,” Sci. Rep. 5, 8918 (2015).
[Crossref]

Agard, D. A.

D. A. Agard, Y. Hiraoka, P. Shaw, and J. W. Sedat, “Fluorescence microscopy in three dimensions,” Methods Cell Biol. 30, 353–377 (1989).
[Crossref]

D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Ann. Rev. Biophys. Bioeng. 13, 191–219 (1984).
[Crossref]

Aides, A.

A. Aides, Y. Y. Schechner, V. Holodovsky, M. J. Garay, and A. B. Davis, “Multi sky-view 3D aerosol distribution recovery,” Opt. Express 21, 25820–25833 (2013).
[Crossref]

A. Aides, A. Levis, V. Holodovsky, Y. Y. Schechner, D. Althausen, and A. Vainiger, “Distributed sky imaging radiometry and tomography,” in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2020), pp. 1–12.

V. Holodovsky, Y. Y. Schechner, A. Levin, A. Levis, and A. Aides, “In-situ multi-view multi-scattering stochastic tomography,” in IEEE International Conference on Computational Photography (ICCP) (2016), pp. 1–12.

Akkaynak, D.

Allen, J. J.

Alterman, M.

M. Alterman, Y. Y. Schechner, M. Vo, and S. G. Narasimhan, “Passive tomography of turbulence strength,” in European Conference on Computer Vision (ECCV) (2014), pp. 47–60.

Althausen, D.

A. Aides, A. Levis, V. Holodovsky, Y. Y. Schechner, D. Althausen, and A. Vainiger, “Distributed sky imaging radiometry and tomography,” in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2020), pp. 1–12.

Ancuti, C.

C. Ancuti, C. O. Ancuti, C. De Vleeschouwer, R. Garcia, and A. C. Bovik, “Multi-scale underwater descattering,” in IEEE International Conference on Pattern Recognition (IEEE, 2016), pp. 4202–4207.

Ancuti, C. O.

C. Ancuti, C. O. Ancuti, C. De Vleeschouwer, R. Garcia, and A. C. Bovik, “Multi-scale underwater descattering,” in IEEE International Conference on Pattern Recognition (IEEE, 2016), pp. 4202–4207.

Anderson, C.

C. Anderson, E. Berdalet, R. Kudela, C. Cusack, J. Silke, E. O’Rourke, D. Dugan, M. McCammon, J. Newton, S. Moore, and K. Paige, “Scaling up from regional case studies to a global harmful algal bloom observing system,” Front. Mar. Sci. 6, 250 (2019).
[Crossref]

Avni, A.

Aykin, M. D.

Baker, M. L.

C. L. Lawson, M. L. Baker, C. Best, C. Bi, M. Dougherty, P. Feng, G. Van Ginkel, B. Devkota, I. Lagerstedt, S. J. Ludtke, and R. Newman, “EMDataBank.org: unified data resource for CryoEM,” Nucleic Acids Res. 39, D456–D464 (2010).
[Crossref]

Batenburg, K. J.

Beardall, J.

Z. V. Finkel, J. Beardall, K. J. Flynn, A. Quigg, T. A. V. Rees, and J. A. Raven, “Phytoplankton in a changing world: cell size and elemental stoichiometry,” J. Plankton Res. 32, 119–137 (2009).
[Crossref]

Bellingham, J. G.

H. Singh, J. G. Bellingham, F. Hover, S. Lemer, B. A. Moran, K. Von der Heydt, and D. Yoerger, “Docking for an autonomous ocean sampling network,” IEEE J. Ocean. Eng. 26, 498–514 (2001).
[Crossref]

Bender, R.

R. Gordon, R. Bender, and G. T. Herman, “Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography,” J. Theor. Biol. 29, 471–481 (1970).
[Crossref]

Benzie, P. W.

H. Sun, P. W. Benzie, N. Burns, D. C. Hendry, M. A. Player, and J. Watson, “Underwater digital holography for studies of marine plankton,” Philos. Trans. R. Soc. A 366, 1789–1806 (2008).
[Crossref]

Berdalet, E.

C. Anderson, E. Berdalet, R. Kudela, C. Cusack, J. Silke, E. O’Rourke, D. Dugan, M. McCammon, J. Newton, S. Moore, and K. Paige, “Scaling up from regional case studies to a global harmful algal bloom observing system,” Front. Mar. Sci. 6, 250 (2019).
[Crossref]

Best, C.

C. L. Lawson, M. L. Baker, C. Best, C. Bi, M. Dougherty, P. Feng, G. Van Ginkel, B. Devkota, I. Lagerstedt, S. J. Ludtke, and R. Newman, “EMDataBank.org: unified data resource for CryoEM,” Nucleic Acids Res. 39, D456–D464 (2010).
[Crossref]

Bi, C.

C. L. Lawson, M. L. Baker, C. Best, C. Bi, M. Dougherty, P. Feng, G. Van Ginkel, B. Devkota, I. Lagerstedt, S. J. Ludtke, and R. Newman, “EMDataBank.org: unified data resource for CryoEM,” Nucleic Acids Res. 39, D456–D464 (2010).
[Crossref]

Birch, J.

J. Ryan, R. Kudela, J. Birch, M. Blum, H. Bowers, F. Chavez, G. Doucette, K. Hayashi, R. Marin, C. Mikulski, and J. Pennington, “Causality of an extreme harmful algal bloom in Monterey Bay, California, during the 2014–2016 northeast Pacific warm anomaly,” Geophys. Res. Lett. 44, 5571–5579 (2017).
[Crossref]

Bleichrodt, F.

Blum, M.

J. Ryan, R. Kudela, J. Birch, M. Blum, H. Bowers, F. Chavez, G. Doucette, K. Hayashi, R. Marin, C. Mikulski, and J. Pennington, “Causality of an extreme harmful algal bloom in Monterey Bay, California, during the 2014–2016 northeast Pacific warm anomaly,” Geophys. Res. Lett. 44, 5571–5579 (2017).
[Crossref]

Bodenmann, A.

T. Nakatani, S. Li, T. Ura, A. Bodenmann, and T. Sakamaki, “3D visual modeling of hydrothermal chimneys using a rotary laser scanning system,” in IEEE Symposium on Underwater Technology and Workshop on Scientific Use of Submarine Cables and Related Technologies (IEEE, 2011), pp. 1–5.

Bork, P.

S. Colin, L. P. Coelho, S. Sunagawa, C. Bowler, E. Karsenti, P. Bork, R. Pepperkok, and C. De Vargas, “Quantitative 3D-imaging for cell biology and ecology of environmental microbial eukaryotes,” Elife 6, e26066 (2017).
[Crossref]

Boss, E.

F. Lombard, E. Boss, A. M. Waite, J. Uitz, L. Stemmann, H. M. Sosik, J. Schulz, J.-B. Romagnan, M. Picheral, and J. Pearlman, “Globally consistent quantitative observations of planktonic ecosystems,” Front. Mar. Sci. 6, 196 (2019).
[Crossref]

Bovik, A. C.

C. Ancuti, C. O. Ancuti, C. De Vleeschouwer, R. Garcia, and A. C. Bovik, “Multi-scale underwater descattering,” in IEEE International Conference on Pattern Recognition (IEEE, 2016), pp. 4202–4207.

Bowers, H.

J. Ryan, R. Kudela, J. Birch, M. Blum, H. Bowers, F. Chavez, G. Doucette, K. Hayashi, R. Marin, C. Mikulski, and J. Pennington, “Causality of an extreme harmful algal bloom in Monterey Bay, California, during the 2014–2016 northeast Pacific warm anomaly,” Geophys. Res. Lett. 44, 5571–5579 (2017).
[Crossref]

Bowler, C.

S. Colin, L. P. Coelho, S. Sunagawa, C. Bowler, E. Karsenti, P. Bork, R. Pepperkok, and C. De Vargas, “Quantitative 3D-imaging for cell biology and ecology of environmental microbial eukaryotes,” Elife 6, e26066 (2017).
[Crossref]

Brandt, G.

E. Acevedo-Trejos, G. Brandt, J. Bruggeman, and A. Merico, “Mechanisms shaping size structure and functional diversity of phytoplankton communities in the ocean,” Sci. Rep. 5, 8918 (2015).
[Crossref]

Briseño-Avena, C.

E. C. Orenstein, D. Ratelle, C. Briseño-Avena, M. L. Carter, P. J. Franks, J. S. Jaffe, and P. L. Roberts, “The Scripps Plankton Camera system: a framework and platform for in situ microscopy,” Limnol. Oceanogr. Methods 18, 681–695 (2020).
[Crossref]

Bruggeman, J.

E. Acevedo-Trejos, G. Brandt, J. Bruggeman, and A. Merico, “Mechanisms shaping size structure and functional diversity of phytoplankton communities in the ocean,” Sci. Rep. 5, 8918 (2015).
[Crossref]

Burns, N.

H. Sun, P. W. Benzie, N. Burns, D. C. Hendry, M. A. Player, and J. Watson, “Underwater digital holography for studies of marine plankton,” Philos. Trans. R. Soc. A 366, 1789–1806 (2008).
[Crossref]

Caimi, F. M.

D. M. Kocak, F. R. Dalgleish, F. M. Caimi, and Y. Y. Schechner, “A focus on recent developments and trends in underwater imaging,” Mar. Technol. Soc. J. 42, 52–67 (2008).
[Crossref]

Cant, J.

Cantoni, M.

A. Veeraraghavan, A. V. Genkin, S. Vitaladevuni, L. Scheffer, S. Xu, H. Hess, R. Fetter, M. Cantoni, G. Knott, and D. Chklovskii, “Increasing depth resolution of electron microscopy of neural circuits using sparse tomographic reconstruction,” in IEEE Conference on Computer Vision and Pattern Recognition (2010), pp. 1767–1774.

Carter, M. L.

E. C. Orenstein, D. Ratelle, C. Briseño-Avena, M. L. Carter, P. J. Franks, J. S. Jaffe, and P. L. Roberts, “The Scripps Plankton Camera system: a framework and platform for in situ microscopy,” Limnol. Oceanogr. Methods 18, 681–695 (2020).
[Crossref]

Chavez, F.

J. Ryan, R. Kudela, J. Birch, M. Blum, H. Bowers, F. Chavez, G. Doucette, K. Hayashi, R. Marin, C. Mikulski, and J. Pennington, “Causality of an extreme harmful algal bloom in Monterey Bay, California, during the 2014–2016 northeast Pacific warm anomaly,” Geophys. Res. Lett. 44, 5571–5579 (2017).
[Crossref]

Chen, M.

D. Ren, C. Ophus, M. Chen, and L. Waller, “A multiple scattering algorithm for three dimensional phase contrast atomic electron tomography,” Ultramicroscopy 208, 112860 (2020).
[Crossref]

Chen, S.

D. Gürsoy, Y. P. Hong, K. He, K. Hujsak, S. Yoo, S. Chen, Y. Li, M. Ge, L. M. Miller, Y. S. Chu, and V. De Andrade, “Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection,” Sci. Rep. 7, 11818 (2017).
[Crossref]

Chklovskii, D.

A. Veeraraghavan, A. V. Genkin, S. Vitaladevuni, L. Scheffer, S. Xu, H. Hess, R. Fetter, M. Cantoni, G. Knott, and D. Chklovskii, “Increasing depth resolution of electron microscopy of neural circuits using sparse tomographic reconstruction,” in IEEE Conference on Computer Vision and Pattern Recognition (2010), pp. 1767–1774.

Chu, Y. S.

D. Gürsoy, Y. P. Hong, K. He, K. Hujsak, S. Yoo, S. Chen, Y. Li, M. Ge, L. M. Miller, Y. S. Chu, and V. De Andrade, “Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection,” Sci. Rep. 7, 11818 (2017).
[Crossref]

Coelho, L. P.

S. Colin, L. P. Coelho, S. Sunagawa, C. Bowler, E. Karsenti, P. Bork, R. Pepperkok, and C. De Vargas, “Quantitative 3D-imaging for cell biology and ecology of environmental microbial eukaryotes,” Elife 6, e26066 (2017).
[Crossref]

Colin, S.

S. Colin, L. P. Coelho, S. Sunagawa, C. Bowler, E. Karsenti, P. Bork, R. Pepperkok, and C. De Vargas, “Quantitative 3D-imaging for cell biology and ecology of environmental microbial eukaryotes,” Elife 6, e26066 (2017).
[Crossref]

Cowen, R. K.

R. K. Cowen and C. M. Guigand, “In situ ichthyoplankton imaging system (I SIIS): system design and preliminary results,” Limnol. Oceanogr. Methods 6, 126–132 (2008).
[Crossref]

Cusack, C.

C. Anderson, E. Berdalet, R. Kudela, C. Cusack, J. Silke, E. O’Rourke, D. Dugan, M. McCammon, J. Newton, S. Moore, and K. Paige, “Scaling up from regional case studies to a global harmful algal bloom observing system,” Front. Mar. Sci. 6, 250 (2019).
[Crossref]

Dabravolski, A.

Dalgleish, F. R.

D. M. Kocak, F. R. Dalgleish, F. M. Caimi, and Y. Y. Schechner, “A focus on recent developments and trends in underwater imaging,” Mar. Technol. Soc. J. 42, 52–67 (2008).
[Crossref]

Davies, E. J.

E. J. Davies and R. Nepstad, “In situ characterisation of complex suspended particulates surrounding an active submarine tailings placement site in a Norwegian fjord,” Regional Stud. Mar. Sci. 16, 198–207 (2017).
[Crossref]

Davis, A. B.

A. Levis, Y. Y. Schechner, A. B. Davis, and J. Loveridge, “Multi-view polarimetric scattering cloud tomography and retrieval of droplet size,” Remote Sens. 12, 2831 (2020).
[Crossref]

A. Aides, Y. Y. Schechner, V. Holodovsky, M. J. Garay, and A. B. Davis, “Multi sky-view 3D aerosol distribution recovery,” Opt. Express 21, 25820–25833 (2013).
[Crossref]

A. Levis, Y. Y. Schechner, and A. B. Davis, “Multiple-scattering microphysics tomography,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 6740–6749.

Davis, C. S.

C. S. Davis, S. M. Gallager, M. Marra, and W. K. Stewart, “Rapid visualization of plankton abundance and taxonomic composition using the video plankton recorder,” Deep Sea Res. II 43, 1947–1970 (1996).
[Crossref]

Davis, R. E.

M. D. Ohman, R. E. Davis, J. T. Sherman, K. R. Grindley, B. M. Whitmore, C. F. Nickels, and J. S. Ellen, “Zooglider: an autonomous vehicle for optical and acoustic sensing of zooplankton,” Limnol. Oceanogr. Methods 17, 69–86 (2019).
[Crossref]

De Andrade, V.

D. Gürsoy, Y. P. Hong, K. He, K. Hujsak, S. Yoo, S. Chen, Y. Li, M. Ge, L. M. Miller, Y. S. Chu, and V. De Andrade, “Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection,” Sci. Rep. 7, 11818 (2017).
[Crossref]

De Beenhouwer, J.

De Vargas, C.

S. Colin, L. P. Coelho, S. Sunagawa, C. Bowler, E. Karsenti, P. Bork, R. Pepperkok, and C. De Vargas, “Quantitative 3D-imaging for cell biology and ecology of environmental microbial eukaryotes,” Elife 6, e26066 (2017).
[Crossref]

De Vleeschouwer, C.

C. Ancuti, C. O. Ancuti, C. De Vleeschouwer, R. Garcia, and A. C. Bovik, “Multi-scale underwater descattering,” in IEEE International Conference on Pattern Recognition (IEEE, 2016), pp. 4202–4207.

Demirci, U.

Devkota, B.

C. L. Lawson, M. L. Baker, C. Best, C. Bi, M. Dougherty, P. Feng, G. Van Ginkel, B. Devkota, I. Lagerstedt, S. J. Ludtke, and R. Newman, “EMDataBank.org: unified data resource for CryoEM,” Nucleic Acids Res. 39, D456–D464 (2010).
[Crossref]

Doucette, G.

J. Ryan, R. Kudela, J. Birch, M. Blum, H. Bowers, F. Chavez, G. Doucette, K. Hayashi, R. Marin, C. Mikulski, and J. Pennington, “Causality of an extreme harmful algal bloom in Monterey Bay, California, during the 2014–2016 northeast Pacific warm anomaly,” Geophys. Res. Lett. 44, 5571–5579 (2017).
[Crossref]

Dougherty, M.

C. L. Lawson, M. L. Baker, C. Best, C. Bi, M. Dougherty, P. Feng, G. Van Ginkel, B. Devkota, I. Lagerstedt, S. J. Ludtke, and R. Newman, “EMDataBank.org: unified data resource for CryoEM,” Nucleic Acids Res. 39, D456–D464 (2010).
[Crossref]

Dugan, D.

C. Anderson, E. Berdalet, R. Kudela, C. Cusack, J. Silke, E. O’Rourke, D. Dugan, M. McCammon, J. Newton, S. Moore, and K. Paige, “Scaling up from regional case studies to a global harmful algal bloom observing system,” Front. Mar. Sci. 6, 250 (2019).
[Crossref]

Ellen, J. S.

M. D. Ohman, R. E. Davis, J. T. Sherman, K. R. Grindley, B. M. Whitmore, C. F. Nickels, and J. S. Ellen, “Zooglider: an autonomous vehicle for optical and acoustic sensing of zooplankton,” Limnol. Oceanogr. Methods 17, 69–86 (2019).
[Crossref]

Feng, P.

C. L. Lawson, M. L. Baker, C. Best, C. Bi, M. Dougherty, P. Feng, G. Van Ginkel, B. Devkota, I. Lagerstedt, S. J. Ludtke, and R. Newman, “EMDataBank.org: unified data resource for CryoEM,” Nucleic Acids Res. 39, D456–D464 (2010).
[Crossref]

Fetter, R.

A. Veeraraghavan, A. V. Genkin, S. Vitaladevuni, L. Scheffer, S. Xu, H. Hess, R. Fetter, M. Cantoni, G. Knott, and D. Chklovskii, “Increasing depth resolution of electron microscopy of neural circuits using sparse tomographic reconstruction,” in IEEE Conference on Computer Vision and Pattern Recognition (2010), pp. 1767–1774.

Finkel, Z. V.

Z. V. Finkel, J. Beardall, K. J. Flynn, A. Quigg, T. A. V. Rees, and J. A. Raven, “Phytoplankton in a changing world: cell size and elemental stoichiometry,” J. Plankton Res. 32, 119–137 (2009).
[Crossref]

Flynn, K. J.

Z. V. Finkel, J. Beardall, K. J. Flynn, A. Quigg, T. A. V. Rees, and J. A. Raven, “Phytoplankton in a changing world: cell size and elemental stoichiometry,” J. Plankton Res. 32, 119–137 (2009).
[Crossref]

Frank, J.

J. Frank, “Single-particle reconstruction of biological macromolecules in electron microscopy–30 years,” Q. Rev. Biophys. 42, 139–158 (2009).
[Crossref]

Franks, P. J.

E. C. Orenstein, D. Ratelle, C. Briseño-Avena, M. L. Carter, P. J. Franks, J. S. Jaffe, and P. L. Roberts, “The Scripps Plankton Camera system: a framework and platform for in situ microscopy,” Limnol. Oceanogr. Methods 18, 681–695 (2020).
[Crossref]

Gallager, S. M.

C. S. Davis, S. M. Gallager, M. Marra, and W. K. Stewart, “Rapid visualization of plankton abundance and taxonomic composition using the video plankton recorder,” Deep Sea Res. II 43, 1947–1970 (1996).
[Crossref]

Garay, M. J.

Garcia, R.

C. Ancuti, C. O. Ancuti, C. De Vleeschouwer, R. Garcia, and A. C. Bovik, “Multi-scale underwater descattering,” in IEEE International Conference on Pattern Recognition (IEEE, 2016), pp. 4202–4207.

Gazzea, M.

M. Leonardi, A. Stahl, M. Gazzea, M. Ludvigsen, I. Rist-Christensen, and S. M. Nornes, “Vision based obstacle avoidance and motion tracking for autonomous behaviors in underwater vehicles,” in IEEE OCEANS (IEEE, 2017), pp. 1–10.

Ge, M.

D. Gürsoy, Y. P. Hong, K. He, K. Hujsak, S. Yoo, S. Chen, Y. Li, M. Ge, L. M. Miller, Y. S. Chu, and V. De Andrade, “Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection,” Sci. Rep. 7, 11818 (2017).
[Crossref]

Genkin, A. V.

A. Veeraraghavan, A. V. Genkin, S. Vitaladevuni, L. Scheffer, S. Xu, H. Hess, R. Fetter, M. Cantoni, G. Knott, and D. Chklovskii, “Increasing depth resolution of electron microscopy of neural circuits using sparse tomographic reconstruction,” in IEEE Conference on Computer Vision and Pattern Recognition (2010), pp. 1767–1774.

Gordon, R.

R. Gordon, R. Bender, and G. T. Herman, “Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography,” J. Theor. Biol. 29, 471–481 (1970).
[Crossref]

Gorsky, G.

M. Picheral, L. Guidi, L. Stemmann, D. M. Karl, G. Iddaoud, and G. Gorsky, “The underwater vision profiler 5: an advanced instrument for high spatial resolution studies of particle size spectra and zooplankton,” Limnol. Oceanogr. Methods 8, 462–473 (2010).
[Crossref]

G. Gorsky, M. Picheral, and L. Stemmann, “Use of the underwater video profiler for the study of aggregate dynamics in the North Mediterranean,” Estuarine Coastal Shelf Sci. 50, 121–128 (2000).
[Crossref]

Gravdahl, J. T.

E. Kelasidi, K. Y. Pettersen, J. T. Gravdahl, and P. Liljebäck, “Modeling of underwater snake robots,” in IEEE International Conference on Robotics and Automation (IEEE, 2014), pp. 4540–4547.

Greenberg, I.

I. Greenberg and Y. Shkolnisky, “Common lines modeling for reference free Ab-initio reconstruction in cryo-EM,” J. Struct. Biol. 200, 106–117 (2017).
[Crossref]

Gregson, J.

J. Gregson, M. Krimerman, M. B. Hullin, and W. Heidrich, “Stochastic tomography and its applications in 3D imaging of mixing fluids,” ACM Trans. Graph. 31, 52 (2012).
[Crossref]

Grindley, K. R.

M. D. Ohman, R. E. Davis, J. T. Sherman, K. R. Grindley, B. M. Whitmore, C. F. Nickels, and J. S. Ellen, “Zooglider: an autonomous vehicle for optical and acoustic sensing of zooplankton,” Limnol. Oceanogr. Methods 17, 69–86 (2019).
[Crossref]

Guidi, L.

M. Picheral, L. Guidi, L. Stemmann, D. M. Karl, G. Iddaoud, and G. Gorsky, “The underwater vision profiler 5: an advanced instrument for high spatial resolution studies of particle size spectra and zooplankton,” Limnol. Oceanogr. Methods 8, 462–473 (2010).
[Crossref]

Guigand, C. M.

R. K. Cowen and C. M. Guigand, “In situ ichthyoplankton imaging system (I SIIS): system design and preliminary results,” Limnol. Oceanogr. Methods 6, 126–132 (2008).
[Crossref]

Gürkan, U. A.

Gürsoy, D.

D. Gürsoy, Y. P. Hong, K. He, K. Hujsak, S. Yoo, S. Chen, Y. Li, M. Ge, L. M. Miller, Y. S. Chu, and V. De Andrade, “Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection,” Sci. Rep. 7, 11818 (2017).
[Crossref]

Hanlon, R. T.

Hayashi, K.

J. Ryan, R. Kudela, J. Birch, M. Blum, H. Bowers, F. Chavez, G. Doucette, K. Hayashi, R. Marin, C. Mikulski, and J. Pennington, “Causality of an extreme harmful algal bloom in Monterey Bay, California, during the 2014–2016 northeast Pacific warm anomaly,” Geophys. Res. Lett. 44, 5571–5579 (2017).
[Crossref]

Hays, G. C.

G. C. Hays, A. J. Richardson, and C. Robinson, “Climate change and marine plankton,” Trends Ecol. Evol. 20, 337–344 (2005).
[Crossref]

He, K.

D. Gürsoy, Y. P. Hong, K. He, K. Hujsak, S. Yoo, S. Chen, Y. Li, M. Ge, L. M. Miller, Y. S. Chu, and V. De Andrade, “Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection,” Sci. Rep. 7, 11818 (2017).
[Crossref]

Heidrich, W.

J. Gregson, M. Krimerman, M. B. Hullin, and W. Heidrich, “Stochastic tomography and its applications in 3D imaging of mixing fluids,” ACM Trans. Graph. 31, 52 (2012).
[Crossref]

Hendry, D. C.

H. Sun, P. W. Benzie, N. Burns, D. C. Hendry, M. A. Player, and J. Watson, “Underwater digital holography for studies of marine plankton,” Philos. Trans. R. Soc. A 366, 1789–1806 (2008).
[Crossref]

Herman, G. T.

R. Gordon, R. Bender, and G. T. Herman, “Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography,” J. Theor. Biol. 29, 471–481 (1970).
[Crossref]

Hess, H.

A. Veeraraghavan, A. V. Genkin, S. Vitaladevuni, L. Scheffer, S. Xu, H. Hess, R. Fetter, M. Cantoni, G. Knott, and D. Chklovskii, “Increasing depth resolution of electron microscopy of neural circuits using sparse tomographic reconstruction,” in IEEE Conference on Computer Vision and Pattern Recognition (2010), pp. 1767–1774.

Hiraoka, Y.

D. A. Agard, Y. Hiraoka, P. Shaw, and J. W. Sedat, “Fluorescence microscopy in three dimensions,” Methods Cell Biol. 30, 353–377 (1989).
[Crossref]

Holodovsky, V.

A. Aides, Y. Y. Schechner, V. Holodovsky, M. J. Garay, and A. B. Davis, “Multi sky-view 3D aerosol distribution recovery,” Opt. Express 21, 25820–25833 (2013).
[Crossref]

V. Holodovsky, Y. Y. Schechner, A. Levin, A. Levis, and A. Aides, “In-situ multi-view multi-scattering stochastic tomography,” in IEEE International Conference on Computational Photography (ICCP) (2016), pp. 1–12.

A. Aides, A. Levis, V. Holodovsky, Y. Y. Schechner, D. Althausen, and A. Vainiger, “Distributed sky imaging radiometry and tomography,” in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2020), pp. 1–12.

Hong, Y. P.

D. Gürsoy, Y. P. Hong, K. He, K. Hujsak, S. Yoo, S. Chen, Y. Li, M. Ge, L. M. Miller, Y. S. Chu, and V. De Andrade, “Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection,” Sci. Rep. 7, 11818 (2017).
[Crossref]

Hopkins, T.

S. Samson, T. Hopkins, A. Remsen, L. Langebrake, T. Sutton, and J. Patten, “A system for high-resolution zooplankton imaging,” IEEE J. Ocean. Eng. 26, 671–676 (2001).
[Crossref]

Hover, F.

H. Singh, J. G. Bellingham, F. Hover, S. Lemer, B. A. Moran, K. Von der Heydt, and D. Yoerger, “Docking for an autonomous ocean sampling network,” IEEE J. Ocean. Eng. 26, 498–514 (2001).
[Crossref]

Hujsak, K.

D. Gürsoy, Y. P. Hong, K. He, K. Hujsak, S. Yoo, S. Chen, Y. Li, M. Ge, L. M. Miller, Y. S. Chu, and V. De Andrade, “Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection,” Sci. Rep. 7, 11818 (2017).
[Crossref]

Hullin, M. B.

J. Gregson, M. Krimerman, M. B. Hullin, and W. Heidrich, “Stochastic tomography and its applications in 3D imaging of mixing fluids,” ACM Trans. Graph. 31, 52 (2012).
[Crossref]

Iddaoud, G.

M. Picheral, L. Guidi, L. Stemmann, D. M. Karl, G. Iddaoud, and G. Gorsky, “The underwater vision profiler 5: an advanced instrument for high spatial resolution studies of particle size spectra and zooplankton,” Limnol. Oceanogr. Methods 8, 462–473 (2010).
[Crossref]

Ihrke, I.

I. Ihrke and M. Magnor, “Image-based tomographic reconstruction of flames,” in ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Eurographics Association, 2004), pp. 365–373.

Jaffe, J. S.

E. C. Orenstein, D. Ratelle, C. Briseño-Avena, M. L. Carter, P. J. Franks, J. S. Jaffe, and P. L. Roberts, “The Scripps Plankton Camera system: a framework and platform for in situ microscopy,” Limnol. Oceanogr. Methods 18, 681–695 (2020).
[Crossref]

J. S. Jaffe, “Underwater optical imaging: the past, the present, and the prospects,” IEEE J. Ocean. Eng. 40, 683–700 (2014).
[Crossref]

A. D. Mullen, T. Treibitz, P. L. Roberts, and J. S. Jaffe, “An underwater microscope for in situ imaging of seafloor organism,” in Novel Techniques in Microscopy (OSA, 2017), paper NTu1C–1.

Janssens, E.

Johnsen, S.

S. Johnsen, The Optics of Life: A Biologist’s Guide to Light in Nature (Princeton University, 2012).

Jordt-Sedlazeck, A.

A. Jordt-Sedlazeck and R. Koch, “Refractive structure-from-motion on underwater images,” in IEEE International Conference on Computer Vision (2013), pp. 57–64.

Karl, D. M.

M. Picheral, L. Guidi, L. Stemmann, D. M. Karl, G. Iddaoud, and G. Gorsky, “The underwater vision profiler 5: an advanced instrument for high spatial resolution studies of particle size spectra and zooplankton,” Limnol. Oceanogr. Methods 8, 462–473 (2010).
[Crossref]

Karsenti, E.

S. Colin, L. P. Coelho, S. Sunagawa, C. Bowler, E. Karsenti, P. Bork, R. Pepperkok, and C. De Vargas, “Quantitative 3D-imaging for cell biology and ecology of environmental microbial eukaryotes,” Elife 6, e26066 (2017).
[Crossref]

Katz, J.

Kelasidi, E.

E. Kelasidi, K. Y. Pettersen, J. T. Gravdahl, and P. Liljebäck, “Modeling of underwater snake robots,” in IEEE International Conference on Robotics and Automation (IEEE, 2014), pp. 4540–4547.

Kim, K.

Knott, G.

A. Veeraraghavan, A. V. Genkin, S. Vitaladevuni, L. Scheffer, S. Xu, H. Hess, R. Fetter, M. Cantoni, G. Knott, and D. Chklovskii, “Increasing depth resolution of electron microscopy of neural circuits using sparse tomographic reconstruction,” in IEEE Conference on Computer Vision and Pattern Recognition (2010), pp. 1767–1774.

Kocak, D. M.

D. M. Kocak, F. R. Dalgleish, F. M. Caimi, and Y. Y. Schechner, “A focus on recent developments and trends in underwater imaging,” Mar. Technol. Soc. J. 42, 52–67 (2008).
[Crossref]

Koch, R.

A. Jordt-Sedlazeck and R. Koch, “Refractive structure-from-motion on underwater images,” in IEEE International Conference on Computer Vision (2013), pp. 57–64.

Krimerman, M.

J. Gregson, M. Krimerman, M. B. Hullin, and W. Heidrich, “Stochastic tomography and its applications in 3D imaging of mixing fluids,” ACM Trans. Graph. 31, 52 (2012).
[Crossref]

Kudela, R.

C. Anderson, E. Berdalet, R. Kudela, C. Cusack, J. Silke, E. O’Rourke, D. Dugan, M. McCammon, J. Newton, S. Moore, and K. Paige, “Scaling up from regional case studies to a global harmful algal bloom observing system,” Front. Mar. Sci. 6, 250 (2019).
[Crossref]

J. Ryan, R. Kudela, J. Birch, M. Blum, H. Bowers, F. Chavez, G. Doucette, K. Hayashi, R. Marin, C. Mikulski, and J. Pennington, “Causality of an extreme harmful algal bloom in Monterey Bay, California, during the 2014–2016 northeast Pacific warm anomaly,” Geophys. Res. Lett. 44, 5571–5579 (2017).
[Crossref]

Lagerstedt, I.

C. L. Lawson, M. L. Baker, C. Best, C. Bi, M. Dougherty, P. Feng, G. Van Ginkel, B. Devkota, I. Lagerstedt, S. J. Ludtke, and R. Newman, “EMDataBank.org: unified data resource for CryoEM,” Nucleic Acids Res. 39, D456–D464 (2010).
[Crossref]

Langebrake, L.

S. Samson, T. Hopkins, A. Remsen, L. Langebrake, T. Sutton, and J. Patten, “A system for high-resolution zooplankton imaging,” IEEE J. Ocean. Eng. 26, 671–676 (2001).
[Crossref]

Lawson, C. L.

C. L. Lawson, M. L. Baker, C. Best, C. Bi, M. Dougherty, P. Feng, G. Van Ginkel, B. Devkota, I. Lagerstedt, S. J. Ludtke, and R. Newman, “EMDataBank.org: unified data resource for CryoEM,” Nucleic Acids Res. 39, D456–D464 (2010).
[Crossref]

Lederman, R. R.

R. R. Lederman and A. Singer, “A representation theory perspective on simultaneous alignment and classification,” Appl. Comput. Harmon. Anal. (2019).
[Crossref]

Lee, K.

Lee, S.

Lemer, S.

H. Singh, J. G. Bellingham, F. Hover, S. Lemer, B. A. Moran, K. Von der Heydt, and D. Yoerger, “Docking for an autonomous ocean sampling network,” IEEE J. Ocean. Eng. 26, 498–514 (2001).
[Crossref]

Leonardi, M.

M. Leonardi, A. Stahl, M. Gazzea, M. Ludvigsen, I. Rist-Christensen, and S. M. Nornes, “Vision based obstacle avoidance and motion tracking for autonomous behaviors in underwater vehicles,” in IEEE OCEANS (IEEE, 2017), pp. 1–10.

Levin, A.

V. Holodovsky, Y. Y. Schechner, A. Levin, A. Levis, and A. Aides, “In-situ multi-view multi-scattering stochastic tomography,” in IEEE International Conference on Computational Photography (ICCP) (2016), pp. 1–12.

Levis, A.

A. Levis, Y. Y. Schechner, A. B. Davis, and J. Loveridge, “Multi-view polarimetric scattering cloud tomography and retrieval of droplet size,” Remote Sens. 12, 2831 (2020).
[Crossref]

A. Aides, A. Levis, V. Holodovsky, Y. Y. Schechner, D. Althausen, and A. Vainiger, “Distributed sky imaging radiometry and tomography,” in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2020), pp. 1–12.

V. Holodovsky, Y. Y. Schechner, A. Levin, A. Levis, and A. Aides, “In-situ multi-view multi-scattering stochastic tomography,” in IEEE International Conference on Computational Photography (ICCP) (2016), pp. 1–12.

A. Levis, Y. Y. Schechner, and R. Talmon, “Statistical tomography of microscopic life,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018), pp. 6411–6420.

A. Levis, Y. Y. Schechner, and A. B. Davis, “Multiple-scattering microphysics tomography,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 6740–6749.

Li, S.

T. Nakatani, S. Li, T. Ura, A. Bodenmann, and T. Sakamaki, “3D visual modeling of hydrothermal chimneys using a rotary laser scanning system,” in IEEE Symposium on Underwater Technology and Workshop on Scientific Use of Submarine Cables and Related Technologies (IEEE, 2011), pp. 1–5.

Li, Y.

D. Gürsoy, Y. P. Hong, K. He, K. Hujsak, S. Yoo, S. Chen, Y. Li, M. Ge, L. M. Miller, Y. S. Chu, and V. De Andrade, “Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection,” Sci. Rep. 7, 11818 (2017).
[Crossref]

Liljebäck, P.

E. Kelasidi, K. Y. Pettersen, J. T. Gravdahl, and P. Liljebäck, “Modeling of underwater snake robots,” in IEEE International Conference on Robotics and Automation (IEEE, 2014), pp. 4540–4547.

Lombard, F.

F. Lombard, E. Boss, A. M. Waite, J. Uitz, L. Stemmann, H. M. Sosik, J. Schulz, J.-B. Romagnan, M. Picheral, and J. Pearlman, “Globally consistent quantitative observations of planktonic ecosystems,” Front. Mar. Sci. 6, 196 (2019).
[Crossref]

Loveridge, J.

A. Levis, Y. Y. Schechner, A. B. Davis, and J. Loveridge, “Multi-view polarimetric scattering cloud tomography and retrieval of droplet size,” Remote Sens. 12, 2831 (2020).
[Crossref]

Ludtke, S. J.

C. L. Lawson, M. L. Baker, C. Best, C. Bi, M. Dougherty, P. Feng, G. Van Ginkel, B. Devkota, I. Lagerstedt, S. J. Ludtke, and R. Newman, “EMDataBank.org: unified data resource for CryoEM,” Nucleic Acids Res. 39, D456–D464 (2010).
[Crossref]

Ludvigsen, M.

M. Leonardi, A. Stahl, M. Gazzea, M. Ludvigsen, I. Rist-Christensen, and S. M. Nornes, “Vision based obstacle avoidance and motion tracking for autonomous behaviors in underwater vehicles,” in IEEE OCEANS (IEEE, 2017), pp. 1–10.

Magnor, M.

I. Ihrke and M. Magnor, “Image-based tomographic reconstruction of flames,” in ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Eurographics Association, 2004), pp. 365–373.

Malkiel, E.

Marin, R.

J. Ryan, R. Kudela, J. Birch, M. Blum, H. Bowers, F. Chavez, G. Doucette, K. Hayashi, R. Marin, C. Mikulski, and J. Pennington, “Causality of an extreme harmful algal bloom in Monterey Bay, California, during the 2014–2016 northeast Pacific warm anomaly,” Geophys. Res. Lett. 44, 5571–5579 (2017).
[Crossref]

Marra, M.

C. S. Davis, S. M. Gallager, M. Marra, and W. K. Stewart, “Rapid visualization of plankton abundance and taxonomic composition using the video plankton recorder,” Deep Sea Res. II 43, 1947–1970 (1996).
[Crossref]

Mass, T.

A. Zweifler, D. Akkaynak, and T. Mass, and T. Treibitz, “In situ analysis of coral recruits using fluorescence imaging,” Front. Mar. Sci. 4, 273 (2017).
[Crossref]

McCammon, M.

C. Anderson, E. Berdalet, R. Kudela, C. Cusack, J. Silke, E. O’Rourke, D. Dugan, M. McCammon, J. Newton, S. Moore, and K. Paige, “Scaling up from regional case studies to a global harmful algal bloom observing system,” Front. Mar. Sci. 6, 250 (2019).
[Crossref]

Merico, A.

E. Acevedo-Trejos, G. Brandt, J. Bruggeman, and A. Merico, “Mechanisms shaping size structure and functional diversity of phytoplankton communities in the ocean,” Sci. Rep. 5, 8918 (2015).
[Crossref]

Mikulski, C.

J. Ryan, R. Kudela, J. Birch, M. Blum, H. Bowers, F. Chavez, G. Doucette, K. Hayashi, R. Marin, C. Mikulski, and J. Pennington, “Causality of an extreme harmful algal bloom in Monterey Bay, California, during the 2014–2016 northeast Pacific warm anomaly,” Geophys. Res. Lett. 44, 5571–5579 (2017).
[Crossref]

Miller, L. M.

D. Gürsoy, Y. P. Hong, K. He, K. Hujsak, S. Yoo, S. Chen, Y. Li, M. Ge, L. M. Miller, Y. S. Chu, and V. De Andrade, “Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection,” Sci. Rep. 7, 11818 (2017).
[Crossref]

Moore, S.

C. Anderson, E. Berdalet, R. Kudela, C. Cusack, J. Silke, E. O’Rourke, D. Dugan, M. McCammon, J. Newton, S. Moore, and K. Paige, “Scaling up from regional case studies to a global harmful algal bloom observing system,” Front. Mar. Sci. 6, 250 (2019).
[Crossref]

Moran, B. A.

H. Singh, J. G. Bellingham, F. Hover, S. Lemer, B. A. Moran, K. Von der Heydt, and D. Yoerger, “Docking for an autonomous ocean sampling network,” IEEE J. Ocean. Eng. 26, 498–514 (2001).
[Crossref]

Mubarok, A.

Mullen, A. D.

A. D. Mullen, T. Treibitz, P. L. Roberts, and J. S. Jaffe, “An underwater microscope for in situ imaging of seafloor organism,” in Novel Techniques in Microscopy (OSA, 2017), paper NTu1C–1.

Nakatani, T.

T. Nakatani, S. Li, T. Ura, A. Bodenmann, and T. Sakamaki, “3D visual modeling of hydrothermal chimneys using a rotary laser scanning system,” in IEEE Symposium on Underwater Technology and Workshop on Scientific Use of Submarine Cables and Related Technologies (IEEE, 2011), pp. 1–5.

Narasimhan, S. G.

M. Alterman, Y. Y. Schechner, M. Vo, and S. G. Narasimhan, “Passive tomography of turbulence strength,” in European Conference on Computer Vision (ECCV) (2014), pp. 47–60.

Negahdaripour, S.

Nepstad, R.

E. J. Davies and R. Nepstad, “In situ characterisation of complex suspended particulates surrounding an active submarine tailings placement site in a Norwegian fjord,” Regional Stud. Mar. Sci. 16, 198–207 (2017).
[Crossref]

Newman, R.

C. L. Lawson, M. L. Baker, C. Best, C. Bi, M. Dougherty, P. Feng, G. Van Ginkel, B. Devkota, I. Lagerstedt, S. J. Ludtke, and R. Newman, “EMDataBank.org: unified data resource for CryoEM,” Nucleic Acids Res. 39, D456–D464 (2010).
[Crossref]

Newton, J.

C. Anderson, E. Berdalet, R. Kudela, C. Cusack, J. Silke, E. O’Rourke, D. Dugan, M. McCammon, J. Newton, S. Moore, and K. Paige, “Scaling up from regional case studies to a global harmful algal bloom observing system,” Front. Mar. Sci. 6, 250 (2019).
[Crossref]

Nickels, C. F.

M. D. Ohman, R. E. Davis, J. T. Sherman, K. R. Grindley, B. M. Whitmore, C. F. Nickels, and J. S. Ellen, “Zooglider: an autonomous vehicle for optical and acoustic sensing of zooplankton,” Limnol. Oceanogr. Methods 17, 69–86 (2019).
[Crossref]

Nornes, S. M.

M. Leonardi, A. Stahl, M. Gazzea, M. Ludvigsen, I. Rist-Christensen, and S. M. Nornes, “Vision based obstacle avoidance and motion tracking for autonomous behaviors in underwater vehicles,” in IEEE OCEANS (IEEE, 2017), pp. 1–10.

O’Rourke, E.

C. Anderson, E. Berdalet, R. Kudela, C. Cusack, J. Silke, E. O’Rourke, D. Dugan, M. McCammon, J. Newton, S. Moore, and K. Paige, “Scaling up from regional case studies to a global harmful algal bloom observing system,” Front. Mar. Sci. 6, 250 (2019).
[Crossref]

Ohman, M. D.

M. D. Ohman, R. E. Davis, J. T. Sherman, K. R. Grindley, B. M. Whitmore, C. F. Nickels, and J. S. Ellen, “Zooglider: an autonomous vehicle for optical and acoustic sensing of zooplankton,” Limnol. Oceanogr. Methods 17, 69–86 (2019).
[Crossref]

Olson, R. J.

R. J. Olson and H. M. Sosik, “A submersible imaging-in-flow instrument to analyze nano-and microplankton: imaging FlowCytobot,” Limnol. Oceanogr. Methods 5, 195–203 (2007).
[Crossref]

Ophus, C.

D. Ren, C. Ophus, M. Chen, and L. Waller, “A multiple scattering algorithm for three dimensional phase contrast atomic electron tomography,” Ultramicroscopy 208, 112860 (2020).
[Crossref]

Orenstein, E. C.

E. C. Orenstein, D. Ratelle, C. Briseño-Avena, M. L. Carter, P. J. Franks, J. S. Jaffe, and P. L. Roberts, “The Scripps Plankton Camera system: a framework and platform for in situ microscopy,” Limnol. Oceanogr. Methods 18, 681–695 (2020).
[Crossref]

Paige, K.

C. Anderson, E. Berdalet, R. Kudela, C. Cusack, J. Silke, E. O’Rourke, D. Dugan, M. McCammon, J. Newton, S. Moore, and K. Paige, “Scaling up from regional case studies to a global harmful algal bloom observing system,” Front. Mar. Sci. 6, 250 (2019).
[Crossref]

Palenstijn, W. J.

Panduwirawan, A.

Park, H.

Park, Y.

Patten, J.

S. Samson, T. Hopkins, A. Remsen, L. Langebrake, T. Sutton, and J. Patten, “A system for high-resolution zooplankton imaging,” IEEE J. Ocean. Eng. 26, 671–676 (2001).
[Crossref]

Pearlman, J.

F. Lombard, E. Boss, A. M. Waite, J. Uitz, L. Stemmann, H. M. Sosik, J. Schulz, J.-B. Romagnan, M. Picheral, and J. Pearlman, “Globally consistent quantitative observations of planktonic ecosystems,” Front. Mar. Sci. 6, 196 (2019).
[Crossref]

Pennington, J.

J. Ryan, R. Kudela, J. Birch, M. Blum, H. Bowers, F. Chavez, G. Doucette, K. Hayashi, R. Marin, C. Mikulski, and J. Pennington, “Causality of an extreme harmful algal bloom in Monterey Bay, California, during the 2014–2016 northeast Pacific warm anomaly,” Geophys. Res. Lett. 44, 5571–5579 (2017).
[Crossref]

Pepperkok, R.

S. Colin, L. P. Coelho, S. Sunagawa, C. Bowler, E. Karsenti, P. Bork, R. Pepperkok, and C. De Vargas, “Quantitative 3D-imaging for cell biology and ecology of environmental microbial eukaryotes,” Elife 6, e26066 (2017).
[Crossref]

Pettersen, K. Y.

E. Kelasidi, K. Y. Pettersen, J. T. Gravdahl, and P. Liljebäck, “Modeling of underwater snake robots,” in IEEE International Conference on Robotics and Automation (IEEE, 2014), pp. 4540–4547.

Picheral, M.

F. Lombard, E. Boss, A. M. Waite, J. Uitz, L. Stemmann, H. M. Sosik, J. Schulz, J.-B. Romagnan, M. Picheral, and J. Pearlman, “Globally consistent quantitative observations of planktonic ecosystems,” Front. Mar. Sci. 6, 196 (2019).
[Crossref]

M. Picheral, L. Guidi, L. Stemmann, D. M. Karl, G. Iddaoud, and G. Gorsky, “The underwater vision profiler 5: an advanced instrument for high spatial resolution studies of particle size spectra and zooplankton,” Limnol. Oceanogr. Methods 8, 462–473 (2010).
[Crossref]

G. Gorsky, M. Picheral, and L. Stemmann, “Use of the underwater video profiler for the study of aggregate dynamics in the North Mediterranean,” Estuarine Coastal Shelf Sci. 50, 121–128 (2000).
[Crossref]

Player, M. A.

H. Sun, P. W. Benzie, N. Burns, D. C. Hendry, M. A. Player, and J. Watson, “Underwater digital holography for studies of marine plankton,” Philos. Trans. R. Soc. A 366, 1789–1806 (2008).
[Crossref]

Quigg, A.

Z. V. Finkel, J. Beardall, K. J. Flynn, A. Quigg, T. A. V. Rees, and J. A. Raven, “Phytoplankton in a changing world: cell size and elemental stoichiometry,” J. Plankton Res. 32, 119–137 (2009).
[Crossref]

Ratelle, D.

E. C. Orenstein, D. Ratelle, C. Briseño-Avena, M. L. Carter, P. J. Franks, J. S. Jaffe, and P. L. Roberts, “The Scripps Plankton Camera system: a framework and platform for in situ microscopy,” Limnol. Oceanogr. Methods 18, 681–695 (2020).
[Crossref]

Raven, J. A.

Z. V. Finkel, J. Beardall, K. J. Flynn, A. Quigg, T. A. V. Rees, and J. A. Raven, “Phytoplankton in a changing world: cell size and elemental stoichiometry,” J. Plankton Res. 32, 119–137 (2009).
[Crossref]

Rees, T. A. V.

Z. V. Finkel, J. Beardall, K. J. Flynn, A. Quigg, T. A. V. Rees, and J. A. Raven, “Phytoplankton in a changing world: cell size and elemental stoichiometry,” J. Plankton Res. 32, 119–137 (2009).
[Crossref]

Remsen, A.

S. Samson, T. Hopkins, A. Remsen, L. Langebrake, T. Sutton, and J. Patten, “A system for high-resolution zooplankton imaging,” IEEE J. Ocean. Eng. 26, 671–676 (2001).
[Crossref]

Ren, D.

D. Ren, C. Ophus, M. Chen, and L. Waller, “A multiple scattering algorithm for three dimensional phase contrast atomic electron tomography,” Ultramicroscopy 208, 112860 (2020).
[Crossref]

Richardson, A. J.

G. C. Hays, A. J. Richardson, and C. Robinson, “Climate change and marine plankton,” Trends Ecol. Evol. 20, 337–344 (2005).
[Crossref]

Rist-Christensen, I.

M. Leonardi, A. Stahl, M. Gazzea, M. Ludvigsen, I. Rist-Christensen, and S. M. Nornes, “Vision based obstacle avoidance and motion tracking for autonomous behaviors in underwater vehicles,” in IEEE OCEANS (IEEE, 2017), pp. 1–10.

Roberts, P. L.

E. C. Orenstein, D. Ratelle, C. Briseño-Avena, M. L. Carter, P. J. Franks, J. S. Jaffe, and P. L. Roberts, “The Scripps Plankton Camera system: a framework and platform for in situ microscopy,” Limnol. Oceanogr. Methods 18, 681–695 (2020).
[Crossref]

A. D. Mullen, T. Treibitz, P. L. Roberts, and J. S. Jaffe, “An underwater microscope for in situ imaging of seafloor organism,” in Novel Techniques in Microscopy (OSA, 2017), paper NTu1C–1.

Robinson, C.

G. C. Hays, A. J. Richardson, and C. Robinson, “Climate change and marine plankton,” Trends Ecol. Evol. 20, 337–344 (2005).
[Crossref]

Romagnan, J.-B.

F. Lombard, E. Boss, A. M. Waite, J. Uitz, L. Stemmann, H. M. Sosik, J. Schulz, J.-B. Romagnan, M. Picheral, and J. Pearlman, “Globally consistent quantitative observations of planktonic ecosystems,” Front. Mar. Sci. 6, 196 (2019).
[Crossref]

Ryan, J.

J. Ryan, R. Kudela, J. Birch, M. Blum, H. Bowers, F. Chavez, G. Doucette, K. Hayashi, R. Marin, C. Mikulski, and J. Pennington, “Causality of an extreme harmful algal bloom in Monterey Bay, California, during the 2014–2016 northeast Pacific warm anomaly,” Geophys. Res. Lett. 44, 5571–5579 (2017).
[Crossref]

Sakamaki, T.

T. Nakatani, S. Li, T. Ura, A. Bodenmann, and T. Sakamaki, “3D visual modeling of hydrothermal chimneys using a rotary laser scanning system,” in IEEE Symposium on Underwater Technology and Workshop on Scientific Use of Submarine Cables and Related Technologies (IEEE, 2011), pp. 1–5.

Samson, S.

S. Samson, T. Hopkins, A. Remsen, L. Langebrake, T. Sutton, and J. Patten, “A system for high-resolution zooplankton imaging,” IEEE J. Ocean. Eng. 26, 671–676 (2001).
[Crossref]

Schechner, Y. Y.

A. Levis, Y. Y. Schechner, A. B. Davis, and J. Loveridge, “Multi-view polarimetric scattering cloud tomography and retrieval of droplet size,” Remote Sens. 12, 2831 (2020).
[Crossref]

A. Vainiger, Y. Y. Schechner, T. Treibitz, A. Avni, and D. S. Timor, “Optical wide-field tomography of sediment resuspension,” Opt. Express 27, A766–A778 (2019).
[Crossref]

A. Aides, Y. Y. Schechner, V. Holodovsky, M. J. Garay, and A. B. Davis, “Multi sky-view 3D aerosol distribution recovery,” Opt. Express 21, 25820–25833 (2013).
[Crossref]

D. M. Kocak, F. R. Dalgleish, F. M. Caimi, and Y. Y. Schechner, “A focus on recent developments and trends in underwater imaging,” Mar. Technol. Soc. J. 42, 52–67 (2008).
[Crossref]

M. Sheinin and Y. Y. Schechner, “The next best underwater view,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 3764–3773.

A. Aides, A. Levis, V. Holodovsky, Y. Y. Schechner, D. Althausen, and A. Vainiger, “Distributed sky imaging radiometry and tomography,” in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2020), pp. 1–12.

A. Levis, Y. Y. Schechner, and R. Talmon, “Statistical tomography of microscopic life,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018), pp. 6411–6420.

V. Holodovsky, Y. Y. Schechner, A. Levin, A. Levis, and A. Aides, “In-situ multi-view multi-scattering stochastic tomography,” in IEEE International Conference on Computational Photography (ICCP) (2016), pp. 1–12.

M. Alterman, Y. Y. Schechner, M. Vo, and S. G. Narasimhan, “Passive tomography of turbulence strength,” in European Conference on Computer Vision (ECCV) (2014), pp. 47–60.

A. Levis, Y. Y. Schechner, and A. B. Davis, “Multiple-scattering microphysics tomography,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 6740–6749.

Scheffer, L.

A. Veeraraghavan, A. V. Genkin, S. Vitaladevuni, L. Scheffer, S. Xu, H. Hess, R. Fetter, M. Cantoni, G. Knott, and D. Chklovskii, “Increasing depth resolution of electron microscopy of neural circuits using sparse tomographic reconstruction,” in IEEE Conference on Computer Vision and Pattern Recognition (2010), pp. 1767–1774.

Schulz, J.

F. Lombard, E. Boss, A. M. Waite, J. Uitz, L. Stemmann, H. M. Sosik, J. Schulz, J.-B. Romagnan, M. Picheral, and J. Pearlman, “Globally consistent quantitative observations of planktonic ecosystems,” Front. Mar. Sci. 6, 196 (2019).
[Crossref]

Sedat, J. W.

D. A. Agard, Y. Hiraoka, P. Shaw, and J. W. Sedat, “Fluorescence microscopy in three dimensions,” Methods Cell Biol. 30, 353–377 (1989).
[Crossref]

Shaw, P.

D. A. Agard, Y. Hiraoka, P. Shaw, and J. W. Sedat, “Fluorescence microscopy in three dimensions,” Methods Cell Biol. 30, 353–377 (1989).
[Crossref]

Sheinin, M.

M. Sheinin and Y. Y. Schechner, “The next best underwater view,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 3764–3773.

Sheng, J.

Sherman, J. T.

M. D. Ohman, R. E. Davis, J. T. Sherman, K. R. Grindley, B. M. Whitmore, C. F. Nickels, and J. S. Ellen, “Zooglider: an autonomous vehicle for optical and acoustic sensing of zooplankton,” Limnol. Oceanogr. Methods 17, 69–86 (2019).
[Crossref]

Shkolnisky, Y.

I. Greenberg and Y. Shkolnisky, “Common lines modeling for reference free Ab-initio reconstruction in cryo-EM,” J. Struct. Biol. 200, 106–117 (2017).
[Crossref]

Y. Shkolnisky and A. Singer, “Viewing direction estimation in cryo-EM using synchronization,” SIAM J. Imaging Sci. 5, 1088–1110 (2012).
[Crossref]

Sijbers, J.

Silke, J.

C. Anderson, E. Berdalet, R. Kudela, C. Cusack, J. Silke, E. O’Rourke, D. Dugan, M. McCammon, J. Newton, S. Moore, and K. Paige, “Scaling up from regional case studies to a global harmful algal bloom observing system,” Front. Mar. Sci. 6, 250 (2019).
[Crossref]

Singer, A.

L. Wang, A. Singer, and Z. Wen, “Orientation determination of Cryo-EM images using least unsquared deviations,” SIAM J. Imaging Sci. 6, 2450–2483 (2013).
[Crossref]

Y. Shkolnisky and A. Singer, “Viewing direction estimation in cryo-EM using synchronization,” SIAM J. Imaging Sci. 5, 1088–1110 (2012).
[Crossref]

R. R. Lederman and A. Singer, “A representation theory perspective on simultaneous alignment and classification,” Appl. Comput. Harmon. Anal. (2019).
[Crossref]

Singh, H.

H. Singh, J. G. Bellingham, F. Hover, S. Lemer, B. A. Moran, K. Von der Heydt, and D. Yoerger, “Docking for an autonomous ocean sampling network,” IEEE J. Ocean. Eng. 26, 498–514 (2001).
[Crossref]

Sosik, H. M.

F. Lombard, E. Boss, A. M. Waite, J. Uitz, L. Stemmann, H. M. Sosik, J. Schulz, J.-B. Romagnan, M. Picheral, and J. Pearlman, “Globally consistent quantitative observations of planktonic ecosystems,” Front. Mar. Sci. 6, 196 (2019).
[Crossref]

R. J. Olson and H. M. Sosik, “A submersible imaging-in-flow instrument to analyze nano-and microplankton: imaging FlowCytobot,” Limnol. Oceanogr. Methods 5, 195–203 (2007).
[Crossref]

Stahl, A.

M. Leonardi, A. Stahl, M. Gazzea, M. Ludvigsen, I. Rist-Christensen, and S. M. Nornes, “Vision based obstacle avoidance and motion tracking for autonomous behaviors in underwater vehicles,” in IEEE OCEANS (IEEE, 2017), pp. 1–10.

Stemmann, L.

F. Lombard, E. Boss, A. M. Waite, J. Uitz, L. Stemmann, H. M. Sosik, J. Schulz, J.-B. Romagnan, M. Picheral, and J. Pearlman, “Globally consistent quantitative observations of planktonic ecosystems,” Front. Mar. Sci. 6, 196 (2019).
[Crossref]

M. Picheral, L. Guidi, L. Stemmann, D. M. Karl, G. Iddaoud, and G. Gorsky, “The underwater vision profiler 5: an advanced instrument for high spatial resolution studies of particle size spectra and zooplankton,” Limnol. Oceanogr. Methods 8, 462–473 (2010).
[Crossref]

G. Gorsky, M. Picheral, and L. Stemmann, “Use of the underwater video profiler for the study of aggregate dynamics in the North Mediterranean,” Estuarine Coastal Shelf Sci. 50, 121–128 (2000).
[Crossref]

Stewart, W. K.

C. S. Davis, S. M. Gallager, M. Marra, and W. K. Stewart, “Rapid visualization of plankton abundance and taxonomic composition using the video plankton recorder,” Deep Sea Res. II 43, 1947–1970 (1996).
[Crossref]

Sun, H.

H. Sun, P. W. Benzie, N. Burns, D. C. Hendry, M. A. Player, and J. Watson, “Underwater digital holography for studies of marine plankton,” Philos. Trans. R. Soc. A 366, 1789–1806 (2008).
[Crossref]

Sunagawa, S.

S. Colin, L. P. Coelho, S. Sunagawa, C. Bowler, E. Karsenti, P. Bork, R. Pepperkok, and C. De Vargas, “Quantitative 3D-imaging for cell biology and ecology of environmental microbial eukaryotes,” Elife 6, e26066 (2017).
[Crossref]

Sutton, T.

S. Samson, T. Hopkins, A. Remsen, L. Langebrake, T. Sutton, and J. Patten, “A system for high-resolution zooplankton imaging,” IEEE J. Ocean. Eng. 26, 671–676 (2001).
[Crossref]

Talmon, R.

A. Levis, Y. Y. Schechner, and R. Talmon, “Statistical tomography of microscopic life,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018), pp. 6411–6420.

Timor, D. S.

Treibitz, T.

A. Vainiger, Y. Y. Schechner, T. Treibitz, A. Avni, and D. S. Timor, “Optical wide-field tomography of sediment resuspension,” Opt. Express 27, A766–A778 (2019).
[Crossref]

A. Zweifler, D. Akkaynak, and T. Mass, and T. Treibitz, “In situ analysis of coral recruits using fluorescence imaging,” Front. Mar. Sci. 4, 273 (2017).
[Crossref]

D. Akkaynak, T. Treibitz, B. Xiao, U. A. Gürkan, J. J. Allen, U. Demirci, and R. T. Hanlon, “Use of commercial off-the-shelf digital cameras for scientific data acquisition and scene-specific color calibration,” J. Opt. Soc. Am. A 31, 312–321 (2014).
[Crossref]

A. D. Mullen, T. Treibitz, P. L. Roberts, and J. S. Jaffe, “An underwater microscope for in situ imaging of seafloor organism,” in Novel Techniques in Microscopy (OSA, 2017), paper NTu1C–1.

Trifonov, B. D.

B. D. Trifonov, “Tomographic reconstruction of transparent objects,” Ph.D. thesis (University of British Columbia, 2006).

Uitz, J.

F. Lombard, E. Boss, A. M. Waite, J. Uitz, L. Stemmann, H. M. Sosik, J. Schulz, J.-B. Romagnan, M. Picheral, and J. Pearlman, “Globally consistent quantitative observations of planktonic ecosystems,” Front. Mar. Sci. 6, 196 (2019).
[Crossref]

Ura, T.

T. Nakatani, S. Li, T. Ura, A. Bodenmann, and T. Sakamaki, “3D visual modeling of hydrothermal chimneys using a rotary laser scanning system,” in IEEE Symposium on Underwater Technology and Workshop on Scientific Use of Submarine Cables and Related Technologies (IEEE, 2011), pp. 1–5.

Vainiger, A.

A. Vainiger, Y. Y. Schechner, T. Treibitz, A. Avni, and D. S. Timor, “Optical wide-field tomography of sediment resuspension,” Opt. Express 27, A766–A778 (2019).
[Crossref]

A. Aides, A. Levis, V. Holodovsky, Y. Y. Schechner, D. Althausen, and A. Vainiger, “Distributed sky imaging radiometry and tomography,” in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2020), pp. 1–12.

van Aarle, W.

Van Ginkel, G.

C. L. Lawson, M. L. Baker, C. Best, C. Bi, M. Dougherty, P. Feng, G. Van Ginkel, B. Devkota, I. Lagerstedt, S. J. Ludtke, and R. Newman, “EMDataBank.org: unified data resource for CryoEM,” Nucleic Acids Res. 39, D456–D464 (2010).
[Crossref]

Veeraraghavan, A.

A. Veeraraghavan, A. V. Genkin, S. Vitaladevuni, L. Scheffer, S. Xu, H. Hess, R. Fetter, M. Cantoni, G. Knott, and D. Chklovskii, “Increasing depth resolution of electron microscopy of neural circuits using sparse tomographic reconstruction,” in IEEE Conference on Computer Vision and Pattern Recognition (2010), pp. 1767–1774.

Vitaladevuni, S.

A. Veeraraghavan, A. V. Genkin, S. Vitaladevuni, L. Scheffer, S. Xu, H. Hess, R. Fetter, M. Cantoni, G. Knott, and D. Chklovskii, “Increasing depth resolution of electron microscopy of neural circuits using sparse tomographic reconstruction,” in IEEE Conference on Computer Vision and Pattern Recognition (2010), pp. 1767–1774.

Vo, M.

M. Alterman, Y. Y. Schechner, M. Vo, and S. G. Narasimhan, “Passive tomography of turbulence strength,” in European Conference on Computer Vision (ECCV) (2014), pp. 47–60.

Vogel, S.

S. Vogel, Life in Moving Fluids: The Physical Biology of Flow-Revised and Expanded Second Edition (Princeton University, 2020).

Von der Heydt, K.

H. Singh, J. G. Bellingham, F. Hover, S. Lemer, B. A. Moran, K. Von der Heydt, and D. Yoerger, “Docking for an autonomous ocean sampling network,” IEEE J. Ocean. Eng. 26, 498–514 (2001).
[Crossref]

Waite, A. M.

F. Lombard, E. Boss, A. M. Waite, J. Uitz, L. Stemmann, H. M. Sosik, J. Schulz, J.-B. Romagnan, M. Picheral, and J. Pearlman, “Globally consistent quantitative observations of planktonic ecosystems,” Front. Mar. Sci. 6, 196 (2019).
[Crossref]

Waller, L.

D. Ren, C. Ophus, M. Chen, and L. Waller, “A multiple scattering algorithm for three dimensional phase contrast atomic electron tomography,” Ultramicroscopy 208, 112860 (2020).
[Crossref]

Wang, L.

L. Wang, A. Singer, and Z. Wen, “Orientation determination of Cryo-EM images using least unsquared deviations,” SIAM J. Imaging Sci. 6, 2450–2483 (2013).
[Crossref]

Wang, Y.

Watson, J.

H. Sun, P. W. Benzie, N. Burns, D. C. Hendry, M. A. Player, and J. Watson, “Underwater digital holography for studies of marine plankton,” Philos. Trans. R. Soc. A 366, 1789–1806 (2008).
[Crossref]

Wen, Z.

L. Wang, A. Singer, and Z. Wen, “Orientation determination of Cryo-EM images using least unsquared deviations,” SIAM J. Imaging Sci. 6, 2450–2483 (2013).
[Crossref]

Whitmore, B. M.

M. D. Ohman, R. E. Davis, J. T. Sherman, K. R. Grindley, B. M. Whitmore, C. F. Nickels, and J. S. Ellen, “Zooglider: an autonomous vehicle for optical and acoustic sensing of zooplankton,” Limnol. Oceanogr. Methods 17, 69–86 (2019).
[Crossref]

Xiao, B.

Xu, S.

A. Veeraraghavan, A. V. Genkin, S. Vitaladevuni, L. Scheffer, S. Xu, H. Hess, R. Fetter, M. Cantoni, G. Knott, and D. Chklovskii, “Increasing depth resolution of electron microscopy of neural circuits using sparse tomographic reconstruction,” in IEEE Conference on Computer Vision and Pattern Recognition (2010), pp. 1767–1774.

Yoerger, D.

H. Singh, J. G. Bellingham, F. Hover, S. Lemer, B. A. Moran, K. Von der Heydt, and D. Yoerger, “Docking for an autonomous ocean sampling network,” IEEE J. Ocean. Eng. 26, 498–514 (2001).
[Crossref]

Yoo, S.

D. Gürsoy, Y. P. Hong, K. He, K. Hujsak, S. Yoo, S. Chen, Y. Li, M. Ge, L. M. Miller, Y. S. Chu, and V. De Andrade, “Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection,” Sci. Rep. 7, 11818 (2017).
[Crossref]

Zweifler, A.

A. Zweifler, D. Akkaynak, and T. Mass, and T. Treibitz, “In situ analysis of coral recruits using fluorescence imaging,” Front. Mar. Sci. 4, 273 (2017).
[Crossref]

ACM Trans. Graph. (1)

J. Gregson, M. Krimerman, M. B. Hullin, and W. Heidrich, “Stochastic tomography and its applications in 3D imaging of mixing fluids,” ACM Trans. Graph. 31, 52 (2012).
[Crossref]

Ann. Rev. Biophys. Bioeng. (1)

D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Ann. Rev. Biophys. Bioeng. 13, 191–219 (1984).
[Crossref]

Appl. Opt. (2)

Deep Sea Res. II (1)

C. S. Davis, S. M. Gallager, M. Marra, and W. K. Stewart, “Rapid visualization of plankton abundance and taxonomic composition using the video plankton recorder,” Deep Sea Res. II 43, 1947–1970 (1996).
[Crossref]

Elife (1)

S. Colin, L. P. Coelho, S. Sunagawa, C. Bowler, E. Karsenti, P. Bork, R. Pepperkok, and C. De Vargas, “Quantitative 3D-imaging for cell biology and ecology of environmental microbial eukaryotes,” Elife 6, e26066 (2017).
[Crossref]

Estuarine Coastal Shelf Sci. (1)

G. Gorsky, M. Picheral, and L. Stemmann, “Use of the underwater video profiler for the study of aggregate dynamics in the North Mediterranean,” Estuarine Coastal Shelf Sci. 50, 121–128 (2000).
[Crossref]

Front. Mar. Sci. (3)

A. Zweifler, D. Akkaynak, and T. Mass, and T. Treibitz, “In situ analysis of coral recruits using fluorescence imaging,” Front. Mar. Sci. 4, 273 (2017).
[Crossref]

F. Lombard, E. Boss, A. M. Waite, J. Uitz, L. Stemmann, H. M. Sosik, J. Schulz, J.-B. Romagnan, M. Picheral, and J. Pearlman, “Globally consistent quantitative observations of planktonic ecosystems,” Front. Mar. Sci. 6, 196 (2019).
[Crossref]

C. Anderson, E. Berdalet, R. Kudela, C. Cusack, J. Silke, E. O’Rourke, D. Dugan, M. McCammon, J. Newton, S. Moore, and K. Paige, “Scaling up from regional case studies to a global harmful algal bloom observing system,” Front. Mar. Sci. 6, 250 (2019).
[Crossref]

Geophys. Res. Lett. (1)

J. Ryan, R. Kudela, J. Birch, M. Blum, H. Bowers, F. Chavez, G. Doucette, K. Hayashi, R. Marin, C. Mikulski, and J. Pennington, “Causality of an extreme harmful algal bloom in Monterey Bay, California, during the 2014–2016 northeast Pacific warm anomaly,” Geophys. Res. Lett. 44, 5571–5579 (2017).
[Crossref]

IEEE J. Ocean. Eng. (3)

J. S. Jaffe, “Underwater optical imaging: the past, the present, and the prospects,” IEEE J. Ocean. Eng. 40, 683–700 (2014).
[Crossref]

S. Samson, T. Hopkins, A. Remsen, L. Langebrake, T. Sutton, and J. Patten, “A system for high-resolution zooplankton imaging,” IEEE J. Ocean. Eng. 26, 671–676 (2001).
[Crossref]

H. Singh, J. G. Bellingham, F. Hover, S. Lemer, B. A. Moran, K. Von der Heydt, and D. Yoerger, “Docking for an autonomous ocean sampling network,” IEEE J. Ocean. Eng. 26, 498–514 (2001).
[Crossref]

J. Opt. Soc. Am. A (1)

J. Opt. Soc. Korea (1)

J. Plankton Res. (1)

Z. V. Finkel, J. Beardall, K. J. Flynn, A. Quigg, T. A. V. Rees, and J. A. Raven, “Phytoplankton in a changing world: cell size and elemental stoichiometry,” J. Plankton Res. 32, 119–137 (2009).
[Crossref]

J. Struct. Biol. (1)

I. Greenberg and Y. Shkolnisky, “Common lines modeling for reference free Ab-initio reconstruction in cryo-EM,” J. Struct. Biol. 200, 106–117 (2017).
[Crossref]

J. Theor. Biol. (1)

R. Gordon, R. Bender, and G. T. Herman, “Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography,” J. Theor. Biol. 29, 471–481 (1970).
[Crossref]

Limnol. Oceanogr. Methods (5)

E. C. Orenstein, D. Ratelle, C. Briseño-Avena, M. L. Carter, P. J. Franks, J. S. Jaffe, and P. L. Roberts, “The Scripps Plankton Camera system: a framework and platform for in situ microscopy,” Limnol. Oceanogr. Methods 18, 681–695 (2020).
[Crossref]

M. Picheral, L. Guidi, L. Stemmann, D. M. Karl, G. Iddaoud, and G. Gorsky, “The underwater vision profiler 5: an advanced instrument for high spatial resolution studies of particle size spectra and zooplankton,” Limnol. Oceanogr. Methods 8, 462–473 (2010).
[Crossref]

R. J. Olson and H. M. Sosik, “A submersible imaging-in-flow instrument to analyze nano-and microplankton: imaging FlowCytobot,” Limnol. Oceanogr. Methods 5, 195–203 (2007).
[Crossref]

M. D. Ohman, R. E. Davis, J. T. Sherman, K. R. Grindley, B. M. Whitmore, C. F. Nickels, and J. S. Ellen, “Zooglider: an autonomous vehicle for optical and acoustic sensing of zooplankton,” Limnol. Oceanogr. Methods 17, 69–86 (2019).
[Crossref]

R. K. Cowen and C. M. Guigand, “In situ ichthyoplankton imaging system (I SIIS): system design and preliminary results,” Limnol. Oceanogr. Methods 6, 126–132 (2008).
[Crossref]

Mar. Technol. Soc. J. (1)

D. M. Kocak, F. R. Dalgleish, F. M. Caimi, and Y. Y. Schechner, “A focus on recent developments and trends in underwater imaging,” Mar. Technol. Soc. J. 42, 52–67 (2008).
[Crossref]

Methods Cell Biol. (1)

D. A. Agard, Y. Hiraoka, P. Shaw, and J. W. Sedat, “Fluorescence microscopy in three dimensions,” Methods Cell Biol. 30, 353–377 (1989).
[Crossref]

Nucleic Acids Res. (1)

C. L. Lawson, M. L. Baker, C. Best, C. Bi, M. Dougherty, P. Feng, G. Van Ginkel, B. Devkota, I. Lagerstedt, S. J. Ludtke, and R. Newman, “EMDataBank.org: unified data resource for CryoEM,” Nucleic Acids Res. 39, D456–D464 (2010).
[Crossref]

Opt. Express (3)

Philos. Trans. R. Soc. A (1)

H. Sun, P. W. Benzie, N. Burns, D. C. Hendry, M. A. Player, and J. Watson, “Underwater digital holography for studies of marine plankton,” Philos. Trans. R. Soc. A 366, 1789–1806 (2008).
[Crossref]

Q. Rev. Biophys. (1)

J. Frank, “Single-particle reconstruction of biological macromolecules in electron microscopy–30 years,” Q. Rev. Biophys. 42, 139–158 (2009).
[Crossref]

Regional Stud. Mar. Sci. (1)

E. J. Davies and R. Nepstad, “In situ characterisation of complex suspended particulates surrounding an active submarine tailings placement site in a Norwegian fjord,” Regional Stud. Mar. Sci. 16, 198–207 (2017).
[Crossref]

Remote Sens. (1)

A. Levis, Y. Y. Schechner, A. B. Davis, and J. Loveridge, “Multi-view polarimetric scattering cloud tomography and retrieval of droplet size,” Remote Sens. 12, 2831 (2020).
[Crossref]

Sci. Rep. (2)

D. Gürsoy, Y. P. Hong, K. He, K. Hujsak, S. Yoo, S. Chen, Y. Li, M. Ge, L. M. Miller, Y. S. Chu, and V. De Andrade, “Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection,” Sci. Rep. 7, 11818 (2017).
[Crossref]

E. Acevedo-Trejos, G. Brandt, J. Bruggeman, and A. Merico, “Mechanisms shaping size structure and functional diversity of phytoplankton communities in the ocean,” Sci. Rep. 5, 8918 (2015).
[Crossref]

SIAM J. Imaging Sci. (2)

L. Wang, A. Singer, and Z. Wen, “Orientation determination of Cryo-EM images using least unsquared deviations,” SIAM J. Imaging Sci. 6, 2450–2483 (2013).
[Crossref]

Y. Shkolnisky and A. Singer, “Viewing direction estimation in cryo-EM using synchronization,” SIAM J. Imaging Sci. 5, 1088–1110 (2012).
[Crossref]

Trends Ecol. Evol. (1)

G. C. Hays, A. J. Richardson, and C. Robinson, “Climate change and marine plankton,” Trends Ecol. Evol. 20, 337–344 (2005).
[Crossref]

Ultramicroscopy (1)

D. Ren, C. Ophus, M. Chen, and L. Waller, “A multiple scattering algorithm for three dimensional phase contrast atomic electron tomography,” Ultramicroscopy 208, 112860 (2020).
[Crossref]

Other (20)

B. D. Trifonov, “Tomographic reconstruction of transparent objects,” Ph.D. thesis (University of British Columbia, 2006).

A. Veeraraghavan, A. V. Genkin, S. Vitaladevuni, L. Scheffer, S. Xu, H. Hess, R. Fetter, M. Cantoni, G. Knott, and D. Chklovskii, “Increasing depth resolution of electron microscopy of neural circuits using sparse tomographic reconstruction,” in IEEE Conference on Computer Vision and Pattern Recognition (2010), pp. 1767–1774.

I. Ihrke and M. Magnor, “Image-based tomographic reconstruction of flames,” in ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Eurographics Association, 2004), pp. 365–373.

A. Levis, Y. Y. Schechner, and A. B. Davis, “Multiple-scattering microphysics tomography,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 6740–6749.

M. Alterman, Y. Y. Schechner, M. Vo, and S. G. Narasimhan, “Passive tomography of turbulence strength,” in European Conference on Computer Vision (ECCV) (2014), pp. 47–60.

V. Holodovsky, Y. Y. Schechner, A. Levin, A. Levis, and A. Aides, “In-situ multi-view multi-scattering stochastic tomography,” in IEEE International Conference on Computational Photography (ICCP) (2016), pp. 1–12.

A. Levis, Y. Y. Schechner, and R. Talmon, “Statistical tomography of microscopic life,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018), pp. 6411–6420.

R. R. Lederman and A. Singer, “A representation theory perspective on simultaneous alignment and classification,” Appl. Comput. Harmon. Anal. (2019).
[Crossref]

H. Sosik, E. E. Peacock, and E. Brownlee, “Annotated plankton images data set for developing and evaluating classification methods,” WHOAS2015, http://darchive.mblwhoilibrary.org/handle/1912/7341 .

“Scripps plankton camera website,” http://spc.ucsd.edu .

S. Vogel, Life in Moving Fluids: The Physical Biology of Flow-Revised and Expanded Second Edition (Princeton University, 2020).

S. Johnsen, The Optics of Life: A Biologist’s Guide to Light in Nature (Princeton University, 2012).

A. D. Mullen, T. Treibitz, P. L. Roberts, and J. S. Jaffe, “An underwater microscope for in situ imaging of seafloor organism,” in Novel Techniques in Microscopy (OSA, 2017), paper NTu1C–1.

A. Jordt-Sedlazeck and R. Koch, “Refractive structure-from-motion on underwater images,” in IEEE International Conference on Computer Vision (2013), pp. 57–64.

C. Ancuti, C. O. Ancuti, C. De Vleeschouwer, R. Garcia, and A. C. Bovik, “Multi-scale underwater descattering,” in IEEE International Conference on Pattern Recognition (IEEE, 2016), pp. 4202–4207.

M. Sheinin and Y. Y. Schechner, “The next best underwater view,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 3764–3773.

A. Aides, A. Levis, V. Holodovsky, Y. Y. Schechner, D. Althausen, and A. Vainiger, “Distributed sky imaging radiometry and tomography,” in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2020), pp. 1–12.

T. Nakatani, S. Li, T. Ura, A. Bodenmann, and T. Sakamaki, “3D visual modeling of hydrothermal chimneys using a rotary laser scanning system,” in IEEE Symposium on Underwater Technology and Workshop on Scientific Use of Submarine Cables and Related Technologies (IEEE, 2011), pp. 1–5.

M. Leonardi, A. Stahl, M. Gazzea, M. Ludvigsen, I. Rist-Christensen, and S. M. Nornes, “Vision based obstacle avoidance and motion tracking for autonomous behaviors in underwater vehicles,” in IEEE OCEANS (IEEE, 2017), pp. 1–10.

E. Kelasidi, K. Y. Pettersen, J. T. Gravdahl, and P. Liljebäck, “Modeling of underwater snake robots,” in IEEE International Conference on Robotics and Automation (IEEE, 2014), pp. 4540–4547.

Data Availability

Data underlying the results presented in this paper are available in Refs. [4,60].

4. “Scripps plankton camera website,” http://spc.ucsd.edu.

60. H. Sosik, E. E. Peacock, and E. Brownlee, “Annotated plankton images data set for developing and evaluating classification methods,” WHOAS2015, http://darchive.mblwhoilibrary.org/handle/1912/7341.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (15)

Fig. 1.
Fig. 1. Specimens viewed in nature have random sizes and poses. Light passes through, yielding a single 2D image per specimen. The goal is to tomographically recover the 3D volumetric distribution of the plankton (on average) and estimate the population size distribution.
Fig. 2.
Fig. 2. (a) Illustration of the Scripps Plankton Camera (SPC). (b) An image of the SPC in situ. The dark-field microscope’s sample volume is between the ports.
Fig. 3.
Fig. 3. 3D Fourier domain and two central slices ${{{\tilde {\textbf y}}}_n},{{{\tilde {\textbf y}}}_m}$ . Each central slice is perpendicular to the direction-vector of the respective projection in the microscope. The two slices intersect at a common tilt axis (common line) in the 3D Fourier domain.
Fig. 4.
Fig. 4. Central slice in 3D Fourier domain (seen in Fig. 3) is equivalent to a 2D Fourier transform of an acquired image. A common line (also seen in Fig. 3) is at respective corresponding angles ${\psi _{n \to m}},{\psi _{m \to n}}$ in the 2D Fourier domains. Noisy data (e.g., due to non-rigid deformation of specimens) leads to erroneous estimation of these angles, ${\hat \psi _{n \to m}},{\hat \psi _{m \to n}}$ .
Fig. 5.
Fig. 5. Histogram of the angular error $\Delta {\psi _{nm}}$ (in degrees) with ${V_{{\max}}} = 9\%$ , when data is affected by simulated random non-rigid deformations. The red curve is a fit of the histogram to Eq. (18). The fit has a coefficient of correlation 0.98.
Fig. 6.
Fig. 6. Influence of the deformation coefficient ${V_{{\max}}}$ on the parameters of the statistical model [Eq. (18)]. The stronger the deformations, the lower is the probability that correlation-based angle pairs are indicative. Pairs that are indicative have a larger uncertainty as ${V_{{\max}}}$ increases.
Fig. 7.
Fig. 7. Top left, the pre-computed PDF of a triplet score $s$ , if the triplet of estimated relative rotations is arbitrary. Top right, the pre-computed PDF of a triplet score $s$ , if the triplet of estimated relative rotations is indicative. Here estimated relative rotations have errors whose standard deviation is $\sigma = {6^\circ}$ . Bottom, empirical histogram ${\cal H}(s)$ of triplet scores. It is modeled as a convex sum of the above PDFs, controlled by $P$ . Fitting this parametric model to ${\cal H}(s)$ yields an estimate of ${{\hat P}},\hat \sigma$ .
Fig. 8.
Fig. 8. (a) 3D volumetric object is randomly scaled, deformed shifted, and projected onto 100 random orientations. (b) Here we show several image samples from this simulation.
Fig. 9.
Fig. 9. Recovery using data described in Fig. 8. Top, visualization of the 3D object density estimated by (a) our method versus (b) that of Ref. [42]. (c) and (d) The respective scatter plots of $\hat \beta$ versus true $\beta$ (values drawn from 1% random voxels). The red line represents a perfect fit. Our approach clearly yields a better fit to ground-truth than [42]. (e) The statistical model [Eq. (18)] fitted to the empirical histogram of angular errors, similarly to Fig. 6. In the process described in Section 3.D, estimated angle pairs are better constrained. After a single iteration, our error model (red plot) has a narrower standard deviation and a higher indicative probability, compared to errors (blue plot) obtained in the method of Ref. [42].
Fig. 10.
Fig. 10. Experiment on a population of Daphnia sp., imaged using our system. Top, a few raw dark-field images. Middle, corresponding cross-validation re-projections, as described in Section 3.A, using our tomographically estimated object. Bottom, corresponding cross-validation re-projections, using the estimation resulting from the method of Ref. [42].
Fig. 11.
Fig. 11. Experiment on a population of Oithona sp., imaged using our system. Top, a few raw dark-field images. Middle, corresponding cross-validation re-projections of our tomographically estimated object. Bottom, corresponding cross-validation re-projections of the object resulting from the method of Ref. [42].
Fig. 12.
Fig. 12. Volumetric density recovery: Oithona sp.
Fig. 13.
Fig. 13. Resulting size distribution histograms (blue) and fitted [Eq. (31)] density (red). (a) Oithona sp. The fit 95% confidence intervals are $\xi \in [6.48\;6.5]$ , $\nu \in [0.138\;0.151]$ . (b) Daphnia sp. The fit 95% confidence intervals are $\xi \in [1.12\;1.21]$ , $\nu \in [0.244\;0.308]$ .
Fig. 14.
Fig. 14. Volumetric density recovery: Pyramimonas longicauda.
Fig. 15.
Fig. 15. Experiment on a population of Pyramimonas longicauda, from dataset [4]. Top, a logarithm operation on raw bright-field images yields a linear model as in Eq. (1). A few samples are shown. Middle, corresponding cross-validation re-projections of our tomographically estimated object. Bottom, corresponding cross-validation re-projections of the object resulting from the method of Ref. [42].

Tables (1)

Tables Icon

Table 1. Results Corresponding to Data Described in Fig. 8 a

Equations (33)

Equations on this page are rendered with MathJax. Learn more.

y n = P x 1 , x 2 ( n ) { β } = β ( R n 1 x ) d x 3 ,
R n 1 [ k 1 , k 2 , 0 ] T ,
[ cos ψ n m sin ψ n m 0 ] = R n 1 R m [ cos ψ m n sin ψ m n 0 ] .
Σ n m ( ψ , ψ , M , τ ) = ρ max ρ max y ~ n ( ρ M , ψ ) y ~ m ( ρ , ψ ) e 2 π i ρ τ d ρ .
{ ψ ^ n m , ψ ^ m n , M ^ n m , τ ^ n m } = a r g m a x ψ , ψ , M , τ Σ n m ( ψ , ψ , M , τ ) Σ n n ( ψ , ψ , M , τ ) Σ m m ( ψ , ψ , M , τ ) .
D s c a l e ( { M ¯ n } n = 1 N ) = n m | M ¯ n M ¯ m log M ^ n m | 2 .
{ M ¯ ^ n } n = 1 N = arg min M ¯ n [ D s c a l e ( { M ¯ n } ) + R s c a l e ( { M ¯ n } ) ] .
{ R ^ n } n = 1 N = argmin R 1 , , R N n m R n [ cos ψ n m sin ψ n m 0 ] R m [ cos ψ m n sin ψ m n 0 ] 2 2 s . t . R n R n T = I , det ( R n ) = 1 , n = 1 , , N .
R m n R n R m 1 .
S = [ I R 1 2 R 1 n R 1 N R 2 1 I R 2 n R 2 N R m 1 R m 2 R m n R m N R N 1 R N 2 R N n I ]
H = [ R 1 R N ] .
( SH ) n = m = 1 N R m n R m = N R n = N ( H ) n ,
SH = N H .
[ R ^ 1 R ^ N ] = [ h 1 , h 2 , h 3 ] .
m w n , m = N , w n , m 0 , n , m .
( S ~ H ) n = m = 1 N w n , m R m n R m = R n m = 1 N w n , m = N ( H ) n .
Δ ψ n m = ψ n m ψ ^ n m .
f ( Δ ψ n m ) = P σ 2 π exp [ ( Δ ψ n m ) 2 2 σ 2 ] + ( 1 P ) 180 .
v c ( n ) = v ^ c ( n ) V ( n ) sin [ 2 π x c / ( G 1 ) + ϕ c ( n ) ] .
R n m R m l R l n I = 0.
s n , m , l s ( R ^ n m R ^ m l R ^ l n ) ,
H ( s ) P 3 f t r i p l e t i n d i c a t i v e ( s | σ ) + ( 1 P 3 ) f t r i p l e t a r b i t r a r y ( s ) .
S n , m { s n , m , l l { n , m } } .
p n , m = P ^ n , m i n d i c a t i v e ( σ ^ ) P ^ n , m i n d i c a t i v e ( σ ^ ) + ( 1 P ^ ) n , m a r b i t r a r y .
w n , m = γ p n , m ,
{ M ^ n m , τ ^ n m } = arg max M , τ Σ n m ( M , τ ) Σ n n ( M , τ ) Σ m m ( M , τ ) ,
ϵ r o t = 1 N min O n = 1 N R n O R ^ n 2 2 ,
ϵ s c a l e = 1 N min M r e l n = 1 N | M n M r e l M ^ n | 2 .
ϵ d e n s = | β ^ ( x ) β ( x ) | d x β ( x ) d x .
ϵ r e - p r o j = 1 N N p i x e l n ~ = 1 N y n ~ y ^ n ~ 2 1 max ( y n ~ ) .
f s i z e ( ϱ ) = 1 ϱ 2 π ν 2 exp { [ ln ( ϱ ) ξ ] 2 2 ν 2 } .
n , m a r b i t r a r y a r b i t r a r y ( S n , m ) = l { n , m } f t r i p l e t a r b i t r a r y ( s n , m , l ) .
n , m i n d i c a t i v e ( σ ^ ) i n d i c a t i v e ( S n , m | σ ^ ) = l { n , m } [ P ^ 2 f t r i p l e t i n d i c a t i v e ( s n , m , l | σ ^ ) + ( 1 P ^ 2 ) f t r i p l e t a r b i t r a r y ( s n , m , l ) ] .

Metrics