1. Introduction

Studies on dynamic deformation and movement of amoeboid cells are required in relation to sustainable maintenance of environmental microbial ecosystem since diverse protists amoeba live everywhere in soil and waters and play a fundamental role [1–5].

Progress of those studies is supported by quantitative microscopic 3D measurement for single-celled amoeboid movement by means of some modern fluorescence microscopy: for instance, light-sheet microscopy [6,7], light-field microscopy [8], spinning disk microscopy [9], laser scanning microscopy [10], two-photon microscopy [11,12], and multi-photon microscopy [13]. These techniques need expensive equipments that are not easy to purchase.

Inexpensive non-fluorescence stereoscopic binocular microscopy is a possible option since the binocular microscope is one of the ordinary equipments in life science laboratories [14]. So far, quantitative binocular stereo-microscope has been proved to be very useful mainly in material sciences [15–18]. But the application to cells and tissues is limited due to the difficulty of high transparency of the cellular specimens. So it is needed to visualize the specimen: stained by fluorescent dye as a marker, or sprayed by fluorescent or non-fluorescent tiny particles on the surface of the specimens, etc. In one of the pioneering works, a visualization technique was applied to 3D observation for deformation of biofilm [19,20].

In this report, we develop a useful method in binocular microscopy for stereoscopic observation of protists amoeboid movement without any prior procedures for staining, while tackling the difficulty: (1) extraction of information from the light that is scattered by intracellular vesicles and organelles, (2) stereo-matching of left and right views, and (3) reconstruction of the cell surface from the limited information.

The reconstruction in this context means extracting 3D positional information of objects from two-dimensional (2D) images taken from different angles. Basically, the reconstruction is the triangulation using information about camera angles and the positions of objects in images taken by these cameras. But to do this, we have to identify objects in images. This process is called stereo matching [21]. This is the most difficult part of the reconstruction process.

The stereo matching is usually achieved by utilizing all the available information such as colors, shapes and even surface patterns of objects [22]. But those information is not available for amoeboid cells. These cells have transparent white colors, undefined shapes and invisible surface patterns. We have to find other ways to do stereo matching.

What we use here is the 2D pattern of graded brightness of amoeboid cell on the dark-field image that is formed by the scattered light from many intrinsic intracellular vesicles and other organelles in the amoeboid cell [23]. On the right and left images, these vesicles and the others appear, in principle, to be bright peaks that are to be considered for stereo-matching. This is our novel idea examined in this report.

We use one notable characteristic of the images taken by microscopes: the projection is almost orthogonal. The orthogonal projection is mathematically much simpler projection compared to the perspective projection as shown in Fig. 1(a). The orthogonal projection is a good approximation to the natural perspective projection when all objects in sight are at almost the same distance from the camera. In other words, this condition is expressed that the distance between the camera and objects is much longer than the width of field of view. This is obviously satisfied in microscopes.

 

Fig. 1. Coordinate system for binocular stereo-microscope and correction for lens distortion. (a) In perspective projection shown in the left figure, an object twice as close appears twice as big. When objects in sight are at the same distance from the camera, orthogonal projection in the right figure is a good approximation to perspective projection. (b) Schematic diagram of stereo microscope. We assume that objects are orthogonally projected on two planes of the sensors of cameras by the projection vectors $p_L$ and $p_R$ respectively. The gray plane is an iso-y plane. (c) Triangulation on the iso-y plane. (d1, d2) Correction for Lens Distortion. Five sets of lattice points near four corners and at the center are recorded. Then the correction parameters are fitted to restore these distorted lattice points to the original correct lattice points. The upper image d1 shows the 5 sets of distorted lattice points positioned at the center. The lower image d2 shows the corrected sets of lattice points.

Download Full Size | PDF

In fact, in our specimen of typical amoeboid cell, the height and width are actually within about 100 micrometers, and since this specimen is observed with an objective lens about 5 cm away, the change in apparent size due to the difference in depth can be ignored. In other words, we will make the most of this special situation. In this regards, two interesting methods have been recently developed : monocular microscopic fringe projection profilometry [24] and microscopic tele centric stereo vision system [25]. These methods were applied to solid hard surfaces such as electronic circuits, allowing observations of a several millimeters in height and width while some special equipment of specimen illumination and careful calibration. Since the observable range of these methods is much larger than that for amoeba, a simpler method may be possible.

For the images obtained by orthogonal projection, stereo matching problem can be reduced to peak matching problem between one variable functions. Thus, the stereo matching becomes more tractable problem solvable approximately by a simple algorithm.

A pair of matched peaks determines the size and the three-dimensional position of a connected component in a cross-section of the amoeboid cell (each cross-section consists of many components with different sizes and positions there), and the total collection of these cross-sections defines the whole body of the cell. Eventually, the cellular surface of the estimated body of the cell is expressed as a collection of triangles. Then the movement of the amoeboid cell can be analyzed in terms of these moving cellular surfaces.

As a test application, our method proposed here was applied to a kind of single-celled protists, Amoeba proteus and enabled us to observe 3D deformation when the amoeba migrated freely. A. proteus is one of the most well-studied protists and shows the drastic deformation of the whole body.

This paper is organized as follows. In Methods, the experimental setup for taking videos for our processing and the detail of the processing is described. We demonstrated large 3D-deformation of cell surface reconstructed by the proposed method. In Discussion, some essential limitations of our processing are considered.

2. Methods for preparation of test sample: Amoeba proteus

2.1 Organisms and culture

Amoeba proteus (wild type) was purchased from Kyoto Kagaku Co., Kyoto, Japan, and was cultured in the standard medium with several particles of rice grain (KCM medium: 7mg/L KCl, 8mg/L CaCl₂, 8mg/L MgSO₄-7H₂O) at 25 degree Celsius in dark in un-sterilized conditions. The medium was refreshed every four or five days. Within two weeks after the purchase from the company, A. proteus was served for the experiment.

2.2 Setup of microscope and observation

A. proteus was gently and carefully sucked by the micro-pipette together with the medium, and was put with the medium on a slide glass and covered by a cover slip with a rubber spacer (1mm thick). This prepared slide was mounted on the binocular microscope (Olympus type SZX16, Olympus Co., Tokyo, Japan) and a migrating A. proteus was observed for several minutes in 25 degree Celsius.

The cellular specimen was observed in the dark field illumination. The cellular specimen was observed in the dark field illumination on the stage of the binocular microscope SZX 16. In the SZX 16, the white light was illuminated almost from the side but a little below and this lateral illumination was performed from all directions around the circular dark stage (ca. 4 cm in diameter) where the specimen was placed. As above, the dark field illumination was homogeneous enough for the measurement. The observation light was scattered by many intrinsic intracellular vesicles and organelles that were distributed throughout the cellular body and moved around the body during the amoeboid movement. The images of specimens were projected on the video camera as a pattern of white dots of scattered light (see Fig. 2(a-c)). Roughly speaking, the specimen appeared as the blinking pattern of white dots while deforming the cell body.

 

Fig. 2. Misalignment and stereo-matching of left and right images. (a) Projected intensity distribution on the vertical axis of this figure with the image rotated by an angle $\phi _L$ around the center of the image. (b) Correction parameters for misalignment of left and right images. (c) Alignment of Left and Right Movies. (d) Local minima in $\phi _L$-$\phi _R$ angle plane. The minima concentrate on $\phi _L - \phi _R\ = const.$ line. The circle indicates the position of the best global minimum as far as we calculated. (e) The values of $\chi ^2(\phi _L,\phi _R,d_y)$ at minima on the line in the previous figure near the global minimum. The circle indicates the position of the global minimum. (f)Stereo-matching of minima of intensity functions. The top thin graph is $I_L(x,y_{fix})$ and the bottom thick graph $I_R(x,y_{fix})$. The thin graph in middle is $F_L(x)$ and the thick graph is $F_R(x)$. The minima matched in our algorithm are connected in the middle figure.

Download Full Size | PDF

Binocular video images (both of right and left images) were detected by monochromatic CCD video cameras (Model XC-77, Sony Co., Tokyo, Japan) and recorded by digital videotape recorder (Handycam DCR-TRV50, Sony Co., Tokyo, Japan) with 640 x 480 pixels. This video image was served for the video image analysis by computer.

Dynamic deformation of amoeboid movement was converted to the spatio-temporal patterns of graded brightness in time series of images. This brightness pattern gave the key information to our stereoscopy.

3. Methods for three-dimensional reconstruction

3.1 Outline for stereoscopic image-processing

The flow of image-processing is summarized below.

Corrections for Lens-distortion The lens-distortion for the cameras are corrected according to the parameters calculated using test-patterns.

Enhancement and Normalization The left and right movies are normalized so as to two movies have the same light intensity distributions.

Corrections for Misalignments The misalignments in optical axes and rotational degrees of freedom between the left and right movies are corrected according to the parameters previously calculated using the actual left and right images.

Stereo Matching By the characteristics of orthogonal projection, the stereo matching in three-dimensional space is reduced to matching peaks of brightness between two single-variable functions of brightness. From a pair of matched peaks, we estimate the position of the cross-section of the amoeba, which is approximated as a disc. From a collection of these discs, we get a probability distribution of amoeba on three-dimensional lattice points.

Probability on Lattice The inevitable stereo matching errors in the previous step are smoothed out by taking the time average of the distribution of amoeba on 3D lattice points.

Marching Cube Identifying the equi-value surface of the distribution of amoeba on the lattice with the cellular surface, we calculate the cellular surface by a straight forward algorithm called marching cube, which generated the surface as a collection of triangles. The error of the positions of vertices is estimated at this point from the steepness of the distribution at the threshold. The steeper the edge, more precise is the position of the surface.

Calculating Various Surface Characteristics From this collection of triangles, by fitting quadratic surface, we can calculate normal vector to the surface, curvatures, and surface velocity at each vertex.

There are two main potential sources of errors in this process. One is the wrong stereo matching of light intensity peaks. Another is the wrong correction parameters, especially the alignment correction.

Because the stereo matching in this process is based on a simple algorithm due to the essential lack of available information, the wrong matching is inevitable. Moreover, the noise of the original movies severely affects this matching.

Since we are interested in the relatively slow movements of cellular surface, not in the faster movements of vesicles, we can use the time average to smooth out this kind of errors. In a step called ‘Probability on Lattice’ in the above process, time average is taken. And in the next step ‘Marching Cube’, the steepness of the probability distribution at the threshold value is used to estimate the size of this kind of errors.

For the systematic errors caused by wrong alignment correction parameters, we can estimate them only by seeing what happens when parameters are deliberately shifted. The results are shown later, which suggests that our parameters are reasonably good.

3.2 Coordinate system for stereo microscope

Throughout this paper, the following system of coordinates is used.

Let $z$ be the unit axis vector perpendicular to the stage of the microscope.

Let $p_L$ and $p_R$ be the unit optical axial vector of the left and right objective lenses. These are the orthogonal projection vectors for the left and right images. The angles, $\theta _L > 0$ and $\theta _R > 0$, made by these vectors with $z$ are precisely determined by the test pattern analysis below. Also, from the same analysis, we confirm that three vectors $z$, $p_L$, and $p_R$ are on the same plane.

We will now construct coordinate systems for left and right images and for real-world three-dimensional space starting from these three vectors.

Let $y = p_L \times p_R / \| p_L \times p_R \|$ be the common y-axis vector for the above three coordinate systems where $\times$ denotes the vector product and $\|v\|$ the length of a vector $v$. This unit vector is perpendicular to vectors $z$, $p_L$ and $p_R$. Then let $x = y \times z$. Thus $x,y,z$ is three-dimensional cartesian coordinate axes of real-world space.

Let $x_L = y_L \times p_L$ and $x_R = y_R \times p_R$ be the x-axis vectors for the left and the right projected images where $y_L = y_R = y$ the common y-axis as depicted in Fig. 1(b).

In these coordinate systems, a point $r = (r_x,r_y,r_z)$ in real-world space is projected to $(r \cdot x_L, r \cdot y_L)$ on the left camera and $(r \cdot x_R, r \cdot y_R)$ on the right camera where $\cdot$ denotes the inner product. But the three coordinate systems share the common $y$-axis, the $y$-coordinate of the projected points is $r_y$.

The $x$-coordinates of the projected points, $r_L = r \cdot x_L$ and $r_R = r \cdot x_R$, are the observable data from the left and right movies. Solving for $r_x$ and $r_z$ from these two equations, we have the following relations using $\theta _L$ and $\theta _R$ as shown in Fig. 1(c). We calculate $r = (r_x,r_y,r_z)$ from observed $r_L$ and $r_R$ using these relations.

3.3 Correction for lens distortion

Discrepancies between optical axes of lenses in a camera lead to the lens distortion where straight lines appear as curves on recorded movies.

This is corrected according to [26] using OpenGL texture warping to generate corrected movies.

The correction parameters are obtained by fitting previously recorded distorted test patterns of plane lattice points by the original square lattice points using the Levenberg-Marquardt algorithm.

Since the optical axes of stereo microscope incline to the left and right sides, a square lattice looks shrunk in the horizontal direction. From this extent of shrinkage, we can calculate the angle between the optical axis and the vertical line of the stage. These parameters are also included in the above fitting parameters.

In Fig. 1(d), we depicted a sample of distorted and corrected lattice points.

3.4 Misalignment of left and right images

The horizontal and vertical axes of the recorded left and right movies are not necessarily identical to those of the coordinate systems defined above. And the center of the left movie does not necessarily coincide with that of the right movie.

The former discrepancies occur due to the rotational degree of freedom of the eyepiece to which camera is attached, and the latter due to the discrepancies of light and right objective lenses.

If the microscope’s images are close enough to orthogonally projected images, both can be corrected by taking advantage of the common y-axis. The light intensity distributions of left and right images should be almost identical when projected on this common y-axis.

An example of the projected distribution is depicted in Fig. 2(a) with a left image rotated by an angle $\phi _L$. Denoting the rotated image by $I_{\phi _L}(x,y)$ where $x,y\in \textbf{Z}$, the projected distribution is denoted by $I_{\phi _L}(y) = \sum _{x} I_{\phi _L}(x,y)$. The rotated image is calculated by OpenGL texture mechanism, then the projected distribution is calculated by reading and summing the light intensity value of pixel.

A similar projected distribution can be obtained by rotating the right image by $\phi _R$ then translating by $d_y$ vertically as depicted in Fig. 2(b). This is denoted by $I_{\phi _R,d_y}(y)$.

The difference of these two distributions can be evaluated by a function like $\chi ^2(\phi _L,\phi _R,d_y) = \sum _{y} (I_{\phi _L}(y) - I_{\phi _R,d_y}(y))^2$.

If we can find the set of values $\phi _L$, $\phi _R$ and $d_y$ which minimizes $\chi ^2(\phi _L,\phi _R,d_y)$ as depicted in Fig. 2(c), then the vertical direction at that values is the common y-axis. The x-axis for each camera is the horizontal direction.

Concerning the discrepancy in the center of the left and right images, the $y$-direction is already corrected by $d_y$. And the discrepancy in $x$-direction only results in the translation of $z$ coordinate as seen from the relation in the previous section, so we do not need any correction.

To find the minimizing set of values $\phi _L$, $\phi _R$ , and $d_y$, we use the Levenberg-Marquardt algorithm to find the minima by following the descending directions. So it is often trapped in local minima before reaching the target global minimum.

Since these three parameters are constant in one experiment, the quantity we try to minimize is a summation of $\chi ^2(\phi _L,\phi _R,d_y)$ at some sampled points from that experiment.

The convergence of the algorithm is very good until it reaches the $\phi _L - \phi _R\ = const.$ line. If the original object is planar, the quantity to minimize is a constant on this line, so no more convergence is possible.

Even for a three-dimensional object, the convergence on this line is very bad. Since there are many local minima, we have to do many trials to reach the global minimum as shown in Fig. 2(d) and Fig. 2(e).

Using these parameters, the corrected movies were calculated.

3.5 Stereo-matching of left and right images

Since the images obtained by microscopes can be adequately approximated by orthogonal projections, and using coordinate systems which share the common $y$-axis, the task of stereo-matching is reduced to finding correspondence between peaks of two intensity functions of $x$ on the same iso-y plane.

More precisely, let $I_L(x,y)$ be the left image, and $I_R(x,y)$ the right image where a pair of integers $1 \leq x \leq x_{max}, 1 \leq y \leq y_{max}$ denoting a pixel position on screen. On an iso-y plane $y = y_{fix}$, we only have to find correspondence between intensity functions $I_L(x,y_{fix})$ and $I_R(x,y_{fix})$.

Our algorithm is based on the following assumptions.

In other words, we assume that the same peaks appearing in both left and right intensity functions of $x$ in the same order, but the positions of peaks are a little different.

These assumptions frequently do not hold true. For example, the light intensity of an isolated particle can be very different in left and right images. A dim light source is practically invisible when a brighter light source overlaps.

These things lead to the stereo matching errors. And this is the main source of errors in our reconstruction. We try to minimize the influence of these errors by heavily averaging over time as described later.

According to the assumptions 1, the peak areas of the same peak should be equal in $I_L(i,y_{fix})$ and $I_R(i,y_{fix})$. According to the assumptions 2, at the tails of n-th peaks from left, the total peak areas from left should be equal.

More precisely, consider the cumulative intensity functions $F_L(x) = \sum _{i=1}^x I_L(i,y_{fix})$ and $F_R(x) = \sum _{i=1}^x I_R(i,y_{fix})$ and let $x_{L,n}$ and $x_{R,n}$ be the positions of tails of n-th peaks in respective intensity functions. We should have $F_L(x_{L,n}) = F_R(x_{R,n})$.

The positions of tails of peaks are taken as the minimum points between the two adjoining peaks or the edge of the amoeba. This is another assumption which fails frequently by overlapping of peaks.

So our stereo matching algorithm is the following.

An example is depicted in Fig. 2(f).

3.6 Reconstruction of cellular surface

3.6.1 Tangent circle

The positions of minima and edges are considered as the positions of the edges of extended light sources viewed from that angles. A pair of adjacent minima and the corresponding pair determine the position and extent of a light source represented as an ellipse approximately tangent to these four lines of sight. An example is shown in Fig. 3(a).

 

Fig. 3. Reconstruction of cellular surface. (a) Ellipses approximately tangent to four lines of sight. The angles of sight are actual ones measured in our stereo microscope. See text for multiple ellipses. (b1) Collection of tangent discs. (b2) Existence probability of amoeba on three-dimensional lattice points. The apparent size of the box on a lattice point is proportional to the probability of the point. (b3) Cellular surface calculated by marching cube algorithm from the distribution in the previous figure which is also depicted in this figure. The cellular surface is depicted as a mesh of triangles. (c)The outlines of cellular surfaces for $T_p=0.1, 0.5, 1, 2, 3$ from outer to thin inner lines viewed from the position of the left camera. The thin black line denotes the outline of equi-value surface for the actually used value $T_p=1$ in this paper. The outline of the original left image is depicted in the thick black line.

Download Full Size | PDF

But as shown in the figure, this tangent ellipse is not unique. All ellipses drawn would look the same in the left and right intensity functions.

In our algorithm, the circular discs are used in these cases so as to minimize the artificial information added in the process.

This means that the volume calculated from the reconstructed surface is meaningless. The ellipses have different areas, as seen from the figure. And the cross section of amoeba would not always be circular. To measure the volume, we should be able to determine the cross section.

3.6.2 Probability distribution on lattice

Now we have a collection of these discs for all $y_{fix}$ values as shown in Fig. 3(b1). Next, we construct probability distributions of the amoeba on three-dimensional cubic lattice points. This facilitates the time-average of results necessary for smoothing out the inevitable stereo matching errors.

Contribution $w_{ijk}$ to cubic lattice points $(x_i,y_j,z_k)$ of lattice size $s$ by a disc of radius $r$ at $(c_x,c_y,c_z)$ is calculated as follows.

Summing up all the contributions from discs, we have the unnormalized probability distribution $f(x_i,y_j,z_k)$ on lattice as in Fig. 3(b2).

A time average is taken at this stage with Gaussian weight of standard deviation 120 frames $\sim$ 4 seconds.

3.6.3 Marching cube algorithm

Assuming one of the equi-value surfaces of this distribution is the cellular surface, we can calculate the cellular surface by another straight forward algorithm called marching cube. In this algorithm, equi-value plane of each cube of the three-dimensional lattice is calculated by linear interpolation. By joining these planes, the equi-value surface is obtained as a collection of triangles. In Fig. 3(b), we depicted the cellular surface of the distribution.

The equi-value surface depends on the surface value $T_f$. The right value for $T_f$ can be determined by comparing it with the experimental data. In Fig. 3(c), the outlines of equi-value surfaces for some values of $T_f$ are shown with that of the original image. These outlines are calculated from the reconstructed data viewed from the position of the left camera so as to be compared with the outline of the original left image.

From Fig. 3(c), we use $T_p=1$ in this paper.

3.7 Error estimations

3.7.1 Stereo matching errors

The error in stereo matching results in the long tail of the probability distribution due to the heavy time average. To estimate this error, we need a more detailed description of the marching cube algorithm.

Consider the equi-value surface for $f=T_f$. If $f(x,y,z) = a < T_f$ and $f(x + 5,y,z) = b > T_f$ (so the surface is between these two lattice points), then the linearly interpolated point $(x + 5(T_f - a)/(b - a),y,z)$ is defined as one of the vertices of this surface in this algorithm.

In this process, the position of the vertex is less sensitive to the value of $T_f$, if the slope of the tail of the distribution is steeper, namely $b - a$ is larger. In Fig. 4(a), we depicted this relation of the existence probability of amoeba and the distribution of cellular surface.

 

Fig. 4. Stereo-matching error. (a)Relation between the slope of tail of distribution of amoeba and the distribution of cellular surface. Solid lines are for the best case in which the distribution of the cellular surface is a uniform distribution between two adjacent lattice points. Broken lines are for not so good case in which the distribution of the cellular surface is much spread. (b)Error bars due to stereo matching errors estimated from the distribution of amoeba in Fig. 3(b2). Notice the dorsal errors tend to be larger than those of sides.

Download Full Size | PDF

We assume the distribution of the cellular surface as the uniform distribution along the tail of the distribution of amoeba as in Fig. 4(a). And we use the standard deviation from the center of this uniform distribution as the error size. If the width of uniform distribution is $w$, the standard deviation is $w/2\sqrt {3}$.

In the above example, the slope of the distribution near the vertex is $(b - a)/5$. So the width of the uniform distribution of the surface is $w = 5f_{max}/(b - a)$, where $f_{max}$ is the maximal value of unnormalized probability $f(x,y,z)$. In this case, we define the error vector as $(5f_{max}/2\sqrt {3}(b - a),0,0)$. The error of the position of the surface in the normal direction of surface is obtained as an inner product of this error vector and the outward normal vector at this vertex.

A typical example is shown in Fig. 4(b). The dorsal and ventral errors tend to be larger than those inside. This is because there is not much room for the error in matching the first or the last peaks as seen from Fig. 2(f).

As shown in Fig. 4(b), most of the dorsal or ventral bumpy patterns are on the same magnitude as the error bars. Just ignore them.

3.7.2 Error due to misalignment

If the correction parameters in Sec.3.4 are wrong, a systematic error occurs which can not be estimated by the methods in the previous section. We will now see what kind of situation we should expect in this case.

Let $\phi _{L,opt}$ and $\phi _{R,opt}$ denote the global minimum point in Sec.3.4. We saw in that section that the local minima are on the line $\phi _L - \phi _{L,opt} = \phi _R - \phi _{R,opt}$.

In Fig. 5(a), we depicted the reconstructed images with alignment correction parameters shifted along this line and along the line perpendicular to this line.

 

Fig. 5. Cell surfaces depending on errors due to misalignment, and estimated deformation velocity of the cell surface. (a)Errors due to misalignment. In the middle row, the optimal image with alignment correction parameters $\phi _{L,opt}$ and $\phi _{R,opt}$. In the bottom row, the reconstructed images with $\phi _{L,opt} \mp 20^\circ$ and $\phi _{R,opt} \mp 20^\circ$. In the top row, the reconstructed images with $\phi _{L,opt} \mp 2^\circ$ and $\phi _{R,opt} \pm 2^\circ$. (b)Surface velocity. White arrows denote expanding parts of the cellular surface, while the dark arrows denote shrinking parts. (One example was shown in Supplementary movie. See Visualization 1.)

Download Full Size | PDF

As seen from this figure, the images are very sensitive to the error in the direction perpendicular to this line. Probably, one tenth of this error is intolerable in most cases. This means that we need more than 0.1$^\circ$ precision in angular control to accomplish our reconstruction.

On the other hand, the images are less sensitive to the error along this line. This reflects the fact that the convergence of the minimum-seeker algorithm is bad along this line.

3.8 Surface velocity and other characteristics

The movement of the unmarked surface is quantified by the surface velocity, which is defined as the velocity of the surface in the direction of normal vector of the surface.

In our image-processing, the normal vector at a vertex is calculated by fitting neighboring vertices by a quadratic surface. Then the nearest intersection of this normal line and the cellular surface at a 60 frames $\sim$ 2 seconds later is calculated. The distance between the vertex and this intersection divided by 2 seconds is the surface velocity at this vertex. The sign is taken so that the outward velocity is plus. An example is shown in Fig. 5(b).

Now each vertex has the following information.

Integration of surface velocity over an area is the volume increasing rate of that area. Using this, the volume increasing rate of each pseudopod can be calculated. Thus we can track the movements of all pseudopods.

The tip of a pseudopod is defined as a connected collection of vertices having surface velocity larger than a pre-defined threshold. Then the volume increasing rate of this tip is defined as $\sum _{i \in tip} v_i s_i$, and the position of this tip as $\sum _{i \in tip} v_i r_i / \sum _{i \in tip} v_i$. The tips in the next frame are identified with those in this frame based on these positions.

Time course of volume increase rates of tips can be plotted, and the movements of amoebas are analyzed using this kind of figures.

4. Discussion

In our reconstruction process, the most significant limitation is that we can not observe the cross sections of amoeba directly. We substitute circular discs for them. Although the positions of the centers of cross sections in three-dimensional space are correct, the shapes of cross sections are totally fictitious.

This affects various aspects of our three-dimensional reconstruction.

For example, the volume is almost meaningless in this situation where the areas of cross sections are uncertain. As shown in Fig. 3(a), we can change areas of cross sections without changing the images from the left and right cameras. Also, the bumpy dorsal and abdominal patterns are meaningless. These are strongly dependent on the shapes of cross sections. We can not hope that our circular discs reproduce the correct dorsal and abdominal surfaces.

On the other hand, our reconstruction is very accurate concerning the relative positions of peripheral parts of cells in three-dimensional space because the chance of mismatching is slim there. For example, whether a pseudo-pod is extending upward or downward? Whether two organisms, seemingly making contact in two-dimensional images, really make contact or not in three-dimensional space? Our reconstruction can give definitive answers to these kinds of questions. This is very useful in analyzing the behaviors of amoebae.

As described in the test study, it was detected how the cell shape deformed when A. proteus crawled freely on the flat plate of glass. It turned out that the novel method introduced in this report was good enough to detect very active amoeboid movements like A. proteus. This technique can be applied to the other species of protists: not only amoeboid movement but also collective motion of colonial or multicellular ciliate and algae. Lastly, although the 3D accuracy of the proposed method is relatively less and to be examined more, we would like to note two advantages : (1) Faster time resolution can be achieved with the high-speed camera because of no need of scanning. (2) Intact cells can be observed because of no need to stain the cells or to introduce marker particles into the cells. Due to convenience, this binocular stereo-microscopy is to be used in the future.

Our method proposed in this manuscript was devised by making the best use of the specific feature that the specimen is within the range of about 100 micrometers in height and width. Although a general model for the optical paths was not always required in this paper, it is important and interesting to consider a general model in order to apply this method more widely to a different size of cellular specimen.