Open Access
Add to BibSonomy
Abstract
Image plane off-axis holograms (IP-OAH) are the most common data captured in digital holographic microscopy and tomography. Due to increasing storage and data transmission requirements, lossy compression of such holograms has been subject of earlier investigations. However, hologram compression can not be allowed to hinder the metrological capabilities of the measurement technique itself. In this work, we present lossy and lossless IP-OAH compression approaches that are based on conventional compression codecs, but optimized with regard to bandwidth of the signal. Both approaches outperform respective conventional codecs, while the lossy approach is shown to uphold the accuracy of holographic phase measurements.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Digital holographic microscopy (DHM) [1] is a mature branch of quantitative phase imaging (QPI) [2]. It is often used to investigate a variety of subjects, both biomedical and engineered [3–6]. With widespread adoption, successful commercialization [7–9] and emerging new modalities [10,11] the problem of data storage becomes significant. For example, raw 8-bit data from a one minute, one frame per second video-like measurement reaches $288\textrm { MB}$, assuming a popular Sony IMX250 sensor. To reach with DHM the capabilities of traditional pathology slide scanners that are able to measure 15 mm $\times$ 15 mm regions [12,13], the storage cost is estimated to explode to over $100 \textrm { GB}$ of raw data. This estimate explains the necessity of hologram compression.
In general, methods for holographic data compression can be classified into two categories [14]:
a) Transform based methods transforming the wavefield into a different basis in the frequency domain, where they can be more efficiently decorrelated and where an approximation of the signal is obtained by storing only a few coefficients. While such techniques are fundamental in lossy natural image compression, they perform sub-optimally when applied directly to hologram compression.
b) Probability distribution based methods based on probability distribution of the coefficients of the wavefield in different representations [15]. Quantization techniques are used for lossy solutions [16,17], while lossless compression performs best on real/imaginary parts of the wavefield with the usage of Lempel-Ziv-Welch, LZ77, Huffman or Burrows-Wheeler algorithms.
Compression of IP-OAH, holograms captured in the image plane (IP) with the off-axis (OA) reference beam, is the subject of this work. This is dictated not only by their popularity in commercial systems [7,18,19], but also by their useful properties discussed in Sec. 2. This specific type of holograms has been investigated from the compression point of view by other researchers.
Jaferzadeh et al. [20] retrieved phase images of red blood cells and compressed them using both lossless and lossy image codecs. For the lossy codecs they measured the degradation of the extracted morphological parameters of the cells given the measured compression ratio (CR) and for lossless codecs the achieved compression ratios were determined. The drawback of their approach is that the distortion measures were based on low frequency features making the investigation very specific to the blood cells investigation. Additionally, compressing only phase leads to loss of amplitude information, even in the case of lossless codecs. Full complex amplitude information is relevant when: a) a need exists to alter the phase-retrieval pipeline after compression; b) both retrieved amplitude and phase are required, for example when there is a need for numerical propagation of the optical field. [21]
Hence, due to these requirements, it is often more desirable to compress the original hologram measurements directly.
In the work of Blinder et al. [22], an extension to JPEG 2000 was proposed for compressing general, out of image plane off-axis holograms (OAH). By extending the wavelet transform with directional wavelet bases and arbitrary packet decompositions of the subbands, the features of the strongly oriented high-frequency interference fringes in the holograms are captured more efficiently – and hence, an improved energy compaction is realized by the transform – enabling gains in compression performance over the standard JPEG 2000 implementation.
In the works of a group from MEPhI [23–25], the applicability of different types of wavelet bases and their effect on the obtained reconstruction quality have been extensively studied. Among the tested wavelets, the best results were obtained by a 3-level Haar wavelet. While they report impressive CRs, they do not provide any distortion metrics at those CRs, and rely on visual inspection that indicated significant distortion.
Finally, the efforts in hologram multiplexing, which is introducing multiple measurements in a single hologram, should be noted. Multiplexing can be achieved either by hardware [26–29] or software [30]. Although the solutions were often grounded on band limitation of the signal which is also being utilized in this work, they did not investigate them from the compression point of view.
The referenced works provided some methods to estimate the distortion, either subjective or objective. However, none addressed directly the degradation of the most important aspect from the QPI perspective, namely the distortion of retrieved unwrapped phase. This distortion can be evaluated by the root mean square error (RMSE) on the retrieved unwrapped phase. In our lossy compression tests, we maintain the usual DHM accuracy of $\textrm {RMSE}=0.05$ rad, i.e. below $\frac {2\pi }{125}$ rad.
A novel lossy compression scheme that can be implemented with any conventional image/video codec has been briefly introduced in our previous work [31]. In this work, we give a more detailed description of our approach, including an analytical justification for the method and a performance assessment of the proposed technique on a representative and diverse test set. Tests show a substantial improvement over direct compression. Additionally, we introduce a novel lossless compression approach that is based on our lossy method, yet fully preserves the original measurements. The lossless technique is assessed on the same extensive test set and is shown to achieve better CRs than conventional lossless codecs. Our approach exploits the following observations: a) the maximum bandwidth of the first-order term can be explicitly calculated based on the operating parameters of the microscope, b) the object transmittance function encoded in the first-order term is slowly varying for many kinds of technical and biological samples, which implies the transmittance function is comprised of predominantly low frequency content that can be compressed with any traditional image/video codec.
The paper is structured as follows. Section 2 describes a typical DHM setup that meets the requirements for efficient and metrologically reliable compression. Subsection 2.2 provides the motivation for our compression approach by analyzing the bandwidth of the optical signal. The design of the proposed compression solutions are discussed in detail in Section 3. In Section 4, we discuss the experimental setup used to evaluate the compression performance of the proposed methods and experimental results are presented and discussed in Section 5. The conclusions and suggestions on future work are given in Section 6.
2. Off-axis digital holographic microscopy
2.1 System setup
DHM hardware setup is typically based on Mach-Zehnder interferometer, with a modified object arm, in order to substitute free-space propagation after the object with the microscopic imaging setup providing high magnification (see Fig. 1).
Fig. 1. Schematic of a generic DHM setup. Optical elements are: LASER - laser light source, C - collimator lens, M - mirror, S - sample, MO - microscope objective, TL - tube lens, CAM - camera. Red colored beam indicates a light scattered by the object.
Off-axis holography, is a holographic scheme with the tilted reference beam as an approach to solve the twin image problem, that appears whenever phase retrieval from intensity images is required from a single measurement. The tilt introduces a carrier frequency that allows to separate +1, -1 and 0 diffraction orders in Fourier domain. In case of full spectral separation, a fast and robust Fourier filtering method [32,33] may be applied, otherwise more sophisticated approaches [34–36] have to be used. However, this simultaneously implies excessive spatial bandwidth utilization - a camera is required to have a much higher sampling rate than what is utilized in diffraction limited phase retrieval. Even more redundancy is required if the setup allows for microscope objective (MO) swapping, due to MO being the band-limiting element in a well-designed microscope system.
On the other hand, image plane hologram capture yields several benefits: a) bypassing the need for numerical propagation, hence simplifying QPI processing and allowing for easy modality switching between modalities such as fluorescent imaging, that do not have numerical refocusing capabilities; b) compact design by shortening the object arm [37]; c) concentrating optimally the bandwidth of individual diffraction orders.
Sanchez-Ortiga et al. [38] discuss the frequency response of a general non-confocal DHM system. However, as they explain, the non-confocal DHM systems are sub-optimal. In the following section, we summarize their work regarding an optimal confocal image-plane DHM systems.
In the remainder of this work, we will – unless stated otherwise – refer to image plane off-axis holograms simply as holograms or IP-OAH.
2.2 System analysis
In the subsequent discussion, we will adhere to the following notation. Vectors are given in bold with $\textbf {x} = (x, y)$ being the spatial coordinates and $\textbf {u} = (u, v)$ frequency coordinates. The 2-dimensional Fourier transform of a signal $f(\textbf {x})$ will be given by its capitalized notation as $F(\textbf {u})$ where
DHM requires an overlap of two mutually coherent beams, namely the object and reference beam, in the image sensor plane. The mutual coherence is ensured by the use of a high temporal and spatial coherence laser of wavelength $\lambda$ and a Mach-Zehnder-based setup with a small optical path difference between its arms. The typical scheme of DHM based on the Mach-Zehnder interferometer architecture is shown in Fig. 1. The beams are originally formed as plane wavefront beams. The object beam is modified upon transmission through the object and the imaging system, whereas the reference beam remains plane. The information about the beams is encoded as an interference fringe pattern on the hologram image captured by the camera. The hologram intensity map $i(\textbf {x})$ can be modeled according to the hologram equation:
Formation of the image $o(\textbf {x})$ is achieved by the microscope system implemented in the object path, but band limits the signal. The imaging system is comprised of a infinity corrected microscope objective (MO) with focal length $f_{\textrm {MO}}$ and a tube lens (TL) with focal length $f_{\textrm {TL}}$ (Fig. 1). The Joint MO-TL magnification is given by $M$. The MO collects the optical field $o_t(\textbf {x})$, that is an object beam right after transmission through the sample. $o_t(\textbf {x})$ is also known as the object transmittance function. The sample is placed in the MO’s front focal plane, so the optical field $O_p(\textbf {u})$ at the MO’s pupil plane, after omitting amplitude scaling factors and phase offsets, may be expressed as in [38]:
with $P(\textbf {u})=\textrm {circ}\left (\frac {|\textbf {u}|}{r}\right )$ being the pupil function of the MO. The pupil radius $r$ may be approximated as $r=\textit {NA}\cdot f_{\textrm {MO}}$ for a sufficiently small MO’s numerical aperture $\textit {NA}$, while $f_{\textrm {MO}}$ acts as the object-to-MO distance in the discussed case. Equation (4) shows clearly that the Fourier spectrum of the optical field at the MO pupil plane is band-limited by the pupil function. The TL transforms the field $o_p(x)$ creating a magnified image as the object beam $o(x)$ that co-produces the hologram (See Eq. (2)). Omitting again amplitude scaling factors and phase offsets, the object transmittance function $o_t(x)$ is related to the object beam as:3. Compression pipeline
The proposed methods exploit the band-limitation (Eqs. (6) and (7)) of the image plane holograms. They are general enough to work in the sub-optimal conditions, such as partial overlap of diffraction orders in hologram spectrum or hologram acquisition in the presence of misalignment aberrations [38]. However, investigation of their performance in such cases is beyond the scope of this work. In this section, a detailed explanation of the proposed compression methods is given.
3.1 Lossy compression
The goal of lossy compression is to approximate the reference (uncompressed) input signal as closely as possible given a limited bit rate budget, according to some application-dependent distance metric. Concretely, one aims to prioritize the encoded information in a signal and discard the least relevant parts for the intended application (such as noise) to reach a given bit rate. Although such strategies always result in a compressed signal that is different from the original signal, lossy techniques can be used to achieve data rates that are several orders of magnitude smaller than the uncompressed representation without suffering from noticeable performance loss. For example, in natural audio/image coding, the strategy involves modelling human sensory system such that the least perceptible information is discarded first.
In the case of IP-OAH, the relevant information is the object transmittance function mentioned in Eq. (5), which is encoded in the +1 order term. While the -1 order term can be trivially obtained as the conjugate of the +1 order term, both the 0 order and -1 order terms are usually considered as noise terms in most applications and are removed during the processing of hologram. Thus for our lossy compression system, we will only store the +1 order term.
As discussed in Section 2.2, no metrologically relevant information is found outside this bandwidth for IP-OAH. While adaptive filtering techniques may be used to improve the reconstructed signal for complex amplitude retrieval [40–42], our goal here is to provide metrological reliability of the compression process with respect to the original measurements. Therefore, all features that lie within the bandwidth of the signal ought to be preserved, no matter their origin: from the object or system errors like coherence artifacts.
For a hologram $i[x,y]$ of size $A \times B$, the filtered first-order term $o_f[x,y]$ in the spatial domain is now converted to the digital frequency domain by applying a 2D-Discrete Fourier Transform of the same size to obtain $O_f[u,v]$. $O_f[u,v]$ is demodulated and bandlimited as given in Eq. (8).
The object transmittance function is slowly varying for typical objects. The demodulated and bandlimited first-order term $o_d[x,y]$ is essentially a rescaled version of the object transmittance function and will have spectral characteristics similar to natural imagery (low frequency dominated signal), which is shown in Fig. 2 by visual comparison of Fourier spectra of a biological specimen and a typical natural image. Although the distortion metrics will ultimately assess the performance, this intuitively justifies that $o_d[x,y]$ should be efficiently compressed using existing image/video codecs. However some transformations must be applied on $o_d[x,y]$ before the codecs can be utilized. Since codecs operate on data in unsigned integer representations (8-bit in our experiments), $o_d[x,y]$ is split into its real and imaginary terms on which 8-bit uniform quantization is applied. We recorded better compression performance while encoding the real and imaginary components, instead of amplitude and wrapped/unwrapped phase, when measuring the signal-to-noise ratio (SNR) of the first-order term. This is in agreement with the literature [43,44]. The quantized real and imaginary terms now serve as inputs to the encoder of the different codecs, whose output is stored.
Fig. 2. Logarithm of normalized absolute valued frequency spectrum (centered around zero frequency) for the demodulated and bandlimited term of an off-axis hologram of a single cell (left) and an image of a mountain (right) of same size.
The decoder for our method will initially reverse the above mentioned steps starting from application of the codec decoder on the encoded terms obtaining $\widehat {o_f}[x,y]$, which is the lossy representation of the first order term $o_f[x,y]$ in the spatial domain. The final decoded lossy representation $\widehat {i}[x,y]$ of the original measurement output is obtained as the superposition of the +1 order and -1 order term and is given by Eq. (9):
Fig. 3. Processing diagram for the proposed lossy compression (yellow arrows) and decompression (red arrows) of off-axis holograms. Solid elliptic lines indicate bandwidth of diffraction orders, whereas yellow dashed rectangle illustrates region that is being preserved. Complex-valued matrices, indicated by $\mathbb {C}$, are visualized as $\log _{10}$ of the absolute value of the matrices in case of frequency domain and as absolute value of matrices in case of spatial domain.
3.2 Lossless compression
Lossless preservation of the holograms by directly using standard image/video codecs is inefficient due to the high carrier frequency of +1 and -1 order terms. These codecs have been designed to compress signals having predominantly low frequency content which affects their performance in the presence of high carrier frequency.
In the previous section, we introduced an approach to lossy compression of IP-OAH. We will now introduce a lossless scheme that utilizes the lossy compression approach in conjunction with any lossless image/video codec.
For a hologram $i[x,y]$ of dimensions $A \times B$, the lossy compression scheme is initially used to obtain the lossy representation $\widehat {i}[x,y]$. The fundamental idea used in the proposed lossless compression is to remove the high-frequency content from the original measurements by subtracting the decoded +1 order and -1 order terms to obtain a low-frequency residual term which can then be more efficiently compressed using lossless image/video codecs. Therefore the residual signal $i_{\mathrm {res}}[x,y]$ is obtained as
The total rate $R$ for the lossless compression scheme will be given by the sum of bits required for the encoding the first order term and the bits required for encoding the residual.
The parameter to be determined for the optimal lossless compression strategy is the bits assigned to the lossy encoder for encoding the first order term. Intuitively, it can be seen that increasing the bitrate for lossy first order term will result in a residual signal that has more of its high frequency removed, which implies lesser number of bits are required for residual compression. It is not directly possible to determine the optimal rate allocation for the lossy first order term that obtains the minimal overall rate. However, in our experiments, we observe that the total rate $R$ as a function of $R_{\mathrm {lossy}}$ is unimodal, which is exemplified in Fig. 4. To estimate optimal $R_{\mathrm {lossy}}$ we use the golden Section search. It is observed that a near optimal value can be obtained in less than 10 iterations of the golden Section algorithm. The lossless compression scheme is summarized in Fig. 5.Fig. 5. Data flow for the proposed lossless compression of off-axis holograms, based on the proposed lossy compression scheme. Note that the higher frequencies related to the +1 and -1 order terms have been removed from the residual image.
4. Experimental setup
In this section, we elucidate the parameters of the optical systems, the test sets, the codecs and objective metrics used in our experiments.
4.1 Codec description
Sections 3.1 and 3.2 discussed the proposed lossless and lossy compression techniques that utilize traditional image/video codecs. The codecs and their various modes assessed in our experiments are given in Table 1. A short description of each codec is given along the function calls used to execute the codecs in Code 1 [45].
In video compression terminology, intra mode means consecutive frames are compressed separately, while inter mode exploits also the redundancy between consecutive frames using motion estimation and compensation techniques. Since intra mode is the most basic mode, only the capability of inter mode is accentuated in Table 1 (column “Video support").
Table 1. Codecs used in the experiments.
4.2 Complex amplitude retrieval
For the complex amplitude retrieval, we use the standard Fourier filtering method [32,33]. The method consists of transformation of the hologram $i(\textbf {x})$ to the frequency domain, appropriate filtering and transformation back to the spatial domain. In our implementation, we introduce two improvement techniques: a) Reduction of high frequency artifacts arising from signal non-periodicity (Gibbs phenomenon), by applying a Tukey window with a $10\%$ lobe. This is done before each transformation - the rectangular window is applied on the hologram and elliptically-shaped window is applied on the extracted frequency components. b) Sub-pixel precision of filtering window placement by multiplying the hologram by a linear phase function.
The reference beam wavevector $\textbf {k'} \approx \textbf {k}$ is estimated by finding maximum energy component in frequency domain. The estimation is further improved by sub-pixel center of mass calculation. The original hologram is then multiplied by a linear phase function. The phase function is created based on the sub-pixel calculation of $\textbf {k'}$ and yields a sub-pixel circular shift that places the diffraction order of interest in the center of frequency domain. The spectrum is cropped to the size $\frac {\textit {NA} \Delta x \textbf {N}}{\lambda M}$ and masked with a elliptic Tukey window isolating a single diffraction order. The spectrum is zero-padded to the original hologram size, the inverse Fourier transform is calculated and the complex phase component is unwrapped [46,47].
4.3 Optical system parameters
We perform the experiments using data captured using two different optical setups, whose parameters are shown in Table 2. The majority of the experiments were obtained using Setup 1. Additional test data was captured with Setup 2, to demonstrate that presented approaches are not limited to a single DHM setup.
Table 2. Optical systems parameters used in the experiments. The last column indicates the size of the rectangular frequency window that contains the +1 order term in the Fourier domain.
4.4 Test sets
All static and time-lapse measurements have been captured using Setup 1, with the exclusion of the phantom [48] hologram that has been captured using Setup 2. (see Table 1), due to the presence of very fine details.
All holograms are 8-bit images, which based on the literature [49] introduces quantization errors below an order of magnitude lower than our RMSE target of 0.05 rad.
Static captures
Static objects have been chosen with the varying complexity in mind. We decided to include both technical and biological samples. The samples are: single HaCaT cell, HaCaT cell culture, slice of brain tissue, artificial cell phantom [48], microspheres and USAF resolution test. The samples are shown in in Fig. 6.
Time-lapse
Fig. 6. Phase retrieved from raw holograms from the single captures test set. (a) single HaCaT cell, (b) HaCaT cell culture, (c) slice of brain tissue, (d) artificial cell phantom [48], (e) microspheres, (e) USAF resolution test.
Time-lapse is a video-like measurement, often used to investigate processes that evolve over an extended period of time. The technique is often used in the context of tracking changes of biological samples. Since each frame of the time-lapse dataset is a single measurement, the data storage requirements grow very quickly, especially when a high sampling rate is needed.
The test sets chosen for our experiment comprises of three time-lapse measurements of cell cultures with various confluence and cell motility. The samples are: neuroblastoma SH-SY5Y cells, keratinocyte HaCaT cells and nasal epithelium cells. They are shown in Fig. 7. All objects were measured with Setup 1.
Fig. 7. Phase retrieved from first raw hologram from captured sequence test set. (a) Neuroblastoma SH-SY5Y cells, (b) keratinocyte HaCaT cells, (c) nasal epithelium cells.
4.5 Assessment metrics
Objective assessment of different lossy compression techniques is done by means of a rate-distortion function, which maps a distortion metric as a function of a rate metric. The distortion metric is calculated by comparing the information retrieved from the original hologram to the information retrieved from the compressed hologram as shown in Fig. 8. The same wavefront retrieval pipeline (Section 4.2) is used for both the original hologram and compressed hologram. For lossless compression, there is no distortion involved and only the rate metric of different techniques are used for assessment.
Rate metrics
Fig. 8. Pipeline for the assessment of the lossy hologram compression influence on the encoded wavefront information.
Compression ratio (CR) is designed to elucidate the improvement in file size due to compression, with respect to the original file. Throughout this work we adhere to the following definition:
The bitrate accounts for the average number of bits per pixel used in the compressed representation of the input hologram (or holograms). The bitrate defined as the number of bits per pixel is given by:
Distortion metrics
We use two objective distortion metrics - Signal to Noise Ratio (SNR) of the +1 order wavefield and the Root Mean Squared Error of Retrieved Phase (RMSE) of the unwrapped phase from the wavefield.
SNR is a mean-squared-error-based metric that gives a measure of how well the original complex wavefield (both amplitude and phase) is preserved by different compression techniques. For the demodulated first order wavefield $O_d[u,v]$ and its compressed version $\widehat {O}_d[u,v]$, the SNR is given by
The Bjøntegaard Delta Signal to Noise Ratio (BD-SNR) [50] is a metric used to compare the SNR performance of two codecs across some rate region. The BD-SNR gain of some codec over a reference codec for the rate region, will be given by the surface area that lies between the rate-SNR curves of the two codecs, where the rate axis is logarithmically scaled.
The compressed first-order wavefield can be used for a variety of metrological purposes, where the phase component is usually more important. Phase-retrieval is a non-linear process due to the phase unwrapping process, which can sometimes introduce strong unwrapping errors even for small errors in the wavefield of the first order term. Measuring the error on the unwrapped phase via RMSE can provide additional insights on the effect of compression on metrological accuracy in practice.
5. Results
For readability purposes, in this section we only show a summary of the obtained compression results. Fully detailed results are provided in supplementary materials (Data File 1-Data File 3 [54–56]) and their preprint version [57].
The results are grouped into two categories to analyze the gain obtained by proposed lossy and lossless method.
a) Default compression - codecs operate directly on the hologram intensity measurements as if they were generic natural images.
b) Proposed compression - codecs operate on the hologram measurements using the proposed method.
5.1 Lossy compression
Static captures
In this section, all codecs that support lossy compression have been evaluated (see Table 1). Based on the BD-SNR and T-RMSE rate – detailed in Data File 1 [54] and Data File 2 [55] respectively – we observe that with the default approach the JPEG DAPD is overall the best performing codec (best in 4 out of 6 cases). This is justified by JPEG DAPD being the only codec designed to compress off-axis holograms. The second best is the HEVC, leading in 2 out of 6 cases.
With the proposed approach, JPEG DAPD is no longer the best performing codec, because after demodulation the compressed signal is much more similar to natural imagery. All tested codecs benefit with the proposed approach where JPEG 2000 is shown to be the best performing codec, leading in 6 out of 6 cases, both in terms of BD-SNR and T-RMSE rate. Table 3 summarizes the compression ratio and bitrate per discrete frequency, achieved by JPEG 2000 on the test holograms, as well as the improvement ratio of T-RMSE rate, with respect to best performance of the default compression. It is shown that the proposed approach achieves 4.68 times smaller files on average (up to 8.5 times smaller). It also shows by the means of bpf (see sec. 4.5 and Eq. (15)) that the achieved compression rates are not only due to reduced number of pixels after truncation of the spectrum to the system’s band limit - despite inputting less data into the codec, the two 8-bit components, real and imaginary, are further compressed between 3 to 14 times.
Table 3. Summary of the proposed lossy compression results for static captures, assessed for the best performing codec (JPEG 2000). Last column gives the improvement in terms of T-RMSE rate offered by JPEG 2000 under proposed compression versus the best performing codec under default compression.
Figure 9 is a visual example of proposed compression performance. It shows that using 0.20 bpp compression rate yields visually impeccable results with the proposed method, yet default method fails with the same bitrate.
Fig. 9. Effect of default and proposed lossy compression on retrieved phase for single cell sample. (a) no compression, (b) default compression with JPEG 2000 at 0.20 bpp, (c) proposed compression with JPEG 2000 at 0.20 bpp.
Figure 10 shows an example of rate-distortion plots that we obtained with both default and proposed methods for the microspheres sample.
Fig. 10. Rate-Distortion curves for the object Microsphere. (a) SNR under default compression vs bitrate (bpp), (b) RMSE under default compression vs bitrate (bpp), (c) SNR under proposed compression vs bitrate (bpp), (d) RMSE under proposed compression vs bitrate (bpp).
Lossy compression of time-lapse
In this section, two codecs that support lossy compression of video data have been evaluated (see Table 1), namely AVC and HEVC.
In both default and proposed compression HEVC Inter is the best performing codec when it comes to compression of hologram sequences. This is shown by both metrics provided in supplementary materials - BD-SNR in Data File 1 [54] and T-RMSE rate in Data File 2 [55]. In comparison to default mode, HEVC Inter in proposed mode produced files that were on average 7.34 times smaller and at best 12.63 times smaller. Those results are summarized in Table 4.
Table 4. Summary of the lossy compression results for time-lapses, assessed for the strongest compression that meets the RMSE target of 0.05 rad. Last column gives the improvement offered by the best performing codec under proposed compression (HEVC Inter) versus the best performing codec under default compression (also HEVC Inter).
Notably, all codecs improved when used with the proposed method, similarly to compression of static captures.
The results detailed in supplementary materials also show that when there is an option to compress holograms as a sequence, one should do so to obtain optimal results - both metrics (Data File 1 [54] and Data File 2 [55]) show better performance of HEVC Inter compared to JPEG 2000 (best Intra codec) operating separately on each time-lapse frame.
5.2 Lossless compression
5.2.1 Static captures
As mentioned in Section 3.2, the proposed lossless compression technique employs two codecs - a lossy codec operating in proposed mode and a lossless codec for compressing the residual signal in default mode. To simplify the analysis we assume that only the best codec to compress the residual signal needs to be determined, as the best performing lossy codec has already been found (JPEG 2000, see sec. 5.1). This justifies testing just 5 lossless codecs paired with JPEG 2000, instead of 35 combinations of lossless and lossy codecs.
Investigation of lossless static capture compression yields one clear winner (see Data File 3 [56]) - JPEG LS achieved best bitrates in 5 out of 6 cases with default compression (bested by lossless JPEG 2000 in microsphere hologram) and 6 out of 6 with proposed compression (ex aequo with PNG in one case). On average JPEG LS allowed for 1.14 times smaller files with proposed method compared with the best performing codecs with default mode.
Our tests revealed also that the overflow/underflow data that may be expected from the proposed lossless mode does not influence significantly the final compression performance - overflow/underflow bitrate was at worst three orders of magnitude smaller than the total bitrate of compressed files.
Table 5 summarizes the comparison between the modes, while detailed bitrates are available as supplementary materials.
Table 5. Summary of the lossless compression results for static captures. Last column gives the improvement offered by the best performing codec under proposed compression versus the best performing codec under default compression.
5.2.2 Lossless compression of time-lapse
Within evaluated codecs just two support video compression, namely AVC and HEVC. Both of them support either lossy and lossless encoding.
Evaluation of lossless video compression in default mode is tested with AVC and HEVC in both inter and intra modes and also with JPEG LS, which is the best performing codec for default lossless intra compression. This set of codecs shows that, similarly to lossy video compression, inter mode should be used to obtain best compression performance - AVC Inter provided best results in 3 out of 3 cases, matched by HEVC Inter in only one of those cases (see supplementary materials).
To evaluate the proposed mode we test the combinations of codecs presented in Table 6. Configurations C3 and C5 provide actual video compression with AVC and HEVC codecs, while configurations C1, C2 and C4 encode consecutive holograms separately posing as a reference to benefits of video compression. Configuration C1 was chosen as the best performing configuration in the single capture case, while C2 and C4 are AVC and HEVC working in intra mode.
Table 6. Codec configurations tested for proposed lossless compression of time-lapse.
The best results have been provided by configuration C3 in all three cases, again showing the benefits of video compression that leverages the temporal correlation of consecutive frames. C3 which is a proposed combination of lossy and lossless modes of AVC Inter codec outperformed in all cases the lossless AVC Inter working in default mode, providing files that are on average 1.09 times smaller.
The case by case comparison is given in Table 7 and detailed bitrates with the contributions of overflow/underflow data are available in Data File 3 [56].
Table 7. Summary of the lossless compression results for time-lapses. Last column gives the improvement offered by the best performing codec under proposed compression versus the best performing codec under default compression.
6. Conclusions
Rigorous metrological assessment of compression methods is a crucial aspect in the holographic QPI systems development towards the large-scale biomedical applications. We have proposed and demonstrated lossy and lossless compression methods using conventional image/video codecs for both single and video-like sequences of image plane off-axis holograms while exploiting the limited bandwidth of the first order term imposed by the imaging system. The assessment of lossy compression throughout this work, both proposed and conventional, was bound to maintaining the metrological accuracy of unwrapped phase at $0.05\textrm { rad}$ RMSE, which is a novel approach in holographic compression to the best of our knowledge. With this approach we address the practical problem of the whole complex amplitude distortion measures, that are insensitive to the unwrapping errors. Object dependence of compression performance has also been taken into account by investigating a variety of objects.
With proposed band-limitation lossy codec JPEG 2000 produces files that are between 2.3 to 8.5 times smaller than the files produced by direct application of the codecs (Table 2).
The proposed lossy method utilizing HEVC codec in video mode resulted in files that are between 3.8 and 12.63 times smaller compared to direct lossy video compression. It also performed better than compression of each hologram separately with JPEG 2000 in proposed mode.
Using proposed lossless compression with JPEG 2000 and JPEG LS codecs we achieve files that are between 1.02 to 1.25 times smaller than the files produced by conventional lossless codecs operating directly on the measurements.
We showed that using lossless video compression utilizing lossy and lossless capabilities of AVC in video mode provides 1.07 to 1.12 times smaller files compared with the direct lossless video compression (best direct compression acheived by HEVC). Comparison with JPEG 2000 and JPEG LS configuration (best performer in single hologram compression) shows that additional improvement has been reached by leveraging temporal redundancy present within the holograms sequence.
For lossless compression, the original measurements (+1,-1,0 order terms and noise) are preserved exactly. Therefore, we are unable to restrict the storage of the measurement to the spectral bandwidth of the +1 order term, like in the case of lossy compression. Additionally, the noise term by its nature is unpredictable and has high entropy, which increases the number of bits to preserve it fully. This disparity between lossy and lossless compression is not unique to holography and can be seen in the fields of image, video and audio compression as well.
This work establishes a new baseline for future research on compression codecs optimized for IP-OAH, as it provides an extensive and metrologically suitable assessment of rate-distortion performance for lossy compression and compression ratio performance for lossless compression, both for single holograms and video-like sequences of holograms.
Funding
Fundacja na rzecz Nauki Polskiej (TEAM TECH/2016-1/4); Narodowe Centrum Badań i Rozwoju (PL-TW/V/5/2018); European Social Fund (POWR.03.03.00-00-PN13/18).
Acknowledgments
The research leading to the described results was carried out within the program TEAM TECH/2016-1/4 of Foundation for Polish Science, co-financed by the European Union under the European Regional Development Fund, Polish-Taiwanese Joint Research Project (PL-TW/V/5/2018) co-financed by the European Social Fund within the framework of the Operational Programme Knowledge Education Development and “Projekt PROM - Miedzynarodowa wymiana stypendialna doktorantow i kadry akademickiej" (POWR.03.03.00-00-PN13/18), financed by the European Social Fund.
Disclosures
The authors declare no conflicts of interest.
References
1. M. K. Kim, “Principles and techniques of digital holographic microscopy,” SPIE Rev. 1(1), 018005 (2010). [CrossRef]
2. Y. Park, C. Depeursinge, and G. Popescu, “Quantitative phase imaging in biomedicine,” Nat. Photonics 12(10), 578–589 (2018). [CrossRef]
3. B. Kemper and G. Von Bally, “Digital holographic microscopy for live cell applications and technical inspection,” Appl. Opt. 47(4), A52–A61 (2008). [CrossRef]
4. M. Mikuła, T. Kozacki, M. Józwik, and J. Kostencka, “Accurate shape measurement of focusing microstructures in fourier digital holographic microscopy,” Appl. Opt. 57(1), A197–A204 (2018). [CrossRef]
5. B. Kemper, D. D. Carl, J. Schnekenburger, I. Bredebusch, M. Schäfer, W. Domschke, and G. von Bally, “Investigation of living pancreas tumor cells by digital holographic microscopy,” J. Biomed. Opt. 11(3), 034005 (2006). [CrossRef]
6. P. Lenz, D. Bettenworth, P. Krausewitz, M. Brückner, S. Ketelhut, G. von Bally, D. Domagk, and B. Kemper, “Digital holographic microscopy quantifies the degree of inflammation in experimental colitis,” Integrative Biol. 5(3), 624–630 (2013). [CrossRef]
7. Tec S. A. Lync, Lync Tec, https://www.lynceetec.com, (2020). [Online; accessed 02-January-2020].
8. Phase Holographic Imaging PHI AB, HoloMonitor, https://phiab.com/holomonitor/holomonitor-system/, (2020). [Online; accessed 21-February-2020].
9. HoloSea, http://4-deep.com/products/submersible-microscope/ (2020). [Online; accessed 21-February-2020].
10. D. Jin, R. Zhou, Z. Yaqoob, and P. T. So, “Tomographic phase microscopy: principles and applications in bioimaging,” J. Opt. Soc. Am. B 34(5), B64–B77 (2017). [CrossRef]
11. M. Kujawińska, W. Krauze, M. Baczewska, A. Kuś, and M. Ziemczonok, “Comparative study of laboratory and commercial limited-angle holographic tomography setups,” in Quantitative Phase Imaging V, vol. 10887 (International Society for Optics and Photonics, 2019), p. 1088708.
12. K. P. N.V., “Ultra Fast Scanner Specifications,” https://www.usa.philips.com/healthcare/product/HCNOCTN442/ultra-fast-scanner-digital-pathology-slide-scanner/specifications (2019). [Online; accessed 17-November-2019].
13. P. Stępień, D. Korbuszewski, and M. Kujawińska, “Digital holographic microscopy with extended field of view using tool for generic image stitching,” ETRI J. 41, 73–83 (2019). [CrossRef]
14. E. Kurbatova, P. Cheremkhin, N. Evtikhiev, V. Krasnov, and S. Starikov, “Methods of compression of digital holograms 4th International Conference of Photonics and Information Optics, PhIO 2015, 28-30 January 2015, Moscow, Russian Federation,” Phys. Procedia 73, 328–332 (2015). [CrossRef]
15. F. Dufaux, Y. Xing, B. Pesquet-Popescu, and P. Schelkens, “Compression of digital holographic data: an overview,” in Applications of Digital Image Processing XXXVIII, vol. 9599A. G. Tescher, ed., International Society for Optics and Photonics (SPIE, 2015), pp. 163–173.
16. G. A. Mills and I. Yamaguchi, “Effects of quantization in phase-shifting digital holography,” Appl. Opt. 44(7), 1216–1225 (2005). [CrossRef]
17. Y. Xing, B. Pesquet-Popescu, and F. Dufaux, “Comparative study of scalar and vector quantization on different phase-shifting digital holographic data representations,” in 2014 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), (2014), pp. 1–4.
18. Tomocube Inc., “Tomocube,” http://www.tomocube.com/, (2020). [Online; accessed 02-January-2020].
19. S. A. Nanolive, “Nanolive,” https://nanolive.ch, (2020). [Online; accessed 02-January-2020].
20. K. Jaferzadeh, S. Gholami, and I. Moon, “Lossless and lossy compression of quantitative phase images of red blood cells obtained by digital holographic imaging,” Appl. Opt. 55(36), 10409–10416 (2016). [CrossRef]
21. P. Langehanenberg, G. von Bally, and B. Kemper, “Autofocusing in digital holographic microscopy,” 3D Res. 2(1), 4 (2011). [CrossRef]
22. D. Blinder, T. Bruylants, H. Ottevaere, A. Munteanu, and P. Schelkens, “JPEG 2000-based compression of fringe patterns for digital holographic microscopy,” Opt. Eng. 53(12), 123102 (2014). [CrossRef]
23. P. A. Cheremkhin and E. A. Kurbatova, “Wavelet compression of off-axis digital holograms using real/imaginary and amplitude/phase parts,” Sci. Rep. 9(1), 7561 (2019). [CrossRef]
24. P. A. Cheremkhin and E. A. Kurbatova, “Quality of reconstruction of compressed off-axis digital holograms by frequency filtering and wavelets,” Appl. Opt. 57(1), A55–A64 (2018). [CrossRef]
25. N. N. Evtikhiev, E. A. Kurbatova, and P. A. Cheremkhin, “Coefficients quantization at off-axis digital hologram wavelet compression,” KnE Energy 3(3), 523–534 (2018). [CrossRef]
26. T. Colomb, F. Dürr, E. Cuche, P. Marquet, H. G. Limberger, R.-P. Salathé, and C. Depeursinge, “Polarization microscopy by use of digital holography: application to optical-fiber birefringence measurements,” Appl. Opt. 44(21), 4461–4469 (2005). [CrossRef]
27. A. Khmaladze, A. Restrepo-Martínez, M. Kim, R. Castañeda, and A. Blandón, “Simultaneous dual-wavelength reflection digital holography applied to the study of the porous coal samples,” Appl. Opt. 47(17), 3203–3210 (2008). [CrossRef]
28. M. Paturzo, P. Memmolo, A. Tulino, A. Finizio, and P. Ferraro, “Investigation of angular multiplexing and de-multiplexing of digital holograms recorded in microscope configuration,” Opt. Express 17(11), 8709–8718 (2009). [CrossRef]
29. M. Rubin, G. Dardikman, S. K. Mirsky, N. A. Turko, and N. T. Shaked, “Six-pack off-axis holography,” Opt. Lett. 42(22), 4611–4614 (2017). [CrossRef]
30. G. Dardikman, N. A. Turko, N. Nativ, S. K. Mirsky, and N. T. Shaked, “Optimal spatial bandwidth capacity in multiplexed off-axis holography for rapid quantitative phase reconstruction and visualization,” Opt. Express 25(26), 33400–33415 (2017). [CrossRef]
31. P. Stepień, R. K. Muhamad, M. Kujawińska, and P. Schelkens, “Hologram compression in quantitative phase imaging,” in Quantitative Phase Imaging VI, vol. 11249Y. Liu, G. Popescu, and Y. Park, eds., International Society for Optics and Photonics (SPIE, 2020), pp. 75–86.
32. M. Takeda, H. Ina, and S. Kobayashi, “Fourier-transform method of fringe-pattern analysis for computer-based topography and interferometry,” J. Opt. Soc. Am. 72(1), 156–160 (1982). [CrossRef]
33. E. Cuche, P. Marquet, and C. Depeursinge, “Spatial filtering for zero-order and twin-image elimination in digital off-axis holography,” Appl. Opt. 39(23), 4070–4075 (2000). [CrossRef]
34. C. Schretter, D. Blinder, S. Bettens, H. Ottevaere, and P. Schelkens, “Regularized non-convex image reconstruction in digital holographic microscopy,” Opt. Express 25(14), 16491–16508 (2017). [CrossRef]
35. N. Pavillon, C. S. Seelamantula, J. Kühn, M. Unser, and C. Depeursinge, “Suppression of the zero-order term in off-axis digital holography through nonlinear filtering,” Appl. Opt. 48(34), H186–H195 (2009). [CrossRef]
36. M. Cywińska, M. Trusiak, and K. Patorski, “Automatized fringe pattern preprocessing using unsupervised variational image decomposition,” Opt. Express 27(16), 22542–22562 (2019). [CrossRef]
37. M. Karray, P. Slangen, and P. Picart, “Comparison between digital fresnel holography and digital image-plane holography: the role of the imaging aperture,” Exp. Mech. 52(9), 1275–1286 (2012). [CrossRef]
38. E. Sánchez-Ortiga, A. Doblas, G. Saavedra, M. Martínez-Corral, and J. Garcia-Sucerquia, “Off-axis digital holographic microscopy: practical design parameters for operating at diffraction limit,” Appl. Opt. 53(10), 2058–2066 (2014). [CrossRef]
39. P. Picart and J. Leval, “General theoretical formulation of image formation in digital fresnel holography,” J. Opt. Soc. Am. A 25(7), 1744–1761 (2008). [CrossRef]
40. X. He, C. V. Nguyen, M. Pratap, Y. Zheng, Y. Wang, D. R. Nisbet, R. J. Williams, M. Rug, A. G. Maier, and W. M. Lee, “Automated fourier space region-recognition filtering for off-axis digital holographic microscopy,” Biomed. Opt. Express 7(8), 3111–3123 (2016). [CrossRef]
41. W. Xiao, Q. Wang, F. Pan, R. Cao, X. Wu, and L. Sun, “Adaptive frequency filtering based on convolutional neural networks in off-axis digital holographic microscopy,” Biomed. Opt. Express 10(4), 1613–1626 (2019). [CrossRef]
42. P. Memmolo, V. Renò, E. Stella, and P. Ferraro, “Adaptive and automatic diffraction order filtering by singular value decomposition in off-axis digital holographic microscopy,” Appl. Opt. 58(34), G155–G161 (2019). [CrossRef]
43. M. V. Bernardo, P. Fernandes, A. Arrifano, M. Antonini, E. Fonseca, P. T. Fiadeiro, A. M. Pinheiro, and M. Pereira, “Holographic representation: Hologram plane vs. object plane,” Signal Process. Image Commun. 68, 193–206 (2018). [CrossRef]
44. P. Cheremkhin and E. Kurbatova, “Wavelet compression of off-axis digital holograms using real/imaginary and amplitude/phase parts,” Sci. Rep. 9(1), 7561 (2019). [CrossRef]
45. P. St, R. K. Muhamad, D. Blinder, P. Schelkens, and M. Kujawika, “Function calls,” https://doi.org/10.6084/m9.figshare.12478784 (2020). [Online; accessed 13-June-2020]“.
46. D. Blinder, H. Ottevaere, A. Munteanu, and P. Schelkens, “Efficient multiscale phase unwrapping methodology with modulo wavelet transform,” Opt. Express 24(20), 23094–23108 (2016). [CrossRef]
47. J. M. Bioucas-Dias and G. Valadao, “Phase unwrapping via graph cuts,” IEEE Trans. on Image Process. 16(3), 698–709 (2007). [CrossRef]
48. M. Ziemczonok, A. Kuś, P. Wasylczyk, and M. Kujawińska, “3d-printed biological cell phantom for testing 3d quantitative phase imaging systems,” Sci. Rep. 9(1), 18872 (2019). [CrossRef]
49. N. Pandey and B. Hennelly, “Quantization noise and its reduction in lensless fourier digital holography,” Appl. Opt. 50(7), B58–B70 (2011). [CrossRef]
50. G. Bjontegaard, “Calculation of average PSNR differences between RD-curves,” ITU-VCEG (2001) (2001).
51. P. Marquet, B. Rappaz, P. J. Magistretti, E. Cuche, Y. Emery, T. Colomb, and C. Depeursinge, “Digital holographic microscopy: a noninvasive contrast imaging technique allowing quantitative visualization of living cells with subwavelength axial accuracy,” Opt. Lett. 30(5), 468–470 (2005). [CrossRef]
52. B. Kemper, P. Langehanenberg, and G. Von Bally, “Digital holographic microscopy: A new method for surface analysis and marker-free dynamic life cell imaging,” Optik & Photonik 2(2), 41–44 (2007). [CrossRef]
53. T. Colomb, J. Kühn, F. Charrière, C. Depeursinge, P. Marquet, and N. Aspert, “Total aberrations compensation in digital holographic microscopy with a reference conjugated hologram,” Opt. Express 14(10), 4300–4306 (2006). [CrossRef]
54. P. Stepeiń, R. K. Muhamad, D. Blinder, P. Schelkens, and M. Kujawińska, “BD-SNR in lossy compression,” https://osapublishing.figshare.com/s/24986ce9f8f99360e419, (2020). [Online; accessed 13-June-2020].
55. P. Stepeiń, R. K. Muhamad, D. Blinder, P. Schelkens, and M. Kujawińska, “Bitrate in lossy compression,” https://osapublishing.figshare.com/s/62453eabd59a9c8a5c4d (2020). [Online; accessed 13-June-2020].
56. P. Stepeiń, R. K. Muhamad, D. Blinder, P. Schelkens, and M. Kujawińska, “Bitrate in lossless compression,” https://osapublishing.figshare.com/s/7dbf19bb6826b81c6df2, (2020). [Online; accessed 13-June-2020].
57. P. Stepeiń, R. K. Muhamad, D. Blinder, P. Schelkens, and M. Kujawińska, “Spatial bandwidth-optimized compression of image plane off-axis holograms with image and video codecs - supplement preprint,” https://doi.org/10.6084/m9.figshare.12678764, (2020). [Online; accessed 20-June-2020].
