1. Introduction

The increasing use of numerical imaging technologies in everyday life has increased the needs for image storage solutions. Materialized storage media such as optical disks, hard disk drives and flash memory or dematerialized solutions such as cloud computing offer valuable answers for storing and sharing huge quantities of data. Although these solutions are satisfactory for general consumer needs, they give no guarantee regarding information access durability.

As pointed out in various studies [1, 2], the availability of digital documents may be jeopardized for several reasons. First of all are media and file format issues. Rapidly developing storage media or customer practices can supersede well established media, which then become obsolete in a rapid, non predictive way. File format is another shortcoming for data durability as the technology of file formats and structures is always subject to improvement. These long term global technological hurdles are often underestimated as the principal customer storage concern is generally that of digital media physical stability.

Digital data, virtually stored as succession of 0/1 bits is recorded as variations of thresholded physical parameters: local changes of surface topologies or refractive index in optical disks, variation of local magnetic polarity in hard disks or coupling charge capacitance in flash memory. Digital information management is built on inherently analogue physical phenomena and is therefore sensitive to ageing. This presents an inherent additional shortcoming linked to its sequential approach: small-scale media degradation can lead to global data loss. A scratched CD doesn’t lead to a scratched photo, it just doesn’t work.

The concern of long term data archiving is particularly sensitive for institutions that have a statutory obligation to store data over long time periods, such as medical or legal institutions. In these cases permanent data migration is often the only viable solution despite the inherent cost and energy consumption.

Analogue archiving media such as microforms offer a good solution to data access durability concerns in the case of graphical file format preservation. The human readable approach releases the constraint of file and media format. Moreover bidimensional graphical representation of data is less sensitive to global data loss and allows progressive, detectable degradation of the media. Storage capacity is one of the main drawbacks of this solution and generally excludes it for cost effectiveness reasons. To release this capacity constraint due to poor information density, a modern approach of human readable media based on up to date lithographic technologies has been proposed [3, 4].

In order to facilitate data access and optimize capacity, an interesting approach is to store additional information inside the analogue data recorded in the media. Metadata are currently an important part of photograph information (date, GPS coordinates …) and have to be archived in the same way. Data access time is another drawback of human readable media, hiding information in the halftone images could help rapid location of images within the media. Another approach to speed up data access time and facilitate the use of the media could be to embed a compressed digital image representation in the raw halftone image. Finally, as the main objective of data archiving is the confidence in the graphical content of data, it may be interesting to hide security information inside the halftone recorded images as an authentication guarantee.

The watermarking of halftone images at a microscopic scale is presented in this paper as a solution to the aforementioned archiving issues. Digital watermarking in halftone images has been widely studied for authentication purposes. It concerns generally frequency modulation halftoning strategies for digital or printed image manipulation [5, 6]. We propose here the application of complementary computer generated holography [7] to introduce information in amplitude modulated halftone images with very low image visual degradation and good diffraction efficiency.

Firstly, we explain in section 2 the general principle of halftone image watermarking with the use of computer generated holograms. In section 3 we focus on the relationship between hologram efficiency and halftoning for the case of elliptic amplitude modulation. Finally, section 4 is devoted to experimental results, followed by the conclusion.

2. Principle of halftone holographic watermarking process

2.1. Computer generated holography

The principle of computer generated holography (CGH) was first introduced by Lohmann et al. at the end of the sixties [8]. It consists of designing a holographic structure that generates an image by diffraction. Since the first significant paper of Brown and Lohmann the concept of CGH has been widely studied and applied in various forms. Recently we have discussed a method of watermarking images based on the principle of complementary holography. In this work the hologram is embedded in a piece of binary graphic art, we extend our concept to gray tone images through halftone watermarking.

The hologram is calculated from the Fourier Transform of a binary computer generated image I^h. This gives the amplitude A and the phase α of the wave front to be encoded as shown on Fig. 1(a):

 

Fig. 1 (a) Principle of the coding of an image by the cell oriented computer generated holography. (b) Description of the cell geometry (d) Picture of a cell oriented CGH with period 5 µm.

Download Full Size | PDF

Where n and m are the pixel coordinates.

We use the original formulation of a Lohmann type hologram with elliptical diffraction patterns [7]. The phase is coded by the offset of the pattern location relative to the address grid and the amplitude by the size wx and wy of the elliptical pattern as shown in Fig. 1(b). Parameters d, wx and wy are normalized with respect to the grid period Λ. Amplitude and phase are expressed as follows:

For hologram manufacturing we use a direct laser writing system derived from a Blu Ray mastering system [9]. A phase transition metal sputtered on a glass wafer is used in a thermal lithography process.

The contrast between the wafer and the metallic layer transmission allows the generation of the amplitude hologram structure. Figure 1c gives a typical microscopic view of a CGH structure (reflective metallic layer in gray, transparent wafer in black). A typical metallic layer thickness of 40 nm is used in our experiment with a transmission coefficient of about 20%.

CGH structures are generally designed in literature with rectangular diffraction patterns. Our choice to use elliptical pattern, introduced in our previous paper, was first motivated by researches on grayscale halftoning process, as halftone pattern are generally made of round dots. As compared to rectangular patterns, elliptical patterns are interesting in halftoning process as they give a large number of gray tones and a better visual rendering.

2.2. Amplitude modulation halftoning

Haftoning is the process that consists of the binary representation of gray tone images. It has been widely studied over the XXth century from the first photography printed in an 1873 newspaper to the latest digital printer technologies [10]. The principle of halftoning is based on the human visual spatial frequency response. Our visual system acts as a low pass filter that shades the binary structure of the halftone image: if the binary halftone pattern is correctly scaled, the image renders a gray tone visual aspect.

Two main halftoning techniques have been developed: frequency modulation, where the gray tone is coded by the repetition frequency of an elementary pattern and amplitude modulation, where it is the size of the pattern that codes the gray tone.

Our work is based on the latter modulation scheme. The gray tone associated with each image pixel is rendered by a change in the halftone pattern size. In a general case the pattern is a round dot with a normalized diameter w directly given by the gray tone image I^g:

Gray tone rendering accuracy is given by the ability to tune precisely the halftone pattern size. To do so we can take advantage of the very small address grid allowed by modern maskless lithographic systems. Typical address grids are around a few tens of nanometers, well below the diffraction limit of the optical writing beam that fix the minimum pattern size to around few hundreds of nanometers.

Figures 2(a) and Fig. 2(b) compare a gray tone image and the result of the halftoning process with amplitude modulation. In this case we have used a circular pattern starting from a minimum pattern size of 800 nm in a pattern grid distribution of period Λ = 4 µm. A theoretical address grid of 100 nm is used.

 

Fig. 2 (a) Detail of the gray tone reference image Ig. (b) Result of a usual amplitude modulation halftoning. (c) Result of halftoning with complementary CGH watermarking. (d) result of halftoning with previous CGH watermarking process.

Download Full Size | PDF

The halftone cell is similar to the holographic cell except for the offset, not introduced in the pattern location, and the pattern size that codes the incoherent reflected light intensity rather than the coherent incoming wave amplitude. Rosen and Javidi have proposed a combination of both concepts of amplitude modulation halftoning and computer generated holography to encode an image in a halftone picture for security applications [11]. We analyze here how this concept can be improved by the use of complementary holography.

2.3. Holographic watermarking

As given by Eq. (3), coding the hologram phase is achieved by lateral deviations of the diffractive patterns positions. To improve this effect it is useful to consider elliptical pattern with a lateral diameter wx less than the longitudinal diameter wy. The gray tone associated to the pattern is expressed as follows:

With $\rho =\frac{wy}{wx}\ge 1$.

Equations (4) and (5) pose a problem at both extremities of the gray tone range where Γ_nm deviates from I^g_nm. On the one side, the minimum pattern size prevents coding some of the lower values and on the other side, the halftone pattern can exceed the cell dimension and interfere with neighboring cells.

To avoid the latter effect we limit the longitudinal size of the elliptical pattern to the cell size so that wy ≤ 1. The ratio ρ is kept constant while the pattern surface area increases, but is forced to decrease when wy reaches its maximum. This choice limits the maximum gray tone to Γ_max = π/4 ~0.8. The image gray tone dynamic has then to be limited in order to use Eq. (5) to define the halftone pattern distribution.

This definition fixes the amplitude of the embedded hologram following Eq. (2). The hologram amplitude A^h given by the gray tone image I^g can then be expressed by:

Figure 3 shows the evolution of the hologram amplitude for various values of the elliptical factor ρ. Observation of the figure leads to the following remarks:

 

Fig. 3 Evolution of the hologram diffraction pattern amplitude as a function of the gray tone associated to the half tone pattern with various elliptical factors ρ.

Download Full Size | PDF

The last remark leads us to consider limiting the range of Eq. (6) to [0:0.5] and to use the concept of complementary holography to cover the whole gray tone range [0:1].

For this purpose, we consider a folded image I^f determined as follows:

As this image is gray tone limited to the range [0:0.5], to recover the gray tone range of I^g, a thresholded image I^t is also considered:

During the halftoning writing process, I^t fixes the polarity of the cells, a value of zero leads to an opaque cell with an open aperture and a value of one leads to a transparent cell with an opaque aperture.

To improve the visual rendering of the halftone structure without compromising the pattern diffraction amplitude, we have to find a compromise between the usual round dot pattern and the highly elliptical hologram pattern. We choose ρ = π/2 ~1.5 so that Eq. (6) and (7) lead to the simple expression:

Hologram phase is modified following the concept of complementary holography based on the Babinet principle. A π phase offset is added to the hologram phase distribution following a mapping defined by I^t.

2.4. Hologram phase determination

The watermarking of image I^g via the binary holographic image I^h may disturb the hologram recovery as its amplitude is fixed by I^g independently of I^h. To correct this perturbation we use a phase retrieval process according to the work of Fienup [12].

The whole design process is described in Fig. 4. Firstly, hologram phase and amplitude are calculated using a Fast Fourier Transform of I^h (step 1). The gray tone image I^g is then separated in two components: the thresholded Image I^t and the final hologram amplitude A^h (step 2). This amplitude replaces the calculated hologram amplitude and the hologram function is inverse Fourier Transformed to form an estimate of the hologram recovery and a phase distribution φ (step 3). This phase is added to the initial amplitude and an iteration process of step 1 and 3 is begun. The squared error between the hologram image source and the estimate gives a measure of the phase iteration process efficiency. When the resulting error is stabilized, the iteration is stopped. The thresholded image I^t, the amplitude A^h and the resulting phase α are merged to produce the watermarked halftone image (step 4).

 

Fig. 4 Description of the phase retrieval process used to design a halftone image with complementary CGH watermarking.

Download Full Size | PDF

Figure 5(a) shows the evolution of the mean squared error of the phase iteration process. The error decreases strongly from the first iteration and the visual aspect of the recovered holographic image estimate shows a satisfying result after only two iterations.

 

Fig. 5 (a) Evolution of the mean squared error between the original holographic image and the estimation of the holographic image recovery. (b) Holographic image recovered at first iteration (detail of the image given below). (c) Holographic image recovered after two iterations. (d) Holographic image recovered after 200 iterations.

Download Full Size | PDF

Figures 5(b)-5(d) show the recovery result after 1, 2 and 200 iterations.

Figures 2(c) and 2(d) give a comparison between the visual aspect of the gray tone reference image watermarked by complementary CGH and by the previous CGH method. In the latter case, the background color is uniform (black in the example) and phase shifts around 2π generate noticeable black dots in the bright regions of the image. In the case of complementary CGH, the background is locally adapted to the mean gray tone so that high phase shifts generate black and white dots in dark and clear regions of the image, with limited visual impact. Moreover, due to the gray tone range limitation, the second watermarking technique leads to a low contrast image. As a result, it appears that our process offers a better visual aspect of the image in terms of noise and contrast.

This advantage should not be balanced by a degradation of hologram recovery quality. Estimation of this characteristic is generally hard to achieve as it is related to numerous quality parameters [13]. In the next section we analyze one of these parameters, the hologram efficiency, with a particular focus, theoretical and experimental, on the relation between halftone amplitude and hologram diffraction. Then in last section the recovered hologram image and halftone image quality are analyzed experimentally.

3. Holographic efficiency over halftoning

3.1. Theory

The hologram function can be described by a set of diffractive apertures located in a bi-dimensional Λ periodic cell distribution [5]:

Where (x_h,y_h) are the coordinates in the hologram plane and (ν_x, ν_y) the spatial frequencies.

In our case we consider an elliptical pattern of transmission t₁ surrounded by a cell of transmission t₂. The cell function C_nm can be expressed with a pattern size dependant disk function D and a normalized rectangular function R:

With the disk and rectangle functions expressed as follows:

For the sake of simplicity we don't consider the offset term d_nm and we calculate the diffraction efficiency of a bi-dimensional grating, similar to the hologram structure. Equation (10) gives the intensity diffracted by the grating:

The ideal case of a maximum contrast between the cell and the aperture (t1 = 0 and t2 = 1) gives, respectively, the following diffraction efficiencies for zero and first orders, in the case of a medium surface cell occupation (Γ = 0.5) and ρ = π/2: η_o,o = 0.25 and η_1,0 = (J1(2)/2)² ~0.08.

This can be compared to the diffraction efficiency of a typical Lohmann type hologram with rectangular pattern size wx and wy. Equation (17) gives the diffraction efficiency in the first order for such a 2D grating. With a 50% duty cycle (Γ = 0.5): wy = 1 and wx = 0.5, a maximum value η'_1,0 = 1/π² ~0.10 is found.

Figure 6 compares the theoretical diffraction efficiency of various elliptical 2D gratings with diffraction pattern sizes given by the gray tone image I^g, for various elliptical factors ρ. The use of complementary holography allows the symmetry of the gray tone range that now covers the whole [0:1] domain. As previously stated, the diffraction efficiency is improved for high ρ values.

 

Fig. 6 Diffraction efficiency in the first order for uniform 2D gratings using complementary holography design as a function of the gray tone associated to the half tone/diffractive pattern for various elliptical factor ρ, dotted lines show the diffraction efficiency for rectangular pattern gratings with associated rectangular factor.

Download Full Size | PDF

The diffraction efficiency is also given in the case of rectangular patterns (associated dotted lines). We notice slightly better diffraction efficiency for elliptical patterns up to approximately ρ = 1.5.

3.2. Measurement

In order to measure easily the diffraction efficiency of the hologram, we consider a simple hologram image I^h in the form of a dark circle embedded in a bright disc. The disc distribution allows an easy measurement of the confined energy. The circle is used to adjust the sensor location at the Fourier lens focus. The inset of Fig. 7 gives a representation of the binary image I^h. A 670 nm wavelength laser, collimated with a spatial filter beam expander, generates the hologram diffraction. The diffracted signal is imaged by a Fourier lens onto a photo detector that measures the diffracted energy. Measurements are normalized by the energy measured through a blank glass wafer.

 

Fig. 7 Experimental results of the first order diffraction efficiency for various uniform gray tone gratings for open and opaque diffraction patterns (resp. open and filled circles), theoretical diffraction efficiency is given for t₁ - t₂ = 0.55 (solid line) and for a +/− 10% error in the transmission value (broken lines).

Download Full Size | PDF

Various holograms are generated with uniform gray tone distributions I^g ranging from 0 to 0.8. The holograms are written with a period of 5 µm and a scale factor, ρ = 1.5 on a 40 nm thick phase transition metal layer. Two complementary polarity configurations are considered: open aperture in an opaque cell or opaque aperture in a transparent cell.

The hologram gray tone equivalent Γ is calculated by statistical microscope measurements of the pattern/cell surface ratio. Due to erosion effects of the laser writing process, higher than expected gray tone values are observed, which explains the maximum value greater than 0.8.

Results of the experiments are plotted in Fig. 7. Solid and broken lines show respectively the theoretical diffraction efficiency in the first order with t₁ = 1 and t₂ = 0.45 with an error +/−10% in t₂. Results are in accordance with the metallic layer transmission t₂² = 20%. As expected, both cell polarities show similar diffraction behavior.

The evolution of the efficiency as a function of ρ in Eq. (16) is evaluated experimentally for a set of constant amplitude holograms. The gray tone is fixed at Γ = 0.4, the elliptical pattern diameter is calculated for various elliptical factors ρ from Eq. (5). We measure the diffraction efficiency and the effective ρ value and plot data in Fig. 8. The results are in good agreement with the theoretical values indicated by the solid line and confirm the increase of diffraction efficiency with the elliptic shape factor.

 

Fig. 8 Experimental results of the first order diffraction efficiency for 0.4 uniform gray tone gratings for various elliptical factors ρ, the solid line shows the theoretical diffraction efficiency for t₁ - t₂ = 0.55.

Download Full Size | PDF

Figure 7 shows that the use of complementary holography for analogical watermarking gives similar results in terms of diffraction efficiency as compared to the previous holographic watermarking technique. A single cell polarity or a dual cell polarity hologram will result in the same diffraction efficiency.

The choice of an elliptical diffraction pattern gives a small advantage in terms of diffraction efficiency as compared to a rectangular profile when the form factor ρ is below 1.5. We examine next the hologram visual aspect and holographic recovery quality.

4. Experimental results

4.1. Image visual aspect

The watermarking process given in Fig. 4 is used to embed a QR code data matrix in the 1000x1000 pixel gray tone image I^g shown in Fig. 9(a). We code a 229 character text from the biography of Jean-Baptiste Fourier in a 69x69 Qr code. An additional text is added to the 2D barcode to provide a direct visual interpretation of the hologram recovery quality. The holographic image I^h is scaled to the gray tone image dimensions. Holograms are designed with a period 4 µm and an elliptical factor ρ = 1.5.

 

Fig. 9 (a) Reference gray tone image. (b), (c) and (d) Microscopic view of the halftone image with complementary CGH watermarking with magnification range x5, x20 and x50.

Download Full Size | PDF

Figures 9(b)-9(d) show microscopic views of the watermarked halftone picture at scale x5, x20 and x50. The correspondence between Fig. 9(a) and 9(b) is degraded by the erosion effect due to the laser writing process and due to the noise added by the phase distribution of the halftone pattern. The former artifact can be controlled by a rigorous calibration of the manufacturing process and attenuated by digital image post-processing. The latter artifact is inherent to the watermarking process, but should be reduced by digital image pre-processing or by improved CGH models [14]. Figures 9(c) and 9(d) show a diffractive pattern structure very similar to the simulations given in Fig. 2(c).

4.2 Hologram recovery

We evaluate the hologram recovery by comparing various holograms generation modes. In each case we add to the hologram phase distribution a 60 mm focal length Fourier Lens phase function so that the recovery set-up only consists of a laser and a camera.

The first design is the result of the process described in Fig. 4 with 300 iterations.

We then generate two holograms with a random phase mask added to the holographic image I^h as described in our previous paper [5], i.e. without a phase retrieval process. The hologram amplitude of the second and third designs are calculated respectively by the holographic image I^h and by a 0.5 uniform gray tone image

Figures 10(a)-10(c) show the visual result measured with our camera for the three designs. A zoom is given beneath each image.

 

Fig. 10 (a) Experimentally recovered holographic image in the case of a phase retrieval process based on image I^g with 300 iterations. (b) and (c) Experimentally recovered holographic image in the case a random phase mask and hologram amplitude given by I^g or by a 0.5 uniform gray tone.

Download Full Size | PDF

QR code reader software is used to test the 2D barcode recovery. The text is decoded in all three cases with similar decoding parameters so that none of the three designs can be considered better than the other. The previous holographic watermarking technique, as shown in Fig. 2(d), also gives good visual and QR decoding results for rectangular or elliptical patterns.

Our experiments clearly confirm that the amplitude has very much a secondary role in the hologram recovery process. They also show that the phase retrieval process is not necessary for the hologram recovery and that a simple random phase mask is sufficient in the hologram design process.

5. Conclusion

We have presented an original approach to gray tone image watermarking. Our concept is based on complementary holography and offers a valuable improvement as compared to the previous analogical watermarking method when the visual aspect of the gray tone image matters as in the case of analogical archiving. The use of elliptical diffraction patterns has been described theoretically and validated experimentally. This allows a better visual rendering of the watermarked image and slightly better diffraction efficiency for a moderate elliptical factor as compared to usual rectangular shape diffraction patterns.

Applications of this watermarking concept may include the embedding of metadata in gray tone image archiving to improve the surface capacity of analogue archiving media, such as that commercialized by our industrial partner Arnano.