Abstract

A formulation of signal-to-noise ratio is constructed that uses temporal integrated images from image sequences. Given a blurred image that drifts horizontally at various speeds and at various linear blurs, we prove that this formulation of the signal-to-noise ratio consistently increases with an increase in speed. This increase is shown to model the trends in the human vision system by which drifting blurred images are perceived with increased sharpness. The existing widely used objective quality techniques fail to model the perceptual increase in sharpness. This new formulation, along with other objective quality measures, is tested on several blurred drifting image sequences. The new formulation reflects the theoretically predicted increase in perceived sharpness.

© 2000 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. H. de Ridder, “Current issues and new techniques in visual quality assessment,” in Proceedings of the IEEE International Conference on Image Processing, Lausanne, Switzerland, 16–19 Sept. 1996, Vol. 1, pp. 869–872 (1996).
  2. C. H. Graham, R. Margaria, “Area and the intensity-time relation in the peripheral retina,” Am. J. Physiol. 113, 299–305 (1935).
  3. H. B. Barlow, “Temporal and spatial summation in human vision at different background intensities,” J. Physiol. (London) 141, 337–350 (1958).
  4. D. C. Burr, “Motion smear,” Nature 284, 164–165 (1980).
    [CrossRef] [PubMed]
  5. P. J. Bex, G. K. Edgar, A. T. Smith, “Sharpening of drifting, blurred images,” Vision Res. 35, 2539–2546 (1995).
    [CrossRef] [PubMed]
  6. S. T. Hammett, “Motion blur and motion sharpening in the human visual system,” Vision Res. 37, 2505–2510 (1997).
    [CrossRef] [PubMed]
  7. S. T. Hammett, M. A. Georgeson, A. Gorea, “Motion blur and motion sharpening: temporal smear and local contrast non-linearity,” Vision Res. 38, 2099–2108 (1998).
    [CrossRef] [PubMed]
  8. J. H. D. M. Westerink, C. Teunissen, “Perceived sharpness in moving images,” in Human Vision and Electronic Imaging: Models, Methods, and Applications, J. P. Allebach, B. E. Rogowitz, eds., Proc. SPIE1249, 78–87 (1990).
  9. J. H. D. M. Westerink, K. Teunissen, “Perceived sharpness in complex moving images,” Displays 16, 89–97 (1995).
    [CrossRef]
  10. J. Jonides, D. Irwin, S. Yantis, “Integrating visual information from successive fixations,” Science 215, 192–194 (1982).
    [CrossRef] [PubMed]
  11. American National Standards Institute, Digital Transport of One-Way Video Signals—Parameters for Objective Performance Assessment, ANSI T1.801.03—1996 (Alliance for Telecommunications Industry Solutions, Washington, D.C., 1996.

1998

S. T. Hammett, M. A. Georgeson, A. Gorea, “Motion blur and motion sharpening: temporal smear and local contrast non-linearity,” Vision Res. 38, 2099–2108 (1998).
[CrossRef] [PubMed]

1997

S. T. Hammett, “Motion blur and motion sharpening in the human visual system,” Vision Res. 37, 2505–2510 (1997).
[CrossRef] [PubMed]

1995

J. H. D. M. Westerink, K. Teunissen, “Perceived sharpness in complex moving images,” Displays 16, 89–97 (1995).
[CrossRef]

P. J. Bex, G. K. Edgar, A. T. Smith, “Sharpening of drifting, blurred images,” Vision Res. 35, 2539–2546 (1995).
[CrossRef] [PubMed]

1982

J. Jonides, D. Irwin, S. Yantis, “Integrating visual information from successive fixations,” Science 215, 192–194 (1982).
[CrossRef] [PubMed]

1980

D. C. Burr, “Motion smear,” Nature 284, 164–165 (1980).
[CrossRef] [PubMed]

1958

H. B. Barlow, “Temporal and spatial summation in human vision at different background intensities,” J. Physiol. (London) 141, 337–350 (1958).

1935

C. H. Graham, R. Margaria, “Area and the intensity-time relation in the peripheral retina,” Am. J. Physiol. 113, 299–305 (1935).

Barlow, H. B.

H. B. Barlow, “Temporal and spatial summation in human vision at different background intensities,” J. Physiol. (London) 141, 337–350 (1958).

Bex, P. J.

P. J. Bex, G. K. Edgar, A. T. Smith, “Sharpening of drifting, blurred images,” Vision Res. 35, 2539–2546 (1995).
[CrossRef] [PubMed]

Burr, D. C.

D. C. Burr, “Motion smear,” Nature 284, 164–165 (1980).
[CrossRef] [PubMed]

de Ridder, H.

H. de Ridder, “Current issues and new techniques in visual quality assessment,” in Proceedings of the IEEE International Conference on Image Processing, Lausanne, Switzerland, 16–19 Sept. 1996, Vol. 1, pp. 869–872 (1996).

Edgar, G. K.

P. J. Bex, G. K. Edgar, A. T. Smith, “Sharpening of drifting, blurred images,” Vision Res. 35, 2539–2546 (1995).
[CrossRef] [PubMed]

Georgeson, M. A.

S. T. Hammett, M. A. Georgeson, A. Gorea, “Motion blur and motion sharpening: temporal smear and local contrast non-linearity,” Vision Res. 38, 2099–2108 (1998).
[CrossRef] [PubMed]

Gorea, A.

S. T. Hammett, M. A. Georgeson, A. Gorea, “Motion blur and motion sharpening: temporal smear and local contrast non-linearity,” Vision Res. 38, 2099–2108 (1998).
[CrossRef] [PubMed]

Graham, C. H.

C. H. Graham, R. Margaria, “Area and the intensity-time relation in the peripheral retina,” Am. J. Physiol. 113, 299–305 (1935).

Hammett, S. T.

S. T. Hammett, M. A. Georgeson, A. Gorea, “Motion blur and motion sharpening: temporal smear and local contrast non-linearity,” Vision Res. 38, 2099–2108 (1998).
[CrossRef] [PubMed]

S. T. Hammett, “Motion blur and motion sharpening in the human visual system,” Vision Res. 37, 2505–2510 (1997).
[CrossRef] [PubMed]

Irwin, D.

J. Jonides, D. Irwin, S. Yantis, “Integrating visual information from successive fixations,” Science 215, 192–194 (1982).
[CrossRef] [PubMed]

Jonides, J.

J. Jonides, D. Irwin, S. Yantis, “Integrating visual information from successive fixations,” Science 215, 192–194 (1982).
[CrossRef] [PubMed]

Margaria, R.

C. H. Graham, R. Margaria, “Area and the intensity-time relation in the peripheral retina,” Am. J. Physiol. 113, 299–305 (1935).

Smith, A. T.

P. J. Bex, G. K. Edgar, A. T. Smith, “Sharpening of drifting, blurred images,” Vision Res. 35, 2539–2546 (1995).
[CrossRef] [PubMed]

Teunissen, C.

J. H. D. M. Westerink, C. Teunissen, “Perceived sharpness in moving images,” in Human Vision and Electronic Imaging: Models, Methods, and Applications, J. P. Allebach, B. E. Rogowitz, eds., Proc. SPIE1249, 78–87 (1990).

Teunissen, K.

J. H. D. M. Westerink, K. Teunissen, “Perceived sharpness in complex moving images,” Displays 16, 89–97 (1995).
[CrossRef]

Westerink, J. H. D. M.

J. H. D. M. Westerink, K. Teunissen, “Perceived sharpness in complex moving images,” Displays 16, 89–97 (1995).
[CrossRef]

J. H. D. M. Westerink, C. Teunissen, “Perceived sharpness in moving images,” in Human Vision and Electronic Imaging: Models, Methods, and Applications, J. P. Allebach, B. E. Rogowitz, eds., Proc. SPIE1249, 78–87 (1990).

Yantis, S.

J. Jonides, D. Irwin, S. Yantis, “Integrating visual information from successive fixations,” Science 215, 192–194 (1982).
[CrossRef] [PubMed]

Am. J. Physiol.

C. H. Graham, R. Margaria, “Area and the intensity-time relation in the peripheral retina,” Am. J. Physiol. 113, 299–305 (1935).

Displays

J. H. D. M. Westerink, K. Teunissen, “Perceived sharpness in complex moving images,” Displays 16, 89–97 (1995).
[CrossRef]

J. Physiol. (London)

H. B. Barlow, “Temporal and spatial summation in human vision at different background intensities,” J. Physiol. (London) 141, 337–350 (1958).

Nature

D. C. Burr, “Motion smear,” Nature 284, 164–165 (1980).
[CrossRef] [PubMed]

Science

J. Jonides, D. Irwin, S. Yantis, “Integrating visual information from successive fixations,” Science 215, 192–194 (1982).
[CrossRef] [PubMed]

Vision Res.

P. J. Bex, G. K. Edgar, A. T. Smith, “Sharpening of drifting, blurred images,” Vision Res. 35, 2539–2546 (1995).
[CrossRef] [PubMed]

S. T. Hammett, “Motion blur and motion sharpening in the human visual system,” Vision Res. 37, 2505–2510 (1997).
[CrossRef] [PubMed]

S. T. Hammett, M. A. Georgeson, A. Gorea, “Motion blur and motion sharpening: temporal smear and local contrast non-linearity,” Vision Res. 38, 2099–2108 (1998).
[CrossRef] [PubMed]

Other

J. H. D. M. Westerink, C. Teunissen, “Perceived sharpness in moving images,” in Human Vision and Electronic Imaging: Models, Methods, and Applications, J. P. Allebach, B. E. Rogowitz, eds., Proc. SPIE1249, 78–87 (1990).

American National Standards Institute, Digital Transport of One-Way Video Signals—Parameters for Objective Performance Assessment, ANSI T1.801.03—1996 (Alliance for Telecommunications Industry Solutions, Washington, D.C., 1996.

H. de Ridder, “Current issues and new techniques in visual quality assessment,” in Proceedings of the IEEE International Conference on Image Processing, Lausanne, Switzerland, 16–19 Sept. 1996, Vol. 1, pp. 869–872 (1996).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1
Fig. 1

Temporal-constrained system is illustrated. No matter how many input frames are given to the system, frame integration is used to force 10-fps processing. The top half shows 3 frames at 10-fps input and 3 quality measures output. The bottom half shows 9 frames at 30-fps input and still 3 quality measures output.

Fig. 2
Fig. 2

This shows a single frame from each of the 5 image sequences used to test the objective metrics. In the experiment the portrait rocks back and forth at various speeds in the rectangular black box. The top image is the original. The other frames are linearly blurred by 2, 4, 5, and 6 pixels.

Fig. 3
Fig. 3

For image sequences of varying quality, the PSNR does not reflect psychophysical evidence. PSNR shows only constant SNR for increases in the speed across the field of view.

Fig. 4
Fig. 4

For image sequences of varying quality, ŝ does not reflect psychophysical evidence. ŝ is constant for increases in the speed across the field of view.

Fig. 5
Fig. 5

Assuming a drifting portrait, degrees and degrees per second is illustrated. In the top illustration, θ1 is the degrees that the portrait spans given that the eye is a distance d from the display. The middle illustration shows θ2 > θ1 for a drifting portrait. θ2 divided by time yields degrees per second. The bottom illustrates that a faster frame rate means an increase in frames and thus an increase in θ3 and an increase in degrees per second.

Fig. 6
Fig. 6

Picture sharpness for varying speeds of a drifting portrait as reported in psychophysical tests performed by Westerink and Teunissen.8,9

Fig. 7
Fig. 7

Picture sharpness as obtained by repeating the psychophysical experiment at the Air Force Research Laboratory by use of the drifting portrait in Fig. 2. For increased picture speed, the blurred images show a trend toward sharpening, whereas the sharper image shows a trend toward blurring.

Fig. 8
Fig. 8

PSNRI agrees with psychophysical evidence8 that shows the trend to increase signal-to-noise ratio while highly degraded images increase in speed across the field of view.

Fig. 9
Fig. 9

PSNRI can be normalized and combined with a linear blur function to allow for a more complete model that accounts for both the sharpening and the separate blurring mechanisms7 found in the psychophysical evidence.

Equations (43)

Equations on this page are rendered with MathJax. Learn more.

eki, j¯=1Kk=1K eki, j=1Kk=1Kfki, j-gki, j.
ek+1i, j=eki, j-δ.
MSIE=i=1Mj=1Nk=1K eki, jK2MN.
PSNRI=20 log10peak value of fki, jMSIE.
MSE=i=1Mj=1N eki, j2MN.
MSIEMSE.
1Kk=12 eki, j2=1Ke1i, j+e2i, j2=1Ke1i, j+e1i, j-12.
MSIE=14MNi=1Me1i, 12+e1i, 2+e1i, 12+e1i, N-3+e1i, N-22+e1i, N-22.
MSIE=12MNi=1Mj=1Ne1i, j2+e1i, j+1e1i, j=12MNi=1Mj=1N-1e1i, j2+e1i, j+1e1i, j.
MSIE=12MNi=1Mj=1N-1e1i, j2+e1i, j+1e1i, j1MNi=1Mj=1N-1 e1i, j21MNi=1Mj=1N e1i, j2=MSE.
i=1Mj=1N-1e1i, je1i, j+1i=1Mj=1N-1 e1i, j2.
a, ba b=a, a.
j=1N-1e1i, je1i, j+1j=1N-1 e1i, j2.
1MSIE1MSE.
PSNR=20 log10peak value of f1i, jMSE, PSNRI=20 log10peak value of fki, jMSIE
peak value of f1i, j=peak value of fki, j,
gki, j=fki, j+fki, j+12
hki, j=gki, j+gki, j+12.
MSEgkMSEhk for k=1,, K.
MSEgk=1MN¯ijfki, j+1-fki, j22,
MSEgk=12MN¯ijfki, j2-fki, jfki, j+1.
MSEhk=116MN¯ijfki, j+2+2fki, j+1-3fki, j2.
MSEhk=116MN¯ij14fki, j2-8fki, jfki, j-6fki, jfki, j+2.
MSEhk-MSEgk=38MN¯ijfki, j2-fki, jfki, j+2.
X=fki, 1fki, 2  fki, N0 0,
Y=0 0 fki, 1fki, 2  fki, N.
ij fki, jfki, j+2=X, YX Y=j fki, j2.
MSEgkMSEhk.
gki, j=fki, j+fki, j+12.
MSIEgk<MSEgk.
MSIE=1MN¯ijk=12 eki, j22,
eki, j=fki, j+1-fki, j2.
fk+1i, j+1=fki, j,
MSIE=116MN¯ijf1i, j+1-f1i, j+f2i, j+1-f2i, j2.
MSIE=18MN¯ijf1i, j2-f1i, jf1i, j+2.
f1i, j-f1i, j+22=f1i, j-f1i, j+1+f1i, j+1-f1i, j+22f1i, j-f1i, j+12+f1i, j+1-f1i, j+22.
-f1i, jf1i, j+2f1i, j+12-f1i, jf1i, j+1-f1i, j+1f1i, j+2.
MSIE18MN¯ijf1i, j2+f1i, j+12-f1i, jfi, j+1-f1i, j+1f1i, j+214MN¯ijf1i, j2-f1i, jf1i, j+1<12MN¯ijf1i, j2-f1i, jf1i, j+1=MSE.
sˆ=4.77-0.922m1-0.272m2-0.356m3.
m1=RMStime5.81SIfk-SIgkSIfk,
m2=ftime(0.108 MAXTIfk-TIgk, 0),
m3=MAXtime4.23 log10TIgkTIfk, 0,
ftime=STDtimeCONVxi, -1, 2, -1.

Metrics