Abstract

In a multiview video plus depth (MVD) based three-dimensional (3D) video system, the generation of the contents with simultaneous resolution and depth adjustments is very challenging. In this paper, we have presented a Multiview Video plus Depth ReTargeting (MVDRT) technique for stereoscopic 3D (S3D) displays. The main motivation of this work is to optimize the resolution and depth of original MVD data so that it is suitable for view synthesis. Our method takes shape preservation, line bending and visual comfort constraints into account, and simultaneously optimizes the horizontal, vertical and depth coordinates in display space. The retargeted MVD data is used to generate the contents for S3D displays. Experimental results demonstrate our method can achieve a better view synthesis performance than other approaches that still preserve the original depth information after retargeting, leading to promising S3D experience.

© 2017 Optical Society of America

1. Introduction

Due to the creation of immersive realistic experiences with depth perception, great variety of developments in 3D display technologies have been occurred in the last years. However, even immersive realistic experiences can be provided by 3D displays, the main challenge to hinder its extensive applications is how to provide satisfactory Quality of Experience (QoE) for users [1]. Therefore, simply extensions of the existing 2D techniques (e.g., image retargeting) to 3D applications usually fail to deliver a satisfactory QoE for users, because the QoE models in 2D and 3D semantics are different [2]. Although some promising holographic, integral imaging and Super MultiView displays [3] are being developed to offer a high QoE, the essence of QoE optimization considered in this work is to eliminate intrinsic content limitations (e.g., accommodation-vergence conflict [4,5]) by taking advantage of new means.

With the diversity of various stereoscopic 3D (S3D) display devices, such as phones, table PCs and TVs, similar to 2D media, 3D media also requires content adaptation to different display devices [6]. In addition to adapting to the device spatial resolution (retargeting along the x and y directions), for S3D displays, depth adaptation is especially critical to adapt the comfort zone of different displays [7], otherwise, undesirable visual discomfort may be occurred. Particularly, the depth ranges should be adjustable on different S3D displays to avoid accommodation-vergence conflicts of the scene. Without a specific explanation, S3D displays in this work refer to binocular 3D displays. According to the accessibility of depth information, sources for S3D displays mainly originate from two aspects, i.e., fixed-viewpoint stereoscopic images and unfixed-viewpoint multiview video plus depth (MVD) data [8]. Currently, many works have been focused on designing effective coding methods for MVD data [9–12]. To manipulate the depth according to different sources, in the first aspect, depth adaptation is achieved by changing the disparity/depth ranges of stereoscopic images [13,14]. In the second aspect, depth map is adjusted according to specific display requirements, and is aided to generate stereoscopic images by depth-image-based rendering (DIBR) technique [15]. However, these methods still rest on the original image resolution to adjust the depth information, but if the display device is changed, it is desirable to control the resolution and depth simultaneously for S3D displays. Since existing depth adaptation methods cannot retarget the size of stereoscopic image, the advantage of this work is to generate resolution and depth adjustable stereoscopic image pairs from MVD data by retargeting.

This work focuses on optimizing the resolution and depth of MVD data to adapt to the comfort zone of the display collecting camera parameters of virtual camera for view synthesis, which are not presented in fixed-viewpoint stereoscopic image system. This paper provides new insights on how the selected virtual camera affects the retargeting process and further affects the 3D perception of view synthesis. A novel method, the Multiview Video plus Depth ReTargeting (MVDRT), aiming at providing a solution to optimize the factors, such as image resolution, depth range, and the perceived visual comfort. Previous methods only consider fixed-viewpoint depth remapping or content retargeting, while the MVDRT utilizes the parameters of virtual camera and viewing conditions to optimize the 3D perception of view synthesis. This optimization provides a very valuable tool to guide the retargeting for MVD data. The major advantages of the MVDRT are as follows:

  • 1) The optimization is performed in display space so that the viewing conditions (including display device and viewing distance) are propagated to the warping process. Thus, it aggregates the contributions of different parameters that influence the 3D perception of view synthesis. Also, the MVDRT takes camera parameters of virtual camera into account.
  • 2) Besides considering the influence of various factors on view synthesis, the perceived visual comfort of the virtual view (influenced by depth range and consistent deformation of the reference views) are particularly designed to address the viewing experience in a more natural way. With these adjustable parameters, the MVDRT has the flexibility for S3D displays.
  • 3) We simultaneously adjust the horizontal, vertical and depth coordinates in a uniform optimization framework. To obtain the retargeted depth map, we use a five-parameter fitting function to remap the original depth range to the target one. It has been validated that the depth remapping step has better performance in view synthesis than previous approaches.

The rest of the paper is organized as follows: we first review the related work in Section 2, detail our method in Section 3, and present results in Section 4 and discussion in Section 5.

2. Related work

3D media retargeting. The problem of 2D image/video retargeting has received much attention over the recent years, followed the classification of discrete and continuous methods [16–18]. Towards retargeting for 3D media, some approaches have been proposed. Seam carving technique is to determine the seams for one view of the stereoscopic image, and find the corresponding seams in another view based on the correspondence constraint as well as the spatial coherence [19–21]. However, seam caving for stereoscopic image will cause shape distortion in or between left and right images. Also, due to depth preservation essence of the seam carving methods, their depth adaptation on different display devices is comparatively poor.

For continuous methods, warping-based retargeting is performed to solve mesh deformation problem with several deformation constraints. The key of this type of methods is to uniformly distribute more distortion on those less important regions and less distortion on more important regions. Chang et al. [22] presented a content-aware method for editing stereoscopic image to different displays, which optimizes energy function based on sparse stereo correspondences and disparity consistency constraints. Lin et al. [23] proposed an object-coherence warping method for content-aware stereoscopic image retargeting, in which the object correspondences between the left and right images are utilized to generate an object-based significance map and preserve object consistency during warping. Li et al. [24] presented a warping-based stereo retargeting framework by imposing both shape preservation and depth preservation constraints, in which region-wise depth-preserving constraints are derived to control the local warping functions for depth preservation. Lee et al. [25] proposed a layer-based stereoscopic image resizing method, in which each layer is warped by its own mesh deformation and the warped layers are composited together to form the retargeted images. However, although these methods can handle MVD data, it is limited to altering the perceived depth without the constraint from user’s visual experience.

Depth adaptation. Another way for depth adaptation is to retarget depth/disparity range, or control depth perception or visual comfort for the 3D content. Disparity shifting methods are relatively simple that adjust the zero disparity plane (ZDP) of a scene but maintain its overall disparity range [26]. Various linear and nonlinear disparity/depth remapping methods are developed to adjust the comfortable viewing zone [27], or adjust the depth perception [28], or manipulate the perceived depth [29], with the aims to enhance the 3D visual experience. However, these methods do not take into account the distortion from display resolution, and their depth adaptation to this situation is poor.

To optimize the disparity/depth by warping, Lang et al. [13] presented a set of disparity mapping operators for stereoscopic images that uses disparity and saliency estimation to compute a deformation of the input views to meet the target disparities. Yan et al. [14] used image warping to simultaneously adjust the positions of image pixels in the left and right images in order to remap the depth range of stereoscopic image. Wang et al. [30] proposed stereoscopic video disparity adjustment framework that takes disparity range, motion, and stereoscopic window violation into account. However, these methods only focus on disparity adjustment for stereoscopic contents, and develop several disparity mapping operations with consistent size constraint. If considering the distortions from different spatial resolutions and different camera geometries, these methods may fail in these situations.

Although multiple virtual views can be conveniently generated via DIBR technique, depth adaptation for view synthesis is considered in some methods. Huang et al. [31] presented a warping-based method for synthesizing virtual views from a binocular stereoscopic image based on dense feature correspondences instead of using DIBR. Li et al. [32] investigated the influences of stereoscopic parameters on multiview synthesis in S3D display and presented a parameter adjustment method for reducing visual discomforts and geometric distortions. Lei et al. [33] proposed a projection-based disparity control method to generate multiple virtual views by shifting the ZDP. However, in the practical applications, it is expected to view the virtual views naturally and comfortably under different viewing conditions. Thus, the virtual camera parameters should be taken into account in synthesizing the multiple virtual views.

In order to illustrate the limitations of the existing 3D retargeting and depth adaptation methods, we list the main properties of some representative works in Table 1. Considering resolution and depth adaptation to generate virtual views from MVD data in this work, we combine image retargeting and depth adaptation techniques to present a possible solution. The motivation of this work mainly comes from two aspects:

Tables Icon

Table 1. Main properties of the representative works

  • 1) Although we can separately retarget color and depth videos, depth is not a simple spatial dimension to be viewed, but inversely to generate the virtual views. Therefore, the resolution and depth should be optimized in a uniform framework.
  • 2) The factors considered in MVD retargeting will be not the same with those in stereoscopic image retargeting, because the factors affected the 3D perception of view synthesis should be addressed to present the viewing experience in a more natural way.

3. The proposed method

3.1 Framework overview

We propose a retargeting framework that aims to generate resolution and depth adjustable stereoscopic images from MVD data. As shown in Fig. 1, taking two-view MVD data as input (denoted as L-Reference and R-Reference respectively), we first transform the input MVD data to the display space based on the target displays and viewing conditions. Then, warping energy function is constructed based on the constraints of shape preservation, line bending, and visual comfort. The energy function is used to optimize the mesh warping procedure to generate the retargeted reference images and the corresponding retargeted depth maps. Finally, we generate the virtual view to construct the stereoscopic images by DIBR. Different with the existing stereoscopic image retargeting methods that preserve the original depth information after retargeting, our method simultaneously optimizes the resolution and depth to match the target displays and viewing conditions.

 

Fig. 1 Illustration of our proposed MVD retargeting framework.

Download Full Size | PPT Slide | PDF

3.2 Relationship between the depth image and the display space

Figure 2 illustrates cameras geometry in different spaces. The scene is captured by a reference camera in 3D space shown in (a), while the viewer perceives the 3D scene in the display space shown in (b). The baseline between reference and virtual cameras is not equal to the interocular distance between two eyes, and the depth of the convergence plane is not equal to the viewing distance. Considering the effects of different displays and viewing conditions, the camera arrangements used for the creation of stereoscopic contents are not unique. Therefore, transformation from the 3D space to display space is the first step to fulfill the following optimization. Given the L-Reference data (IL(x,y)andDL(x,y)), the real depth zx,y of a pixel point inIL(x,y)can be calculated from the quantized depth value Dx,y inDL(x,y)by

zx,y=1Dx,y255(1znear1zfar)+1zfar
where znear and zfar are the farthest and nearest depth values.

 

Fig. 2 Illustration of camera geometry in different spaces.

Download Full Size | PPT Slide | PDF

Considering the relationship between the camera coordinate and 3D world coordinate, the imaging geometry of the reference camera is formulated by projecting a matrix [uL,vL,1]in the camera coordinate to the matrix[X,Y,Z]in the world coordinate as follows:

zx,y[uLvL1]=[fu0u00fvv0001][r11r12r13txr21r22r23tyr31r32r33tz][XYZ]
[XYZ]=[r11r12r13r21r22r23r31r32r33][fu0u00fvv0001]1[uLvL1]zx,y+[txtytz]=R1A11[uLvL1]zx,y+T1
where fu and fv are the focal lengths along the x and y axes, and (u0,v0) is the coordinate of the principal point, A1 (3 × 3) and R1 (3 × 3) are the intrinsic and rotation matrices of the reference camera, and T1 (3 × 1) is the translation vector of the reference camera. Taking the virtual camera into consideration, the same world coordinate[X,Y,Z]is projected into the virtual image plane via
[u'v'w']=A2R21[XYZ]A2R21T2
where (u',v',w')is the homogeneous coordinates of the virtual image plane, and A2, R2 and T2 are the intrinsic matrix, rotation matrix and translation vector of the virtual camera, respectively. The corresponding pixel location in the virtual image plane is(uv,vv)=(u'/w',v'/w'). The matrices A1, R1 and T1, as well as A2, R2, T2, are known for specific cameras. Since(uL,vL)and(uV,vV)are the same points in both cameras, the disparity d between the reference and virtual cameras is

d=uVuL

From (5), it can be seen that the binocular disparity d incorporates the effects of camera distance (T1 and T2), focus length (A1 and A2) and camera rotation (R1 and R2), as well as the distance of objects to the camera (z). These parameters are common for a given camera arrangement. However, as demonstrated in Figs. 2(a) and 2(b), scene geometry from different camera sensors and screens will be not the same. Therefore, the distortions in image plane may be not the same with the perceived distortions on displays. Since the actual size of a real scene is generally too large or too small to be reproduced on different displays, the scaling on all dimensions besides horizontal and vertical dimensions is necessary. Taking the left and virtual images as a stereoscopic image pair, based on the principal of binocular vision, the point coordinates perceived by the viewer in the display space can be determined as follows [32]:

{XD=de(uL+de2)ded(W/R)de2YD=devLded(W/R)ZD=deLDded(W/R)
where LD be the viewing distance from the viewer to the screen, de be the interocular distance between the viewer’s two eyes, W and R denote the width and horizontal resolution of display. By optimizing the horizontal, vertical and depth coordinates in the display space, the factors of displays (W and R) and viewing conditions (LD) are integrated to promote user’s experience.

3.3 Construction of warping energy function

For resolution and depth adaptation, warping energy optimization is a feasible way to describe the relationship among camera arrangement, display, and stereoscopic viewing conditions. These factors are usually utilized in the energy functions to characterize the overall QoE perceived by the viewers when watching 3D content. In the practical situations, the QoE will be affected by different viewing conditions. Given the inputs IL(x,y)andDL(x,y), a set of 2D grid meshes with size of 40 × 40 in the reference image plane are denoted asUL={UL,k}, UL,k={uL,k1,uL,k2,uL,k3,uL,k4}. For each vertex (taking uL,k1 as example), it hasuL,k1={xL,k1,yL,k1}, wherexL,k1and yL,k1 denote the horizontal and vertical coordinates in the reference image plane, respectively. By transforming the coordinates to the display space by (6), a set of 3D surface meshesV={Vk} are created to cover the reference and virtual cameras. For each 3D surface mesh Vk, it consists of four vertices,Vk={vk1,vk2,vk3,vk4}, where vk1, vk2, vk3and vk4 denote the left-top, left-bottom, right-top, and right-bottom vertices, respectively. The coordinate of each vertex (takingvk1 as example) is represented asvk1={Xk1,Yk1,Zk1}, whereXk1, Yk1 andZk1denote the horizontal, vertical and depth coordinates in the display space, respectively.

1) Shape preservation energy: Since our method uses four vertices of a mesh to control the selected object, the shape and depth of the selected object may be deformed while implementing warping. This problem is solved by introducing a shape preservation energy term that preserves the shapes of the high-significance objects as much as possible, while allows to conduct large deformation for those low-significance objects. For each mesh Vk with four vertices, the distortion energy for the mesh is defined as

ESP(Vk)=ρkVkV˜k2
where ρk is a scale parameter to be optimized. By a linear least-squares solutionAP=b, we obtain the optimal ρk as
ρk=(AkTAk)1AkTb˜k
where

Ak=[Xk1Yk10100Yk1Xk1001000Zk1001Xk4Yk40100Yk4Xk4001000Zk4001],b˜k=[X˜k1Y˜k1Z˜k1X˜k4Y˜k4Z˜k4]

Considering visual importance of each mesh, the total shape preservation energy is the weighted sum of the distortions of all meshes, defined as

ESD=VkVS(k)Ak(AkTAk)1AkTb˜kb˜k2=VkVS(k)Ckb˜k2
where Ck=Ak(AkTAk)1AkTIand I is the identity matrix. To acquire the visual importance of each mesh, 3D saliency and depth sensitivity of each vertex are combined by:
Si(k)=ζi(k)+αψi(k)
where α is weigh for depth sensitivity. Here, the 3D saliency value ζi(k)and depth sensitivity value ψi(k)are computed based on our previous works [34,35]. The importance of the depth sensitivity included in the significance map is to propagate the influence of depth deformation on view synthesis to the warping process, so that the geometry changes induced by the depth deformation can be relieved in the virtual views. The significance value of a mesh is calculated by averaging the significance values of all pixels within the mesh. Figure 3 shows an example of the 3D saliency map and the depth sensitivity map.

 

Fig. 3 Example of the 3D saliency map and the depth sensitivity map.

Download Full Size | PPT Slide | PDF

2) Line bending energy: Refer to [22], to minimize the bending of the mesh edges, we define a line bending energy by measuring the angle between the deformed edge and its original edge. Let e=vivjbe original edge between two vertices vi and vj, and e^=v^iv^jbe the deformed edge between two vertices v^i andv^j, the angle between e ande^is approximated by

Δ(e^)=seee^2
where se is a scale parameter. Refer to the solution in (8), minimization of (12) yields a linear least-squares solution as

Δ(e^)=e(eTe)1eTe^e^2

Finally, the total line bending energy is defined as

ELB=vi,vjV(e(eTe)1eTI)e^2

3) Visual comfort energy: According to our warping framework, the depth of each mesh will be changed together with the resolution. To ensure the naturalness and visual comfort of the retargeted color-plus-depth data, we propose to adjust the depth range to adapt the new camera geometry. Considering different displays and viewing distances, the original depth range in the 3D space is mapped to the display space, obtaining a new depth range[ZDfar,ZDnear]. To ensure the depth range[ZDfar,ZDnear]fall into the comfort zone, we define a comfortable depth range based on the Percival’s Zone of Comfort (PZC) as follows:

Zmax=deLDdeη1LD,Zmin=deLDdeη2LD
where η1 and η2 denote the negative and positive retinal disparity limits (e.g., ± 1° disparity for the PZC).

The scaling factor is defined to scale the original depth range to a new comfortable depth range by:

K=ZmaxZminZDfarZDnear

The depth scaling function is defined to remap the original depth ZD to a new depth by

f(ZD)=K(ZDZDnear)+Zmin

Since objects at the screen yield null disparity (d = 0), while objects in front of and behind the screen produce negative and positive disparity, respectively, it leaves large space for the objects at the screen to adjust their depth information. Also, since the depth scaling function in (17) linearly remaps the depth values, the scene layout will be still the same with the original one. Considering the impacts of different disparity planes on visual comfort, the visual comfort energy is defined as

EVC=VkVi=14ω(Zki)f(Zki)Z˜ki2
ω(Zki)=exp(|ZkiLD|ZDfarZDnear)

3.4 Optimization for virtual color-and-depth generation

By combining the above shape preservation, line bending and visual comfort energy terms, the final optimization for the mesh warping is formulated as

argminV˜(ESP+EVC+ELB)

Minimizing the energy function corresponds to solving a least-squares linear systemAV˜T=band finds a set of deformed meshesV˜, which is satisfied the boundary conditions. However, in a typical DIBR system, typically two views of color and depth videos are used to synthesize a virtual view. That is, the quality of the virtual view will be affected by the detailed view blending in DIBR, that combines the synthesized virtual views from different reference views. To ensure the consistent deformation across different reference views, by re-projecting the deformed verticesV˜to the left and right reference image planes, the deformed verticesU˜LandU˜Rin the left and right reference images are obtained. Further, by establishing the warping relationships between the original verticesULand the deformed verticesU˜L, and between the original verticesURand the deformed verticesU˜R, we can warp the left and right images to the target resolution, denoted asI˜L(x',y)=IL(x,y)andI˜R(x'',y)=IR(x,y), respectively.

Since the depth coordinate of each vertex in changed after the above mesh optimization, we cannot directly use the same warping relationship on the depth maps. To obtain the retargeted depth map, the retargeted depth values are inversely transformed from the display space to 3D space. Then, we establish a five-parameter fitting function by minimizing the reconstruction error between the original and retargeted depth values for all vertices as [36]

f(D)=β1(1211+exp(β2(Dβ3)))+β4D+β5
where β1, β2, β3, β4 and β5 are parameters determined by using the original and retargeted depth values. With the fitting function and the same warping relationship between the original and deformed meshes, we can obtain the retargeted depth maps asD˜L(x',y)=f(DL(x,y))andD˜R(x'',y)=f(DR(x,y)).

Finally, with the retargetedI˜L(x',y)andD˜L(x',y), I˜R(x'',y)andD˜R(x'',y), the virtual view is synthesized by DIBR technique. Different with the general procedure that synthesizes the virtual view first and retargets it to target resolution, the advantage of the framework is that the viewing condition and the location of virtual camera are propagated to the warping process, so that the virtual camera parameters are considered and stereoscopic vision is addressed in a more natural way. Especially, in multiview displays, multiple virtual views should be synthesized simultaneously from the reference cameras. In this situation, the same warping relationship derived from the two-view case can be propagated to other virtual views to add the efficiency of multi-view view synthesis. Since virtual views are synthesized from the adjacent two views (i.e., L-Reference and R-Reference in Fig. 1), we only consider two-view MVD format in this work, and it can be easily extended to multiple views. For example, for three-view MVD format, the virtual view can be synthesized from two adjacent views.

4. Experimental results

In the experiment, seven typical 3D video sequences, ‘Balloons’ (1024 × 768), ‘Kendo’ (1024 × 768), ‘Bookarrival’ (1024 × 768), ‘Newspaper’ (1024 × 768), ‘GT_Fly’ (1920 × 1088), ‘Undo_Dancer’ (1920 × 1088) and ‘Poznan_Street’ (1920 × 1088) are tested. The selections of left, right and virtual views for these sequences are shown in Table 2. For the view synthesis, view synthesis reference software (VSRS) 1D fast [37] is employed. Although different parameter setting will have certain influences on the retargeting results, based on the practical subjective tests, we set the interocular distance de = 65mm, the width of display W = 750mm, the horizontal resolution R = 1920 and the viewing distance LD = 800mm.

Tables Icon

Table 2. Selected views for experiment.

4.1 Analysis the influence of depth remapping

As an important step of the proposed method, the final retargeted depth maps are obtained by remapping the original depth values to its retargeted values. Figure 4 shows the examples of the retargeted depth maps and the corresponding synthesized virtual images with and without depth remapping step (for better display of the depth maps, the depth values are mapped to [0, 255]). The retargeted depth maps with depth remapping step have distinctly different appearances with those of the retargeted depth maps without depth remapping step, because the depth values are simultaneously optimized in our warping framework, while the retargeted depth maps without depth remapping step omit this property (still preserve the original depth values). Using the depth maps without depth remapping step, the synthesized images have significant ghost in the boundary area, while the proposed method can eliminate the phenomenon. Also, duo to the influence of incorrect depth maps in the existing 3D video sequences, which are estimated by depth estimation software, the synthesized images will introduce a certain geometric change especially at the edge areas, while optimizing the depth in the display space can somewhat reduce the influence. More results will be presented in the next sections.

 

Fig. 4 Examples of the retargeted depth maps and the corresponding synthesized images with and without depth remapping step.

Download Full Size | PPT Slide | PDF

4.2 Comparison to other methods

Since the proposed method is very related to the stereoscopic image retargeting in the two-view case, we compare our method with state-of-the-art stereoscopic image retargeting methods, including linear scaling (SCL), stereoscopic seam carving (SSC) [20], and single-layer warping (SLW) [22]. For the SCL method, we independently apply the same scaling factor on each reference view. For the SSC and SLW methods, the left reference image and right reference image are taken as a stereoscopic image pair. The same retargeting relationships are directly performed on the depth maps for these methods. Figure 5, Fig. 6, and Fig. 7 illustrate the retargeting results on ‘Poznan_Street’, ‘Undo_Dancer’ and ‘Newspaper’ test sequences. In these figures, all images are shrunk by 40%. The first row in the figures shows the retargeted left reference images. The second row shows the retargeted left depth maps and the third row show the corresponding histograms of the depth maps. The synthesized images from the left and right reference views are shown in the fourth row. As shown in the retargeted left reference images, the SCL method may distort images due to the lack of object information, and the SSC method will suffer from more or less object shape distortion due to the essence of discrete seam removal. Although it is effective in shape preservation, the SLW method will lost the boundary contents, e.g., the car in Fig. 5 and girl in Fig. 7. As shown in the histograms of the depth maps, due to the nature of depth preservation, the SCL and SLW methods have the similar depth distributions with the original ones, while our method changes the depth distributions due to depth remapping. Using the above retargeted reference images and the retargeted depth maps to synthesize the virtual images, the synthesized images with the SCL and SLW methods have significant ghost in the boundary area. The reasons may be that, on one hand, the inconsistent deformation on the left and right reference views with the methods may change the inter-view geometry, while image retargeting in fact does not consider this factor. On the other hand, due to depth preservation essence, a reference point will be projected to the original instead of the retargeted location in the virtual view, while the matched reference points in the left and right reference views will be changed after retargeting. Even the ghost effect is not obvious for the SSC method, the synthesized images still suffer from significant shape deformation due to the poor quality of the retargeted reference images. In contrast, the proposed method preserves most of the structure information without the ghost effect in the synthesized images.

 

Fig. 5 Comparison between our method and the SCL, SSC and SLW on ‘Poznan_Street’ test sequence.

Download Full Size | PPT Slide | PDF

 

Fig. 6 Comparison between our method and the SCL, SSC and SLW on ‘Undo_Dancer’ test sequence.

Download Full Size | PPT Slide | PDF

 

Fig. 7 Comparison between our method and the SCL, SSC and SLW on ‘Newspaper’ test sequence.

Download Full Size | PPT Slide | PDF

4. 3 Subjective evaluation results

A subjective assessment test is conducted to validate the proposed MVDRT model and investigate the influence of the above-mentioned factors (e.g., shape preservation energy, line bending energy and visual comfort energy) on the perceived QoE. The original motivation of this study is to optimize the MVD retargeting for multiview displays with multiple virtual views. The difference between the stereoscopic and multiview displays is that, stereoscopic displays delivers the same pair of information across the entire view zone, whereas multiview displays treats the same space as discrete pieces and provides a different view of the virtual world. However, due to the lack of multiview displays, this test is carried out on a stereoscopic display with a given stereoscopic image pair. The main characteristics of the test are detailed as follows:

Methodology: The Absolute Category Rating (ACR) [38] methodology was used to evaluate the test materials. During the test, the observers were asked to provide a score for the overall 3D QoE, consid ering shape preservation, depth perception, visual comfort, etc. They used the five-grade quality scale. After detecting and discarding outliers of all opinion scores, the final mean opinion score (MOS) for each stereoscopic image was calculated as the mean of the remaining opinion scores.

Environment: The subjective tests were conducted in the laboratory designed for subjective tests according to the recommendation BT.500-11 [39]. A 65” Samsung UA65F9000 stereoscopic display war used to carry out the subjective tests. It is an Ultra HD 3D-LED display with active shutter glasses. The viewing distance was 3 times the height of the screen.

Observers: A total of 25 undergraduate students (12 males and 13 females) in Ningbo University were participated in the subjective test, all of them having normal vision. The ages of the participants were between range 20 and 25. The participants were asked to rank the stereoscopic images based on their own judgment.

The user study results on seven MVD sequences are reported in Table 3. To construct a stereoscopic image pair, the reference left image and the synthesized virtual image are used. Considering the factors of shape preservation, depth perception and visual comfort in evaluating the QoE, we evaluate the user’s visual experience in viewing a stereoscopic image pair. For the SCL and SLW methods, due to the effect of ghost, visual discomfort is obvious in viewing the stereoscopic image pairs, leading to lower opinion scores. For the SSC method, shape deformation is particularly serious for ‘Balloons’, ‘Newspaper’ and ‘Undo_Dancer’ test sequences (also can be found in Figs. 5-7), leading to very poor visual experience, while for some sequences, e.g., ‘Balloons’ and ‘GT_Fly’, the object shape preservation performances are better than our method. Overall, our approach can achieve a better tradeoff among shape preservation, depth perception and visual comfort factors to provide a better visual experience for users.

Tables Icon

Table 3. Quantitative subjective assessment results of different MVD sequences.

5. Conclusions

We have presented a retargeting framework for MVD data to optimize the visual experience for S3D displays. The most important technical innovation of our framework is that the parameters of virtual camera and viewing conditions are utilized to optimize the resolution and depth of MVD data for view synthesis. Our results demonstrate the effectiveness and power of our approach. Currently, we generate fixed-viewpoint S3D output from MVD data. For dense virtual view generation, other constraint conditions, such as displaying parameters, should be integrated into our MVDRT framework.

Funding

National Natural Science Foundation of China (NSFC) (grant 61622109, 61271021, 61471212, and U1301257); K.C. Wong Magna Fund in Ningbo University.

References and links

1. S. Baek and C. Lee, “Depth perception estimation of various stereoscopic displays,” Opt. Express 24(21), 23618–23634 (2016). [CrossRef]   [PubMed]  

2. A. K. Moorthy and A. C. Bovik, “A survey on 3D quality of experience and 3D quality assessment,” Proc. SPIE 8651, 86510M (2013). [CrossRef]  

3. H. Urey, K. V. Chellappan, E. Erden, and P. Surman, “State of the art in stereoscopic and autostereoscopic displays,” Proc. IEEE 99(4), 540–555 (2011). [CrossRef]  

4. D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, “Vergence-accommodation conflicts hinder visual performance and cause visual fatigue,” J. Vis. 8(3), 33 (2008). [CrossRef]   [PubMed]  

5. M. Lambooij, M. Fortuin, H. Heynderickx, and W. IJsselsteijn, “Visual discomfort and visual fatigue of stereoscopic displays: A review,” J. Imaging Sci. Technol. 53(3), 030201 (2009). [CrossRef]  

6. W. J. Kim, S. D. Kim, J. Kim, and N. Hur, “Resizing of stereoscopic images for display adaptation,” Proc. SPIE 7237, 72371S (2009). [CrossRef]  

7. M. Urvoy, M. Barkowsky, and P. Le Callet, “How visual fatigue and discomfort impact 3D-TV quality of experience: A comprehensive review of technological, psychophysical, and psychological factors,” Ann. Telecommun. 68(11–12), 641–655 (2013). [CrossRef]  

8. P. Merkle, A. Smolic, K. Muller, and T. Wiegand, “Multi-view video plus depth representation and coding,” in Proc. IEEE International Conference on Image Processing (2007), pp. 201–204. [CrossRef]  

9. Y. Liu, Q. Dai, Z. You, and W. Xu, “Rate-prediction structure complexity analysis for multi-view video coding using hybrid genetic algorithms,” Proc. SPIE 6508, 650804 (2007). [CrossRef]  

10. A. De Abreu, P. Frossard, and F. Pereira, “Optimized MVC prediction structures for interactive multiview video streaming,” IEEE Signal Process. Lett. 20(6), 603–606 (2013). [CrossRef]  

11. A. De Abreu, P. Frossard, and F. Pereira, “Fast MVC prediction structure selection for interactive multiview video streaming,” in Proc. Picture Coding Symp. (2013), pp. 169–172. [CrossRef]  

12. A. Fiandrotti, J. Chakareski, and P. Frossard, “Popularity-aware rate allocation in multi-view video coding,” Proc. SPIE 7744, 77440Q (2010). [CrossRef]  

13. M. Lang, A. Hornung, O. Wang, S. Poulakos, A. Smolic, and M. Gross, “Nonlinear disparity mapping for stereoscopic 3D,” ACM Trans. Graph. 29(4), 75 (2010). [CrossRef]  

14. T. Yan, R. W. H. Lau, Y. Xu, and L. Huang, “Depth mapping for stereoscopic videos,” Int. J. Comput. Vis. 102(1–3), 293–307 (2013). [CrossRef]  

15. M. S. Farid, M. Lucenteforte, and M. Grangetto, “Depth image based rendering with inverse mapping,” in Proc. IEEE International Workshop on Multimedia Signal Processing (2013), pp. 135–140. [CrossRef]  

16. M. Rubinstein, D. Gutierrez, O. Sorkine, and A. Shamir, “A comparative study of image retargeting,” ACM Trans. Graph. 29(6), 160 (2010). [CrossRef]  

17. S. Avidan and A. Shamir, “Seam carving for content-aware image resizing,” ACM Trans. Graph. 26(3), 118 (2007). [CrossRef]  

18. G. X. Zhang, M. M. Cheng, S. M. Hu, and R. R. Martin, “A shape-preserving approach to image resizing,” Comput. Graph. Forum 28(7), 1897–1906 (2009). [CrossRef]  

19. K. Utsugi, T. Shibahara, T. Koike, K. Takahashi, and T. Naemura, “Seam carving for stereo images,” in Proc. of 3DTV-Conference: The True Vision -Capture, Transmission and Display of 3D Video (2010), pp. 1–4. [CrossRef]  

20. T. Dekel Basha, Y. Moses, and S. Avidan, “Stereo seam carving a geometrically consistent approach,” IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2513–2525 (2013). [CrossRef]   [PubMed]  

21. F. Shao, W. Lin, W. Lin, G. Jiang, M. Yu, and R. Fu, “Stereoscopic visual attention guided seam carving for stereoscopic image retargeting,” J. Disp. Technol. 12(1), 22–30 (2016). [CrossRef]  

22. C. H. Chang, C. K. Liang, and Y. Y. Chuang, “Content-aware display adaptation and interactive editing for stereoscopic images,” IEEE Trans. Multimed. 13(4), 589–601 (2011). [CrossRef]  

23. S. S. Lin, C. H. Lin, S. H. Chang, and T. Y. Lee, “Object-coherence warping for stereoscopic image retargeting,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 759–768 (2014). [CrossRef]  

24. B. Li, L. Y. Duan, C. W. Lin, T. Huang, and W. Gao, “Depth-preserving warping for stereo image retargeting,” IEEE Trans. Image Process. 24(9), 2811–2826 (2015). [CrossRef]   [PubMed]  

25. K. Y. Lee, C. D. Chung, and Y. Y. Chuang, “Scene warping: Layer-based stereoscopic image resizing,” in Proc. of IEEE Computer Vision and Pattern Recognition (2012), pp. 49–56.

26. F. Shao, Z. Li, Q. Jiang, G. Jiang, M. Yu, and Z. Peng, “Visual discomfort relaxation for stereoscopic 3D images by adjusting zero-disparity plane for projection,” Displays 39, 125–132 (2015). [CrossRef]  

27. H. Sohn, Y. J. Jung, S. Lee, F. Speranza, and Y. M. Ro, “Visual comfort amelioration technique for stereoscopic images: Disparity remapping to mitigate global and local discomfort causes,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 745–758 (2014). [CrossRef]  

28. J. Lei, S. Li, B. Wang, K. Fang, and C. Hou, “Stereoscopic visual attention guided disparity control for multiview images,” J. Disp. Technol. 10(5), 373–379 (2014). [CrossRef]  

29. F. Shao, Q. Jiang, R. Fu, M. Yu, and G. Jiang, “Optimizing visual comfort for stereoscopic 3D display based on color-plus-depth signals,” Opt. Express 24(11), 11640–11653 (2016). [CrossRef]   [PubMed]  

30. M. Wang, X. J. Zhang, J. B. Liang, S. H. Zhang, and R. R. Martin, “Comfort-driven disparity adjustment for stereoscopic video,” Comput. Visual Media 2(1), 3–17 (2016). [CrossRef]  

31. Y. H. Huang, T. K. Huang, Y. H. Huang, W. C. Chen, and Y. Y. Chuang, “Warping-based novel view synthesis from a binocular image for autostereoscopic displays,” in Proc. of IEEE International Conference on Multimedia and Expo (2012), pp. 302–307. [CrossRef]  

32. D. Li, X. Qiao, D. Zang, L. Wang, and M. Zhang, “On adjustment of stereo parameters in multiview synthesis for planar 3D displays,” J. Soc. Inf. Disp. 23(10), 491–502 (2015). [CrossRef]  

33. J. Lei, M. Wang, B. Wang, K. Fan, and C. Hou, “Projection-based disparity control for toed-in multiview images,” Opt. Express 22(9), 11192–11204 (2014). [CrossRef]   [PubMed]  

34. Q. Jiang, F. Shao, G. Jiang, M. Yu, Z. Peng, and C. Yu, “A depth perception and visual comfort guided computational model for stereoscopic 3D visual saliency,” Signal Process. Image Commun. 38, 57–69 (2015). [CrossRef]  

35. F. Shao, W. Lin, G. Jiang, M. Yu, and Q. Dai, “Depth map coding for view synthesis based on distortion analyses,” IEEE J. Em. Sel. Top. C. 4(1), 106–117 (2014).

36. P. G. Gottschalk and J. R. Dunn, “The five-parameter logistic: a characterization and comparison with the four-parameter logistic,” Anal. Biochem. 343(1), 54–65 (2005). [CrossRef]   [PubMed]  

37. VSRS-1D-Fast. [Online]. Available: https://hevc.hhi.fraunhofer.de/svn/ svn_3DVCSoftware, 2014.

38. ITU-T P.910, “Subjective video quality assessment methods for multimedia applications,” (1999).

39. ITU-R BT-500.11, “Methodology for the subjective assessment of the quality of television pictures,” (2012).

References

  • View by:
  • |
  • |
  • |

  1. S. Baek and C. Lee, “Depth perception estimation of various stereoscopic displays,” Opt. Express 24(21), 23618–23634 (2016).
    [Crossref] [PubMed]
  2. A. K. Moorthy and A. C. Bovik, “A survey on 3D quality of experience and 3D quality assessment,” Proc. SPIE 8651, 86510M (2013).
    [Crossref]
  3. H. Urey, K. V. Chellappan, E. Erden, and P. Surman, “State of the art in stereoscopic and autostereoscopic displays,” Proc. IEEE 99(4), 540–555 (2011).
    [Crossref]
  4. D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, “Vergence-accommodation conflicts hinder visual performance and cause visual fatigue,” J. Vis. 8(3), 33 (2008).
    [Crossref] [PubMed]
  5. M. Lambooij, M. Fortuin, H. Heynderickx, and W. IJsselsteijn, “Visual discomfort and visual fatigue of stereoscopic displays: A review,” J. Imaging Sci. Technol. 53(3), 030201 (2009).
    [Crossref]
  6. W. J. Kim, S. D. Kim, J. Kim, and N. Hur, “Resizing of stereoscopic images for display adaptation,” Proc. SPIE 7237, 72371S (2009).
    [Crossref]
  7. M. Urvoy, M. Barkowsky, and P. Le Callet, “How visual fatigue and discomfort impact 3D-TV quality of experience: A comprehensive review of technological, psychophysical, and psychological factors,” Ann. Telecommun. 68(11–12), 641–655 (2013).
    [Crossref]
  8. P. Merkle, A. Smolic, K. Muller, and T. Wiegand, “Multi-view video plus depth representation and coding,” in Proc. IEEE International Conference on Image Processing (2007), pp. 201–204.
    [Crossref]
  9. Y. Liu, Q. Dai, Z. You, and W. Xu, “Rate-prediction structure complexity analysis for multi-view video coding using hybrid genetic algorithms,” Proc. SPIE 6508, 650804 (2007).
    [Crossref]
  10. A. De Abreu, P. Frossard, and F. Pereira, “Optimized MVC prediction structures for interactive multiview video streaming,” IEEE Signal Process. Lett. 20(6), 603–606 (2013).
    [Crossref]
  11. A. De Abreu, P. Frossard, and F. Pereira, “Fast MVC prediction structure selection for interactive multiview video streaming,” in Proc. Picture Coding Symp. (2013), pp. 169–172.
    [Crossref]
  12. A. Fiandrotti, J. Chakareski, and P. Frossard, “Popularity-aware rate allocation in multi-view video coding,” Proc. SPIE 7744, 77440Q (2010).
    [Crossref]
  13. M. Lang, A. Hornung, O. Wang, S. Poulakos, A. Smolic, and M. Gross, “Nonlinear disparity mapping for stereoscopic 3D,” ACM Trans. Graph. 29(4), 75 (2010).
    [Crossref]
  14. T. Yan, R. W. H. Lau, Y. Xu, and L. Huang, “Depth mapping for stereoscopic videos,” Int. J. Comput. Vis. 102(1–3), 293–307 (2013).
    [Crossref]
  15. M. S. Farid, M. Lucenteforte, and M. Grangetto, “Depth image based rendering with inverse mapping,” in Proc. IEEE International Workshop on Multimedia Signal Processing (2013), pp. 135–140.
    [Crossref]
  16. M. Rubinstein, D. Gutierrez, O. Sorkine, and A. Shamir, “A comparative study of image retargeting,” ACM Trans. Graph. 29(6), 160 (2010).
    [Crossref]
  17. S. Avidan and A. Shamir, “Seam carving for content-aware image resizing,” ACM Trans. Graph. 26(3), 118 (2007).
    [Crossref]
  18. G. X. Zhang, M. M. Cheng, S. M. Hu, and R. R. Martin, “A shape-preserving approach to image resizing,” Comput. Graph. Forum 28(7), 1897–1906 (2009).
    [Crossref]
  19. K. Utsugi, T. Shibahara, T. Koike, K. Takahashi, and T. Naemura, “Seam carving for stereo images,” in Proc. of 3DTV-Conference: The True Vision -Capture, Transmission and Display of 3D Video (2010), pp. 1–4.
    [Crossref]
  20. T. Dekel Basha, Y. Moses, and S. Avidan, “Stereo seam carving a geometrically consistent approach,” IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2513–2525 (2013).
    [Crossref] [PubMed]
  21. F. Shao, W. Lin, W. Lin, G. Jiang, M. Yu, and R. Fu, “Stereoscopic visual attention guided seam carving for stereoscopic image retargeting,” J. Disp. Technol. 12(1), 22–30 (2016).
    [Crossref]
  22. C. H. Chang, C. K. Liang, and Y. Y. Chuang, “Content-aware display adaptation and interactive editing for stereoscopic images,” IEEE Trans. Multimed. 13(4), 589–601 (2011).
    [Crossref]
  23. S. S. Lin, C. H. Lin, S. H. Chang, and T. Y. Lee, “Object-coherence warping for stereoscopic image retargeting,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 759–768 (2014).
    [Crossref]
  24. B. Li, L. Y. Duan, C. W. Lin, T. Huang, and W. Gao, “Depth-preserving warping for stereo image retargeting,” IEEE Trans. Image Process. 24(9), 2811–2826 (2015).
    [Crossref] [PubMed]
  25. K. Y. Lee, C. D. Chung, and Y. Y. Chuang, “Scene warping: Layer-based stereoscopic image resizing,” in Proc. of IEEE Computer Vision and Pattern Recognition (2012), pp. 49–56.
  26. F. Shao, Z. Li, Q. Jiang, G. Jiang, M. Yu, and Z. Peng, “Visual discomfort relaxation for stereoscopic 3D images by adjusting zero-disparity plane for projection,” Displays 39, 125–132 (2015).
    [Crossref]
  27. H. Sohn, Y. J. Jung, S. Lee, F. Speranza, and Y. M. Ro, “Visual comfort amelioration technique for stereoscopic images: Disparity remapping to mitigate global and local discomfort causes,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 745–758 (2014).
    [Crossref]
  28. J. Lei, S. Li, B. Wang, K. Fang, and C. Hou, “Stereoscopic visual attention guided disparity control for multiview images,” J. Disp. Technol. 10(5), 373–379 (2014).
    [Crossref]
  29. F. Shao, Q. Jiang, R. Fu, M. Yu, and G. Jiang, “Optimizing visual comfort for stereoscopic 3D display based on color-plus-depth signals,” Opt. Express 24(11), 11640–11653 (2016).
    [Crossref] [PubMed]
  30. M. Wang, X. J. Zhang, J. B. Liang, S. H. Zhang, and R. R. Martin, “Comfort-driven disparity adjustment for stereoscopic video,” Comput. Visual Media 2(1), 3–17 (2016).
    [Crossref]
  31. Y. H. Huang, T. K. Huang, Y. H. Huang, W. C. Chen, and Y. Y. Chuang, “Warping-based novel view synthesis from a binocular image for autostereoscopic displays,” in Proc. of IEEE International Conference on Multimedia and Expo (2012), pp. 302–307.
    [Crossref]
  32. D. Li, X. Qiao, D. Zang, L. Wang, and M. Zhang, “On adjustment of stereo parameters in multiview synthesis for planar 3D displays,” J. Soc. Inf. Disp. 23(10), 491–502 (2015).
    [Crossref]
  33. J. Lei, M. Wang, B. Wang, K. Fan, and C. Hou, “Projection-based disparity control for toed-in multiview images,” Opt. Express 22(9), 11192–11204 (2014).
    [Crossref] [PubMed]
  34. Q. Jiang, F. Shao, G. Jiang, M. Yu, Z. Peng, and C. Yu, “A depth perception and visual comfort guided computational model for stereoscopic 3D visual saliency,” Signal Process. Image Commun. 38, 57–69 (2015).
    [Crossref]
  35. F. Shao, W. Lin, G. Jiang, M. Yu, and Q. Dai, “Depth map coding for view synthesis based on distortion analyses,” IEEE J. Em. Sel. Top. C. 4(1), 106–117 (2014).
  36. P. G. Gottschalk and J. R. Dunn, “The five-parameter logistic: a characterization and comparison with the four-parameter logistic,” Anal. Biochem. 343(1), 54–65 (2005).
    [Crossref] [PubMed]
  37. VSRS-1D-Fast. [Online]. Available: https://hevc.hhi.fraunhofer.de/svn/ svn_3DVCSoftware , 2014.
  38. ITU-T P.910, “Subjective video quality assessment methods for multimedia applications,” (1999).
  39. ITU-R BT-500.11, “Methodology for the subjective assessment of the quality of television pictures,” (2012).

2016 (4)

S. Baek and C. Lee, “Depth perception estimation of various stereoscopic displays,” Opt. Express 24(21), 23618–23634 (2016).
[Crossref] [PubMed]

F. Shao, W. Lin, W. Lin, G. Jiang, M. Yu, and R. Fu, “Stereoscopic visual attention guided seam carving for stereoscopic image retargeting,” J. Disp. Technol. 12(1), 22–30 (2016).
[Crossref]

F. Shao, Q. Jiang, R. Fu, M. Yu, and G. Jiang, “Optimizing visual comfort for stereoscopic 3D display based on color-plus-depth signals,” Opt. Express 24(11), 11640–11653 (2016).
[Crossref] [PubMed]

M. Wang, X. J. Zhang, J. B. Liang, S. H. Zhang, and R. R. Martin, “Comfort-driven disparity adjustment for stereoscopic video,” Comput. Visual Media 2(1), 3–17 (2016).
[Crossref]

2015 (4)

D. Li, X. Qiao, D. Zang, L. Wang, and M. Zhang, “On adjustment of stereo parameters in multiview synthesis for planar 3D displays,” J. Soc. Inf. Disp. 23(10), 491–502 (2015).
[Crossref]

B. Li, L. Y. Duan, C. W. Lin, T. Huang, and W. Gao, “Depth-preserving warping for stereo image retargeting,” IEEE Trans. Image Process. 24(9), 2811–2826 (2015).
[Crossref] [PubMed]

F. Shao, Z. Li, Q. Jiang, G. Jiang, M. Yu, and Z. Peng, “Visual discomfort relaxation for stereoscopic 3D images by adjusting zero-disparity plane for projection,” Displays 39, 125–132 (2015).
[Crossref]

Q. Jiang, F. Shao, G. Jiang, M. Yu, Z. Peng, and C. Yu, “A depth perception and visual comfort guided computational model for stereoscopic 3D visual saliency,” Signal Process. Image Commun. 38, 57–69 (2015).
[Crossref]

2014 (5)

F. Shao, W. Lin, G. Jiang, M. Yu, and Q. Dai, “Depth map coding for view synthesis based on distortion analyses,” IEEE J. Em. Sel. Top. C. 4(1), 106–117 (2014).

S. S. Lin, C. H. Lin, S. H. Chang, and T. Y. Lee, “Object-coherence warping for stereoscopic image retargeting,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 759–768 (2014).
[Crossref]

H. Sohn, Y. J. Jung, S. Lee, F. Speranza, and Y. M. Ro, “Visual comfort amelioration technique for stereoscopic images: Disparity remapping to mitigate global and local discomfort causes,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 745–758 (2014).
[Crossref]

J. Lei, S. Li, B. Wang, K. Fang, and C. Hou, “Stereoscopic visual attention guided disparity control for multiview images,” J. Disp. Technol. 10(5), 373–379 (2014).
[Crossref]

J. Lei, M. Wang, B. Wang, K. Fan, and C. Hou, “Projection-based disparity control for toed-in multiview images,” Opt. Express 22(9), 11192–11204 (2014).
[Crossref] [PubMed]

2013 (5)

T. Dekel Basha, Y. Moses, and S. Avidan, “Stereo seam carving a geometrically consistent approach,” IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2513–2525 (2013).
[Crossref] [PubMed]

A. De Abreu, P. Frossard, and F. Pereira, “Optimized MVC prediction structures for interactive multiview video streaming,” IEEE Signal Process. Lett. 20(6), 603–606 (2013).
[Crossref]

T. Yan, R. W. H. Lau, Y. Xu, and L. Huang, “Depth mapping for stereoscopic videos,” Int. J. Comput. Vis. 102(1–3), 293–307 (2013).
[Crossref]

A. K. Moorthy and A. C. Bovik, “A survey on 3D quality of experience and 3D quality assessment,” Proc. SPIE 8651, 86510M (2013).
[Crossref]

M. Urvoy, M. Barkowsky, and P. Le Callet, “How visual fatigue and discomfort impact 3D-TV quality of experience: A comprehensive review of technological, psychophysical, and psychological factors,” Ann. Telecommun. 68(11–12), 641–655 (2013).
[Crossref]

2011 (2)

H. Urey, K. V. Chellappan, E. Erden, and P. Surman, “State of the art in stereoscopic and autostereoscopic displays,” Proc. IEEE 99(4), 540–555 (2011).
[Crossref]

C. H. Chang, C. K. Liang, and Y. Y. Chuang, “Content-aware display adaptation and interactive editing for stereoscopic images,” IEEE Trans. Multimed. 13(4), 589–601 (2011).
[Crossref]

2010 (3)

M. Rubinstein, D. Gutierrez, O. Sorkine, and A. Shamir, “A comparative study of image retargeting,” ACM Trans. Graph. 29(6), 160 (2010).
[Crossref]

A. Fiandrotti, J. Chakareski, and P. Frossard, “Popularity-aware rate allocation in multi-view video coding,” Proc. SPIE 7744, 77440Q (2010).
[Crossref]

M. Lang, A. Hornung, O. Wang, S. Poulakos, A. Smolic, and M. Gross, “Nonlinear disparity mapping for stereoscopic 3D,” ACM Trans. Graph. 29(4), 75 (2010).
[Crossref]

2009 (3)

G. X. Zhang, M. M. Cheng, S. M. Hu, and R. R. Martin, “A shape-preserving approach to image resizing,” Comput. Graph. Forum 28(7), 1897–1906 (2009).
[Crossref]

M. Lambooij, M. Fortuin, H. Heynderickx, and W. IJsselsteijn, “Visual discomfort and visual fatigue of stereoscopic displays: A review,” J. Imaging Sci. Technol. 53(3), 030201 (2009).
[Crossref]

W. J. Kim, S. D. Kim, J. Kim, and N. Hur, “Resizing of stereoscopic images for display adaptation,” Proc. SPIE 7237, 72371S (2009).
[Crossref]

2008 (1)

D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, “Vergence-accommodation conflicts hinder visual performance and cause visual fatigue,” J. Vis. 8(3), 33 (2008).
[Crossref] [PubMed]

2007 (2)

Y. Liu, Q. Dai, Z. You, and W. Xu, “Rate-prediction structure complexity analysis for multi-view video coding using hybrid genetic algorithms,” Proc. SPIE 6508, 650804 (2007).
[Crossref]

S. Avidan and A. Shamir, “Seam carving for content-aware image resizing,” ACM Trans. Graph. 26(3), 118 (2007).
[Crossref]

2005 (1)

P. G. Gottschalk and J. R. Dunn, “The five-parameter logistic: a characterization and comparison with the four-parameter logistic,” Anal. Biochem. 343(1), 54–65 (2005).
[Crossref] [PubMed]

Akeley, K.

D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, “Vergence-accommodation conflicts hinder visual performance and cause visual fatigue,” J. Vis. 8(3), 33 (2008).
[Crossref] [PubMed]

Avidan, S.

T. Dekel Basha, Y. Moses, and S. Avidan, “Stereo seam carving a geometrically consistent approach,” IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2513–2525 (2013).
[Crossref] [PubMed]

S. Avidan and A. Shamir, “Seam carving for content-aware image resizing,” ACM Trans. Graph. 26(3), 118 (2007).
[Crossref]

Baek, S.

Banks, M. S.

D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, “Vergence-accommodation conflicts hinder visual performance and cause visual fatigue,” J. Vis. 8(3), 33 (2008).
[Crossref] [PubMed]

Barkowsky, M.

M. Urvoy, M. Barkowsky, and P. Le Callet, “How visual fatigue and discomfort impact 3D-TV quality of experience: A comprehensive review of technological, psychophysical, and psychological factors,” Ann. Telecommun. 68(11–12), 641–655 (2013).
[Crossref]

Bovik, A. C.

A. K. Moorthy and A. C. Bovik, “A survey on 3D quality of experience and 3D quality assessment,” Proc. SPIE 8651, 86510M (2013).
[Crossref]

Chakareski, J.

A. Fiandrotti, J. Chakareski, and P. Frossard, “Popularity-aware rate allocation in multi-view video coding,” Proc. SPIE 7744, 77440Q (2010).
[Crossref]

Chang, C. H.

C. H. Chang, C. K. Liang, and Y. Y. Chuang, “Content-aware display adaptation and interactive editing for stereoscopic images,” IEEE Trans. Multimed. 13(4), 589–601 (2011).
[Crossref]

Chang, S. H.

S. S. Lin, C. H. Lin, S. H. Chang, and T. Y. Lee, “Object-coherence warping for stereoscopic image retargeting,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 759–768 (2014).
[Crossref]

Chellappan, K. V.

H. Urey, K. V. Chellappan, E. Erden, and P. Surman, “State of the art in stereoscopic and autostereoscopic displays,” Proc. IEEE 99(4), 540–555 (2011).
[Crossref]

Chen, W. C.

Y. H. Huang, T. K. Huang, Y. H. Huang, W. C. Chen, and Y. Y. Chuang, “Warping-based novel view synthesis from a binocular image for autostereoscopic displays,” in Proc. of IEEE International Conference on Multimedia and Expo (2012), pp. 302–307.
[Crossref]

Cheng, M. M.

G. X. Zhang, M. M. Cheng, S. M. Hu, and R. R. Martin, “A shape-preserving approach to image resizing,” Comput. Graph. Forum 28(7), 1897–1906 (2009).
[Crossref]

Chuang, Y. Y.

C. H. Chang, C. K. Liang, and Y. Y. Chuang, “Content-aware display adaptation and interactive editing for stereoscopic images,” IEEE Trans. Multimed. 13(4), 589–601 (2011).
[Crossref]

K. Y. Lee, C. D. Chung, and Y. Y. Chuang, “Scene warping: Layer-based stereoscopic image resizing,” in Proc. of IEEE Computer Vision and Pattern Recognition (2012), pp. 49–56.

Y. H. Huang, T. K. Huang, Y. H. Huang, W. C. Chen, and Y. Y. Chuang, “Warping-based novel view synthesis from a binocular image for autostereoscopic displays,” in Proc. of IEEE International Conference on Multimedia and Expo (2012), pp. 302–307.
[Crossref]

Chung, C. D.

K. Y. Lee, C. D. Chung, and Y. Y. Chuang, “Scene warping: Layer-based stereoscopic image resizing,” in Proc. of IEEE Computer Vision and Pattern Recognition (2012), pp. 49–56.

Dai, Q.

F. Shao, W. Lin, G. Jiang, M. Yu, and Q. Dai, “Depth map coding for view synthesis based on distortion analyses,” IEEE J. Em. Sel. Top. C. 4(1), 106–117 (2014).

Y. Liu, Q. Dai, Z. You, and W. Xu, “Rate-prediction structure complexity analysis for multi-view video coding using hybrid genetic algorithms,” Proc. SPIE 6508, 650804 (2007).
[Crossref]

De Abreu, A.

A. De Abreu, P. Frossard, and F. Pereira, “Optimized MVC prediction structures for interactive multiview video streaming,” IEEE Signal Process. Lett. 20(6), 603–606 (2013).
[Crossref]

A. De Abreu, P. Frossard, and F. Pereira, “Fast MVC prediction structure selection for interactive multiview video streaming,” in Proc. Picture Coding Symp. (2013), pp. 169–172.
[Crossref]

Dekel Basha, T.

T. Dekel Basha, Y. Moses, and S. Avidan, “Stereo seam carving a geometrically consistent approach,” IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2513–2525 (2013).
[Crossref] [PubMed]

Duan, L. Y.

B. Li, L. Y. Duan, C. W. Lin, T. Huang, and W. Gao, “Depth-preserving warping for stereo image retargeting,” IEEE Trans. Image Process. 24(9), 2811–2826 (2015).
[Crossref] [PubMed]

Dunn, J. R.

P. G. Gottschalk and J. R. Dunn, “The five-parameter logistic: a characterization and comparison with the four-parameter logistic,” Anal. Biochem. 343(1), 54–65 (2005).
[Crossref] [PubMed]

Erden, E.

H. Urey, K. V. Chellappan, E. Erden, and P. Surman, “State of the art in stereoscopic and autostereoscopic displays,” Proc. IEEE 99(4), 540–555 (2011).
[Crossref]

Fan, K.

Fang, K.

J. Lei, S. Li, B. Wang, K. Fang, and C. Hou, “Stereoscopic visual attention guided disparity control for multiview images,” J. Disp. Technol. 10(5), 373–379 (2014).
[Crossref]

Farid, M. S.

M. S. Farid, M. Lucenteforte, and M. Grangetto, “Depth image based rendering with inverse mapping,” in Proc. IEEE International Workshop on Multimedia Signal Processing (2013), pp. 135–140.
[Crossref]

Fiandrotti, A.

A. Fiandrotti, J. Chakareski, and P. Frossard, “Popularity-aware rate allocation in multi-view video coding,” Proc. SPIE 7744, 77440Q (2010).
[Crossref]

Fortuin, M.

M. Lambooij, M. Fortuin, H. Heynderickx, and W. IJsselsteijn, “Visual discomfort and visual fatigue of stereoscopic displays: A review,” J. Imaging Sci. Technol. 53(3), 030201 (2009).
[Crossref]

Frossard, P.

A. De Abreu, P. Frossard, and F. Pereira, “Optimized MVC prediction structures for interactive multiview video streaming,” IEEE Signal Process. Lett. 20(6), 603–606 (2013).
[Crossref]

A. Fiandrotti, J. Chakareski, and P. Frossard, “Popularity-aware rate allocation in multi-view video coding,” Proc. SPIE 7744, 77440Q (2010).
[Crossref]

A. De Abreu, P. Frossard, and F. Pereira, “Fast MVC prediction structure selection for interactive multiview video streaming,” in Proc. Picture Coding Symp. (2013), pp. 169–172.
[Crossref]

Fu, R.

F. Shao, W. Lin, W. Lin, G. Jiang, M. Yu, and R. Fu, “Stereoscopic visual attention guided seam carving for stereoscopic image retargeting,” J. Disp. Technol. 12(1), 22–30 (2016).
[Crossref]

F. Shao, Q. Jiang, R. Fu, M. Yu, and G. Jiang, “Optimizing visual comfort for stereoscopic 3D display based on color-plus-depth signals,” Opt. Express 24(11), 11640–11653 (2016).
[Crossref] [PubMed]

Gao, W.

B. Li, L. Y. Duan, C. W. Lin, T. Huang, and W. Gao, “Depth-preserving warping for stereo image retargeting,” IEEE Trans. Image Process. 24(9), 2811–2826 (2015).
[Crossref] [PubMed]

Girshick, A. R.

D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, “Vergence-accommodation conflicts hinder visual performance and cause visual fatigue,” J. Vis. 8(3), 33 (2008).
[Crossref] [PubMed]

Gottschalk, P. G.

P. G. Gottschalk and J. R. Dunn, “The five-parameter logistic: a characterization and comparison with the four-parameter logistic,” Anal. Biochem. 343(1), 54–65 (2005).
[Crossref] [PubMed]

Grangetto, M.

M. S. Farid, M. Lucenteforte, and M. Grangetto, “Depth image based rendering with inverse mapping,” in Proc. IEEE International Workshop on Multimedia Signal Processing (2013), pp. 135–140.
[Crossref]

Gross, M.

M. Lang, A. Hornung, O. Wang, S. Poulakos, A. Smolic, and M. Gross, “Nonlinear disparity mapping for stereoscopic 3D,” ACM Trans. Graph. 29(4), 75 (2010).
[Crossref]

Gutierrez, D.

M. Rubinstein, D. Gutierrez, O. Sorkine, and A. Shamir, “A comparative study of image retargeting,” ACM Trans. Graph. 29(6), 160 (2010).
[Crossref]

Heynderickx, H.

M. Lambooij, M. Fortuin, H. Heynderickx, and W. IJsselsteijn, “Visual discomfort and visual fatigue of stereoscopic displays: A review,” J. Imaging Sci. Technol. 53(3), 030201 (2009).
[Crossref]

Hoffman, D. M.

D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, “Vergence-accommodation conflicts hinder visual performance and cause visual fatigue,” J. Vis. 8(3), 33 (2008).
[Crossref] [PubMed]

Hornung, A.

M. Lang, A. Hornung, O. Wang, S. Poulakos, A. Smolic, and M. Gross, “Nonlinear disparity mapping for stereoscopic 3D,” ACM Trans. Graph. 29(4), 75 (2010).
[Crossref]

Hou, C.

J. Lei, M. Wang, B. Wang, K. Fan, and C. Hou, “Projection-based disparity control for toed-in multiview images,” Opt. Express 22(9), 11192–11204 (2014).
[Crossref] [PubMed]

J. Lei, S. Li, B. Wang, K. Fang, and C. Hou, “Stereoscopic visual attention guided disparity control for multiview images,” J. Disp. Technol. 10(5), 373–379 (2014).
[Crossref]

Hu, S. M.

G. X. Zhang, M. M. Cheng, S. M. Hu, and R. R. Martin, “A shape-preserving approach to image resizing,” Comput. Graph. Forum 28(7), 1897–1906 (2009).
[Crossref]

Huang, L.

T. Yan, R. W. H. Lau, Y. Xu, and L. Huang, “Depth mapping for stereoscopic videos,” Int. J. Comput. Vis. 102(1–3), 293–307 (2013).
[Crossref]

Huang, T.

B. Li, L. Y. Duan, C. W. Lin, T. Huang, and W. Gao, “Depth-preserving warping for stereo image retargeting,” IEEE Trans. Image Process. 24(9), 2811–2826 (2015).
[Crossref] [PubMed]

Huang, T. K.

Y. H. Huang, T. K. Huang, Y. H. Huang, W. C. Chen, and Y. Y. Chuang, “Warping-based novel view synthesis from a binocular image for autostereoscopic displays,” in Proc. of IEEE International Conference on Multimedia and Expo (2012), pp. 302–307.
[Crossref]

Huang, Y. H.

Y. H. Huang, T. K. Huang, Y. H. Huang, W. C. Chen, and Y. Y. Chuang, “Warping-based novel view synthesis from a binocular image for autostereoscopic displays,” in Proc. of IEEE International Conference on Multimedia and Expo (2012), pp. 302–307.
[Crossref]

Y. H. Huang, T. K. Huang, Y. H. Huang, W. C. Chen, and Y. Y. Chuang, “Warping-based novel view synthesis from a binocular image for autostereoscopic displays,” in Proc. of IEEE International Conference on Multimedia and Expo (2012), pp. 302–307.
[Crossref]

Hur, N.

W. J. Kim, S. D. Kim, J. Kim, and N. Hur, “Resizing of stereoscopic images for display adaptation,” Proc. SPIE 7237, 72371S (2009).
[Crossref]

IJsselsteijn, W.

M. Lambooij, M. Fortuin, H. Heynderickx, and W. IJsselsteijn, “Visual discomfort and visual fatigue of stereoscopic displays: A review,” J. Imaging Sci. Technol. 53(3), 030201 (2009).
[Crossref]

Jiang, G.

F. Shao, W. Lin, W. Lin, G. Jiang, M. Yu, and R. Fu, “Stereoscopic visual attention guided seam carving for stereoscopic image retargeting,” J. Disp. Technol. 12(1), 22–30 (2016).
[Crossref]

F. Shao, Q. Jiang, R. Fu, M. Yu, and G. Jiang, “Optimizing visual comfort for stereoscopic 3D display based on color-plus-depth signals,” Opt. Express 24(11), 11640–11653 (2016).
[Crossref] [PubMed]

F. Shao, Z. Li, Q. Jiang, G. Jiang, M. Yu, and Z. Peng, “Visual discomfort relaxation for stereoscopic 3D images by adjusting zero-disparity plane for projection,” Displays 39, 125–132 (2015).
[Crossref]

Q. Jiang, F. Shao, G. Jiang, M. Yu, Z. Peng, and C. Yu, “A depth perception and visual comfort guided computational model for stereoscopic 3D visual saliency,” Signal Process. Image Commun. 38, 57–69 (2015).
[Crossref]

F. Shao, W. Lin, G. Jiang, M. Yu, and Q. Dai, “Depth map coding for view synthesis based on distortion analyses,” IEEE J. Em. Sel. Top. C. 4(1), 106–117 (2014).

Jiang, Q.

F. Shao, Q. Jiang, R. Fu, M. Yu, and G. Jiang, “Optimizing visual comfort for stereoscopic 3D display based on color-plus-depth signals,” Opt. Express 24(11), 11640–11653 (2016).
[Crossref] [PubMed]

Q. Jiang, F. Shao, G. Jiang, M. Yu, Z. Peng, and C. Yu, “A depth perception and visual comfort guided computational model for stereoscopic 3D visual saliency,” Signal Process. Image Commun. 38, 57–69 (2015).
[Crossref]

F. Shao, Z. Li, Q. Jiang, G. Jiang, M. Yu, and Z. Peng, “Visual discomfort relaxation for stereoscopic 3D images by adjusting zero-disparity plane for projection,” Displays 39, 125–132 (2015).
[Crossref]

Jung, Y. J.

H. Sohn, Y. J. Jung, S. Lee, F. Speranza, and Y. M. Ro, “Visual comfort amelioration technique for stereoscopic images: Disparity remapping to mitigate global and local discomfort causes,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 745–758 (2014).
[Crossref]

Kim, J.

W. J. Kim, S. D. Kim, J. Kim, and N. Hur, “Resizing of stereoscopic images for display adaptation,” Proc. SPIE 7237, 72371S (2009).
[Crossref]

Kim, S. D.

W. J. Kim, S. D. Kim, J. Kim, and N. Hur, “Resizing of stereoscopic images for display adaptation,” Proc. SPIE 7237, 72371S (2009).
[Crossref]

Kim, W. J.

W. J. Kim, S. D. Kim, J. Kim, and N. Hur, “Resizing of stereoscopic images for display adaptation,” Proc. SPIE 7237, 72371S (2009).
[Crossref]

Lambooij, M.

M. Lambooij, M. Fortuin, H. Heynderickx, and W. IJsselsteijn, “Visual discomfort and visual fatigue of stereoscopic displays: A review,” J. Imaging Sci. Technol. 53(3), 030201 (2009).
[Crossref]

Lang, M.

M. Lang, A. Hornung, O. Wang, S. Poulakos, A. Smolic, and M. Gross, “Nonlinear disparity mapping for stereoscopic 3D,” ACM Trans. Graph. 29(4), 75 (2010).
[Crossref]

Lau, R. W. H.

T. Yan, R. W. H. Lau, Y. Xu, and L. Huang, “Depth mapping for stereoscopic videos,” Int. J. Comput. Vis. 102(1–3), 293–307 (2013).
[Crossref]

Le Callet, P.

M. Urvoy, M. Barkowsky, and P. Le Callet, “How visual fatigue and discomfort impact 3D-TV quality of experience: A comprehensive review of technological, psychophysical, and psychological factors,” Ann. Telecommun. 68(11–12), 641–655 (2013).
[Crossref]

Lee, C.

Lee, K. Y.

K. Y. Lee, C. D. Chung, and Y. Y. Chuang, “Scene warping: Layer-based stereoscopic image resizing,” in Proc. of IEEE Computer Vision and Pattern Recognition (2012), pp. 49–56.

Lee, S.

H. Sohn, Y. J. Jung, S. Lee, F. Speranza, and Y. M. Ro, “Visual comfort amelioration technique for stereoscopic images: Disparity remapping to mitigate global and local discomfort causes,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 745–758 (2014).
[Crossref]

Lee, T. Y.

S. S. Lin, C. H. Lin, S. H. Chang, and T. Y. Lee, “Object-coherence warping for stereoscopic image retargeting,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 759–768 (2014).
[Crossref]

Lei, J.

J. Lei, S. Li, B. Wang, K. Fang, and C. Hou, “Stereoscopic visual attention guided disparity control for multiview images,” J. Disp. Technol. 10(5), 373–379 (2014).
[Crossref]

J. Lei, M. Wang, B. Wang, K. Fan, and C. Hou, “Projection-based disparity control for toed-in multiview images,” Opt. Express 22(9), 11192–11204 (2014).
[Crossref] [PubMed]

Li, B.

B. Li, L. Y. Duan, C. W. Lin, T. Huang, and W. Gao, “Depth-preserving warping for stereo image retargeting,” IEEE Trans. Image Process. 24(9), 2811–2826 (2015).
[Crossref] [PubMed]

Li, D.

D. Li, X. Qiao, D. Zang, L. Wang, and M. Zhang, “On adjustment of stereo parameters in multiview synthesis for planar 3D displays,” J. Soc. Inf. Disp. 23(10), 491–502 (2015).
[Crossref]

Li, S.

J. Lei, S. Li, B. Wang, K. Fang, and C. Hou, “Stereoscopic visual attention guided disparity control for multiview images,” J. Disp. Technol. 10(5), 373–379 (2014).
[Crossref]

Li, Z.

F. Shao, Z. Li, Q. Jiang, G. Jiang, M. Yu, and Z. Peng, “Visual discomfort relaxation for stereoscopic 3D images by adjusting zero-disparity plane for projection,” Displays 39, 125–132 (2015).
[Crossref]

Liang, C. K.

C. H. Chang, C. K. Liang, and Y. Y. Chuang, “Content-aware display adaptation and interactive editing for stereoscopic images,” IEEE Trans. Multimed. 13(4), 589–601 (2011).
[Crossref]

Liang, J. B.

M. Wang, X. J. Zhang, J. B. Liang, S. H. Zhang, and R. R. Martin, “Comfort-driven disparity adjustment for stereoscopic video,” Comput. Visual Media 2(1), 3–17 (2016).
[Crossref]

Lin, C. H.

S. S. Lin, C. H. Lin, S. H. Chang, and T. Y. Lee, “Object-coherence warping for stereoscopic image retargeting,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 759–768 (2014).
[Crossref]

Lin, C. W.

B. Li, L. Y. Duan, C. W. Lin, T. Huang, and W. Gao, “Depth-preserving warping for stereo image retargeting,” IEEE Trans. Image Process. 24(9), 2811–2826 (2015).
[Crossref] [PubMed]

Lin, S. S.

S. S. Lin, C. H. Lin, S. H. Chang, and T. Y. Lee, “Object-coherence warping for stereoscopic image retargeting,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 759–768 (2014).
[Crossref]

Lin, W.

F. Shao, W. Lin, W. Lin, G. Jiang, M. Yu, and R. Fu, “Stereoscopic visual attention guided seam carving for stereoscopic image retargeting,” J. Disp. Technol. 12(1), 22–30 (2016).
[Crossref]

F. Shao, W. Lin, W. Lin, G. Jiang, M. Yu, and R. Fu, “Stereoscopic visual attention guided seam carving for stereoscopic image retargeting,” J. Disp. Technol. 12(1), 22–30 (2016).
[Crossref]

F. Shao, W. Lin, G. Jiang, M. Yu, and Q. Dai, “Depth map coding for view synthesis based on distortion analyses,” IEEE J. Em. Sel. Top. C. 4(1), 106–117 (2014).

Liu, Y.

Y. Liu, Q. Dai, Z. You, and W. Xu, “Rate-prediction structure complexity analysis for multi-view video coding using hybrid genetic algorithms,” Proc. SPIE 6508, 650804 (2007).
[Crossref]

Lucenteforte, M.

M. S. Farid, M. Lucenteforte, and M. Grangetto, “Depth image based rendering with inverse mapping,” in Proc. IEEE International Workshop on Multimedia Signal Processing (2013), pp. 135–140.
[Crossref]

Martin, R. R.

M. Wang, X. J. Zhang, J. B. Liang, S. H. Zhang, and R. R. Martin, “Comfort-driven disparity adjustment for stereoscopic video,” Comput. Visual Media 2(1), 3–17 (2016).
[Crossref]

G. X. Zhang, M. M. Cheng, S. M. Hu, and R. R. Martin, “A shape-preserving approach to image resizing,” Comput. Graph. Forum 28(7), 1897–1906 (2009).
[Crossref]

Merkle, P.

P. Merkle, A. Smolic, K. Muller, and T. Wiegand, “Multi-view video plus depth representation and coding,” in Proc. IEEE International Conference on Image Processing (2007), pp. 201–204.
[Crossref]

Moorthy, A. K.

A. K. Moorthy and A. C. Bovik, “A survey on 3D quality of experience and 3D quality assessment,” Proc. SPIE 8651, 86510M (2013).
[Crossref]

Moses, Y.

T. Dekel Basha, Y. Moses, and S. Avidan, “Stereo seam carving a geometrically consistent approach,” IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2513–2525 (2013).
[Crossref] [PubMed]

Muller, K.

P. Merkle, A. Smolic, K. Muller, and T. Wiegand, “Multi-view video plus depth representation and coding,” in Proc. IEEE International Conference on Image Processing (2007), pp. 201–204.
[Crossref]

Peng, Z.

Q. Jiang, F. Shao, G. Jiang, M. Yu, Z. Peng, and C. Yu, “A depth perception and visual comfort guided computational model for stereoscopic 3D visual saliency,” Signal Process. Image Commun. 38, 57–69 (2015).
[Crossref]

F. Shao, Z. Li, Q. Jiang, G. Jiang, M. Yu, and Z. Peng, “Visual discomfort relaxation for stereoscopic 3D images by adjusting zero-disparity plane for projection,” Displays 39, 125–132 (2015).
[Crossref]

Pereira, F.

A. De Abreu, P. Frossard, and F. Pereira, “Optimized MVC prediction structures for interactive multiview video streaming,” IEEE Signal Process. Lett. 20(6), 603–606 (2013).
[Crossref]

A. De Abreu, P. Frossard, and F. Pereira, “Fast MVC prediction structure selection for interactive multiview video streaming,” in Proc. Picture Coding Symp. (2013), pp. 169–172.
[Crossref]

Poulakos, S.

M. Lang, A. Hornung, O. Wang, S. Poulakos, A. Smolic, and M. Gross, “Nonlinear disparity mapping for stereoscopic 3D,” ACM Trans. Graph. 29(4), 75 (2010).
[Crossref]

Qiao, X.

D. Li, X. Qiao, D. Zang, L. Wang, and M. Zhang, “On adjustment of stereo parameters in multiview synthesis for planar 3D displays,” J. Soc. Inf. Disp. 23(10), 491–502 (2015).
[Crossref]

Ro, Y. M.

H. Sohn, Y. J. Jung, S. Lee, F. Speranza, and Y. M. Ro, “Visual comfort amelioration technique for stereoscopic images: Disparity remapping to mitigate global and local discomfort causes,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 745–758 (2014).
[Crossref]

Rubinstein, M.

M. Rubinstein, D. Gutierrez, O. Sorkine, and A. Shamir, “A comparative study of image retargeting,” ACM Trans. Graph. 29(6), 160 (2010).
[Crossref]

Shamir, A.

M. Rubinstein, D. Gutierrez, O. Sorkine, and A. Shamir, “A comparative study of image retargeting,” ACM Trans. Graph. 29(6), 160 (2010).
[Crossref]

S. Avidan and A. Shamir, “Seam carving for content-aware image resizing,” ACM Trans. Graph. 26(3), 118 (2007).
[Crossref]

Shao, F.

F. Shao, W. Lin, W. Lin, G. Jiang, M. Yu, and R. Fu, “Stereoscopic visual attention guided seam carving for stereoscopic image retargeting,” J. Disp. Technol. 12(1), 22–30 (2016).
[Crossref]

F. Shao, Q. Jiang, R. Fu, M. Yu, and G. Jiang, “Optimizing visual comfort for stereoscopic 3D display based on color-plus-depth signals,” Opt. Express 24(11), 11640–11653 (2016).
[Crossref] [PubMed]

F. Shao, Z. Li, Q. Jiang, G. Jiang, M. Yu, and Z. Peng, “Visual discomfort relaxation for stereoscopic 3D images by adjusting zero-disparity plane for projection,” Displays 39, 125–132 (2015).
[Crossref]

Q. Jiang, F. Shao, G. Jiang, M. Yu, Z. Peng, and C. Yu, “A depth perception and visual comfort guided computational model for stereoscopic 3D visual saliency,” Signal Process. Image Commun. 38, 57–69 (2015).
[Crossref]

F. Shao, W. Lin, G. Jiang, M. Yu, and Q. Dai, “Depth map coding for view synthesis based on distortion analyses,” IEEE J. Em. Sel. Top. C. 4(1), 106–117 (2014).

Smolic, A.

M. Lang, A. Hornung, O. Wang, S. Poulakos, A. Smolic, and M. Gross, “Nonlinear disparity mapping for stereoscopic 3D,” ACM Trans. Graph. 29(4), 75 (2010).
[Crossref]

P. Merkle, A. Smolic, K. Muller, and T. Wiegand, “Multi-view video plus depth representation and coding,” in Proc. IEEE International Conference on Image Processing (2007), pp. 201–204.
[Crossref]

Sohn, H.

H. Sohn, Y. J. Jung, S. Lee, F. Speranza, and Y. M. Ro, “Visual comfort amelioration technique for stereoscopic images: Disparity remapping to mitigate global and local discomfort causes,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 745–758 (2014).
[Crossref]

Sorkine, O.

M. Rubinstein, D. Gutierrez, O. Sorkine, and A. Shamir, “A comparative study of image retargeting,” ACM Trans. Graph. 29(6), 160 (2010).
[Crossref]

Speranza, F.

H. Sohn, Y. J. Jung, S. Lee, F. Speranza, and Y. M. Ro, “Visual comfort amelioration technique for stereoscopic images: Disparity remapping to mitigate global and local discomfort causes,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 745–758 (2014).
[Crossref]

Surman, P.

H. Urey, K. V. Chellappan, E. Erden, and P. Surman, “State of the art in stereoscopic and autostereoscopic displays,” Proc. IEEE 99(4), 540–555 (2011).
[Crossref]

Urey, H.

H. Urey, K. V. Chellappan, E. Erden, and P. Surman, “State of the art in stereoscopic and autostereoscopic displays,” Proc. IEEE 99(4), 540–555 (2011).
[Crossref]

Urvoy, M.

M. Urvoy, M. Barkowsky, and P. Le Callet, “How visual fatigue and discomfort impact 3D-TV quality of experience: A comprehensive review of technological, psychophysical, and psychological factors,” Ann. Telecommun. 68(11–12), 641–655 (2013).
[Crossref]

Wang, B.

J. Lei, S. Li, B. Wang, K. Fang, and C. Hou, “Stereoscopic visual attention guided disparity control for multiview images,” J. Disp. Technol. 10(5), 373–379 (2014).
[Crossref]

J. Lei, M. Wang, B. Wang, K. Fan, and C. Hou, “Projection-based disparity control for toed-in multiview images,” Opt. Express 22(9), 11192–11204 (2014).
[Crossref] [PubMed]

Wang, L.

D. Li, X. Qiao, D. Zang, L. Wang, and M. Zhang, “On adjustment of stereo parameters in multiview synthesis for planar 3D displays,” J. Soc. Inf. Disp. 23(10), 491–502 (2015).
[Crossref]

Wang, M.

M. Wang, X. J. Zhang, J. B. Liang, S. H. Zhang, and R. R. Martin, “Comfort-driven disparity adjustment for stereoscopic video,” Comput. Visual Media 2(1), 3–17 (2016).
[Crossref]

J. Lei, M. Wang, B. Wang, K. Fan, and C. Hou, “Projection-based disparity control for toed-in multiview images,” Opt. Express 22(9), 11192–11204 (2014).
[Crossref] [PubMed]

Wang, O.

M. Lang, A. Hornung, O. Wang, S. Poulakos, A. Smolic, and M. Gross, “Nonlinear disparity mapping for stereoscopic 3D,” ACM Trans. Graph. 29(4), 75 (2010).
[Crossref]

Wiegand, T.

P. Merkle, A. Smolic, K. Muller, and T. Wiegand, “Multi-view video plus depth representation and coding,” in Proc. IEEE International Conference on Image Processing (2007), pp. 201–204.
[Crossref]

Xu, W.

Y. Liu, Q. Dai, Z. You, and W. Xu, “Rate-prediction structure complexity analysis for multi-view video coding using hybrid genetic algorithms,” Proc. SPIE 6508, 650804 (2007).
[Crossref]

Xu, Y.

T. Yan, R. W. H. Lau, Y. Xu, and L. Huang, “Depth mapping for stereoscopic videos,” Int. J. Comput. Vis. 102(1–3), 293–307 (2013).
[Crossref]

Yan, T.

T. Yan, R. W. H. Lau, Y. Xu, and L. Huang, “Depth mapping for stereoscopic videos,” Int. J. Comput. Vis. 102(1–3), 293–307 (2013).
[Crossref]

You, Z.

Y. Liu, Q. Dai, Z. You, and W. Xu, “Rate-prediction structure complexity analysis for multi-view video coding using hybrid genetic algorithms,” Proc. SPIE 6508, 650804 (2007).
[Crossref]

Yu, C.

Q. Jiang, F. Shao, G. Jiang, M. Yu, Z. Peng, and C. Yu, “A depth perception and visual comfort guided computational model for stereoscopic 3D visual saliency,” Signal Process. Image Commun. 38, 57–69 (2015).
[Crossref]

Yu, M.

F. Shao, Q. Jiang, R. Fu, M. Yu, and G. Jiang, “Optimizing visual comfort for stereoscopic 3D display based on color-plus-depth signals,” Opt. Express 24(11), 11640–11653 (2016).
[Crossref] [PubMed]

F. Shao, W. Lin, W. Lin, G. Jiang, M. Yu, and R. Fu, “Stereoscopic visual attention guided seam carving for stereoscopic image retargeting,” J. Disp. Technol. 12(1), 22–30 (2016).
[Crossref]

F. Shao, Z. Li, Q. Jiang, G. Jiang, M. Yu, and Z. Peng, “Visual discomfort relaxation for stereoscopic 3D images by adjusting zero-disparity plane for projection,” Displays 39, 125–132 (2015).
[Crossref]

Q. Jiang, F. Shao, G. Jiang, M. Yu, Z. Peng, and C. Yu, “A depth perception and visual comfort guided computational model for stereoscopic 3D visual saliency,” Signal Process. Image Commun. 38, 57–69 (2015).
[Crossref]

F. Shao, W. Lin, G. Jiang, M. Yu, and Q. Dai, “Depth map coding for view synthesis based on distortion analyses,” IEEE J. Em. Sel. Top. C. 4(1), 106–117 (2014).

Zang, D.

D. Li, X. Qiao, D. Zang, L. Wang, and M. Zhang, “On adjustment of stereo parameters in multiview synthesis for planar 3D displays,” J. Soc. Inf. Disp. 23(10), 491–502 (2015).
[Crossref]

Zhang, G. X.

G. X. Zhang, M. M. Cheng, S. M. Hu, and R. R. Martin, “A shape-preserving approach to image resizing,” Comput. Graph. Forum 28(7), 1897–1906 (2009).
[Crossref]

Zhang, M.

D. Li, X. Qiao, D. Zang, L. Wang, and M. Zhang, “On adjustment of stereo parameters in multiview synthesis for planar 3D displays,” J. Soc. Inf. Disp. 23(10), 491–502 (2015).
[Crossref]

Zhang, S. H.

M. Wang, X. J. Zhang, J. B. Liang, S. H. Zhang, and R. R. Martin, “Comfort-driven disparity adjustment for stereoscopic video,” Comput. Visual Media 2(1), 3–17 (2016).
[Crossref]

Zhang, X. J.

M. Wang, X. J. Zhang, J. B. Liang, S. H. Zhang, and R. R. Martin, “Comfort-driven disparity adjustment for stereoscopic video,” Comput. Visual Media 2(1), 3–17 (2016).
[Crossref]

ACM Trans. Graph. (3)

M. Rubinstein, D. Gutierrez, O. Sorkine, and A. Shamir, “A comparative study of image retargeting,” ACM Trans. Graph. 29(6), 160 (2010).
[Crossref]

S. Avidan and A. Shamir, “Seam carving for content-aware image resizing,” ACM Trans. Graph. 26(3), 118 (2007).
[Crossref]

M. Lang, A. Hornung, O. Wang, S. Poulakos, A. Smolic, and M. Gross, “Nonlinear disparity mapping for stereoscopic 3D,” ACM Trans. Graph. 29(4), 75 (2010).
[Crossref]

Anal. Biochem. (1)

P. G. Gottschalk and J. R. Dunn, “The five-parameter logistic: a characterization and comparison with the four-parameter logistic,” Anal. Biochem. 343(1), 54–65 (2005).
[Crossref] [PubMed]

Ann. Telecommun. (1)

M. Urvoy, M. Barkowsky, and P. Le Callet, “How visual fatigue and discomfort impact 3D-TV quality of experience: A comprehensive review of technological, psychophysical, and psychological factors,” Ann. Telecommun. 68(11–12), 641–655 (2013).
[Crossref]

Comput. Graph. Forum (1)

G. X. Zhang, M. M. Cheng, S. M. Hu, and R. R. Martin, “A shape-preserving approach to image resizing,” Comput. Graph. Forum 28(7), 1897–1906 (2009).
[Crossref]

Comput. Visual Media (1)

M. Wang, X. J. Zhang, J. B. Liang, S. H. Zhang, and R. R. Martin, “Comfort-driven disparity adjustment for stereoscopic video,” Comput. Visual Media 2(1), 3–17 (2016).
[Crossref]

Displays (1)

F. Shao, Z. Li, Q. Jiang, G. Jiang, M. Yu, and Z. Peng, “Visual discomfort relaxation for stereoscopic 3D images by adjusting zero-disparity plane for projection,” Displays 39, 125–132 (2015).
[Crossref]

IEEE J. Em. Sel. Top. C. (1)

F. Shao, W. Lin, G. Jiang, M. Yu, and Q. Dai, “Depth map coding for view synthesis based on distortion analyses,” IEEE J. Em. Sel. Top. C. 4(1), 106–117 (2014).

IEEE Signal Process. Lett. (1)

A. De Abreu, P. Frossard, and F. Pereira, “Optimized MVC prediction structures for interactive multiview video streaming,” IEEE Signal Process. Lett. 20(6), 603–606 (2013).
[Crossref]

IEEE Trans. Circ. Syst. Video Tech. (2)

S. S. Lin, C. H. Lin, S. H. Chang, and T. Y. Lee, “Object-coherence warping for stereoscopic image retargeting,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 759–768 (2014).
[Crossref]

H. Sohn, Y. J. Jung, S. Lee, F. Speranza, and Y. M. Ro, “Visual comfort amelioration technique for stereoscopic images: Disparity remapping to mitigate global and local discomfort causes,” IEEE Trans. Circ. Syst. Video Tech. 24(5), 745–758 (2014).
[Crossref]

IEEE Trans. Image Process. (1)

B. Li, L. Y. Duan, C. W. Lin, T. Huang, and W. Gao, “Depth-preserving warping for stereo image retargeting,” IEEE Trans. Image Process. 24(9), 2811–2826 (2015).
[Crossref] [PubMed]

IEEE Trans. Multimed. (1)

C. H. Chang, C. K. Liang, and Y. Y. Chuang, “Content-aware display adaptation and interactive editing for stereoscopic images,” IEEE Trans. Multimed. 13(4), 589–601 (2011).
[Crossref]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

T. Dekel Basha, Y. Moses, and S. Avidan, “Stereo seam carving a geometrically consistent approach,” IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2513–2525 (2013).
[Crossref] [PubMed]

Int. J. Comput. Vis. (1)

T. Yan, R. W. H. Lau, Y. Xu, and L. Huang, “Depth mapping for stereoscopic videos,” Int. J. Comput. Vis. 102(1–3), 293–307 (2013).
[Crossref]

J. Disp. Technol. (2)

F. Shao, W. Lin, W. Lin, G. Jiang, M. Yu, and R. Fu, “Stereoscopic visual attention guided seam carving for stereoscopic image retargeting,” J. Disp. Technol. 12(1), 22–30 (2016).
[Crossref]

J. Lei, S. Li, B. Wang, K. Fang, and C. Hou, “Stereoscopic visual attention guided disparity control for multiview images,” J. Disp. Technol. 10(5), 373–379 (2014).
[Crossref]

J. Imaging Sci. Technol. (1)

M. Lambooij, M. Fortuin, H. Heynderickx, and W. IJsselsteijn, “Visual discomfort and visual fatigue of stereoscopic displays: A review,” J. Imaging Sci. Technol. 53(3), 030201 (2009).
[Crossref]

J. Soc. Inf. Disp. (1)

D. Li, X. Qiao, D. Zang, L. Wang, and M. Zhang, “On adjustment of stereo parameters in multiview synthesis for planar 3D displays,” J. Soc. Inf. Disp. 23(10), 491–502 (2015).
[Crossref]

J. Vis. (1)

D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, “Vergence-accommodation conflicts hinder visual performance and cause visual fatigue,” J. Vis. 8(3), 33 (2008).
[Crossref] [PubMed]

Opt. Express (3)

Proc. IEEE (1)

H. Urey, K. V. Chellappan, E. Erden, and P. Surman, “State of the art in stereoscopic and autostereoscopic displays,” Proc. IEEE 99(4), 540–555 (2011).
[Crossref]

Proc. SPIE (4)

A. K. Moorthy and A. C. Bovik, “A survey on 3D quality of experience and 3D quality assessment,” Proc. SPIE 8651, 86510M (2013).
[Crossref]

W. J. Kim, S. D. Kim, J. Kim, and N. Hur, “Resizing of stereoscopic images for display adaptation,” Proc. SPIE 7237, 72371S (2009).
[Crossref]

Y. Liu, Q. Dai, Z. You, and W. Xu, “Rate-prediction structure complexity analysis for multi-view video coding using hybrid genetic algorithms,” Proc. SPIE 6508, 650804 (2007).
[Crossref]

A. Fiandrotti, J. Chakareski, and P. Frossard, “Popularity-aware rate allocation in multi-view video coding,” Proc. SPIE 7744, 77440Q (2010).
[Crossref]

Signal Process. Image Commun. (1)

Q. Jiang, F. Shao, G. Jiang, M. Yu, Z. Peng, and C. Yu, “A depth perception and visual comfort guided computational model for stereoscopic 3D visual saliency,” Signal Process. Image Commun. 38, 57–69 (2015).
[Crossref]

Other (9)

K. Y. Lee, C. D. Chung, and Y. Y. Chuang, “Scene warping: Layer-based stereoscopic image resizing,” in Proc. of IEEE Computer Vision and Pattern Recognition (2012), pp. 49–56.

Y. H. Huang, T. K. Huang, Y. H. Huang, W. C. Chen, and Y. Y. Chuang, “Warping-based novel view synthesis from a binocular image for autostereoscopic displays,” in Proc. of IEEE International Conference on Multimedia and Expo (2012), pp. 302–307.
[Crossref]

M. S. Farid, M. Lucenteforte, and M. Grangetto, “Depth image based rendering with inverse mapping,” in Proc. IEEE International Workshop on Multimedia Signal Processing (2013), pp. 135–140.
[Crossref]

A. De Abreu, P. Frossard, and F. Pereira, “Fast MVC prediction structure selection for interactive multiview video streaming,” in Proc. Picture Coding Symp. (2013), pp. 169–172.
[Crossref]

K. Utsugi, T. Shibahara, T. Koike, K. Takahashi, and T. Naemura, “Seam carving for stereo images,” in Proc. of 3DTV-Conference: The True Vision -Capture, Transmission and Display of 3D Video (2010), pp. 1–4.
[Crossref]

P. Merkle, A. Smolic, K. Muller, and T. Wiegand, “Multi-view video plus depth representation and coding,” in Proc. IEEE International Conference on Image Processing (2007), pp. 201–204.
[Crossref]

VSRS-1D-Fast. [Online]. Available: https://hevc.hhi.fraunhofer.de/svn/ svn_3DVCSoftware , 2014.

ITU-T P.910, “Subjective video quality assessment methods for multimedia applications,” (1999).

ITU-R BT-500.11, “Methodology for the subjective assessment of the quality of television pictures,” (2012).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1
Fig. 1 Illustration of our proposed MVD retargeting framework.
Fig. 2
Fig. 2 Illustration of camera geometry in different spaces.
Fig. 3
Fig. 3 Example of the 3D saliency map and the depth sensitivity map.
Fig. 4
Fig. 4 Examples of the retargeted depth maps and the corresponding synthesized images with and without depth remapping step.
Fig. 5
Fig. 5 Comparison between our method and the SCL, SSC and SLW on ‘Poznan_Street’ test sequence.
Fig. 6
Fig. 6 Comparison between our method and the SCL, SSC and SLW on ‘Undo_Dancer’ test sequence.
Fig. 7
Fig. 7 Comparison between our method and the SCL, SSC and SLW on ‘Newspaper’ test sequence.

Tables (3)

Tables Icon

Table 1 Main properties of the representative works

Tables Icon

Table 2 Selected views for experiment.

Tables Icon

Table 3 Quantitative subjective assessment results of different MVD sequences.

Equations (21)

Equations on this page are rendered with MathJax. Learn more.

z x,y = 1 D x,y 255 ( 1 z near 1 z far )+ 1 z far
z x,y [ u L v L 1 ]=[ f u 0 u 0 0 f v v 0 0 0 1 ][ r 11 r 12 r 13 t x r 21 r 22 r 23 t y r 31 r 32 r 33 t z ][ X Y Z ]
[ X Y Z ]=[ r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 ] [ f u 0 u 0 0 f v v 0 0 0 1 ] 1 [ u L v L 1 ] z x,y +[ t x t y t z ] = R 1 A 1 1 [ u L v L 1 ] z x,y + T 1
[ u' v' w' ]= A 2 R 2 1 [ X Y Z ] A 2 R 2 1 T 2
d= u V u L
{ X D = d e ( u L + d e 2 ) d e d(W/R) d e 2 Y D = d e v L d e d(W/R) Z D = d e L D d e d(W/R)
E SP ( V k )= ρ k V k V ˜ k 2
ρ k = ( A k T A k ) 1 A k T b ˜ k
A k =[ X k 1 Y k 1 0 1 0 0 Y k 1 X k 1 0 0 1 0 0 0 Z k 1 0 0 1 X k 4 Y k 4 0 1 0 0 Y k 4 X k 4 0 0 1 0 0 0 Z k 4 0 0 1 ], b ˜ k =[ X ˜ k 1 Y ˜ k 1 Z ˜ k 1 X ˜ k 4 Y ˜ k 4 Z ˜ k 4 ]
E SD = V k V S(k) A k ( A k T A k ) 1 A k T b ˜ k b ˜ k 2 = V k V S(k) C k b ˜ k 2
S i (k)= ζ i (k)+α ψ i (k)
Δ( e ^ )= s e e e ^ 2
Δ( e ^ )= e ( e T e) 1 e T e ^ e ^ 2
E LB = v i , v j V (e ( e T e) 1 e T I) e ^ 2
Z max = d e L D d e η 1 L D , Z min = d e L D d e η 2 L D
K= Z max Z min Z D far Z D near
f( Z D )=K( Z D Z D near )+ Z min
E VC = V k V i=1 4 ω( Z k i ) f( Z k i ) Z ˜ k i 2
ω( Z k i )=exp( | Z k i L D | Z D far Z D near )
argmin V ˜ ( E SP + E VC + E LB )
f(D)= β 1 ( 1 2 1 1+exp( β 2 (D β 3 )) )+ β 4 D+ β 5

Metrics