## Abstract

By making use of the changes in optical properties such as absorption and dispersion around the resonance generated via electromagnetically induced transparency (EIT), we theoretically and experimentally investigate a “∞”-shape optical bistability (OB) versus frequency on the probe transmission with a Λ-shape EIT window in a rubidium atomic ensemble confined in a three-mirror optical ring cavity. Compared to the traditional OB reflected by a hysteresis loop versus power, such newly demonstrated optical bistable behavior (represented by a “∞”-shape non-overlapping region) by scanning probe and cavity detuning can experience dual bistabilities and be more sensitive to the change of experimental parameters. Further, we study the relationship between vacuum Rabi splitting and the “∞”-shape OB. Such study on frequency-induced OB could effectively improve the applications related to OB such as logic-gate devices and optical information processing.

© 2017 Optical Society of America

## 1. Introduction

The optical bistability (OB) in a composite cavity-atom system has been extensively demonstrated experimentally and theoretically in the past three decades due to its promising applications in all-optical devices including switches, memories elements, and transistors [1,2]. Generally, by confining a multi-level atomic ensemble into an optical ring cavity, the bistable behavior can be obtained on the cavity transmission within a definite range of input intensities [3], which can be attributed to the enhancement of nonlinear optical properties of the atomic medium, such as the intra-cavity dark state [4] and Kerr nonlinearity [5,6]. Actually, intensity change of the input field can modulate the refractive index of the intra-cavity atoms and results in the shift of the energy on the optical mode sustained in the cavity, which can provide a feedback as an essential factor for the generation of bistability [7]. In past few years, OB with wide controllability are thoroughly studied in both three- and four-level atomic configurations by displaying a hysteresis cycle as a function of the intensity of incident driving field [8,9]. Particularly, a multi-level atomic configuration with electromagnetically induced transparency (EIT) [10,11] is a desirable platform to demonstrate the OB effect by considering the advantages including the multi-parameter controllable nature and the dramatically enhanced/modified dispersion and nonlinearity [2,12]. Also, the OB behavior versus frequency is demonstrated by a hysteresisloop in an atomic configuration involving Rydberg excitation [13], where the feedback is due to resonant dipole-dipole interactions instead of an optical cavity. In the meanwhile, the dynamical behaviors in a composite atom-cavity system with controllable linear and nonlinear optical properties are extensively studied. For example, the vacuum Rabi splitting (VRS) [14] characterized as a double-peaked transmission spectrum and associated with “bright polaritons” can be effectively modulated as two pairs of peak under the EIT regime due to the modified intra-cavity dispersion properties [15], which can make the atom-cavity interaction achieve the “superstrong-coupling” condition with the atom-cavity coupling strength manipulated to be close or larger than the cavity free spectral range [16,17].

In this paper, we theoretically and experimentally investigate a “∞”-shape OB effect on the transmitted probe signal with a ladder-type EIT window (induced by a probe field and a coupling field) in a ^{85}Rb atomic ensemble confined in a three-mirror optical ring cavity, which essentially provides a dressing feedback effect to the cavity output. The classical hysteresis loop representing the traditional OB is observed by scanning the input intensity of the cavity. Also, a new kind of dual bistablilty is demonstrated through a two-dimension non-overlapping region (different frequency in the horizontal *x* direction and different intensity in the longitudinal *y* direction) on the transmitted spectrum versus the detuning of cavity or probe field. The lineshape of the non-overlapping region can be approximately viewed as the symbol “∞”, so we name this new kind of OB as “∞”-shape OB. The non-reversibility on frequency is caused by the feedback dressing effect from the cavity while the intensity difference is caused by the different requirements for suppression and enhancement. Experimentally, the degree of the “∞”-shape OB are controlled by the intensities of the probe and coupling fields, and the frequency detuning of the cavity and probe field. Finally, by comparing the size of non-overlapping regions of the OB versus detuning and the traditional one versus input power, we conclude that the “∞”-shape OB is more sensitive to the change of experimental parameters. Further, the observations show that the “∞”-shape OB can become more obvious when the VRS is larger, which initially indicates the relationship between OB and VRS. Such study on the nonreversible OB effect could effectively improve the applications arising from the atomic OB effect such as all-optical devices and optical information processing [8].

## 2. Experimental setup

The experimental study is implemented in an optical ring cavity filled with thermal ^{85}Rb atomic vapor as shown in Fig. 1(a). The 39 cm long cavity is formed by a plane mirror M3 (with a reflectivity of 97.5% at 780 nm) and two plate-concave mirrors M1 (99.9% at 780 nm) and M2 (97.5% at 780 nm) with the same curvature radius of 100 mm. The distance between M1 and M2 is approximately 13 cm. A long stroke PZT is mounted on the back of M1 to adjust and control the cavity length. Also, the feedback electronic signal for holding the cavity length is also added onto the cavity through the PZT [18]. The 7cm long atomic vapor cell (placed between M2 and M3) is wrapped with *μ*-metal sheets and heated by a heater tape to set the temperature of the cell as 60°C to provide an atomic density of 2.5 × 10^{11} cm^{−3}. The horizontally polarized probe beam *E*_{1} enters the cavity by passing through M2 after shaped by a convex lens L1. Also, an electro-optic modulator (EOM) is applied to continuously modifying the input intensity to the cavity by adding a triangle wave with a frequency of 20Hz to the driver of EOM. The probe filed propagates through the polarization beam splitter (PBS), cell and then is reflected by M1, M3 and M2 to form a circular route. Besides, the transmission of M3 is received by an avalanche photo diode (APD) to receive the cavity modes. The vertically polarized coupling beam *E*_{2} is coupled into the center of cell by a high-reflectivity mirror M4 and reflected out the cavity by the PBS. In this setup, *E*_{1} field counter-propagates with *E*_{2} field so that we can obtain an EIT window in the probe field with Δ_{1} + Δ_{2} = 0 satisfied. Here Δ* _{i}* = Ω

*−*

_{i}*ω*is the detuning between the resonant transition frequency Ω

_{i}*and the laser frequency*

_{i}*ω*of

_{i}

*E**. Figure 1(b) shows the atomic energy level diagram. Beam*

_{i}

*E*_{1}(frequency

*ω*

_{1}, wavevector

*k*_{1}, Rabi frequency

*G*

_{1}) probes the lower transition 5S

_{1/2}↔5P

_{3/2}, and beam

*E*_{2}(

*ω*

_{2},

*k*_{2},

*G*

_{2}) couples the transition 5P

_{3/2}↔5D

_{3/2}. Here the Rabi frequency between |

*i*〉↔|

*j*〉 is defined as

*G*=

_{i}*μ*/

_{ij}E_{i}*ħ*(

*i*= 1, 2…), where

*E*is the electric field of laser

_{i}

*E*_{i}and

*μ*is the dipole momentum. Figure 1(c) demonstrates the experimentally observed cavity modes (lower curve) by scanning the frequency detuning Δ

_{ij}_{ac}(Δ

_{ac}= Ω

_{1}-

*ω*with

_{c}*ω*set as the resonant frequency of the cavity) of the cavity with a repetition of 10Hz. In the current experiment, the scanning speed (tunable between 10Hz to 20Hz) can have no clear influence on the generated spectrum signals as well as the obtained OB behaviors. The upper triangle wave represents the scanning signal with two periods Ramp1 and Ramp2. There exist a rising edge and a falling edge in one scanning ramp. The lower curve is the observed cavity mode by scanning the cavity length over two periods. Obviously, the signal peaks generated on the rising (falling) edge of Ramp1 and the falling (rising) edge of Ramp2 are different in both intensity and frequency. The lower peak is the cavity mode corresponding to rising edge (with a feedback intensity of

_{c}*I*

_{up}) while the higher one is the cavity mode from the falling edge (with a feedback intensity of

*I*

_{down}). Actually, different feedback in the rising edge and decreasing edge of a scanning ramp can lead to the nonequilibrium output.

## 3. Basic theory

#### 3.1 The transmitted probe field with cavity feedback dressing

Actually, the nonequilibrium OB response of the transmitted probe field in the current system is introduced by the cavity feedback dressing effect. First, with only beam *E*_{1} turned on, the perturbation chain corresponding to the transition |0〉↔|1〉 is${\rho}_{00}{}^{\left(0\right)}\stackrel{\omega 1}{\to}{\rho}_{\text{1}0}{}^{\left(\text{1}\right)}$ (*ρ _{ij}* is the density matrix element for transition |

*i*〉↔|

*j*〉) and the first-order density matrix element

*ρ*

_{10}

^{(1)}is written as

*d*

_{1}is defined as

*d*

_{1}= Γ

_{10}+

*i*Δ

_{1}, where Γ

*= (Γ*

_{ij}*+ Γ*

_{i}*)/2 is the decoherence rate between |*

_{j}*i*〉 and |

*j*〉 and Γ

*is the transverse relaxation rate. With both probe and coupling fields turned on to establish a ladder-type three-level configuration, the first-order density matrix element with the dressing effects from*

_{i}

*E*_{1}and

*E*_{2}is given aswhere term

*d*

_{2}is defined as

*d*

_{2}= Γ

_{20}+

*i*(Δ

_{1}+ Δ

_{2}).

Then, the probe signal with dressing effects circulates inside the ring cavity and we can obtain the cavity modes, which indicates that the cavity couples with the atomic ensemble with a strength of$g\sqrt{N}$, where *g* and *N* represent the single-atom-cavity coupling strength and the atomic population, respectively. According to the weak cavity field limitation and the assumption that all the atoms initially locate at the ground state, the output cavity modes and the density matrix elements obey the following linear relations (derived from the master equation in the atom-cavity coupling picture):

*t*;

*γ*is the decay rate for the cavity;

*a*is the cavity field,

*G*

_{T}is intensity of cavity feedback. As a result in the steady state, with

*d*

_{c}written as

*d*

_{c}=

*γ*+

*i*(Δ

_{1}−Δ

_{ac}), the cavity mode generated by the dressed probe field can be given as

*I*

_{o}of the output cavity mode and the intensity

*I*

_{i}of the input field are proportional to

**|**

*a*

**|**

^{2}and

**|**

*G*

_{1}

**|**

^{2}respectively. Consequently, we have

*g*

^{2}

*N*/

*d*

_{c}has the similar format as the dressing term

**|**

*G*

_{2}

**|**

^{2}/

*d*

_{2}. Therefore,

*g*

^{2}

*N*/

*d*

_{c}can be considered as a vacuum induced dressing effect from the cavity. Such sequential-cascade type dressing structure [20] can be viewed as a coherent superposition of the two dressing processes. The output term

*I*

_{o}on the right-hand side of Eq. (5) represents the feedback dressing effect to the cavity transmission. Essentially, the feedback can result in the nonreversible OB behavior characterized as the non-overlapping region (different in both frequency and intensity) between two signals from the rising and falling ramps by continuously modifying Δ

_{1}or Δ

_{ac}.

#### 3.2 Nonreversible bistability in the “∞”-shape non-overlapping regions

Similarly to the proposed concept of vacuum induced transparency [21], the vacuum induced nonequilibrium responses are caused by the atom-cavity coupling strength $g\sqrt{N}$. The OB behavior of zero-order mode in the composite atom-cavity system can be understood through master equation formalism theory [22]. The generated OB is visually reflected by the size of non-overlapping region between the signal lineshapes in the frequency-increasing and frequency-decreasing processes (corresponding to the rising and falling edges in one frequency scanning round trip, respectively), where the respective feedback intensities *I*_{up} and *I*_{down} are different due to the combined dressing effects. Such non-overlapping area can map the degree of nonreversible bistability quantificationally by comparing the difference (in frequency and intensity) between the spectra on the rising and falling edges for scanning the frequency detuning (Δ_{1} and Δ_{ac}).

On one hand, for the frequency difference by scanning the detuning, the nonreversibility can be interpreted by the change of nonlinear refractive index Δ*n'*, which is given as

*σ*= Δ

*υn*

_{1}

*l*/

*c*is the phase delay and Δ

*υ*is the frequency difference that can reflect the OB phenomenon directly.

*n*

_{2up}(

*n*

_{2down}) is the nonlinear refractive index coefficient corresponding to

*I*

_{up}(

*I*

_{down}). Here the Kerr nonlinear coefficient

*n*

_{2}can be generally expressed as

*n*

_{2up}≈

*n*

_{2down}≈

*n*

_{2}= Re[

*χ*

^{(3)}/(

*ε*

_{0}

*cn*

_{0})], which is the mainly dominated by field

*E*_{2}. The nonlinear susceptibility is

*n*′ =

*Nn*

_{2}(

*I*

_{up}−

*I*

_{down}) can be modulated by manipulating the parameters in the expression of ${\tilde{\rho}}_{10}^{(3)}$.

On the other hand, the intensity difference of the generated cavity modes at the frequency-rising and -falling edges can also reflect the OB effect. Physically, the intensity difference of the cavity modes can be understood through the interpretation that the requirements for suppression/enhancement are different for the rising and the falling edges.

In the current system, the primary dressing state |*G*_{2 ±} 〉 is split by field *E*_{2}, and corresponding eigenvalues are *λ* _{±} = [∆_{2} ± (∆_{2}^{2} + 4|*G*_{2}|^{2})^{1/2}]/2. So the suppression and enhancement conditions are ∆_{1} + ∆_{2} = 0 and ∆_{1} + ∆_{2} + [∆_{2} ± (∆_{2}^{2} + 4|*G*_{2}|^{2})^{1/2}]/2 = 0, respectively. The secondary dressing splitting is caused by the term *G _{T}*, whose eigenvalues are

*λ*

_{+ ±}= [∆

_{2}′

^{2}± (∆

_{2}′

^{2}+ 4|

*G*

_{T}|

^{2})

^{1/2}]/2 (∆

_{2}′ = ∆

_{2}−

*λ*

_{+}). Consequently, the suppression and enhancement conditions are modified as ∆

_{1}+ ∆

_{2}′ = 0 and ∆

_{1}+ ∆

_{2}′ + [−∆

_{2}′

^{2}± (∆

_{2}′

^{2}+ 4|

*G*|

_{T}^{2})

^{1/2}]/2 = 0 (∆

_{1}+ λ

_{±}+

*λ*

_{+ ±}= 0), respectively. Similarly, the suppression and enhancement conditions for the case of scanning ∆

_{ac}can be expressed as ∆

_{ac}+ ∆

_{2}′ = 0 and ∆

_{ac}+ ∆

_{2}′ + [−∆

_{2}′

^{2}± (∆

_{2}′

^{2}+ 4|

*G*|

_{T}^{2})

^{1/2}]/2 = 0 (∆

_{ac}+ λ

_{±}+

*λ*

_{+ ±}= 0), respectively. Such difference (caused by different

*G*) in conditions for suppression/enhancement can lead to the intensity difference at the rising edge and falling edge.

_{T}What’s more, the cavity mode spectrum versus the probe frequency detuning can exhibit two peaks, which can be interpreted as the so called VRS effect. The VRS of the composite system with a frequency distance of $2g\sqrt{N}$ is derived from the atom-cavity coupling effect, and determined mainly by the self-dressing term${g}^{2}N$. According to the multiple dressed-state theory for explaining the VRS phenomena [22], the two eigenvalues corresponding to dressed states $|\pm \u3009$ induced by term ${g}^{2}N$ are${\lambda}_{\pm}={\Delta}_{ac}\pm \sqrt{{\Delta}_{ac}^{2}+4{\left|g\sqrt{N}\right|}^{2}}/2$. As a result, the frequency distance of VSR can be given as

For the VRS phenomenon, term *g*^{2}*N* serves as the atom-cavity coupling effect and the dressing cavity field. The OB behavior in the atom-cavity system results from the self-Kerr nonlinearity and is closely related the self-dressing effect of |*G*_{T}|^{2}, which has a similar influence with the self-dressing effect of *g*^{2}*N* and the internal-dressing effects of |*G*_{2}|^{2}. Term *g*^{2}*N* can establish the internal relationship between OB and VRS. Physically, the evolutions of VRS in frequency domain and output-input relationship both result from the change of the absorption and dispersion properties of the atoms.

## 4. Results and discussions

Figure 2 experimentally shows the traditional OB by scanning the power of the incident field and the nonreversible OB by scanning the length of the cavity. For the traditional case, the widely known hysteresis cycles [8] are observed as Figs. 2(a1)-2(a4), where every curve represents the cavity mode by scanning the probe power *P*_{1} at different power *P*_{2} of the coupling field. The right (left) curve in each figure is obtained by continuously increasing (decreasing) *P*_{1}. There exists a square non-overlapping region S_{OB} when we fold the two mode curves. Here, the OB results from the self-dressing effect of |*G*_{T}|^{2}/Γ_{00} in Eq. (4) and can be reflected by the area size of S_{OB}, which is mainly determined by the power difference between the two thresholds. The distance dependence of thresholds on *P*_{2} is shown in Fig. 2(a5), which demonstrates that the OB effect can be strengthened with the decrease of coupling-field power. This is most likely caused by the enhancement of the Kerr nonlinearity in the three-level atomic ensemble when we change *P*_{2} to modify term |*G*_{2}|^{2}/*d*_{2} [24].

Further, we obtain the nonreversible bistability by scanning the cavity length and folding the generated cavity modes along the central dashed line of the triangle wave for scanning ∆_{ac}. As shown in Figs. 2(b1)-2(b4), by scanning ∆_{ac} over two periods, the transmitted probe signals can have frequency difference and intensity difference, which are explained as the nonreversible bistability. Compared to the normal OB in Fig. 2(a), this new kind of OB by scanning detuning can be reflected via a “∞”-shape non-overlapping region (S_{NB}), which is composed of two non-overlapping regions S_{n1} and S_{n2}. We can also name the “∞”-shape OB as dual optical bistability (DOB).

The red signal (with a higher left peak) shows the transmitted signal from Ramp1 and the blue one (with a higher right peak) shows the signal from Ramp2. To be specific, the observed four peaks in Fig. 2(b1), from left to right, correspond to *I*_{up} of Ramp1, *I*_{down} of Ramp2, *I*_{down} of Ramp1 and *I*_{up} of Ramp2, respectively. Considering of the symmetry in scanning triangle wave as well as the output behaviors, we just analyze the pair of the partially overlapped cavity modes (*I*_{up} of Ramp1 and *I*_{down} of Ramp2) at ∆_{ac}<0.

Figures 2(b1)-2(b4) show the observed DOB versus Δ_{ac} by discretely decreasing the power *P*_{2} of the coupling field. In Fig. 2(b1), on one hand, the two observed peaks at ∆_{ac} = −157.9 MHz and −126.3 MHz represent the enhancement of the cavity mode induced by self-dressing term *G*_{1} and the cavity polariton from term *g*^{2}*N*/*d*_{c} in Eq. (5). The transmission peak reaches the maximum value when Δ_{ac} = 0 and Δ_{1}−Δ_{ac} = 0 are simultaneously satisfied. However, the feedback from the output field at rising and falling regions are different in signal intensity and the third-order nonlinear coefficient *n*_{2}, and such difference can result in the frequency offset of the generated cavity modes. Because of the difference on the feedback term |*G _{T}*|

^{2}/Γ

_{00}for

*I*

_{up}and

*I*

_{down}in Eq. (4), we can conclude Δ

*n'*≠0 according to Eq. (6), which can lead to the frequency shift in DOB.

The frequency distance of two signal peaks increases from 33.0MHz to 56.3MHz when we attenuate *P*_{2} from 20mW to 5mW. The change of *P*_{2} can directly affect the dressing term |*G _{2}*|

^{2}/

*d*

_{2}, which further imposes influence on the third-order nonlinear coefficient

*n*

_{2}according to${n}_{2}\propto {\tilde{\rho}}_{10}^{(3)}$. To be specific, the decrease of

*G*

_{2}can lead to the growing of

*n*

_{2}, which can render frequency offset become bigger according to Eq. (7), where term (

*I*

_{up}-

*I*

_{down}) is a constant. The larger frequency shift means the more efficient OB and the dependence curve is shown in Fig. 2(b4), where the solid curve is the fitting simulation and the triangles represent the experimental observations.

On the other hand, the observed double peaks at ∆_{ac}<0 also have a clear difference in the intensity (along the longitudinal axis), which comes from the different requirements for suppression/enhancement. The two partially overlapped peaks can form a “∞”-shape region. Actually, both the two peaks meet the enhancement condition ∆_{ac} + ∆_{2}′ + [−∆_{2}′^{2} ± (∆_{2}′^{2} + 4|*G _{T}*|

^{2})

^{1/2}]/2 = 0. However, the dressing-state level on the falling edge is closer to the original level than that on the rising one, which means the falling-edge case can meet the enhancement condition better than the rising-edge one. So, the difference of term |

*G*|

_{T}^{2}in Eq. (4) and the enhancement condition at different scanning ramps are responsible for the intensity difference between two peaks. With the decreasing of

*P*

_{2}, the intensity of the cavity mode decreases because of the variation of term $|{G}_{2}{|}^{2}/{d}_{2}$ in Eq. (4). Further, the intensity difference between the left and right peaks gradually increases as shown in Figs. 2(b1)-2(b4). Here the changing trend is essentially consistent with that of frequency offset. By comparing the results versus frequency and power, it can be easily observed that S

_{NB}= (S

_{N1}+ S

_{N2})>S

_{OB}at the same

*P*

_{2}, which indicates that the nonreversible OB is more sensitive to ∆

_{ac}than the traditional OB to

*P*

_{1}. Actually, the DOB is easier to be observed than classical OB, which can also advocate the “∞”-shapeOB is more sensitive.

Figure 3 demonstrates the nonreversible OB effect versus the cavity detuning ∆_{ac} and probe detuning ∆_{1} at different power *P*_{1}. The observed cavity modes in Fig. 3(a1) are generated by scanning ∆_{ac}. According to Figs. 3(a1)-3(a4), we find that the height of transmitted peak and the frequency shift can obviously increase with power *P*_{1}. The growth of the transmission can be attributed to the increase of dominated numerator term *G*_{1} in Eq. (4) by considering *G*_{1}∝*P*_{1}. Also, the “∞”-shape nonreversible OB effect, which including the frequency shift along the transverse axis and intensity difference along the longitudinal axis, can also get enhanced by increasing *P*_{1}. The enlargement (from 0 to 0.35) of the intensity difference can be understood through Eq. (4), where cavity output *I*_{o} can increase with *G*_{1}. The growth indicates that the cavity feedback term *G _{T}* proportional to

*I*can grow with

_{o}*G*

_{1}and the OB behavior can be more obvious at larger

*G*

_{1}. Further, based on Eq. (6), we can understand that the frequency shift Δ

*υ*can increase with

*n*

_{2}, which is consistent with the observations. The stronger

*G*

_{1}can induce stronger nonlinearity, namely, stronger

*n*

_{2}. The frequency distance increases from 31.1MHz to 57.7MHz on the whole when

*P*

_{1}increases from 0.2mW to 0.8mW. Figure 3(a5) shows the measured (squares) frequency shift at different

*P*

_{1}according to Figs. 3(a1)-3(a4), and the solid curve is the simulation.

Figures 3(b1)-(b4) investigate the “∞”-shape OB versus the frequency detuning ∆_{1} by increasing power *P*_{1}. In Fig. 3(b1), the cavity modes appear at the positions (∆_{1} = −138.2MHz, −105.7MHz, 83.7MHz and 102.4MHz) where the enhancement condition ∆_{1} + ∆_{2}′ + [−∆_{2}′^{2} ± (∆_{2}′^{2} + 4|*G _{T}*|

^{2})

^{1/2}]/2 = 0 is satisfied. These peaks can also be enhanced by dressing terms

*g*

^{2}

*N*/

*d*

_{c}and |

*G*

_{2}|

^{2}/

*d*

_{2}in Eq. (4) when ∆

_{1}−∆

_{ac}= 0 and the two-photon condition ∆

_{1}+ ∆

_{2}= 0 are simultaneously achieved.

In Figs. 3(b1)-3(b4), the double-peak phenomenon in a single ramp is the so-called VRS caused by the cavity dressing effect. From left to right, the first peak (at ∆_{1} = −138.2MHz) and the fourth peak (at ∆_{1} = 102.4MHz) are the signals from the rising edge while the second (at ∆_{1} = −105.7MHz) and third ones (at ∆_{1} = 83.7MHz) are from the falling edge. Obviously, the OB effect (mainly the frequency shift) reflected by peak1 and peak2 can increase with *P*_{1}, which is similar to that reflected by peak3 and peak4. Taking the two peaks at ∆_{1}<0 for instance, the increase of *P*_{1} can result in the enhancement of the nonlinearity as well as term Δ*υ* according to Eq. (6) and Eq. (7). The difference between *I*_{up} and *I*_{down} is caused by the feedback term |*G*_{T}|^{2}/Γ_{00}. Here the intensity difference it not obvious, which is most likely caused by the interaction between the two dressing terms |*G*_{2}|^{2}/*d*_{2} and |*G*_{T}|^{2}/Γ_{00} in Eq. (5). Such interaction may partially balance the intensity difference caused by the feedback. By comparing the results versus cavity detuning ∆_{ac} and probe detuning ∆_{1} at the same *P*_{1}, one can find that the nonreversible OB can be more sensitive to scanning ∆_{ac} than to ∆_{1}.

Figure 4 illustrates the evolution of the output cavity mode versus *P*_{1} and ∆_{1} by discretely changing ∆* _{ac}* and reveals the relationship between traditional OB and VRS by observing the evolution of the hysteresis cycle.

For the case of scanning power shown in Figs. 4(a1)-4(a3), the normal OB is visually reflected by the size of S_{OB} (the hysteresis cycle in each figure). Based on the denominator term *d*_{c} = *γ* + *i*(Δ_{1}−Δ_{ac}) in Eq. (4), the increase of ∆_{ac} can enhance the output intensity, which indicates the feedback term |*G _{T}*|

^{2}/Γ

_{00}can increase with ∆

_{ac}and further induce a stronger OB effect. The size dependence of the OB hysteresis cycle on ∆

_{ac}according to Figs. 4(a1)-4(a3) is demonstrated in Fig. 4(a4), where the squares are the experimental results and the solid curve is the simulation. These OB behaviors are observed with Δ

_{1}fixed at the left peak in Figs. 4(b1)-4(b3), where the curves represent the cavity mode versus detuning ∆

_{1}at different ∆

*. In Fig. 4(b1) with ∆*

_{ac}*= 0, there are only two bright states at ∆*

_{ac}_{1}= −44.6MHz and 36.8MHz. Here the two bright states are generated by cavity dressing term

*g*

^{2}

*N*/

*d*, which can induce two first-level dressing states | ± 1〉 as well as VRS. Then the dark state appears between the two bright states at ∆

_{c}*= 10MHz and ∆*

_{ac}*= 20MHz, and the three states successively correspond to ∆*

_{ac}_{1}= −56.3MHz, −4.8 MHz and 42.5 MHz in Fig. 4(b2) and ∆

_{1}= −62.0MHz, 19.9MHz and 59.1MHz in Fig. 4(b3). The additional dark-state peak is caused by the second-level dressing states, which indicates the level | ± 1〉 is dressed as | + ± 1〉 and |− ± 1〉 by the coupling field

*E*_{2}. Consequently, the one dark state and two bright states result from the combined effects of the vacuum cavity field and the coupling field, which is described by term |

*G*

_{2}|

^{2}/

*d*

_{2}+

*g*

^{2}

*N*/

*d*in Eq. (4). By tuning ∆

_{c}*from 0 to 20MHz*

_{ac}_{,}the resonance condition ∆

_{1}−∆

*= 0 is fully meet and the intensity of the transmitted peaks can be greatly enhanced at ∆*

_{ac}*= 20MHz. In the meanwhile, the modification of ∆*

_{ac}*can also move the first-level dressing states, one of which can be secondly split by field*

_{ac}

*E*_{2}(with a fixed detuning ∆

_{2}= 0). Consequently, the middle peak can appear either at the position of ∆

_{1}>0 or ∆

_{1}<0, which can be explained as the field

*E*_{2}interacts with the split level | + 1〉 or |−1〉, respectively.

Figure 4(b4) shows that the frequency difference of two bright-state peaks can change from 75.0 MHz to 127.8 MHz and indicates that VRS becomes more pronounced with larger ∆_{ac}. To show the VSR more clearly, we rearrange the transmission spectra in Figs. 4(b1)-4(b3) from top down in Fig. 4(c). Theoretically, according to Eq. (8), the difference *λ*_{+}−*λ*_{−} (frequency distance) between the two eigenvalues (*λ*_{+} and *λ*_{−}, corresponding to dressed states induced by term *g*^{2}*N*) can increase with ∆_{ac} and term *g*^{2}*N* at ∆_{ac}>0, which is consistent with the experimental observations.

Furthermore, we experimentally verify that the OB and VRS have the similar changing trend by increasing ∆_{ac}, namely, the more obvious VRS is, the more pronounced OB becomes, which can be easily understand as the dressing term *g*^{2}*N*/*d*_{c} (with *d*_{c} = *γ* + *i*(Δ_{1}−Δ_{ac})) can grow with Δ_{ac}. According to Eq. (4), the increase of *g*^{2}*N*/*d*_{c} can induce a stronger output *I*_{o}, which can provide stronger feedback *G*_{T} as well as more obvious OB behavior_{.} Such observations support that there exists a cascaded relationship between term *I*_{o} and term *g*^{2}*N* proposed according to right-hand side of Eq. (7) [21].

## 5. Conclusion

In summary, we have investigated the comparison between the “∞”-shape OB versus the detuning (of cavity or probe field) and traditional OB phenomena versus the input power on the cavity modes of the probe field in a three-level cavity-atom composite system both experimentally and theoretically. Our work shows detailed study on the comparison between the two kinds of OB by settling different experimental parameters. As a result, the “∞”-shape OB are more sensitive to the change of parameters than the normal OB. What’s more, the VRS and the normal OB simultaneously resulting from the significant change of the absorption and dispersion characteristics of the medium can experience the same evolution tendency with the variation of cavity detuning. Our research could improve the applications derived from the normal OB in an atomic system such as building more efficient all-optical switches and logic-gate devices and quantum information processing [8].

## Funding

National Natural Science Foundation of China (NSFC) (61605154, 11474228), China Postdoctoral Science Foundation (2016M600776), and Key Scientific and Technological Innovation Team of Shaanxi Province (2014KCT-10).

## References and links

**1. **H. M. Gibbs, S. L. McCall, and T. N. C. Venkatesan, “Differential gain and bistability using a sodium-filled Fabry-Perot interferometer,” Phys. Rev. Lett. **36**(19), 1135–1138 (1976).

**2. **A. Joshi and M. Xiao, “Atomic optical bistability in two- and three-level systems: perspectives and prospects,” J. Mod. Opt. **57**(14–15), 1196–1220 (2001).

**3. **H. Wang, D. J. Goorskey, and M. Xiao, “Bistability and instability of three-level atoms inside an optical cavity,” Phys. Rev. A **65**(1), 011801 (2001).

**4. **G. Hernandez, J. Zhang, and Y. Zhu, “Vacuum Rabi splitting and intracavity dark state in a cavity-atom system,” Phys. Rev. A **76**(5), 053814 (2007).

**5. **S. Gupta, K. L. Moore, K. W. Murch, and D. M. Stamper-Kurn, “Cavity nonlinear optics at low photon numbers from collective atomic motion,” Phys. Rev. Lett. **99**(21), 213601 (2007). [PubMed]

**6. **H. Wang, D. Goorskey, and M. Xiao, “Enhanced Kerr nonlinearity via atomic coherence in a three-level atomic system,” Phys. Rev. Lett. **87**(7), 073601 (2001). [PubMed]

**7. **A. Joshi and M. Xiao, “Optical bistability in a three-level semiconductor quantum-well system,” Appl. Phys. B **79**(1), 65–69 (2004).

**8. **A. Joshi, A. Brown, H. Wang, and M. Xiao, “Controlling optical bistability in a three-level atomic system,” Phys. Rev. A **67**(4), 041801 (2003).

**9. **X. Lü, J. Li, J. Liu, and J. Luo, “Optical bistability via quantum interference in a four-level atomic medium,” J. Phys. At. Mol. Opt. Phys. **39**(24), 5161–5171 (2006).

**10. **Z. Zhang, Y. Zhang, J. Sheng, L. Yang, M. A. Miri, D. N. Christodoulides, B. He, Y. Zhang, and M. Xiao, “Observation of parity-time symmetry in optically induced atomic lattices,” Phys. Rev. Lett. **117**(12), 123601 (2016). [PubMed]

**11. **J. Gea-Banacloche, Yq. Li, Sz. Jin, and M. Xiao, “Electromagnetically induced transparency in ladder-type inhomogeneously broadened media: theory and experiment,” Phys. Rev. A **51**(1), 576–584 (1995). [PubMed]

**12. **A. Brown, A. Joshi, and M. Xiao, “Controlled steady-state switching in optical bistability,” Appl. Phys. Lett. **83**(7), 1301–1303 (2003).

**13. **C. Carr, R. Ritter, C. G. Wade, C. S. Adams, and K. J. Weatherill, “Nonequilibrium phase transition in a dilute Rydberg ensemble,” Phys. Rev. Lett. **111**(11), 113901 (2013). [PubMed]

**14. **Y. Zhu, D. J. Gauthier, S. E. Morin, Q. Wu, H. J. Carmichael, and T. W. Mossberg, “Vacuum Rabi splitting as a feature of linear-dispersion theory: Analysis and experimental observations,” Phys. Rev. Lett. **64**(21), 2499–2502 (1990). [PubMed]

**15. **H. Wu, J. Gea-Banacloche, and M. Xiao, “Splitting of atom-cavity polariton peaks for three-level atoms in an optical cavity,” Phys. Rev. A **80**(3), 033806 (2009).

**16. **X. Yu, D. Xiong, H. Chen, P. Wang, M. Xiao, and J. Zhang, “Multi-normal-mode splitting of a cavity in the presence of atoms: A step towards the superstrong-coupling regime,” Phys. Rev. A **79**(6), 061803 (2009).

**17. **X. Yu and J. Zhang, “Multi-normal mode-splitting for an optical cavity with electromagnetically induced transparency medium,” Opt. Express **18**(5), 4057–4065 (2010). [PubMed]

**18. **Z. Zhang, H. Zheng, Y. Tian, H. Chen, S. Yang, and Y. Zhang, “Stable scanning of dressing fields on multiwave mixing in optical ring cavity,” IEEE J. Quantum Electron. **50**(7), 575–580 (2014).

**19. **H. Wu, J. Gea-Banacloche, and M. Xiao, “Observation of intracavity electromagnetically induced transparency and polariton resonances in a doppler-broadened medium,” Phys. Rev. Lett. **100**(17), 173602 (2008). [PubMed]

**20. **Z. Nie, H. Zheng, P. Li, Y. Yang, Y. Zhang, and M. Xiao, “Interacting multiwave mixing in a five-level atomic system,” Phys. Rev. A **77**(6), 063829 (2008).

**21. **H. Tanji-Suzuki, W. Chen, R. Landig, J. Simon, and V. Vuletić, “Vacuum-induced transparency,” Science **333**(6047), 1266–1269 (2011). [PubMed]

**22. **J. Yuan, W. Feng, P. Li, X. Zhang, Y. Zhang, H. Zheng, and Y. Zhang, “Controllable vacuum Rabi splitting and optical bistability of multi-wave-mixing signal inside a ring cavity,” Phys. Rev. A **86**(6), 063820 (2012).

**23. **Y. Zhang, Z. Nie, H. Zheng, C. Li, J. Song, and M. Xiao, “Electromagnetically induced spatial nonlinear dispersion of four-wave mixing,” Phys. Rev. A **80**(1), 013835 (2009).

**24. **H. Wang, D. Goorskey, and M. Xiao, “Dependence of enhanced Kerr nonlinearity on coupling power in a three-level atomic system,” Opt. Lett. **27**(4), 258–260 (2002). [PubMed]