1. Introduction

Optical camera communication (OCC), as a complementary technique for wireless cellular communication, has garnered increasing attention from cutting-edge scholars and industries. With the abundance of large-scale display screens and extensive deployment of cameras in urban infrastructure, it is worth noting that OCC techniques offer the possibility of line-of-sight (LOS) optical communication without the need for additional investment in proprietary communication devices. Not only in the field of public infrastructure, but also in every aspect of people’s work and life, the prevalence of smart devices has empowered and provided wide possibilities for OCC systems [1], and screen-shooting resilient watermarking for IP protection [2]. As a pragmatic implementation of visible light communication (VLC), OCC employs the image sensor (IS) assembled in pervasive consumer devices, such as smartphones, portable personal computers (PCs), iPads, and virtual reality/augmented reality (VR/AR) devices, as transmitters. Off-the-shelf cameras are employed as detectors to collect the distributed light, which conveys spatial light information in terms of system composition [3]. OCC systems have been widely used in various applications, such as vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) in Intelligent Transportation Systems (ITSs) [4,5], multichannel acoustic beamforming measurement via optical wireless microphones [6,7], and electronic health (eHealth) solutions applied to hospitals, pharmacies and homes [8]. In contrast to the screen-camera communication which pixel composite regions can be modulated independently, the LED-camera communication can be more flexible in tuning frame rates. From the hardware perspective, OCC deployment requires only consumer electronic devices with embedded software-defined modulator and decoders, such as scanning a dynamic QR code. From the perspective of vehicle-to-everything (V2X) communication, OCC is expected to become an end-to-end, fast decision-making-based means of V2V and V2I collaboration for future 6G V2X and industrial internet of vehicles (IoV) applications, and it offers viable solutions for eye-free headlight to vehicle-embedded commercial camera V2X communications.

In view of relative standardization, the collaboration of the IEEE 802.15.7m optical wireless communication (OWC) task group (TG7m) has witnessed technical contributions with OWC solutions, including the OCC baseband modulation techniques [9]. Since 2016, the TG7m committee has approved the supplementary of the IS-based OWC physical layer (PHY) operating modes technical proposals (from PHY IV to PHY VIII). Specifically for newly added PHY VI modes in TG7m, the asynchrony between senders and detectors are partially considered and primarily intended for use in video dislay scenarios, including the hidden asynchronous quick link (HA-QL), asynchronous quick link (A-QL), variable transparent amplitude-shape-color (VTASC), sequential scalable two-dimensional color (SS2DC) and the invisible data embedding (IDE) algorithms modulation schemes [10]. Nevertheless, the theoretical framework has not yet been rigorously analyzed and OCC performance is still limited by low frame rate [11], nonlinear distortions [12], asynchrony [13], and RoI tracking issues [14], which still urgently call for in-depth investigation.

Over the past decade, many off-the-shelf cameras integrated with complementary metal-oxide-semiconductor (CMOS) IS have lacked a global shutter (GS) mode, leading to pixels being exposed at different times across various scanlines within the camera chip. For high-frame-rate (HFR) cameras used in manufacturing assembly lines, the rolling shutter (RS) mode is preferred over the inside GS mode. However, the use of RS mode can cause geometric distortions, especially when either the camera or the object being photographed is in motion, as compared to the use of GS mode inside the camera. Multiple nonlinear factors, including aperture size, exposure time, frame rate, imaging optical pathway, moving object tracking, ambient noises, and shutter timing, can impact the exposure level and imaging quality for built-in IS. After real-time video processing, the mechanical structure can automatically adjust the aperture size and exposure time window to regulate the incident light, enabling smart cameras to achieve high dynamic range (HDR) imaging and enhance the visual experience. Asynchronous problems have received comparatively less research attention. Nevertheless, a dedicated and computationally-efficient asynchronous mitigation scheme is indispensable to recover each independent frame for subsequent data decoding. The asynchrony between smart displays and IS-based cameras is a typical issue that arises from the absence of synchronous clock between optical senders and decoders. Moreover, random variations in the oscillation frequency of each crystal and temperature drift, camera internal noise, GS and RS mechanism are crucial factors that affect bottom-level OCC synchronizations [15,16]. To deal with such issues, in [13], the heterochronous frame detection algorithm utilizes the maximum-likelihood sequence detection (MLSD), which is implemented by the Viterbi algorithm, and the pixel combining weights from adjacent frames are separated by the modified maximal-ratio combining (MRC) detection algorithm. In [5], two asynchrony schemes for OCC-based infrastructure-to-vehicle (I2V) communication are proposed to cancel the effect of the exposure time and mitigate the variation in the camera frame rate during each sampling operation, and it allows the optical detectors to select proper neighbouring frames for decoding under a fixed oversampling rate or varied oversampling rate using pre-set reference light-emitting diodes (LEDs). Performance evaluation shows the feasibility for use in future OCC-based wireless communication applications and services in vehicular communication scenarios. In [17], the variational Bayesian Gaussian-mixture model (VB-GMM) and the maximum-likelihood sequence detection method are employed to estimate the channel and synchronization parameters, achieving a communication data rate of approximately 10 kbps. In [18], an effective time-sharing-based technique with visual-MIMO characteristics named the generalized color modulation (GCM) based visual-MIMO is proposed to overcome the asynchrony issue and maintain the color uniformity under a camera frame rate of 30 fps where the human eye cannot detect flickering for color-independent V2X communication. The basic architecture and overview of the OCC-based V2V and V2X communication systems is shown in Fig. 1. However, large-scale time-synchronized symbols are embedded in the data structure, which on the other hand wastes frame cycles to transmit information symbols. Although the above studies have provided algorithms and schemes for asynchronous correction to varying degrees, for the asynchronous problem of RS-mode cameras, the asynchronous overlap ratio varies with the electronic scanning position, resulting in different overlap ratios for different shooting frame positions. This increases the difficulty of solving and recovering asynchronous crosstalk.

 

Fig. 1. The basic architecture and overview of the OCC-based V2V and V2X communication systems.

Download Full Size | PDF

High-order statistics (HOS) have been widely utilized in various fields, such as modulation format recognition, channel estimation, blind source separation, and multi-user detection. Specifically, HOS can be used to identify different pulse amplitude modulation (PAM) and quadrature amplitude modulation (QAM) orders due to their distinguishing features. To achieve super-Nyquist optical transmission, we have fully utilized the characteristics of HOS to mitigate subband overlapping. Specifically, we utilize the decision-feedback-kurtosis-minimum (DFKM) [19] and optimal skewness-based criteria (OSBC) inter-band interference (IBI) mitigation algorithm [20], which have been validated to mitigate the crosstalk from adjacent carriers at the symbol level. Among all HOS, kurtosis is a statistical measure that quantifies the degree of peakedness or flatness of a probability distribution, and is calculated as the fourth-order statistical moment. It provides information about whether a distribution has more extreme or outlier values than a normal distribution. Higher kurtosis values correspond to more outlier values, while lower kurtosis indicates fewer ones. By leveraging the difference between regular signals and noise, the HOS technique can be used to design adaptive filter estimators, such as the variable step-size least-mean-square (LMS) algorithm [21], the 2nd- and 4th-order hybrid multiuser equalizer [22], and the kurtosis-based complex independent component analysis (CICA) scheme [23].

Gamma distortion has found globally utility in imaging systems, serving not only to enhance overall display performance, but also in applications such as image dehazing [24], fringe-projection profilometry [25], and image enhancement [26,27]. In color image acquisition system, light response for the independent R, G, and B can be transformed by a nonlinear curve, i.e., the gamma distortion. In 1990s, the standardization of high-definition television (HDTV) spawned the creation for basic parameters of the first version ITU-R BT.709 [28,29]. Initially, these R, G, B transformed signals are utilized to compensate for the nonlinearities of cathode ray tubes (CRT) used in television sets and computer monitors to obtain a linear response on viewing device as perceived by human user. However, it may lead to nonlinear shift of pixel color and misleading the object recognition, edge detection and classification for automatic image processing algorithms. In V2V and V2X scenarios, the on-board camera serves multiple purposes, including providing safety alerts when parked, supplying visual information for autonomous driving, and potentially carrying out the work of OCC in the near future. However, the Gamma correction algorithm embedded in on-board cameras introduces nonlinearity and results in higher pixelated intensity modulated bits error rate.

In this paper, we propose the kurtosis-based asynchronous interference cancellation (K-AIC) algorithm for synchronizing temporally superposed pixelated data captured by RS-mode cameras. It is capable of recovering temporally adjacent and superposed LED array frames captured by cameras, offering low complexity and benefiting from wide frame overlapping ratio and Gamma correction compatibility. Initially, we formulate the LOS I-MIMO channel OCC system model, introduce the challenge of asynchronous OCC system problem, and provide detailed theoretical derivations of the state-of-the-art K-AIC optimization scheme. Subsequently, numerical simulations are conducted across various inter-frame overlapping ratios and Gamma distortion levels to verify the global effectiveness and performance enhancement performance of K-AIC. Furthermore, the LED-matrix-to-camera (LM2C) OCC experiment is performed to validate the temporally crosstalk mitigation and Gamma correction performance with frame group BER and Q-factor as metrics. Furthermore, the 32$\times$32 LED array utilized in this paper represents the maximum number of LED elements selected for experimental investigation based on our comprehensive literature review.

2. Theoretical framework

In this subsection, we commence by establishing a theoretical model for the LOS LM2C OCC system. Then, the primary focus then shifts to the physical model encompassing temporal inter-frame crosstalk and Gamma distortion. Subsequently, we provide a comprehensive derivation of the state-of-the-art low-complexity K-AIC frame algorithm based on high-order statistics convex optimizations. Furthermore, extensive numerical analyses are conducted under varied asynchronization misalignment, Gamma distortion levels and optical signal-noise-ratio (OSNR) conditions. The top-level overview of the OCC system, along with the frame data processing procedure, is depicted in Fig. 2.

 

Fig. 2. The block diagram of the asynchronous OCC system with the main offline data processing steps, including the state-of-the-art K-AIC algorithm.

Download Full Size | PDF

2.1 Optical camera communication system model

To initiate our analysis of the LOS LM2C OCC system, we make certain assumptions. Specifically, we model it as an imaging multiple-in-multiple-out (I-MIMO) system comprising independently modulated LED elements with flexible shapes, and IS-based commercial cameras serving as optical transmitters and detectors. Then, we assume a short-range free-space communication scenario, where the OCC medium is free from significantly disruptive factors such as LOS refractions, reflections, and other non-line-of-sight (NLOS) attenuation factors. This simplification of the optical channel allows us to focus on the core issue of asynchronous OCC.

Since LED elements are current-driven to produce light through the recombination of injected holes and electrons in P-N junction, the LED luminous intensity is ordinarily controlled by the current flowing through it. Here we assume that the relationship between LED driving current $I(t)$ and the optical power $P(t)$ can be simplified as the linear relationship $P(t)=kI(t)$, where $k$ is simplified as the linear coefficient factor that characterized by the inherent LED physical properties. It should be emphasized that although LED current and emitted optical power are not linear over full range, we can fully leverage the linear region to achieve high efficiency transmission. Furthermore, we assume that each LED element can be modulated independently to maximize the throughput. An equiprobable PAM-$n$ data stream is generated, linearly transformed, and matrix-reshaped based on the LED matrix size. This resulting matrix can be denoted as $\mathbf {X}_k$ with $AB$ elements, and its individual element marked as $x_k(a,b)$, representing the $k^{th}$ frame PAM-$n$ data located in the $a^{th}$ row and $b^{th}$ column. Under these assumptions, the instantaneous LED optical power can be expressed as:

Referring to [30,31], we employ the point spreading function (PSF) to mimic the spatial optical power distribution. By adjusting $\sigma ^2$, PSF offers varying diffusion degrees to describe the light extending from the imaging center of the LED to edges or fringes. Subsequently, we let the bold letter $\mathbf {Z}$ as an imaging area, which contains X$\times$Y imaging pixels. Specifically, the PSF is convolved with the intensity-modulated optical signals to get the imaged signals on the IS, and it is more often applied to estimate the channel impulse response (CIR) and subsequent equalization to mitigate the effect of distortions. To specify and characterize the location of pixels, we utilize indices $x$ and $y$ as unique identifiers. Finally, the optical spatial distribution PSF $g_{x,y}$ can be written as:

Nevertheless, since the PSF model treats the luminous LED as the point light source, the lens mounted on the LED chip with desired light distribution is not given sufficient consideration. To provide more detailed optical channel path model, many researchers have established and adopted the imaging optical transmission models under varied communication application scenarios to achieve higher precision channel estimation, such as LOS, NLOS [32], fog and atmospheric turbulence links, scattering-dominant links [33,34], and indoor and outdoor links [30]. In this paper, we use more universal and generalized LOS channel as the fundamental imaging OCC channel. Without sacrificing generality, the Lambertian-pattern LED matrix radiant intensity distribution is identified as the LED irradiance field, and we let $H_{los}(x,y)$ be the LOS I-MIMO channel that projected light onto pixel location $(x,y)$, which can be given as:

Based on Eq. 6, it can be tentatively concluded that the received optical power $P_{k'}(x,y;t)$ is determined by camera shutter mode, exposure time period, photo-electrical conversion efficiency, and the generalized feature for imaging optical channels. Particularly for RS-mode cameras, the unique IS exposure mechanism captures each frame sequentially by scanning row-by-row, instead of capturing the entire frame at once. This IS mechanism leads to complex asynchrony conditions, which makes synchronization more challenging. The captured optical power by IS pixel during the exposure time-slot ranges from $t$ to $t+T$ can be detected in forms of photon stream $I_t$. Based on above definitions, the relationship between $I_t$ and $P_{k'}(x,y;t)$ can be expressed as:

2.2 Kurtosis-based asynchronous interference cancellation scheme

From a systemic perspective, we assume that the IS-based camera can capture the entire LED emitting panel on the focal plane, ensuring the structured light can be well projected when frame-switching occurs simultaneously on the IS-based camera. Under such assumption, we define $n$(an integer with a positive power of two) as the number of distinguishable optical intensity levels for LED elements. Therefore, $M$ bits are conveyed when $2^M=N$ under pulse amplitude modulated $N$-level (PAM-$n$) signaling. Then, we define $F$ as the manually set frame refresh rate (FRR) for both LED matrix and camera. Although the same frame rate is set on both sides, the specific clock phase and frequency will drift slightly. Rigorously speaking, it is necessary here to classify the asynchronized status into three types, i.e., the mesochronous, plesiochronous and heterochronous types. The mesochronous type refers to two signals that have the same average frequency but differ only in their phases. The plesiochronous type refers to two signals that have nominally the same average frequencies but differ slightly. Unlike the above types, the heterochronous type refers to two signals that have nominally different average frequencies. Compared to the above two types, the plesiochronous type is most likely to occur when the same frame rates are set for LED matrix and camera, but the actual optical emitting interval may slightly differ from the camera’s IS exposure time-slot due to factors such as crystal oscillator temporal drift, mechanical shutter response delay variation. To address this issue, the K-AIC algorithm is specifically designed to mitigate asynchrony, ensuring high efficient and low cross-talk data transmission. Based on the above principle derivation, the LM2C OCC system transmission data rate is $C=\frac {ABM}{T}$, where A and B denote the number of rows and columns in the LED matrix, and M denotes the number of consecutive frames used for batch processing. The detailed information for K-AIC will be introduced in the following section. Here we assume that each LED element has the same electro-optical conversion and luminous efficiency, and the kurtosis of the modulated PAM data in the $k^{th}$ LED frame $\mathbf {X}_k$ can be expressed as:

 

Fig. 3. The overview of the asynchronous OCC system with the RS mechanism. (a) Physical effect of blocking and through for RS on the IS plane; (b) relationship between the IS exposure time-delay and IS row indices; (c) received frame mixing cuboid space, in which adjacent elements in the adjacency frame overlapping ratio matrix $\mathbf {M}$ is in superposed state by RS camera; (d) the varied frame misalignment conditions between LED matrix and row-wise IS.

Download Full Size | PDF

Table 1. Normalized kurtosis for uniform PAM-$n$ and Gaussian signals

View Table | View all tables in this article

Since $\alpha$ is primarily dependent on the frame index $k$ and the corresponding LED matrix row index $a$ in RS mode, we define $\alpha (a,k) \in S, S \in [0,1]$ as a metric to quantify the contribution of neighboring frames to the IS plane over the RS opening period. In Fig. 3(d), the alphabet $\tau$ denotes the RS exposure period, which satisfy $\tau \leq T$. Conceptually, time interval $\tau$ is similar to duty cycle in pulse width modulation (PWM), and can be adjusted to control the total luminous flux that exposed on IS. In Fig. 3(d), exposure cycle $\tau$ can be divided into two segments: before and after the transmit frame switching. Specifically, we define the start of the IS’s exposure as time zero, and the LED display frame changes at time $\alpha (a,k)\tau$. Furthermore, we define the consecutive imaging matrix $\boldsymbol {Y}_{(AKB)}$ as the concatenation and permutation of the individual captured frames ${\mathbf {Y}_1,\mathbf {Y}_2,\ldots,\mathbf {Y}_K}$, and we have:

When a camera working in RS shutter mode, each row pixel is scanned in sequential over time to fill each image. Therefore, we assume the exposure time interval within each scanline is instantaneous. Building on such assumption, we define the transmitted vector $\mathbf {x}_{a,k}$ as extracted from $\boldsymbol {X}_{(AKB)}$, which has a total $B$ elements. Then, we adopt $\mathbf {x}_{a,k}$ as the fundamental transmitter calculation unit, and share the same overlapping ratio $\alpha (a,k)$ through the row pixels $\mathbf {x}_{a,k}$. Correspondingly, we use $\mathbf {y}_{a,k}$ as the basic demixing vectors for IS receiver. During the simulation and experimental processes, it should be noted that the row index $k$ can be flexibly selected within a continuous range corresponding to the LED matrix row indices. The overall trade-off between the number of rows per frame and the estimation precision of $\alpha$ is established before solving the convex optimization problem. Therefore, the linear combination, specifically the superposed relationship for independent and identically distributed (i.i.d) structured light and IS captured grayscale frames can be built. When the asynchronous OCC works in plesiochronous condition, $\alpha (a,k)$ with adjacent position (both in temporally adjacent and row indices adjacent directions) can be numerically close, and the frame mixing coefficients matrix $\mathbf {M}$ can be utilized to characterize the variation of the frame superposition state. Based on the plesiochronous assumptions, $\mathbf {M}$ should be a upper or lower 1-band matrix, with non-zeros entries in the main diagonal and one of the adjacent main diagonal (super- or sub-diagonal) elements. The determination of whether the superposed matrix $\mathbf {M}$ is an upper triangular matrix or a lower triangular matrix hinges on whether the phase relationship between the emitted frame and the IS array detected frame is leading or lagging.

Based on the above derivation, theoretical framework of the Gamma distorted plesiochronous OCC system model is established. To obtain the minimized row-by-row local kurtosis coefficient $\alpha (a,k)$, here we first arrange the temporally consecutive transmitted frames under the same row index $a$ together, which can be denoted by $\mathbf {F}=[\mathbf {x}_{a,1},\mathbf {x}_{a,2},\ldots,\mathbf {x}_{a,K}]$. Similarly, we let $\mathbf {G}=[\mathbf {y}_{a,1},\mathbf {y}_{a,2},\ldots,\mathbf {y}_{a,K'}]$ as the consecutive received imaging frames with $a$ denotes the row index. For the model completeness and rigour, we let $\mathbf {H}_{los}$ be the free-space I-MIMO spatial optical intensity grayscale response matrix, which has the same size as LED matrix. Under such assumptions, a temporally consecutive Gamma distorted asynchronous OCC model can theoretically be expressed as:

It is noteworthy to emphasize that the convex optimizer K-AIC can be start by estimating $\hat {\alpha }$ and $\hat \gamma$ in the joint linear and nonlinear parameter space from the initially received first two frames after the first frame demapped, which is denoted as $\mathbf {y}_{a,k-1}^{\frac {1}{\hat {\gamma }}}$. Specifically, the estimation process for the initial parameters of $\hat {\alpha _1}$ and $\hat {\gamma _1}$ can be represented by:

Upon obtaining the first $\hat \alpha$ and $\hat \gamma$ through K-AIC, we incorporate these parameters into Eq. 14 to derive the estimate of the subsequent frames with grayscale level Euclidean-distance-based demapper, which serves as the known PAM-$n$ sequence in recovering the following frames. Numerically, since the system works in plesiochronous condition, the optimal $\hat \alpha _k$ and $\hat \gamma _k$ can be quite close with the frame crosstalk being separated incrementally. With the progression of K-AIC for current frame group, the frame overlapping matrix $\mathbf {M}$ and $\gamma _k$ can be sequentially estimated. Finally, as the matrix $\mathbf {M}$ is non-square, we utilize the Moore-Penrose pseudoinverse denoted as $\mathbf {M}^+$ to determine $\mathbf {x}$ such that $\mathbf {x=M^+y}$ within the grayscale space $\mathbf {F}$ by minimizing the L2-norm of the residual $\left \lVert \mathbf {MF-G}\right \rVert _2$. By utilizing the spectral theorem and Tikhonov’s regularization method [35], the Moore-Penrose pseudoinverse $\mathbf {M^+}$ can be computed through the following equation:

Having provided a detailed theoretical derivation of the K-AIC algorithm, it is imperative to conduct a comprehensive evaluation of the computational complexity. Given that the recovery of inter-frame crosstalk involves two distinct stages, namely the K-AIC optimization algorithm for solving $\alpha -\gamma$ stage, and the Moore-Penrose pseudo-inverse for recovering the interference-free frame stage [36–38]. To sum up, the computational complexity $\mathbf {C}$ of these two stages can be written as:

3. Numerical simulations

In the first place, lateral comparisons are conducted for the camera recorder operating in RS- and GS-mode under the I-MIMO channel, as derived in Eq. 5. The comparison is based on their respective shutter mechanism, as illustrated in Fig. 4. In Fig. 4(a), (b), the generated two PAM-2 frames are individually detected by IS with no neighbouring frame interference and no shutter mode issue introduced. Compared to the upper figure, Fig. 4(c), (d) illustrates the condition that the successive two frames are superposed in a single frame received by IS under RS- and GS-mode, respectively. In the examination of shutter modes and their corresponding optical spatial distributions, it is observed that the RS-shutter mode manifests multiple intensity levels throughout the entire IS pixel grid, as opposed to the GS-shutter mode, which exhibits only two intensity levels. Comparative analysis indicates that the timeslot misalignment specific to row pixels in RS-mode is inherently more intricate.

 

Fig. 4. The spatial optical distribution for RoI detection in an IS-based camera. (a) Spatial optical distribution detected from the preceding frame transmitted by the LED matrix; (b) the spatial optical distribution measured from the subsequent frame sent by LED matrix; (c) spatial optical distribution falling on the IS plane for the camera operating in RS-mode with $\alpha$ ranging from 0.2 to 0.4 to simulate the shutter moving mechanism; (d) spatial optical distribution falling on IS plane under $\alpha =0.3$ for the camera operating in GS-mode.

Download Full Size | PDF

In addition to qualitative analysis of the asynchronous optical intensity distribution, we conduct quantitative analysis under various degrees of Gaussian noise and inter-frame crosstalk ratios. In the case of transmitters, LED elements are modulated with equiprobable PAM-4 signals independently, denoted as $\mathbf {x}_{a, k} \in [1, 3, 5, 7]$. In Fig. 5(a), we extract and compile peak grayscale levels detected on the IS through image feature extraction under perfectly synchronized conditions (i.e., $\alpha =0$). Fig. 5(b) presents the grayscale distribution for asynchronous OCC under $\alpha =0.5$, where each camera frame comprises two consecutive transmitter frames with a precisely maintained 5:5 duty cycle. In this scenario, the peak grayscale levels of the LED elements imaged can be achieved as $\mathbf {x}\in [1, 2, 3,\ldots, 7]$ with a non-uniform probability distribution. To enhance the evaluation, we introduce a nonlinear Gamma distortion factor $\gamma$, and the imaging peak grayscale levels are accumulated, with the resulting histogram shown in Fig. 5(c). It is imperative to emphasize that, in addressing the asynchronous nature of OCC systems, numerical simulation does not take into account factors such as high dynamic range, white balance, and image jitter(which refers to the small, random variations or oscillations in the position of an image during its capture or display).

 

Fig. 5. Probability density distribution of the peak grayscale levels projected from the structured light generated by the LED matrix. The probability density distribution under simulation conditions: (a) the precisely synchronized case; (b) the asynchronous case with $\alpha =0.5$; (c) the asynchronous case with $\alpha =0.5$ accompanied by nonlinear Gamma distortion $\gamma =2.4$.

Download Full Size | PDF

The interdependence among signal-noise-ratio (SNR), $\alpha$ and normalized kurtosis is investigated in Fig. 6 for PAM-$n$ ($n$=2, 4, 8). It is evident that inter-frame overlapping significantly impairs the PAM-$n$ signal, leading to the superposed PAM signals adopting a distribution closely resembling a Gaussian profile, where the normalized kurtosis approaches zero. By horizontally comparing Fig. 6(a), (b) and (c), it is apparent that, within PAM-$n$, an increase in PAM order leads to a corresponding rise in normalized kurtosis. The minimum normalized kurtosis can be observed in PAM-2 signal transmission with SNR=30 dB. In contrast, the normalized kurtosis corresponding to the central vertical line at $\alpha =0.5$ represents the maximum value under the same SNR conditions, irrespective of PAM order $n$. Furthermore, it can be readily inferred that, under the condition of $\alpha =0.5$, with an increase in PAM order, the comprehensive distribution of inter-frame crosstalk tends to approach a Gaussian distribution.

 

Fig. 6. The normalized kurtosis distribution in joint SNR-$\alpha$ parameter space. (a) PAM-2; (b) PAM-4; (c) PAM-8.

Download Full Size | PDF

Subsequently, the distribution for normalized kurtosis across the joint $\alpha -\gamma$ space is investigated, aiming to observe and analyze the unique minimum kurtosis. To evaluate the overall effectiveness and uniqueness of the convex optimization K-AIC scheme, the multi-frame PAM-4 signal streams are generated, and we preset $\gamma =2.4$ and $\alpha =0.5$ to mimic a generalized Gamma nonlinearity along with frame overlapping asynchronous OCC model. Constrained by space limitations, we opt for the centrally positioned PAM-4 among PAM-2, PAM-4, and PAM-8 as the primary modulation scheme. This choice is predominantly based on the fundamental theory of the K-AIC convex optimization scheme, which demonstrates that the kurtosis of framed overlapping signals is higher than that of non-overlapping PAM signals. To verify the effectiveness and uniqueness of the K-AIC scheme, numerical simulations were performed for PAM-2 and PAM-8 signals, and the results align with the conclusions outlined for PAM-4. Then we employ Eq. 12 to compute the kurtosis for each intersection point in $\alpha -\gamma$ parameter space. Specifically, an extensive traversal of the joint $\alpha -\gamma$ parameter space is performed by initializing $\mathbf {F}$ and $\gamma$ with equi-probable PAM-4 signal under SNR=20 dB. Results reveal that Eq. 12 exhibits a unique minimum in the joint $\alpha -\gamma$ space, as shown in Fig. 7(a). Consequently, through the K-AIC optimization algorithm with several iterations, the state-of-the-art scheme can successfully search for the optimal $\alpha$ and $\gamma$, and finally obtain the corresponding $\mathbf {M^+}$ to recover the linear overlapping and nonlinear distortion contaminated PAM signals. Moreover, it is essential to evaluate and analyze the convergence efficiency of the K-AIC scheme to illustrate its practical computational complexity. In Fig. 7(b), the iterative convergence performance of the K-AIC optimizer is comprehensively examined for PAM-4 signals with SNR values of 12, 15, 20, 30 dB, respectively. It is noteworthy that, under various SNR conditions, the K-AIC scheme typically converges within a maximum of three iterations to determine the optimal values of $\hat {\alpha }$ and $\hat \gamma$ .

 

Fig. 7. (a) Normalized kurtosis of the PAM signal contaminated by joint linear and nonlinear factors in an asynchronous OCC system under $\alpha =0.5$ and $\gamma =2.4$. (b) The iterative convergence performance of the convex optimization K-AIC algorithm under varied SNR conditions.

Download Full Size | PDF

The comprehensive numerical simulations of the K-AIC scheme under diverse SNR conditions are extensively assessed, utilizing the bit-error-rate (BER) as the metric for evaluating both linear and nonlinear interference factors, as illustrated in Fig. 8(a). It can be clearly seen that the probability distribution of grayscale values reflect a significant change in the grayscale distribution before and after employing the K-AIC scheme. This transformation is primarily attributed to the combined effects of Gaussian noise, grayscale range, adjacent frame overlapping, and Gamma distortion. The key numerical simulation parameters for generating the grayscale distribution are summarized in Table 2. To be specific, we simulate the SNR at 12, 15, and 18 dB, with the linear interframe overlapping parameter $\alpha$ varying from 0 to 0.5 and the Gamma distortion parameter $\gamma$ configured as 1 (no nonlinearity), 1.8, 2.7 (with nonlinearity) for a comprehensive analysis. Results indicate that the K-AIC convex optimization algorithm has the capability to equalize and compensate for both linear grayscale frame overlaps and nonlinear distortions. Moreover, we systematically explore the parameter space with $\alpha =[0,0.1,0.2,0.3,0.4,0.5]$ and $\gamma =[1,1.2,1.5,1.8,2.1,2.4,2.7]$, utilizing a PAM-4 signal within SNR ranges spanning from 12 to 19 dB. This investigation aims to validate the ten-frame averaged BER and Q-factor ($Q_{dB}$) performance across a comprehensive set of 42 conditions for each SNR value, encompassing various challenging asynchronous communication scenarios. Here we utilize $Q_{dB}$ as the SNR in decibels (dB), which quantifies the quality of a digital communication system’s performance in the presence of noise. The relationship between $Q_{dB}$ and BER can be written as:

Results demonstrate that the K-AIC scheme efficiently leverages the kurtosis convex optimization strategy to search for the optimal values of $\alpha,\gamma$ with a Q-factor standard deviation (SD) of less than 0.5 dB across all SNR cases, as shown in Fig. 8(b). This ensures remarkable consistency and robustness in compensating for both linear and nonlinear effects in asynchronous OCC systems.

 

Fig. 8. (a) Numerical simulations on varied $\alpha$ and $\gamma$ conditions when SNR=12, 15, 18 dB, and the RoI grayscale distributions before and after the K-AIC scheme. (b) Q-factor analysis for SNR ranges from 12 to 19 dB through traverse $\alpha$ and $\gamma$ conditions with the SD Q-factor distribution in boxplot form.

Download Full Size | PDF

Table 2. The parameter sets utilized to depict the grayscale distribution in Fig. 8(a).

View Table | View all tables in this article

For the theoretical assessment of the computational complexity associated with the proposed K-AIC scheme, a linear relationship with the iteration time is observed. Consequently, we compile the iteration counts corresponding to kurtosis convex optimization under diverse simulation parameters. Specifically, simulations are conducted for SNR values of [12, 15, 18], $\alpha$ ranging from 0 to 0.5 in increments of 0.1, and $\gamma$ values of [1, 1.5, 2, 2.5]. The averaged iteration counts for successive asynchronous 9-frame scenarios are presented in Fig. 10. The results indicate that, generally, the iteration counts increase with an increase in $\alpha$. This suggests that the two consecutive frames are captured sequentially, with exposure times that are relatively close rather than significantly disparate.

Finally, we compare the lateral Bit Error Rate (BER) performance between the Higher Order Statistics (HOS)-based K-AIC and the VB-GMM scheme. We consider a configuration with 1024 LED elements ($A \times B$), SNR values of 14 and 18 dB, respectively, and $\gamma$ ranging from 1 to 2.7. Additionally, we set $\alpha$ to 0.2 and 0.5 to simulate various asynchronous conditions. The simulation results are presented in Fig. 9. The comparison indicates that the Viterbi-based scheme demonstrates comparable performance to the K-AIC scheme. However, as the Gamma distortion increases, the non-uniform Euclidean-distance among the received grayscale levels for PAM-4 misguides the sequential detection for VB-GMM. Consequently, this leads to an increase in the frame-averaged BER.

 

Fig. 9. The iteration counts of the convex-optimization-based K-AIC scheme under varied SNR cases. (a) SNR=12 dB; (b) SNR=15 dB; (c) SNR=18 dB.

Download Full Size | PDF

 

Fig. 10. The lateral comparison of frame-averaged BER between the K-AIC and VB-GMM scheme. (a) $\alpha =0.2$; (b) $\alpha =0.5$.

Download Full Size | PDF

4. Experimental demonstrations

An OCC system experimental setup is implemented to assess the enhancement in practical asynchronous communication performance achieved through the utilization of the K-AIC scheme, as shown in Fig. 11. To begin with, LED matrix is configured using surface-mount LED board, forming a 32$\times$32 LED array. For real-time frame-by-frame display, an LED-driving component is employed to establish a connection between the computer graphics card and the LED board. Specifically, we employ an FPGA-embedded LED driving board to synchronize the change in illumination states for all LED elements upon the arrival of the clock.

 

Fig. 11. Experimental setup of the short-range indoor OCC system.

Download Full Size | PDF

Pseudo-random bit sequences are generated with PAM signals, and then converted bit-wise to grayscale images, which are subsequently transformed into video clips. Subsequently, the 32$\times$32 resolution video is played and exhibited in real-time, maintaining a one-to-one correspondence with the LED array, ensuring no spatial crosstalk. The LED display frame rate is set to 120, and the distance between the LED matrix and the camera is 4.5 m. For the off-the-shelf camera, we choose the 4K high-dynamic-range (HDR) camcorder SONY FDR-AX700. This camera is equipped with a 1-inch stacked Exmor reverse stacked CMOS sensor, featuring tunable RS- and GS- modes. To regulate the luminous flux, the internal neutral density (ND) filter is adjusted to 1/32 or 1/64 to prevent overexposure for IS. The frame rate for camera is also set to 120 to simulate the plesiochronous type OCC transmissions. Based on the aforementioned experimental parameter settings, it is evident that, at a frame rate of 120 and an actual data frame rate of approximately 98, the effective data rate of the OCC system is estimated to be around 200 kbps.

In the context of frame data format, a dedicated LED-array frame format is designated, incorporating two benchmark frame patterns to streamline real-time perception and full-frame compensation of spatial light intensity distribution, illustrated in Fig. 12(a)(b). More precisely, the benchmark frame serves to identify the exact position of each individual LED element projected onto the imaging sensor, enabling subsequent data capture at precise pixel positions. Based on the computational complexity analysis of the proposed algorithm as presented in Eq. 15, the length of consecutive overlapping frames should not be excessively long to avoid heavy pseudo-inverse calculations. In this context, we set the length of consecutive data frames to nine frames per group to balance the computational complexity and the data transmission rate.

 

Fig. 12. Group frame structure and the comparative quantitative analysis between with and without the K-AIC scheme. (a) Benchmark pattern A; (b) benchmark pattern B; (c) inter-frame overlap and Gamma distortion when the camera works in RS mode; (d) camera captured data frame containing two temporally adjacent PAM-2 data frames; (e) measured average group BER (in light blue) and Q-factor performance (in orange) by with and without the K-AIC algorithm (represented by square and circle shape, respectively).

Download Full Size | PDF

Qualitative and quantitative analyses are accomplished after extensive spatial light camera communications. It is crucial to emphasize that, prior to numerical analysis, cross-correlation is essential for identifying whether a frame is a benchmark frame or a data frame. After benchmark frames are precisely captured, individual extraction of data frames, characterized by linear inter-frame overlaps and nonlinear distortions, are performed, as shown in Fig. 12(c)(d). The group frame BER and Q-factor serve as fundamental metrics. We extract and analyze 450 consecutive frames (approximately 40 group frames) in video footage with offline image processing. Results show that the K-AIC algorithm can effectively mitigate the real-time plesiochronous transmission. Numerically, the Q factor yields a gain of approximately 13 dB with the state-of-the-art K-AIC scheme when comparing with direct decoding case, as shown in Fig. 12(e). Moreover, under the condition of relatively uniform optical intensity modulation send to the LED array, the grayscale distribution of light pixels in the imaging region-of-interest (RoI) pixels exhibits a nonlinear distribution due to the nonlinear effects of electro-optical conversion and gamma distortion. The experimentally measured Gamma nonlinearity is approximately 1.8. Moreover, due to the prevalence of extensive consecutive asynchronous transmissions in this OCC experiment, there is a proficient exploration of diverse $\alpha$ for LED array and camera. Consequently, the validation of this experiment underscores its universal effectiveness in practical short-distance OCC scenarios. Notably, the Q-factor results reveal fluctuations of less than 1 dB.

It is crucial to emphasize that, the more complex and dynamic real-word environment, such as V2V scenarios, the optical channel can manifest greater complexity. Addressing image jitter resulting from dynamic focusing on the LED board requires the integration of real-time image processing techniques along with a camera gimbal for effective tracking. In a preliminary outlook, with the extension of communication distances, adjacent LEDs may experience blurring, leading to partial overlap of brightness information from neighboring elements. In our future work, we aim to explore the design of spatial diversity and multiplexing schemes tailored for the more challenging OCC conditions to enhance optical imaging communication capabilities.

5. Conclusions

In this paper, we address synchronization challenges in OCC scenarios, including Vehicle-to-Vehicle (V2V), Internet of Vehicles (IoV), and high-speed underwater VLC (UVLC) OCC scenarios. We employ the state-of-the-art K-AIC algorithm to achieve synchronization between the LED array and the camera, thereby ensuring high reliability and robustness in transmissions. Extensive simulations and short-range OCC experimental demonstrations comprehensively validate the enhancement of asynchronous communication performance, resulting in Q-factor gain of 13 dB compared to the non-K-AIC compensation case. Furthermore, both simulation and experimental results indicate Q-factor fluctuations of less than 0.5 dB and 1 dB, respectively, confirming the consistency and robustness of the proposed scheme.