Red, green, blue (RGB) light-emitting-diodes (LEDs) are used to increase the visible light communication (VLC) transmission capacity via wavelength-division-multiplexing (WDM), and the color image sensor in mobile phone is used to separate different color signals via a color filter array. However, due to the wide optical bandwidths of the color filters, there is a high spectral overlap among different channels, and a high inter-channel interference (ICI) happens. Here, we propose and demonstrate an RGB VLC transmission using CMOS image sensor with multi-input multi-output (MIMO) technique to mitigate the ICI and retrieve the three independent color channels in the rolling shutter pattern. Data pattern extinction-ratio (ER) enhancement and thresholding are deployed.
© 2016 Optical Society of America
As there is a shortage of radio-frequency (RF) spectrum for the traditional wireless communications, it is believed that wireless communication using visible light, known as visible light communication (VLC) can become one of the promising candidates for the 5th generation (5G) wireless communication  as well as for the internet of things (IOT) applications . VLC [3–7] can be integrated with the light-emitting-diode (LED) illuminance systems; hence it can be a value-added service for the lighting industries. VLC is very directional; hence little interference occurs for the neighboring channels. This is an important feature to increase the wireless communication capacity needed for the 5G. For the last decade, the complementary-metal-oxide-semiconductor (CMOS) cameras have been widely used in mobile-phone and vehicles. It is interesting to use these cameras as the receivers (Rxs) for VLC since the VLC deployment cost can be reduced. Although the data rate is limited by the CMOS cameras, it may be enough for sending secure ID, authorization information, or positioning data in indoor navigation system. Ref . demonstrated using tailor-made car camera with specific high-speed photo-diode (PD) pixels for VLC and low-speed pixels for imaging. Ref . demonstrated using red-green-blue (RGB) camera for 60 m long VLC transmission, suitable for car-to-car communications. However its data rate is limited to 3 x 50 bit/s (using RGB). Previously, it was demonstrated that the rolling shutter effect of the CMOS image sensor Rx can increase the data rate by using the row-by-row pixel activation [10, 11]. By demodulating this rolling shutter pattern (bright and dark fringes), ~1 kbit/s data rate can be achieved. Ref . analyzed several properties of the rolling shutter effect, and refs [12, 13]. demonstrated the “blooming” mitigation, extinction-ratio (ER) enhancement and thresholding to improve the transmission performance.
Recently, red, green, blue (RGB) LEDs  have been widely used in many places; and the color-tuning ability can provide fancy lighting conditions for different applications. Besides, color-shift-keying (CSK) using RGB has been standardized to provide additional modulation dimensions to enhance the transmission data rate . In this work, RGB LEDs are used to increase the VLC transmission capacity by using wavelength division multiplexing (WDM); and the color image sensor can separate different color signals via a color filter array on top of the color-insensitive sensing element. However, due to the wide optical bandwidths of the color filters, there is a high spectral overlap among different channels, and a high WDM inter-channel interference (ICI) happens. Here, we propose and demonstrate for the first time up to the authors’ knowledge a RGB VLC transmission using CMOS image sensor with multi-input multi-output (MIMO) technique to mitigate the ICI and retrieve the three independent color channels in the rolling shutter pattern. A second order polynomial fitting is applied to enhance the data pattern ER, and then another second order polynomial fitting is applied for thresholding. Experimental bit-error-rate (BER) performance is improved when MIMO is used, satisfying the forward-error-correction (FEC) limits.
2. Proposed architecture
As mentioned above, the color camera can separate the RGB signals according to their wavelengths; hence the VLC transmission capacity can be increased. As the CMOS sensing elements are color-insensitive, color filter array on top of the sensing element is used to separate the R, G and B color signals. However, as the optical filter bandwidths are wide, and may not exactly wavelength match with the wavelengths of the RGB LEDs. This will create color misjudge by the image sensor and result in ICI. Figure 1 shows the proposed architecture of the RGB VLC transmission using MIMO to demodulate the rolling shutter pattern. The RGB LED transmitter (Tx) used is from Lumileds Luxeon Z. The color chips used are “deep red”, “green” and “royal blue”; and the typical peak wavelengths are 660 nm, 530 nm and 450 nm respectively; with typical spectral half-widths (3-dB bandwidth of optical spectrum) of 20 nm, 100 nm and 20 nm respectively. The color Rx used is an iPhone5 mobile phone with image sensor resolution of 1080 x 1920 pixels. The typical peak color filter wavelengths of the R, G, and B in iPhone5 are 600 nm, 530 nm and 440 nm respectively, with spectral half-widths of 70 nm, 80 nm, and 20 nm respectively. The optical spectra sketches of the LEDs and Rx are shown in Fig. 1(a) and 1(b) respectively. We can observe that there is a high wavelength mismatch between the LEDs and the Rx; and there is a high spectral overlap among different channels in the sensor filter spectra. If there is no spectral overlap of the optical filter bandwidths in the image sensor, and the R, G, B signal line-widths are narrow enough and can match with the optical filter wavelengths; the signal performance will improve and MIMO can be unnecessary. Figure 1(c) shows the RGB signals received by the color-insensitive sensing elements via the color filters.
To mitigate the ICI of the received rolling shutter pattern, MIMO technique [16, 17] is applied. The optical power received at the Rxs, transmitted by the LED Txs, the channel gain, and the noise can be represented by Y, X, H and N respectively as shown in Eq. (1).Eq. (2), where, for example, hrg is the channel gain from the red LED Tx and received by the green Rx.Eq. (3).Eq. (4).
3. Experiment, results and discussion
Figure 2 shows the proof-of-concept demonstration of the RGB VLC transmission using MIMO to demodulate the rolling shutter pattern. The three independent data signals are programmed using Matlab, which is then transferred to two arbitrary waveform generators (AWGs) to perform the digit-to-analog (DAC) conversions. The two AWGs used have the sampling rates and bandwiths of 2 GSample/s and 240 MHz (Tektronix, AFG 3252C); and 50 Msample/s and 20 MHz (Agilent 33220A) respectively. The three data signals are then applied to the R, G, and B Txs (Lumileds Luxeon Z) as shown in Fig. 2(a). The R, G, B and white (not used in this demonstration) LED chips are densely soldered on an aluminum starboard with spacing of 0.2 mm. The color chips used are “deep red”, “green” and “royal blue”; and the “deep red” LED is based on AllnGaP material; while the “deep red” and “royal blue” LEDs are based on InGaN material. The RGB signals are then received by an iPhone5 Rx. Figure 2(b) shows a typical RGB rolling shutter pattern showing different color fringes. Then, the recorded signal by the iPhone5 is demodulated by a Matlab program in a computer.
When using mobile phone CMOS image sensor as Rx, one difficulty is the frame-to-frame processing time gap during recording. As the RGB LEDs are continuously sending data, there will be a data loss during this frame-to-frame time gap. The iPhone5 has frame rate of 30 fps and processing time gap of ~38% of the frame duration. The frame-to-frame processing time gap can be reduced by better design and architecture of the CMOS readout circuits . The image sensor size is 1/3.2” with pixel size of 1.4 μm. For video recording, it can operate at 1080p at 30 frames per second. Hence, in the proof-of-concept demonstration, the transmitted signals are packet-based; and each packet is transmitted successively for three times in order to guarantee a complete packet is received in an image frame. Figure 3 shows the structure of the data packet. The channel gain matrix H can be obtained from the two training signals. For the training signal 1, the R, G, and B LEDs will emit signal in turn, and each transmitting a 11-bit training signal. This is to make sure only monochromatic signal can be received. For the training signal 2, the RGB LEDs will emit 11-bit training signals simultaneously. It is worth to method that the training signal 1 and 2 are used at the beginning for building up the VLC link. The payload performance can be continuously monitored in the mobile-phone; if the VLC performance decreases due to the link condition changes, the feedback signal could be sent from the user’s side (mobile-phone) to the LED Tx side to request sending the training data again for re-establishing the transmission link. The data payload is 32-bit, and the training signal 1, 2 and the data payload are in on-off keying (OOK) format. In order to easily identify the header signal, the header consists of 4-bit periodic Manchester coded signal. This is enough for our clock recovery.
In the demodulation process, the color rolling shutter pattern including many color fringes shown in Fig. 2(b) will be first separated into three different rolling shutter patterns based on the color filter of the iPhone5 as described in Fig. 1(c). Then each image file will be converted into grayscale format, in which 0 represents completely darkness while 255 represents completely brightness. A column matrix of pixel will be selected in each grayscale image frame. We have proposed and demonstrated using a second order polynomial fitting to select a column pixel in an image frame in order to mitigate the “blooming effect” as discussed in detail in . After the column matrix selection, a matrix of grayscale value having 1080 elements can be obtained for the R, G, and B colors. However, severe ICI will result due to the wavelength mismatch between the LEDs and the color filters as discussed above. Then MIMO described in Section II is used, and Figs. 4(a)-4(c) show the data pattern of R, G, and B grayscale values respectively after the MIMO demodulation. High ER fluctuation can be observed. Then a second order polynomial fitting (black curve) is applied for smoothing.
In the smoothing process, the grayscale values greater than the polynomial equation are assigned equal to the values of that equation; while the grayscale values smaller than that equation are assigned equal to zero. Hence, the ER fluctuation in grayscale data pattern is significantly reduced, as shown in Figs. 5(a)-5(c) respectively. After this, another second order polynomial fitting curve can be constructed for thresholding. The thresholding is to distinguish the logic 1 and 0; hence the BER can be evaluated. In Figs. 5(a)-5(c), as the header is Manchester encoded, much narrower periodic data can be easily distinguished from the payload OOK data. By estimating the frequency of these narrower data, the clock and sampling rate for the payload data can be obtained. If the LED source only shines part of the image sensor, the other part of the image sensor cannot reveal the bright and dark fringes due to low optical signal received. In this case, more complicated signal enhancement scheme should be used, such as using the Sobel method to enhance the signal . However, the processing complexity is increased.
Finally, the BER performance of the RGB VLC transmission without and with the MIMO technique is shown in Fig. 6. When using the 32-bit payload, the effective data rate (by removing the header and duplicated payload) for the RGB transmission is about 3 x 30 frame/s x 32 bit/frame = 2.88 kbit/s. Besides, we can observe from Fig. 6 that the BER is significantly improved by more than one order of magnitude when MIMO is used (satisfying the FEC; BER = 3.8 x 10−3), owing to the mitigation of the ICI. In Fig. 6, the blue channel performs the best when compared with the red and green channels because the blue color filter of the image sensor allows higher transmission; hence the blue channel has better sensitivity. As also shown in Fig. 6, a relatively high received illuminance is needed here due to no suitable focusing lens is used. The total included angles (total angle at which 90% of total luminous flux captured) are > 140° for the R, G, and B LED chips. In this proof-of-concept experiment, only single R, G, B LED is used without proper lens for focusing; hence the transmission distance at ~500 lux is only <10 cm. Using a RGB LED array or focusing lens can increase the transmission distance. The BER performance is mainly determined by the decoding of the image frame received by the CMOS image sensor. Improved BER can be obtained by correctly decoding every bright/dark fringe in an image frame; and this can be achieved by increasing the illumination (putting the Rx closer to the LED or using focusing lenses) or decreasing the data rate (so that more pixels are used to represent each bit).
In this work, RGB LEDs were used to increase the VLC transmission capacity by WDM; and the color image sensor in mobile-phone was used. However, due to the wide optical bandwidth of the color filters, there was a high spectral overlap among different channels, and a high ICI happened. Here, we proposed and demonstrated a RGB VLC transmission using CMOS image sensor with MIMO to mitigate the ICI and to demodulate the rolling shutter pattern. A second order polynomial fitting was proposed and applied to enhance the data pattern ER, and then another second order polynomial fitting was applied for thresholding. BER performance was significantly improved when MIMO was used.
This work was supported by Ministry of Science and Technology, Taiwan, ROC, MOST-104-2628-E-009-011-MY3, MOST-103-2221-E-009-030-MY3, Aim for the Top University Plan, Taiwan, and Ministry of Education, Taiwan.
References and links
1. S. Wu, H. Wang, and C. H. Youn, “Visible light communications for 5G wireless networking systems: from fixed to mobile communications,” IEEE Netw. 28(6), 41–45 (2014). [CrossRef]
2. H. Haas and C. Chen, “What is LiFi?” in Proc. ECOC (2015), paper 0871.
3. C. W. Chow, C. H. Yeh, Y. Liu, and Y. F. Liu, “Digital signal processing for light emitting diode based visible light communication,” IEEE Photonics Soc. Newslett. 26, 9–13 (2012).
4. B. Janjua, H. M. Oubei, J. R. D. Retamal, T. K. Ng, C.-T. Tsai, H.-Y. Wang, Y.-C. Chi, H.-C. Kuo, G.-R. Lin, J.-H. He, and B. S. Ooi, “Going beyond 4 Gbps data rate by employing RGB laser diodes for visible light communication,” Opt. Express 23(14), 18746–18753 (2015). [CrossRef] [PubMed]
5. Y. C. Chi, D. H. Hsieh, C. T. Tsai, H. Y. Chen, H. C. Kuo, and G. R. Lin, “450-nm GaN laser diode enables high-speed visible light communication with 9-Gbps QAM-OFDM,” Opt. Express 23(10), 13051–13059 (2015). [CrossRef] [PubMed]
7. Z. Wang, C. Yu, W. D. Zhong, J. Chen, and W. Chen, “Performance of a novel LED lamp arrangement to reduce SNR fluctuation for multi-user visible light communication systems,” Opt. Express 20(4), 4564–4573 (2012). [CrossRef] [PubMed]
8. I. Takai, S. Ito, K. Yasutomi, K. Kagawa, M. Andoh, and S. Kawahito, “LED and CMOS image sensor based optical wireless communication system for automotive applications,” IEEE Photonics J. 5(5), 6801418 (2013). [CrossRef]
9. P. Luo, M. Zhang, Z. Ghassemlooy, H. L. Minh, H. M. Tsai, X. Tang, L. C. Png, and D. Han, “Experimental demonstration of RGB LED-based optical camera communications,” IEEE Photonics J. 7, 7904242 (2015).
10. C. Danakis, M. Afgani, G. Povey, I. Underwood, and H. Haas, “Using a CMOS camera sensor for visible light communication,” in Proc. OWC (2012), pp. 1244–1248. [CrossRef]
12. C. W. Chow, C. Y. Chen, and S. H. Chen, “Enhancement of signal performance in LED visible light communications using mobile phone camera,” IEEE Photonics J. 7(5), 7903607 (2015). [CrossRef]
13. Y. Liu, H. Y. Chen, K. Liang, C. W. Hsu, C. W. Chow, and C. H. Yeh, “Visible light communication using receivers of camera image sensor and solar cell,” IEEE Photonics J. 8(1), 7800107 (2016). [CrossRef]
14. J. Y. Sung, C. W. Chow, and C. H. Yeh, “Dimming-discrete-multi-tone (DMT) for simultaneous color control and high speed visible light communication,” Opt. Express 22(7), 7538–7543 (2014). [CrossRef] [PubMed]
15. S. Rajagopal, R. Roberts, and S. K. Lim, “IEEE 802.15.7 visible light communication: modulation schemes and dimming support,” IEEE Commun. Mag. 50(3), 72–82 (2012). [CrossRef]
16. H. H. Lu, Y. P. Lin, P. Y. Wu, C. Y. Chen, M. C. Chen, and T. W. Jhang, “A multiple-input-multiple-output visible light communication system based on VCSELs and spatial light modulators,” Opt. Express 22(3), 3468–3474 (2014). [CrossRef] [PubMed]
17. C. H. Chang, C. Y. Li, H. H. Lu, C. Y. Lin, J. H. Chen, Z. W. Wan, and C. J. Cheng, “A 100-Gb/s multiple-input multiple-output visible laser light communication system,” J. Lightwave Technol. 32(24), 4723–4729 (2014). [CrossRef]