Abstract

In an RGB-LED-based optical camera communication (OCC) system, the inter-symbol interference and inter-channel interference deteriorate the transmission performance considerably. In this paper, a two-dimensional CNN structure is proposed for data recovery by learning features between color channels and neighboring symbols in the rolling shutter based OCC system under random data transmission. Moreover, we further propose an XOR-based data loss compensation method to realize 21% data rate improvement by restoring the lost data during the transmission. A record-high data rate at 47 kbit/s has been experimentally achieved for an RGB-LED-based OCC system using a rolling shutter camera in a smartphone.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Decoding scheme based on CNN for mobile optical camera communication

Ke Yu, Jing He, and Zheng Huang
Appl. Opt. 59(23) 7109-7113 (2020)

Correlation-based receiver for optical camera communications

Cristo Jurado-Verdu, Vicente Matus, Jose Rabadan, Victor Guerra, and Rafael Perez-Jimenez
Opt. Express 27(14) 19150-19155 (2019)

Performance evaluation of neural network assisted motion detection schemes implemented within indoor optical camera based communications

Shivani Rajendra Teli, Stanislav Zvanovec, and Zabih Ghassemlooy
Opt. Express 27(17) 24082-24092 (2019)

References

  • View by:
  • |
  • |
  • |

  1. Y. Hong, T. Wu, and L. K. Chen, “On the performance of adaptive MIMO-OFDM indoor visible light communications,” IEEE Photonics Technol. Lett. 28(8), 907–910 (2016).
    [Crossref]
  2. M. Z. Chowdhury, M. T. Hossan, A. Islam, and Y. M. Jang, “A comparative survey of optical wireless technologies: architectures and applications,” IEEE Access 6, 9819–9840 (2018).
    [Crossref]
  3. J. Wang, W. Huang, and Z. Xu, “Demonstration of a covert camera-screen communication system,” in Proceedings of International Wireless Communications and Mobile Computing Conference (IEEE, 2017), pp. 910–915.
  4. C. Danakis, M. Afgani, G. Povey, I. Underwood, and H. Haas, “Using a CMOS camera sensor for visible light communication,” Globecom Workshops (IEEE, 2012), pp. 1244–1248.
  5. R. Deng, J. He, Y. Hong, J. Shi, and L. Chen, “2.38 Kbits/frame WDM transmission over a CVLC system with sampling reconstruction for SFO mitigation,” Opt. Express 25(24), 30575–30581 (2017).
    [Crossref]
  6. H. Chen, X. Z. Lai, P. Chen, Y. T. Liu, M. Y. Yu, Z. H. Liu, and Z. J. Zhu, “Quadrichromatic LED based mobile phone camera visible light communication,” Opt. Express 26(13), 17132–17144 (2018).
    [Crossref]
  7. Y. Q. Xu, J. Hua, Z. Gong, W. Zhao, Z. Q. Zhang, C. Y. Xie, Z. T. Chen, and J. F. Chen, “Visible light communication using dual camera on one smartphone,” Opt. Express 26(26), 34609–34621 (2018).
    [Crossref]
  8. L. Liu, R. Deng, and L. K. Chen, “Spatial and Time Dispersions Compensation with Double-equalization for Optical Camera Communications,” Photon. Technol. Lett. (posted 3 October 2019, in press).
    [Crossref]
  9. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural network,” in Proceedings of Conference on Neural Information Processing Systems (ACM, 2012), pp. 1097–1105.
  10. W. Guan, J. Li, S. Wen, X. Zhang, Y. Ye, J. Zheng, and J. Jiang, “The detection and recognition of RGB-LED-ID based on visible light communication using convolutional neural network,” Appl. Sci. 9(7), 1400 (2019).
    [Crossref]
  11. M. Meingast, C. Geyer, and S. Sastry, “Geometric models of rolling-shutter cameras,” https://arxiv.org/abs/cs/0503076 .
  12. P. Hu, P. H. Pathak, X. Feng, H. Fu, and P. Mohapatra, “ColorBars: increasing data rate of LED-to-camera communication using color shift keying,” in Proceedings of International Conference on emerging Networking EXperiments and Technologies (ACM, 2015), pp. 1–13.
  13. D. T. Nguyen and Y. Park, “Data rate enhancement of optical camera communications by compensating inter-frame gaps,” Opt. Commun. 394, 56–61 (2017).
    [Crossref]
  14. I. Goodfellow, Y. Bengio, and A. Courville, “Deep feedforward networks,” in Deep Learning, (The Massachusetts Institute of Technology Press, 2016).

2019 (1)

W. Guan, J. Li, S. Wen, X. Zhang, Y. Ye, J. Zheng, and J. Jiang, “The detection and recognition of RGB-LED-ID based on visible light communication using convolutional neural network,” Appl. Sci. 9(7), 1400 (2019).
[Crossref]

2018 (3)

2017 (2)

R. Deng, J. He, Y. Hong, J. Shi, and L. Chen, “2.38 Kbits/frame WDM transmission over a CVLC system with sampling reconstruction for SFO mitigation,” Opt. Express 25(24), 30575–30581 (2017).
[Crossref]

D. T. Nguyen and Y. Park, “Data rate enhancement of optical camera communications by compensating inter-frame gaps,” Opt. Commun. 394, 56–61 (2017).
[Crossref]

2016 (1)

Y. Hong, T. Wu, and L. K. Chen, “On the performance of adaptive MIMO-OFDM indoor visible light communications,” IEEE Photonics Technol. Lett. 28(8), 907–910 (2016).
[Crossref]

Afgani, M.

C. Danakis, M. Afgani, G. Povey, I. Underwood, and H. Haas, “Using a CMOS camera sensor for visible light communication,” Globecom Workshops (IEEE, 2012), pp. 1244–1248.

Bengio, Y.

I. Goodfellow, Y. Bengio, and A. Courville, “Deep feedforward networks,” in Deep Learning, (The Massachusetts Institute of Technology Press, 2016).

Chen, H.

Chen, J. F.

Chen, L.

Chen, L. K.

Y. Hong, T. Wu, and L. K. Chen, “On the performance of adaptive MIMO-OFDM indoor visible light communications,” IEEE Photonics Technol. Lett. 28(8), 907–910 (2016).
[Crossref]

L. Liu, R. Deng, and L. K. Chen, “Spatial and Time Dispersions Compensation with Double-equalization for Optical Camera Communications,” Photon. Technol. Lett. (posted 3 October 2019, in press).
[Crossref]

Chen, P.

Chen, Z. T.

Chowdhury, M. Z.

M. Z. Chowdhury, M. T. Hossan, A. Islam, and Y. M. Jang, “A comparative survey of optical wireless technologies: architectures and applications,” IEEE Access 6, 9819–9840 (2018).
[Crossref]

Courville, A.

I. Goodfellow, Y. Bengio, and A. Courville, “Deep feedforward networks,” in Deep Learning, (The Massachusetts Institute of Technology Press, 2016).

Danakis, C.

C. Danakis, M. Afgani, G. Povey, I. Underwood, and H. Haas, “Using a CMOS camera sensor for visible light communication,” Globecom Workshops (IEEE, 2012), pp. 1244–1248.

Deng, R.

R. Deng, J. He, Y. Hong, J. Shi, and L. Chen, “2.38 Kbits/frame WDM transmission over a CVLC system with sampling reconstruction for SFO mitigation,” Opt. Express 25(24), 30575–30581 (2017).
[Crossref]

L. Liu, R. Deng, and L. K. Chen, “Spatial and Time Dispersions Compensation with Double-equalization for Optical Camera Communications,” Photon. Technol. Lett. (posted 3 October 2019, in press).
[Crossref]

Feng, X.

P. Hu, P. H. Pathak, X. Feng, H. Fu, and P. Mohapatra, “ColorBars: increasing data rate of LED-to-camera communication using color shift keying,” in Proceedings of International Conference on emerging Networking EXperiments and Technologies (ACM, 2015), pp. 1–13.

Fu, H.

P. Hu, P. H. Pathak, X. Feng, H. Fu, and P. Mohapatra, “ColorBars: increasing data rate of LED-to-camera communication using color shift keying,” in Proceedings of International Conference on emerging Networking EXperiments and Technologies (ACM, 2015), pp. 1–13.

Geyer, C.

M. Meingast, C. Geyer, and S. Sastry, “Geometric models of rolling-shutter cameras,” https://arxiv.org/abs/cs/0503076 .

Gong, Z.

Goodfellow, I.

I. Goodfellow, Y. Bengio, and A. Courville, “Deep feedforward networks,” in Deep Learning, (The Massachusetts Institute of Technology Press, 2016).

Guan, W.

W. Guan, J. Li, S. Wen, X. Zhang, Y. Ye, J. Zheng, and J. Jiang, “The detection and recognition of RGB-LED-ID based on visible light communication using convolutional neural network,” Appl. Sci. 9(7), 1400 (2019).
[Crossref]

Haas, H.

C. Danakis, M. Afgani, G. Povey, I. Underwood, and H. Haas, “Using a CMOS camera sensor for visible light communication,” Globecom Workshops (IEEE, 2012), pp. 1244–1248.

He, J.

Hinton, G. E.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural network,” in Proceedings of Conference on Neural Information Processing Systems (ACM, 2012), pp. 1097–1105.

Hong, Y.

R. Deng, J. He, Y. Hong, J. Shi, and L. Chen, “2.38 Kbits/frame WDM transmission over a CVLC system with sampling reconstruction for SFO mitigation,” Opt. Express 25(24), 30575–30581 (2017).
[Crossref]

Y. Hong, T. Wu, and L. K. Chen, “On the performance of adaptive MIMO-OFDM indoor visible light communications,” IEEE Photonics Technol. Lett. 28(8), 907–910 (2016).
[Crossref]

Hossan, M. T.

M. Z. Chowdhury, M. T. Hossan, A. Islam, and Y. M. Jang, “A comparative survey of optical wireless technologies: architectures and applications,” IEEE Access 6, 9819–9840 (2018).
[Crossref]

Hu, P.

P. Hu, P. H. Pathak, X. Feng, H. Fu, and P. Mohapatra, “ColorBars: increasing data rate of LED-to-camera communication using color shift keying,” in Proceedings of International Conference on emerging Networking EXperiments and Technologies (ACM, 2015), pp. 1–13.

Hua, J.

Huang, W.

J. Wang, W. Huang, and Z. Xu, “Demonstration of a covert camera-screen communication system,” in Proceedings of International Wireless Communications and Mobile Computing Conference (IEEE, 2017), pp. 910–915.

Islam, A.

M. Z. Chowdhury, M. T. Hossan, A. Islam, and Y. M. Jang, “A comparative survey of optical wireless technologies: architectures and applications,” IEEE Access 6, 9819–9840 (2018).
[Crossref]

Jang, Y. M.

M. Z. Chowdhury, M. T. Hossan, A. Islam, and Y. M. Jang, “A comparative survey of optical wireless technologies: architectures and applications,” IEEE Access 6, 9819–9840 (2018).
[Crossref]

Jiang, J.

W. Guan, J. Li, S. Wen, X. Zhang, Y. Ye, J. Zheng, and J. Jiang, “The detection and recognition of RGB-LED-ID based on visible light communication using convolutional neural network,” Appl. Sci. 9(7), 1400 (2019).
[Crossref]

Krizhevsky, A.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural network,” in Proceedings of Conference on Neural Information Processing Systems (ACM, 2012), pp. 1097–1105.

Lai, X. Z.

Li, J.

W. Guan, J. Li, S. Wen, X. Zhang, Y. Ye, J. Zheng, and J. Jiang, “The detection and recognition of RGB-LED-ID based on visible light communication using convolutional neural network,” Appl. Sci. 9(7), 1400 (2019).
[Crossref]

Liu, L.

L. Liu, R. Deng, and L. K. Chen, “Spatial and Time Dispersions Compensation with Double-equalization for Optical Camera Communications,” Photon. Technol. Lett. (posted 3 October 2019, in press).
[Crossref]

Liu, Y. T.

Liu, Z. H.

Meingast, M.

M. Meingast, C. Geyer, and S. Sastry, “Geometric models of rolling-shutter cameras,” https://arxiv.org/abs/cs/0503076 .

Mohapatra, P.

P. Hu, P. H. Pathak, X. Feng, H. Fu, and P. Mohapatra, “ColorBars: increasing data rate of LED-to-camera communication using color shift keying,” in Proceedings of International Conference on emerging Networking EXperiments and Technologies (ACM, 2015), pp. 1–13.

Nguyen, D. T.

D. T. Nguyen and Y. Park, “Data rate enhancement of optical camera communications by compensating inter-frame gaps,” Opt. Commun. 394, 56–61 (2017).
[Crossref]

Park, Y.

D. T. Nguyen and Y. Park, “Data rate enhancement of optical camera communications by compensating inter-frame gaps,” Opt. Commun. 394, 56–61 (2017).
[Crossref]

Pathak, P. H.

P. Hu, P. H. Pathak, X. Feng, H. Fu, and P. Mohapatra, “ColorBars: increasing data rate of LED-to-camera communication using color shift keying,” in Proceedings of International Conference on emerging Networking EXperiments and Technologies (ACM, 2015), pp. 1–13.

Povey, G.

C. Danakis, M. Afgani, G. Povey, I. Underwood, and H. Haas, “Using a CMOS camera sensor for visible light communication,” Globecom Workshops (IEEE, 2012), pp. 1244–1248.

Sastry, S.

M. Meingast, C. Geyer, and S. Sastry, “Geometric models of rolling-shutter cameras,” https://arxiv.org/abs/cs/0503076 .

Shi, J.

Sutskever, I.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural network,” in Proceedings of Conference on Neural Information Processing Systems (ACM, 2012), pp. 1097–1105.

Underwood, I.

C. Danakis, M. Afgani, G. Povey, I. Underwood, and H. Haas, “Using a CMOS camera sensor for visible light communication,” Globecom Workshops (IEEE, 2012), pp. 1244–1248.

Wang, J.

J. Wang, W. Huang, and Z. Xu, “Demonstration of a covert camera-screen communication system,” in Proceedings of International Wireless Communications and Mobile Computing Conference (IEEE, 2017), pp. 910–915.

Wen, S.

W. Guan, J. Li, S. Wen, X. Zhang, Y. Ye, J. Zheng, and J. Jiang, “The detection and recognition of RGB-LED-ID based on visible light communication using convolutional neural network,” Appl. Sci. 9(7), 1400 (2019).
[Crossref]

Wu, T.

Y. Hong, T. Wu, and L. K. Chen, “On the performance of adaptive MIMO-OFDM indoor visible light communications,” IEEE Photonics Technol. Lett. 28(8), 907–910 (2016).
[Crossref]

Xie, C. Y.

Xu, Y. Q.

Xu, Z.

J. Wang, W. Huang, and Z. Xu, “Demonstration of a covert camera-screen communication system,” in Proceedings of International Wireless Communications and Mobile Computing Conference (IEEE, 2017), pp. 910–915.

Ye, Y.

W. Guan, J. Li, S. Wen, X. Zhang, Y. Ye, J. Zheng, and J. Jiang, “The detection and recognition of RGB-LED-ID based on visible light communication using convolutional neural network,” Appl. Sci. 9(7), 1400 (2019).
[Crossref]

Yu, M. Y.

Zhang, X.

W. Guan, J. Li, S. Wen, X. Zhang, Y. Ye, J. Zheng, and J. Jiang, “The detection and recognition of RGB-LED-ID based on visible light communication using convolutional neural network,” Appl. Sci. 9(7), 1400 (2019).
[Crossref]

Zhang, Z. Q.

Zhao, W.

Zheng, J.

W. Guan, J. Li, S. Wen, X. Zhang, Y. Ye, J. Zheng, and J. Jiang, “The detection and recognition of RGB-LED-ID based on visible light communication using convolutional neural network,” Appl. Sci. 9(7), 1400 (2019).
[Crossref]

Zhu, Z. J.

Appl. Sci. (1)

W. Guan, J. Li, S. Wen, X. Zhang, Y. Ye, J. Zheng, and J. Jiang, “The detection and recognition of RGB-LED-ID based on visible light communication using convolutional neural network,” Appl. Sci. 9(7), 1400 (2019).
[Crossref]

IEEE Access (1)

M. Z. Chowdhury, M. T. Hossan, A. Islam, and Y. M. Jang, “A comparative survey of optical wireless technologies: architectures and applications,” IEEE Access 6, 9819–9840 (2018).
[Crossref]

IEEE Photonics Technol. Lett. (1)

Y. Hong, T. Wu, and L. K. Chen, “On the performance of adaptive MIMO-OFDM indoor visible light communications,” IEEE Photonics Technol. Lett. 28(8), 907–910 (2016).
[Crossref]

Opt. Commun. (1)

D. T. Nguyen and Y. Park, “Data rate enhancement of optical camera communications by compensating inter-frame gaps,” Opt. Commun. 394, 56–61 (2017).
[Crossref]

Opt. Express (3)

Other (7)

L. Liu, R. Deng, and L. K. Chen, “Spatial and Time Dispersions Compensation with Double-equalization for Optical Camera Communications,” Photon. Technol. Lett. (posted 3 October 2019, in press).
[Crossref]

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural network,” in Proceedings of Conference on Neural Information Processing Systems (ACM, 2012), pp. 1097–1105.

J. Wang, W. Huang, and Z. Xu, “Demonstration of a covert camera-screen communication system,” in Proceedings of International Wireless Communications and Mobile Computing Conference (IEEE, 2017), pp. 910–915.

C. Danakis, M. Afgani, G. Povey, I. Underwood, and H. Haas, “Using a CMOS camera sensor for visible light communication,” Globecom Workshops (IEEE, 2012), pp. 1244–1248.

M. Meingast, C. Geyer, and S. Sastry, “Geometric models of rolling-shutter cameras,” https://arxiv.org/abs/cs/0503076 .

P. Hu, P. H. Pathak, X. Feng, H. Fu, and P. Mohapatra, “ColorBars: increasing data rate of LED-to-camera communication using color shift keying,” in Proceedings of International Conference on emerging Networking EXperiments and Technologies (ACM, 2015), pp. 1–13.

I. Goodfellow, Y. Bengio, and A. Courville, “Deep feedforward networks,” in Deep Learning, (The Massachusetts Institute of Technology Press, 2016).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1.
Fig. 1. (a) Pre-processing for signal extraction and scaling (signals on the green channel are taken as an example). (b) The 2D-CNN structure proposed for RGB-LED-based OCC system.
Fig. 2.
Fig. 2. (a-c) Different cases of data loss.
Fig. 3.
Fig. 3. (a) Repetitive transmission and proposed XDLC scheme. (b) Experimental setup. (c) Decoding diagram.
Fig. 4.
Fig. 4. Parameter optimization. (a) Gap-time estimation. (b) BER performance under different symbol number and oversampling ratio for the input layer of 2D-CNN.
Fig. 5.
Fig. 5. BER performance and data rate comparison. (a) BER performance by using various algorithms. (b) Data rate versus symbol rate and (c) BER performance w/ and w/o XDLC.

Equations (1)

Equations on this page are rendered with MathJax. Learn more.

W l ( i , j , v ) = f ( ( W l 1 K v l ) ( i , j ) + b ( v ) ) , i = 1 , , X , j = 1 , , Y , v = 1 , , D .

Metrics