Abstract

We experimentally validate a complete optical packet switched interconnection network, implementing the SPINet architecture. The scalable photonic integrated network (SPINet) architecture capitalizes on wavelength division multiplexing (WDM) to provide very large transmission bandwidths, simplify network design, and reduce the network's power dissipation. Contention resolution is performed in the optical domain, and a novel physical layer acknowledgement protocol is employed to mitigate the associated latency and performance penalties. Moreover, the SPINet architecture is specifically designed to enable on-chip integration by not using any kind of optical delay lines. Experiments presented include a complete functionality verification, error-free routing of 80Gb/s wavelength-striped optical packets (8 wavelengths each modulated at 10 Gb/s) with a bit-error rate (BER) better than $10^{{-}12}$, and novel performance-enhancement techniques such as path adjustments and load balancing.

© 2009 IEEE

PDF Article

References

  • View by:
  • |
  • |

  1. G. E. Moore, "Cramming more components onto integrated circuits," Electronics 38, 114-117 (1965).
  2. D. A. B. Miller, "Rationale and challenges for optical interconnects to electronic chips," Proc. IEEE 88, 728-749 (2000).
  3. W. J. Dally, B. Towles, Principles and Practices of Interconnection Networks (Morgan Kaufmann, 2004).
  4. J. L. Hennessy, D. A. Patterson, Computer Architecture: A Quantitative Approach (Morgan Kaufmann, 2002).
  5. M. Kistler, M. Perrone, F. Petrini, "Cell multiprocessor communication network: Built for speed," IEEE Micro 26, 10-23 (2006).
  6. T. Mudge, "Power: A first-class architectural design constraint," IEEE Computer 34, 52-58 (2001).
  7. L. A. Barroso, J. Dean, U. Hölzle, "Web search for a planet: The Google cluster architecture," IEEE Micro 23, 22-28 (2003).
  8. J. Liu, X. Lin, "Equalization in high-speed communication systems," IEEE Circuits Syst. Mag. 4, 4-17 (2004).
  9. G. P. Agrawal, Fiber-Optic Communication Systems (Wiley, 2002).
  10. K. Fukuchi, T. Kasamatsu, M. Morie, R. Ohhira, T. Ito, K. Sekiya, D. Ogasahara, T. Ono, "10.92-Tb/s (273$\,\times\,$40-Gb/s) triple-band/ultradense WDM optical-repeatered transmission experiment," Optical Fiber Communications Conf. (OFC) (2001) paper PDP-24.
  11. A. F. Benner, M. Ignatowski, J. A. Kash, D. M. Kuchta, M. B. Ritter, "Exploitation of optical interconnects in future server architectures," IBM J. Res. Dev. 49, 755-775 (2005).
  12. R. Ramaswami, K. N. Sivarajan, Optical Networks: A Practical Perspective (Morgan Kaufmann, 2002).
  13. R. Luijten, C. Minkenberg, B. R. Hemenway, M. Sauer, R. Grzybowski, "Viable opto-electronic HPC interconnect fabrics," Proc. ACM/IEEE (SC—05) Conf. Supercomp. (2005) pp. 18.
  14. A. K. Kodi, A. Louri, "Design of a high-speed optical interconnect for scalable shared-memory multiprocessors," IEEE Micro 25, 41-49 (2005).
  15. I. H. White, K. A. Williams, R. V. Penty, T. Lin, A. Wonfor, E. T. Aw, M. Glick, M. Dales, D. McAuley, "Control architecture for high capacity multistage photonic switch circuits," J. Optical Networking 6, 180-188 (2007).
  16. A. Shacham, B. A. Small, O. Liboiron-Ladouceur, K. Bergman, "A fully implemented 12$\,\times\,$12 data vortex optical packet switching interconnection network," J. Lightw. Technol. 23, 3066-3075 (2005).
  17. R. D. Chamberlain, M. A. Franklin, C. S. Baw, "Gemini: An optical interconnection network for parallel processing," IEEE Trans. Parallel Distrib. Syst. 13, 1038-1055 (2002).
  18. A. Shacham, B. G. Lee, K. Bergman, "A scalable, self-routed, terabit capacity, photonic interconnection network," Proc. Hot Interconnects: IEEE 13th Ann. Symp. High Performance Interconnects (2005) pp. 147-150.
  19. A. Shacham, K. Bergman, "Building ultralow latency interconnection networks using photonic integration," IEEE Micro 27, 6-20 (2007).
  20. R. Nagarajan, "Large-scale photonic integrated circuits," IEEE J. Sel. Topics Quantum Electron. 11, 50-65 (2005).
  21. R. Nagarajan, "Large-scale photonic integrated circuits for long-haul transmission and switching," J. Opt. Netw. 6, 102-111 (2007).
  22. R. Nagarajan, "Monolithic, 10 and 40 channel inp receiver photonic integrated circuits with on-chip amplification," Proc. Opt. Fiber Commun. Conf. (OFC) (2007).
  23. H. Park, A. W. Fang, O. Cohen, R. Jones, M. J. Paniccia, J. E. Bowers, "A hybrid AlGaInAs-silicon evanescent amplifier," IEEE Photon. Technol. Lett. 19, 230-232 (2007).
  24. C. Gunn, "CMOS photonics for high-speed interconnects," IEEE Micro 26, 58-66 (2006).
  25. S. Sur, M. J. Koop, L. Chai, D. K. Panda, "Performance analysis and evaluation of Mellanox ConnectX InfiniBand architecture with multi-core platforms," Hot Interconnects: IEEE 15th Ann. Symp. High Performance Interconnects (2007).
  26. A. Shacham, K. Bergman, "Utilizing path diversity in optical packet switched interconnection networks," Opt. Fiber Commun. Conf. (OFC) (2006) paper OTuN5.
  27. A. Shacham, B. G. Lee, K. Bergman, "A wideband, non-blocking, 2$\,\times\,$2 switching node for a SPINet network," IEEE Photon. Technol. Lett. 17, 2742-2744 (2005).
  28. A. Shacham, H. Wang, K. Bergman, "Experimental demonstration of a complete SPINet optical packet switched interconnection network," Opt. Fiber Commun. Conf. (OFC) (2007) paper OThF7.
  29. I. Armstrong, I. Andonovic, A. E. Kelly, "Semiconductor optical amplifiers: Performance and applications in optical packet switching," J. Opt. Netw. 3, 882-897 (2004).
  30. Q. Xu, B. Schmidt, S. Pradhan, M. Lipson, "Micrometre-scale silicon electro-optic modulator," Nature 435, 325-327 (2005).
  31. A. Pattavina, Switching Theory—Architecture and Performance in Broadband ATM Networks (Wiley, 1998).
  32. Mellanox InfiniScale III Switch Overview http://www.mellanox.com/products/switch silicon.php.
  33. T. Lin, K. A. Williams, R. V. Penty, I. H. White, M. Glick, "Capacity scaling in a multihost wavelength-striped SOA-based switch fabric," J. Lightw. Technol. 25, 655-663 (2007).
  34. L. A. B. Windover, "Parallel-optical interconnects ${>}100$ Gb/s," J. Lightw. Technol. 22, 2055-2073 (2004).
  35. B. E. Lemoff, M. E. Ali, G. Panotopoulos, G. M. Flower, B. Madhavan, A. F. J. Levi, D. W. Dolfi, "MAUI: Enabling fiber-to-the-processor with parallel multiwavelength optical interconnects," J. Lightw. Technol. 22, 2043-2054 (2004).
  36. A. Narasimha, B. Analui, Y. Liang, T. Sleboda, C. Gunn, "A fully integrated 4$\,\times\,$10 Gb/s DWDM optoelectronic transceiver in a standard 0.13 $\mu$m CMOS SOI," Int. Solid State Circuits Conf. (2007) paper 2.1.
  37. Infinera DTN Product Brief http://www.infinera.com/products/dtn.html.
  38. M. J. Connelly, Semiconductor Optical Amplifiers (Kluwer Academic, 2002).
  39. O. Liboiron-Ladouceur, B. A. Small, K. Bergman, "Physical layer scalability of WDM optical packet interconnection networks," J. Lightw. Technol. 24, 262-270 (2006).

2007 (5)

I. H. White, K. A. Williams, R. V. Penty, T. Lin, A. Wonfor, E. T. Aw, M. Glick, M. Dales, D. McAuley, "Control architecture for high capacity multistage photonic switch circuits," J. Optical Networking 6, 180-188 (2007).

A. Shacham, K. Bergman, "Building ultralow latency interconnection networks using photonic integration," IEEE Micro 27, 6-20 (2007).

R. Nagarajan, "Large-scale photonic integrated circuits for long-haul transmission and switching," J. Opt. Netw. 6, 102-111 (2007).

H. Park, A. W. Fang, O. Cohen, R. Jones, M. J. Paniccia, J. E. Bowers, "A hybrid AlGaInAs-silicon evanescent amplifier," IEEE Photon. Technol. Lett. 19, 230-232 (2007).

T. Lin, K. A. Williams, R. V. Penty, I. H. White, M. Glick, "Capacity scaling in a multihost wavelength-striped SOA-based switch fabric," J. Lightw. Technol. 25, 655-663 (2007).

2006 (3)

O. Liboiron-Ladouceur, B. A. Small, K. Bergman, "Physical layer scalability of WDM optical packet interconnection networks," J. Lightw. Technol. 24, 262-270 (2006).

C. Gunn, "CMOS photonics for high-speed interconnects," IEEE Micro 26, 58-66 (2006).

M. Kistler, M. Perrone, F. Petrini, "Cell multiprocessor communication network: Built for speed," IEEE Micro 26, 10-23 (2006).

2005 (6)

A. F. Benner, M. Ignatowski, J. A. Kash, D. M. Kuchta, M. B. Ritter, "Exploitation of optical interconnects in future server architectures," IBM J. Res. Dev. 49, 755-775 (2005).

A. K. Kodi, A. Louri, "Design of a high-speed optical interconnect for scalable shared-memory multiprocessors," IEEE Micro 25, 41-49 (2005).

A. Shacham, B. G. Lee, K. Bergman, "A wideband, non-blocking, 2$\,\times\,$2 switching node for a SPINet network," IEEE Photon. Technol. Lett. 17, 2742-2744 (2005).

R. Nagarajan, "Large-scale photonic integrated circuits," IEEE J. Sel. Topics Quantum Electron. 11, 50-65 (2005).

A. Shacham, B. A. Small, O. Liboiron-Ladouceur, K. Bergman, "A fully implemented 12$\,\times\,$12 data vortex optical packet switching interconnection network," J. Lightw. Technol. 23, 3066-3075 (2005).

Q. Xu, B. Schmidt, S. Pradhan, M. Lipson, "Micrometre-scale silicon electro-optic modulator," Nature 435, 325-327 (2005).

2004 (4)

L. A. B. Windover, "Parallel-optical interconnects ${>}100$ Gb/s," J. Lightw. Technol. 22, 2055-2073 (2004).

B. E. Lemoff, M. E. Ali, G. Panotopoulos, G. M. Flower, B. Madhavan, A. F. J. Levi, D. W. Dolfi, "MAUI: Enabling fiber-to-the-processor with parallel multiwavelength optical interconnects," J. Lightw. Technol. 22, 2043-2054 (2004).

I. Armstrong, I. Andonovic, A. E. Kelly, "Semiconductor optical amplifiers: Performance and applications in optical packet switching," J. Opt. Netw. 3, 882-897 (2004).

J. Liu, X. Lin, "Equalization in high-speed communication systems," IEEE Circuits Syst. Mag. 4, 4-17 (2004).

2003 (1)

L. A. Barroso, J. Dean, U. Hölzle, "Web search for a planet: The Google cluster architecture," IEEE Micro 23, 22-28 (2003).

2002 (1)

R. D. Chamberlain, M. A. Franklin, C. S. Baw, "Gemini: An optical interconnection network for parallel processing," IEEE Trans. Parallel Distrib. Syst. 13, 1038-1055 (2002).

2001 (1)

T. Mudge, "Power: A first-class architectural design constraint," IEEE Computer 34, 52-58 (2001).

2000 (1)

D. A. B. Miller, "Rationale and challenges for optical interconnects to electronic chips," Proc. IEEE 88, 728-749 (2000).

1965 (1)

G. E. Moore, "Cramming more components onto integrated circuits," Electronics 38, 114-117 (1965).

Electronics (1)

G. E. Moore, "Cramming more components onto integrated circuits," Electronics 38, 114-117 (1965).

IBM J. Res. Dev. (1)

A. F. Benner, M. Ignatowski, J. A. Kash, D. M. Kuchta, M. B. Ritter, "Exploitation of optical interconnects in future server architectures," IBM J. Res. Dev. 49, 755-775 (2005).

IEEE Micro (1)

A. K. Kodi, A. Louri, "Design of a high-speed optical interconnect for scalable shared-memory multiprocessors," IEEE Micro 25, 41-49 (2005).

IEEE Circuits Syst. Mag. (1)

J. Liu, X. Lin, "Equalization in high-speed communication systems," IEEE Circuits Syst. Mag. 4, 4-17 (2004).

IEEE Computer (1)

T. Mudge, "Power: A first-class architectural design constraint," IEEE Computer 34, 52-58 (2001).

IEEE J. Sel. Topics Quantum Electron. (1)

R. Nagarajan, "Large-scale photonic integrated circuits," IEEE J. Sel. Topics Quantum Electron. 11, 50-65 (2005).

IEEE Micro (4)

C. Gunn, "CMOS photonics for high-speed interconnects," IEEE Micro 26, 58-66 (2006).

L. A. Barroso, J. Dean, U. Hölzle, "Web search for a planet: The Google cluster architecture," IEEE Micro 23, 22-28 (2003).

M. Kistler, M. Perrone, F. Petrini, "Cell multiprocessor communication network: Built for speed," IEEE Micro 26, 10-23 (2006).

A. Shacham, K. Bergman, "Building ultralow latency interconnection networks using photonic integration," IEEE Micro 27, 6-20 (2007).

IEEE Photon. Technol. Lett. (1)

A. Shacham, B. G. Lee, K. Bergman, "A wideband, non-blocking, 2$\,\times\,$2 switching node for a SPINet network," IEEE Photon. Technol. Lett. 17, 2742-2744 (2005).

IEEE Photon. Technol. Lett. (1)

H. Park, A. W. Fang, O. Cohen, R. Jones, M. J. Paniccia, J. E. Bowers, "A hybrid AlGaInAs-silicon evanescent amplifier," IEEE Photon. Technol. Lett. 19, 230-232 (2007).

IEEE Trans. Parallel Distrib. Syst. (1)

R. D. Chamberlain, M. A. Franklin, C. S. Baw, "Gemini: An optical interconnection network for parallel processing," IEEE Trans. Parallel Distrib. Syst. 13, 1038-1055 (2002).

J. Lightw. Technol. (2)

B. E. Lemoff, M. E. Ali, G. Panotopoulos, G. M. Flower, B. Madhavan, A. F. J. Levi, D. W. Dolfi, "MAUI: Enabling fiber-to-the-processor with parallel multiwavelength optical interconnects," J. Lightw. Technol. 22, 2043-2054 (2004).

O. Liboiron-Ladouceur, B. A. Small, K. Bergman, "Physical layer scalability of WDM optical packet interconnection networks," J. Lightw. Technol. 24, 262-270 (2006).

J. Optical Networking (1)

I. H. White, K. A. Williams, R. V. Penty, T. Lin, A. Wonfor, E. T. Aw, M. Glick, M. Dales, D. McAuley, "Control architecture for high capacity multistage photonic switch circuits," J. Optical Networking 6, 180-188 (2007).

J. Lightw. Technol. (3)

A. Shacham, B. A. Small, O. Liboiron-Ladouceur, K. Bergman, "A fully implemented 12$\,\times\,$12 data vortex optical packet switching interconnection network," J. Lightw. Technol. 23, 3066-3075 (2005).

T. Lin, K. A. Williams, R. V. Penty, I. H. White, M. Glick, "Capacity scaling in a multihost wavelength-striped SOA-based switch fabric," J. Lightw. Technol. 25, 655-663 (2007).

L. A. B. Windover, "Parallel-optical interconnects ${>}100$ Gb/s," J. Lightw. Technol. 22, 2055-2073 (2004).

J. Opt. Netw. (2)

Nature (1)

Q. Xu, B. Schmidt, S. Pradhan, M. Lipson, "Micrometre-scale silicon electro-optic modulator," Nature 435, 325-327 (2005).

Proc. IEEE (1)

D. A. B. Miller, "Rationale and challenges for optical interconnects to electronic chips," Proc. IEEE 88, 728-749 (2000).

Other (16)

W. J. Dally, B. Towles, Principles and Practices of Interconnection Networks (Morgan Kaufmann, 2004).

J. L. Hennessy, D. A. Patterson, Computer Architecture: A Quantitative Approach (Morgan Kaufmann, 2002).

G. P. Agrawal, Fiber-Optic Communication Systems (Wiley, 2002).

K. Fukuchi, T. Kasamatsu, M. Morie, R. Ohhira, T. Ito, K. Sekiya, D. Ogasahara, T. Ono, "10.92-Tb/s (273$\,\times\,$40-Gb/s) triple-band/ultradense WDM optical-repeatered transmission experiment," Optical Fiber Communications Conf. (OFC) (2001) paper PDP-24.

R. Ramaswami, K. N. Sivarajan, Optical Networks: A Practical Perspective (Morgan Kaufmann, 2002).

R. Luijten, C. Minkenberg, B. R. Hemenway, M. Sauer, R. Grzybowski, "Viable opto-electronic HPC interconnect fabrics," Proc. ACM/IEEE (SC—05) Conf. Supercomp. (2005) pp. 18.

A. Shacham, B. G. Lee, K. Bergman, "A scalable, self-routed, terabit capacity, photonic interconnection network," Proc. Hot Interconnects: IEEE 13th Ann. Symp. High Performance Interconnects (2005) pp. 147-150.

A. Pattavina, Switching Theory—Architecture and Performance in Broadband ATM Networks (Wiley, 1998).

Mellanox InfiniScale III Switch Overview http://www.mellanox.com/products/switch silicon.php.

R. Nagarajan, "Monolithic, 10 and 40 channel inp receiver photonic integrated circuits with on-chip amplification," Proc. Opt. Fiber Commun. Conf. (OFC) (2007).

S. Sur, M. J. Koop, L. Chai, D. K. Panda, "Performance analysis and evaluation of Mellanox ConnectX InfiniBand architecture with multi-core platforms," Hot Interconnects: IEEE 15th Ann. Symp. High Performance Interconnects (2007).

A. Shacham, K. Bergman, "Utilizing path diversity in optical packet switched interconnection networks," Opt. Fiber Commun. Conf. (OFC) (2006) paper OTuN5.

A. Shacham, H. Wang, K. Bergman, "Experimental demonstration of a complete SPINet optical packet switched interconnection network," Opt. Fiber Commun. Conf. (OFC) (2007) paper OThF7.

A. Narasimha, B. Analui, Y. Liang, T. Sleboda, C. Gunn, "A fully integrated 4$\,\times\,$10 Gb/s DWDM optoelectronic transceiver in a standard 0.13 $\mu$m CMOS SOI," Int. Solid State Circuits Conf. (2007) paper 2.1.

Infinera DTN Product Brief http://www.infinera.com/products/dtn.html.

M. J. Connelly, Semiconductor Optical Amplifiers (Kluwer Academic, 2002).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.