Abstract

The memory over optical network (MONet) system is a disaggregated data center architecture where serial (HMC)/parallel (DDR4) memory resources can be accessed over optically switched interconnects within and between racks. An FPGA/ASIC-based custom hardware IP (ReMAT) supports heterogeneous memory pools, accommodates optical-to-electrical conversion for remote access, performs the required serial/parallel conversion, and hosts the necessary local memory controller. An optically interconnected HMC-based (serial I/O type) memory card is accessed by a memory controller embedded in the compute card, simplifying the hardware near the memory modules. This substantially reduces overheads on latency, cost, power consumption, and space. We characterize CPU–memory performance by experimentally demonstrating the impact of distance, number of switching hops, transceivers, channel bonding, and bit rate per transceiver on the bit error rate, power consumption, additional latency, sustained remote memory bandwidth/throughput (using industry standard benchmark STREAMS), and cloud workload performance (such as operations per second, average added latency, and retired instructions per second memcached with YCSB cloud workloads). MONet pushes the CPU–memory operational limit from a few centimeters to tens of meters, yet applications can experience as low as 10% performance penalty (at 36 m) compared to a direct-attached equivalent. Using the proposed parallel topology, a system can support up to 100,000 disaggregated cards.

© 2021 Optical Society of America

Full Article  |  PDF Article
More Like This
Optically Disaggregated Data Centers With Minimal Remote Memory Latency: Technologies, Architectures, and Resource Allocation [Invited]

Georgios Zervas, Hui Yuan, Arsalan Saljoghei, Qianqiao Chen, and Vaibhawa Mishra
J. Opt. Commun. Netw. 10(2) A270-A285 (2018)

RDON: a rack-scale disaggregated data center network based on a distributed fast optical switch

Xiaotao Guo, Fulong Yan, Jingyan Wang, George Exarchakos, Yuanyuan Peng, Xuwei Xue, Bitao Pan, and Nicola Calabretta
J. Opt. Commun. Netw. 12(8) 251-263 (2020)

In Compute/Memory Dynamic Packet/Circuit Switch Placement for Optically Disaggregated Data Centers

Adaranijo Peters, George Oikonomou, and Georgios Zervas
J. Opt. Commun. Netw. 10(7) B164-B178 (2018)

References

You do not have subscription access to this journal. Citation lists with outbound citation links are available to subscribers only. You may subscribe either as an OSA member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access OSA Member Subscription

Cited By

You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an OSA member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access OSA Member Subscription

Figures (19)

You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an OSA member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access OSA Member Subscription

Tables (2)

You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an OSA member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access OSA Member Subscription

Equations (1)

You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an OSA member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access OSA Member Subscription