« journal navigation
|
A single-stage optical load-balanced switch for data centers |
Optics Express, Vol. 20, Issue 22, pp. 25014-25021 (2012)
http://dx.doi.org/10.1364/OE.20.025014
Acrobat PDF (2989 KB)
Article Navigation
▸ Article Outline
– Abstract
1. Introduction
2. Load-balanced switch architecture
3. The proposed switch architecture and principle
4. Experiments and results
5. Conclusion
– References and links
– Abstract
1. Introduction
2. Load-balanced switch architecture
3. The proposed switch architecture and principle
4. Experiments and results
5. Conclusion
– References and links
Article Tools
Full Text
Article Info
References
Cited By
Figures
Metrics
Download PDF
Viewing Equations in the
HTML Article
Optimal Viewing Recommendations
Abstract
Load balancing is an attractive technique to achieve maximum throughput and optimal resource utilization in large-scale switching systems. However current electronic load-balanced switches suffer from severe problems in implementation cost, power consumption and scaling. To overcome these problems, in this paper we propose a single-stage optical load-balanced switch architecture based on an arrayed waveguide grating router (AWGR) in conjunction with fast tunable lasers. By reuse of the fast tunable lasers, the switch achieves both functions of load balancing and switching through the AWGR. With this architecture, proof-of-concept experiments have been conducted to investigate the feasibility of the optical load-balanced switch and to examine its physical performance. Compared to three-stage load-balanced switches, the reported switch needs only half of optical devices such as tunable lasers and AWGRs, which can provide a cost-effective solution for future data centers.
© 2012 OSA
1. Introduction
With the development of cloud computing in the Internet, many applications are being hosted in data centers. The exponential increase in data traffic demands a corresponding growth of server racks in data centers. A modern data center may comprise hundreds or even thousands of server racks and each rack hosts up to 40 servers [1]. With the increase in the number of racks and the line speeds, the switch fabrics with centralized scheduler have become even more difficult to be implemented and they cannot guarantee a fixed throughput performance for different traffic patterns [2].
C. F. Lam, H. Liu, B. Koley, X. Zhao, V. Kamalov, and V. Gill, “Fiber optical communication technologies: what’s needed for datacenter network operations,” IEEE Commun. Mag. 48(7), 32–39 (2010). [CrossRef]
In order to address these issues, load-balanced routers/switches have been deployed in data centers to distribute the traffic across server racks for optimal resource utilization [3]. The load-balanced switch, which was first proposed in [4], has several attractive features: 1) it can achieve close to 100% throughput for a wide class of traffic; 2) it removes the usage of centralized scheduler by local scheduler thus has a large scalability; 3) it can avoid overload in some of switching nodes/ports and optimal resource utilization. However since the racks in data centers are interconnected through optical fiber links, the use of electronics load-balanced switches requires a large number of optical/electrical/optical (O/E/O) converters, which causes severe problems in implementation cost, power consumption and scaling.
C. S. Chang, D. S. Lee, and Y. S. Jou, “Load balanced Birkoff-von Neuman switches, part I: one-stage buffering,” Comput. Commun. 25(6), 611–622 (2002). [CrossRef]
Optical switching technique provides a possible solution to realize load-balanced switch fabric with transparency to data formats and significant reduction in power consumption. The IRIS architecture reported in [5–7] was a typical three-stage load-balanced switch of optical version. The IRIS architecture has a very large capacity as it can provide a total of N2 simultaneous connections between N input ports and N output ports. However due to the three-stage architecture, it adds relatively high power penalty to optical signals and large amount of optical components such as wavelength converters. A load-balanced optical packet switching fabric using two-stage time-slot interchangers was proposed in [8], in which the first-in-first-out (FIFO) buffer could be replaced by optical timeslot interchangers (TSI) to simplify the switch fabric, but the optical TSI is still far from practical implementation. Ref [9]. proposed a single-stage load-balanced switch fabric based on a mesh of N2 fixed wavelength lasers and the switch fabric is scalable to support high throughput.
J. Gripp, D. Stiliadis, J. E. Simsarian, P. Bernasconi, J. D. Le Grange, L. Zhang, L. Buhl, and D. T. Neilson, “IRIS optical packet router [Invited],” J. Opt. Netw. 5(8), 589–597 (2006). [CrossRef]
J. E. Simsarian, J. Gripp, J. D. LeGrange, A. L. Adamiecki, P. Bernasconi, L. L. Buhl, E. C. Burrows, J.-Y. Dupuy, C. D. Howland, F. Jorge, A. Konczykowska, and D. T. Neilson, “A load-balanced optical packet router interconnected with a 10-GbEthernet electronic router,” IEEE Photon. Technol. Lett. 23(16), 1124–1126 (2011). [CrossRef]
Motivated by these, we propose an optical load-balanced switch architecture based on an AWGR and N fast reconfigurable linecards. The switch can achieve both functions of load balancing and switching by reuse of the linecards through the AWGR, which leads to a significant reduction in implementation cost in terms of optical devices. Moreover, the switch solves packet contention by electronic buffers but without adding more O/E/O converters.
2. Load-balanced switch architecture
Before describing the proposed optical load-balanced switch, we first introduce the concept of typical load-balanced switch. Figure 1
shows the architecture of a typical three-stage load-balanced switch [4]. It consists of two identical crossbar switches and an intermediate buffer sandwiched between them. The intermediate buffer has N first-in-first-out (FIFO) queues for each input port, forming an array of virtual output queues (VOQs). Each VOQ is corresponding to an output port of the switch. The first crossbar switch is responsible for load balancing function by distributing the incoming packets to its output ports with equal probability, regardless of their actual destinations. Once a packet traverses the first-stage switch, it will be put into a VOQ according to its actual destination. Then the second crossbar switch will select the VOQs through a periodic sequence patterns (such as round-robin scheme) and deliver the packets in the selected VOQs to their actual destined output ports.
C. S. Chang, D. S. Lee, and Y. S. Jou, “Load balanced Birkoff-von Neuman switches, part I: one-stage buffering,” Comput. Commun. 25(6), 611–622 (2002). [CrossRef]
Fig. 1 Typical architecture of load-balanced switch [4].
C. S. Chang, D. S. Lee, and Y. S. Jou, “Load balanced Birkoff-von Neuman switches, part I: one-stage buffering,” Comput. Commun. 25(6), 611–622 (2002). [CrossRef]
The load-balanced switch can achieve close to 100% throughput under a wide class of traffic patterns such as bursty traffic and hot-spot traffic without dynamic scheduling algorithm. This is because the input traffic is shaped into uniform traffic by the first-stage switch. The intermediate buffer and the second-stage switch can be viewed as an input-buffered switch, which can achieve near 100% throughput under uniform traffic [10]. Although the load-balanced switch has three stages, the load balancing stage and the switching stage may be identical in terms of architecture and operation, which means that it is possible to integrate these two functions within a single-stage switch for reducing the complexity in terms of component counts. In next section, we will describe how to realize the functions of load balancing and switching in a single-stage optical switch.
3. The proposed switch architecture and principle
Figure 2(a)
depicts the proposed N × N optical load-balanced switch architecture. It is composed of an N × N AWGR connected by N fast-reconfigurable linecards. The configuration of the fast-reconfigurable linecard is shown in Fig. 2(b). It consists of four sections: tunable transmitter (Tx), optical receiver (Rx), a VOQ buffer and a local scheduler. Each linecard has an array of N VOQs corresponding to N output ports of the AWGR. From the architecture point of view, the proposed load-balanced switch is similar to most of the AWG-based optical switches, but the linecard has two operation modes: load balancing and switching.
Fig. 2 (a) Architecture of the proposed optical load-balanced switch. (b) The fast-reconfigurable linecard.
In load balancing mode as illustrated in Fig. 3(a)
, the arriving electronic packets in each linecard are converted into optical packets by the tunable transmitter. The local scheduler tunes the tunable transmitter to appropriate wavelengths such that the optical packets can be routed to all of the linecards through the AWGR. In order to convert the input traffic into uniform traffic, a cyclic-shift scheme such as round-robin scheduling scheme is employed to tune the tunable transmitters in consecutive linecard timeslot (operation period). Figure 3(b) shows an example of wavelength tuning scheme for a 4 × 4 switch in load balancing mode. Assume that the initial wavelength of a tunable transmitter is λi and 1≤i≤N, it will be tuned to λj at timeslot t, where j = [(i-1 + t) mod N] + 1. Accordingly, the optical packet in timeslot t will be routed to appropriate output port of the AWGR. By such a cyclic-shift pattern, optical packets from a linecard in consecutive timeslots are uniformly distributed to all of the linecards. Meanwhile, this linecard receives the packets from all of the linecards. Upon receiving an optical packet, the receiver will convert it to electronic packet and forward the electronic packet to appropriate VOQ according to its actual destination.
Fig. 3 (a) Load balancing mode of the fast-reconfigurable linecard. (b) Wavelength tuning scheme for a 4 × 4 switch in load balancing mode. t is operation period (linecard timeslot).
The operation of switching mode is illustrated in Fig. 4(a)
. The local scheduler selects one out of N VOQs and sends the packet at the header of selected VOQ to the tunable transmitter for transmission. In order to avoid output contention, a round-robin scheduling scheme is used again for selecting VOQs in the linecards. For example, in linecard timeslot t, linecard 1 selects 1th VOQ, linecard 2 selects 2th VOQ, …, linecard N selects Nth VOQ; in timeslot t + 1, linecard 1 selects 2th VOQ, linecard 2 selects 3th VOQ, …, linecard N selects 1th VOQ. Figure 4(b) illustrates the scheduling for a 4 × 4 switch in switching mode. The packets from other linecards are directly received at the optical receiver.
Fig. 4 (a) Switching mode of the fast-reconfigurable linecard. (b) Round-robin scheduling scheme for wavelength allocation in switching mode. t is operation period (linecard timeslot).
It should be noted that the linecard is required to complete a load balancing mode and a switching mode within an operation period. Since the wavelengths in these two modes depend on the scheduling schemes they employ, there may be two types of operation periods. In one scenario, if the scheduling scheme in the switching mode is different from that in the load balancing mode, wavelength retuning is required because the wavelength in these two modes are different) (Fig. 5
). Given that the incoming packets are of fixed durations, the time required for the load balancing and switching modes are identical. Each operation period includes a packet duration and a laser wavelength reconfiguration time. In another scenario, if the scheduling scheme in the switching mode is the same as that in the load balancing mode (i.e., the wavelengths of these two modes are the same, thus no wavelength retuning is required between them), then the time duration of the load balancing mode includes a packet duration and a wavelength reconfiguration time, while the time duration of the switching mode is only one packet duration long (Fig. 6
).
Fig. 5 Operation period of the linecard for the scenario whereby wavelength tuning is required between load balancing and switching modes.
Fig. 6 Operation period of the linecard for the scenario whereby no wavelength tuning is required between load balancing and switching modes.
To allow the linecard to switch between the load balancing and switching modes, the reconfiguration time includes the wavelength tuning time and the controlling time of the local scheduler. In addition, the guard time of the incoming packet for the scenario shown in Fig. 5 should be at least the sum of one packet duration and the duration of two wavelength reconfiguration time slots; for the scenario shown in Fig. 6, the guard time should be at least the sum of a packet duration and the duration of one wavelength reconfiguration time slot.
In the proposed switch architecture, the functions of load balancing and switching are both implemented in the fast-reconfigurable linecards through the AWGR. This scheme can simplify the switch architecture by replacing the centralized scheduler with load scheduler, which can significantly enhance the scalability in terms of scheduling. Furthermore, the AWGR in the architecture can be replaced by high port-count wavelength-assisted crossconnects [11–13] for large-scale data centers. Compared to three-stage load-balanced switches, the proposed switch utilizes only half of optical components such as AWGRs, tunable transmitters/wavelength converters and optical receivers due to its single stage design. However, since it requires the linecards to complete two modes within an operation period, this may result in a throughput reduction.
Y. K. Yeo, Z. Xu, D. Wang, J. Liu, Y. Wang, and T. H. Cheng, “High-speed optical switch fabrics with large port count,” Opt. Express 17(13), 10990–10997 (2009). [CrossRef] [PubMed]
4. Experiments and results
Since the load balancing and switching modes of a linecard in the proposed switch can operate independently, it is possible to emulate the operation of a single linecard by two linecards (one is in the load balancing mode and the other is in the switching mode). Thus we can now examine the feasibility and physical performance of the proposed switch architecture by a proof-of-concept experiment. The experimental setup is shown in Fig. 7
. Two linecards are used. Linecard 1 is configured in load balancing mode and Linecard 2 is configured in switching mode. The AWGR has 6 dB average insertion loss and 100 GHz channel spacing. In each linecard, the tunable transmitter is composed of a fast tunable laser module (Syntune S3500) reported in [13], a 10Gb/s LiNbO3 intensity modulator and a modulator driver (model: Picosecond 5865). The optical Rx has a sensitivity of −18 dBm with AC-coupled output. An optical attenuator (Att.) is used for adjusting the optical power for each optical Rx. The local scheduler is implemented using FPGA (ALTERA Stratix III) with an embedded lookup table for the wavelength selection of the tunable laser. For simplicity, the linecards were designed without electronic buffers but if these are included, it is possible to perform the experiment by using just a single linecard.
Fig. 7 (a) Experimental setup of the proposed optical load-balanced switch. (b) Photo of the linecard (the optical receiver is not shown here).
Consecutive packet sequences of 400 ns packet duration are generated by programming the pulse pattern generator (PPG) as the input electronic packets in Linecard 1. To allow sufficient time for tuning the wavelength, the guard time between two adjacent packets is set to 600 ns. In Linecard 1, a periodical command is sent by the PC to the local scheduler to tune the wavelength of the tunable laser to λa (1553.33 nm) and λb (1554.13 nm) alternatively so that the input packets are distributed to output 2 and output 3. Figure 8(a)
shows the packet traces of Linecard 1 and output 2 and output 3, respectively. It is observed that the input packets from linecard 1 are uniformly distributed to output 2 (connected to linecard 2) and output 3 (no linecard is connected though) in load balancing mode. In Linecard 2, we assume that the packets distributed to linecard 2 are destined for output 1 and output 4 alternatively. Then the tunable laser is tuned alternatively to wavelengths λa (1553.33 nm) and λc (1555.76 nm) for routing the packets to output 1 and output 4 respectively. The packet traces of Linecard 2, output 1, and output 4 are shown in Fig. 8(b). Correct switching function is observed for the packet received at Linecard 2 in switching mode.
Next, we measure the physical performance of the proposed optical load-balanced switch architecture. The input packet sequence is replaced by 231-1 pseudorandom binary sequence (PRBS). Figure 9(a)
compares the spectra of the input and output optical signals (λ = 1554.13 nm) in load balancing mode for linecard 1. From the spectrum, it is evident that the bandwidth of the input optical signal is larger than the bandwidth of the AWGR channel, which can result in degradation to signal quality. To study this degradation Fig. 9(b) shows the bit-error-rate (BER) performance in load balancing mode for the switch. The power penalty at 10−9 BER is about 0.7 dB for the optical signal at output 3. When the optical signal is tuned to output 2, there is no significant change in the BER performance for output signals, which reflects the good channel uniformity of the switch. Similar results can also be observed in switching mode as shown in Fig. 10
. It is also found that the power penalty in switching mode is close to that in load balancing mode. This is because the optical signals experience the same AWGR in load balancing mode and switching mode. This feature can avoid the discrepancy in signal quality that is caused by the non-uniformity of different stages in a multiple-stage load balance switch.
5. Conclusion
We have proposed an optical load-balanced switch architecture based on single AWGR and fast-reconfigurable linecards. The switch enables both functions of load balancing and switching to be implemented in the linecards through the AWGR. Compared to three-stage load-balanced optical switches, the proposed one needs only half of optical devices due to its single-stage architecture. This can lead to a significant reduction in implementation cost as well as power consumption. We have conducted proof-of-concept experiments to examine the feasibility and physical performance of the switch architecture. The experimental results have successfully demonstrated the functions of load balancing and switching in the switch and there is no significant degradation in optical signals quality for load balancing mode and switching mode. To achieve larger scalability, it is possible to replace the AWGR with high port-count wavelength-assisted crossconnects in the proposed switch architecture.
Acknowledgments
The authors acknowledge the funding support by the Singapore-Japan (A*STAR-JST) Joint research grant under Contract No. 102 163 0070.
References and links
C. F. Lam, H. Liu, B. Koley, X. Zhao, V. Kamalov, and V. Gill, “Fiber optical communication technologies: what’s needed for datacenter network operations,” IEEE Commun. Mag. 48(7), 32–39 (2010). [CrossRef] | |
H. J. Chao and J. S. Park, “Centralized contention resolution schemes for a large-capacity optical ATM switch,” in Proceedings of IEEE ATM Workshop, Fairfax, Virginia (1998), pp. 11–16. | |
A. Greenbery, P. Lahiri, D. A. Maltz, P. Patel, and S. Sengupta, “Towards a next generation data center architecture: scalability and commoditization,” in Proceedings of ACM workshop on Programmable Routers for Extensible Services of Tomorrow (PRESTO’08) (2008), pp. 57–62. | |
C. S. Chang, D. S. Lee, and Y. S. Jou, “Load balanced Birkoff-von Neuman switches, part I: one-stage buffering,” Comput. Commun. 25(6), 611–622 (2002). [CrossRef] | |
J. Gripp, D. Stiliadis, J. E. Simsarian, P. Bernasconi, J. D. Le Grange, L. Zhang, L. Buhl, and D. T. Neilson, “IRIS optical packet router [Invited],” J. Opt. Netw. 5(8), 589–597 (2006). [CrossRef] | |
J. Gripp, J. E. Simsarian, J. D. LeGrange, P. Bernasconi, and D. T. Neilson, “Photonic terabit router: the IRIS project,” in Proceedings of Optical Fiber Commun. Conf. (OFC) 2010 (2010), OThP3. | |
J. E. Simsarian, J. Gripp, J. D. LeGrange, A. L. Adamiecki, P. Bernasconi, L. L. Buhl, E. C. Burrows, J.-Y. Dupuy, C. D. Howland, F. Jorge, A. Konczykowska, and D. T. Neilson, “A load-balanced optical packet router interconnected with a 10-GbEthernet electronic router,” IEEE Photon. Technol. Lett. 23(16), 1124–1126 (2011). [CrossRef] | |
A. Cassinelli, A. Goulet, M. Naruse, F. Kubota, and M. Ishikawa, “Load-balanced optical packet switching using two-stage time-slot interchangers,” in Proceedings of IEICE Conference, 2004, 49–50 (2004). | |
I. Keslassy, S. Chuang, K. Yu, D. Miller, M. Horowitz, O. Solgaard, and N. Mckeown, “Scaling Internet routers using optics,” in Proceedings of SIGCOMM’03 (2003), pp. 189–200. | |
N. McKeown, V. Anatharam, and J. Walrand, “Achieving 100% throughput in an input-queued switch,” Proc. IEEE INFOCOM 96, 296–302 (1996). | |
Y. K. Yeo, Z. Xu, D. Wang, J. Liu, Y. Wang, and T. H. Cheng, “High-speed optical switch fabrics with large port count,” Opt. Express 17(13), 10990–10997 (2009). [CrossRef] [PubMed] | |
S. Qiu, S. Cao, L. Wei, H. Zhao, C. Ding, J. Xiang, Q. Zhong, M. Ye, H. Zhou, N. Deng, K. Jordan, W. Saulsberry, Z. Feng, and Q. Xiong, “A cost-effective scheme of high-radix optical burst switch based on fast tunable lasers and cyclic AWG,” in Proceedings of Optical Fiber Commun. Conf. (OFC) (2012), OTh1G4. | |
Y. K. Yeo, Z. Xu, C. Y. Liaw, D. Wang, Y. Wang, and T. H. Cheng, “A 448×448 optical cross-connect for high performance computers and multi-terabit/s routers,” in Proceedings of Optical Fiber Commun. Conf. (OFC) 2010 (2010), OMP6. |
OCIS Codes
(060.1810) Fiber optics and optical communications : Buffers, couplers, routers, switches, and multiplexers
(060.2340) Fiber optics and optical communications : Fiber optics components
(060.6719) Fiber optics and optical communications : Switching, packet
ToC Category:
Fiber Optics and Optical Communications
History
Original Manuscript: July 30, 2012
Revised Manuscript: October 2, 2012
Manuscript Accepted: October 2, 2012
Published: October 17, 2012
Citation
Qirui Huang, Yong-Kee Yeo, and Luying Zhou, "A single-stage optical load-balanced switch for data centers," Opt. Express 20, 25014-25021 (2012)
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-20-22-25014
Sort: Year | Journal | Reset
References
- C. F. Lam, H. Liu, B. Koley, X. Zhao, V. Kamalov, and V. Gill, “Fiber optical communication technologies: what’s needed for datacenter network operations,” IEEE Commun. Mag.48(7), 32–39 (2010). [CrossRef]
- H. J. Chao and J. S. Park, “Centralized contention resolution schemes for a large-capacity optical ATM switch,” in Proceedings of IEEE ATM Workshop, Fairfax, Virginia (1998), pp. 11–16.
- A. Greenbery, P. Lahiri, D. A. Maltz, P. Patel, and S. Sengupta, “Towards a next generation data center architecture: scalability and commoditization,” in Proceedings of ACM workshop on Programmable Routers for Extensible Services of Tomorrow (PRESTO’08) (2008), pp. 57–62.
- C. S. Chang, D. S. Lee, and Y. S. Jou, “Load balanced Birkoff-von Neuman switches, part I: one-stage buffering,” Comput. Commun.25(6), 611–622 (2002). [CrossRef]
- J. Gripp, D. Stiliadis, J. E. Simsarian, P. Bernasconi, J. D. Le Grange, L. Zhang, L. Buhl, and D. T. Neilson, “IRIS optical packet router [Invited],” J. Opt. Netw.5(8), 589–597 (2006). [CrossRef]
- J. Gripp, J. E. Simsarian, J. D. LeGrange, P. Bernasconi, and D. T. Neilson, “Photonic terabit router: the IRIS project,” in Proceedings of Optical Fiber Commun. Conf. (OFC) 2010 (2010), OThP3.
- J. E. Simsarian, J. Gripp, J. D. LeGrange, A. L. Adamiecki, P. Bernasconi, L. L. Buhl, E. C. Burrows, J.-Y. Dupuy, C. D. Howland, F. Jorge, A. Konczykowska, and D. T. Neilson, “A load-balanced optical packet router interconnected with a 10-GbEthernet electronic router,” IEEE Photon. Technol. Lett.23(16), 1124–1126 (2011). [CrossRef]
- A. Cassinelli, A. Goulet, M. Naruse, F. Kubota, and M. Ishikawa, “Load-balanced optical packet switching using two-stage time-slot interchangers,” in Proceedings of IEICE Conference, 2004, 49–50 (2004).
- I. Keslassy, S. Chuang, K. Yu, D. Miller, M. Horowitz, O. Solgaard, and N. Mckeown, “Scaling Internet routers using optics,” in Proceedings of SIGCOMM’03 (2003), pp. 189–200.
- N. McKeown, V. Anatharam, and J. Walrand, “Achieving 100% throughput in an input-queued switch,” Proc. IEEE INFOCOM96, 296–302 (1996).
- Y. K. Yeo, Z. Xu, D. Wang, J. Liu, Y. Wang, and T. H. Cheng, “High-speed optical switch fabrics with large port count,” Opt. Express17(13), 10990–10997 (2009). [CrossRef] [PubMed]
- S. Qiu, S. Cao, L. Wei, H. Zhao, C. Ding, J. Xiang, Q. Zhong, M. Ye, H. Zhou, N. Deng, K. Jordan, W. Saulsberry, Z. Feng, and Q. Xiong, “A cost-effective scheme of high-radix optical burst switch based on fast tunable lasers and cyclic AWG,” in Proceedings of Optical Fiber Commun. Conf. (OFC) (2012), OTh1G4.
- Y. K. Yeo, Z. Xu, C. Y. Liaw, D. Wang, Y. Wang, and T. H. Cheng, “A 448×448 optical cross-connect for high performance computers and multi-terabit/s routers,” in Proceedings of Optical Fiber Commun. Conf. (OFC) 2010 (2010), OMP6.
Cited By |
 Alert me when this paper is cited |
OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.
« Previous Article | Next Article »
OSA is a member of 