“The SC Conference has always been about innovation and showing at a small scale what will be possible on a large scale a year or so later. Just imagine the scale 5 years from now!” – Rob Vietzke, Internet2 VP of Network Services
The SuperComputing (SC) Conference once again brought together members of the international high-performance computing, networking, storage and analysis community to demonstrate advanced computing resources and share the latest technologies, research and ideas. Built annually by conference volunteers, SCinet is the fastest and most powerful high-performance network in the world, providing SC attendees with the innovative network platform necessary to globally interconnect, transport and display computing research during the SC Conference.
As 400 gigabit Ethernet (400GbE) is now hitting prime time, it is achieving higher volumes and lower costs per gigabit of traffic that are critical to network operators, optics vendors, silicon manufacturers and router and switch vendors alike. Leading network and data center operators will rely on 400GbE to power the global economy and lead us into the next era of networking – one with greater accuracy, faster deployments, denser configurations and a more attractive price-per-port.
Reinforcing SCinet’s reputation as the most powerful network in the world, Internet2, in collaboration with Juniper Networks, Ciena and CenturyLink, successfully delivered 400 gigabit Ethernet live traffic from Denver to Chicago – the longest recorded distance in history at just over 1,367 miles (2,200 kilometers) and the first occurrence for an education and research network such as SCinet. To put this in perspective, the distance from Denver to Chicago is about a 20-hour drive – pretty much the same amount of time it takes to drive on the 101 on a Friday afternoon. SCinet delivered an unprecedented amount of bandwidth at 4.2 Tbps and set a new record for the longest live 400GbE connection to-date.
Internet2, a Juniper Networks customer, has played a fundamental role in the SCinet for the past 20 years. This year, the company was challenged by the science community to provide 2.1 Tbps capacity to Denver for the SC Conference week. The innovative and collaborative culture of the SC Conference allows for the constant mingling and development of professions, ideas, technologies and experiences – a key reason for Juniper and Internet2’s longtime support and commitment to SCinet. Comprised of network engineers, researchers and information technology experts, volunteer teams demonstrated the new innovations of 400G at the SC Conference and set the stage for the technologies that lie ahead.
We recently had the opportunity to sit down with Rob Vietzke, the VP of Network Services at Internet2, to talk about our collaboration at SC19 and what this higher capacity, 400GbE milestone means to the future of networking.
Q: How was Internet2 involved with the SC Conference this year?
Internet2 contributes to the SC Conference each year in support of the amazing science demonstrations its members present. This year, we were initially challenged to deliver 2.1 terabits per second of temporary capacity to the SC Conference floor from points all around the country. The stretch goal was to deliver a 400 gigabit Ethernet link from Denver to Chicago. We handled half of the overall network bandwidth to the conference exhibit hall – with many providers making up the other 2.1 Tb – connecting the entirety of the Colorado Convention Center to the broader Internet and private research networks around the world. The SC Conference has always been the premier event for innovation, allowing us to showcase what we’ll be deploying for production next year, which is 400G.
Q: What were a few of the technologies showcased at this year’s SC19 Conference?
The SC Conference has always been a place to showcase the latest and greatest cutting-edge technologies in network, science information systems, hardware, software and security – and this year was no different. The floor itself delivered scale and a real-world noisy spectrum to host a 5G CBRS demo in exploration of private LTE network reliability. With the increased availability of 400GbE packet optical switches and systems, it was the perfect time to showcase the power of 400GbE and what a network capable of 400G and 800G wavelengths looks like.
Internet2 was also involved in a very ‘cool’ demo with the IceCube Neutrino Observatory and Open Science Grid (OSG). In this very impressive experiment, the IceCube/OSG team grabbed and combined every available GPU resource from three main cloud providers: Amazon, Microsoft and Google in all of the Americas, Europe and Australia. The experiment was able to achieve 80% of the capacity of the world’s largest supercomputer on demand – a truly meaningful demo for science, cloud and networking and a pathfinder for exoscale computing.
Q: What did the success of this live 400GbE connection mean – both for Internet2 and the future market?
For Internet2, the ability to demonstrate we could do router-to-router, switch-to-switch 400G over long distance shows that what we’ll be deploying for production in 2021 is doable. It not only foreshadows important future technologies, but it confirms our thoughts that the market will be ready to support 400G backbone links beginning this year.
Q: What challenges were involved with getting the 400G link live and successful?
This was actually a new optical path for us, so at that distance we had a lot of blocking and tackling to clean fibers. It was tuning new gear along a path with a couple dozen amplifiers on new fibers. We had an amazing team at Juniper, Ciena, CenturyLink and Internet2 and a deadline to motivate us. For the most part, the turn-up work went smoothly. Once we had a clean optical path, the 400G popped up and worked between the routers and switches when we plugged it in on both ends.
Q: Can you elaborate on Juniper’s involvement particularly in SC19?
We’re always looking to explore and demonstrate new technologies like 400GbE to understand their strengths and limitations before they become available. Internet2 was able to provision a 400G link from Chicago to Denver using our fiber footprint, but we needed to connect to a router to use it.
Juniper stepped up to that challenge with two Juniper PTX10003 Core Routers that aggregated traffic on each, linked to a QFX5220 Switch with 400G QSFP56-DD pluggables. Juniper additionally provided a core router to the SCinet network itself. What’s unique about Internet2 and partners like Juniper is that we get to collaborate on projects like this that are purely about innovation and proving what’s next and what’s possible – we really enjoy being a part of that.
Q: Going forward, what emerging trends can we expect out of SC19?
Traditionally, Internet2 has upgraded core “big chassis” and changed line cards as technology changes on a multi-year timescale; now, we’re looking at smaller form factors and how to upgrade networks with optimization, security, automation and programmability in a much more agile and rapid way. The SC Conference was a place where you saw that trend emerging, both in the amazing SCinet network and also in many booth demonstrations – that was good to see.
The other trend you saw was the deeper integration of high-performance storage, compute and programmable networks and the accelerated development of software to support automation, orchestration and analytics. The SC Conference is where evolutionary changes like these appear first.
More than 200 volunteers came together in early March to discuss ideas and ask contributors what’s possible. After an immense amount of planning, hard work and dedication, this year’s event was a huge success, smashing records for the longest 400G live connection at 1,367 miles. In acknowledgement of their collaborative innovation in pushing the boundaries of high-performance networking, Internet2, Juniper Networks, Ciena and CenturyLink were recognized with this year’s Spirit of Innovation Award. With the support of SCinet contributors, dedicated volunteers and devoted companies like Internet2 and Juniper, members of the HPC community at the SC Conference were able to demonstrate the advanced computing resources of their institutions and showcase cutting-edge applications, experiments, technology and ideas – all symbols of the SC Conference.
Join us on June 10 (US/EMEA) and June 11 (APAC) to discover how network architectural shifts create business value at our Juniper Virtual Summit for Cloud & Service Providers. Register here.
