Apple Inc. announced plans to accelerate spending in the United States, citing $350 billion of spending over the next five years. The company has cited recent tax rules and its status as being the largest US taxpayer. The company specifically earmarked "over $10 billion" for "investments in data centers across the US." We estimate that this will add about $2 billion more per year than the company was already spending, which the company says has resulted in datacenters in seven US states, including North Carolina, Oregon, Nevada, Arizona and a planned project in Iowa. Based on these estimates, we believe Apple's US datacenter spending rate will now challenge the capital spending rates of Facebook. The company also announced plans to build a Reno, Nevada datacenter.
This capital spending acceleration on datacenters has been timed with the completion of its Cupertino-based mega-campus, which was a significant capital expenditure.
With Apple's datacenter plans are clearly accelerating, it is poised to tap suppliers for more datacenter equipment. We expect that the main suppliers of network equipment will be fighting hard for Apple's business. Examples of such suppliers competing for the new capital spending plan will likely be, in optical equipment, Nokia, Ciena, Finisar, in routing, Nokia, Cisco, and in switching, Cisco, Broadcom, and Arista. It is possible that with Apple's increasing scope of datcenter building, it may seek to bring more equipment design in-house, more similar to larger datacenters, including Facebook, Microsoft, Amazon and Google. Additionally, as the datacenters become more numerous and larger, it will almost certainly require that Apple will implement different network architectures.
The market is in a period of rapid adoption of higher speeds led by the hypserscalers. The industry used 2016 and 2017 to adopt 25 Gbps and 100 Gbps port speeds based on 25 Gbps SERDES technology. As we enter 2018, those same hyperscalers are about to adopt 50 Gbps, 200 Gbps, and 400 Gbps port speeds based on 50 Gbps SERDES at a record shattering pace. In the data center alone, there are now eight unique port speeds, with countless more unique variations of form factor and pluggable distance.
The market will need additional bandwidth beyond what is currently available today. Several of these technologies were highlighted at the OIF Forum conference. 100 Gbps SERDES will help drive the industry towards that goal. Looking forward, 100 Gbps SERDES will help drive wave two of 400 Gbps, which will help enable Ethernet to extend its reach well outside of short reach data center distances. At the same time, it will also have a long life, with use cases ranging from enterprise to service provider.
The big question often asked is why after so many years for the market to adopt 10 Gbps, will we suddenly see a more rapid pace of adoption going forward?
There are many reason why, but we should look at a few things are different this time. First, the hyperscalers are a new type of customer. Hyperscalers truly bring a new scale to networking and compute in a way that makes the traditional SPs look small. Second, SDN, the hyperscalers have done something unique here that often gets overlooked that is occurring right now, in the second half of this decade. Hyperscalers are increasing the utilization rate of their compute and networking resources. For compute, this is approaching 100% utilization so the industry is in a period where hyperscalers, using SDN are able to grow network bandwidth at a pace faster than what the CPU is scaling.
This more rapid pace will not continue forever, but is one of the reasons why innovation over the next several years will occur more rapidly than historic norms and why it will be important for the industry to think about how to invest across speeds and technologies in order to better leverage existing investments. If not, the pace of innovation will simply be too much to recoup investment in the compresses timelines we are currently in.
Broadcom joined both Innovium and Nephos by publicly announcing 12.8 Tbps fabrics with its Tomahawk 3 product line. We love new data center silicon from all vendors, it is something we track closely and we see these as a disruptive technologies to the networking ecosystem and an enabler of next generation cloud architectures. There will be many more such announcements in 2018. Here are some of our takeaways as we enter 2018.
More rapid innovation cycle – Even noted in the Broadcom's Tomahawk 3 press release, we see the demand requirements of the hyperscalers as driving a more rapid cycle of silicon over the next couple generations. Tomahawk 3 is being introduced less than the typical 24 months we see separating prior between generations of data center fabric semiconductors. This will put significant pressure on parts of the supply chain, especially on optics vendors. Optics vendors are still ramping for 100 Gbps and now must support both OSFP and DD-QSFP for 400 Gbps, essentially doubling their product diversity needs. Not only are there more form factors, but there are also different variations of distance and specifications that increase the complexity.
What next – We see two waves of 400 Gbps, the first being based on 56 Gbps SERDES, the second coming in the 2020 timeframe based on 112 Gbps SERDES. We believe 800 Gbps is not that far off in the horizon as hyperscalers like Amazon and Google continue to grow. We note that the hyperscalers are about to be 3-4 generations ahead of the enterprise. This type of lead and technology expertise really changes the conversation around Cloud. We saw this at Amazon re:Invent with their Annapurna NIC, the Cloud is doing things that just aren’t possible in the enterprise, especially around AI, machine learning, and other new applications that take advantage of the hyperscalers size.
2018, the Year of 200 Gbps and 400 Gbps – In 2018 we will see commercial shipments of both 200 Gbps and 400 Gbps switch ports. We see significant vendor share changes because of this. Simply put the Cloud, especially the hyperscalers will be that much bigger by the end of 2018 and they buy a different class of equipment then everyone else. This will continue to cause the vendor landscape to evolve.
New CTO - It was great to see and hear Juniper’s new CTO, Bikash Koley at NXTWORK. The message was very clear that new networks need to be built, not only from the speeds and feeds technology point of view, but from the operator point of view. Scale and simplicity are very important and only going to be more critical as billions of IoT devices dump traffic and data on the network. Juniper is looking to take the network it is building for some of the hyperscalers and Service providers and take help the tier 2/3 Cloud as well as the enterprise customer adopt a cloud architecture for the future.
Contrail – The big announcement was OpenContrail joining the Linux foundation. What we saw in talking to customers and listening to the talks was that Contrail has significant commercial adoption, definitely larger than perceived in the marketplace. We see the largest hyperscalers and service providers as having their own controllers, but see Juniper leveraging its expertise in this area into smaller Cloud and service providers. This will give Juniper the opportunity to build, operate, and transfer for many customers.
We were impressed with the caliber of customers that Juniper has. Juniper has premier Cloud, Service Provider, and Enterprise customers. We enjoyed listening to Twitter’s VP of engineering on stage as an example. We see Juniper as one of just a handful of vendors that can support a customer base with this breath and complexity. We note Juniper mentioned complexity and simplicity throughout the sessions; we only see networking getting more complex, especially as we move beyond 100 Gbps. It is up to vendors to help the human scale with that complexity and we saw Juniper give many good examples during NXTWORK 2017.
Nokia was bullish on its technology developments and cautious on telco capex. CEO Rajeev Suri described the telecom market as having tough market conditions and expects telco equipment market revenues down in 2018. He said the Swedish competitor pricing aggressively and that it will be difficult to keep share in China with 5G ramp. He said Nokia is executing on plan to grow non telco verticals – and it will take 3+ years before it is sizable enough to potentially offset telco challenges. He emphasized a key strength is its focus on cost reductions. He highlighted success in cable MSO market and that its FP4-based router will ship in a few days. Suri thinks 5G could roll out in China first, maybe tied with US. Nokia CEO's comments were quite similar about 5G rollout timing expectations compared to comments made at Huawei’s recent conference (2019 initial deployments; chipsets 2019; broadening deployments in 2020).
Some more details from Rajeev Suri's presentation:
Ericsson. We were surprised that the CEO of Nokia took the opportunity to take some digs at Ericsson. He said that Ericsson is pricing aggressively; it also shared some quantitative statistics about competitive take-outs of Ericsson installed base. He argued that it has broadest portfolio in industry (fixed, software are examples).
China. Two interesting comments about China – a) will be difficult to maintain share in China as 5G rolls out, b) China might be first to deploy 5G.
Emphasized that Amazon Web Services is making a presentation at this conference. Said its new FP4-based routers are more efficient than any competitor and will be so for at least the next year.
Other presenters made comments about 5G mainly. Here are some interesting comments:
Artificial Intelligence. The company has a lot of network automation technology that is it working on but would not share details about this technology. We guess Nokia is more open with its customers and that it'll make announcements at MWC '18 in Barcelona.
5G Radio. Beamforming is a key technology that will be highlighted in 5G. Also, the company's separation of Stage 1 and Stage 2 MIMO processing makes the bandwidth needs from baseband to array be much less than competitors. Additionally, the company explains that its expected systems will have dramatically higher efficiency than competitors - again, the company kept its secrets here under wraps. Late 2018 will see first 5G deployments, going into 2019.
IP/Optical. Basil Alwan, President IP/Optical division said the first FP4-based product went to production end of the last week. Major customers will take shipment before the end of this quarter. The company will ship its 57 Terabit router during 1Q18. SD-WAN will replace MPLS VPNs over time, perhaps at a rapid pace.
Service Provider presentations:
Amazon Web Services IoT. Satyam Yadav, GM of AWS IoT made a presentation at this meeting. The presentation focused on how Amazon's IoT software and its services would be used in a partnership with Nokia to deliver Amazon IoT services.
Sprint. Ron Marquardt, CTO. He is not sure how much customers might pay for lower-latency connections available from 5G. Now that the uncertainty of the T-Mobile US acquisition is beyond us, Sprint says it is rapidly focusing on spending to upgrade its network. Sprint also said it plans to deal with fewer vendors in the future.
Elisa. The Finnish operator presented data about its extraordinary data traffic growth and its per-subscriber data usage being far above competitors as well as other service providers in the world.
This week, we attended the Global Mobile Broadband Forum, held in London, and found several interesting points we thought we would share. Much of the focus of the conference was about 5G wireless networks, and since the show was in London, many of the service providers who we met with and listened to were European. The sponsor of this event was Huawei.
Back on 5/9/17, we discussed how Alcatel-Lucent Enterprise (ALE) had announced its in-house developed enterprise-class WLAN products. A couple days ago (on 9/20/17), ALE announced further developments to its WLAN product line and now has a broadening product line of Stellar WLAN products. Initially, ALE announced a lower end Stellar product. Now, with this 9/20/17 announcement, in Stellar products have moved up-market and include the AP1220, AP1230 and AP1251. ALE continues to offer the HPE Aruba products to customers, but it is clear, in looking at the amount of space on the WLAN products page at ALE's website, that it is increasingly promoting its in-house Stellar brand - and why not?
It is encouraging to see a new set of products enter the enterprise-class WLAN market with ALE's announcement. We will be tracking ALE's progress in the coming quarters, but initial results of the Stellar product line have shown it is growing strongly as a proportion of total WLAN revenues.
Yesterday, Comcast announced its ActiveCore SDN platform. We are attending Comcast’s analyst day and learned quite a bit on day one, but wanted to focus this blog on Routing.
To date, there has been an argument of SD-WAN and it ability to replace expensive telco MPLS solutions. If a customer continues using MPLS, it is likely they will keep their old router or buy a new router. But what happens when a startup uses ActiveCore without a legacy infrastructure or builds a new branch(greenfield)? In many cases, the CPE box provided by Comcast and 1 Gbps bandwidth is more than enough, especially for a mobile Cloud based workforce(many new companies fall into this category).
We continue to believe MPLS will live on and have a very long tail, but are seeing capable platforms threaten traditional vendors and the branch/access router market. It will be interesting to see what other SPs around the world do for SD-WAN as well as how traditional vendors and startups address this changing dynamic in businesses of a mobile workforce that uses Cloud platforms and a VPN tunnel is enough to connect back to headquarters without MPLS and/or without the need for a traditional router.
We attended the Deutsche Bank Tech conference this week and met with a ton of companies. It is always interesting to see the difference in questions from investors vs. those directly in the industry. During the conference, each company put spin and had different definitions of Data Center Interconnect (DCI) that helped address their specific portfolio. This is very similar to the early Cloud days where every vendor and component manufacturer said they sold into the Cloud. Fast forward to today, and very few vendors sell to the Cloud. We see a similar end game with many suppliers being squeezed out of the DCI market as it matures.
The lack of clarity created confusion amongst the investors as they went from session to session and we think is a short term negative to the market.
We are very excited to have holistic DCI coverage. One that looks at legacy approaches around Optical and the new approach of using switching and routing. We are hopeful that the market will move towards one consistent definition of DCI as that will be better for the market itself and the suppliers in that market, but see that as unlikely as many vendors seem to be digging into a definition that is self serving and more focused on legacy products vs. what customers will want in the future.
We look forward to many future conversations on DCI.
Last week we attended Huawei Connect conference in Shanghai which is turning into a massive event for Huawei with significant customer attendance. It was tons of fun to talk to customers, catch up with friends, and the different parts of Huawei while on the show floor. While there are ton of highlights from the show, here are a few highlights that peaked our interest.