Today we’re making the AWS Graviton3 processor generally available in the AWS EC2 C7g Instances.
Graviton3 and EC2 C7g Instance Type General Availability Video
Graviton3 is the third generation of the AWS Graviton CPUs and it continues to raise the bar on performance. Graviton is one of our 4 semiconductor product lines here at AWS. It’s our general-purpose CPU delivering the best price-performance of all instance types across all of EC2. We also have Trainium, for machine learning training, Inferentia, for ML inference, and Nitro, which is the foundation of our hardware accelerated hypervisor allowing us to support network hardware offload, storage hardware offload, hypervisor offload, and security.
When I joined AWS a bit more than 13 years ago I wouldn’t have guessed we would be designing and delivering custom processors but scale supports innovation at every level. We are now a leading semiconductor design house, we do custom servers, custom storage racks, our own networking equipment, AWS optimized hypervisor and operating systems, and of course the AWS services above.
I’m particularly proud of the innovations we’ve made in silicon over the years. We launched the first-generation Graviton CPU back in 2018 and it delivered significant cost savings for scale-out workloads. Only 1 year later we followed up with Graviton2 delivering up to 40% better price-performance than comparable X86 processors. And today with the announcement of the Graviton3-powered C7g instances, we deliver even higher performance for compute-intensive workloads.
AWS EC2 C7g General Availability
Today we announce the general availability of C7g. These are the first EC2 instances powered by the latest generation Graviton3 processor. C7g provides up to 25% better performance than the predecessor C6g, which already offered up to 40% better price-performance over comparable x86 instances. It’s only been two years since we launched Graviton2, and I believe that the 25% generational performance gain we’re delivering in Graviton3 is a big step forward for customers.
The AWS Annapurna team is behind all 4 of our semiconductor product lines and they continue to take on aggressive goals, work through the challenges that inevitably come up during large engineering projects, and yet still they reliably hit schedules and consistently over-deliver on performance. Graviton3 is a big step forward for us.
Graviton3 Details
Comparing Graviton3 to our previous generation processor, we see up to 2x better floating-point performance, up to 2x faster crypto performance, and up to 3x better machine learning performance. It’s based on the Arm Neoverse-V1 core where each core has a private 64KB L1 as well as a private 1MB L2 cache.
Graviton3 is the first server in the AWS fleet to support DDR5 memory and it delivers 50% more bandwidth than the previous generation DDR4. Throughout the Graviton program, we have always focused on the importance of memory bandwidth to real-world application performance and, with Graviton3, we continue that focus by offering 8 lanes of DDR5 per socket.
Each Graviton3 package has 55 billion transistors and is made up of 7 chiplets, 1 main compute-die housing all the cores, 4 DRAM controller dies, and 2 PCIe dies.
We elected to implement Graviton3 as a multi-chip module to reduce cost and to increase yields as the size and complexity of modern processors continues to grow. This improves manufacturing economics but it also allows future Gravitons to be fabricated using a leading process node while, at the same time, some of the less rapidly evolving I/O components, for example the PCIe dies, can remain unchanged.
Unlike some contemporary server CPUs, the Graviton3 compute die is monolithic where all 64 cores are on a single die. This allows us to deliver all 64 cores with uniform memory access times rather than exposing application developers to the complexity of managing different local and remote memory access latencies. Graviton3 memory latency is uniform for all cores over all memory.
AWS C7 Instance Details
Looking “logically” above Graviton to the C7g board design, each Graviton3 motherboard supports 3 servers. And, you can see on the slide, all three servers are supported by a single Nitro processor further improving overall cost and efficiency.
We exploit the power efficiency of Graviton3 to cost effectively deploy 3 servers per Rack Unit (1.75“) without requiring exotic or expensive cooling solutions. This reduces costs but the energy efficiency of Graviton3 also helps customers reduce their carbon footprints. Graviton3 instances use up to 60% less energy for the same performance than comparable EC2 instances.
Graviton3 Customer Feedback
Graviton3 has the performance needed for modern compute intensive applications such as gaming, video encoding, machine learning, and high-performance computing while also delivering the power-performance and price-performance customers seek for less demanding, general-purpose workloads.
We announced a preview of C7g at the AWS re:Invent conference last year, and hundreds of customers have since benchmarked their workloads. Let look at just a few of examples. Twitter ran a number of benchmarks representative of their workloads, and found that C7g delivered up to 80% better performance than the previous generation C6g. In addition, they reported a reduction in tail latency of as much as 35%.
Formula 1 ran their Computational Fluid Dynamics workloads on C7g and found them to deliver 40% better performance than C6gn. Honeycomb.io, an early adopter of Graviton, saw a 35% performance improvement and a 30% reduction in latency compared with C6g for their telemetry ingestion workload.
It’s exciting to see such great results from our customers and I’m looking forward to seeing more data points like these now that C7g is generally available.
Launch Video: https://youtu.be/QKo7yDAn75k
More detail on Graviton: aws.amazon.com/ec2/graviton