AMD Instinct MI350 Launch Event Coverage

AMD Instinct MI350 Series MI350X And MI355X

The AMD Instinct MI350 is the company’s new series to go head-to-head with the NVIDIA B200 generation. The new chips come in two flavors, the AMD Instinct MI350X and AMD Instinct MI355X. Today, we will get into the new series, some of the performance claims, and even some of the future plans.

AMD Instinct MI350 Series Launch with the MI350X and MI355X

First off, a check on where we are at in the journey. In 2023, AMD shipped the Instinct MI300 series. The MI300A found great success at places like HPC with El Capitan (and the GIGABYTE G383-R80-AAP1) while the MI300X series did well at Microsoft Azure for AI. The MI325X is effectively an update to the MI300X much like the NVIDIA H200 updates the H100. The MI350 series is more than that with some architectural changes an a pivot to focus more on AI than HPC.

AMD Instinct Roadmap 2023 To 2026

Headline features are 20PF of FP4 and FP6 performance. AMD actually is putting a lot in this generation on FP6 which we will get into in the deeper dive. There is also 288GB of HBM3E memory.

AMD Instinct MI350 Series Key Specs

The UBB8 form factor is the 8-GPU form factor so that it can work in the same systems as NVIDIA HGX 8-GPU platforms. We covered this in our Substack a few months ago.

The MI300 series comes in two flavors, the AMD Instinct MI350X and AMD Instinct MI355X. Both are very similar and use 3nm for the compute complexes.

AMD Instinct MI350 Series MI350X And MI355X

Here is the side-by-side comparison. You will notice that the MI355X is the faster of the two. The key is really that the MI355X runs at 1400W and is liquid-cooled.

AMD Instinct MI350X And MI355X Spec Side By Side

Just for some level of comparison, here is AMD’s competitive slide versus the GB200 and B200. AMD’s point is that it now has specs that are equal if not better than NVIDIA Blackwell (B200) parts.

AMD Instinct MI355X V NVIDIA GB200 B200 Comparison

Here are the AMD Instinct MI350 platforms where we can see the air cooled and liquid cooled version. The UBB8 platforms are designed for lower desnity air-cooled and higher-density liquid-cooled racks.

AMD Instinct MI350X Air Cooled And MI355X Liquid Cooled

Next, let us get to the performance claims.

AMD Instinct MI355X versus NVIDIA GB200 and B200 Inference and Training Performance

On the performance side, AMD is doing something here that we need to poitn out. They are using the MI300X not the MI325X as the comparison point as well as the MI355X instead of the MI350X. Still, the gains are huge.

AMD Instinct MI355X To MI300X Performance 1

Getting into popular models like DeepSeek R1, Llama .3 70B and Llama 4 Maverick, AMD is claiming 3-3.3x more performance.

AMD Instinct MI355X To MI300X Performance 2

AMD also is doing some competitive benchmarking disucssing its parts compared to NVIDIA GB200 and B200. We should expect a response from NVIDIA soon on this one. Perhaps the takeaway is that AMD thinks it is in the ballpark of NVIDIA.

AMD MI355X To NVIDIA GB200 And B200 Performance 1

On the training side, AMD also shows massive changes. Something to keep in mind on both sides is that memory capacity and bandwidth can matter quite a bit. If you have to move off a GPU and access remote GPU memory, then it can lead to a big performance hit.

AMD MI355X To MI350X Performance 3

AMD also has some numbers on pre-training and fine-tuning versus NVIDIA Blackwell.

AMD MI355X To NVIDIA GB200 And B200 Performance 2

To be clear, we expect NVIDIA is going to sell a lot of Blackwell. AMD is coming in saying it is in the ballpark in terms of performance, and then it is lower in cost than the NVIDIA Blackwell platforms leading to lower $/token figures.

AMD Instinct MI350 Rack Scale Infrastructure

Today, training clusters and inference clusters are rarely a single 8-GPU system. Instead, folks are focused on rack-scale infrastructure as AI clusters are being designed alongside data centers these days.

AMD Instinct MI350 Rack Scale Infrastructure

AMD’s vision is to use open standards like UltraEthernet Consortium (UEC) and UALink for scale out and scale up. Some of those are next-generation technologies, but for now, the focus is showing the racks with 64 to 128 GPUs per rack and offering up to 36TB of HBM3E memory in a single rack.

AMD Instinct MI350 Rack Scale Infrastructure Solutions

Today AMD will get into more than just these accelerators, including its roadmap around rack-scale, so you should think of this as the current-generation.

Final Words

New chips are always exciting, we are going to have a deeper dive on some of the architectural changes and the other components today. What will be interesting to see is that AMD is focusing on the value-based sales with this generation versus NVIDIA. The next-generation is where things get a lot more exciting. Still, unlike Intel, AMD is commited to shipping new generations of AI focused GPU products at an annual cadence.