16128 cores, 24GB GDDR6X, 450W, 2x faster than RTX 3090

Specifications for NVIDIA’s top-of-the-line Ada Lovelace graphics card have reportedly been leaked. The GeForce RTX 4090 leveraging the AD102 chip will feature a total of 16,128 FP32 cores in 126 SM, 63 TPC, and 11 GPC. This chip will be paired with 24GB of 21Gbps GDDR6X memory on a 384-bit bus, just like the RTX 3090 Ti. Lovelace is likely to borrow some of Hopper’s features, especially Thread Block Memory Sharing, which together with the 96 MB of L2 cache drastically increase SM utilization and bandwidth, respectively.

To make programming on GPUs with more than 100 SMs more efficient, Hopper and Lovelace will group every two thread blocks in a GPC into a cluster. The main purpose of thread block clusters is to improve multithreading and SM utilization. These clusters run concurrently in SM on a GPC.

OK, let’s do a new summary.
RTX 4090, AD102-300, 16128FP32, 21Gbps 24G GDDR6X, 450W, ~2×3090.
I am disappointed with RDNA3.
That’s all.

– kopite7kimi (@ kopite7kimi) May 16, 2022

Thanks to an SM-to-SM network between the two thread blocks in a cluster, data can be shared efficiently between them. This is going to be one of the key features that promotes scalability in Hopper and Lovelace, which is a key requirement when increasing the number of cores/ALUs by more than 50%.

Lastly, let’s not forget that the RTX 4090 will not feature the full AD102 chip, however it will offer twice the performance of its predecessor. The TGP will eventually only be 450W, a far cry from the previously rumored 600-900W abominations. The RTX 4090 Ti, which may launch later with the AD102 die fully enabled, is more likely to come with a 600W TGP.