# Single Cell Analysts and Investors are fucking wrong, and NVIDIA interest in INTEL proves it
AI runs on GPUs, right? RIGHT?
The answer is it depends. And most people (including developers) don't know that, but they will know, eventually
Let's begin with memory bandwidth from now on BW
Each token generation requires accessing ALL model weights, that means for each output token you need to load the entire model so how fast you load that model in memory, for that bw was the major limitation of CPUs ultill the next gen
The current top xeon has 0.614TB/s which is a ton, and could be working on dual configuration with 6TB RAM at 1,7TB/s but Top GPUs like A100 have 2TB/s and H100 has 3TB/s, the next generation of Intel Xeon 18A is expected to have \~2TB/s. Positioning near A100.
BW tl;dr: \*\*Usually doubling bandwidth roughly doubles token generation speed
BW deepdive: [https://www.reddit.com/r/LocalLLaMA/comments/1brcnps/is\_inferencing\_memory\_bandwidth\_limited/](https://www.reddit.com/r/LocalLLaMA/comments/1brcnps/is_inferencing_memory_bandwidth_limited/)
Let's keep going, usually big chunk papa models cannot and will not work on a CPU period, but advances in quantization(from now on Q\_) made possible to run big papa models like gpt-oss 120b run on a CPU with a loss in accuracy near to 2%
Usually models are based on 16B and Q\_ down to 8 or 4, each time you half the number of bits, it halves the model size. For example GPT-OSS could run Q\_8 with only 60GB what makes possible to run at any modern workstation CPU RAM. You have 2 modules of 32GB DDR5? Boom you can run Q\_ big papa models
Q\_ tl;dr: CPUs can run big models on RAM now
Q\_ deepdive: [https://developer.nvidia.com/blog/optimizing-llms-for-performance-and-accuracy-with-post-training-quantization/](https://developer.nvidia.com/blog/optimizing-llms-for-performance-and-accuracy-with-post-training-quantization/)
[https:\/\/developer.nvidia.com\/blog\/optimizing-llms-for-performance-and-accuracy-with-post-training-quantization\/](https://preview.redd.it/xjqr7wqojcrf1.png?width=1080&format=png&auto=webp&s=7b50372aec5d17a684cfa3882534067d4bd58844)
Let's talk about money, while a GPU is the king and surpasses the CPU in every metric possible, CPUs are crazy energy efficient and versatile, but what does it mean? It means that server providers are more prone to buy a CPU because they have a wider set of applications, and not only inference so they can have less idle time and more money back from their chips, since we discussed earlier, not everyone needs the biggest model available and token generation speed in new Xeon processors with CPU tokens per second sucks vs GPU they are not even close, but! Look at performance:
[https:\/\/www.nscale.com\/blog\/cpu-inferencing-why-not](https://preview.redd.it/8h79t8kqjcrf1.png?width=1080&format=png&auto=webp&s=6e3f3f23a5491adf4064be81e5af3be9a53dd86c)
Oh boy, you can bet small to mid-sized businesses are going to notice this, since this graph is not suitable for large environments where inference is handled by a cluster using vLLM or other software that allows load distribution and pipeline parallelism.
And they are not only crazy efficient but Intel is positioning at the most competitive chips for LLM inference per dollar:
[https://www.reddit.com/r/LocalLLaMA/comments/1jxwk05/intel\_6944p\_the\_most\_cost\_effective\_cpu\_solution/](https://www.reddit.com/r/LocalLLaMA/comments/1jxwk05/intel_6944p_the_most_cost_effective_cpu_solution/)
Lastly, the necessity of private LLM execution for small and mid-sized businesses is a niche that perfectly meets CPU inference.
[https://openmetal.io/resources/blog/private-ai-cpu-vs-gpu-inference/](https://openmetal.io/resources/blog/private-ai-cpu-vs-gpu-inference/)
On top of all of this, Intel and NVIDIA reached an agreement which announcement literally says: "For data centers, Intel will build NVIDIA-custom x86 CPUs that NVIDIA will integrate into its AI infrastructure platforms and offer to the market."
[https://newsroom.intel.com/artificial-intelligence/intel-and-nvidia-to-jointly-develop-ai-infrastructure-and-personal-computing-products](https://newsroom.intel.com/artificial-intelligence/intel-and-nvidia-to-jointly-develop-ai-infrastructure-and-personal-computing-products)
So you can bet every board in a data center that works with Nvidia will have an Intel-manufactured CPU, which means that NVIDIA could be one of the first clients of the new Foundries that could be followed by other manufacturers and fabless chip designers.
tl;dr: buy intel
positons: [https://www.reddit.com/r/wallstreetbets/comments/1fw14ev/in\_intc\_i\_trust/](https://www.reddit.com/r/wallstreetbets/comments/1fw14ev/in_intc_i_trust/)
