Windows 12 and the coming AI chip war

For more than a year, we’ve been hearing rumors that Windows 12, the next major release for Microsoft’s venerated operating system, could be coming as soon as 2024. As always, Microsoft has refused to confirm or deny those rumors. So when Intel’s chief financial officer David Zinsner recently hinted at a Windows release expected next year that would drive new PC (and thus processor) sales, tongues began wagging.

Zinsner didn’t set out to tip Microsoft’s hand — he did it accidentally during a conversation with a financial analyst at Citigroup’s Global Technology Conference in September. In a transcript posted by investor site Seeking Alpha (free registration required), he is quoted as saying, “We actually think ’24 is going to be a pretty good year for client [processor sales], in particular, because of the Windows refresh. And we still think that the installed [PC] base is pretty old and does require a refresh, and we think next year may be the start of that, given the Windows catalyst.”

Most pundits came to the conclusion that he was referring to Windows 12, but the operative word here is “refresh” and not a version number. Whatever Microsoft has planned for next year, the label is irrelevant. If it requires new hardware, it will be a significant update to the Windows operating system, with heavy emphasis on (what else?) artificial intelligence and well beyond the capabilities of the current Copilot for Windows, its generative AI tool.

“[This is] something much more rich into Windows that will drive higher compute demands,” said Bajarin. “For the first time in a long time, you’re going to see software that requires levels of compute that we don’t have today, which is great for everyone in silicon. A lot of it’s based around all this AI stuff.”

GenAI on the desktop?

The explosion of generative AI tools like ChatGPT and Google Bard — and the large language models (LLMs) that underlie them — brought on server farms with thousands of GPUs. What could one desktop PC bring to the table? The answer is complex.

First, the AI on a client will be inferencing, not training. The training portion of genAI is the process-intensive part. Inference is simply matching and requires a much less powerful processor.

And enterprises are extremely uncomfortable with using a public cloud to share or use their company’s data as a part of cloud programs like ChatGPT. “The things that I hear consistently coming back from CIOs and CSOs are data sovereignty and privacy. They want models running locally,” said Bajarin.

AI training is very expensive to run, either in the cloud or on-premises, he adds. Inferencing is not as power hungry but still uses a lot of juice at scale.

As models get more efficient and compute gets better, you’re better off running inferencing locally, because it’s cheaper to run it on local hardware than it is on the cloud. So data sovereignty and security are driving the desire to process AI locally rather than in the cloud.

Interesting times in the chip industry

Here’s the plot twist. Both Intel and AMD have spoken about adding AI processing to CPUs, with Intel in the lead for now. However, they have new competition. First is Qualcomm, which acquired an Arm-based desktop CPU startup called Nuvia in 2021 for $1.4 billion. It is now preparing to release desktop chips to compete with Intel and AMD under the brand name Snapdragon X Elite.

Bajarin noted that the Qualcomm AI processor was capable of executing 45 AI TOPS (trillions of operations per second) alone, and with the CPU it can do 75 TOPS. Intel’s latest processor is only capable of managing 10 TOPS, and AMD hasn’t released any figures yet.

“When it comes to the part that’s most important for AI, which is the NPU (neural processing unit), [Qualcomm’s chip] is far more capable than anything Intel or AMD has coming out to market in the next 12 months,” he said.

And if that was not enough, Reuters reports that Nvidia is looking at developing its own desktop processor based on the Arm architecture. It already has a processor for servers called Grace Hopper. It wouldn’t take a lot of work on Nvidia’s part to repurpose the processor for a desktop.

And Microsoft has been very encouraging of supporting Arm on both the server and the desktop. So as the AI battle moves to the client and other endpoints, the two-horse race of Intel and AMD suddenly becomes a four-horse race with Qualcomm and Nvidia added to the mix.

So it appears that IT is about to get more choices when it comes to desktop systems. That can be good or bad, depending on your perspective.