CPUs and GPUs are becoming outdated. Nowadays, the forefront of technology is all about NPUs, and hardware manufacturers are emphasizing NPU capabilities.
The NPU is a computing component crafted to enhance AI activities in an energy-efficient manner, which opens possibilities for innovative Windows desktop applications featuring robust AI functionalities. At least, that’s the goal.
While NPUs will eventually be integrated into all PCs, only a limited number of laptops currently include them. Here’s a comprehensive guide to NPUs and why they’re such a trending subject in the technology sector today.
What is an NPU?
NPU refers to neural processing unit. It’s a distinct type of processor specifically designed for AI and machine learning operations.
The term stems from the fact that AI models utilize neural networks. A neural network can be simply understood as a complex web of interconnected nodes that share information among themselves. (This concept was inspired by the workings of the human brain.)
An NPU isn’t something you purchase separately and connect, like a GPU, for instance. Rather, an NPU is integrated with a contemporary processor platform—such as Intel’s Core Ultra, AMD’s Ryzen AI, and Qualcomm’s Snapdragon X Elite and X Plus. These platforms combine a CPU, an integrated GPU, and an NPU.
NPU vs. CPU vs. GPU: Understanding the differences
For several years, computers have performed tasks either on the central processing unit (CPU) or the graphics processing unit (GPU). This practice continues on AI PCs (i.e., machines featuring NPUs).
The CPU is responsible for executing most tasks on the computer. However, the GPU, despite its designation, is not solely for visual and gaming functions. The GPU is fundamentally optimized for a different kind of computation, explaining why GPUs have been essential for non-gaming applications such as cryptocurrency mining and executing local AI models efficiently. In fact, GPUs are quite effective for AI duties—but they consume a lot of power.
This is where NPUs become relevant. An NPU operates faster than a CPU when managing AI tasks, but it doesn’t compare to a GPU in terms of speed. The benefit is that an NPU consumes significantly less energy than a GPU while executing the same AI operations. Moreover, while the NPU takes care of AI-related tasks, both the CPU and GPU can focus on their specific jobs, enhancing the overall system efficiency.
Why choose an NPU over a CPU or GPU?
If you’re utilizing AI image generation software like Stable Diffusion (or any similar AI model) on your hardware and require peak performance, a GPU will serve you best. This is precisely why Nvidia markets its GPUs as “premium AI” equipment rather than NPUs.
Nonetheless, there are instances where you might need to run AI functionalities that are too demanding for a standard CPU but don’t necessarily require the utmost power of a GPU. Alternatively, you might be using a laptop and wish to leverage AI capabilities without exhausting battery life through GPU consumption.
With an NPU, a laptop can handle on-device AI tasks without generating excessive heat and while conserving battery energy—plus, it can execute those AI operations without monopolizing CPU and GPU resources, allowing your PC to manage other tasks simultaneously.
Even if you aren’t particularly interested in AI, you can still benefit from NPUs for various other applications. At CES 2024, HP demonstrated gaming streaming software that utilizes the NPU for video-related tasks, allowing the GPU to focus on the game itself. By harnessing the NPU’s additional computational power, the streaming application avoids using any GPU resources…and it operates much more swiftly than if the CPU were responsible for the same function.
But what practical applications can Windows PC software utilize with an NPU?
In an AI-enabled PC, both the operating system and installed applications can leverage the NPU.
For instance, if you have a laptop equipped with Intel Meteor Lake hardware, the integrated NPU will enable you to use Windows Studio Effects, which are AI-enhanced webcam features that allow functionalities like background blurring and enforced eye contact in any application utilizing your webcam.
Microsoft’s Copilot+ PCs — the initial models of which were equipped with Qualcomm Snapdragon X processors — feature AI capabilities that leverage the NPUs integrated into that system. For instance, the Windows Recall function will depend on an NPU.
Earlier this year, AMD and Intel processors began accessing Copilot+ functionalities that were initially reserved for Qualcomm PCs — but only with the new AMD Ryzen AI 300 series and Intel Core Ultra Series 2 (Lunar Lake) processors.
These are merely the built-in features of Windows; app developers will also have the opportunity to utilize the NPU in various ways. It wouldn’t be surprising to see plugins for Audacity and GIMP that deliver AI-enhanced audio and photo editing powered by a computer’s NPU.
The potential applications are limitless, but it’s still the early stages for the hardware, so it may take some time before its full capabilities are realized.
Why would I need an NPU if my computer can already handle AI applications?
As of now, many AI-featured applications — including Microsoft’s Copilot chatbot — do not yet take advantage of an NPU. Instead, they rely on remote cloud servers to run their AI models. This is why tools like Microsoft Copilot, ChatGPT, Google Gemini, Adobe Firefly, and other AI options can operate on virtually any device, be it an older Windows PC, a Chromebook, a Mac, an Android smartphone, or something else.
However, running these AI models in the cloud can be costly. For example, Microsoft incurs significant expenses processing tasks for Copilot AI in data centers. Companies are eager to transfer those AI workloads to your personal computer, thus lowering their cloud computing costs.
Of course, it’s not solely about cutting expenses. You also gain the advantage of executing computationally intensive AI tasks on your own device. For example, those AI features will remain functional even when you’re offline, and you can keep your information private rather than constantly uploading it to cloud servers. (This is particularly important for businesses that wish to maintain control over their data for privacy and security reasons.)
Is it advisable to purchase a PC with an NPU now or to wait?
Honestly, embracing the latest technology can be a gamble.
If you sought out a Meteor Lake laptop with the intention of future-proofing your device for AI capabilities, you may have been disappointed to learn later that the NPUs in Meteor Lake were deemed too slow for Copilot+ PC functionalities. (Intel, however, countered that users still benefit from the features those laptops originally came with.)
The positive aspect? The current Copilot+ PC features aren’t particularly compelling. Additionally, most of the leading AI tools — including ChatGPT, Adobe Firefly, and others — don’t utilize NPUs at all.
Still, if I were in the market for a new laptop, I would prefer to have an NPU if possible. You don’t actually need to search extensively for one; it typically comes as part of modern hardware platforms. These up-to-date processor platforms also offer significant advantages, such as improved battery life.
Conversely, these high-speed NPUs are only found on the latest laptops, and you can often discover excellent discounts on older laptops that perform nearly as well as the most recent models. If you find a last-generation laptop that is heavily reduced in price, it may not be wise to spend much more just for the NPU, especially if you don’t intend to run any AI applications.
As of now, the majority of significant AI tools still operate in the cloud — or at least provide that as an option — so you will be able to access them on any Windows laptop, Chromebook, or even Android tablets and iPads.
In the case of desktops, the situation with NPUs is different. Intel’s desktop processors currently lack NPUs. You could look for an AMD desktop chip equipped with a Ryzen AI NPU, but AMD’s Ryzen 7000 and 8000 series models do not support Copilot+ PC features anyway. So, if you are building a desktop PC, you can disregard the NPU for the present. It simply isn’t essential at this moment.
Leave a Reply