NPUs are now available from almost every major hardware manufacturer. But what kind of chip is it? What can it do and what makes it so special? We explain it to you
Apple has it, Samsung has it, AMD has it and Intel now has it too: its own NPU.
This is a chip that is primarily used in AI applications. The widespread use across manufacturers shows: The NPU is clearly not a passing fad, but is here to stay.
But what does an NPU do? Why does every manufacturer want one? What does it do for you in everyday life and might it even replace the GPU at some point?
- What is an NPU?
- NPUs: The end of graphics chips?
- Why every laptop will soon have an NPU
- What an NPU can do for you in practice
What is an NPU?
The NPU (Neural Processing Unit) is a processor, just like a CPU or GPU. However, its area of responsibility is not graphics or general tasks. Instead, the NPU is designed to perform machine learning tasks quickly and efficiently
This not only provides a performance boost, but should also reduce the load on the CPU and GPU. To this end, the NPU has been optimized for the mathematical calculations that take place during machine learning.
Similar to graphics chips, it uses parallelization to tackle many small tasks at once. At the same time, it is even more specialized.
In these calculations, NPUs are therefore not only more efficient than graphics chips, but in some cases even up to 10,000 times faster.
NPUs: The end for graphics chips?
That sounds pretty powerful at first – and almost like a swan song for the GPU in terms of scale. Will we no longer need GPUs in the future if NPUs are more economical and much faster?
At the moment, the answer is quite clear: yes
Because NPUs currently use their parallelization primarily to tackle smaller, repetitive tasks as efficiently as possible: Blurring the background in a video call or letting an AI chatbot reply.
GPUs, on the other hand, take on larger tasks such as (increasingly AI-based) graphics calculations, video editing or training AI models – making them the hardware of choice even in the AI sector if power consumption is not an issue.
At the same time, there is already the concept of a GPNPU (General Purpose NPU), i.e. a GPU-NPU hybrid. This is a single chip that is intended to strike a healthy balance between the specialized NPU and the general purpose weapon
GPU.
It is quite possible that this will work so well in the future that GPU and NPU will merge into one chip again, especially for hardware that is particularly space-limited and dependent on battery life, such as smartphones and ultrabooks.
Why every laptop will soon have an NPU
Anyone who thinks: I don’t need that
despite all the predicted advantages of an NPU will soon have no choice at all. Because there’s a good chance that in a few years’ time, at least on the mobile market – for smartphones and laptops – you’ll have to go for a model with an NPU
Almost every major chip manufacturer has already developed its own form of NPU.
There is Qualcomm, for example, which has been using NPUs since the launch of the Snapdragon 855 in 2018. At that time, the main aim was to catch up with Huawei, which was one of the first to use an NPU for its in-house Kirin 970 processor, but now it tends to compete with Samsung’s Exynos processors, which of course also use an NPU.
Apple has also been using NPUs in its products for some time – especially under the name Neural Engine. It is installed in the processors of the M series (e.g. Mac) as well as the A series (e.g. iPhone). One feature in which the NPU comes into play is Face ID. Using AI, the image scanner can also recognize you if you are wearing glasses or have grown a beard.
Intel only really jumped on the NPU bandwagon in January. In addition to the CPU and integrated GPU, each of the new Ultra chips also contains an NPU – one of the reasons why Intel decided to switch to a chiplet design with Meteor Lake.
AMD is already a little further ahead. Only around three weeks ago, they presented the second generation of their in-house NPU under the name XDNA 2, which is to be used in the Strix Point processors in 2024
What an NPU can do for you in practice
The actual benefits of many NPUs are still limited. The chips already provide a certain advantage in terms of performance and runtime, especially where hardware and software come from a single source (cough, Apple)
But above all in the foreseeable future, the advantage of NPUs should really be played out. Because at the moment, it doesn’t look like AI applications are just a passing trend.
Predictions are difficult in such a rapidly developing field, but the more apps and applications that jump on the bandwagon and integrate generative AI into their toolbox, the more powerful NPUs are likely to become.
At the same time, operating systems are likely to become even better at distributing AI tasks between CPU, GPU and NPU, which will allow the hardware to solve tasks even more efficiently.
What do you think? Will NPUs play an increasingly important role in computers in the future? Will they join the CPU and GPU as the third in the group – or will they merge with the latter again in a few years? Which applications do you use yourself where an NPU could help in everyday life – or perhaps already does? Let us know in the comments!
