Qualcomm, a leading provider of wireless communications and processors, has made significant strides in the field of artificial intelligence (AI) at Mobile World Congress Barcelona. The company’s primary goal is to run AI at scale on edge devices such as phones, computers, and other devices.
Qualcomm has successfully run the world’s first large language (LLM) and large multimodal model on an Android phone, allowing for a chat-like, multi-stage conversation between the user and their device. Additionally, the company has announced plans to accelerate AI on a wide range of edge devices, including next-generation computers, vehicles, phones, industrial IoT devices, Wi-Fi access points, and cellular infrastructure.
Easy-to-use Qualcomm AI Hub
To facilitate the development of AI applications, Qualcomm has created the AI Hub, which provides developers with a vast library of AI models optimized for high-performance on Snapdragon and other Qualcomm platforms. The platform offers many features, such as image and text rendering, and up to four times faster inference on devices powered by Snapdragon or other Qualcomm platforms.
The AI Hub offers a user-friendly experience, allowing developers to choose from more than 75 AI models and select between TensorFlow, PyTorch, or Onyx working environments. Models are also available on Hugging Face and GitHub, making it easy for developers to integrate AI into their applications.
What is LoRA?
Qualcomm also reached another technical achievement by running its first low-order implementation (LoRA) on an Android phone. Developed by Microsoft, LoRA aims to reduce model training complexity in terms of latency, cost and hardware requirements. LoRA offers a novel approach to image generation that is different from generative AI tools such as DALL-E. By reducing model complexity, LoRA reduces memory requirements, improves efficiency, and allows lightweight versions to be customized without requiring downloading or fine-tuning the entire model.
LoRA takes a different approach to traditional AI rendering models. These models learn a pattern of pixels to create the image. LoRA uses a list of vectors to create the image. Vectors represent the basic features of the image (shape, color, texture, etc.).
LoRA’s vector-based approach significantly reduces the complexity and size of the model. This allows the model to be trained faster, requiring less hardware and consuming less energy.
Snapdragon X Elite: The designed for AI applications
Designed to run AI, the Snapdragon X Elite features a 4nm SoC architecture, a 12-core Qualcomm Oryon CPU with dual-core support, and an integrated Qualcomm Adreno GPU. It offers days of use on a single charge, eliminating the need for constant recharging. One of its most important features is that it can also run generative AI LLM models with 13 billion parameters.
To demonstrate the SoC’s capabilities, Qualcomm ran a performance benchmark between the Snapdragon X Elite and a competing x86 processor with the most efficient GPU, CPU, and NPU configuration. The benchmark used GIMP, a powerful open-source raster graphics editor equipped with the Stable Diffusion plugin for GAI image generation. The results showed that Snapdragon X Elite with a “45 TOPS” NPU can render images three times faster than its x86 competitor. To better understand how powerful 45 TOPS processing power is, we can look at the example below:
- A human brain can perform about 100 trillion operations per second.
- A desktop computer can perform about 10 trillion operations per second.
- A smartphone can perform about 1 trillion operations per second.
As can be seen, 45 TOPS processing power is 45 times more than a smartphone and 4.5 times more than a desktop computer. This shows that the Snapdragon X Elite platform is a very powerful platform for AI applications.
NPU, whose name we started to hear
NPU stands for Neural Processing Unit, a specialized processor for artificial intelligence and machine learning applications. NPUs provide superior performance, especially in calculating mathematical operations used by artificial neural networks (ANNs). These mathematical operations include matrix multiplication, convolution, and activation functions.
NPUs are designed to work more efficiently with low-precision data, making them ideal for AI workloads. They are also much more power-efficient than CPUs and GPUs for AI computing.
Qualcomm’s innovations in AI
The company’s bullishness on the subject aims to ensure that artificial intelligence is used efficiently on many different platforms. The company’s Snapdragon 8 Gen 3 phone processor and Snapdragon X Elite PC platforms can perform important functions in areas such as generative AI, computer vision, and natural language processing.
Qualcomm is also offering advanced AI capabilities to developers through the Qualcomm AI Hub. The hub provides developers with resources, including more than 75 AI models optimized for high performance on Snapdragon and other Qualcomm platforms.
The use of artificial intelligence in smartphones is limited, but if the company keeps its word and develops these technologies as it wants, we will be able to use artificial intelligence in our phones as we use it in our computers. AI also brings a whole new dimension to the gaming experience, allowing virtual game assistants to generate new game content and dynamically adjust game rules and gameplay based on the player’s knowledge or desire for a more challenging experience.
Qualcomm’s scalable commercialization of on-device AI is a major contribution to the democratization of AI. These visionary steps make Qualcomm a key player in AI, and the company can further accelerate large-scale AI adoption.
Developments in artificial intelligence continue. At this rate, artificial intelligence, which first entered our computers, will reach our phones efficiently. I think it also shows how much this sector will grow going forward, with processors built just for AI to understand. Although its ethics and morality are currently being debated, in the coming days, the use of this technology will somehow be accepted by all users. With Qualcomm’s advancements, we can expect to see AI-powered devices that can perform complex tasks with ease, making our lives more convenient and efficient.
Featured image credit: Qualcomm
