> ## Documentation Index
> Fetch the complete documentation index at: https://qualcomm-3.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Real-Time NPU Monitoring on Qualcomm Ubuntu with libqcperf
> Get live, programmatic NPU utilization data on Qualcomm Ubuntu using libqcperf, including Q6, HVX, HMX, and clock metrics over FastRPC.
## Introduction
When multiple AI models run concurrently on a Qualcomm device, utilization visibility becomes critical. On CPU workloads, `/proc/stat` offers a direct view of load. On Hexagon DSP-backed NPU workloads, there is no equivalent standard Linux interface that exposes Q6, HVX (Hexagon Vector eXtensions), and HMX (Hexagon Matrix eXtensions) utilization in real time.
Without direct telemetry, inference latency is only an indirect signal. It can show that performance changed, but not why. Teams cannot reliably tell whether the accelerators are engaged, whether the DSP is saturated, or whether there is headroom for additional models.
This guide walks through building [`libqcperf`](https://github.com/qualcomm/libqcperf) from source and writing a minimal C program that streams live NPU metrics into your own application.
## Why Existing Paths Fall Short
The official Qualcomm Profiler is not suitable for many open workflows because it requires NDA access.
SysmonApp in the Hexagon SDK can query CDSP utilization over FastRPC, but it is an offline flow: capture to a binary `.bin`, transfer to a host, then post-process into HTML or CSV. This works for one-time profiling, not continuous on-device telemetry in application code.
Hexagon QuRT PMU counters are another option, but they require DSP-side instrumentation and deployment with Hexagon toolchain artifacts. That is a high barrier when the goal is application-layer monitoring from standard Linux processes.
## What You Will Do
1. Confirm FastRPC is present on the device.
2. Clone and build `libqcperf` with the NPU backend.
3. Write and build a minimal C program using the `libqcperf` API.
4. Run it and observe live Q6, HVX, and HMX metrics streaming to stdout.
## Prerequisites
`libqcperf` communicates with the CDSP over FastRPC. Before anything below works, the device needs its Qualcomm peripherals enabled and the FastRPC userland present. It's also necessary to install the headers for the DSP services.
Set this up first by following the IQ8 device pages, then come back here:
* [First-time setup for the Dragonwing IQ8](/Ubuntu/devices/iq8275-evk/setup)
* [Install the required software packages](/Ubuntu/devices/iq8275-evk/Install_required_software_packages)
After the reboot, confirm FastRPC is present:
```bash theme={null}
ls /dev/fastrpc-cdsp # must exist
ldconfig -p | grep cdsprpc # libcdsprpc.so[.1] present
```
If `/dev/fastrpc-cdsp` does not exist, the kernel lacks FastRPC support. That is a BSP or image problem, not something you can fix in userland.
You'll need to add your user to the fastrpc group by running the command below, then log out and log back in.
```bash theme={null}
sudo usermod -aG fastrpc $USER
```
You also need standard build tools along with the DSP headers:
```bash theme={null}
sudo apt-get install -y git cmake build-essential qcom-dspservices-headers-dev
```
## Build libqcperf
All work lives in `~/libqcperf-build`. Every code block starts with its own `cd`, so you can paste any block into a fresh terminal without tracking which directory you are in.
### Clone the repository
```bash theme={null}
mkdir -p ~/libqcperf-build
cd ~/libqcperf-build
git clone https://github.com/qualcomm/libqcperf.git
```
### Configure and build
The NPU backend is off by default. Enable it explicitly. This build targets the host device directly (native aarch64), so no cross-compile toolchain is needed:
```bash theme={null}
cd ~/libqcperf-build
cmake -S libqcperf/qcperf -B build \
-DCMAKE_BUILD_TYPE=Release \
-DProjectVersion="0.1.0.0" \
-DBACKENDS="NPU"
cmake --build build --parallel
```
The build produces the static library archives the C example links against:
```text theme={null}
build/libqcperfCore.a
build/libQcPerfDspNpuBackend.a
build/libQcPerfQCv.a
build/libQcPerfQMutex.a
build/libQcPerfQSleep.a
build/libQcPerfQThread.a
build/libQcPerfQTime.a
build/libQcPerfqlist.a
build/libQcPerfQcomDsp.so
```
## Write a C Integration
For application-layer integration — embedding NPU telemetry directly in your inference loop, correlating metrics with latency measurements, or triggering adaptive behavior — use the `libqcperf` API directly.
The full lifecycle is nine steps. Here is a minimal but complete program that streams all four NPU metrics to stdout.
### The program
Create the source file:
```bash theme={null}
mkdir -p ~/libqcperf-build/example
```
```c theme={null}
/* npu_monitor.c — minimal libqcperf NPU integration example */
#define _POSIX_C_SOURCE 200809L
#include
#include
#include
#include
#include "qcperf.h"
#include "qcperf_common.h"
/* ── Shared state ─────────────────────────────────────────────────────────── */
static volatile sig_atomic_t g_running = 1;
/* Deep copy of backend info so the callback can resolve metric names. */
static struct QcPerfBackendInfo *g_info = NULL;
/* ── Signal handler ───────────────────────────────────────────────────────── */
static void on_signal(int sig) {
(void)sig;
g_running = 0;
}
/* ── Data callback ────────────────────────────────────────────────────────── */
/*
* Called by the libqcperf background thread once per streaming interval
* (1000 ms in this example). data->metric_response holds all samples
* collected during that window; we print only the most recent value for
* each metric_id.
*/
static enum QcPerfReturnCode on_data(struct QcPerfData *data) {
uint32_t idx = 0;
uint64_t written = 0;
if (NULL == data) {
return QC_PERF_RETURN_CODE_FAILED;
}
printf("--- NPU snapshot ---\n");
for (idx = data->metric_response_len; idx > 0; idx--) {
uint16_t mid = data->metric_response[idx - 1].metric_id;
uint64_t bit = (mid < 64) ? ((uint64_t)1 << mid) : 0;
if (0 == bit || 0 != (written & bit)) {
continue; /* skip: out of range or already printed */
}
written |= bit;
/* Resolve the metric name from the deep-copied backend info. */
const char *name = NULL;
if (NULL != g_info && NULL != g_info->capabilities_list) {
uint8_t cap = data->capabilityId;
if (cap < g_info->capabilities_list_length) {
struct QcPerfCapabilityInfo *ci = &g_info->capabilities_list[cap];
for (uint8_t m = 0; m < ci->metric_ids_list_len; m++) {
if (ci->metric_ids_list[m].metric_id == mid) {
name = ci->metric_ids_list[m].metric_name;
break;
}
}
}
}
if (NULL == name) {
printf(" metric_%u: ", (unsigned)mid);
} else {
printf(" %-20s ", name);
}
struct QcPerfGenericType *v = &data->metric_response[idx - 1].metric_value;
switch (v->data_type) {
case QC_PERF_DATA_TYPE_DOUBLE: printf("%.2f\n", v->double_value); break;
case QC_PERF_DATA_TYPE_UINT64: printf("%llu\n", (unsigned long long)v->uint64_value); break;
case QC_PERF_DATA_TYPE_INT64: printf("%lld\n", (long long)v->int64_value); break;
case QC_PERF_DATA_TYPE_BOOL: printf("%s\n", v->bool_value ? "true" : "false"); break;
default: printf("(unknown type)\n"); break;
}
}
return QC_PERF_RETURN_CODE_SUCCESS;
}
/* ── Message callback ─────────────────────────────────────────────────────── */
static enum QcPerfReturnCode on_message(struct QcPerfMessage *msg) {
if (NULL == msg || NULL == msg->message) {
return QC_PERF_RETURN_CODE_FAILED;
}
if (msg->message_level != QC_PERF_MESSAGE_LEVEL_DEBUG) {
fprintf(stderr, "[backend] %s\n", msg->message);
}
return QC_PERF_RETURN_CODE_SUCCESS;
}
/* ── main ─────────────────────────────────────────────────────────────────── */
int main(void) {
enum QcPerfReturnCode rc = QC_PERF_RETURN_CODE_FAILED;
struct QcPerfBackendInfo *info = NULL;
struct QcPerfRequest *req = NULL;
int exit_code = 0;
/* Install signal handlers for clean shutdown. */
struct sigaction sa = {0};
sa.sa_handler = on_signal;
sigemptyset(&sa.sa_mask);
sigaction(SIGTERM, &sa, NULL);
sigaction(SIGINT, &sa, NULL);
/* Step 1: Initialise the library. */
rc = qcperf_init();
if (QC_PERF_RETURN_CODE_SUCCESS != rc) {
fprintf(stderr, "qcperf_init failed (%d)\n", (int)rc);
return 1;
}
/* Step 2: Connect to the NPU backend, registering the message callback. */
rc = qcperf_connect_backend(QC_PERF_BACKEND_DSP_NPU, &on_message);
if (QC_PERF_RETURN_CODE_SUCCESS != rc) {
fprintf(stderr, "qcperf_connect_backend failed (%d)\n", (int)rc);
exit_code = 1;
goto deinit;
}
/* Step 3: Query capabilities and deep-copy for use in the callback. */
info = (struct QcPerfBackendInfo *)calloc(1, sizeof(struct QcPerfBackendInfo));
if (NULL == info) {
fprintf(stderr, "calloc failed\n");
exit_code = 1;
goto disconnect;
}
rc = qcperf_get_capabilities_info(QC_PERF_BACKEND_DSP_NPU, info);
if (QC_PERF_RETURN_CODE_SUCCESS != rc) {
fprintf(stderr, "qcperf_get_capabilities_info failed (%d)\n", (int)rc);
exit_code = 1;
goto disconnect;
}
/*
* Deep-copy into g_info so the data callback (called from a background
* thread) can safely look up metric names without touching the stack-local
* `info` pointer.
*/
g_info = (struct QcPerfBackendInfo *)calloc(1, sizeof(struct QcPerfBackendInfo));
if (NULL != g_info) {
g_info->backend_id = info->backend_id;
g_info->capabilities_list_length = info->capabilities_list_length;
g_info->capabilities_list = (struct QcPerfCapabilityInfo *)calloc(
info->capabilities_list_length, sizeof(struct QcPerfCapabilityInfo));
if (NULL != g_info->capabilities_list) {
for (uint8_t c = 0; c < info->capabilities_list_length; c++) {
g_info->capabilities_list[c] = info->capabilities_list[c];
uint8_t mlen = info->capabilities_list[c].metric_ids_list_len;
g_info->capabilities_list[c].metric_ids_list =
(struct QcPerfMetricInfo *)calloc(mlen, sizeof(struct QcPerfMetricInfo));
if (NULL != g_info->capabilities_list[c].metric_ids_list) {
for (uint8_t m = 0; m < mlen; m++) {
g_info->capabilities_list[c].metric_ids_list[m] =
info->capabilities_list[c].metric_ids_list[m];
}
}
}
}
}
/* Step 4: Register the data callback. */
rc = qcperf_set_data_callback(QC_PERF_BACKEND_DSP_NPU, &on_data);
if (QC_PERF_RETURN_CODE_SUCCESS != rc) {
fprintf(stderr, "qcperf_set_data_callback failed (%d)\n", (int)rc);
exit_code = 1;
goto disconnect;
}
/* Step 5: Build the request and start monitoring. */
req = (struct QcPerfRequest *)calloc(1, sizeof(struct QcPerfRequest));
if (NULL == req) {
fprintf(stderr, "calloc failed\n");
exit_code = 1;
goto disconnect;
}
req->capability_id = info->capabilities_list[0].capability_id;
req->sampling_rate = 100; /* poll CDSP every 100 ms */
req->streaming_rate = 1000; /* deliver callback every 1000 ms */
rc = qcperf_start(QC_PERF_BACKEND_DSP_NPU, req);
if (QC_PERF_RETURN_CODE_SUCCESS != rc) {
fprintf(stderr, "qcperf_start failed (%d)\n", (int)rc);
exit_code = 1;
goto disconnect;
}
fprintf(stderr, "Streaming NPU metrics — press Ctrl-C to stop\n");
/* Step 6: Run until signalled. */
while (0 != g_running) {
sleep(1);
}
/* Step 7: Stop monitoring. */
qcperf_stop(QC_PERF_BACKEND_DSP_NPU, req);
free(req);
req = NULL;
disconnect:
/* Step 8: Disconnect the backend. */
free(req);
req = NULL;
qcperf_disconnect_backend(QC_PERF_BACKEND_DSP_NPU);
deinit:
/* Step 9: Deinitialise the library. */
qcperf_deinit();
/* Free caller-owned memory. */
if (NULL != info) {
free(info);
}
if (NULL != g_info) {
if (NULL != g_info->capabilities_list) {
for (uint8_t c = 0; c < g_info->capabilities_list_length; c++) {
free(g_info->capabilities_list[c].metric_ids_list);
}
free(g_info->capabilities_list);
}
free(g_info);
}
return exit_code;
}
```
Save this as `~/libqcperf-build/example/npu_monitor.c`.
### Build the example
The example links against the same static library archive produced by the earlier build:
```bash theme={null}
cd ~/libqcperf-build
gcc -std=c11 \
-I libqcperf/qcperf/core/inc \
-I libqcperf/qcperf/backends/inc \
-I build/include \
example/npu_monitor.c \
build/libqcperfCore.a \
build/libQcPerfDspNpuBackend.a \
build/libQcPerfQCv.a \
build/libQcPerfQMutex.a \
build/libQcPerfQSleep.a \
build/libQcPerfQThread.a \
build/libQcPerfQTime.a \
build/libQcPerfqlist.a \
-L build -lQcPerfQcomDsp \
-lcdsprpc \
-lpthread \
-o example/npu_monitor
```
### Run it
```bash theme={null}
cd ~/libqcperf-build
export LD_LIBRARY_PATH=build
./example/npu_monitor
```
Expected output (one block per second while a model is running):
```text theme={null}
Streaming NPU metrics — press Ctrl-C to stop
--- NPU snapshot ---
Q6 Utilization 42.50
Q6 Clock 614400.00
HVX Utilization 12.30
HMX Utilization 8.70
--- NPU snapshot ---
Q6 Utilization 67.10
Q6 Clock 729600.00
HVX Utilization 31.80
HMX Utilization 55.20
```
Press `Ctrl-C` to stop. The library shuts down cleanly on `SIGINT`.
## Under the Hood
### Sampling rate vs. streaming rate
These two parameters are independent and serve different purposes.
The **sampling rate** (100 ms in the examples above) controls how often the background thread calls into the CDSP over FastRPC to read raw hardware counters. Lower values give finer time resolution but increase FastRPC overhead. The NPU backend supports 1, 5, 10, 50, 100, and 200 ms.
The **streaming rate** (1000 ms) controls how often the background thread fires your data callback. Each callback delivery includes all samples collected since the last delivery — ten samples at 100 ms sampling / 1000 ms streaming. The callback receives them as a flat `metric_response` array; the example above uses a bitmask to extract only the most recent sample per metric.
The supported streaming rates are 100 ms through 1000 ms in 100 ms steps.
### The FastRPC path
`libqcperf` does not open a kernel driver or read a sysfs file. It calls `sysmonquery_get_profdata` over FastRPC — the same inter-processor RPC mechanism that llama.cpp and LiteRT-LM use to dispatch compute to the CDSP. The call crosses the kernel FastRPC bridge (`/dev/fastrpc-cdsp`) and returns a struct with the four hardware counter values directly from the DSP firmware.
The runtime dependency is `libcdsprpc.so`. This shared library is already present on Qualcomm Ubuntu images as part of the FastRPC userland. If it is absent, the dynamic linker will fail to start the process before `main` is reached.
### The background thread
`qcperf_start` spawns a single background thread named `qcperf_dsp_npu_thread`. This thread owns the FastRPC session for the duration of the monitoring session. Your data callback is called from this thread, not from the thread that called `qcperf_start`. Keep the callback fast; any blocking work should be handed off to a queue.
## Interpreting the Metrics
Live telemetry turns the NPU from a black box into an observable subsystem.
| Metric | Unit | What it tells you |
| --------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Q6 Utilization | % of max Q6 clock | Scalar DSP pressure. High values mean the Q6 core is busy — either running non-vectorized code or managing HVX/HMX dispatch overhead. |
| Q6 Clock | KHz | The actual CDSP clock frequency chosen by DCVS. Correlate this with utilization: 80% utilization at 614 MHz is a very different situation from 80% at 1 GHz. |
| HVX Utilization | % of max Q6 clock | Hexagon Vector eXtensions engagement. HVX handles 128-byte SIMD operations — convolutions, activations, element-wise ops. |
| HMX Utilization | % of max Q6 clock | Hexagon Matrix eXtensions engagement. HMX is the dedicated matrix-multiply accelerator used for quantized linear layers. |
A few patterns worth knowing:
**Low HMX during quantized inference** is the most common surprise. If you expect a quantized model to be running on the NPU but HMX utilization is near zero, the workload is not taking the intended accelerator path. Common causes: the model was not compiled with HMX ops enabled, the QNN context binary version does not match the on-device runtime, or the model is falling back to CPU.
**HVX high, HMX low** suggests the model is running vectorized but not matrix-accelerated — typical of FP16 or non-quantized paths, or of models that use HVX-friendly ops (pooling, normalization) but not INT8/INT4 matmuls.
**Q6 clock stepping up under load** is DCVS working correctly. If the clock does not step up when utilization is high, check whether a power profile is capping the CDSP frequency.
**All metrics near zero** while inference is running usually means the workload is executing on the CPU, not the DSP. Confirm with `htop` and check your model's backend configuration.
## Troubleshooting
| Symptom | Likely cause | Fix |
| --------------------------------------------------------------------------------------- | ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `error while loading shared libraries: libcdsprpc.so` | `libcdsprpc.so` not installed | Follow the [IQ8 software setup](/Ubuntu/devices/iq8275-evk/Install_required_software_packages) guide; the library ships with the FastRPC userland package |
| `qcperf_connect_backend` returns `QC_PERF_RETURN_CODE_FAILED` with `[ERROR]` in journal | CDSP is inaccessible | Confirm `/dev/fastrpc-cdsp` exists; if not, the kernel or firmware does not have FastRPC enabled |
| All metric values are `0.00` | No DSP workload is active | The counters are hardware-accurate; zero means the CDSP is idle. Start an NPU inference workload |
| Build fails: `QCPERF_ENABLED_QCOM_LINUX_NPU` not set | CMake did not detect aarch64 Linux | Confirm you are building on or for `linux-aarch64`; the NPU backend is gated to that platform |
## Next Steps
With live NPU telemetry in place, the natural next step is to watch a real model run:
* [Run Gemma-4 E2B on the IQ8 NPU with LiteRT-LM](/tutorials/gemma-litert-lm-on-iq8) — run `npu_monitor` alongside LiteRT-LM and watch HMX utilization climb during prefill
* [Run LLMs with llama.cpp on Dragonwing](/Ubuntu/ai-workflows/llama-cpp) — correlate Q6 clock steps with llama.cpp's token throughput
* [libqcperf API reference](/Ubuntu/tools/libqcperf) — full documentation for all backends, return codes, and struct fields
* [libqcperf on GitHub](https://github.com/qualcomm/libqcperf) — source, issue tracker, and DEVELOPMENT-GUIDE for adding new backends