Real-Time NPU Monitoring on Qualcomm Ubuntu with libqcperf

AI / ML

Sam Freund·Jun 30, 2026·← All posts

Introduction

When multiple AI models run concurrently on a Qualcomm device, utilization visibility becomes critical. On CPU workloads, /proc/stat offers a direct view of load. On Hexagon DSP-backed NPU workloads, there is no equivalent standard Linux interface that exposes Q6, HVX (Hexagon Vector eXtensions), and HMX (Hexagon Matrix eXtensions) utilization in real time. Without direct telemetry, inference latency is only an indirect signal. It can show that performance changed, but not why. Teams cannot reliably tell whether the accelerators are engaged, whether the DSP is saturated, or whether there is headroom for additional models. This guide walks through building libqcperf from source and writing a minimal C program that streams live NPU metrics into your own application.

Why Existing Paths Fall Short

The official Qualcomm Profiler is not suitable for many open workflows because it requires NDA access. SysmonApp in the Hexagon SDK can query CDSP utilization over FastRPC, but it is an offline flow: capture to a binary .bin, transfer to a host, then post-process into HTML or CSV. This works for one-time profiling, not continuous on-device telemetry in application code. Hexagon QuRT PMU counters are another option, but they require DSP-side instrumentation and deployment with Hexagon toolchain artifacts. That is a high barrier when the goal is application-layer monitoring from standard Linux processes.

What You Will Do

Confirm FastRPC is present on the device.
Clone and build libqcperf with the NPU backend.
Write and build a minimal C program using the libqcperf API.
Run it and observe live Q6, HVX, and HMX metrics streaming to stdout.

Prerequisites

libqcperf communicates with the CDSP over FastRPC. Before anything below works, the device needs its Qualcomm peripherals enabled and the FastRPC userland present. It’s also necessary to install the headers for the DSP services. Set this up first by following the IQ8 device pages, then come back here:

After the reboot, confirm FastRPC is present:

ls /dev/fastrpc-cdsp                 # must exist
ldconfig -p | grep cdsprpc           # libcdsprpc.so[.1] present

If /dev/fastrpc-cdsp does not exist, the kernel lacks FastRPC support. That is a BSP or image problem, not something you can fix in userland. You’ll need to add your user to the fastrpc group by running the command below, then log out and log back in.

sudo usermod -aG fastrpc $USER

You also need standard build tools along with the DSP headers:

sudo apt-get install -y git cmake build-essential qcom-dspservices-headers-dev

Build libqcperf

All work lives in ~/libqcperf-build. Every code block starts with its own cd, so you can paste any block into a fresh terminal without tracking which directory you are in.

Clone the repository

mkdir -p ~/libqcperf-build
cd ~/libqcperf-build
git clone https://github.com/qualcomm/libqcperf.git

Configure and build

The NPU backend is off by default. Enable it explicitly. This build targets the host device directly (native aarch64), so no cross-compile toolchain is needed:

cd ~/libqcperf-build
cmake -S libqcperf/qcperf -B build \
    -DCMAKE_BUILD_TYPE=Release \
    -DProjectVersion="0.1.0.0" \
    -DBACKENDS="NPU"
cmake --build build --parallel

The build produces the static library archives the C example links against:

build/libqcperfCore.a
build/libQcPerfDspNpuBackend.a
build/libQcPerfQCv.a
build/libQcPerfQMutex.a
build/libQcPerfQSleep.a
build/libQcPerfQThread.a
build/libQcPerfQTime.a
build/libQcPerfqlist.a
build/libQcPerfQcomDsp.so

Write a C Integration

For application-layer integration — embedding NPU telemetry directly in your inference loop, correlating metrics with latency measurements, or triggering adaptive behavior — use the libqcperf API directly. The full lifecycle is nine steps. Here is a minimal but complete program that streams all four NPU metrics to stdout.

The program

Create the source file:

mkdir -p ~/libqcperf-build/example

/* npu_monitor.c — minimal libqcperf NPU integration example */
#define _POSIX_C_SOURCE 200809L

#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

#include "qcperf.h"
#include "qcperf_common.h"

/* ── Shared state ─────────────────────────────────────────────────────────── */

static volatile sig_atomic_t g_running = 1;

/* Deep copy of backend info so the callback can resolve metric names. */
static struct QcPerfBackendInfo *g_info = NULL;

/* ── Signal handler ───────────────────────────────────────────────────────── */

static void on_signal(int sig) {
    (void)sig;
    g_running = 0;
}

/* ── Data callback ────────────────────────────────────────────────────────── */

/*
 * Called by the libqcperf background thread once per streaming interval
 * (1000 ms in this example).  data->metric_response holds all samples
 * collected during that window; we print only the most recent value for
 * each metric_id.
 */
static enum QcPerfReturnCode on_data(struct QcPerfData *data) {
    uint32_t idx = 0;
    uint64_t written = 0;

    if (NULL == data) {
        return QC_PERF_RETURN_CODE_FAILED;
    }

    printf("--- NPU snapshot ---\n");

    for (idx = data->metric_response_len; idx > 0; idx--) {
        uint16_t mid = data->metric_response[idx - 1].metric_id;
        uint64_t bit = (mid < 64) ? ((uint64_t)1 << mid) : 0;

        if (0 == bit || 0 != (written & bit)) {
            continue;   /* skip: out of range or already printed */
        }
        written |= bit;

        /* Resolve the metric name from the deep-copied backend info. */
        const char *name = NULL;
        if (NULL != g_info && NULL != g_info->capabilities_list) {
            uint8_t cap = data->capabilityId;
            if (cap < g_info->capabilities_list_length) {
                struct QcPerfCapabilityInfo *ci = &g_info->capabilities_list[cap];
                for (uint8_t m = 0; m < ci->metric_ids_list_len; m++) {
                    if (ci->metric_ids_list[m].metric_id == mid) {
                        name = ci->metric_ids_list[m].metric_name;
                        break;
                    }
                }
            }
        }

        if (NULL == name) {
            printf("  metric_%u: ", (unsigned)mid);
        } else {
            printf("  %-20s ", name);
        }

        struct QcPerfGenericType *v = &data->metric_response[idx - 1].metric_value;
        switch (v->data_type) {
        case QC_PERF_DATA_TYPE_DOUBLE:  printf("%.2f\n",  v->double_value);                    break;
        case QC_PERF_DATA_TYPE_UINT64:  printf("%llu\n",  (unsigned long long)v->uint64_value); break;
        case QC_PERF_DATA_TYPE_INT64:   printf("%lld\n",  (long long)v->int64_value);           break;
        case QC_PERF_DATA_TYPE_BOOL:    printf("%s\n",    v->bool_value ? "true" : "false");    break;
        default:                        printf("(unknown type)\n");                              break;
        }
    }

    return QC_PERF_RETURN_CODE_SUCCESS;
}

/* ── Message callback ─────────────────────────────────────────────────────── */

static enum QcPerfReturnCode on_message(struct QcPerfMessage *msg) {
    if (NULL == msg || NULL == msg->message) {
        return QC_PERF_RETURN_CODE_FAILED;
    }
    if (msg->message_level != QC_PERF_MESSAGE_LEVEL_DEBUG) {
        fprintf(stderr, "[backend] %s\n", msg->message);
    }
    return QC_PERF_RETURN_CODE_SUCCESS;
}

/* ── main ─────────────────────────────────────────────────────────────────── */

int main(void) {
    enum QcPerfReturnCode rc = QC_PERF_RETURN_CODE_FAILED;
    struct QcPerfBackendInfo *info = NULL;
    struct QcPerfRequest *req = NULL;
    int exit_code = 0;

    /* Install signal handlers for clean shutdown. */
    struct sigaction sa = {0};
    sa.sa_handler = on_signal;
    sigemptyset(&sa.sa_mask);
    sigaction(SIGTERM, &sa, NULL);
    sigaction(SIGINT,  &sa, NULL);

    /* Step 1: Initialise the library. */
    rc = qcperf_init();
    if (QC_PERF_RETURN_CODE_SUCCESS != rc) {
        fprintf(stderr, "qcperf_init failed (%d)\n", (int)rc);
        return 1;
    }

    /* Step 2: Connect to the NPU backend, registering the message callback. */
    rc = qcperf_connect_backend(QC_PERF_BACKEND_DSP_NPU, &on_message);
    if (QC_PERF_RETURN_CODE_SUCCESS != rc) {
        fprintf(stderr, "qcperf_connect_backend failed (%d)\n", (int)rc);
        exit_code = 1;
        goto deinit;
    }

    /* Step 3: Query capabilities and deep-copy for use in the callback. */
    info = (struct QcPerfBackendInfo *)calloc(1, sizeof(struct QcPerfBackendInfo));
    if (NULL == info) {
        fprintf(stderr, "calloc failed\n");
        exit_code = 1;
        goto disconnect;
    }

    rc = qcperf_get_capabilities_info(QC_PERF_BACKEND_DSP_NPU, info);
    if (QC_PERF_RETURN_CODE_SUCCESS != rc) {
        fprintf(stderr, "qcperf_get_capabilities_info failed (%d)\n", (int)rc);
        exit_code = 1;
        goto disconnect;
    }

    /*
     * Deep-copy into g_info so the data callback (called from a background
     * thread) can safely look up metric names without touching the stack-local
     * `info` pointer.
     */
    g_info = (struct QcPerfBackendInfo *)calloc(1, sizeof(struct QcPerfBackendInfo));
    if (NULL != g_info) {
        g_info->backend_id = info->backend_id;
        g_info->capabilities_list_length = info->capabilities_list_length;
        g_info->capabilities_list = (struct QcPerfCapabilityInfo *)calloc(
            info->capabilities_list_length, sizeof(struct QcPerfCapabilityInfo));
        if (NULL != g_info->capabilities_list) {
            for (uint8_t c = 0; c < info->capabilities_list_length; c++) {
                g_info->capabilities_list[c] = info->capabilities_list[c];
                uint8_t mlen = info->capabilities_list[c].metric_ids_list_len;
                g_info->capabilities_list[c].metric_ids_list =
                    (struct QcPerfMetricInfo *)calloc(mlen, sizeof(struct QcPerfMetricInfo));
                if (NULL != g_info->capabilities_list[c].metric_ids_list) {
                    for (uint8_t m = 0; m < mlen; m++) {
                        g_info->capabilities_list[c].metric_ids_list[m] =
                            info->capabilities_list[c].metric_ids_list[m];
                    }
                }
            }
        }
    }

    /* Step 4: Register the data callback. */
    rc = qcperf_set_data_callback(QC_PERF_BACKEND_DSP_NPU, &on_data);
    if (QC_PERF_RETURN_CODE_SUCCESS != rc) {
        fprintf(stderr, "qcperf_set_data_callback failed (%d)\n", (int)rc);
        exit_code = 1;
        goto disconnect;
    }

    /* Step 5: Build the request and start monitoring. */
    req = (struct QcPerfRequest *)calloc(1, sizeof(struct QcPerfRequest));
    if (NULL == req) {
        fprintf(stderr, "calloc failed\n");
        exit_code = 1;
        goto disconnect;
    }

    req->capability_id  = info->capabilities_list[0].capability_id;
    req->sampling_rate  = 100;   /* poll CDSP every 100 ms */
    req->streaming_rate = 1000;  /* deliver callback every 1000 ms */

    rc = qcperf_start(QC_PERF_BACKEND_DSP_NPU, req);
    if (QC_PERF_RETURN_CODE_SUCCESS != rc) {
        fprintf(stderr, "qcperf_start failed (%d)\n", (int)rc);
        exit_code = 1;
        goto disconnect;
    }

    fprintf(stderr, "Streaming NPU metrics — press Ctrl-C to stop\n");

    /* Step 6: Run until signalled. */
    while (0 != g_running) {
        sleep(1);
    }

    /* Step 7: Stop monitoring. */
    qcperf_stop(QC_PERF_BACKEND_DSP_NPU, req);
    free(req);
    req = NULL;

disconnect:
    /* Step 8: Disconnect the backend. */
    free(req);
    req = NULL;
    qcperf_disconnect_backend(QC_PERF_BACKEND_DSP_NPU);

deinit:
    /* Step 9: Deinitialise the library. */
    qcperf_deinit();

    /* Free caller-owned memory. */
    if (NULL != info) {
        free(info);
    }
    if (NULL != g_info) {
        if (NULL != g_info->capabilities_list) {
            for (uint8_t c = 0; c < g_info->capabilities_list_length; c++) {
                free(g_info->capabilities_list[c].metric_ids_list);
            }
            free(g_info->capabilities_list);
        }
        free(g_info);
    }

    return exit_code;
}

Save this as ~/libqcperf-build/example/npu_monitor.c.

Build the example

The example links against the same static library archive produced by the earlier build:

cd ~/libqcperf-build
gcc -std=c11 \
    -I libqcperf/qcperf/core/inc \
    -I libqcperf/qcperf/backends/inc \
    -I build/include \
    example/npu_monitor.c \
    build/libqcperfCore.a \
    build/libQcPerfDspNpuBackend.a \
    build/libQcPerfQCv.a \
    build/libQcPerfQMutex.a \
    build/libQcPerfQSleep.a \
    build/libQcPerfQThread.a \
    build/libQcPerfQTime.a \
    build/libQcPerfqlist.a \
    -L build -lQcPerfQcomDsp \
    -lcdsprpc \
    -lpthread \
    -o example/npu_monitor

Run it

cd ~/libqcperf-build
export LD_LIBRARY_PATH=build
./example/npu_monitor

Expected output (one block per second while a model is running):

Streaming NPU metrics — press Ctrl-C to stop
--- NPU snapshot ---
  Q6 Utilization      42.50
  Q6 Clock            614400.00
  HVX Utilization     12.30
  HMX Utilization     8.70
--- NPU snapshot ---
  Q6 Utilization      67.10
  Q6 Clock            729600.00
  HVX Utilization     31.80
  HMX Utilization     55.20

Press Ctrl-C to stop. The library shuts down cleanly on SIGINT.

Under the Hood

Sampling rate vs. streaming rate

These two parameters are independent and serve different purposes. The sampling rate (100 ms in the examples above) controls how often the background thread calls into the CDSP over FastRPC to read raw hardware counters. Lower values give finer time resolution but increase FastRPC overhead. The NPU backend supports 1, 5, 10, 50, 100, and 200 ms. The streaming rate (1000 ms) controls how often the background thread fires your data callback. Each callback delivery includes all samples collected since the last delivery — ten samples at 100 ms sampling / 1000 ms streaming. The callback receives them as a flat metric_response array; the example above uses a bitmask to extract only the most recent sample per metric. The supported streaming rates are 100 ms through 1000 ms in 100 ms steps.

The FastRPC path

libqcperf does not open a kernel driver or read a sysfs file. It calls sysmonquery_get_profdata over FastRPC — the same inter-processor RPC mechanism that llama.cpp and LiteRT-LM use to dispatch compute to the CDSP. The call crosses the kernel FastRPC bridge (/dev/fastrpc-cdsp) and returns a struct with the four hardware counter values directly from the DSP firmware. The runtime dependency is libcdsprpc.so. This shared library is already present on Qualcomm Ubuntu images as part of the FastRPC userland. If it is absent, the dynamic linker will fail to start the process before main is reached.

The background thread

qcperf_start spawns a single background thread named qcperf_dsp_npu_thread. This thread owns the FastRPC session for the duration of the monitoring session. Your data callback is called from this thread, not from the thread that called qcperf_start. Keep the callback fast; any blocking work should be handed off to a queue.

Interpreting the Metrics

Live telemetry turns the NPU from a black box into an observable subsystem.

Metric	Unit	What it tells you
Q6 Utilization	% of max Q6 clock	Scalar DSP pressure. High values mean the Q6 core is busy — either running non-vectorized code or managing HVX/HMX dispatch overhead.
Q6 Clock	KHz	The actual CDSP clock frequency chosen by DCVS. Correlate this with utilization: 80% utilization at 614 MHz is a very different situation from 80% at 1 GHz.
HVX Utilization	% of max Q6 clock	Hexagon Vector eXtensions engagement. HVX handles 128-byte SIMD operations — convolutions, activations, element-wise ops.
HMX Utilization	% of max Q6 clock	Hexagon Matrix eXtensions engagement. HMX is the dedicated matrix-multiply accelerator used for quantized linear layers.

A few patterns worth knowing: Low HMX during quantized inference is the most common surprise. If you expect a quantized model to be running on the NPU but HMX utilization is near zero, the workload is not taking the intended accelerator path. Common causes: the model was not compiled with HMX ops enabled, the QNN context binary version does not match the on-device runtime, or the model is falling back to CPU. HVX high, HMX low suggests the model is running vectorized but not matrix-accelerated — typical of FP16 or non-quantized paths, or of models that use HVX-friendly ops (pooling, normalization) but not INT8/INT4 matmuls. Q6 clock stepping up under load is DCVS working correctly. If the clock does not step up when utilization is high, check whether a power profile is capping the CDSP frequency. All metrics near zero while inference is running usually means the workload is executing on the CPU, not the DSP. Confirm with htop and check your model’s backend configuration.

Troubleshooting

Symptom	Likely cause	Fix
`error while loading shared libraries: libcdsprpc.so`	`libcdsprpc.so` not installed	Follow the IQ8 software setup guide; the library ships with the FastRPC userland package
`qcperf_connect_backend` returns `QC_PERF_RETURN_CODE_FAILED` with `[ERROR]` in journal	CDSP is inaccessible	Confirm `/dev/fastrpc-cdsp` exists; if not, the kernel or firmware does not have FastRPC enabled
All metric values are `0.00`	No DSP workload is active	The counters are hardware-accurate; zero means the CDSP is idle. Start an NPU inference workload
Build fails: `QCPERF_ENABLED_QCOM_LINUX_NPU` not set	CMake did not detect aarch64 Linux	Confirm you are building on or for `linux-aarch64`; the NPU backend is gated to that platform

Next Steps

With live NPU telemetry in place, the natural next step is to watch a real model run:

Run Gemma-4 E2B on the IQ8 NPU with LiteRT-LM — run npu_monitor alongside LiteRT-LM and watch HMX utilization climb during prefill
Run LLMs with llama.cpp on Dragonwing — correlate Q6 clock steps with llama.cpp’s token throughput
libqcperf API reference — full documentation for all backends, return codes, and struct fields
libqcperf on GitHub — source, issue tracker, and DEVELOPMENT-GUIDE for adding new backends

​Introduction

​Why Existing Paths Fall Short

​What You Will Do

​Prerequisites

​Build libqcperf

​Clone the repository

​Configure and build

​Write a C Integration

​The program

​Build the example

​Run it

​Under the Hood

​Sampling rate vs. streaming rate

​The FastRPC path

​The background thread

​Interpreting the Metrics

​Troubleshooting

​Next Steps