Detection and OCR under 100 ms, on a board that did not exist yet.
The product needed object detection plus text recognition running on device, fast enough to feel instant, on hardware cheap enough to ship in volume. No cloud round trip.
And one more constraint: the same C++ binary had to run on a developer desktop and on the embedded board, or iteration speed would collapse.
One pipeline, three inference backends.
The vision pipeline is written once against an inference interface with three implementations: ONNX Runtime for the desktop, RKNN for the NPU on device, OpenCV DNN as the fallback. Swapping backends is a config change, not a rewrite.
Frames never leave GPU memory between capture and inference: GStreamer moves them zero-copy, which is where most of the latency budget was won.
Software and the board it runs on.
C++17, YOLO for detection, PP-OCRv4 (DBNet for text detection, CRNN for recognition) for reading. The device presents itself as a USB HID gadget so the host needs no driver. systemd supervises it.
The board is ours too: an 8-layer PCB around an RK3576 with LPDDR4X, laid out in KiCad.
30 fps, on device, no cloud.
Detection and OCR run entirely on the board. The same binary debugs on a laptop, which is why the thing shipped at all.
Need something like this?
Start a project