is there example code for running multiple models on one hailo8 device using c++ API of hailort. In my case object detection + image classification of two different streams is required.
This comment in the example code
// Constructor for when using multiple models on the same device
AsyncModelInfer(const std::string &hef_path, const std::string &group_id);
hints that it should be possible using VDevice and passing a group_id. But the doc on these classes / methods is relatively thin. Any further hints on doc / reference examples for such application.
For what you’re looking to do, there’s actually a really good C++ example that shows how to work with multiple HEFs. Check this one out:
Here’s how you’d approach it:
To run multiple models on a single Hailo-8 device using the HailoRT C++ API, you’ll want to use the VDevice (Virtual Device) abstraction along with a group_id when setting up your models. This lets the runtime handle coordination between multiple models through the Concurrent Pipeline Processing (CCPP) scheduler.
VDevice: This abstracts your physical device(s) into a single logical device - works even if you’re just using one Hailo-8
group_id: This associates multiple models (like object detection + classification) to the same scheduler context so they can run concurrently
AsyncModelInfer: The class from our application examples that handles inference with async scheduling
Here’s the basic structure:
std::string group_id = "multi_model_app";
// Create VDevice (you can also specify device IDs if needed)
auto vdevice = hailort::VDevice::create().value();
// Load your HEFs
auto model1 = std::make_shared<AsyncModelInfer>("yolov8s.hef", group_id);
auto model2 = std::make_shared<AsyncModelInfer>("yolov5-seg.hef", group_id);
// Run inference on each model
model1->infer(...);
model2->infer(...);