@shashi I have been looking to test face detection on a pi 5 with an AI HAT+ (hailo8) with a Pi Camera using the degirum PySDK, and while the it runs ok and i see the preview window showing the detected face, it does not appear to be using the NPU. I was wondering if you have any advice or if I am misreading the situation. Thank you in advance for any help you can provide.
Here are the details:
Program output:
(venv_pi_demo) conorroche@picam1:~/pi-demo $ python degirum_simple.py
[0:04:08.369897242] [1983] INFO Camera camera_manager.cpp:326 libcamera v0.5.0+59-d83ff0a4
[0:04:08.376901719] [2010] INFO RPI pisp.cpp:720 libpisp version v1.2.1 981977ff21f3 29-04-2025 (14:13:50)
[0:04:08.386291847] [2010] INFO RPI pisp.cpp:1179 Registered camera /base/axi/pcie@1000120000/rp1/i2c@88000/ov5647@36 to CFE device /dev/media3 and ISP device /dev/media0 using PiSP variant BCM2712_D0
[0:04:08.389187776] [1983] INFO Camera camera.cpp:1205 configuring streams: (0) 640x480-RGB888 (1) 640x480-GBRG_PISP_COMP1
[0:04:08.389298297] [2010] INFO RPI pisp.cpp:1483 Sensor: /base/axi/pcie@1000120000/rp1/i2c@88000/ov5647@36 - Selected sensor format: 640x480-SGBRG10_1X10 - Selected CFE format: 640x480-PC1g
the cpu spikes to ~120%
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1983 conorro+ 20 0 3407552 340944 173504 S 119.9 4.1 0:29.42 python
774 root 20 0 1106032 24160 18128 S 30.9 0.3 0:06.77 hailort_service
2073 conorro+ 20 0 324496 63808 34400 S 24.6 0.8 0:05.13 python3
2067 conorro+ 20 0 324496 64720 34864 S 21.9 0.8 0:05.01 python3
and hailortcli monitor shows:
Monitor did not retrieve any files. This occurs when there is no application currently running.
If this is not the case, verify that environment variable 'HAILO_MONITOR' is set to 1.
I have set the HAILO_MONITOR env var in the terminal i am running the python script from.
(venv_pi_demo) conorroche@picam1:~/pi-demo $ echo $HAILO_MONITOR
1
(venv_pi_demo) conorroche@picam1:~/pi-demo $ python degirum_simple.py
If i run the model directly
(venv_pi_demo) conorroche@picam1:~/pi-demo $ hailortcli run .models/scrfd_10g--640x640_quant_hailort_hailo8_1/scrfd_10g--640x640_quant_hailort_hailo8_1.hef -t 30
Running streaming inference (.models/scrfd_10g--640x640_quant_hailort_hailo8_1/scrfd_10g--640x640_quant_hailort_hailo8_1.hef):
Transform data: true
Type: auto
Quantized: true
Network scrfd_10g/scrfd_10g: 23% | 2127 | FPS: 303.54 | ETA: 00:00:2
Then cpu is fine:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
32645 conorro+ 20 0 2143712 36160 26496 S 47.8 0.4 0:11.92 hailortcli
And I can see it is using the NPU as expected:
conorroche@picam1:~ $ hailortcli monitor
Device ID Utilization (%) Architecture
----------------------------------------------------------------------------------------------------------------------------------------------------------------
0001:01:00.0 100.0 HAILO8
Model Utilization (%) FPS PID
----------------------------------------------------------------------------------------------------------------------------------------------------------------
scrfd_10g 100.0 303.6 32689
Model Stream Direction Frames Queue
Avg Max Min Capacity
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
scrfd_10g scrfd_10g/input_layer1 H2D 0.50 1 0 4
scrfd_10g scrfd_10g/conv58 D2H 0.50 1 0 4
scrfd_10g scrfd_10g/conv56 D2H 0.50 1 0 4
scrfd_10g scrfd_10g/conv51 D2H 0.50 1 0 4
scrfd_10g scrfd_10g/conv50 D2H 0.50 1 0 4
scrfd_10g scrfd_10g/conv57 D2H 0.50 1 0 4
Degirum Sys Info:
(venv_pi_demo) conorroche@picam1:~/pi-demo $ degirum sys-info
Devices:
HAILORT/HAILO8:
- '@Index': 0
Board Name: Hailo-8
Device Architecture: HAILO8
Firmware Version: 4.20.0
ID: '0001:01:00.0'
Part Number: ''
Product Name: ''
Serial Number: ''
N2X/CPU:
- '@Index': 0
TFLITE/CPU:
- '@Index': 0
- '@Index': 1
Software Version: 0.16.2
The python code i am running is as follows:
import degirum as dg
import degirum_tools
from picamera2 import Picamera2
import matplotlib.pyplot as plt
import numpy as np
import cv2
# Define a frame generator: a function that yields frames from the Picamera2
def frame_generator():
picam2 = Picamera2()
# Configure the camera (optional: set the resolution or other settings)
picam2.configure(picam2.create_preview_configuration({'format': 'RGB888'}))
# Start the camera
picam2.start()
try:
while True:
# Capture a frame as a numpy array
frame = picam2.capture_array()
# Yield the frame
yield frame
finally:
picam2.stop() # Stop the camera when the generator is closed
# Specify the model name
face_det_model_name = "scrfd_10g--640x640_quant_hailort_hailo8_1"
# Specify the inference host address
#inference_host_address = "@cloud" # Use "@cloud" for cloud inference
inference_host_address = "@local" # Use "@local" for local inference
# Specify the zoo_url
#zoo_url = "degirum/models_hailort"
zoo_url = ".models" # For local model files
token = '' # Leave empty for local inference
# Load the face detection model
face_det_model = dg.load_model(
model_name=face_det_model_name,
inference_host_address=inference_host_address,
zoo_url=zoo_url,
token=token,
overlay_color=(0, 255, 0) # Green color for bounding boxes
)
# Process the video stream by AI model using model.predict_batch():
for result in face_det_model.predict_batch(frame_generator()):
# Display the frame with AI annotations in a window named 'AI Inference'
cv2.imshow("AI Inference", result.image_overlay)
# Process GUI events and break the loop if 'q' key was pressed
if cv2.waitKey(1) & 0xFF == ord("q"):
break
# Destroy any remaining OpenCV windows after the loop finishes
cv2.destroyAllWindows()