Depth estimation rpi5 + hailo kit

General
1

Hi
we would like to use a rpi camera (wide angle) mounted on the ceiling of a small room, to see if there are objects in the room and how much space they occupy. The final goal is to provide a percentage of space occupied in the room to another system.
So it’s not really object classification…as it can be any object there. We are thinking of fast sam or depth estimation.

But it’s not too clear how to use compiled models like the one described in the github rep (from model zoo)
HAILO8L_depth_estimation.rst

or
HAILO8L_zero_shot_instance_segmentation.rst

Thanks

2

Hi @dario.ravarro,
This seems like an interesting project! You have to remember that NN are quite good at what they was trained to do, and becomes less so as farther you devaite form the original usecase.
We have the example listed below SCDepthV3, it wan’t trained on camera angle from the ceiling of the room, so I’m not sure how well will it perform. I still think it’s worth the check.
Hailo-Application-Code-Examples/runtime/cpp/depth_estimation/scdepthv3 at a30762227e2f8304f9cb782519a9e0b778e732e5 · hailo-ai/Hailo-Application-Code-Examples (github.com)

3

I compiled the cpp in rpi5, but I get an error that hef format is not compatible with this device.
Device arch: HAILO8L, HEF arch : HAILO8

4

update: I downloaded the hef file for Hailo8L from model zoo, docs, folder Hailo8l
as suggested in

but, launching ./build/depth_estimation_example_cpp -hef=scdepthv3.hef -path=input_video.mp4

outputs this

CHECK failed - HEF file length does not match
CHECK_SUCCESS failed…
failed parsing HEF file
failed creating HEF
CHEC_SUCCESS failed…
Failed to configure network group scdepthv3.hef

5

@dario.ravarro it could be that either the file is corrupted or there is an issue with version mismatch.

You could try re-downloading and re-copying the file. If that doesn’t solve it, please try this hef version instead:
https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ModelZoo/Compiled/v2.11.0/hailo8l/scdepthv3.hef

6

I also tried v2.10.0 and it worked.
Is there an example code to run depth estimation live on a rtsp stream?
Also any example code in python that uses scdepthv3.hef ?

7

Glad to hear!

Not sure which host you are using, but TAPPAS has an example pipeline for multistream over rtsp.

Not currently, depending on the demand we may add it in the future, but so far it hasn’t been highly requested.

Implementing such a scdepthv3 Python script is quite simple though. You can take any of our Python examples and switch the post process to perform the below:

mask = tf.math.reciprocal(tf.math.sigmoid(DEVICE_OUTPUT) * 10 + 0.009)

You can then visualize the results as it is done here.

9

How to modify the depth_estimation.cpp in Hailo application code examples to provide an output like an array of depth for each pixel in the image, frame by frame? This would be useful to evaluate if a room is empty or occupied by a certain percentage