CLIP on live feed from rpi camera

General
1

I was able to install CLIP with RPI5 and hailo, following the instructions in the repository.
The demo video works also well.
I would like to run CLIP on live feed from RPI camera. Using /dev/video0 or other videoX doesn’t work.

2024-09-19 12:37:56,920 - clip_app.logger_setup - INFO - Loading model RN50x4 on device cpu, this might take a while…
2024-09-19 12:38:06,251 - clip_app.logger_setup - INFO - Using clip for text embedding
2024-09-19 12:38:06,251 - clip_app.logger_setup - INFO - Loading embeddings from /home/kone/hailo-CLIP/embeddings.json

2024-09-19 12:38:06,253 - clip_app.logger_setup - INFO - Text box 0 is set to negative: True
2024-09-19 12:38:06,348 - clip_app.logger_setup - INFO - State change failed.
2024-09-19 12:38:06,410 - clip_app.logger_setup - ERROR - Error: gst-stream-error-quark: Internal data stream error. (1) …/libs/gst/base/gstbasesrc.c(3132): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
2024-09-19 12:38:06,410 - clip_app.logger_setup - INFO - Sending EOS event to the pipeline…
2024-09-19 12:38:06,411 - clip_app.logger_setup - ERROR - Error: gst-stream-error-quark: Stream doesn’t contain enough data. (4) …/plugins/elements/gsttypefindelement.c(1012): gst_type_find_element_chain_do_typefinding (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind:
Can’t typefind stream

2

Hi are you trying raspberry pi camera or USB camera?
For RPi camera it is not supported yet on this app you can copy the libcamerasrc pipeline from the rpi5 examples repo.
We will add rpi camera support for the CLIP app soon.
If you are using USB camera make sure you are using the correct device. It’s not necessarily /dev/video0. Make sure it is by testing it with ffplay /dev/video0. If it does not work try other video device. video2, video4 etc …

3

Hello, is there any update on whether or not RPi camera is supported yet? I’ve gotten the CLIP app to work but I’m having the same issues as dario (OP). From what I can tell, /dev/video0 is the camera’s address but I get the same error as seen above. Any updates would be appreciated.

4

Hi Luke,
Actually that was missing from the example.

I’ve added this code:

image

In the clip_pipeline.py. Now, you can call clip_app --input rpi, and it will work! We will add it to the app itself.

1 Like