CLIP on live feed from rpi camera

General
1

I was able to install CLIP with RPI5 and hailo, following the instructions in the repository.
The demo video works also well.
I would like to run CLIP on live feed from RPI camera. Using /dev/video0 or other videoX doesn’t work.

2024-09-19 12:37:56,920 - clip_app.logger_setup - INFO - Loading model RN50x4 on device cpu, this might take a while…
2024-09-19 12:38:06,251 - clip_app.logger_setup - INFO - Using clip for text embedding
2024-09-19 12:38:06,251 - clip_app.logger_setup - INFO - Loading embeddings from /home/kone/hailo-CLIP/embeddings.json

2024-09-19 12:38:06,253 - clip_app.logger_setup - INFO - Text box 0 is set to negative: True
2024-09-19 12:38:06,348 - clip_app.logger_setup - INFO - State change failed.
2024-09-19 12:38:06,410 - clip_app.logger_setup - ERROR - Error: gst-stream-error-quark: Internal data stream error. (1) …/libs/gst/base/gstbasesrc.c(3132): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
2024-09-19 12:38:06,410 - clip_app.logger_setup - INFO - Sending EOS event to the pipeline…
2024-09-19 12:38:06,411 - clip_app.logger_setup - ERROR - Error: gst-stream-error-quark: Stream doesn’t contain enough data. (4) …/plugins/elements/gsttypefindelement.c(1012): gst_type_find_element_chain_do_typefinding (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind:
Can’t typefind stream

2

Hi are you trying raspberry pi camera or USB camera?
For RPi camera it is not supported yet on this app you can copy the libcamerasrc pipeline from the rpi5 examples repo.
We will add rpi camera support for the CLIP app soon.
If you are using USB camera make sure you are using the correct device. It’s not necessarily /dev/video0. Make sure it is by testing it with ffplay /dev/video0. If it does not work try other video device. video2, video4 etc …