I have the following configuration:
hailo-CLIP runs fine on my rpi5. It takes the image from a connected rpi3 camera and sends the output and detection results to the desktop and in parallel to a rtsp server. I edited therefore the detection_pipeline and “tee’d” the stream over to my Homeassistant rtsp server in parallel.
Everything works as desired.
Now I want to deploy the rpi and the camera to a remote location to survey the cats in my backyard. The rpi then is no longer connected to a monitor nor mouse/keyboard.
When I now ssh into the rpi, and then start the clip application, I get the following gstreamer error:
$ ./catcam_CLIP_start.sh
Setting up the environment...
Setting up the environment for hailo-tappas-core...
TAPPAS_VERSION is 3.30.0. Proceeding...
You are not in the hailo_clip_venv virtual environment.
Virtual environment exists. Activating...
TAPPAS_POST_PROC_DIR set to /usr/lib/aarch64-linux-gnu/hailo/tappas/post_processes
DEVICE_ARCHITECTURE is set to: HAILO8
2025-01-22 16:09:48,350 - clip_app.logger_setup - INFO - PIPELINE:
gst-launch-1.0 appsrc name=app_source is-live=true leaky-type=downstream max-buffers=3 ! videoflip name=videoflip video-direction=horiz ! video/x-raw, format=RGB, width=1280, height=720 ! queue name=source_scale_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0 ! videoscale name=source_videoscale n-threads=2 ! queue name=source_convert_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0 ! videoconvert n-threads=3 name=source_convert qos=false ! video/x-raw, pixel-aspect-ratio=1/1, format=RGB, width=1280, height=720 ! tee name=clip_t hailomuxer name=clip_hmux clip_t. ! queue name=clip_bypass_q leaky=no max-size-buffers=20 max-size-bytes=0 max-size-time=0 ! clip_hmux.sink_0 clip_t. ! queue name=clip_muxer_queue leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0 ! videoscale n-threads=4 qos=false ! queue name=clip_inference_scale_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0 ! videoscale name=clip_inference_videoscale n-threads=2 qos=false ! queue name=clip_inference_convert_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0 ! video/x-raw, pixel-aspect-ratio=1/1 ! videoconvert name=clip_inference_videoconvert n-threads=2 ! queue name=clip_inference_hailonet_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0 ! hailonet name=clip_inference_hailonet hef-path=/home/StephanS/hailo-CLIP/resources/clip_resnet_50x4.hef batch-size=8 vdevice-group-id=1 scheduler-timeout-ms=1000 scheduler-priority=16 force-writable=true ! queue name=clip_inference_hailofilter_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0 ! hailofilter name=clip_inference_hailofilter so-path=/home/StephanS/hailo-CLIP/resources/libclip_post.so qos=false ! queue name=clip_inference_output_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0 ! clip_hmux.sink_1 clip_hmux. ! queue name=clip_hmux_queue leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0 ! hailopython name=pyproc module=/home/StephanS/hailo-CLIP/clip_app/clip_hailopython.py qos=false ! queue name=clip_postprocess_queue leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0 ! identity name=identity_callback ! queue name=hailo_display_overlay_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0 ! hailooverlay name=hailo_display_overlay ! textoverlay text="Detection Pipeline" valignment=top halignment=right font-desc="Areal, 12" ! tee name=myrtspstream ! queue name=hailo_display_videoconvert_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0 ! videoconvert name=hailo_display_videoconvert n-threads=2 qos=false ! queue name=hailo_display_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0 ! fpsdisplaysink name=hailo_display video-sink=autovideosink sync=false text-overlay=True signal-fps-measurements=true myrtspstream. ! videoconvert ! video/x-raw, format=I420 ! x264enc tune=zerolatency bitrate=500 speed-preset=superfast ! video/x-h264, profile=baseline ! rtspclientsink location=rtsp://localhost:8554/mystream
appsrc properties: <__gi__.GstAppSrc object at 0x7fffb0590040 (GstAppSrc at 0xd6e4460)>
[0:05:51.510238826] [4339] INFO Camera camera_manager.cpp:325 libcamera v0.3.2+99-1230f78d
[0:05:51.517894656] [4340] INFO RPI pisp.cpp:695 libpisp version v1.0.7 28196ed6edcf 29-08-2024 (16:33:32)
[0:05:51.528761246] [4340] INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a to CFE device /dev/media3 and ISP device /dev/media1 using PiSP variant BCM2712_C0
[0:05:51.532516077] [4339] INFO Camera camera.cpp:1197 configuring streams: (0) 1280x720-RGB888 (1) 1280x720-RGB888 (2) 1536x864-BGGR_PISP_COMP1
[0:05:51.532645929] [4340] INFO RPI pisp.cpp:1450 Sensor: /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected CFE format: 1536x864-PC1B
Picamera2 configuration: width=1280, height=720, format=RGB
picamera_process started
2025-01-22 16:09:48,873 - clip_app.logger_setup - INFO - No text embedding runtime selected, adding new text is disabled. Loading /home/StephanS/hailo-CLIP/embeddings.json
2025-01-22 16:09:48,873 - clip_app.logger_setup - INFO - Loading embeddings from /home/StephanS/hailo-CLIP/embeddings.json
2025-01-22 16:09:48,875 - clip_app.logger_setup - INFO - Text box 0 is set to negative: True
2025-01-22 16:09:48,875 - clip_app.logger_setup - INFO - Text box 5 is set to negative: True
2025-01-22 16:09:48,876 - clip_app.logger_setup - INFO - Setting detection threshold to: 0.76
2025-01-22 16:09:51,692 - clip_app.logger_setup - INFO - State change failed.
2025-01-22 16:09:51,730 - clip_app.logger_setup - ERROR - Error: gst-resource-error-quark: GStreamer encountered a general resource error. (1) ../sys/kms/gstkmssink.c(2032): gst_kms_sink_show_frame (): /GstPipeline:pipeline0/GstFPSDisplaySink:hailo_display/GstAutoVideoSink:autovideosink0/GstKMSSink:autovideosink0-actual-sink-kms:
drmModeSetPlane failed: Permission denied (13)
Is there any solution to run CLIP from remote, not directly from the desktop?