Hello everyone,
I would like to share my project bult for Hailo8 and Hailo10 and Raspberry Pi.
It’s similar to Frigate - it allows you to connect multiple RSTP camera feeds, grab a frame and draw ‘forbidden zones’ on it. Then you can select classes for object detection and define actions triggered by object detection.
I built it to detect my cats attacking plants in my living room.
The solution includes occlusion detection implemented with open cv. If the drawn zone is much different from the captured frame - it will be marked as occluded and stop detecting in that zone - i implemented it because i very often place a drying rack for my laundry between my camera and a plant.
Optionally you can enable depth detection gating → A cat (or a person or a dog or a bird) overlapping with the drawn zone might trigger a lot of false positive so I run also depthanything visual transformer that let’s you trigger alerts only when the object overlaps with the drawn zone but also is in a roughly the same distance from the camera.
Eventually for Hailo10H users there is also optional VLM gate - you can write your custom prompt that will run on succesful detection (e.g. Is the cat attacking the plant?, but i can also imagine something like ‘is this person wearing a helmet’) - and if it’s enabled then the alert will be triggered only if the VLM outputs ‘Yes’
Among configurable alerts you have an SMB File upload, Telegram notification and custom webhook - so that you can wire it to any kind of automation you would like.
I tested it on Raspberry Pi 5 (16GB and 8GB) and Hailo10H and Hailo8 - of course VLM gating doesn’t work on Hailo8.
The repo includes an install script that pretty much does everything automatically for you.
I am happy with the current implementation - I get alerted when my cats do nasty things although VLM gating isn’t super reliable due to the quantized models you can run on hailo. I installed it also for my father to play sounds to scare away doves that sit on the power lines above his driveway
repository: GitHub - PGielniak/catyolo: monorepo for catyolo projects · GitHub
screenshots:
Adding scenes (I tested on 2 cameras at the same time. Capped at 15FPS and it works okay)
