Detection of generic objects (camouflaged intruders) using Hailo 8

General Discussion
1

Hi Hailo Team,

  I’m trying to use HAILO 8 (26Tops) for intrusion detection purposes. I would like to be able to accurately detect human intrusion camouflaged or not. 
  By camouflaged intrusion I understand an intruder potentially wearing a sheet, a gillie, a cardboard, a beach umbrella or any other (Halloween) disguise that can hide all its body making him/her unrecognizable as a class by a YOLO type model (example).
  So the need is to detect a generic/unknown object of a minimum given size/area appearing inside a camera view. 
  The criteria should be:
   new object + coherent shape + movement + protected-zone crossing + persistence

 The problem is that YOLO or any other stdc1 or resnet models I tried with Hailo 8 do not have an "unknown object" class, nor temporal awareness.
 The problem is that (if I'm right) the most sensitive type of intrusion, the camouflaged one, falls in the domain of MOG2 algorithm that runs on the CPU, not on HAILO. As far as I understand, MOG2 and is a background substractor prone to false positives, etc.
 If my reasoning is correct, in case of camouflaged intrusion, if no class is found for the object a HAILO running YOLO or resnet, etc may discard the object and ignore the intrusion altogether.
 I would like to be able to use HAILO 8 to do the heavy lifting of unknown object detection. 
  

  Is this possible? Could you recommend a model or a path to follow (Hailo +CPU, etc.) ?

Note: I tinkered also with Frigate but it also uses an Yolo model that, if I'm right, may miss the intrusion if based solely on classes.

Thank you four your insight!

Best Regards,
2

Hi @Cristian_Bogus,

YOLO’s 80 COCO classes have no “unknown object” concept.
The best path on Hailo-8 is a hybrid approach : use lightweight CPU-side background subtraction (MOG2 or frame differencing) to detect any new blob of sufficient size/persistence in your protected zone (this is inherently class-agnostic), then send those candidate ROIs to the Hailo-8 running CLIP hailo-apps/hailo_apps/python/pipeline_apps/clip at main · hailo-ai/hailo-apps · GitHub for open-vocabulary verification with prompts like “person hiding”, “foreign object”, “normal background” - if it’s not “normal background,” alert. This way the CPU handles the class-agnostic motion/change detection (your criteria: new object + shape + movement + zone + persistence), while Hailo does the heavy neural lifting to confirm the anomaly, giving you both sensitivity to disguised intruders and robustness against false positives.

Regarding Frigate - it’s worth verifying whether it only alerts on YOLO-recognized classes or if it can also surface raw motion events that passed its internal motion detection but matched no class.

Thanks,

3

Hi Michael,

Thank You for your message and for the very interesting CLI suggestion.

I’m a newcomer in this field. Since I think my problem is a general intrusion problem I’ll try to see if Frigate already solved it and I’ll proceed from there!

Thanks again!

Best Regards,

Cristian

1 Like