Detection app with hailo_platform, how to track_id properly?

General Discussion
41

This is great question. Yes, zone coordinates are specified in pixels of input image. Say, your camera outputs FullHD, 1920x1080 and you want to detect only in the top half of the frame. Then you specify zone like [[0,0], [1920, 0], [1920,540], [0,540]]. If your image changes size, you need to recalculate zone coordinates to new size. Current implementation of zone counter does not accept zone coordinates as percentage of the frame size yet. But you can do it in your code, if you know the frame size in advance.

Since the model itself does image resizing to the model input size, and then automatically recalculates the bbox coordinates to original image size, you don’t care about that conversion - PySDK will do it for you. So the whole pipeline operates in the input image coordinate system (unless you explicitly insert some resizer gizmo in the middle).

42

degirum-tools has nifty feature to adjust zone coordinates in OpenCV window by mouse. See this example:
PySDKExamples/examples/specialized/object_in_zone_counting_video_stream.ipynb at main · DeGirum/PySDKExamples

All you need to do is to specify window_name parameter in ZoneCounter() constructor call to be equal to OpenCV window title used for display. In that example you do it by with degirum_tools.Display(window_name) as display: call - you pass the same window_name var there.

Once you do it, you may drag zones by mouse:

  • Left-click and drag to move entire polygon.
  • Right-click and drag to move individual vertices.
    It updates internal polygon state on changes, so you can access updated zone coordinates in zone counter object _polygons member.

Of course, this is not for production. But can be used as handy tool.

43

Do not know why but I had a prefeeling that you would be pointing me to that example, that’s exactly why I said ‘I don’t plan to have a window for mouse selection’ :rofl:

Did a little bit of research and found the method get_video_properties() I can use in dgstreams.VideoSourceGizmo(video_source)which gives back a tuple of ( width, heigth, fps)

So I can get the ‘size’ of the input stream. So if I got it right in a 1920x1080 image a full zone would be [[0,0], [1920, 0], [1920,1080], [0,1080]] ?¿

44

I have one more question…

How can I config the notification destination? or that cannot be done at runtime?¿

I’m reading the apprise documentation and does not understand how it works…
Managed to send a email via console, but I don’t get how to use it here…

Do i need to know the username and password of the destination email?¿¿
This has no sense at all !!!

If destination cannot be configured at runtime then do I need a configuration file per each destination?¿

45

I’m uunable to make work the smart nvr example :frowning:

It throws the next execption:

[[0, 0], [1920, 0], [1920, 1080], [0, 1080]]
Traceback (most recent call last):
  File "/root/api_ai_inference/stream_service.py", line 116, in <module>
    dgstreams.Composition(cam_source >> detector >> streamer).start()
  File "/usr/local/lib/python3.10/dist-packages/degirum_tools/streams/base.py", line 695, in start
    self.wait()
  File "/usr/local/lib/python3.10/dist-packages/degirum_tools/streams/base.py", line 771, in wait
    raise Exception(errors)
Exception: Error detected during execution of AiSimpleGizmo:
  <class 'degirum.exceptions.DegirumException'>: Failed to perform model 'yolo11s_coco--640x640_quant_hailort_hailo8_1' inference: OpenCV(4.11.0) /io/opencv/modules/imgproc/src/drawing.cpp:2426: error: (-215:Assertion failed) p.checkVector(2, CV_32S) > 0 in function 'fillPoly'


degirum.exceptions.DegirumException: OpenCV(4.11.0) /io/opencv/modules/imgproc/src/drawing.cpp:2426: error: (-215:Assertion failed) p.checkVector(2, CV_32S) > 0 in function 'fillPoly'


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/degirum_tools/streams/base.py", line 662, in gizmo_run
    gizmo.run()
  File "/usr/local/lib/python3.10/dist-packages/degirum_tools/streams/gizmos.py", line 689, in run
    for result in self.model.predict_batch(source()):
  File "/usr/local/lib/python3.10/dist-packages/degirum/model.py", line 293, in predict_batch
    for res in self._predict_impl(source):
  File "/usr/local/lib/python3.10/dist-packages/degirum/model.py", line 1233, in _predict_impl
    raise DegirumException(msg) from saved_exception
degirum.exceptions.DegirumException: Failed to perform model 'yolo11s_coco--640x640_quant_hailort_hailo8_1' inference: OpenCV(4.11.0) /io/opencv/modules/imgproc/src/drawing.cpp:2426: error: (-215:Assertion failed) p.checkVector(2, CV_32S) > 0 in function 'fillPoly'

If I remove the notitifer, detector and counter the error dissapear.

I also tried with all the yolo models I was able to find, and every one gives the same error, with the oonly difference being the model name of course

This is how the code looks:

hw_location="10.0.0.2:8778"
#model_name = "yolo11n_visdrone_person--640x640_quant_hailort_multidevice_1"
#model_name = "yolo11n_coco--640x640_quant_hailort_hailo8_1"
model_name = "yolo11s_coco--640x640_quant_hailort_hailo8_1"
#model_name = "yolov8s_coco--320x320_quant_hailort_hailo8_1"
#model_name = "yolov8n_coco--640x640_quant_hailort_hailo8_1"
#model_name = "yolov8m_coco--640x640_quant_hailort_hailo8_1"
#model_name = "yolov8s_coco--640x640_quant_hailort_hailo8_1"
#model_name = "yolov5s_coco--640x640_quant_hailort_multidevice_1"
#model_name = "yolov5n_relu6_coco--640x640_quant_hailort_hailo8_1"
#model_name = "yolov5s_relu6_coco--640x640_quant_hailort_hailo8_1"

model_zoo_url="aiserver://home/pi/DeGirum/zoo"
video_source = args.input
video_output= args.output
classes = {"clock"}
device_type = "HAILORT/HAILO8"

model_manager = dg.connect(
    inference_host_address=hw_location,
    zoo_url = model_zoo_url
)

model = model_manager.load_model(
    model_name=model_name,
    device_type=device_type,
    #output_confidence_threshold=0.3,
    input_pad_method="letterbox",
    image_backend='opencv',
    overlay_color=[255,0,0],
    output_class_set=classes
)

anchor = degirum_tools.AnchorPoint.CENTER

# create object tracker
tracker = degirum_tools.ObjectTracker(
    class_list=classes,
    track_thresh=0.35,
    track_buffer=100,
    match_thresh=0.9999,
    trail_depth=20,
    anchor_point=anchor,
    show_only_track_ids = True,
    #show_overlay = True,
    annotation_color = [255,0,0]
)


cam_source = dgstreams.VideoSourceGizmo(video_source)

width=cam_source.get_video_properties()[0]
height=cam_source.get_video_properties()[1]
zones=[[0,0],[width,0],[width,height],[0,height]]
print(zones)

#
# create analyzers:
#

event_name = "object_detected"

zone_counter = degirum_tools.ZoneCounter(
    zones,
    use_tracking=True,
    triggering_position=[anchor],
    annotation_color=(0, 255, 0),
)


zone_detector = degirum_tools.EventDetector(
      f"""
      Trigger: {event_name}
      when: ZoneCount
      is greater than: 0
      during: [10, frames]
      for at least: [90, percent]
      """,
      show_overlay=False,
)

holdoff_sec = 3.0
notification_config = "notifyconf.txt"  # file with apprise url

# event notifier
notifier = degirum_tools.EventNotifier(
      event_name,
      event_name,
      message="{time}: person is detected in zone",
      holdoff=holdoff_sec,
      notification_config=notification_config,
      clip_save=False,
)

degirum_tools.attach_analyzers(model, [tracker, zone_counter, zone_detector, notifier])
#degirum_tools.attach_analyzers(model, [tracker])

detector = dgstreams.AiSimpleGizmo(model)

streamer = dgstreams.VideoStreamerGizmo(video_output, show_ai_overlay=True)

dgstreams.Composition(cam_source >> detector >> streamer).start()
46

Why this forum does not give me the option to edit¿¿?¿?¿?

I found the problem,

My zone deeclaration was like

zones = [[0, 0], [1920, 0], [1920, 1080], [0, 1080]]

and it should be
`zones = [ [[0, 0], [1920, 0], [1920, 1080], [0, 1080]], ]

1 Like
47

I found the way to change destinations in runtime

The notifier does not only allow to put the config file in ‘notification_config’ parameter, but also the actual mailto string, so the destination can be cahnged in runtime having it cofigured like:

email_destination="perico@lospalotes.com"
notifier = degirum_tools.EventNotifier(
      event_name,
      event_name,
      message="{time}: person is detected in zone",
      holdoff=holdoff_sec,
      notification_config=f"mailtos://mail.server.com?user=email@server.com?pass=pass?from="AI Detection Service<noreply@server.com>"?to={email_destination}"
1 Like
48

I kept reading the docs, and noticed something… please correct me if I’m wrong:

As my detection zone is always the full video area, doing the calculation for zone points using the input video resolution and also creating a ZoneCounter are in fact not the best option isn’t?

There is a mention to a function in the event detector named ObjectCount and the documentation also say “This metric does not require any auxiliary analyzer”…

So I think for anyone with ony 1 zone is better to use this ObjectCount function instead of ZoneCounter more resource conservative, isn’t?

49

Hi @Dario_Quesada

Yes, if you have only one zone, you do not need a zone counter as results need not be filtered.

50

about the notifier…

I am unable to find information about this:

if instead a video-clip I wanted to send a screenshot how can that be done?

also do you know if it is possible to trigger another function from the script like a callback using the notifier?

51

I mean if the clip is only 1 frame it will be sending a video of only 1 frame or it is ‘intelligent’ enough to send only a image?

52

Hi @Dario_Quesada

In your application, you want the screenshot of the first frame that triggered the event? We currently do not have such capability, but you can write a custom notifier class to do that. Also, the notifier is not intelligent enough to send only 1 image at this point.

53

@Dario_Quesada ,

we just released degirum_tools 0.22.3, which saves single .jpg image instead of video clip if you specify video clip duration equal to 1 frame.

1 Like
54

Uppss, updated to 0.22.3 and my script stopped working…

‘That is what happens when you mod the source code of the original packages’ :sweat_smile:

Edited the video_support.py to add RTMP support and started working again…

Aalso I was able to ‘see’ the changes you made in it to be able to export the jpg when clip_duration=1 :smiley: (didn’t know the clip saving optios was there until today)

55

@Dario_Quesada , RTMP support will be added in the next release. Actually, it is already implemented.

56

Nice , Thanks!

Do you know if it is possible to trigger another actions using the notifier?

Like making a sound, calling another function or something like that?

57

@Dario_Quesada , currently not - notifier only sends notification via Apprise - whatever Apprise supports. Since Apprise supports general webhooks, this is the way to invoke any web app.

Notifier is designed to not to slow down the main pipeline, therefore it does all the job in a worker process, asynchronously from the main process. This limits the argument types you can pass to a callback (if we would implement such functionality). Also that callback would be called in another process. May be this is not what you want.

But you can implement simple analyzer for calling your callback:

class CallbackCaller(degirum_tools.ResultAnalyzerBase):
    def __init__(self, callback: callable):
        super().__init__()
        self.callback = callback

    def analyze(self, result):
        if result.notifications:
            self.callback(result)
        return result

Then just add this analyzer directly after your notifier. It works this way: if there was any notification issued (result.notifications dict is not empty) then your callback is called.

58

@Dario_Quesada , we just released degirum_tools 0.22.4 with your RTMP support - no need to patch it locally :slight_smile:

59

Hi, thanks for your answers @Vlad_Klimov :smiley:

Very cool about the 0.22.4… Amazing! you’re all beasts, keep pushing forward :smiley:

Let me see if I understood your code

This callback is getting the result of the notification from apprise, so with this we are able to know if the notification was sent or if it wasn’t, isn’t?

So if I want to ‘register’ the succesful notification in the database I should do something like:

class CallbackCaller(degirum_tools.ResultAnalyzerBase):
    def __init__(self, callback: callable):
        super().__init__()
        self.callback = callback

    def analyze(self, result):
        if result.notifications:
            try:
                database.create_record(table="notifications", data={"notification_source":origin_stream, "notification_time":time.datetime(), "destination":destination_email })
            except:
                logger.error("Error writing to database") 
            self.callback(result)
        return result

I said ‘notifier’ because that was the component we were talking about but didn’t meant that I wanted to use the notifier for no reason or to modify it in any way, I think the apprise implementation is perfect for the notification system, it provides so muh flexibility :wink:

Maybe I should had choosen the word ‘eventDetector’ in my last question instead notifier, but in eventDetector’s documentation the only reference to something customizable I was able to find was a mention to a ‘CustomMetric’ deep burried in the documentation page, in the 'Event Definition Schema (YAML)…

But I’m pretty sure that ‘custommetric’ is not useful here as the ‘ObjectCount’ is the only ‘analyzer’ I think I need by the moment…

I thought that what I wanted was very simple and easy to achieve: to call something other function than the notifier when the EventDetector is triggered…

But looks like is not :frowning:

60

@Dario_Quesada ,

Notifier currently implements two important functions (besides notifications and clip saving):

  1. It detects the first time the event is triggered
  2. It supports hold off period.

Why #1 is important? Because event detector will continue generating events for every frame when condition is hold. This will cause your code to do something on each frame. Notifier implements logic which issues notification only on the first event in a sequence, like on the “rising edge” of event signal.

Why #2 is important? To suppress jitter of events, when event comes and goes in rapid manner.

This is similar to what oscilloscopes do with their triggers.