Video Recording if people detected

[yimingliu0216]

Hey,

I am playing around hailo example, just want to achieve following:

• List item

When people detected, start video recording.
When no people detected, stop video recording.

The detection example can easily do the detection part here.

I tried to add recording function in user_data

    def recording(self, has_people):
        if has_people : 
            if not self.is_recording:
                self.is_recording = True
                tz = datetime.timezone.utc
                ft = "%Y-%m-%dT%H:%M:%S"
                t = datetime.datetime.now(tz=tz).strftime(ft)
                file_name = f'People Detection - {t}.mp4'
                
                self.picam2 = Picamera2()
                self.picam2.start()
                video_output = FfmpegOutput(file_name)
                encoder = H264Encoder(10000000)
                print("Starting video recording.")
                self.picam2.start_recording(encoder,output=video_output)
            
        else:
            if self.is_recording:
                print(" Stop video recording")
                self.is_recording = False
                self.picam2.stop_recording()
                self.picam2.stop()

And I got following error

I tried to use the frame from frame object in line 80 with following

def recording(self, has_people, frame):
        if has_people : 
            if not self.is_recording:
                print(" Starting record")
                self.is_recording = True
                if self.video_out_stream:
                    self.video_out_stream.release()
                    self.video_out_stream = None
                tz = datetime.timezone.utc
                ft = "%Y-%m-%dT%H:%M:%S"
                t = datetime.datetime.now(tz=tz).strftime(ft)
                file_name = f'People Detection - {t}'
                fourcc = cv2.VideoWriter_fourcc(*'mp4v')
                fps = 30
                frame_size = (640, 480)
                # Create a VideoWriter object
                self.video_out_stream = cv2.VideoWriter(file_name+'.mp4', fourcc, fps, frame_size)

            print(" Recording ...")
            import pdb; pdb.set_trace()
            self.video_out_stream.write(frame)  # Write the frame to the video
        else:
            if self.is_recording:
                print(" Stop record")
                self.is_recording = False
                if self.video_out_stream:
                    self.video_out_stream.release()
                    self.video_out_stream = None

The mp4 file always empty.

Could you have any suggestion how I can record video if detection_count is greater than zero?

[omria]

Hey @yimingliu0216

Here’s a rewritten version of your message:

The IPAManager error you’re seeing is related to your Pi camera setup and is connected to the issue of recording video when people are detected. Let’s break down the problems and their solutions:

1. IPAManager Error (Multiple instances):
   This happens when the camera is initialized more than once in your pipeline. The Hailo example you’re using might be trying to access the camera for both object detection and video recording simultaneously.

2. Empty MP4 Files:
   Your script is recording video based on detection, but the files are empty. This could be because the camera feed isn’t being accessed correctly when detection and recording occur at the same time.

To solve these issues:

1. Avoid Multiple Camera Instances:
   Make sure you only initialize the camera once. Share this instance between detection and recording functions. Here’s an updated version of your recording function:

   def recording(self, has_people, frame):
       if has_people:
           if not self.is_recording:
               print("Starting recording")
               self.is_recording = True
               tz = datetime.timezone.utc
               t = datetime.datetime.now(tz=tz).strftime("%Y-%m-%dT%H:%M:%S")
               file_name = f'People Detection - {t}.mp4'
               fourcc = cv2.VideoWriter_fourcc(*'mp4v')
               fps = 30
               frame_size = (frame.shape[1], frame.shape[0])
               self.video_out_stream = cv2.VideoWriter(file_name, fourcc, fps, frame_size)
           print("Recording...")
           self.video_out_stream.write(frame)
       else:
           if self.is_recording:
               print("Stopping recording")
               self.is_recording = False
               if self.video_out_stream:
                   self.video_out_stream.release()
                   self.video_out_stream = None
   

2. Reset Camera Resources:
   Before running your script, restart the camera services:

   sudo systemctl restart libcamera-daemon
   

3. Integrate Recording in Detection Loop:
   Run the recording logic only when detection is positive. Use a shared self.picam2 object for both tasks:

   def process_frame(self, frame):
       detections = self.detect_people(frame)
       has_people = len(detections) > 0
       self.recording(has_people, frame)
   

4. Test and Debug:
   Run your modified code and check if the camera initializes once and if recording triggers correctly based on detection.

These changes should address both the camera initialization error and the empty MP4 files issue. Let me know if you need any further clarification or assistance!

[yimingliu0216]

Thanks @omria , really appreciated your comprehensive exlpanation.

The camera initialization error has been solved and can save the mp4 file as well.

However, the video image doesn’t correct

I added following code

user_data.recording(detection_count > 0, frame)

in line 81, (added your recording function in hailo_rip_common.py line 51 )

Do I grab the frame in wrong place?