Saving Detection Output to a Video File with Hailo RPI5 example

General Discussion
1

Hi everyone,

I’m working on the Hailo RPI5 example: GitHub Link.

I’m using an .mp4 file as input, and when I run the detection, a window appears showing the detections on the video.

But I want to save the output with the detections applied, so I can watch it later without having to rerun the script.

I noticed that there is a FILE_SINK_PIPELINE in gstreamer_helper_pipelines, so I tried to use it.

In the detection_pipeline file, inside get_pipeline_string, I added these two lines:

file_sink_pipeline = FILE_SINK_PIPELINE(output_file = 'test_output.mkv')

f'{file_sink_pipeline}'

and i commented

#display_pipeline = DISPLAY_PIPELINE(video_sink=self.video_sink, sync=self.sync, show_fps=self.show_fps)

#f'{display_pipeline}'

so my function get_pipeline_string is now :

def get_pipeline_string(self):
        source_pipeline = SOURCE_PIPELINE(self.video_source, self.video_width, self.video_height)
        detection_pipeline = INFERENCE_PIPELINE(
            hef_path=self.hef_path,
            post_process_so=self.post_process_so,
            post_function_name=self.post_function_name,
            batch_size=self.batch_size,
            config_json=self.labels_json,
            additional_params=self.thresholds_str)
        detection_pipeline_wrapper = INFERENCE_PIPELINE_WRAPPER(detection_pipeline)
        tracker_pipeline = TRACKER_PIPELINE(class_id=1)
        user_callback_pipeline = USER_CALLBACK_PIPELINE()
        #display_pipeline = DISPLAY_PIPELINE(video_sink=self.video_sink, sync=self.sync, show_fps=self.show_fps)
        file_sink_pipeline = FILE_SINK_PIPELINE(output_file = 'test_output.mkv')

        pipeline_string = (
            f'{source_pipeline} ! '
            f'{detection_pipeline_wrapper} ! '
            f'{tracker_pipeline} ! '
            f'{user_callback_pipeline} ! '
            f'{file_sink_pipeline}'
            #f'{display_pipeline}'
        )
        print(pipeline_string)
        return pipeline_string

When I run detection.py, it seems to work because the frame count keeps increasing until the end, but then I get this error:

Frame count: 350

Frame count: 351

Frame count: 352

End-of-stream
Error rewinding the video

An output file is created, but it’s just the same as the input video, without any detections applied.

Also, in the FILE_SINK_PIPELINE function, there is a recommendation to run ffmpeg after recording to fix the file header:

ffmpeg -i output.mkv -c copy fixed_output.mkv

I’m not sure if I followed the correct process.

Has anyone managed to save the output with detections using detection.py? Or does anyone know how to fix this?

2

Hi Thomas! The FILE_SINK_PIPELINE does not support looping. To avoid the error I disabled looping in the gstreamer_app.py file by commenting out the looping function and ending the process at end of stream;

    def on_eos(self):
        self.shutdown()
        #if self.source_type == "file":
             # Seek to the start (position 0) in nanoseconds
            #success = self.pipeline.seek_simple(Gst.Format.TIME, Gst.SeekFlags.FLUSH, 0)
            #if success:
            #    print("Video rewound successfully. Restarting playback...")
            #else:
            #    print("Error rewinding the video.", file=sys.stderr)
        #else:
            #self.shutdown()
1 Like
3

Hi I was using so edited pipeline in this function def get_pipeline_string(self)

 def get_pipeline_string(self):
        source_pipeline = SOURCE_PIPELINE(
            video_source=self.video_source,
            video_width=self.video_width, 
            video_height=self.video_height,
            frame_rate=self.frame_rate, sync=self.sync
        ) 
        infer_pipeline = INFERENCE_PIPELINE( 
            hef_path=self.hef_path, 
            post_process_so=self.post_process_so, 
            post_function_name=self.post_process_function,
            batch_size=self.batch_size
        ) 
        infer_pipeline_wrapper = INFERENCE_PIPELINE_WRAPPER(infer_pipeline) 
        tracker_pipeline = TRACKER_PIPELINE(class_id=0)
        user_callback_pipeline = USER_CALLBACK_PIPELINE()

        if self.save_video:
            display_pipeline = (
                f'hailooverlay name=hailo_display_overlay ! '
                f'clockoverlay halignment=right valignment=top '
                f'time-format="%I:%M:%S %p" font-desc="Sans, 10" ! '
                f'videoconvert n-threads=2 qos=false ! '
                f'x264enc speed-preset=ultrafast tune=zerolatency bitrate=4000 ! '
                f'h264parse ! '
                f'matroskamux ! '
                f'filesink location=**/home/path/output.mkv** sync=false'
            )
        else:
            display_pipeline = DISPLAY_PIPELINE(
                video_sink=self.video_sink,
                sync=self.sync,
                show_fps=self.show_fps
            )

        pipeline_string = (
            f'{source_pipeline} ! '
            f'{infer_pipeline_wrapper} ! '
            f'{tracker_pipeline} ! '
            f'{user_callback_pipeline} ! '
            f'{display_pipeline}'
        )
        print(pipeline_string)
        return pipeline_string

For me it’s working and need add this line to init in same file:

 def __init__(self, app_callback, user_data, parser=None):
        # False = preview, 
        # True = save to file
        self.save_video = False

Maybe it’s help some other people