Cannot get correct out put for yolov5m from hailo_infer() API

General
1

Now I get succeed to compile yolov5m model to with/without nms, and get correct detection results when I use the sample code with API pair:
hailo_vstream_write_raw_buffer(…) and hailo_vstream_read_raw_buffer(…).

I also get correct detection with yolov5m with nms with API:
hailo_infer(…).

Now my problem is that I cannot get correct detection results with API hailo_infer(…) of yolov5m without nms. I can invoke the function correctly, and get 3 layers of output data, but the detection results is wrong.
I compared the 3 output streams between hailo_infer() and hailo_vstream_write_raw_buffer(…), hailo_vstream_read_raw_buffer(…) on same yolov5m.hef, they are different. I post the code here, can you please help me to figure it out?

hailo_status status = HAILO_UNINITIALIZED;
hailo_stream_raw_buffer_by_name_t input_buffer = { 0 };
hailo_stream_raw_buffer_by_name_t output_buffer[3];
_output_data.resize(3);

vector<vector<uint8_t>> output_data2(3);
hailo_stream_raw_buffer_by_name_t output_buffer2[3];

size_t frame_size = 0;
for (size_t i = 0; i < vstreams_infos_size; i++)
{
	if (HAILO_H2D_STREAM == vstreams_infos[i].direction)
	{
		memcpy(input_buffer.name, vstreams_infos[i].name, HAILO_MAX_STREAM_NAME_SIZE);
		status = hailo_get_vstream_frame_size(&(vstreams_infos[i]), &(vstreams_infos[i].format), &frame_size);

		//status = hailo_get_input_vstream_frame_size(input_vstreams[i], &input_frame_sizes[i]);

		if (status != HAILO_SUCCESS)
		{
			return status;
		}
		input_buffer.raw_buffer.size = frame_size * INFER_FRAME_COUNT;
		input_buffer.raw_buffer.buffer = _im.data;
	}
	else
	{
		size_t output_size;
		memcpy(output_buffer[3-i].name, vstreams_infos[i].name, HAILO_MAX_STREAM_NAME_SIZE);
		status = hailo_get_vstream_frame_size(&(vstreams_infos[i]), &(vstreams_infos[i].format), &output_size);
		if (status != HAILO_SUCCESS)
		{
			return status;
		}
		_output_data[3 - i].resize(output_size * INFER_FRAME_COUNT);
		output_buffer[3 - i].raw_buffer.size = (output_size * INFER_FRAME_COUNT);
		output_buffer[3 - i].raw_buffer.buffer = _output_data[3 - i].data();
	}
}
size_t frames_count = INFER_FRAME_COUNT;
status = hailo_infer(network_group,
	input_vstream_params, &input_buffer, 1,
	output_vstream_params, output_buffer, 3,
	frames_count);
3

Hi @dliu,
When you don’t apply the Hailo NMS on your model, it means that in order to get the corresponding results you need to perform by yourself the bbox decoding, bbox extraction and NMS.
Did you apply all of the above and still got different results compared to the HEF with the Hailo NMS?

Regards,

4

It is right that the results of the three output streams are different.

In yolov5, each output stream means a feature value with a different scale.

In other words, as Omer says, it is necessary to gather each output value and decode it.

1 Like
5

Hi Omer,

Yes, I do decode the 3 streams with different size. Actually, I can decode and extract the boxes correctly if I use the API hailo_vstream_write_raw_buffer() and hailo_vstream_read_raw_buffer() on same yolov5m.hef without nms. But When I use hailo_infere(…) API, I cannot get correct output streams. I use exactly same input stream, and then compare the 3 output streams from hailo_vstream-read_raw_buffer() and hailo_infere(…), and found they are different. But they are supposed to be same as I use exactly same yolov5m.hef without nms.

Do I miss anything?

6

Hi @dliu,
The outputs should be the same as the hailo_infer is basically a wrapper for multi inputs\outputs inference.
Can you please explain what exactly is the difference?

Regards,

7

Hi Omer,

I’m sorry that made a mistake to phase the output data. Now they are some.

Thank you very much.

1 Like