Hey I want to build my own custom postprocessing .so

saurabh · October 18, 2024, 2:18pm

@omria
I have figured out the post process issue with yoolv8_pose on my trained model.
See the keypoints output type. those are different.

To confirm this issue. I build another hef with all 8bit. And this works.
In my case I need as much accurate and precise value I can get.
So can you tell me at what places I need to modify the code to handle this 16bit data. I applied modifications at a few places but no change.

Actual Issue with 16 Bits -
In c++ post processing, those keypoints values are converted to uint8_t type due to how it’s is implemented.
Everywhere tensors are considered as uint8_t. I believe that’s the issue.

github.com

hailo-ai/tappas/blob/daffd36ecab5110d47107255fd7ec4c779758f2e/core/hailo/libs/postprocesses/pose_estimation/yolov8pose_postprocess.cpp#L223


      
          int num_proposals = output_b.shape(0) * output_b.shape(1);
          auto output_boxes = xt::view(output_b, xt::all(), xt::all(), xt::all());
          xt::xarray<uint8_t> quantized_boxes = xt::reshape_view(output_boxes, {num_proposals, 4, regression_length + 1});
          
          auto shape = {quantized_boxes.shape(1), quantized_boxes.shape(2)};
          
          // Keypoints setup
          float32_t qp_scale_kpts = raw_keypoints[i]->vstream_info().quant_info.qp_scale;
          float32_t qp_zp_kpts = raw_keypoints[i]->vstream_info().quant_info.qp_zp;
          
          auto output_keypoints = common::get_xtensor(raw_keypoints[i]);        
          int num_proposals_keypoints = output_keypoints.shape(0) * output_keypoints.shape(1);
          auto output_keypoints_quantized = xt::view(output_keypoints, xt::all(), xt::all(), xt::all());
          xt::xarray<uint8_t> quantized_keypoints = xt::reshape_view(output_keypoints_quantized, {num_proposals_keypoints, 17, 3});
          
          auto keypoints_shape = {quantized_keypoints.shape(1), quantized_keypoints.shape(2)};
          
          // Bbox decoding
          for (uint j = 0; j < uint(num_proposals); j++) {
              confidence =    xt::row(scores, instance_index)(0);
              instance_index++;

the get_xtensor by default return uint8 type. This can be changed to

auto output_keypoints = common::get_xtensor_uint16(raw_keypoints[i]);

github.com

hailo-ai/tappas/blob/daffd36ecab5110d47107255fd7ec4c779758f2e/core/hailo/libs/postprocesses/common/tensors.hpp#L24-L37


      
          xt::xarray<uint8_t> get_xtensor(HailoTensorPtr &tensor)
          {
              // Adapt a HailoTensorPtr to an xarray (quantized)
              xt::xarray<uint8_t> xtensor = xt::adapt(tensor->data(), tensor->size(), xt::no_ownership(), tensor->shape());
              return xtensor;
          }
          
          xt::xarray<uint16_t> get_xtensor_uint16(HailoTensorPtr &tensor)
          {
              // Adapt a HailoTensorPtr to an xarray (quantized)
              uint16_t *data = (uint16_t *)(tensor->data());
              xt::xarray<uint16_t> xtensor = xt::adapt(data, tensor->size(), xt::no_ownership(), tensor->shape());
              return xtensor;
          }

But this is not working.

Can you tell me at what places I need to modify?

Topic		Replies	Views
How can I write alls script for yolov8n_pose General	2	66	September 30, 2024
Missing source code of tappas 3.28.2 & on customizing postproc General	2	98	July 10, 2024
Custom TAPPAS Python postprocessing, empty tensors General tappas , error	5	286	July 8, 2024
Extracting Keypoints from a Custom Dataset Model for Pose Estimation General dfc , hailort , raspberry-pi	1	98	August 18, 2024
Cannot find post processing stages General raspberry-pi	3	202	August 22, 2024

Hey I want to build my own custom postprocessing .so

Related topics