Dear Hailo Community,
I am trying to reproduce this example from Hugging Face using Qwen2-VL-2B-Instruct for zero-shot object detection.
I modified the simple_vlm_chat example from Hailo-apps and fed the API the same prompts and images as the HuggingFace example.
prompt = [
{
"role": "user",
"content": [
{"type": "image"},
{"type": "text", "text": 'You are a helpfull assistant to detect objects in images. When asked to detect elements based on a description you return bounding boxes for all elements in the form of [xmin, ymin, xmax, ymax] whith the values beeing scaled to 1000 by 1000 pixels. When there are more than one result, answer with a list of bounding boxes in the form of [[xmin, ymin, xmax, ymax], [xmin, ymin, xmax, ymax], ...].'},
{"type": "text", "text": 'detect goats'}
]
}
]
However, instead of the expected bounding boxes, I got the following when using Qwen2-VL-2B-Instruct.hef
[[-0. (0, 0), 0. (1, �)], [0. (0, 0), 0. (<input>), 0. (<input>)]]
I also tried
prompt = [
{
"role": "system",
"content": [{"type": "text", "text": 'You are a helpfull assistant to detect objects in images. When asked to detect elements based on a description you return bounding boxes for all elements in the form of [xmin, ymin, xmax, ymax] whith the values beeing scaled to 1000 by 1000 pixels. When there are more than one result, answer with a list of bounding boxes in the form of [[xmin, ymin, xmax, ymax], [xmin, ymin, xmax, ymax], ...].'}]
},
{
"role": "user",
"content": [
{"type": "image"},
{"type": "text", "text": 'detect goats'}
]
}
]
However, the response was still not even conforming to the requested format, regardless of the accuracy.
[[-100, 4: 100, ~], [1S, <: 100, 100, 100]]
I am wondering if I missed something, or perhaps this has something to do with the quantization process.
Thanks in advance!