Does Hailo 8 support BF16 format? Has anyone tried converting Llama 3.2 to .HAR and HEF formats? Need support @system
The Hailo-8 can perform model inference much more efficiently than a typical GPU. One key reason is that it uses integer computation rather than floating-point, which significantly reduces power consumption.
The Hailo-8 supports 4-bit, 8-bit (default) and 16-bit integer.
We have announced a new generation AI accelerator called Hailo-10H that will allow running LLM.