I followed the guide here. https://www.raspberrypi.com/documentation/computers/ai.html#LLMs When I get to Step 2. 3 with llama3.2:3b the responses seem to start looping either individual repeating words or looping over the same group of words. I did some poking around and found the command hailortcli logs runtime so I directed the output to a text file and see lots of issues such as. below. Is this me or a bug?
Mar 9 15:44:13 hailo10 user.info HailoRT-Server: [pipeline.cpp:896] [print_deep_description] PushQEl4Llama-3_2-3B-Instruct__tbt/input_layer2 | inputs: PreInferEl4Llama-3_2-3B-Instruct__tbt/input_layer2[0] | outputs: AsyncHwEl(running in thread_id: 708)
Mar 9 15:44:18 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ‘The’
Mar 9 15:44:19 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ translation’
Mar 9 15:44:19 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ is’
Mar 9 15:44:20 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ‘: ’
Mar 9 15:44:20 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ‘Le’
Mar 9 15:44:20 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ chat’
Mar 9 15:44:21 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ est’
Mar 9 15:44:21 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ sur’
Mar 9 15:44:21 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ la’
Mar 9 15:44:22 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ table’
Mar 9 15:44:22 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ‘. ’
Mar 9 15:44:23 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ‘Note’
Mar 9 15:44:23 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ‘:’
Mar 9 15:44:23 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ The’
Mar 9 15:44:24 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ word’
Mar 9 15:44:24 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ order’
Mar 9 15:44:24 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ in’
Mar 9 15:44:25 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ French’
Mar 9 15:44:25 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ is’
Mar 9 15:44:26 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ similar’
Mar 9 15:44:26 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ to’
Mar 9 15:44:26 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ English’
Mar 9 15:44:27 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ‘,’
Mar 9 15:44:27 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ with’
Mar 9 15:44:27 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.err HailoRT-Server: [d2h_events_parser.cpp:428] [D2H_EVENTS__parse_nn_core_crc_error_notification] Got NN-Core CRC Error in CSM unit
Mar 9 15:44:28 hailo10 user.err HailoRT-Server: [d2h_events_parser.cpp:428] [D2H_EVENTS__parse_nn_core_crc_error_notification] Got NN-Core CRC Error in CSM unit
Mar 9 15:44:28 hailo10 user.err HailoRT-Server: [d2h_events_parser.cpp:428] [D2H_EVENTS__parse_nn_core_crc_error_notification] Got NN-Core CRC Error in CSM unit
Mar 9 15:44:28 hailo10 user.warn HailoRT-Server: [device_internal.cpp:572] [d2h_notification_thread_main] Aborting Infer, Device [integrated] got CSM CRC-error from NN-core
Mar 9 15:44:28 hailo10 user.warn HailoRT-Server: [device_internal.cpp:572] [d2h_notification_thread_main] Aborting Infer, Device [integrated] got CSM CRC-error from NN-core
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [network_group.cpp:768] [operator()] Infer request was aborted by user
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ a’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:28 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:38 hailo10 user.warn HailoRT-Server: [pipeline.cpp:441] [acquire_buffer] Failed to acquire buffer because the buffer pool is empty. This could be caused by uneven reading and writing speeds, with a short user-defined timeout. (timeout=10000ms)
Mar 9 15:44:38 hailo10 user.warn HailoRT-Server: [pipeline.cpp:441] [acquire_buffer] Failed to acquire buffer because the buffer pool is empty. This could be caused by uneven reading and writing speeds, with a short user-defined timeout. (timeout=10000ms)
Mar 9 15:44:38 hailo10 user.warn HailoRT-Server: [pipeline.cpp:441] [acquire_buffer] Failed to acquire buffer because the buffer pool is empty. This could be caused by uneven reading and writing speeds, with a short user-defined timeout. (timeout=10000ms)
Mar 9 15:44:38 hailo10 user.warn HailoRT-Server: [pipeline.cpp:441] [acquire_buffer] Failed to acquire buffer because the buffer pool is empty. This could be caused by uneven reading and writing speeds, with a short user-defined timeout. (timeout=10000ms)
Mar 9 15:44:38 hailo10 user.warn HailoRT-Server: [pipeline.cpp:441] [acquire_buffer] Failed to acquire buffer because the buffer pool is empty. This could be caused by uneven reading and writing speeds, with a short user-defined timeout. (timeout=10000ms)
Mar 9 15:44:38 hailo10 user.err HailoRT-Server: [pipeline.cpp:472] [get_available_buffer] CHECK_SUCCESS failed with status=HAILO_TIMEOUT(4) - Failed to acquire buffer with status=HAILO_TIMEOUT(4)
Mar 9 15:44:38 hailo10 user.err HailoRT-Server: [pipeline.cpp:472] [get_available_buffer] CHECK_SUCCESS failed with status=HAILO_TIMEOUT(4) - Failed to acquire buffer with status=HAILO_TIMEOUT(4)
Mar 9 15:44:38 hailo10 user.warn HailoRT-Server: [pipeline.cpp:441] [acquire_buffer] Failed to acquire buffer because the buffer pool is empty. This could be caused by uneven reading and writing speeds, with a short user-defined timeout. (timeout=10000ms)
Mar 9 15:44:38 hailo10 user.err HailoRT-Server: [pipeline.cpp:472] [get_available_buffer] CHECK_SUCCESS failed with status=HAILO_TIMEOUT(4) - Failed to acquire buffer with status=HAILO_TIMEOUT(4)
Mar 9 15:44:38 hailo10 user.err HailoRT-Server: [filter_elements.cpp:171] [action] CHECK failed - PreInferEl17Llama-3_2-3B-Instruct__tbt/input_layer5 (H2D) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
Mar 9 15:44:38 hailo10 user.err HailoRT-Server: [pipeline.cpp:472] [get_available_buffer] CHECK_SUCCESS failed with status=HAILO_TIMEOUT(4) - Failed to acquire buffer with status=HAILO_TIMEOUT(4)
Mar 9 15:44:38 hailo10 user.err HailoRT-Server: [pipeline.cpp:472] [get_available_buffer] CHECK_SUCCESS failed with status=HAILO_TIMEOUT(4) - Failed to acquire buffer with status=HAILO_TIMEOUT(4)
Mar 9 15:44:38 hailo10 user.err HailoRT-Server: [pipeline.cpp:472] [get_available_buffer] CHECK_SUCCESS failed with status=HAILO_TIMEOUT(4) - Failed to acquire buffer with status=HAILO_TIMEOUT(4)
Mar 9 15:44:38 hailo10 user.err HailoRT-Server: [filter_elements.cpp:171] [action] CHECK failed - PreInferEl19Llama-3_2-3B-Instruct__tbt/input_layer3 (H2D) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
Mar 9 15:44:38 hailo10 user.err HailoRT-Server: [filter_elements.cpp:171] [action] CHECK failed - PreInferEl16Llama-3_2-3B-Instruct__tbt/input_layer6 (H2D) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
Mar 9 15:44:38 hailo10 user.err HailoRT-Server: [filter_elements.cpp:171] [action] CHECK failed - PreInferEl1Llama-3_2-3B-Instruct__tbt/input_layer1 (H2D) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
Mar 9 15:44:38 hailo10 user.err HailoRT-Server: [filter_elements.cpp:171] [action] CHECK failed - PreInferEl4Llama-3_2-3B-Instruct__tbt/input_layer2 (H2D) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
Mar 9 15:44:38 hailo10 user.err HailoRT-Server: [filter_elements.cpp:171] [action] CHECK failed - PreInferEl18Llama-3_2-3B-Instruct__tbt/input_layer4 (H2D) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
Mar 9 15:44:38 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:38 hailo10 user.info HailoRT-Server: [llm_server.cpp:522] [handle_abort_request] Abort requested during generation
Mar 9 15:44:48 hailo10 user.warn HailoRT-Server: [pipeline.cpp:441] [acquire_buffer] Failed to acquire buffer because the buffer pool is empty. This could be caused by uneven reading and writing speeds, with a short user-defined timeout. (timeout=10000ms)
Mar 9 15:44:48 hailo10 user.warn HailoRT-Server: [pipeline.cpp:441] [acquire_buffer] Failed to acquire buffer because the buffer pool is empty. This could be caused by uneven reading and writing speeds, with a short user-defined timeout. (timeout=10000ms)
Mar 9 15:44:48 hailo10 user.warn HailoRT-Server: [pipeline.cpp:441] [acquire_buffer] Failed to acquire buffer because the buffer pool is empty. This could be caused by uneven reading and writing speeds, with a short user-defined timeout. (timeout=10000ms)
Mar 9 15:44:48 hailo10 user.warn HailoRT-Server: [pipeline.cpp:441] [acquire_buffer] Failed to acquire buffer because the buffer pool is empty. This could be caused by uneven reading and writing speeds, with a short user-defined timeout. (timeout=10000ms)
Mar 9 15:44:48 hailo10 user.err HailoRT-Server: [pipeline.cpp:472] [get_available_buffer] CHECK_SUCCESS failed with status=HAILO_TIMEOUT(4) - Failed to acquire buffer with status=HAILO_TIMEOUT(4)
Mar 9 15:44:48 hailo10 user.warn HailoRT-Server: [pipeline.cpp:441] [acquire_buffer] Failed to acquire buffer because the buffer pool is empty. This could be caused by uneven reading and writing speeds, with a short user-defined timeout. (timeout=10000ms)
Mar 9 15:44:48 hailo10 user.err HailoRT-Server: [pipeline.cpp:472] [get_available_buffer] CHECK_SUCCESS failed with status=HAILO_TIMEOUT(4) - Failed to acquire buffer with status=HAILO_TIMEOUT(4)
Mar 9 15:44:48 hailo10 user.err HailoRT-Server: [pipeline.cpp:472] [get_available_buffer] CHECK_SUCCESS failed with status=HAILO_TIMEOUT(4) - Failed to acquire buffer with status=HAILO_TIMEOUT(4)
Mar 9 15:44:48 hailo10 user.err HailoRT-Server: [pipeline.cpp:472] [get_available_buffer] CHECK_SUCCESS failed with status=HAILO_TIMEOUT(4) - Failed to acquire buffer with status=HAILO_TIMEOUT(4)
Mar 9 15:44:48 hailo10 user.err HailoRT-Server: [pipeline.cpp:472] [get_available_buffer] CHECK_SUCCESS failed with status=HAILO_TIMEOUT(4) - Failed to acquire buffer with status=HAILO_TIMEOUT(4)
Mar 9 15:44:48 hailo10 user.err HailoRT-Server: [filter_elements.cpp:171] [action] CHECK failed - PreInferEl1Llama-3_2-3B-Instruct__tbt/input_layer1 (H2D) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
Mar 9 15:44:48 hailo10 user.warn HailoRT-Server: [pipeline.cpp:441] [acquire_buffer] Failed to acquire buffer because the buffer pool is empty. This could be caused by uneven reading and writing speeds, with a short user-defined timeout. (timeout=10000ms)
Mar 9 15:44:48 hailo10 user.err HailoRT-Server: [filter_elements.cpp:171] [action] CHECK failed - PreInferEl4Llama-3_2-3B-Instruct__tbt/input_layer2 (H2D) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
Mar 9 15:44:48 hailo10 user.err HailoRT-Server: [filter_elements.cpp:171] [action] CHECK failed - PreInferEl17Llama-3_2-3B-Instruct__tbt/input_layer5 (H2D) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
Mar 9 15:44:48 hailo10 user.err HailoRT-Server: [filter_elements.cpp:171] [action] CHECK failed - PreInferEl19Llama-3_2-3B-Instruct__tbt/input_layer3 (H2D) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
Mar 9 15:44:48 hailo10 user.err HailoRT-Server: [filter_elements.cpp:171] [action] CHECK failed - PreInferEl16Llama-3_2-3B-Instruct__tbt/input_layer6 (H2D) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
Mar 9 15:44:48 hailo10 user.err HailoRT-Server: [pipeline.cpp:472] [get_available_buffer] CHECK_SUCCESS failed with status=HAILO_TIMEOUT(4) - Failed to acquire buffer with status=HAILO_TIMEOUT(4)
Mar 9 15:44:48 hailo10 user.err HailoRT-Server: [filter_elements.cpp:171] [action] CHECK failed - PreInferEl18Llama-3_2-3B-Instruct__tbt/input_layer4 (H2D) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
Mar 9 15:44:48 hailo10 user.info HailoRT-Server: [llm_server.cpp:898] [handle_generation_completion] Applied recovery sequence for ungraceful ending
Mar 9 15:44:48 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ’ the’
Mar 9 15:44:48 hailo10 user.info HailoRT-Server: [llm_server.cpp:728] [handle_next_token] Next token: ‘<|eot_id|>’
Mar 9 15:45:19 hailo10 user.info HailoRT-Server: [device.cpp:53] [Device] OS Version: Linux 5.15.325.15.32-yocto-standard-01576-g55eda3b378bf #1 SMP PREEMPT Wed Oct 29 18:22:56 UTC 2025 aarch64
Mar 9 15:45:19 hailo10 user.info HailoRT-Server: [control.cpp:102] [control__parse_identify_results] firmware_version is: 5.1.1
woody@raspberrypi:~ $ cat hailortlog.txt
Mar 9 15:44:02 hailo10 kern.info kernel: hailo_pci_ep_driver 222000.hailo_pci_ep_driver: Changed device state to D0
Mar 9 15:44:02 hailo10 user.info HailoRT-Server: [vdevice.cpp:227] [create] Creating vdevice with params: device_count: 1, scheduling_algorithm: ROUND_ROBIN, multi_process_service: false
Mar 9 15:44:02 hailo10 user.info HailoRT-Server: [device.cpp:53] [Device] OS Version: Linux 5.15.325.15.32-yocto-standard-01576-g55eda3b378bf #1 SMP PREEMPT Wed Oct 29 18:22:56 UTC 2025 aarch64
Mar 9 15:44:02 hailo10 user.info HailoRT-Server: [control.cpp:102] [control__parse_identify_results] firmware_version is: 5.1.1
Mar 9 15:44:02 hailo10 user.info HailoRT-Server: [vdevice.cpp:367] [create] VDevice Infos: [integrated]
Mar 9 15:44:02 hailo10 user.info HailoRT-Server: [device.cpp:53] [Device] OS Version: Linux 5.15.325.15.32-yocto-standard-01576-g55eda3b378bf #1 SMP PREEMPT Wed Oct 29 18:22:56 UTC 2025 aarch64
Mar 9 15:44:02 hailo10 user.info HailoRT-Server: [control.cpp:102] [control__parse_identify_results] firmware_version is: 5.1.1
Mar 9 15:44:02 hailo10 user.info HailoRT-Server: [utils.hpp:54] [handle_check_hef_exists_request] HEF file ‘/usr/share/hailo-ollama/models/blob/sha256_1129f5f8384e4e45c5890104dc4ec1aee77e800ce1484ddc3aa942399aada425’ does not exist on device
Mar 9 15:44:02 hailo10 user.info HailoRT-Server: [vdevice.cpp:227] [create] Creating vdevice with params: device_count: 1, scheduling_algorithm: ROUND_ROBIN, multi_process_service: false
Mar 9 15:44:07 hailo10 user.info HailoRT-Server: [llm_server.cpp:225] [handle_create_llm_request] hef buffer of size ‘3370416230’, lora: ‘’
Mar 9 15:44:08 hailo10 user.info HailoRT-Server: [internal_buffer_manager.cpp:77] [print_execution_results] Planned internal buffer memory: CMA=23601664 CMA-Desc=0 Pinned=0. memory to edge layer usage factor is 0.02598919
Mar 9 15:44:08 hailo10 user.info HailoRT-Server: [internal_buffer_manager.cpp:86] [print_execution_results] Default Internal buffer planner executed successfully