System freeze using Hailo8 PCIe card on a Beelink EQ14 N150 with Frigate container on Debian 13

General
1

I installed a Hailo8 PCIe card on a EQ14 N150. On it I have installed Debian 13 in which I compiled the drivers 4.21 and running Frigate container.

In the past days I found several times the system freezed and I had to power off to restart.

I see that everytime I start Frigate I have several thousand lines of warnings all similar to this:

------------[ cut here ]------------
WARNING: CPU: 3 PID: 3250 at include/linux/rwsem.h:80 find_vma+0x59/0x70
Modules linked in: tls nf_conntrack_netlink xt_nat xt_tcpudp veth xt_conntrack xt_MASQUERADE bridge xt_set ip_set nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_addrtype nft_compat nf_tables libcrc32c xfrm_user xfrm_algo overlay 8021q garp stp mrp llc snd_hda_codec_realtek snd_hda_codec_generic snd_sof_pci_intel_tgl snd_hda_scodec_component snd_sof_pci_intel_cnl snd_sof_intel_hda_generic soundwire_intel soundwire_generic_allocation soundwire_cadence intel_rapl_msr snd_sof_intel_hda_common intel_rapl_common x86_pkg_temp_thermal snd_soc_hdac_hda nls_ascii snd_sof_intel_hda_mlink nls_cp437 snd_sof_intel_hda vfat intel_powerclamp snd_hda_codec_hdmi fat coretemp snd_sof_pci snd_sof_xtensa_dsp snd_sof kvm_intel snd_sof_utils snd_soc_acpi_intel_match snd_soc_acpi i915 iwlmvm soundwire_bus kvm snd_soc_avs mac80211 snd_soc_hda_codec snd_hda_ext_core snd_soc_core irqbypass crct10dif_pclmul ghash_clmulni_intel snd_compress snd_pcm_dmaengine sha512_ssse3 libarc4 snd_hda_intel sha256_ssse3
snd_intel_dspcfg sha1_ssse3 snd_intel_sdw_acpi iwlwifi btusb snd_hda_codec drm_buddy drm_display_helper btrtl aesni_intel btintel snd_hda_core gf128mul btbcm crypto_simd btmtk cryptd snd_hwdep bluetooth cec snd_pcm rc_core hailo_pci(OE) cfg80211 rapl ttm evdev intel_cstate snd_timer wmi_bmof intel_uncore mei_me drm_kms_helper snd ee1004 pcspkr mei soundcore rfkill i2c_algo_bit igen6_edac intel_pmc_core intel_vsec pmt_telemetry pmt_class acpi_pad acpi_tad button drm configfs efi_pstore nfnetlink efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic spi_pxa2xx_platform dw_dmac dw_dmac_core spi_pxa2xx_core xhci_pci iTCO_wdt intel_pmc_bxt xhci_hcd iTCO_vendor_support nvme watchdog usbcore crc32_pclmul i2c_i801 crc32c_intel i2c_smbus nvme_core igc intel_lpss_pci intel_lpss nvme_auth idma64 usb_common fan video wmi pinctrl_alderlake
CPU: 3 UID: 0 PID: 3250 Comm: frigate.detecto Tainted: G W OE 6.12.48+deb13-amd64 #1 Debian 6.12.48-1
Tainted: [W]=WARN, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: AZW EQ/EQ, BIOS TWEQF003 03/31/2025
RIP: 0010:find_vma+0x59/0x70
Code: c7 40 48 c7 c2 ff ff ff ff 48 89 e6 e8 30 d8 8f 00 48 8b 54 24 08 65 48 2b 14 25 28 00 00 00 75 0d 48 83 c4 10 c3 cc cc cc cc <0f> 0b eb d0 e8 9e 42 97 00 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f
RSP: 0018:ffffb845079cb9b0 EFLAGS: 00010246
RAX: 0000000000000000 RBX: 00007f66c6b78000 RCX: 0000000000000000
RDX: 0000000000000080 RSI: 00007f66c6b78000 RDI: ffff9f7e0006d800
RBP: ffffb845079cba48 R08: 0000000000000080 R09: ffff9f7e0232bb80
R10: ffffb845079cb9c8 R11: 0000000000000000 R12: 0000000000000000
R13: ffff9f7e0232bb80 R14: 0000000000000000 R15: 0000000000000000
FS: 00007f66ae1ca6c0(0000) GS:ffff9f816fb80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f66b598c140 CR3: 00000001076d8002 CR4: 0000000000f72ef0
PKRU: 55555554
Call Trace:

hailo_vdma_buffer_map+0x90/0x500 [hailo_pci]
hailo_vdma_buffer_map_ioctl+0xc0/0x1b0 [hailo_pci]
hailo_pcie_fops_unlockedioctl+0x1ca/0x490 [hailo_pci]
__x64_sys_ioctl+0x91/0xd0
do_syscall_64+0x82/0x190
? __iommu_map+0x129/0x270
? arch_exit_to_user_mode_prepare.isra.0+0x16/0xa0
? syscall_exit_to_user_mode+0x37/0x1b0
? do_syscall_64+0x8e/0x190
? iommu_map_sg+0x10e/0x1c0
? iommu_dma_map_sg+0x28b/0x420
? __finalise_sg.isra.0+0xfb/0x1b0
? __dma_map_sg_attrs+0x117/0x190
? dma_map_sg_attrs+0xe/0x20
? hailo_vdma_buffer_map+0x304/0x500 [hailo_pci]
? hailo_vdma_buffer_map_ioctl+0x112/0x1b0 [hailo_pci]
? up+0x56/0xa0
? hailo_pcie_fops_unlockedioctl+0x12b/0x490 [hailo_pci]
? arch_exit_to_user_mode_prepare.isra.0+0x16/0xa0
? syscall_exit_to_user_mode+0x37/0x1b0
? do_syscall_64+0x8e/0x190
? __count_memcg_events+0x53/0xf0
? count_memcg_events.constprop.0+0x1a/0x30
? handle_mm_fault+0x1bb/0x2c0
? do_user_addr_fault+0x36c/0x620
? arch_exit_to_user_mode_prepare.isra.0+0x16/0xa0
entry_SYSCALL_64_after_hwframe+0x76/0x7e
RIP: 0033:0x7f66e565cd1b
Code: 00 48 89 44 24 18 31 c0 48 8d 44 24 60 c7 04 24 10 00 00 00 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 1c 48 8b 44 24 18 64 48 2b 04 25 28 00 00
RSP: 002b:00007f66ae1c6870 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f66ae1c6b70 RCX: 00007f66e565cd1b
RDX: 00007f66ae1c6920 RSI: 00000000c0287604 RDI: 0000000000000021
RBP: 0000000019a4b2b0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000001000 R11: 0000000000000246 R12: 0000000019a4b340
R13: 0000000019a4b368 R14: 00007f66c6b78000 R15: 0000000019a4b2b0

—[ end trace 0000000000000000 ]—

or this:

------------[ cut here ]------------
WARNING: CPU: 3 PID: 3250 at include/linux/rwsem.h:80 find_vma+0x59/0x70
Modules linked in: tls nf_conntrack_netlink xt_nat xt_tcpudp veth xt_conntrack xt_MASQUERADE bridge xt_set ip_set nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_addrtype nft_compat nf_tables libcrc32c xfrm_user xfrm_algo overlay 8021q garp stp mrp llc snd_hda_codec_realtek snd_hda_codec_generic snd_sof_pci_intel_tgl snd_hda_scodec_component snd_sof_pci_intel_cnl snd_sof_intel_hda_generic soundwire_intel soundwire_generic_allocation soundwire_cadence intel_rapl_msr snd_sof_intel_hda_common intel_rapl_common x86_pkg_temp_thermal snd_soc_hdac_hda nls_ascii snd_sof_intel_hda_mlink nls_cp437 snd_sof_intel_hda vfat intel_powerclamp snd_hda_codec_hdmi fat coretemp snd_sof_pci snd_sof_xtensa_dsp snd_sof kvm_intel snd_sof_utils snd_soc_acpi_intel_match snd_soc_acpi i915 iwlmvm soundwire_bus kvm snd_soc_avs mac80211 snd_soc_hda_codec snd_hda_ext_core snd_soc_core irqbypass crct10dif_pclmul ghash_clmulni_intel snd_compress snd_pcm_dmaengine sha512_ssse3 libarc4 snd_hda_intel sha256_ssse3
snd_intel_dspcfg sha1_ssse3 snd_intel_sdw_acpi iwlwifi btusb snd_hda_codec drm_buddy drm_display_helper btrtl aesni_intel btintel snd_hda_core gf128mul btbcm crypto_simd btmtk cryptd snd_hwdep bluetooth cec snd_pcm rc_core hailo_pci(OE) cfg80211 rapl ttm evdev intel_cstate snd_timer wmi_bmof intel_uncore mei_me drm_kms_helper snd ee1004 pcspkr mei soundcore rfkill i2c_algo_bit igen6_edac intel_pmc_core intel_vsec pmt_telemetry pmt_class acpi_pad acpi_tad button drm configfs efi_pstore nfnetlink efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic spi_pxa2xx_platform dw_dmac dw_dmac_core spi_pxa2xx_core xhci_pci iTCO_wdt intel_pmc_bxt xhci_hcd iTCO_vendor_support nvme watchdog usbcore crc32_pclmul i2c_i801 crc32c_intel i2c_smbus nvme_core igc intel_lpss_pci intel_lpss nvme_auth idma64 usb_common fan video wmi pinctrl_alderlake
CPU: 3 UID: 0 PID: 3250 Comm: frigate.detecto Tainted: G W OE 6.12.48+deb13-amd64 #1 Debian 6.12.48-1
Tainted: [W]=WARN, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: AZW EQ/EQ, BIOS TWEQF003 03/31/2025
RIP: 0010:find_vma+0x59/0x70
Code: c7 40 48 c7 c2 ff ff ff ff 48 89 e6 e8 30 d8 8f 00 48 8b 54 24 08 65 48 2b 14 25 28 00 00 00 75 0d 48 83 c4 10 c3 cc cc cc cc <0f> 0b eb d0 e8 9e 42 97 00 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f
RSP: 0018:ffffb845079cbd80 EFLAGS: 00010246
RAX: 0000000000000000 RBX: 00007f66a62b4000 RCX: 0000000000000000
RDX: 0000000000000080 RSI: 00007f66a62b4000 RDI: ffff9f7e0006d800
RBP: ffffb845079cbe18 R08: 0000000000000080 R09: ffff9f7e01f39a80
R10: ffffb845079cbd98 R11: 00000000ffffffff R12: 0000000000000000
R13: ffff9f7e01f39a80 R14: 0000000000000000 R15: 0000000000000000
FS: 00007f66ae1ca6c0(0000) GS:ffff9f816fb80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f66b59353e0 CR3: 00000001076d8002 CR4: 0000000000f72ef0
PKRU: 55555554
Call Trace:

hailo_vdma_buffer_map+0x90/0x500 [hailo_pci]
hailo_vdma_buffer_map_ioctl+0xc0/0x1b0 [hailo_pci]
hailo_pcie_fops_unlockedioctl+0x1ca/0x490 [hailo_pci]
__x64_sys_ioctl+0x91/0xd0
do_syscall_64+0x82/0x190
? do_syscall_64+0x8e/0x190
entry_SYSCALL_64_after_hwframe+0x76/0x7e
RIP: 0033:0x7f66e565cd1b
Code: 00 48 89 44 24 18 31 c0 48 8d 44 24 60 c7 04 24 10 00 00 00 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 1c 48 8b 44 24 18 64 48 2b 04 25 28 00 00
RSP: 002b:00007f66ae1c8510 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f66ae1c8810 RCX: 00007f66e565cd1b
RDX: 00007f66ae1c85c0 RSI: 00000000c0287604 RDI: 0000000000000021
Missed 8 kernel messages
snd_intel_dspcfg sha1_ssse3 snd_intel_sdw_acpi iwlwifi btusb snd_hda_codec drm_buddy drm_display_helper btrtl aesni_intel btintel snd_hda_core gf128mul btbcm crypto_simd btmtk cryptd snd_hwdep bluetooth cec snd_pcm rc_core hailo_pci(OE) cfg80211 rapl ttm evdev intel_cstate snd_timer wmi_bmof intel_uncore mei_me drm_kms_helper snd ee1004 pcspkr mei soundcore rfkill i2c_algo_bit igen6_edac intel_pmc_core intel_vsec pmt_telemetry pmt_class acpi_pad acpi_tad button drm configfs efi_pstore nfnetlink efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic spi_pxa2xx_platform dw_dmac dw_dmac_core spi_pxa2xx_core xhci_pci iTCO_wdt intel_pmc_bxt xhci_hcd iTCO_vendor_support nvme watchdog usbcore crc32_pclmul i2c_i801 crc32c_intel i2c_smbus nvme_core igc intel_lpss_pci intel_lpss nvme_auth idma64 usb_common fan video wmi pinctrl_alderlake
Missed 27 kernel messages
? hailo_vdma_buffer_map_ioctl+0x112/0x1b0 [hailo_pci]
Missed 20 kernel messages
snd_intel_dspcfg sha1_ssse3 snd_intel_sdw_acpi iwlwifi btusb snd_hda_codec drm_buddy drm_display_helper btrtl aesni_intel btintel snd_hda_core gf128mul btbcm crypto_simd btmtk cryptd snd_hwdep bluetooth cec snd_pcm rc_core hailo_pci(OE) cfg80211 rapl ttm evdev intel_cstate snd_timer wmi_bmof intel_uncore mei_me drm_kms_helper snd ee1004 pcspkr mei soundcore rfkill i2c_algo_bit igen6_edac intel_pmc_core intel_vsec pmt_telemetry pmt_class acpi_pad acpi_tad button drm configfs efi_pstore nfnetlink efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic spi_pxa2xx_platform dw_dmac dw_dmac_core spi_pxa2xx_core xhci_pci iTCO_wdt intel_pmc_bxt xhci_hcd iTCO_vendor_support nvme watchdog usbcore crc32_pclmul i2c_i801 crc32c_intel i2c_smbus nvme_core igc intel_lpss_pci intel_lpss nvme_auth idma64 usb_common fan video wmi pinctrl_alderlake
Missed 4 kernel messages
Code: c7 40 48 c7 c2 ff ff ff ff 48 89 e6 e8 30 d8 8f 00 48 8b 54 24 08 65 48 2b 14 25 28 00 00 00 75 0d 48 83 c4 10 c3 cc cc cc cc <0f> 0b eb d0 e8 9e 42 97 00 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f
Missed 20 kernel messages
? up+0x56/0xa0
Missed 12 kernel messages
R10: 0000000000000780 R11: 0000000000000246 R12: 0000000019a4b340
Missed 6 kernel messages
snd_intel_dspcfg sha1_ssse3 snd_intel_sdw_acpi iwlwifi btusb snd_hda_codec drm_buddy drm_display_helper btrtl aesni_intel btintel snd_hda_core gf128mul btbcm crypto_simd btmtk cryptd snd_hwdep bluetooth cec snd_pcm rc_core hailo_pci(OE) cfg80211 rapl ttm evdev intel_cstate snd_timer wmi_bmof intel_uncore mei_me drm_kms_helper snd ee1004 pcspkr mei soundcore rfkill i2c_algo_bit igen6_edac intel_pmc_core intel_vsec pmt_telemetry pmt_class acpi_pad acpi_tad button drm configfs efi_pstore nfnetlink efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic spi_pxa2xx_platform dw_dmac dw_dmac_core spi_pxa2xx_core xhci_pci iTCO_wdt intel_pmc_bxt xhci_hcd iTCO_vendor_support nvme watchdog usbcore crc32_pclmul i2c_i801 crc32c_intel i2c_smbus nvme_core igc intel_lpss_pci intel_lpss nvme_auth idma64 usb_common fan video wmi pinctrl_alderlake
Missed 5 kernel messages
RSP: 0018:ffffb845079cba20 EFLAGS: 00010246
Missed 47 kernel messages
snd_intel_dspcfg sha1_ssse3 snd_intel_sdw_acpi iwlwifi btusb snd_hda_codec drm_buddy drm_display_helper btrtl aesni_intel btintel snd_hda_core gf128mul btbcm crypto_simd btmtk cryptd snd_hwdep bluetooth cec snd_pcm rc_core hailo_pci(OE) cfg80211 rapl ttm evdev intel_cstate snd_timer wmi_bmof intel_uncore mei_me drm_kms_helper snd ee1004 pcspkr mei soundcore rfkill i2c_algo_bit igen6_edac intel_pmc_core intel_vsec pmt_telemetry pmt_class acpi_pad acpi_tad button drm configfs efi_pstore nfnetlink efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic spi_pxa2xx_platform dw_dmac dw_dmac_core spi_pxa2xx_core xhci_pci iTCO_wdt intel_pmc_bxt xhci_hcd iTCO_vendor_support nvme watchdog usbcore crc32_pclmul i2c_i801 crc32c_intel i2c_smbus nvme_core igc intel_lpss_pci intel_lpss nvme_auth idma64 usb_common fan video wmi pinctrl_alderlake
CPU: 3 UID: 0 PID: 3250 Comm: frigate.detecto Tainted: G W OE 6.12.48+deb13-amd64 #1 Debian 6.12.48-1
Tainted: [W]=WARN, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Missed 2 kernel messages
Code: c7 40 48 c7 c2 ff ff ff ff 48 89 e6 e8 30 d8 8f 00 48 8b 54 24 08 65 48 2b 14 25 28 00 00 00 75 0d 48 83 c4 10 c3 cc cc cc cc <0f> 0b eb d0 e8 9e 42 97 00 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f
Missed 29 kernel messages
RSP: 002b:00007f66ae1c8510 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Missed 17 kernel messages
RAX: 0000000000000000 RBX: 00007f66c6b5b000 RCX: 0000000000000000
Missed 19 kernel messages
RSP: 002b:00007f66ae1c8510 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Missed 10 kernel messages
snd_intel_dspcfg sha1_ssse3 snd_intel_sdw_acpi iwlwifi btusb snd_hda_codec drm_buddy drm_display_helper btrtl aesni_intel btintel snd_hda_core gf128mul btbcm crypto_simd btmtk cryptd snd_hwdep bluetooth cec snd_pcm rc_core hailo_pci(OE) cfg80211 rapl ttm evdev intel_cstate snd_timer wmi_bmof intel_uncore mei_me drm_kms_helper snd ee1004 pcspkr mei soundcore rfkill i2c_algo_bit igen6_edac intel_pmc_core intel_vsec pmt_telemetry pmt_class acpi_pad acpi_tad button drm configfs efi_pstore nfnetlink efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic spi_pxa2xx_platform dw_dmac dw_dmac_core spi_pxa2xx_core xhci_pci iTCO_wdt intel_pmc_bxt xhci_hcd iTCO_vendor_support nvme watchdog usbcore crc32_pclmul i2c_i801 crc32c_intel i2c_smbus nvme_core igc intel_lpss_pci intel_lpss nvme_auth idma64 usb_common fan video wmi pinctrl_alderlake
CPU: 3 UID: 0 PID: 3250 Comm: frigate.detecto Tainted: G W OE 6.12.48+deb13-amd64 #1 Debian 6.12.48-1
Missed 7 kernel messages
RBP: ffffb845079cbc98 R08: 0000000000000080 R09: ffff9f7e54887600
Missed 33 kernel messages
Modules linked in: tls nf_conntrack_netlink xt_nat xt_tcpudp veth xt_conntrack xt_MASQUERADE bridge xt_set ip_set nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_addrtype nft_compat nf_tables libcrc32c xfrm_user xfrm_algo overlay 8021q garp stp mrp llc snd_hda_codec_realtek snd_hda_codec_generic snd_sof_pci_intel_tgl snd_hda_scodec_component snd_sof_pci_intel_cnl snd_sof_intel_hda_generic soundwire_intel soundwire_generic_allocation soundwire_cadence intel_rapl_msr snd_sof_intel_hda_common intel_rapl_common x86_pkg_temp_thermal snd_soc_hdac_hda nls_ascii snd_sof_intel_hda_mlink nls_cp437 snd_sof_intel_hda vfat intel_powerclamp snd_hda_codec_hdmi fat coretemp snd_sof_pci snd_sof_xtensa_dsp snd_sof kvm_intel snd_sof_utils snd_soc_acpi_intel_match snd_soc_acpi i915 iwlmvm soundwire_bus kvm snd_soc_avs mac80211 snd_soc_hda_codec snd_hda_ext_core snd_soc_core irqbypass crct10dif_pclmul ghash_clmulni_intel snd_compress snd_pcm_dmaengine sha512_ssse3 libarc4 snd_hda_intel sha256_ssse3
Missed 2 kernel messages
Tainted: [W]=WARN, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Missed 11 kernel messages
CR2: 00007f66b59353e0 CR3: 00000001076d8002 CR4: 0000000000f72ef0
Missed 19 kernel messages
RAX: ffffffffffffffda RBX: 00007f66ae1c8810 RCX: 00007f66e565cd1b
Missed 9 kernel messages
snd_intel_dspcfg sha1_ssse3 snd_intel_sdw_acpi iwlwifi btusb snd_hda_codec drm_buddy drm_display_helper btrtl aesni_intel btintel snd_hda_core gf128mul btbcm crypto_simd btmtk cryptd snd_hwdep bluetooth cec snd_pcm rc_core hailo_pci(OE) cfg80211 rapl ttm evdev intel_cstate snd_timer wmi_bmof intel_uncore mei_me drm_kms_helper snd ee1004 pcspkr mei soundcore rfkill i2c_algo_bit igen6_edac intel_pmc_core intel_vsec pmt_telemetry pmt_class acpi_pad acpi_tad button drm configfs efi_pstore nfnetlink efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic spi_pxa2xx_platform dw_dmac dw_dmac_core spi_pxa2xx_core xhci_pci iTCO_wdt intel_pmc_bxt xhci_hcd iTCO_vendor_support nvme watchdog usbcore crc32_pclmul i2c_i801 crc32c_intel i2c_smbus nvme_core igc intel_lpss_pci intel_lpss nvme_auth idma64 usb_common fan video wmi pinctrl_alderlake
CPU: 3 UID: 0 PID: 3250 Comm: frigate.detecto Tainted: G W OE 6.12.48+deb13-amd64 #1 Debian 6.12.48-1
Missed 4 kernel messages
RSP: 0018:ffffb845079cbd70 EFLAGS: 00010246
Missed 17 kernel messages
? do_syscall_64+0x8e/0x190
Missed 14 kernel messages
snd_intel_dspcfg sha1_ssse3 snd_intel_sdw_acpi iwlwifi btusb snd_hda_codec drm_buddy drm_display_helper btrtl aesni_intel btintel snd_hda_core gf128mul btbcm crypto_simd btmtk cryptd snd_hwdep bluetooth cec snd_pcm rc_core hailo_pci(OE) cfg80211 rapl ttm evdev intel_cstate snd_timer wmi_bmof intel_uncore mei_me drm_kms_helper snd ee1004 pcspkr mei soundcore rfkill i2c_algo_bit igen6_edac intel_pmc_core intel_vsec pmt_telemetry pmt_class acpi_pad acpi_tad button drm configfs efi_pstore nfnetlink efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic spi_pxa2xx_platform dw_dmac dw_dmac_core spi_pxa2xx_core xhci_pci iTCO_wdt intel_pmc_bxt xhci_hcd iTCO_vendor_support nvme watchdog usbcore crc32_pclmul i2c_i801 crc32c_intel i2c_smbus nvme_core igc intel_lpss_pci intel_lpss nvme_auth idma64 usb_common fan video wmi pinctrl_alderlake
CPU: 3 UID: 0 PID: 3250 Comm: frigate.detecto Tainted: G W OE 6.12.48+deb13-amd64 #1 Debian 6.12.48-1
Missed 4 kernel messages
RSP: 0018:ffffb845079cba20 EFLAGS: 00010246
Missed 23 kernel messages
? hailo_vdma_buffer_map_ioctl+0x112/0x1b0 [hailo_pci]
Missed 20 kernel messages
Modules linked in: tls nf_conntrack_netlink xt_nat xt_tcpudp veth xt_conntrack xt_MASQUERADE bridge xt_set ip_set nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_addrtype nft_compat nf_tables libcrc32c xfrm_user xfrm_algo overlay 8021q garp stp mrp llc snd_hda_codec_realtek snd_hda_codec_generic snd_sof_pci_intel_tgl snd_hda_scodec_component snd_sof_pci_intel_cnl snd_sof_intel_hda_generic soundwire_intel soundwire_generic_allocation soundwire_cadence intel_rapl_msr snd_sof_intel_hda_common intel_rapl_common x86_pkg_temp_thermal snd_soc_hdac_hda nls_ascii snd_sof_intel_hda_mlink nls_cp437 snd_sof_intel_hda vfat intel_powerclamp snd_hda_codec_hdmi fat coretemp snd_sof_pci snd_sof_xtensa_dsp snd_sof kvm_intel snd_sof_utils snd_soc_acpi_intel_match snd_soc_acpi i915 iwlmvm soundwire_bus kvm snd_soc_avs mac80211 snd_soc_hda_codec snd_hda_ext_core snd_soc_core irqbypass crct10dif_pclmul ghash_clmulni_intel snd_compress snd_pcm_dmaengine sha512_ssse3 libarc4 snd_hda_intel sha256_ssse3
Missed 1 kernel messages
CPU: 3 UID: 0 PID: 3250 Comm: frigate.detecto Tainted: G W OE 6.12.48+deb13-amd64 #1 Debian 6.12.48-1
Missed 22 kernel messages
? iommu_dma_map_sg+0x28b/0x420
Missed 20 kernel messages
R13: 0000000019a4b368 R14: 00007f66c6b56000 R15: 0000000019a4b2b0
Missed 5 kernel messages
snd_intel_dspcfg sha1_ssse3 snd_intel_sdw_acpi iwlwifi btusb snd_hda_codec drm_buddy drm_display_helper btrtl aesni_intel btintel snd_hda_core gf128mul btbcm crypto_simd btmtk cryptd snd_hwdep bluetooth cec snd_pcm rc_core hailo_pci(OE) cfg80211 rapl ttm evdev intel_cstate snd_timer wmi_bmof intel_uncore mei_me drm_kms_helper snd ee1004 pcspkr mei soundcore rfkill i2c_algo_bit igen6_edac intel_pmc_core intel_vsec pmt_telemetry pmt_class acpi_pad acpi_tad button drm configfs efi_pstore nfnetlink efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic spi_pxa2xx_platform dw_dmac dw_dmac_core spi_pxa2xx_core xhci_pci iTCO_wdt intel_pmc_bxt xhci_hcd iTCO_vendor_support nvme watchdog usbcore crc32_pclmul i2c_i801 crc32c_intel i2c_smbus nvme_core igc intel_lpss_pci intel_lpss nvme_auth idma64 usb_common fan video wmi pinctrl_alderlake
Missed 3 kernel messages
RIP: 0010:find_vma+0x59/0x70
Missed 29 kernel messages
RSP: 002b:00007f66ae1c8510 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Missed 10 kernel messages
snd_intel_dspcfg sha1_ssse3 snd_intel_sdw_acpi iwlwifi btusb snd_hda_codec drm_buddy drm_display_helper btrtl aesni_intel btintel snd_hda_core gf128mul btbcm crypto_simd btmtk cryptd snd_hwdep bluetooth cec snd_pcm rc_core hailo_pci(OE) cfg80211 rapl ttm evdev intel_cstate snd_timer wmi_bmof intel_uncore mei_me drm_kms_helper snd ee1004 pcspkr mei soundcore rfkill i2c_algo_bit igen6_edac intel_pmc_core intel_vsec pmt_telemetry pmt_class acpi_pad acpi_tad button drm configfs efi_pstore nfnetlink efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic spi_pxa2xx_platform dw_dmac dw_dmac_core spi_pxa2xx_core xhci_pci iTCO_wdt intel_pmc_bxt xhci_hcd iTCO_vendor_support nvme watchdog usbcore crc32_pclmul i2c_i801 crc32c_intel i2c_smbus nvme_core igc intel_lpss_pci intel_lpss nvme_auth idma64 usb_common fan video wmi pinctrl_alderlake
CPU: 3 UID: 0 PID: 3250 Comm: frigate.detecto Tainted: G W OE 6.12.48+deb13-amd64 #1 Debian 6.12.48-1
Tainted: [W]=WARN, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Missed 3 kernel messages
RSP: 0018:ffffb845079cbb30 EFLAGS: 00010246
Missed 33 kernel messages
RSP: 002b:00007f66ae1c8510 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Missed 10 kernel messages
snd_intel_dspcfg sha1_ssse3 snd_intel_sdw_acpi iwlwifi btusb snd_hda_codec drm_buddy drm_display_helper btrtl aesni_intel btintel snd_hda_core gf128mul btbcm crypto_simd btmtk cryptd snd_hwdep bluetooth cec snd_pcm rc_core hailo_pci(OE) cfg80211 rapl ttm evdev intel_cstate snd_timer wmi_bmof intel_uncore mei_me drm_kms_helper snd ee1004 pcspkr mei soundcore rfkill i2c_algo_bit igen6_edac intel_pmc_core intel_vsec pmt_telemetry pmt_class acpi_pad acpi_tad button drm configfs efi_pstore nfnetlink efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic spi_pxa2xx_platform dw_dmac dw_dmac_core spi_pxa2xx_core xhci_pci iTCO_wdt intel_pmc_bxt xhci_hcd iTCO_vendor_support nvme watchdog usbcore crc32_pclmul i2c_i801 crc32c_intel i2c_smbus nvme_core igc intel_lpss_pci intel_lpss nvme_auth idma64 usb_common fan video wmi pinctrl_alderlake
CPU: 3 UID: 0 PID: 3250 Comm: frigate.detecto Tainted: G W OE 6.12.48+deb13-amd64 #1 Debian 6.12.48-1
Missed 2 kernel messages
RIP: 0010:find_vma+0x59/0x70
Missed 21 kernel messages
RSP: 002b:00007f66ae1c8510 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Missed 10 kernel messages
snd_intel_dspcfg sha1_ssse3 snd_intel_sdw_acpi iwlwifi btusb snd_hda_codec drm_buddy drm_display_helper btrtl aesni_intel btintel snd_hda_core gf128mul btbcm crypto_simd btmtk cryptd snd_hwdep bluetooth cec snd_pcm rc_core hailo_pci(OE) cfg80211 rapl ttm evdev intel_cstate snd_timer wmi_bmof intel_uncore mei_me drm_kms_helper snd ee1004 pcspkr mei soundcore rfkill i2c_algo_bit igen6_edac intel_pmc_core intel_vsec pmt_telemetry pmt_class acpi_pad acpi_tad button drm configfs efi_pstore nfnetlink efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic spi_pxa2xx_platform dw_dmac dw_dmac_core spi_pxa2xx_core xhci_pci iTCO_wdt intel_pmc_bxt xhci_hcd iTCO_vendor_support nvme watchdog usbcore crc32_pclmul i2c_i801 crc32c_intel i2c_smbus nvme_core igc intel_lpss_pci intel_lpss nvme_auth idma64 usb_common fan video wmi pinctrl_alderlake
CPU: 3 UID: 0 PID: 3250 Comm: frigate.detecto Tainted: G W OE 6.12.48+deb13-amd64 #1 Debian 6.12.48-1
Missed 4 kernel messages
RSP: 0018:ffffb845079cbc40 EFLAGS: 00010246
Missed 20 kernel messages
? syscall_exit_to_user_mode+0x37/0x1b0
Missed 15 kernel messages
snd_intel_dspcfg sha1_ssse3 snd_intel_sdw_acpi iwlwifi btusb snd_hda_codec drm_buddy drm_display_helper btrtl aesni_intel btintel snd_hda_core gf128mul btbcm crypto_simd btmtk cryptd snd_hwdep bluetooth cec snd_pcm rc_core hailo_pci(OE) cfg80211 rapl ttm evdev intel_cstate snd_timer wmi_bmof intel_uncore mei_me drm_kms_helper snd ee1004 pcspkr mei soundcore rfkill i2c_algo_bit igen6_edac intel_pmc_core intel_vsec pmt_telemetry pmt_class acpi_pad acpi_tad button drm configfs efi_pstore nfnetlink efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic spi_pxa2xx_platform dw_dmac dw_dmac_core spi_pxa2xx_core xhci_pci iTCO_wdt intel_pmc_bxt xhci_hcd iTCO_vendor_support nvme watchdog usbcore crc32_pclmul i2c_i801 crc32c_intel i2c_smbus nvme_core igc intel_lpss_pci intel_lpss nvme_auth idma64 usb_common fan video wmi pinctrl_alderlake
CPU: 3 UID: 0 PID: 3250 Comm: frigate.detecto Tainted: G W OE 6.12.48+deb13-amd64 #1 Debian 6.12.48-1
Missed 4 kernel messages
RSP: 0018:ffffb845079cbb70 EFLAGS: 00010246
Missed 30 kernel messages
RSP: 002b:00007f66ae1c8510 EFLAGS: 00000246 ORIG_RAX: 0000000000000010

and continue for several hundreds lines.

After one freze I found this at the end of logs:

list_del corruption. next->prev should be ffffe136c74b5dc8, but was f7ffe136c74b5dc8. (next=ffffe136c74b5d48)
------------[ cut here ]------------
kernel BUG at lib/list_debug.c:65!
BUG: unable to handle page fault for address: 000003fffffffff8
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page

Any idea of what the matter is?

2

Every few days the PC freeze and I shall power off manually. I did not found any other error message in the logs, but all the warnings seems related to the Hailo card and appears when Frigate starts to use it.

Anyone may suggest a method to debug?

3

I found that my problem was already reported in this post by @pat_ko with an older release of the driver and Frigate.

Are there any news about it? Is this problem solved in any release newer than 4.21?

4

Hey @Roberto_Franceschini,

I am currently working on updating frigate to 4.23 and checking that everything works well !
Hopefully this update will be in dev soon and the rpi will move to hailo version 4.23 very soon!

5

I’m testing on N150 with Debian and Ubuntu, the problem is with Frigate that use 4.21.

How can I test the version 4.23? I believe hailortcli may help me, but I’m not sure how to use it.

6

I tested the driver 4.23 with the command:

sudo hailortcli run yolov11m.hef

the problem persist. Without support from developers I believe I shall give up using Hailo.

7

Hey @Roberto_Franceschini,

Can you try running this command and let me know what happens?

hailortcli run2 set-net yolov11m.hef

Regarding Frigate - we’re running into a bit of a version mismatch situation. Frigate jumped to 4.21 for Home Assistant support, but we’re not on 4.20 or 4.23, so we’re working on getting everything migrated to 4.23. The catch is that for the 4.23 HEF files, you’ll need model zoo version 2.17.

I’m actually putting together a new guide on getting Frigate working with 4.23 and Hailo, and I’ll update this thread with a link once it’s ready.

In the meantime, if you want to try this yourself, here’s what you’ll need to do:

If you can run through these steps and send me the logs or any output you’re getting, I’ll be able to help troubleshoot and get Frigate up and running for you!

8

I’m a bit frustrated by receiving answer from AI that seems out of contest and asking me to waste my time with pointless tests.

Frigate is running, there is no problem with that. I also could test the driver 4.23 and the reality is that the kernel panic messages still appears, therefore I would expect some suggestion on how to avoid these messages or information that the developers are working on this issue in next versions, but nothing like this happens.

I’ll buy a Coral processor and give a try to it.

9

Hey @Roberto_Franceschini ,

Sorry for the inconvenience, but i assure you its a real person!

I provided the Frigate configuration for 4.23 because that’s what I thought you wanted, along with how to run a model!

I asked for the run2 command because the run command is being deprecated in the 4.23 version, and run2 is the new way to run it for now.

Regarding the kernel messages and logs, I’ve already shared these with our R&D team. If you’re still experiencing the same log issues or have something new in 4.23, I’d be happy to move it to our R&D team so we can fix it , i will also ask for a patch that can fix this!

10

Dear @omria , few things:

  1. Last time I used the command you suggested and I got the message:
    [HailoRT CLI] [warning] "hailortcli run2" is not optimized for single model usage. It is recommended to use "hailortcli run" command for a single model
    it looks weird you suggest me something that is not recommended.
  2. In my messages I wrote that the kernel panic messages appears also with 4.23, but you suggested to use it with Frigate, I cannot understand the reason.
  3. Changing the version 4.23 in the file of Frigate installation cannot help because Frigate developer does not upload the files for that version, therefore your suggestion will never work.

Failing 3 suggestions out of 3 is not usual for me to see, even considering you are trying to help I think you should be more careful of what you write.

Meanwhile I purchased and give a try installing the Hailo8 card on a EQi12 with Intel I1235U and found exactly the same problem, but now frigate fails to start due to “Failure in hailort driver ioctl”, I think Iĺl open another thread on this

11

The ioctl failure was due to a stupid mistake in my docker file, now solved. I’ll keep testing if the kernel messages are the cause of the freezing

12

Hey @Roberto_Franceschini,

Regarding the kernel messages: I’ve successfully reproduced these on my end and have opened a ticket with the team. We’re working on a fix that should be included in the next release. It would be helpful if you could let me know if these messages appear when using something other than Frigate. I’ll update you once this is resolved.

Regarding the command warning: hailortcli run2 is actually the correct approach for HailoRT 4.23 and version 5+. I understand the warning message is confusing – we maintained the run command for API compatibility, but the warning itself is actually a bug. I’ve escalated this to R&D to have it corrected.

Regarding the version 4.23 files: You’re correct that the Frigate developer doesn’t host those files, but this is easily addressed. Here are the direct links to the required files:

For Hailo-8:

x86_64/amd64:

aarch64/arm64:

(Replace {PY_TAG} with your Python version: cp310-cp310, cp311-cp311, cp312-cp312, or cp313-cp313)

13

As I wrote in my previous message the error apears as well using the commands hailortcli run and hailortcli run2 therefore has nothing to do with Frigate.