Running Inference from within Docker Container

General
1

I’m trying to install and run inference inside of a container that resides on a Raspberry PI 5. I’m trying to install hailort_4.18.0_arm64.deb into the container, but it seems like sudo and systemd are needed, so I’m pausing and wanting to understand what I need to install on the host (I’m assuming at least the pcie-driver) and what can be installed in the container (hailort-4.18.0-cp311-cp311-linux_aarch64.whl).

Does hailort_4.18.0_arm64.deb need to be installed in the container or on the host? Is it even possible to do inference from a container?

2

That’s right, the PCI driver should be installed outside, and the libraries inside.

3

But it seems like hailort_4.18.0_arm64.deb needs systemd and even the Ubuntu base image doesn’t have it as I have:

FROM ubuntu:20.04

...
# Fake Sudo 
RUN echo '#!/bin/sh\nexec "$@"' > /usr/bin/sudo && chmod +x /usr/bin/sudo

COPY ./hailort/hailort_4.18.0_arm64.deb /setup/hailort_4.18.0_arm64.deb
RUN dpkg -i /setup/hailort_4.18.0_arm64.deb

Gives me:

2.647 Setting up hailort (4.18.0) ...
2.989 System has not been booted with systemd as init system (PID 1). Can't operate.
2.990 Failed to connect to bus: Host is down
3.016 Failed. Exited with status 1. See /var/log/hailort.deb.log
3.058 dpkg: error processing package hailort (--install):
3.058  installed hailort package post-installation script subprocess returned error exit status 1
3.113 Errors were encountered while processing:
3.113  hailort
4

You will need some basic build tools packages installed on top of the Ubutnu image, like build-essential

5

Is there an example or a comprehensive list? Just adding build-essential doesn’t add systemd.

6

This is a list that is used to build debug dockers. I believe that it complete, but might also contain packages that are not critical.

RUN apt-get update && apt-get install -y
sudo
git
bsdmainutils
build-essential
dkms
openssh-client
virtualenv
rsync
vim
nano
wget
build-essential
pkg-config
software-properties-common
pciutils
net-tools
gdb
feh
lshw
tmux
less
silversearcher-ag
ffmpeg
libjpeg-dev
gedit
ffmpeg
x11-utils
python3-dev
python3-pip
python3-setuptools
libgirepository1.0-dev
g+±12
python-gi-dev
pkg-config libcairo2-dev
libgstreamer1.0-dev
cmake
libgstreamer-plugins-base1.0-dev
libzmq3-dev rsync
libgstreamer-plugins-bad1.0-dev
gstreamer1.0-plugins-base
gstreamer1.0-plugins-good
gstreamer1.0-plugins-bad
gstreamer1.0-libav
gstreamer1.0-tools
gstreamer1.0-x libopencv-dev

7

Can you share the entire working Dockerfile for RPI5?

8

Yeah

FROM ubuntu:20.04

ENV DEBIAN_FRONTEND=noninteractive

# Set the working directory in the container
WORKDIR /app

RUN apt-get update

# Update the package list and install prerequisites
RUN apt-get update && apt-get install -y \
    software-properties-common \
    curl \
    wget \
    build-essential \
    libssl-dev \
    zlib1g-dev \
    libbz2-dev \
    libreadline-dev \
    libsqlite3-dev \
    libffi-dev \
    liblzma-dev \
    libncurses5-dev \
    libgdbm-dev \
    tk-dev \
    uuid-dev \
    libnss3-dev \
    vim

# Add the deadsnakes PPA for Python 3.11
RUN add-apt-repository ppa:deadsnakes/ppa

# Update the package list again and install Python 3.11
RUN apt-get update && apt-get install -y python3.11 python3.11-venv python3.11-dev python3.11-distutils

# Optionally, set Python 3.11 as the default Python version
RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 1

# Install pip for Python 3.11
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.11

# Set the default Python and pip versions for the container
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.11 1
RUN update-alternatives --install /usr/bin/pip pip /usr/local/bin/pip3.11 1

# Verify Python and pip versions
RUN python --version
RUN pip --version

RUN pip install numpy
RUN pip install zenlog
RUN pip install loguru
RUN pip install Pillow>=8.1.2

RUN echo '#!/bin/sh\nexec "$@"' > /usr/bin/sudo && chmod +x /usr/bin/sudo

RUN apt-get install -y build-essential cmake

RUN echo '#!/bin/bash\nexit 0' > /usr/bin/systemctl && chmod +x /usr/bin/systemctl

RUN mkdir -p /run/systemd && echo 'docker' > /run/systemd/container

RUN apt-get install -y git

RUN mkdir -p hailort
RUN git clone https://github.com/hailo-ai/hailort.git hailort/sources
RUN cd hailort/sources && git checkout v4.18.0

RUN cd hailort/sources && cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release -DHAILO_BUILD_EXAMPLES=1 && sudo cmake --build build --config release --target install

COPY ./hailort/hailort-4.18.0-cp311-cp311-linux_aarch64.whl /setup/hailort-4.18.0-cp311-cp311-linux_aarch64.whl
RUN pip install /setup/hailort-4.18.0-cp311-cp311-linux_aarch64.whl

COPY hailo-test.py .
COPY lib/ ./lib/
COPY ./test_data ./test_data

COPY ./models ./models

RUN pip install opencv-python
RUN pip install numpy

CMD exec python hailo-test.py

Granted inference isn’t working for me but I don’t think it’s cause of a dependency missing. Hope this helps.