Which processes are required when only inference is performed?

General
1

Sorry for the basic question.

When actually performing inference at the edge, I would like to know how much RAM should be allocated to the edge device.
When running the Inference Tutorial in the tutorial environment (hailo tutorial), the following three processes are running on the container.

  • python
  • hailort_service
  • hailo

The hailo process (consuming approximately 500 MB) has existed since the tutorial environment was launched. Is it okay to ignore this process when only performing inference, or are all three processes necessary?

2

Hey @Hit_ASA,

That’s actually a good question! The answer really depends on what example you want to run and which programming language you prefer.

If you want to use the Python API, then you’ll need both Python and the HailoRT service. However, if you want to run C++, you don’t need Python at all.

Here are the main example repositories we have available:

For more specific help, please let me know how you want to use it - are you looking to work with GStreamer-based solutions or direct API calls?

3

Thanks for your reply, @omria.

We will probably use the C++ inference API directly (of course, no Python process is required; instead, a C++ application process will be launched).

If the HailoRT service is required, does that mean we don’t need to consider the hailo process in the tutorial environment?

4

Hey @Hit_ASA ,

When using the C++ InferModel APIs, just make sure hailort.service is running (if you need multi-process mode). You can safely ignore the Python-based hailo process from the tutorial environment.

1 Like