Deep-Live-Cam [show&tell+looking for partners]

Community Projects
1

TLDR:
Super nice face swapping app.
I. Looking for pointers or partners for how to adapt it better for the AI kit so it will harness its abilities.
II. Quick guide for how to run it on your Pi. (link to discussion on the projectโ€™s repo)

Last week I have encountered this post showing a very cool realtime face swapping:

Link to project: GitHub - hacksider/Deep-Live-Cam: real time face swap and one-click video deepfake with only a single image

After succeeding running it on a Macbook, I have tried making it work on my RPI5+AI kit.
It took me the weekend, but I was eventually able to make it run, even if very laggy:

image
image1499ร—906 159 KB

This wonderful project is able to run on โ€œsimpleโ€ machines, not just heavily reinforced with GPU.

As far as I understand, it unfortunately not harnessing any of the Hailo 8L abilities at the current configuration,
are there members here that have experimented with such libraries and applications over the Pi5+8L kit?
Any similar projects that have done this that are a good reference?

1 Like
2

Link to โ€œhow to set the project to RPI5โ€

1 Like
3

Hi, I have tried your project, but I get error:

(dlc3103) ain@raspberrypi:~/Deep-Live-Cam $ python run.py 
Frame processor face_enhancer not found
Frame processor face_enhancer not found
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /home/ain/.insightface/models/buffalo_l/1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /home/ain/.insightface/models/buffalo_l/2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /home/ain/.insightface/models/buffalo_l/det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /home/ain/.insightface/models/buffalo_l/genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /home/ain/.insightface/models/buffalo_l/w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (640, 640)
Exception in Tkinter callback
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/tkinter/__init__.py", line 1921, in __call__
    return self.func(*args)
  File "/home/ain/dlc3103/lib/python3.10/site-packages/customtkinter/windows/widgets/ctk_button.py", line 554, in _clicked
    self._command()
  File "/home/ain/Deep-Live-Cam/modules/ui.py", line 114, in <lambda>
    live_button = ctk.CTkButton(root, text='Live', cursor='hand2', command=lambda: webcam_preview())
  File "/home/ain/Deep-Live-Cam/modules/ui.py", line 356, in webcam_preview
    temp_frame = frame_processor.process_frame(source_image, temp_frame)
  File "/home/ain/Deep-Live-Cam/modules/processors/frame/face_swapper.py", line 64, in process_frame
    temp_frame = swap_face(source_face, target_face, temp_frame)
  File "/home/ain/Deep-Live-Cam/modules/processors/frame/face_swapper.py", line 48, in swap_face
    return get_face_swapper().get(temp_frame, target_face, source_face, paste_back=True)
  File "/home/ain/Deep-Live-Cam/modules/processors/frame/face_swapper.py", line 43, in get_face_swapper
    FACE_SWAPPER = insightface.model_zoo.get_model(model_path, providers=modules.globals.execution_providers)
  File "/home/ain/dlc3103/lib/python3.10/site-packages/insightface/model_zoo/model_zoo.py", line 96, in get_model
    model = router.get_model(providers=providers, provider_options=provider_options)
  File "/home/ain/dlc3103/lib/python3.10/site-packages/insightface/model_zoo/model_zoo.py", line 40, in get_model
    session = PickableInferenceSession(self.onnx_file, **kwargs)
  File "/home/ain/dlc3103/lib/python3.10/site-packages/insightface/model_zoo/model_zoo.py", line 25, in __init__
    super().__init__(model_path, **kwargs)
  File "/home/ain/dlc3103/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 419, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/home/ain/dlc3103/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 452, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidProtobuf: [ONNXRuntimeError] : 7 : INVALID_PROTOBUF : Load model from /home/ain/Deep-Live-Cam/models/inswapper_128_fp16.onnx failed:Protobuf parsing failed.
4

Hi, Have you tried looking it up on the original projectโ€™s repo issues?
I think I saw it there.