Noticed the documentation keeps mentioning the preferred inference method is asynchronous but cant find an explanation for why. Is it due to deprecation or performance?
Hey @Jared_Bowling,
both actually - async can optimize our hardware usage, and also because async will be consistent across all our devices, while the sync version is being deprecated for future devices and versions.
1 Like