Question regarding model inference consistency

[Adblu]

Hi. When you look at the inference predictions on provided image, when face/eye direction barely changes, inference predictions vary a lot sometimes. I was wondering if I can improve it a bit.

I was wondering about kalman filter but I will lose some fps on it. Are there some changes on architectural level I can do ? Or maybe something else ?

Thanks !