[Discussion] Experimenting with generating a synthetic dataset for gaze prediction, why do models struggle to learn on the data?

I've setup an Unreal Engine 5 project that has a metahuman look at random points on a virtual screen, it then captures a photo through a virtual webcam I've setup, and saves it. Below is a simple dataset of around 6.5K images (I say simple because I froze a lot of variables I previously had, here the only things that change image to image is the lighting, the character's position, the screen coordinate, and some slight facial movement I've hacked …

data dataset learn look machinelearning photo prediction project random setup simple struggle synthetic through unreal unreal engine unreal engine 5 virtual webcam