Filter out people in pose estimation?

I want to perform pose estimation on table tennis matches for a project. I wish to perform the pose estimation on the two players only. Currently any algorithm perform pose estimation on all the humans in video.
Is there a way to filter out everyone other than the players?

I don't want to have an approach which involves in cutting the video and then perform the algorithm.
I also wish to not have an approach which involves creating a dataset …

algorithm computervision filter humans people project table tennis video