If the environment exists random, can the problem be modeled as MDP? __mdp

Li Hongyi Teacher's course: Https://www.youtube.com/watch?v=W8XF3ME8G2I

Teacher said, for the same observation/state (Atari game screen), also not necessarily will take the same action, because some actor is stochastic, select action has a certain randomness, this good understanding ...

The teacher also said that even if actor take the same action, the reward and next state are not necessarily the same, because game itself has some randomness. I noticed this very early on , that if the game itself exists randomness, the transfer between the game and State,state to the state is also Markov in nature. If not, the application of RL these methods is not very far-fetched. or the RL is just a randomness on the basis of a good average policy.