Q-learning is a model-free reinforcement learning algorithm used to find the optimal action-selection policy for a given finite Markov decision process. It uses a Q-table where each entry corresponds to a state-action pair, and the value indicates the expected future rewards of taking that action from that state. The algorithm updates the Q-values iteratively using the Bellman equation: $𝑄(𝑠,𝑎)\leftarrow 𝑄(𝑠,𝑎)+𝛼(𝑟+𝛾{\max }_{{𝑎}^{\prime }}𝑄({𝑠}^{\prime },{𝑎}^{\prime })-𝑄(𝑠,𝑎\left)\right)$Q(s,a)←Q(s,a)+α(r+γmaxa′​Q(s′,a′)−Q(s,a)) where $𝑠$ is the current state, $𝑎$ is the action taken, $𝑟$ is the reward received, ${𝑠}^{\prime }$ is the next state, $𝛼$ is the learning rate, and $𝛾$ is the discount factor.

The SARSA (State-Action-Reward-State-Action) algorithm is also a model-free reinforcement learning method but follows an on-policy approach. It updates the Q-values based on the action actually taken in the next state: $𝑄(𝑠,𝑎)\leftarrow 𝑄(𝑠,𝑎)+𝛼(𝑟+𝛾𝑄({𝑠}^{\prime },{𝑎}^{\prime })-𝑄(𝑠,𝑎\left)\right)$Q(s,a)←Q(s,a)+α(r+γQ(s′,a′)−Q(s,a)) where $𝑠$s is the current state, $𝑎$ is the current action, $𝑟$ is the reward, ${𝑠}^{\prime }$ is the next state, and ${𝑎}^{\prime }$ is the next action chosen according to the current policy. SARSA emphasizes learning the action-value function based on the policy being followed, incorporating both exploration and exploitation during learning.