EMZETT.
Login

Emzett Wiki

Reinforcement learning

Properties1
tagsreinforcement, rl, belohnung, q-learning

1 min read

Reinforcement learning

In short: In reinforcement learning an agent tries actions in an environment, receives rewards or penalties and learns to maximise the sum of rewards.

In more detail: There are no ready-made correct answers, only feedback after acting. The agent must weigh trying things out (exploration) against exploiting what it knows (exploitation).

In detail

  • Terms: state, action, reward, policy (strategy), value function.
  • Q-learning: learns how good each action is in each state (Q-table). With neural networks: Deep Q-Network (DQN), played Atari.
  • Policy gradient: the strategy is learned directly as a network and improved using the reward (REINFORCE, PPO).
  • PPO (Proximal Policy Optimization): stable method with limited change per step, often used for RLHF.
  • Successes: AlphaGo/AlphaZero (board games), robotics, data-centre cooling, fusion plasma control.
  • Problems: many trials needed, formulating the reward correctly (reward hacking: the agent finds loopholes).
  • Language models: see RLHF and preference optimisation and Reasoning models and chain of thought.

See also: Kinds of learning: supervised, unsupervised, reinforcement, RLHF and preference optimisation

Backlinks