Can AI improve how call centers operate? The paper “Optimising Call Centre Operations using Reinforcement Learning: Value Iteration versus Proximal Policy Optimisation” by Kwong Ho Li and Wathsala Karunarathne shows that it can — and with strong results. The authors compare two reinforcement learning (RL) approaches to optimize call routing: the classical Value Iteration (VI) and the modern Proximal Policy Optimisation (PPO).

What is Reinforcement Learning?

Reinforcement Learning is an AI method where an agent takes actions in an environment and receives rewards based on how good those actions are. The goal is to maximize the cumulative reward — essentially, to learn the best decisions.

In a call center, the agent is the system routing incoming calls to staff members, and the environment is the entire service operation. The goal is to minimize wait time and maximize employee utilization.

Compared Approaches

Value Iteration (VI) is a classic model-based method. It assumes full knowledge of the system dynamics (like arrival probabilities) and uses this to calculate the best policy analytically.

Proximal Policy Optimisation (PPO) is a modern model-free method. It learns the best policy by trial and error, without knowing the full environment.

Simulation Environment

The authors built a call center simulation using Discrete Event Simulation and OpenAI Gym. The model included:

  • multiple customer and agent types (skills-based routing),
  • Poisson-arrival processes for customers,
  • Exponential distributions for handling and abandonment times.

Results

After a series of simulations, PPO delivered the best performance:

  • shortest customer waiting time,
  • least employee idle time,
  • highest RL reward.

Although VI performed decently, PPO adapted better to unknown conditions and proved more robust.

Conclusion

Modern RL techniques like PPO offer real benefits for optimizing call center operations. This is a solid example of how AI isn’t just theory — it can bring measurable improvements to real-world business workflows.