Model-free Intelligent Control using Reinforcement Learning and Temporal Abstraction-applied to pH Control

This article presents a solution to pH control based on model-free intelligent control (MFIC) using reinforcement learning. This control technique is proposed because the algorithm gives a general solution for acid-base system, yet simple enough for its implementation in existing control hardware. In standard reinforcement learning, the interaction between an agent and the environment is based on a fixed time scale: during learning, the agent can select several primitive actions depending on the system state. A novel solution is presented, using multi-step actions (MSA): actions on multiple time scales consist of several identical primitive actions. This solves the problem of determining a suitable fixed time scale to select control actions so as to trade off accuracy in control against learning complexity. The application of multi-step actions on a simulated pH process shows that the proposed MFIC learns to control adequately the neutralization process. Copyright © 2005 IFAC