View article

Multi-Agent Reinforcement Learning (MARL) is a powerful Machine Learning paradigm, where multiple autonomous agents can learn to improve the performance of a system through experience. In this paper, we examine the application of MARL to a Dynamic Economic Emissions Dispatch (DEED) problem. This is a multi-objective problem domain, where the conflicting objectives of fuel cost and emissions must be minimised. Here we use the framework of Stochastic Games to reformulate this problem as a sequential decision making process, thus making it suitable for the application of MARL. We evaluate the performance of several different MARL credit assignment structures in this domain, including local rewards, global rewards, difference rewards and Counterfactual as Potential, along with two different objective scalarisation methods. We also introduce a new variant of the DEED problem, where a random generator fails during the simulation, with the goal of testing the robustness of the various MARL approaches. Difference rewards are found to offer the best performance of all the MARL credit assignment structures tested, learning Pareto optimal solutions that dominate those of the other MARL approaches. Our experimental results also show that MARL can produce comparable solutions to those published previously using the traditional DEED problem format, including those computed by Genetic Algorithms and Particle Swarm Optimisation.