Using Petri Nets as an Integrated Constraint Mechanism for Reinforcement Learning Tasks

The lack of trust in algorithms is usually an issue

when using Reinforcement Learning (RL) agents for control

in real-world domains such as production plants, autonomous

vehicles, or traffic-related infrastructure, partly due to the lack

of verifiability of the model itself. In such scenarios, Petri

nets (PNs) are often available for flowcharts or process steps,

as they are versatile and standardized. In order to facilitate

integration of RL models and as a step towards increasing AI

trustworthiness, we propose an approach that uses PNs with

three main advantages over typical RL approaches: Firstly, the

agent can now easily be modeled with a combined state including

both external environmental observations and agent-specific

state information from a given PN. Secondly, we can enforce

constraints for state-dependent actions through the inherent PN

model. And lastly, we can increase trustworthiness by verifying

PN properties through techniques such as model checking. We

test our approach on a typical four-way intersection traffic

light control setting and present our results, beating cycle-based

baselines.