RLlib: Abstractions for Distributed Reinforcement Learning

We identify abstractions of algorithmic components that are common in successful reinforcement learning (RL) algorithms. We argue that these components form a convenient standard modularization of RL workflows, which encapsulates parallelism and resource requirements within short-running compute tasks, and provides interfaces for easy development of new algorithms and reuse of existing implementations. We offer RLlib, a library of scalable RL primitives built on Ray, as demonstrated by its state-of-the-art results for popular and cutting-edge RL algorithms.

Eric Liang*, Richard Liaw*, Robert Nishihara, Philipp Moritz, Roy Fox, Ken Goldberg, Joseph Gonzalez, Michael Jordan, and Ion Stoica, ICML 2018

Symposia and Workshops

Ray RLlib: A Composable and Scalable Reinforcement Learning Library

We argue for building composable RL components by encapsulating parallelism and resource requirements within individual components, which can be achieved by building on top of a flexible task-based programming model. We demonstrate this principle by building Ray RLlib on top of Ray and show that we can implement a wide range of state-of-the-art algorithms by composing and reusing a handful of standard components. This composability does not come at the cost of performance — in our experiments, RLlib matches or exceeds the performance of highly optimized reference implementations.

Eric Liang*, Richard Liaw*, Robert Nishihara, Philipp Moritz, Roy Fox, Joseph Gonzalez, Ken Goldberg, and Ion Stoica, DeepRL Symposium, NIPS 2017
* Equal contribution