<a href="https://youtu.be/m2pC9md0ixY" target="_blank">
Sign in to like and favorite skills
This repository implements a Advantage Actor-Critic agent baseline for the pysc2 environment as described in the DeepMind paper StarCraft II: A New Challenge for Reinforcement Learning. We use a synchronous variant of A3C (A2C) to effectively train on GPUs and otherwise stay as close as possible to the agent described in the paper.
This repository is part of a research project at the Autonomous Systems Labs , TU Darmstadt by Daniel Palenicek, Marcel Hussing, and Simon Meister.
This project is licensed under the MIT License (refer to the LICENSE file for details).
On the mini games, we get the following results:
| Map | best mean score (ours) | best mean score (DeepMind) | episodes (ours) |
|---|---|---|---|
| MoveToBeacon | 26 | 26 | 8K |
| CollectMineralShards | 97 | 103 | 300K |
| FindAndDefeatZerglings | 45 | 45 | 450K |
| DefeatRoaches | 65 | 100 | - |
| DefeatZerglingsAndBanelings | 68 | 62 | - |
| CollectMineralsAndGas | - | 3978 | - |
| BuildMarines | - | 3 | - |
In the following we show plots for the score over episodes.
Note that the DeepMind mean scores are their best individual scores after 100 runs for each game, where the initial learning rate was randomly sampled for each run. We use a constant initial learning rate for a much smaller number of runs due to limited hardware. All agents use the same FullyConv agent.
With default settings (32 environments), learning MoveToBeacon well takes between 3K and 8K total episodes. This varies each run depending on random initialization and action sampling.
pip install numpy tensorflow-gpu pysc2==1.2StarcraftII/Maps/python run.py my_experiment --map MoveToBeaconpython run.py my_experiment --map MoveToBeacon --evalYou can visualize the agents during training or evaluation with the
--visrun.pySummaries are written to
out/summary/<experiment_name>out/models/<experiment_name>The code in
rl/environment.pyrl/agents/a2c/runner.py