Skip to content

Latest commit

 

History

History
52 lines (43 loc) · 2.77 KB

README.md

File metadata and controls

52 lines (43 loc) · 2.77 KB

CAVE Implementation

This is the codebase for Communication-Augmented Value Estimation. The implementation is in two domains: Predator-Prey and Traffic-Junction.

Requirements

  • OpenAI Gym
  • PyTorch 1.5 (CPU)
  • Predator-Prey and Traffic Junction Environments

Testing Environment Setup

  • Predator-Prey and Traffic Junction (from IC3Net) 
    cd envs/ic3net-envs
python setup.py develop

Training CAVE and Baselines (except MAGIC)

  • cd baselines
  • Run python run_baselines.py --help to check all the options.
  • Use --commnet to enable CommNet, --ic3net to enable IC3Net, --tarcomm and --ic3net to enable TarMAC-IC3Net, and --gacomm to enable GA-Comm.
  • Use --cave to enable CAVE with TarMAC or GA-Comm.
  • Each combination of method and environment has its own bash script, in /scripts/.
  • Most of these scripts were designed to be used in SLURM for HPC. For example, run TarMAC+CAVE in the Predator-Prey 10-agent scenario with sbatch train_pp_hard_tarmac_cave.sh [seed=1].
  • Most scripts should still work as standard bash scripts. For example, run TarMAC+CAVE in the Predator-Prey 10-agent scenario with sh train_pp_hard_tarmac_cave.sh [seed=1].
    • If using sh, we recommend piping the output to a LOG file. This can be done by adding | tee <log_filename>.log to the Python call.
  • A seed can be provided as an optional argument when calling a bash script. By default, the seed is 1.
  • Multiple runs with different seeds can be set up using sbatch train_multiple_experiments <experiment_script>.sh [num_runs].

Training MAGIC

  • Run python main.py --help to check all the options.
  • Use --first_graph_complete and --second_graph_complete to set the corresponding communication graph of the first round and second round to be complete (disable the sub-scheduler), respectively.
  • Use --comm_mask_zero to block the communication.
  • Example calls
    • Predator-Prey 5-agent scenario: sh train_pp_medium.sh
    • Predator-Prey 5-agent scenario: sbatch train_pp_medium_slurm.sh
    • Predator-Prey 5-agent scenario: sbatch train_pp_medium_slurm_cave.sh
    • Predator-Prey 10-agent scenario: sbatch train_pp_hard_slurm.sh

Visualization

  • Plot with multiple log files
    Use plot_script.py (log files in saved/): 
    python plot.py <path/to/file with all data for an environment> <figure filename> Reward
python plot.py <path/to/file with all data for an environment> <figure filename> Success
python plot.py <path/to/file with all data for an environment> <figure filename> Steps-Taken

References

The training framework is adapted from MAGIC

MAGIC's repository was, in turn, adapted from IC3Net