Drones

This environment is forked from the gym-pybullet-drones, which is a gym environment with pybullet physics for reinforcement learning of multi-agent quadcopter control. It supports both single and multiple drones control. According to the official repository, it provides the following five kinds of action types:

• rpm: rounds per minutes (RPMs);

• pid: PID control;

• vel: Velocity input (using PID control);

• one_d_rpm: 1D (identical input to all motors) with RPMs;

• one_d_pid: 1D (identical input to all motors) with PID control.

You also have permission to customize the scenarios and tasks in this environment for your needs.

Official Address: https://github.com/utiasDSL/gym-pybullet-drones.git

Paper: https://arxiv.org/pdf/2103.02142.pdf

Installation

小技巧

Before preparing the software packages for this simulator, it is recommended to create a new conda environment with Python 3.10.

Open terminal and type the following commands, then a new conda environment for xuance with drones could be built:

conda create -n xuance_drones python=3.10
conda activate xuance_drones
pip install xuance  # refer to the installation of XuanCe.

git clone https://github.com/utiasDSL/gym-pybullet-drones.git
cd gym-pybullet-drones/
pip install --upgrade pip
pip install -e .  # if needed, `sudo apt install build-essential` to install `gcc` and build `pybullet`

During the installation of gym-pybullet-drones, you might encounter the errors like:

错误

gym-pybullet-drones 2.0.0 requires numpy<2.0,>1.24, but you have numpy 1.22.4 which is incompatible.

gym-pybullet-drones 2.0.0 requires scipy<2.0,>1.10, but you have scipy 1.7.3 which is incompatible.

Solution: Upgrade the above incompatible packages.

pip install numpy==1.24.0
pip install scipy==1.12.0

Try An Example

Create a python file named, e.g., “demo_drones.py”.

import argparse
from xuance import get_runner

def parse_args():
    parser = argparse.ArgumentParser("Run a demo.")
    parser.add_argument("--algo", type=str, default="iddpg")
    parser.add_argument("--env", type=str, default="drones")
    parser.add_argument("--env-id", type=str, default="MultiHoverAviary")
    parser.add_argument("--test", type=int, default=0)
    parser.add_argument("--device", type=str, default="cuda:0")
    parser.add_argument("--parallels", type=int, default=10)
    parser.add_argument("--benchmark", type=int, default=1)
    parser.add_argument("--test-episode", type=int, default=5)

    return parser.parse_args()

if __name__ == '__main__':
    parser = parse_args()
    runner = get_runner(method=parser.algo,
                        env=parser.env,
                        env_id=parser.env_id,
                        parser_args=parser,
                        is_test=parser.test)
    if parser.benchmark:
        runner.benchmark()
    else:
        runner.run()

Open the terminal and type the python command:

python demo_drones.py

Then, you can brew a cup of coffee, and wait for the training process to finish.

Finally, test the trained model and view the effectiveness.

python demo_drones.py --benchmark 0 --test 1

APIs

gym-pybullet-drones GitHub: https://github.com/utiasDSL/gym-pybullet-drones.git Note: The version of Python should be >= 3.10.

class xuance.environment.multi_agent_env.drones.Drones_MultiAgentEnv(*args, **kwargs)

基类：RawMultiAgentEnv

agent_mask()

Returns boolean mask variables indicating which agents are currently alive.

avail_actions()

Returns a boolean mask indicating which actions are available for each agent.

close()

Closes the environment.

render(*args, **kwargs)

Renders the environment.

返回:

The images used to visualize the environment.

返回类型:

rgb_images (np.ndarray or list)

reset()

Resets the environment with kwargs.

返回:

The initial observations of the agent. info (MultiAgentDict): The information about the environment.

返回类型:

observation (MultiAgentDict)

space_reshape(gym_space)

state()

Returns the global state of the environment.

step(actions)

Steps through the environment with action.

参数:

action_dict (MultiAgentDict) – A dict that contains all agents’ actions.

返回:

The next step observations after executing actions. reward (MultiAgentDict): The rewards returned by the environment. terminated(MultiAgentDict): A dict of bool values that indicates if the environment should be terminated. truncated(bool): A bool value that indicates if the environment should be truncated. info (MultiAgentDict): The information about the environment.

返回类型:

observation (MultiAgentDict)

class xuance.environment.multi_agent_env.drones.MultiHoverAviary(drone_model: None = None, num_drones: int = 2, neighbourhood_radius: float = inf, initial_xyzs=None, initial_rpys=None, physics: None = None, pyb_freq: int = 240, ctrl_freq: int = 30, gui=False, record=False, obs: None = None, act: None = None)

基类：object

Multi-agent RL problem: leader-follower.