Submitting an Agent

By submitting your agent to the CARLA RAI Autonomous Driving Leaderboard you are accepting the terms of use.

1. Submission steps

In order to create and submit your agent you should have a copy of the CARLA leaderboard-1.0, rai-leaderboard, and scenario_runner. If this is not the case, please visit the 'Get started section' first. Note that you would need to build your agent into a docker image, this is the image you would upload. See the previous section on how to run your agent with docker.

Please take note that any changes you make to the CARLA Leaderboard, RAI Leaderboard or Scenario Runner repositories will be overwritten when you submit your entry to the cloud.

1.1. Register a new user at Eval AI

Our partners at Eval AI have developed the user interface for the leaderboard. To make a submission you need to register a user on the website. Make sure to fill in your affiliation, otherwise you may not be accepted to participate in the challenge.

1.2 Create a new team

After registering a user at Eval AI create a new team by clicking in the Participant Teams section as shown below. All participants need to register a team in the CARLA RAI AD Challenge.

1.3. Apply to the CARLA RAI Autonomous Driving Challenge

Go to the All Challenges section and find the CARLA RAI Autonomous Driving Challenge, or use this link. Select Participate, then choose one of your participant teams to apply to the challenge. You will need to wait for your team to be verified by the CARLA RAI Challenge organisers. Once your team is verified, you can start to make submissions.

1.4. Make your submission

Once your team has been verified, you will be able to submit. In the Overview tab you will find general information about the CARLA RAI AD Challenge, go to the Submit section to make your submission. There you will find instructions on how to submit.

  1. First, create a Conda environment and install EvalAI, which is EvalAI's command-line interface package: 
    2. 
  conda create -n evalai-cli python=3.7
  conda activate evalai-cli
  pip install evalai~=1.3.18
  2. Next, you need to identify your user with your token using the following command
    3. evalai set_token <token>

    You can retrieve your token from the Submit section of the EvalAI website. You may copy and paste the command directly from the submission instructions:

    Please note that the token is specific to your user, each user will see a different token in this section of the website. Do not share tokens or there will be confusion over the submission author.

  3. If the Conda environment already exists, update the Conda environment path in rai/scripts/submit_evalai.sh and execute the script. This will activate your Conda environment and submit the agent. 
    4. 
  bash rai/scripts/submit_evalai.sh -t <image:tag>

  4. You also have to go to the challenge on Eval AI and complete a short questionnaire about your submission.

  5. Congratulations! You have now made a submission to the CARLA RAI Autonomous Driving challenge.