Overview
We conceptualise Responsible AI (RAI) as the conscious effort in designing, developing, and deploying artificial intelligence (AI) systems to maximize their benefits while minimising their risks. As we witness advancements in vehicle autonomy with safety and trust concerns, the need for responsible AI indicators in autonomous driving becomes more pressing. However, the varying perceptions of key RAI principles in the autonomous vehicle (AV) community seem to create misaligned evaluation metrics, complicating the assessment and certification of autonomous driving (AD) agents. In addition to performance scores comparison, this CARLA RAI challenge provides an initial accessible simulation-based framework with robustness, environmental sustainability, and transparency indicators which facilitate AD agent assessment and comparisons under the RAI lenses.
CARLA RAI Autonomous Driving Challenge assesses the robustness, environmental sustainability (carbon emissions) of the submitted agents at inference. See instructions on how to get started
Task
The main goal of the challenge is to assess how autonomous agents comply with core responsible AI concepts such as robustness, environmental sustainability (i.e., carbon emissions at inference), and transparency/interpretability. Similar to the previous CARLA Leaderboard Challenge, this challenge subjects driving agents to rigorous tests in realistic traffic situations, as defined in CARLA Scenario Runner, but with a focus on RAI. In contrast to the previous CARLA Leaderboard challenge, only the SENSOR track is available in this challenge.
Participation
To participate in this challenge, go to the Participate tab and select one of your existing teams, or create a new team to join. The Leaderboard admins will verify your team as soon as possible. Make sure to include your user affiliation, as the admins will use this information for verification. Once your team is verified, you will be able to submit your entries. We also encourage all participants to join our Slack Channel for discussions related to the challenge. Feel free to also contact rai_challenge@robots.ox.ac.uk for any specific questions.
Timeline
| Actions | Dates |
|---|---|
| Submission Opens | September 2nd, 2024 |
| Submission Closes | November 30th, 2024 |
| Workshop for Result Presentations | TBC |