Comma Controls Challenge!

Machine learning models can drive cars, paint beautiful pictures and write passable rap. But they famously suck at doing low level controls. Your goal is to write a good controller. This repo contains a model that simulates the lateral movement of a car, given steering commands. The goal is to drive this "car" well for a given desired trajectory.

Geting Started

We'll be using a synthetic dataset based on the comma-steering-control dataset for this challenge. These are actual routes with actual car and road states.

# download necessary dataset (~0.6G)
bash ./download_dataset.sh

# Test this works
python tinyphysics.py --model_path ./models/tinyphysics.onnx --data_path ./data/00000.csv --debug --controller simple


# Batch Metrics on lots of routes
python tinyphysics.py --model_path ./models/tinyphysics.onnx --data_path ./data --num_segs 100 --controller simple

# Generate a report comparing two controllers
python eval.py --model_path ./models/tinyphysics.onnx --data_path ./data --num_segs 100 --test_controller simple --baseline_controller open

There's also a notebook at experiment.ipynb, to explore.

TinyPhysics

This is a "simulated car" that has been trained to mimic a very simple physics model (bicycle model) based simulator, given realistic driving noise. It is an autoregressive model similar to ML Controls Sim in architecture. It's inputs are the car velocity (v_ego), forward acceleration (a_ego), lateral acceleration due to road roll (road_lataccel), current car lateral acceleration (current_lataccel) and a steer input (steer_action) and predicts the resultant lateral acceleration fo the car.

Controllers

Your controller should implement an update function that returns the steer_action. This controller is then run in-loop, in the simulator to autoregressively predict the car's response.

Evaluation

Each rollout will result in 2 costs:

• lataccel_cost: ${\displaystyle \frac{\mathrm{\Sigma }(actua{l}_{l}a{t}_{a}ccel-targe{t}_{l}a{t}_{a}ccel{)}^2}{steps}}\ast 100$

• jerk_cost: ${\displaystyle \frac{\mathrm{\Sigma }((actua{l}_{l}a{t}_{a}cce{l}_t-actua{l}_{l}a{t}_{a}cce{l}_{t-1})/\mathrm{\Delta }t{)}^2}{steps-1}}\ast 100$

It is important to minimize both costs. total_cost: $(latacce{l}_{c}ost\ast 5)+jer{k}_{c}ost$

Submission

Run the following command, and send us a link to your fork of this repo, and the report.html this script generates.

python eval.py --model_path ./models/tinyphysics.onnx --data_path ./data --num_segs 5000 --test_controller <insert your controller name> --baseline_controller simple