USING DEEP LEARNING FOR TRAFFIC LIGHT RECOGNITION

Challenge Overview:

We’ve all been there: a light turns green and the car in front of you doesn’t budge. No one likes to get stuck behind a vehicle that doesn’t notice when a light changes. Also, a system that can countdown on red light the time remaining until a change to green can save a significant quantities of fuel in city driving (e.g., restart engine five seconds before green) and advise driver to start braking early if it will not make it through a green light. That’s where you get into the picture: in this competition, you will develop a model to recognize traffic-light state in the car driving direction.

How do I get started:

1. Apply for challenge to join the challenge team and for downloading the dataset.
2. Join the challenge Slack team and join the challenge-one channel (invitation will be sent to your email account after applying to the challenge).
3. Begin building and testing your deep network using one of the popular deep-learning frameworks: Caffe, TensorFlow, Theano, Torch, or MXNet (optional: for quick setup of Amazon EC2 machine with Caffe, we recommend to use cs231n_caffe_torch7_keras_lasagne_v2, AMI ID: ami-125b2c72. image can be found on west-1 region).
4. Submit your results in the submission system with csv results file, trained model, code for training and testing the model (shared private GitHub repository)

Prizes:

• First Place : $5000
• Second Place: $2000
• Third Place: iPhone 7
• The top 5 participants will get an automatic invitation to the challenge final event to present their work. They will also be invited to interview and join Nexar’s Deep-Learning Team.
• All submissions will receive a Nexar car mount

Evaluation & Submission:

Nexar will provide the participants with two datasets, training and testing (dataset was splitted randomly to train and test).

The training-set consists of 18,659 labeled images for training and validating the CNN model (i.e., train and validation set) with the following labels:

• 0:= No traffic light in driving direction
• 1:= Red traffic light in driving direction
• 2:= Green traffic light in driving direction

The testing-set consists of 500,000 unlabeled images for testing the final solution. Test-set will be available to download by the submission server open date (see section Timeline).

The participants will build a CNN model using the training-set, use it to predict on the testing-set, and create a file with predicted labels for test-set. More specifically, for each image in the test set, participants will predict a label for its id (0=no traffic light in driving direction, 1=Red light in traffic direction, 2=Green light in traffic direction) and the probability of each class -> (image_fname, label_id,p0,p1,p2) .

Performance will be evaluated on classification accuracy (i.e., percentage of correctly labeled images in test-set) while taking into consideration the trained model size to prefer networks with lower model size.

Classification accuracy will be calculated as follows:

classification_accuracy = number of correctly labeled images/number of predictions

Minimum success criteria for submission acceptance: 0.95 (i.e., 95%) classification accuracy.

Then, normalized model size score will be calculated as follow:

model_size_score = exp(-model_size_mb/100)

Finally, challenge score will be calculated as follow:

challenge_score = classification_accuracy*model_size_score

For example, a straightforward fine-tuning of the GoogLeNet to the Nexar train-set using Caffe yielded test classification accuracy of 0.93 (i.e., 93%) with 41MB model size and challenge_score of 0.6172 .

Competition Rules:

• One account per participant
• No private sharing of code or data
• Code must run on Ubuntu platform and use one of the popular deep-learning frameworks (Caffe, TensorFlow, Theano, Torch, or MXNet) with python interface.
• A submission will be considered ineligible if it was developed using code containing or depending on software that is not approved by the Open Source Initiative, or a license that prohibits commercial use.
• Participants must submit a runnable code with shareable private GitHub repository, including documentation and resources/dependencies required to train and test the model, with reproducible results.
• No hand-labelling of test dataset allowed.

Timeline:

• Start Date: 02/11/2016
• Submission Server is Open (test-set available to download): 01/12/2016
• Submission Deadline: 31/12/2016

All deadlines are at 11:59 PM UTC on the corresponding day unless otherwise noted. Nexar challenge organizers reserve the right to update the contest timeline if they deem it necessary.