Cloning a car to mimic human driving

Mohan Karthik
Dec 23, 2016 · 11 min read

Data

Data source

My own data

Udacity data

Udacity data set: Angles over time
Udacity data set: Angle distribution

Data Augmentation

Randomly choosing between left, center or right image

img_choice = np.random.randint(3)if img_choice == 0:
img_path = os.path.join(PATH, df.left.iloc[idx].strip())
angle += OFF_CENTER_IMG
elif img_choice == 1:
img_path = os.path.join(PATH, df.center.iloc[idx].strip())
else:
img_path = os.path.join(PATH, df.right.iloc[idx].strip())
angle -= OFF_CENTER_IMG

Flipping the image

if np.random.randint(2) == 0:
img = np.fliplr(img)
new_angle = -new_angle

Changing brightness

temp = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)# Compute a random brightness value and apply to the image
brightness = BRIGHTNESS_RANGE + np.random.uniform()
temp[:, :, 2] = temp[:, :, 2] * brightness
# Convert back to RGB and return
return cv2.cvtColor(temp, cv2.COLOR_HSV2RGB)

Changing X and Y translation

# Compute X translation
x_translation = (TRANS_X_RANGE * np.random.uniform()) - (TRANS_X_RANGE / 2)
new_angle = angle + ((x_translation / TRANS_X_RANGE) * 2) * TRANS_ANGLE
# Randomly compute a Y translation
y_translation = (TRANS_Y_RANGE * np.random.uniform()) - (TRANS_Y_RANGE / 2)
# Form the translation matrix
translation_matrix = np.float32([[1, 0, x_translation], [0, 1, y_translation]])
# Translate the image
return cv2.warpAffine(img, translation_matrix, (img.shape[1], img.shape[0]))

Biasing towards non-0 value

# Choose left / right / center image and compute new angle
# Do translation and modify the angle again
# Define a random threshold for each image taken
threshold = np.random.uniform()
# If the newly augmented angle + the bias falls below the threshold
# then discard this angle / img combination and look again
if (abs(angle) + bias) < threshold:
return None, None

Data Preprocessing

# Remove the unwanted top scene and retain only the track
roi = img[60:140, :, :]
# Resize the image
resize = cv2.resize(roi, (IMG_ROWS, IMG_COLS), interpolation=cv2.INTER_AREA)
# Return the image sized as a 4D array
return np.resize(resize, (1, IMG_ROWS, IMG_COLS, IMG_CH))

Visualizing data after the augmentation

Model

Network Architecture
model.add(Lambda(lambda x: x/127.5 - .5,
input_shape=(IMG_ROWS, IMG_COLS, IMG_CH),
output_shape=(IMG_ROWS, IMG_COLS, IMG_CH)))
model.add(Convolution2D(3, 1, 1, border_mode='same', name='color_conv'))

Regularization

Training

bias = 1. / (num_runs + 1.)

Driving

Track 1

Track 1

Track 2

Track 2

Reflections

Future work

Acknowledgements

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade