本篇是自動駕駛系列的第二篇，在后臺留言索取代碼會提供源碼鏈接潜腻。這次的目標(biāo)是編寫一個軟件流水線來識別汽車前置攝像頭的視頻中的車道邊界。攝像機(jī)標(biāo)定圖像，試驗(yàn)路圖像和視頻項(xiàng)目都可以在這里儲存币旧。

這次試驗(yàn)的目標(biāo)/步驟如下：

計(jì)算相機(jī)校準(zhǔn)矩陣和給定一組棋盤圖像的失真系數(shù)。

對原始圖像應(yīng)用畸變校正猿妈。

使用顏色變換吹菱，漸變等創(chuàng)建閾值二值圖像。

應(yīng)用透視變換來糾正二值圖像（“鳥瞰”）彭则。

檢測車道像素鳍刷，找到車道邊界。

確定車道和車輛相對于中心的曲率俯抖。

將檢測到的車道邊界轉(zhuǎn)回到原始圖像上输瓜。

輸出車道邊界的視覺顯示和車道曲率和車輛位置的數(shù)值估計(jì)。

相機(jī)校準(zhǔn)矩陣和失真系數(shù)

當(dāng)照相機(jī)查看真實(shí)世界中的3D對象并將其轉(zhuǎn)換為2D圖像時，會發(fā)生圖像失真; 這個轉(zhuǎn)變并不完美尤揣。失真實(shí)際上改變了這些3D對象的形狀和大小搔啊。因此，分析相機(jī)圖像的第一步是消除這種失真北戏，以便從中獲得正確和有用的信息坯癣。

真實(shí)的相機(jī)使用彎曲的鏡頭來形成圖像，而光線在這些鏡頭的邊緣往往會彎曲得太多或太少最欠。這會產(chǎn)生扭曲圖像邊緣的效果示罗，使線條或物體看起來或多或少比實(shí)際彎曲。這被稱為徑向失真芝硬，這是最常見的失真類型蚜点。

另一種失真是切向失真。當(dāng)相機(jī)的鏡頭沒有完全平行于相機(jī)膠片或傳感器所在的成像平面時拌阴，會發(fā)生這種情況绍绘。這使圖像看起來傾斜，使一些物體看起來比實(shí)際距離或距離更近迟赃。

有三個系數(shù)需要校正徑向失真：k1陪拘，k2和k3，以及2對于切向失真：p1纤壁，p2左刽。在這個項(xiàng)目中，使用OpenCV和具有9×6角的棋盤面板來執(zhí)行相機(jī)校準(zhǔn)酌媒。

import os, glob, pickle

import numpy as np

import cv2

import matplotlib.pyplot as plt

classCameraCalibrator:

'''Compute the camera calibration matrix and distortion coefficients given a set of chessboard images.'''

def__init__(self, image_directory, image_filename, binary_filename, nx, ny):

print ('Initializing CameraCalibrator ...')

self.__image_directory = image_directory

self.__image_filename = image_filename

self.__binary_filename = binary_filename

self.__nx = nx# the number of inside corners in x

self.__ny = ny# the number of inside corners in y

self.mtx = None

self.dist = None

self.rvecs = None

self.tvecs = None

self.__calibrated = False

def__calibrate(self):

# Read in and make a list of calibration images

calibration_filenames = glob.glob(self.__image_directory+'/'+self.__image_filename)

# Arrays to store object points and image points from all the images

object_points = []# 3D points in real world space

image_points = []# 2D points in image plane

# Prepare object points, like (0,0,0), (1,0,0), (2,0,0), ...,(7,5,0)

object_p = np.zeros((self.__ny*self.__nx,3),np.float32)

object_p[:,:2] = np.mgrid[0:self.__nx,0:self.__ny].T.reshape(-1,2)# s,y coordinates

# Extract the shape of any image

image = cv2.imread(calibration_filenames[1])

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

shape = gray.shape[::-1]# (width,height)

# Process each calibration image

forimage_filenameincalibration_filenames:

# Read in each image

image = cv2.imread(image_filename)

image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)# RGB is standard in matlibplot

# Convert to grayscale

gray = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)

# Find the chessboard corners

ret, corners = cv2.findChessboardCorners(gray, (self.__nx,self.__ny), None)

# If found, draw corners

ifret ==True:

# Store the corners found in the current image

object_points.append(object_p)# how it should look like

image_points.append(corners)# how it looks like

# Draw and display the corners

cv2.drawChessboardCorners(image, (self.__nx,self.__ny), corners, ret)

plt.figure()

plt.imshow(image)

plt.show()

# Do the calibration

ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(object_points, image_points, shape, None, None)

#print(ret, mtx, dist, rvecs, tvecs)

# Pickle to save time for subsequent runs

binary = {}

binary["mtx"] = mtx

binary["dist"] = dist

binary["rvecs"] = rvecs

binary["tvecs"] = tvecs

pickle.dump(binary, open(self.__image_directory +'/'+self.__binary_filename,"wb"))

self.mtx = mtx

self.dist = dist

self.rvecs = rvecs

self.tvecs = tvecs

self.__calibrated = True

def__load_binary(self):

'''Load previously computed calibration binary data'''

with open(self.__image_directory +'/'+self.__binary_filename, mode='rb') asf:

binary = pickle.load(f)

self.mtx = binary['mtx']

self.dist = binary['dist']

self.rvecs = binary['rvecs']

self.tvecs = binary['tvecs']

self.__calibrated = True

defget_data(self):

'''Getter for the calibration data. At the first call it gerenates it.'''

ifos.path.isfile(self.__image_directory +'/'+self.__binary_filename):

self.__load_binary()

else:

self.__calibrate()

returnself.mtx,self.dist,self.rvecs,self.tvecs

defundistort(self, image):

ifself.__calibrated ==False:

self.get_data()

returncv2.undistort(image,self.mtx,self.dist, None,self.mtx)

deftest_undistort(self, image_filename, plot=False):

'''A method to test the undistort and to plot its result.'''

image = cv2.imread(image_filename)

image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)# RGB is standard in matlibplot

image_undist =self.undistort(image)

# Ploting both images Original and Undistorted

f, (ax1, ax2) = plt.subplots(1,2, figsize=(20,10))

ax1.set_title('Original/Distorted')

ax1.imshow(image)

ax2.set_title('Undistorted')

ax2.imshow(image_undist)

plt.show()

開始準(zhǔn)備“對象點(diǎn)”欠痴，這將是世界上棋盤角的（x，y秒咨，z）坐標(biāo)喇辽。在這里，我假設(shè)棋盤固定在z = 0處的（x雨席，y）平面上菩咨，使得每個校準(zhǔn)圖像的目標(biāo)點(diǎn)是相同的。因此陡厘，objp只是一個復(fù)制的坐標(biāo)數(shù)組抽米，每當(dāng)我成功檢測到測試圖像中的所有棋盤角時，objpoints都會附加一個副本雏亚。每個成功的棋盤檢測將會在圖像平面中的每個角落附加（x缨硝，y）像素位置。

然后罢低，我使用輸出對象和imgpoint來使用OpenCV cv2.calibrateCamera（）函數(shù)來計(jì)算相機(jī)校準(zhǔn)和失真系數(shù)查辩。我使用cv2.undistort（）函數(shù)將此畸變校正應(yīng)用于測試圖像胖笛，并獲得了以下結(jié)果：

該步驟的代碼包含在文件“./camera_calibration.py”中。將這一步應(yīng)用于一個示例圖像宜岛，你會得到這樣的結(jié)果：

使用顏色變換长踊，漸變等創(chuàng)建閾值二值圖像

使用顏色和漸變閾值的組合來生成二進(jìn)制圖像，方法compute_binary_image（）可以在lane_detection.py中找到萍倡。有各種顏色和梯度閾值的組合來生成車道線清晰可見的二值圖像身弊。

defcompute_binary_image(self, color_image, plot=False):

# Convert to HLS color space and separate the S channel

#Note:img is the undistorted image

hls = cv2.cvtColor(color_image, cv2.COLOR_RGB2HLS)

s_channel = hls[:,:,2]

# Grayscale image

#NOTE:we already saw that standard grayscaling lost color information for the lane lines

# Explore gradients in other colors spaces / color channels to see what might work better

gray = cv2.cvtColor(color_image, cv2.COLOR_RGB2GRAY)

# Sobel x

sobelx = cv2.Sobel(gray, cv2.CV_64F,1,0)# Take the derivative in x

abs_sobelx = np.absolute(sobelx)# Absolute x derivative to accentuate lines away from horizontal

scaled_sobel = np.uint8(255*abs_sobelx/np.max(abs_sobelx))

# Threshold x gradient

thresh_min =20

thresh_max =100

sxbinary = np.zeros_like(scaled_sobel)

sxbinary[(scaled_sobel >= thresh_min) & (scaled_sobel <= thresh_max)] =1

# Threshold color channel

s_thresh_min =170

s_thresh_max =255

s_binary = np.zeros_like(s_channel)

s_binary[(s_channel >= s_thresh_min) & (s_channel <= s_thresh_max)] =1

# Combine the two binary thresholds

combined_binary = np.zeros_like(sxbinary)

combined_binary[(s_binary ==1) | (sxbinary ==1)] =1

if(plot):

# Ploting both images Original and Binary

f, (ax1, ax2) = plt.subplots(1,2, figsize=(20,10))

ax1.set_title('Undistorted/Color')

ax1.imshow(color_image)

ax2.set_title('Binary/Combined S channel and gradient thresholds')

ax2.imshow(combined_binary, cmap='gray')

plt.show()

returncombined_binary

這是我為這一步輸出的一個例子。

應(yīng)用透視變換來糾正二值圖像（“鳥瞰”）

接下來列敲，我們要為透視變換確定四個源點(diǎn)阱佛。在這種情況下，我們可以假設(shè)道路是平面的戴而。這不是嚴(yán)格的凑术，但它可以作為這個項(xiàng)目的近似值。我們希望從上面俯瞰道路時所意，選取四個梯形（類似于區(qū)域遮擋）的點(diǎn)淮逊，這個梯形代表一個矩形。

要做到這一點(diǎn)扶踊，最簡單的方法是調(diào)查車道線是直線的圖像泄鹏，并找到沿線的四個點(diǎn)，在透視變換之后秧耗，從鳥瞰視角使線看起來筆直且垂直备籽。

我的透視變換的代碼包括2個函數(shù)調(diào)用compute_perspective_transform()和apply_perspective_transform()，這出現(xiàn)在文件中l(wèi)ane_detection.py绣版。所述compute_perspective_transform()構(gòu)建的變換矩陣M通過使用apply_perspective_transform()變換的二進(jìn)制圖像胶台。

defcompute_perspective_transform(self, binary_image):

# Define 4 source and 4 destination points = np.float32([[,],[,],[,],[,]])

shape = binary_image.shape[::-1]# (width,height)

w = shape[0]

h = shape[1]

transform_src = np.float32([ [580,450], [160,h], [1150,h], [740,450]])

transform_dst = np.float32([ [0,0], [0,h], [w,h], [w,0]])

M = cv2.getPerspectiveTransform(transform_src, transform_dst)

returnM

defapply_perspective_transform(self, binary_image, M, plot=False):

warped_image = cv2.warpPerspective(binary_image, M, (binary_image.shape[1], binary_image.shape[0]), flags=cv2.INTER_NEAREST)# keep same size as input image

if(plot):

# Ploting both images Binary and Warped

f, (ax1, ax2) = plt.subplots(1,2, figsize=(20,10))

ax1.set_title('Binary/Undistorted and Tresholded')

ax1.imshow(binary_image, cmap='gray')

ax2.set_title('Binary/Undistorted and Warped Image')

ax2.imshow(warped_image, cmap='gray')

plt.show()

returnwarped_image

以下面的方式選擇硬編碼的源和目標(biāo)點(diǎn)：

transform_src= np.float32([ [580,450], [160,h], [1150,h], [740,450]])

transform_dst= np.float32([ [0,0], [0,h], [w,h], [w,0]])

這兩個方法的輸出如下所示：

檢測車道像素，找到車道邊界

現(xiàn)在有一個閾值扭曲的圖像杂抽，我們準(zhǔn)備繪制出車道線！有很多方法可以解決這個問題韩脏，但是在直方圖中使用峰值效果很好缩麸。

在對道路圖像進(jìn)行校準(zhǔn)，閾值處理和透視變換之后赡矢，我們應(yīng)該有一個二值圖像杭朱，車道線清晰可見。但是吹散，我們?nèi)匀恍枰鞔_地確定哪些像素是線的一部分弧械，哪些屬于左邊線，哪些屬于右邊線空民。

我首先沿著圖像下半部分的所有列采用直方圖刃唐，如下所示：

importnumpyasnp

histogram = np.sum(img[img.shape[0]/2:,:], axis=0)

plt.plot(histogram)

使用這個直方圖羞迷，我將圖像中每列的像素值相加。在我的閾值二進(jìn)制圖像中画饥，像素是0或1衔瓮，所以這個直方圖中最突出的兩個峰值將成為車道線底部x坐標(biāo)的良好指標(biāo)。我可以用它作為尋找線條的起點(diǎn)抖甘。從這一點(diǎn)上热鞍，我可以使用一個滑動的窗口，放置在線條中心周圍衔彻，找到并遵循框架頂部的線條薇宠。

defextract_lanes_pixels(self, binary_warped):

# Take a histogram of the bottom half of the image

histogram = np.sum(binary_warped[binary_warped.shape[0]/2:,:], axis=0)

#plt.plot(histogram)

#plt.show()

# Create an output image to draw on and ?visualize the result

out_img = np.dstack((binary_warped, binary_warped, binary_warped))*255

# Find the peak of the left and right halves of the histogram

# These will be the starting point for the left and right lines

midpoint = np.int(histogram.shape[0]/2)

leftx_base = np.argmax(histogram[:midpoint])

rightx_base = np.argmax(histogram[midpoint:]) + midpoint

# Choose the number of sliding windows

nwindows =9

# Set height of windows

window_height = np.int(binary_warped.shape[0]/nwindows)

# Identify the x and y positions of all nonzero pixels in the image

nonzero = binary_warped.nonzero()

nonzeroy = np.array(nonzero[0])

nonzerox = np.array(nonzero[1])

# Current positions to be updated for each window

leftx_current = leftx_base

rightx_current = rightx_base

# Set the width of the windows +/- margin

margin =100

# Set minimum number of pixels found to recenter window

minpix =50

# Create empty lists to receive left and right lane pixel indices

left_lane_inds = []

right_lane_inds = []

# Step through the windows one by one

forwindowinrange(nwindows):

# Identify window boundaries in x and y (and right and left)

win_y_low = binary_warped.shape[0] - (window+1)*window_height

win_y_high = binary_warped.shape[0] - window*window_height

win_xleft_low = leftx_current - margin

win_xleft_high = leftx_current + margin

win_xright_low = rightx_current - margin

win_xright_high = rightx_current + margin

# Draw the windows on the visualization image

cv2.rectangle(out_img,(win_xleft_low,win_y_low),(win_xleft_high,win_y_high),(0,255,0),2)

cv2.rectangle(out_img,(win_xright_low,win_y_low),(win_xright_high,win_y_high),(0,255,0),2)

# Identify the nonzero pixels in x and y within the window

good_left_inds = ((nonzeroy >= win_y_low) & (nonzeroy < win_y_high) & (nonzerox >= win_xleft_low) & (nonzerox < win_xleft_high)).nonzero()[0]

good_right_inds = ((nonzeroy >= win_y_low) & (nonzeroy < win_y_high) & (nonzerox >= win_xright_low) & (nonzerox < win_xright_high)).nonzero()[0]

# Append these indices to the lists

left_lane_inds.append(good_left_inds)

right_lane_inds.append(good_right_inds)

# If you found > minpix pixels, recenter next window on their mean position

iflen(good_left_inds) >minpix:

leftx_current = np.int(np.mean(nonzerox[good_left_inds]))

iflen(good_right_inds) >minpix:

rightx_current = np.int(np.mean(nonzerox[good_right_inds]))

# Concatenate the arrays of indices

left_lane_inds = np.concatenate(left_lane_inds)

right_lane_inds = np.concatenate(right_lane_inds)

# Extract left and right line pixel positions

leftx = nonzerox[left_lane_inds]

lefty = nonzeroy[left_lane_inds]

rightx = nonzerox[right_lane_inds]

righty = nonzeroy[right_lane_inds]

returnleftx, lefty, rightx, righty, left_lane_inds, right_lane_inds

defpoly_fit(self, leftx, lefty, rightx, righty, left_lane_inds, right_lane_inds, binary_warped,plot:False):

# Fit a second order polynomial to each

left_fit = np.polyfit(lefty, leftx,2)

right_fit = np.polyfit(righty, rightx,2)

# Generate x and y values for plotting

ploty = np.linspace(0, binary_warped.shape[0]-1, binary_warped.shape[0] )

left_fitx = left_fit[0]*ploty**2+ left_fit[1]*ploty + left_fit[2]

right_fitx = right_fit[0]*ploty**2+ right_fit[1]*ploty + right_fit[2]

# Identify the x and y positions of all nonzero pixels in the image

nonzero = binary_warped.nonzero()

nonzeroy = np.array(nonzero[0])

nonzerox = np.array(nonzero[1])

out_img = np.dstack((binary_warped, binary_warped, binary_warped))*255

out_img[nonzeroy[left_lane_inds], nonzerox[left_lane_inds]] = [255,0,0]

out_img[nonzeroy[right_lane_inds], nonzerox[right_lane_inds]] = [0,0,255]

if(plot):

plt.imshow(out_img)

plt.plot(left_fitx, ploty, color='yellow')

plt.plot(right_fitx, ploty, color='yellow')

plt.xlim(0,1280)

plt.ylim(720,0)

plt.show()

returnleft_fit, right_fit, ploty, left_fitx, right_fitx

輸出應(yīng)該是這樣的：

確定車道和車輛相對于中心的曲率

自駕車需要被告知正確的轉(zhuǎn)向角度，左轉(zhuǎn)或右轉(zhuǎn)艰额。如果我們知道汽車的速度和動力以及車道彎曲的程度昼接，我們就可以計(jì)算出這個角度。計(jì)算車道線曲率的一種方法是將二次多項(xiàng)式擬合到該線上悴晰，由此我們可以容易地提取有用的信息慢睡。

對于接近垂線的車道線，我們可以用這個公式擬合一條直線：f（y）= Ay ^ 2 + By + C铡溪，其中A漂辐，B和C是系數(shù)。A給出了車道線的曲率棕硫，B給出了該線所指向的標(biāo)題或方向髓涯，并且C根據(jù)距離圖像的最左邊多遠(yuǎn)來給出線的位置（y = 0 ）。

執(zhí)行：

defcompute_curvature(self, left_fit, right_fit, ploty, left_fitx, right_fitx, leftx, lefty, rightx, righty):

# Define conversions in x and y from pixels space to meters

ym_per_pix =30/720# meters per pixel in y dimension

xm_per_pix =3.7/700# meters per pixel in x dimension

y_eval = np.max(ploty)

fit_cr_left = np.polyfit(ploty * ym_per_pix, left_fitx * xm_per_pix,2)

curverad_left = ((1+ (2* left_fit[0] * y_eval /2. + fit_cr_left[1]) **2) **1.5) / np.absolute(2* fit_cr_left[0])

fit_cr_right = np.polyfit(ploty * ym_per_pix, right_fitx * xm_per_pix,2)

curverad_right = ((1+ (2* left_fit[0] * y_eval /2. + fit_cr_right[1]) **2) **1.5) / np.absolute(2* fit_cr_right[0])

return(curverad_left + curverad_right) /2

輸出車道邊界的視覺顯示和車道曲率和車輛位置的數(shù)值估計(jì)

lane_detection.py中的函數(shù)render_curvature_and_offset用于將檢測到的車道線返回到原始圖像上哈扮，并使用填充的多邊形繪制檢測到的車道纬纪。它還繪制了圖像或視頻幀的左上角和底部的曲率和位置。

所有六個測試圖像的結(jié)果：