Yolov3 Inference Using DNN on Nvidia GPU

In My previous article I had cover the building of DNN module with Cuda support. Given Link below.

Now Today I give you the steps to use this library in visual studio 2015.

Step 1: After building Install project, it installs the library inside the Install folder which is inside the build folder. Directory structure showing below.

Step 2: Mainly we need three folders which includes,bin and lib. Bin and lib folder which you will find inside the x64>vc14>lib/bin. Now we have to configure all these in our application so that we can use the functionality.

Step 3: Start visual studio and select Nvidia cuda project so that Nvidia related configuration automatically attache to your project.Also provide the details like Name,Location,Solution name and then click on OK.

Step 4: Add C++ file to the current project using the following steps. Right click on project >Add>New Item and click. Select C++ file and provide the name of the source file for example like main.cpp and click on Add.

Step 5: Add bin folder to environment variable using the following steps. Type environment in search and click on icon.You will get the system variable window.

Click on Environment variables. You you get the below window.

Select Path variable in System variables and click on Edit.

Click on New and include all bin folder to this. If you see above I included two folders. C:\opencv_4.2\bin and C:\opencv_4.2\x64\vc14\bin. After adding click on OK.

Step 6: Now select build type in visual studio and platform. I selected Release and x64.

Step 7: We have to setup the project properties. Right click on Project > click on properties. This will opencv project properties window. Select C/C++ > General > Additional Include Directories and include your opencv include path.

Step 8: Go in Linker >General >Additional Library Directories. Put your opencv library path as shown in figure.

Step 9: set linker input. Linker > Input > Additional Dependencies. click you will get additional dependencies window and copy the lib name and paste and click on OK.

opencv_aruco420.lib
opencv_bgsegm420.lib
opencv_bioinspired420.lib
opencv_calib3d420.lib
opencv_ccalib420.lib
opencv_core420.lib
opencv_cudaarithm420.lib
opencv_cudabgsegm420.lib
opencv_cudacodec420.lib
opencv_cudafeatures2d420.lib
opencv_cudafilters420.lib
opencv_cudaimgproc420.lib
opencv_cudalegacy420.lib
opencv_cudaobjdetect420.lib
opencv_cudaoptflow420.lib
opencv_cudastereo420.lib
opencv_cudawarping420.lib
opencv_cudev420.lib
opencv_datasets420.lib
opencv_dnn420.lib
opencv_dnn_objdetect420.lib
opencv_dnn_superres420.lib
opencv_dpm420.lib
opencv_face420.lib
opencv_features2d420.lib
opencv_flann420.lib
opencv_fuzzy420.lib
opencv_gapi420.lib
opencv_hfs420.lib
opencv_highgui420.lib
opencv_imgcodecs420.lib
opencv_imgproc420.lib
opencv_img_hash420.lib
opencv_line_descriptor420.lib
opencv_ml420.lib
opencv_objdetect420.lib
opencv_optflow420.lib
opencv_phase_unwrapping420.lib
opencv_photo420.lib
opencv_plot420.lib
opencv_quality420.lib
opencv_reg420.lib
opencv_rgbd420.lib
opencv_saliency420.lib
opencv_shape420.lib
opencv_stereo420.lib
opencv_stitching420.lib
opencv_structured_light420.lib
opencv_superres420.lib
opencv_surface_matching420.lib
opencv_text420.lib
opencv_tracking420.lib
opencv_video420.lib
opencv_videoio420.lib
opencv_videostab420.lib
opencv_xfeatures2d420.lib
opencv_ximgproc420.lib
opencv_xobjdetect420.lib
opencv_xphoto420.lib

Step 10: Download darknet yolov3 weight file, class file and configuration file to use in our code from darknet site. Configure the path of the following files: weight file, class file, configuration file and input file as shown and figure and click on run. It will show out put on windows as well as write the output in same directory.

#include <fstream>#include <sstream>#include <opencv2/dnn.hpp>#include <opencv2/imgproc.hpp>#include <opencv2/highgui.hpp>#include <opencv2/opencv.hpp>using namespace cv;using namespace dnn;float confThreshold, nmsThreshold;
std::vector<std::string> classes;
void postprocess(Mat& frame, const std::vector<Mat>& out, Net& net);
void drawPred(int classId, float conf, int left, int top, int right, int bottom, Mat& frame);
int main(int argc, char** argv){confThreshold = 0.5;nmsThreshold = 0.4;float scale = 0.00392;Scalar mean = { 0,0,0 };bool swapRB = true;int inpWidth = 416;
int inpHeight = 416;
String modelPath = "D:/test_dnn_module/models/yolov3.weights";
String configPath = "D:/test_dnn_module/models/yolov3.cfg";
String framework = "";
int backendId = cv::dnn::DNN_BACKEND_CUDA;
int targetId = cv::dnn::DNN_TARGET_CUDA;
String classesFile = "D:/test_dnn_module/models/coco.txt";
// Open file with classes names.
if (!classesFile.empty()) {const std::string& file = classesFile;std::ifstream ifs(file.c_str());if (!ifs.is_open())CV_Error(Error::StsError, "File " + file + " not found");std::string line;while (std::getline(ifs, line)) {classes.push_back(line);}}// Load a model.Net net = readNet(modelPath, configPath, framework);net.setPreferableBackend(backendId);net.setPreferableTarget(targetId);std::vector<String> outNames = net.getUnconnectedOutLayersNames();// Create a windowstatic const std::string Win = "DNN Test";// Open a video file or an image file or a camera stream.VideoCapture cap;cap.open("D:/test_dnn_module/models/road2.mp4");// Process frames.Mat frame, blob;VideoWriter vid_writer("outcpp.avi", VideoWriter::fourcc('X', 'V', 'I', 'D'), 5, Size(1280, 720));while (waitKey(1) < 0) {cap >> frame;if (frame.empty()) {waitKey();break;}// Create a 4D blob from a frame.Size inpSize(inpWidth > 0 ? inpWidth : frame.cols,inpHeight > 0 ? inpHeight : frame.rows);blobFromImage(frame, blob, scale, inpSize, mean, swapRB, false);// Run a model.net.setInput(blob);if (net.getLayer(0)->outputNameToIndex("im_info") != -1) // Faster-RCNN or R-FCN{resize(frame, frame, inpSize);Mat imInfo = (Mat_<float>(1, 3) << inpSize.height, inpSize.width, 1.6f);net.setInput(imInfo, "im_info");}std::vector<Mat> outs;net.forward(outs, outNames);postprocess(frame, outs, net);// Put efficiency information.std::vector<double> layersTimes;double freq = getTickFrequency() / 1000;double t = net.getPerfProfile(layersTimes) / freq;std::string label = format("Inference time: %.2f ms", t);putText(frame, label, Point(0, 15), FONT_HERSHEY_SIMPLEX, 0.5, Scalar(0, 255, 0));vid_writer.write(frame);imshow(Win, frame);}vid_writer.release();return 0;}void postprocess(Mat& frame, const std::vector<Mat>& outs, Net& net){static std::vector<int> outLayers = net.getUnconnectedOutLayers();static std::string outLayerType = net.getLayer(outLayers[0])->type;std::vector<int> classIds;std::vector<float> confidences;std::vector<Rect> boxes;if (net.getLayer(0)->outputNameToIndex("im_info") != -1) // Faster-RCNN or R-FCN{// Network produces output blob with a shape 1x1xNx7 where N is a number of// detections and an every detection is a vector of values// [batchId, classId, confidence, left, top, right, bottom]CV_Assert(outs.size() == 1);float* data = (float*)outs[0].data;for (size_t i = 0; i < outs[0].total(); i += 7) {float confidence = data[i + 2];if (confidence > confThreshold) {int left = (int)data[i + 3];int top = (int)data[i + 4];int right = (int)data[i + 5];int bottom = (int)data[i + 6];int width = right - left + 1;int height = bottom - top + 1;classIds.push_back((int)(data[i + 1]) - 1); // Skip 0th background class id.boxes.push_back(Rect(left, top, width, height));confidences.push_back(confidence);}}}else if (outLayerType == "DetectionOutput") {// Network produces output blob with a shape 1x1xNx7 where N is a number of// detections and an every detection is a vector of values// [batchId, classId, confidence, left, top, right, bottom]CV_Assert(outs.size() == 1);float* data = (float*)outs[0].data;for (size_t i = 0; i < outs[0].total(); i += 7) {float confidence = data[i + 2];if (confidence > confThreshold) {int left = (int)(data[i + 3] * frame.cols);int top = (int)(data[i + 4] * frame.rows);int right = (int)(data[i + 5] * frame.cols);int bottom = (int)(data[i + 6] * frame.rows);int width = right - left + 1;int height = bottom - top + 1;classIds.push_back((int)(data[i + 1]) - 1); // Skip 0th background class id.boxes.push_back(Rect(left, top, width, height));confidences.push_back(confidence);}}}else if (outLayerType == "Region") {for (size_t i = 0; i < outs.size(); ++i) {// Network produces output blob with a shape NxC where N is a number of// detected objects and C is a number of classes + 4 where the first 4// numbers are [center_x, center_y, width, height]float* data = (float*)outs[i].data;for (int j = 0; j < outs[i].rows; ++j, data += outs[i].cols) {Mat scores = outs[i].row(j).colRange(5, outs[i].cols);Point classIdPoint;double confidence;minMaxLoc(scores, 0, &confidence, 0, &classIdPoint);if (confidence > confThreshold) {int centerX = (int)(data[0] * frame.cols);int centerY = (int)(data[1] * frame.rows);int width = (int)(data[2] * frame.cols);int height = (int)(data[3] * frame.rows);int left = centerX - width / 2;int top = centerY - height / 2;classIds.push_back(classIdPoint.x);confidences.push_back((float)confidence);boxes.push_back(Rect(left, top, width, height));}}}}elseCV_Error(Error::StsNotImplemented, "Unknown output layer type: " + outLayerType);std::vector<int> indices;NMSBoxes(boxes, confidences, confThreshold, nmsThreshold, indices);for (size_t i = 0; i < indices.size(); ++i) {int idx = indices[i];Rect box = boxes[idx];drawPred(classIds[idx], confidences[idx], box.x, box.y,box.x + box.width, box.y + box.height, frame);}}void drawPred(int classId, float conf, int left, int top, int right, int bottom, Mat& frame){rectangle(frame, Point(left, top), Point(right, bottom), Scalar(0, 255, 0));std::string label = format("%.2f", conf);if (!classes.empty()) {CV_Assert(classId < (int)classes.size());label = classes[classId] + ": " + label;}int baseLine;Size labelSize = getTextSize(label, FONT_HERSHEY_SIMPLEX, 0.5, 1, &baseLine);top = max(top, labelSize.height);rectangle(frame, Point(left, top - labelSize.height),Point(left + labelSize.width, top + baseLine), Scalar::all(255), FILLED);putText(frame, label, Point(left, top), FONT_HERSHEY_SIMPLEX, 0.5, Scalar());}

Output:

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store