opencv +數(shù)字識別

現(xiàn)在很多場景需要使用的數(shù)字識別姆坚,比如銀行卡識別惧眠,以及車牌識別等,在AI領(lǐng)域有很多圖像識別算法嗤放,大多是居于opencv 或者谷歌開源的tesseract 識別.

由于公司業(yè)務(wù)需要档悠,需要開發(fā)一個客戶端程序廊鸥,同時需要在xp這種老古董的機子上運行,故研究了如下幾個數(shù)字識別方案:

ocr 識別的不同選擇方案

  • tesseract
    • 放棄:谷歌的開源tesseract ocr識別目前最新版本不支持xp系統(tǒng)
  • 云端ocr 識別接口(不適用)
    • 費用比較貴:
    • 場景不同辖所,我們的需求是可能毫秒級別就需要調(diào)用一次ocr 識別
  • opencv
  • 概念:OpenCV是一個基于BSD許可(開源)發(fā)行的跨平臺計算機視覺庫惰说,可以運行在Linux、Windows缘回、Android和Mac OS操作系統(tǒng)上吆视。它輕量級而且高效——由一系列 C 函數(shù)和少量 C++ 類構(gòu)成,同時提供了Python酥宴、Ruby啦吧、MATLAB等語言的接口,實現(xiàn)了圖像處理和計算機視覺方面的很多通用算法拙寡。

以上幾種ocr 識別比較授滓,最后選擇了opencv 的方式進行ocr 數(shù)字識別,下面講解通過ocr識別的基本流程和算法.

opencv 數(shù)字識別流程及算法解析

要通過opencv 進行數(shù)字識別離不開訓(xùn)練庫的支持,需要對目標(biāo)圖片進行大量的訓(xùn)練肆糕,才能做到精準(zhǔn)的識別出目標(biāo)數(shù)字;下面我會分別講解圖片訓(xùn)練的過程及識別的過程.

opencv 識別算法原理

  1. 比如下面一張圖片般堆,需要從中識別出正確的數(shù)字,需要對圖片進行灰度诚啃、二值化淮摔、腐蝕、膨脹始赎、尋找數(shù)字輪廓和橙、切割等一系列操作.

原圖

image

灰度化圖

image

二值化圖

image

尋找輪廓

image

識別后的結(jié)果圖

image

以上就是簡單的圖片進行灰度化、二值化造垛、尋找數(shù)字輪廓得到的識別結(jié)果(==這是基于我之前訓(xùn)練過的數(shù)字模型下得到的識別結(jié)果==)
有些圖片比較賦值魔招,比如存在背景斜杠等的圖片則需要一定的腐蝕或者膨脹等處理,才能尋找到正確的數(shù)字輪廓.

上面的說到我這里使用的是opencv 圖像處理庫進行的ocr 識別筋搏,那我這里簡單介紹下C# 怎么使用opencv 圖像處理看;

為了在xp上能夠運行 我這里通過nuget 包引用了 OpenCvSharp-AnyCPU 第三方庫,它使用的是opencv 2410 版本,你們?nèi)绻豢紤]xp系統(tǒng)的情況下開源使用最新的版本,最新版本支持了更多的識別算法.

右擊你的個人項目厕隧,選擇“管理Nuget程序包”奔脐。在包管理器頁面中,點擊“瀏覽”選項吁讨,然后在搜索框中鍵入“OpenCvSharp-AnyCPU”髓迎。選擇最頂端的正確項目,并在右側(cè)詳情頁中點擊“安裝”建丧,等待安裝完成即可排龄。

以上的核心代碼如下:

      private void runSimpleOCR(string pathName)
       {
            //構(gòu)造opcvOcr 庫,這里的是我單獨對opencv 庫進行的一次封裝,加載訓(xùn)練庫模板
            var opencvOcr = new OpencvOcr($"{path}Template\\Traindata.xml", opencvOcrConfig: new OCR.Model.OpencvOcrConfig()
            {
                ErodeLevel = 2.5,
                ThresholdType = OpenCvSharp.ThresholdType.Binary,
                ZoomLevel = 2,
            });

            var img = new Bitmap(this.txbFilaName.Text);

            var mat = img.ToMat();
            
            //核心識別方法
            var str = opencvOcr.GetText(mat, isDebug: true);
            this.labContent.Content = str;
        }

opencvOcr 的核心代碼如下


        #region Constructor

        const double Thresh = 80;
        const double ThresholdMaxVal = 255;
        const int _minHeight = 35;
        bool _isDebug = false;
        CvKNearest _cvKNearest = null;
        OpencvOcrConfig _config = new OpencvOcrConfig() { ZoomLevel = 2, ErodeLevel = 3 };
        #endregion

        /// <summary>
        /// 構(gòu)造函數(shù)
        /// </summary>
        /// <param name="path">訓(xùn)練庫完整路徑</param>
        /// <param name="opencvOcrConfig">OCR相關(guān)配置信息</param>
        public OpencvOcr(string path, OpencvOcrConfig opencvOcrConfig = null)
        {
            if (string.IsNullOrEmpty(path))
                throw new ArgumentNullException("path is not null");

            if (opencvOcrConfig != null)
                _config = opencvOcrConfig;

            this.LoadKnearest(path);
        }
        
        /// <summary>
        /// 加載Knn 訓(xùn)練庫模型
        /// </summary>
        /// <param name="dataPathFile"></param>
        /// <returns></returns>
        private CvKNearest LoadKnearest(string dataPathFile)
        {
            if (_cvKNearest == null)
            {

                using (var fs = new FileStorage(dataPathFile, FileStorageMode.Read))
                {
                    var samples = fs["samples"].ReadMat();
                    var responses = fs["responses"].ReadMat();
                    this._cvKNearest = new CvKNearest();
                    this._cvKNearest.Train(samples, responses);
                }
            }
            return _cvKNearest;
        }

        /// <summary>
        /// OCR 識別,僅僅只能識別單行數(shù)字 
        /// </summary>
        /// <param name="kNearest">訓(xùn)練庫</param>
        /// <param name="path">要識別的圖片路徑</param>
        public override string GetText(Mat src, bool isDebug = false)
        {
            this._isDebug = isDebug;

            #region 圖片處理
            var respMat = MatProcessing(src, isDebug);
            if (respMat == null)
                return "";
            #endregion

            #region 查找輪廓
            var sortRect = FindContours(respMat.FindContoursMat);
            #endregion

            return GetText(sortRect, respMat.ResourcMat, respMat.RoiResultMat);
        }
        
         /// <summary>
        /// 查找輪廓
        /// </summary>
        /// <param name="src"></param>
        /// <returns></returns>
        private List<Rect> FindContours(Mat src)
        {
            try
            {
                #region 查找輪廓
                Point[][] contours;
                HierarchyIndex[] hierarchyIndexes;
                Cv2.FindContours(
                    src,
                    out contours,
                    out hierarchyIndexes,
                    mode: OpenCvSharp.ContourRetrieval.External,
                    method: OpenCvSharp.ContourChain.ApproxSimple);

                if (contours.Length == 0)
                    throw new NotSupportedException("Couldn't find any object in the image.");
                #endregion

                #region 單行排序(目前僅僅支持單行文字,多行文字順序可能不對翎朱,按照x坐標(biāo)進行排序)
                var sortRect = GetSortRect(contours, hierarchyIndexes);
                sortRect = sortRect.OrderBy(item => item.X).ToList();
                #endregion

                return sortRect;
            }
            catch { }

            return null;
        }
        
        /// <summary>
        /// 獲得切割后的數(shù)量列表
        /// </summary>
        /// <param name="contours"></param>
        /// <param name="hierarchyIndex"></param>
        /// <returns></returns>
        private List<Rect> GetSortRect(Point[][] contours, HierarchyIndex[] hierarchyIndex)
        {
            var sortRect = new List<Rect>();

            var _contourIndex = 0;
            while ((_contourIndex >= 0))
            {
                var contour = contours[_contourIndex];
                var boundingRect = Cv2.BoundingRect(contour); //Find bounding rect for each contour

                sortRect.Add(boundingRect);
                _contourIndex = hierarchyIndex[_contourIndex].Next;
            }
            return sortRect;
        }


        /// <summary>
        /// 是否放大
        /// </summary>
        /// <param name="src"></param>
        /// <returns></returns>
        private bool IsZoom(Mat src)
        {
            if (src.Height <= _minHeight)
                return true;

            return false;
        }
        

        private List<EnumMatAlgorithmType> GetAlgoritmList(Mat src)
        {
            var result = new List<EnumMatAlgorithmType>();
            var algorithm = this._config.Algorithm;

            #region 自定義的算法
            try
            {
                if (algorithm.Contains("|"))
                {
                    result = algorithm.Split('|').ToList()
                        .Select(item => (EnumMatAlgorithmType)Convert.ToInt32(item))
                        .ToList();

                    if (!IsZoom(src))
                        result.Remove(EnumMatAlgorithmType.Zoom);

                    return result;
                }
            }
            catch { }

            #endregion

            #region 默認(rèn)算法
            if (IsZoom(src))
            {
                result.Add(EnumMatAlgorithmType.Zoom);
            }
            if (this._config.ThresholdType == ThresholdType.Binary)
            {
                //result.Add(EnumMatAlgorithmType.Blur);

                result.Add(EnumMatAlgorithmType.Gray);
                result.Add(EnumMatAlgorithmType.Thresh);
                if (this._config.DilateLevel > 0)
                    result.Add(EnumMatAlgorithmType.Dilate);

                result.Add(EnumMatAlgorithmType.Erode);
                return result;
            }
            //result.Add(EnumMatAlgorithmType.Blur);

            result.Add(EnumMatAlgorithmType.Gray);
            result.Add(EnumMatAlgorithmType.Thresh);
            if (this._config.DilateLevel > 0)
                result.Add(EnumMatAlgorithmType.Dilate);

            result.Add(EnumMatAlgorithmType.Erode);
            return result;
            #endregion
        }


        /// <summary>
        /// 對查找的輪廓數(shù)據(jù)進行訓(xùn)練模型匹配橄维,這里使用的是KNN 匹配算法
        /// </summary>
        private string GetText(List<Rect> sortRect, Mat source, Mat roiSource)
        {
            var response = "";
            try
            {
                if ((sortRect?.Count ?? 0) <= 0)
                    return response;

                var contourIndex = 0;
                using (var dst = new Mat(source.Rows, source.Cols, MatType.CV_8UC3, Scalar.All(0)))
                {
                    sortRect.ForEach(boundingRect =>
                    {
                        try
                        {
                            #region 繪制矩形
                            if (this._isDebug)
                            {
                                Cv2.Rectangle(source, new Point(boundingRect.X, boundingRect.Y),
                                new Point(boundingRect.X + boundingRect.Width, boundingRect.Y + boundingRect.Height),
                                new Scalar(0, 0, 255), 1);

                                Cv2.Rectangle(roiSource, new Point(boundingRect.X, boundingRect.Y),
                                   new Point(boundingRect.X + boundingRect.Width, boundingRect.Y + boundingRect.Height),
                                   new Scalar(0, 0, 255), 1);
                            }
                            #endregion

                            #region 單個ROI
                            var roi = roiSource.GetROI(boundingRect); //Crop the image
                            roi = roi.Compress();
                            var result = roi.ConvertFloat();
                            #endregion

                            #region KNN 匹配
                            var results = new Mat();
                            var neighborResponses = new Mat();
                            var dists = new Mat();
                            var detectedClass = (int)this._cvKNearest.FindNearest(result, 1, results, neighborResponses, dists);
                            var resultText = detectedClass.ToString(CultureInfo.InvariantCulture);
                            #endregion

                            #region 匹配
                            var isDraw = false;
                            if (detectedClass >= 0)
                            {
                                response += detectedClass.ToString();
                                isDraw = true;
                            }
                            if (detectedClass == -1 && !response.Contains("."))
                            {
                                response += ".";
                                resultText = ".";
                                isDraw = true;
                            }
                            #endregion

                            #region 繪制及輸出切割信息庫
                            try
                            {
                                //if (this._isDebug)
                                //{
                                Write(contourIndex, detectedClass, roi);
                                //}
                            }
                            catch { }

                            if (this._isDebug && isDraw)
                            {
                                Cv2.PutText(dst, resultText, new Point(boundingRect.X, boundingRect.Y + boundingRect.Height), 0, 1, new Scalar(0, 255, 0), 2);
                            }
                            #endregion

                            result?.Dispose();
                            results?.Dispose();
                            neighborResponses?.Dispose();
                            dists?.Dispose();
                            contourIndex++;
                        }
                        catch (Exception ex)
                        {
                            TextHelper.Error("GetText ex", ex);
                        }
                    });

                    #region 調(diào)試模式顯示過程
                    source.IsDebugShow("Segmented Source", this._isDebug);
                    dst.IsDebugShow("Detected", this._isDebug);
                    dst.IsDebugWaitKey(this._isDebug);
                    dst.IsDebugImWrite("dest.jpg", this._isDebug);
                    #endregion
                }
            }
            catch
            {
                throw;
            }
            finally
            {
                source?.Dispose();
                roiSource?.Dispose();
            }
            return response;
        }
        
        /// <summary>
        /// 圖片處理算法
        /// </summary>
        /// <param name="src"></param>
        /// <param name="isDebug"></param>
        /// <returns></returns>
        public ImageProcessModel MatProcessing(Mat src, bool isDebug = false)
        {
            src.IsDebugShow("原圖", isDebug);

            var list = GetAlgoritmList(src);
            var resultMat = new Mat();
            src.CopyTo(resultMat);
            var isZoom = IsZoom(src);
            list?.ForEach(item =>
            {
                switch (item)
                {
                    case EnumMatAlgorithmType.Dilate:
                        resultMat = resultMat.ToDilate(Convert.ToInt32(this._config.DilateLevel));
                        resultMat.IsDebugShow(EnumMatAlgorithmType.Dilate.GetDescription(), isDebug);
                        break;
                    case EnumMatAlgorithmType.Erode:
                        var eroderLevel = isZoom ? this._config.ErodeLevel * this._config.ZoomLevel : this._config.ErodeLevel;
                        resultMat = resultMat.ToErode(eroderLevel);
                        resultMat.IsDebugShow(EnumMatAlgorithmType.Erode.GetDescription(), isDebug);
                        break;
                    case EnumMatAlgorithmType.Gray:
                        resultMat = resultMat.ToGrey();
                        resultMat.IsDebugShow(EnumMatAlgorithmType.Gray.GetDescription(), isDebug);
                        break;
                    case EnumMatAlgorithmType.Thresh:
                        var thresholdValue = this._config.ThresholdValue <= 0 ? resultMat.GetMeanThreshold() : this._config.ThresholdValue;
                        resultMat = resultMat.ToThreshold(thresholdValue, thresholdType: this._config.ThresholdType);
                        resultMat.IsDebugShow(EnumMatAlgorithmType.Thresh.GetDescription(), isDebug);
                        break;
                    case EnumMatAlgorithmType.Zoom:
                        resultMat = resultMat.ToZoom(this._config.ZoomLevel);
                        src = resultMat;
                        resultMat.IsDebugShow(EnumMatAlgorithmType.Zoom.GetDescription(), isDebug);
                        break;
                    case EnumMatAlgorithmType.Blur:
                        resultMat = resultMat.ToBlur();
                        src = resultMat;
                        resultMat.IsDebugShow(EnumMatAlgorithmType.Blur.GetDescription(), isDebug);
                        break;
                }
            });

            var oldThreshImage = new Mat();
            resultMat.CopyTo(oldThreshImage);

            return new ImageProcessModel()
            {
                ResourcMat = src,
                FindContoursMat = oldThreshImage,
                RoiResultMat = resultMat
            };
        }

opencv 圖片處理開放出去的配置對象實體如下:

 public class OpencvOcrConfig
    {
        /// <summary>
        /// 放大程度級別 默認(rèn)2
        /// </summary>
        public double ZoomLevel { set; get; }

        /// <summary>
        /// 腐蝕級別 默認(rèn)2.5
        /// </summary>
        public double ErodeLevel { set; get; }

        /// <summary>
        /// 膨脹
        /// </summary>
        public double DilateLevel { set; get; }

        /// <summary>
        /// 閥值
        /// </summary>
        public double ThresholdValue { set; get; }

        /// <summary>
        /// 圖片處理算法,用逗號隔開
        /// </summary>
        public string Algorithm { set; get; }

        /// <summary>
        /// 二值化方式
        /// </summary>
        public ThresholdType ThresholdType { set; get; } = ThresholdType.BinaryInv;

        /// <summary>
        /// 通道模式
        /// </summary>
        public OcrChannelTypeEnums ChannelType { set; get; } = OcrChannelTypeEnums.BlackBox;

    }

opencv 圖片處理算法擴展方法如下:

 public static partial class OpenCvExtensions
    {
        private const int Thresh = 200;
        private const int ThresholdMaxVal = 255;

        /// <summary>
        /// Bitmap Convert Mat
        /// </summary>
        /// <param name="bitmap"></param>
        /// <returns></returns>
        public static Mat ToMat(this System.Drawing.Bitmap bitmap)
        {
            return OpenCvSharp.Extensions.BitmapConverter.ToMat(bitmap);
        }

        /// <summary>
        /// Bitmap Convert Mat
        /// </summary>
        /// <param name="bitmap"></param>
        /// <returns></returns>
        public static System.Drawing.Bitmap ToBitmap(this Mat mat)
        {
            return OpenCvSharp.Extensions.BitmapConverter.ToBitmap(mat);
        }


        public static bool MatIsEqual(this Mat mat1, Mat mat2)
        {
            try
            {
                if (mat1.Empty() && mat2.Empty())
                {
                    return true;
                }
                if (mat1.Cols != mat2.Cols || mat1.Rows != mat2.Rows || mat1.Dims() != mat2.Dims() ||
                    mat1.Channels() != mat2.Channels())
                {
                    return false;
                }
                if (mat1.Size() != mat2.Size() || mat1.Type() != mat2.Type())
                {
                    return false;
                }
                var nrOfElements1 = mat1.Total() * mat1.ElemSize();
                if (nrOfElements1 != mat2.Total() * mat2.ElemSize())
                    return false;

                return MatPixelEqual(mat1, mat2);
            }
            catch (Exception ex)
            {
                TextHelper.Error("MatIsEqual 異常", ex);
                return true;
            }
        }

        /// <summary>
        /// 灰度
        /// </summary>
        /// <param name="mat"></param>
        /// <returns></returns>
        public static Mat ToGrey(this Mat mat)
        {
            try
            {
                Mat grey = new Mat();
                Cv2.CvtColor(mat, grey, OpenCvSharp.ColorConversion.BgraToGray);
                return grey;
            }
            catch
            {
                return mat;
            }
        }

        /// <summary>
        /// 二值化
        /// </summary>
        /// <param name="data"></param>
        /// <returns></returns>
        public static Mat ToThreshold(this Mat data, double threshValue = 0, ThresholdType thresholdType = ThresholdType.BinaryInv)
        {
            Mat threshold = new Mat();

            if (threshValue == 0)
                threshValue = Thresh;
            Cv2.Threshold(data, threshold, threshValue, ThresholdMaxVal, thresholdType);
            if (threshold.IsBinaryInv())
            {
                Cv2.Threshold(threshold, threshold, threshValue, ThresholdMaxVal, ThresholdType.BinaryInv);
            }


            //Mat threshold = new Mat();

            //if (threshValue == 0)
            //    threshValue = Thresh;
            //Cv2.AdaptiveThreshold(data, threshold, ThresholdMaxVal,AdaptiveThresholdType.MeanC, thresholdType,3,0);
            //if (threshold.IsBinaryInv())
            //{
            //    Cv2.AdaptiveThreshold(threshold, threshold, ThresholdMaxVal, AdaptiveThresholdType.MeanC, ThresholdType.BinaryInv,3, 0);
            //}
            //Cv2.AdaptiveThreshold()
            // Threshold to find contour
            //var threshold = data.Threshold(80, 255, ThresholdType.BinaryInv);
            //Cv2.Threshold(data, threshold, Thresh, ThresholdMaxVal, ThresholdType.BinaryInv); // Threshold to find contour

            //Cv2.AdaptiveThreshold(data, threshold, 255, AdaptiveThresholdType.MeanC, ThresholdType.BinaryInv, 11, 2);

            //Cv2.Threshold(data, data, Thresh, ThresholdMaxVal, OpenCvSharp.ThresholdType.BinaryInv); // Threshold to find contour
            //Cv2.AdaptiveThreshold(data, threshold, ThresholdMaxVal, AdaptiveThresholdType.GaussianC, OpenCvSharp.ThresholdType.Binary, 3, 0); // Threshold to find contour
            //Cv2.AdaptiveThreshold(data, threshold, 255, AdaptiveThresholdType.MeanC, ThresholdType.Binary, 3, 0);
            //CvInvoke.AdaptiveThreshold(data, data, 255, Emgu.CV.CvEnum.AdaptiveThresholdType.GaussianC, Emgu.CV.CvEnum.ThresholdType.Binary, 3, 0);
            return threshold;
            //var mat = data.Threshold(100, 255,ThresholdType.Binary);
            //return mat;
        }

        /// <summary>
        /// 是否調(diào)試顯示
        /// </summary>
        /// <param name="src"></param>
        /// <param name="name"></param>
        /// <param name="isDebug"></param>
        public static void IsDebugShow(this Mat src, string name, bool isDebug = false)
        {
            if (!isDebug)
                return;

            Cv2.ImShow(name, src);
        }

        public static void IsDebugWaitKey(this Mat src, bool isDebug = false)
        {
            if (!isDebug)
                return;

            Cv2.WaitKey();
        }

        public static void IsDebugImWrite(this Mat src, string path, bool isDebug = false)
        {
            if (!isDebug)
                return;

            try
            {
                Cv2.ImWrite(path, src);
            }
            catch { }
        }

        /// <summary>
        /// Mat 轉(zhuǎn)成另外一種存儲矩陣方式
        /// </summary>
        /// <param name="roi"></param>
        /// <returns></returns>
        public static Mat ConvertFloat(this Mat roi)
        {
            var resizedImage = new Mat();
            var resizedImageFloat = new Mat();
            Cv2.Resize(roi, resizedImage, new Size(10, 10)); //resize to 10X10
            resizedImage.ConvertTo(resizedImageFloat, MatType.CV_32FC1); //convert to float
            var result = resizedImageFloat.Reshape(1, 1);
            return result;
        }

        /// <summary>
        /// 腐蝕
        /// </summary>
        /// <param name="mat"></param>
        /// <returns></returns>
        public static Mat ToErode(this Mat mat, double level)
        {

            #region level 2.5時默認(rèn)的,自動會判斷是否需要腐蝕
            if (level < 1)
            {
                return mat;
            }
            if (level == 2.5)
            {
                if (!mat.IsErode())
                    return mat;
            }
            #endregion

            var erode = new Mat();

            var copyMat = new Mat();
            mat.CopyTo(copyMat);

            Cv2.Erode(mat, erode, Cv2.GetStructuringElement(StructuringElementShape.Ellipse, new Size(level, level)));
            return erode;
        }

        /// <summary>
        /// 膨脹
        /// </summary>
        /// <param name="mat"></param>
        /// <returns></returns>
        public static Mat ToDilate(this Mat mat, int level)
        {
            if (level <= 0)
                return mat;
            var dilate = new Mat();
            Cv2.Dilate(mat, dilate, Cv2.GetStructuringElement(StructuringElementShape.Ellipse, new Size(level, level)));
            return dilate;
            //return mat;
        }

        /// <summary>
        /// mat 轉(zhuǎn)Roi
        /// </summary>
        /// <param name="image"></param>
        /// <param name="boundingRect"></param>
        /// <returns></returns>
        public static Mat GetROI(this Mat image, Rect boundingRect)
        {
            try
            {
                return new Mat(image, boundingRect); //Crop the image
            }
            catch
            {

            }
            return null;
        }

        /// <summary>
        /// 獲取平均閥值
        /// </summary>
        /// <param name="mat"></param>
        /// <returns></returns>
        public static int GetMeanThreshold(this Mat mat)
        {
            var width = mat.Width;
            var height = mat.Height;

            var m = mat.Reshape(1, width * height);
            return (int)m.Sum() / (width * height);
        }

        /// <summary>
        /// 獲得二值化閥值
        /// </summary>
        /// <param name="bitmap"></param>
        /// <returns></returns>
        public static int GetMeanThreshold(this System.Drawing.Bitmap bitmap)
        {
            using (var mat = bitmap.ToMat())
            using (var grap = mat.ToGrey())
            {
                return grap.GetMeanThreshold();
            }
        }

        public static bool IsErode(this System.Drawing.Bitmap bitmap)
        {
            using (var mat = bitmap.ToMat())
            using (var grap = mat.ToGrey())
            {

                var thresholdValue = grap.GetMeanThreshold();
                using (var threshold = grap.ToThreshold(thresholdValue, ThresholdType.BinaryInv))
                {
                    return threshold.IsErode();
                }
            }
        }

        /// <summary>
        /// 放大
        /// </summary>
        /// <param name="img"></param>
        /// <param name="times"></param>
        /// <returns></returns>
        public static Mat ToZoom(this Mat img, double times)
        {
            if (times <= 0)
                return img;
            var width = img.Width * times;
            var height = img.Height * times;

            img = img.Resize(new Size(width, height), 0, 0, Interpolation.NearestNeighbor);
            return img;
        }

        /// <summary>
        /// 均值濾波
        /// </summary>
        /// <param name="img"></param>
        /// <returns></returns>
        public static Mat ToBlur(this Mat img)
        {
            return img.Blur(new Size(3, 3));
        }

        public static Mat Compress(this Mat img)
        {
            var width = 28.0 * img.Width / img.Height;

            var fWidth = width / img.Width;
            var fHeight = 28.0 / img.Height;

            img = img.Resize(new Size(width, 28), fWidth, fHeight, Interpolation.NearestNeighbor);
            return img;
        }

        public static bool MatPixelEqual(this Mat src, Mat are)
        {
            var width = src.Width;
            var height = src.Height;
            var sum = width * height;

            for (int row = 0; row < height; row++)
            {
                for (int col = 0; col < width; col++)
                {
                    byte p = src.At<byte>(row, col); //獲對應(yīng)矩陣坐標(biāo)的取像素
                    byte pAre = are.At<byte>(row, col);
                    if (p != pAre)
                        return false;
                }
            }
            return true;
        }

        public static int GetSumPixelCount(this Mat threshold)
        {
            var width = threshold.Width;
            var height = threshold.Height;
            var sum = width * height;

            var value = 0;
            for (int row = 0; row < height; row++)
            {
                for (int col = 0; col < width; col++)
                {
                    byte p = threshold.At<byte>(row, col); //獲對應(yīng)矩陣坐標(biāo)的取像素
                    value++;
                }
            }
            return value;
        }

        public static int GetPixelCount(this Mat threshold, System.Drawing.Color color)
        {
            var width = threshold.Width;
            var height = threshold.Height;
            var sum = width * height;

            var value = 0;
            for (int row = 0; row < height; row++)
            {
                for (int col = 0; col < width; col++)
                {
                    byte p = threshold.At<byte>(row, col); //獲對應(yīng)矩陣坐標(biāo)的取像素
                    if (Convert.ToInt32(p) == color.R)
                    {
                        value++;
                    }
                }
            }
            return value;
        }

        /// <summary>
        /// 是否需要二值化反轉(zhuǎn)
        /// </summary>
        /// <param name="threshold"></param>
        /// <returns></returns>
        public static bool IsBinaryInv(this Mat threshold)
        {
            var width = threshold.Width;
            var height = threshold.Height;
            var sum = Convert.ToDouble(width * height);

            var black = GetPixelCount(threshold, System.Drawing.Color.Black);

            return (Convert.ToDouble(black) / sum) < 0.5;
        }

        /// <summary>
        /// 是否需要腐蝕
        /// </summary>
        /// <param name="mat"></param>
        /// <returns></returns>
        public static bool IsErode(this Mat mat)
        {
            var percent = mat.GetPercent();
            return percent >= 0.20;
        }

        /// <summary>
        /// 獲得白色像素占比
        /// </summary>
        /// <param name="threshold"></param>
        /// <returns></returns>
        public static double GetPercent(this Mat threshold)
        {
            var width = threshold.Width;
            var height = threshold.Height;
            var sum = Convert.ToDouble(width * height);

            var white = GetPixelCount(threshold, System.Drawing.Color.White);
            return (Convert.ToDouble(white) / sum);
        }

        /// <summary>
        /// 根據(jù)模板查找目標(biāo)圖片的在原圖標(biāo)中的開始位置坐標(biāo)
        /// </summary>
        /// <param name="source"></param>
        /// <param name="template"></param>
        /// <param name="matchTemplateMethod"></param>
        /// <returns></returns>
        public static Point FindTemplate(this Mat source, Mat template, MatchTemplateMethod matchTemplateMethod = MatchTemplateMethod.SqDiffNormed)
        {
            if (source == null)
                return new OpenCvSharp.CPlusPlus.Point();

            var result = new Mat();
            Cv2.MatchTemplate(source, template, result, matchTemplateMethod);

            Cv2.MinMaxLoc(result, out OpenCvSharp.CPlusPlus.Point minVal, out OpenCvSharp.CPlusPlus.Point maxVal);

            var topLeft = new OpenCvSharp.CPlusPlus.Point();
            if (matchTemplateMethod == MatchTemplateMethod.SqDiff || matchTemplateMethod == MatchTemplateMethod.SqDiffNormed)
            {
                topLeft = minVal;
            }
            else
            {
                topLeft = maxVal;
            }
            return topLeft;
        }
    }

以上代碼中開源對圖片進行輪廓切割尺铣,同時會生成切割后的圖片代碼如下

#region 繪制及輸出切割信息庫
    try
    {

        Write(contourIndex, detectedClass, roi);

    }
    catch { }
#endregion

private void Write(int contourIndex, int detectedClass, Mat roi)
{
    Task.Factory.StartNew(() =>
    {
        try
        {
            var templatePath = $"{AppDomain.CurrentDomain.BaseDirectory}template";
            FileHelper.CreateDirectory(templatePath);
            var templatePathFile = $"{templatePath}/{contourIndex}_{detectedClass.ToString()}.png";
            Cv2.ImWrite(templatePathFile, roi);
            if (!roi.IsDisposed)
            {
                roi.Dispose();
            }
        }
        catch {}
   });
}

切割后的圖片如下:


image

這里我已經(jīng)對數(shù)字進行切割好了,接下來就是需要對0-9 這些數(shù)字進行分類(建立文件夾進行數(shù)字歸類)争舞,如下:


image

圖中的每一個分類都是我事先切割好的數(shù)字圖片,圖中有-1 和-2 這兩個特殊分類凛忿,-1 里面我是放的是“.”好的分類,用于訓(xùn)練“.”的圖片竞川,這樣就可以識別出小數(shù)點的數(shù)字支持.
-2 這個分類主要是其他一些無關(guān)緊要的圖片店溢,也就是不是數(shù)字和點的都?xì)w為這一類中.

現(xiàn)在訓(xùn)練庫分類已經(jīng)建立好了,接下來我們需要對這些分類數(shù)字進行歸一化處理委乌,生成訓(xùn)練模型. 代碼如下:

        private void Button_Click_1(object sender, RoutedEventArgs e)
        {
            var opencvOcr = new OpencvOcr($"{path}Template\\Traindata.xml", opencvOcrConfig: null);
            opencvOcr.Save($"{path}Template\\NumberWrite", outputPath: $"{path}Template\\Traindata.xml");
            MessageBox.Show("生成訓(xùn)練庫成功");
            //var img = new Bitmap(this.txbFilaName.Text);

            //var str = opencvOcr.GetText(img.ToMat(), isDebug: true);
            //this.labContent.Content = str;
        }
        
        /// <summary>
        /// 保存訓(xùn)練模型
        /// </summary>
        /// <param name="dataPath"></param>
        /// <param name="trainExt"></param>
        /// <param name="dataPathFile"></param>
        public void Save(string dataPath, string trainExt = "*.png", string outputPath = "")
        {
            if (string.IsNullOrEmpty(outputPath))
                throw new ArgumentNullException("save dataPath is not null");

            var trainingImages = this.ReadTrainingImages(dataPath, trainExt);
            var samples = GetSamples(trainingImages);
            var response = GetResponse(trainingImages);

            //寫入到訓(xùn)練庫中
            using (var fs = new FileStorage(outputPath, FileStorageMode.WriteText))
            {
                fs.Write("samples", samples);
                fs.Write("responses", response);
            }
        }

        /// <summary>
        /// 根據(jù)目錄加載文件
        /// </summary>
        /// <param name="path"></param>
        /// <param name="ext"></param>
        /// <returns></returns>
        private IList<ImageInfo> ReadTrainingImages(string path, string ext)
        {
            var images = new List<ImageInfo>();
            var imageId = 1;
            foreach (var dir in new DirectoryInfo(path).GetDirectories())
            {
                var groupId = int.Parse(dir.Name);
                foreach (var imageFile in dir.GetFiles(ext))
                {
                    var srcMat = new Mat(imageFile.FullName, OpenCvSharp.LoadMode.GrayScale);
                    var image = srcMat.ConvertFloat();
                    if (image == null)
                    {
                        continue;
                    }

                    images.Add(new ImageInfo
                    {
                        Image = image,
                        ImageId = imageId++,
                        ImageGroupId = groupId
                    });
                }
            }
            return images;
        }
        
        /// <summary>
        /// Mat 轉(zhuǎn)成另外一種存儲矩陣方式
        /// </summary>
        /// <param name="roi"></param>
        /// <returns></returns>
        public static Mat ConvertFloat(this Mat roi)
        {
            var resizedImage = new Mat();
            var resizedImageFloat = new Mat();
            Cv2.Resize(roi, resizedImage, new Size(10, 10)); //resize to 10X10
            resizedImage.ConvertTo(resizedImageFloat, MatType.CV_32FC1); //convert to float
            var result = resizedImageFloat.Reshape(1, 1);
            return result;
        }
        
        /// <summary>
        /// 獲取Samples
        /// </summary>
        /// <param name="trainingImages"></param>
        /// <returns></returns>
        private Mat GetSamples(IList<ImageInfo> trainingImages)
        {
            var samples = new Mat();
            foreach (var trainingImage in trainingImages)
            {
                samples.PushBack(trainingImage.Image);
            }
            return samples;
        }
        
        private Mat GetResponse(IList<ImageInfo> trainingImages)
        {
            var labels = trainingImages.Select(x => x.ImageGroupId).ToArray();
            var responses = new Mat(labels.Length, 1, MatType.CV_32SC1, labels);
            var tmp = responses.Reshape(1, 1); //make continuous
            var responseFloat = new Mat();
            tmp.ConvertTo(responseFloat, MatType.CV_32FC1); // Convert  to float

            return responses;
        }

到這里ocr 訓(xùn)練模型以及建立好了床牧,會在目錄中生成一個Traindata.xml 的訓(xùn)練模型庫,我們來打開這個訓(xùn)練模型庫文件探索它的神秘的容顏.


image

image

到這里opencv + 數(shù)字識別分享已經(jīng)完成,它的神秘面紗也就到此結(jié)束了
到這里opencv + 數(shù)字識別分享已經(jīng)完成遭贸,它的神秘面紗也就到此結(jié)束了
歡迎各位大佬關(guān)注公眾號


dotNET 博士
最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
  • 序言:七十年代末戈咳,一起剝皮案震驚了整個濱河市,隨后出現(xiàn)的幾起案子革砸,更是在濱河造成了極大的恐慌除秀,老刑警劉巖,帶你破解...
    沈念sama閱讀 211,743評論 6 492
  • 序言:濱河連續(xù)發(fā)生了三起死亡事件算利,死亡現(xiàn)場離奇詭異册踩,居然都是意外死亡,警方通過查閱死者的電腦和手機效拭,發(fā)現(xiàn)死者居然都...
    沈念sama閱讀 90,296評論 3 385
  • 文/潘曉璐 我一進店門暂吉,熙熙樓的掌柜王于貴愁眉苦臉地迎上來拉宗,“玉大人刁俭,你說我怎么就攤上這事赠堵《悴椋” “怎么了颖系?”我有些...
    開封第一講書人閱讀 157,285評論 0 348
  • 文/不壞的土叔 我叫張陵阿逃,是天一觀的道長林束。 經(jīng)常有香客問我饮笛,道長判导,這世上最難降的妖魔是什么嫉父? 我笑而不...
    開封第一講書人閱讀 56,485評論 1 283
  • 正文 為了忘掉前任,我火速辦了婚禮眼刃,結(jié)果婚禮上绕辖,老公的妹妹穿的比我還像新娘。我一直安慰自己擂红,他們只是感情好仪际,可當(dāng)我...
    茶點故事閱讀 65,581評論 6 386
  • 文/花漫 我一把揭開白布。 她就那樣靜靜地躺著,像睡著了一般树碱。 火紅的嫁衣襯著肌膚如雪肯适。 梳的紋絲不亂的頭發(fā)上,一...
    開封第一講書人閱讀 49,821評論 1 290
  • 那天赴恨,我揣著相機與錄音疹娶,去河邊找鬼。 笑死伦连,一個胖子當(dāng)著我的面吹牛雨饺,可吹牛的內(nèi)容都是我干的。 我是一名探鬼主播惑淳,決...
    沈念sama閱讀 38,960評論 3 408
  • 文/蒼蘭香墨 我猛地睜開眼额港,長吁一口氣:“原來是場噩夢啊……” “哼!你這毒婦竟也來了歧焦?” 一聲冷哼從身側(cè)響起移斩,我...
    開封第一講書人閱讀 37,719評論 0 266
  • 序言:老撾萬榮一對情侶失蹤,失蹤者是張志新(化名)和其女友劉穎绢馍,沒想到半個月后向瓷,有當(dāng)?shù)厝嗽跇淞掷锇l(fā)現(xiàn)了一具尸體,經(jīng)...
    沈念sama閱讀 44,186評論 1 303
  • 正文 獨居荒郊野嶺守林人離奇死亡舰涌,尸身上長有42處帶血的膿包…… 初始之章·張勛 以下內(nèi)容為張勛視角 年9月15日...
    茶點故事閱讀 36,516評論 2 327
  • 正文 我和宋清朗相戀三年猖任,在試婚紗的時候發(fā)現(xiàn)自己被綠了。 大學(xué)時的朋友給我發(fā)了我未婚夫和他白月光在一起吃飯的照片瓷耙。...
    茶點故事閱讀 38,650評論 1 340
  • 序言:一個原本活蹦亂跳的男人離奇死亡朱躺,死狀恐怖,靈堂內(nèi)的尸體忽然破棺而出搁痛,到底是詐尸還是另有隱情长搀,我是刑警寧澤,帶...
    沈念sama閱讀 34,329評論 4 330
  • 正文 年R本政府宣布鸡典,位于F島的核電站源请,受9級特大地震影響,放射性物質(zhì)發(fā)生泄漏彻况。R本人自食惡果不足惜谁尸,卻給世界環(huán)境...
    茶點故事閱讀 39,936評論 3 313
  • 文/蒙蒙 一、第九天 我趴在偏房一處隱蔽的房頂上張望疗垛。 院中可真熱鬧症汹,春花似錦硫朦、人聲如沸贷腕。這莊子的主人今日做“春日...
    開封第一講書人閱讀 30,757評論 0 21
  • 文/蒼蘭香墨 我抬頭看了看天上的太陽泽裳。三九已至瞒斩,卻和暖如春,著一層夾襖步出監(jiān)牢的瞬間涮总,已是汗流浹背胸囱。 一陣腳步聲響...
    開封第一講書人閱讀 31,991評論 1 266
  • 我被黑心中介騙來泰國打工, 沒想到剛下飛機就差點兒被人妖公主榨干…… 1. 我叫王不留瀑梗,地道東北人烹笔。 一個月前我還...
    沈念sama閱讀 46,370評論 2 360
  • 正文 我出身青樓,卻偏偏與公主長得像抛丽,于是被迫代替她去往敵國和親谤职。 傳聞我的和親對象是個殘疾皇子,可洞房花燭夜當(dāng)晚...
    茶點故事閱讀 43,527評論 2 349

推薦閱讀更多精彩內(nèi)容