Orion Nelson

Blog

April 30th 2021 An Update on the Status of my OpenCv Project

    The goal of my current OpenCV Project is to make a proper barcode wrapper that takes a photo of the back of a Drivers licence and extracts all the data required. While I have not completed the project, I have made progress and this post will show the steps I have used to get where I currently am.

  1. Croping The Licence

    2. We start off with detecting a rectangle for the card. To do this we do some image processing and take an edged view of the card. Shown below.

    Edged Image Version

    Edged Image Version.

  2. Finding Contours in the edged version

    3. We then use cv2.findContours(edged.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE) and then iterating through the found contours approximate the shape with approx = cv2.approxPolyDP(c, 0.0225 * peri, True) where peri = cv2.arcLength(c, True). Using this method we then find the largest one of these boxes.

    Here is the largest approximated contour we found using this method

  3. Performing a Perspective Warp and Crop

    4. Using the points from the largest quadrilateral approximate contour for a perspective transformation to 1080p.

    ratio = image.shape[0] / 300.0 # original countour masked version is smaller size
    lbox = order_points(box) # cv2.boxPoints(rect) does not order the points correctly
    screenCnt = (lbox * ratio)
    M = cv2.getPerspectiveTransform(screenCnt, dst_pts)
    warp = cv2.warpPerspective(orig, M, (1920, 1080))

    After this step we are left with a higher quality original cropped version of the card below.

    Here is the Transformed version of the Card at this stage.

  4. Obtaining the Barcode From the Cropped Transformed Card

    5. While it might seem the best strategy to go for an eroded image of the barcode. In my attempts to do this the scanproofing surface and lighting made this an issue.Since my goal was to only catch the barcode I instead found that the best way to achieve this result is first apply a mask by pattern matching. I did this using a similar sized corrupted barcode.

    Here is the top half of my template

    Using a template and pattern matching we are able to remove what is certainly not a barcode the result being area that likely contains a barcode. The two small black squares are for privacy reasons.

    res = cv2.matchTemplate(gray,template,cv2.TM_CCOEFF_NORMED)
    threshold = 0.04
    loc = np.where( res >= threshold)
    for pt in zip(*loc[::-1]):
       closed2 = cv2.rectangle(gray, pt, (pt[0] + w, pt[1] + h), (0,0,0), -1)
    Here is the result after pattern Matching

    Here is the result after pattern Matching

  5. Eroding the Rectangle to Retrive the Barcode and Cropping

    6. Im not going to bore you with this part performed a series of erosions and dilations as well as added blur in order to get the rectangle shape of the barcode.

    Here is the full Correct Barcode

    We then adjust the image to mean OTSU and by doing this we get a clearer barcode image

    cimg2 = cv2.morphologyEx(cimg, cv2.MORPH_CLOSE, cv2.getStructuringElement(cv2.MORPH_RECT, (1, 5)))
    dens = np.sum(cimg2, axis=0)
    mean = np.mean(dens)
    thresh = cimg2.copy()
    for idx, val in enumerate(dens):
      if val< 10800:
      thresh[:,idx] = 0
    (_, thresh2) = cv2.threshold(thresh, 128, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)

    From this image we are then able to scan the barcode using zxing open source barcode library and parse the text inside.

    Here is the full Correct Barcode

    Thanks For Reading.