I want to OCR numbers from the desktop screen.
The work would entail the following:
1) Start a new console project, that starts up with the window minimized.
2) Grab the screen and save it as screen.bmp. (I already have code for you.)
3) Get rectangle coordinates from command line.
4) Clip out the rectangle and save this as rectangle.bmp. (I already have code to write a BMP file, but you will need to write the code that copies the data from the larger BMP already in memory.)
5) Scan the image and extract the digits. (The background will be a single pixel value, taken from the upper left pixel. You will need to scan from left to right, find a different color, trace the edge to extract each digit, OCR it, repeat until all the digits are read.)
6) Print out the number found.
Step 5 is all the work.
5.1) Get the background color 0x00bbggrr
5.2) Scan the columns until you find a non background pixel. (Assume there is a complete background line separating each digit, no overhangs.)
5.3) Flood fill the digit (any non background color, the digit may be shaded.)
5.4) Clip out the digit and convert to black / white.
5.5) Resize to a 28x28 image.
5.6) OCR it. See which digit this is.
5.7) Repeat until the end of the rectangle.
The OCR algorithm is straightforward.
1) Download the MNIST database. This contains 60,000 training images of 28x28 digits.
2) For each digit 0 - 9, make a grayscale image that represents the average of all images. These 10 resulting images will be the template. Save these to a file to be loaded into the program.
3) Given a black/white image, see which digit it is closest to.
* Must compile with Visual Studio Community 2022, on Windows 10.
* No 3rd party libraries, only built in C++.
The deliverables will be 2 Visual Studio solutions.
1) MNIST Template generator.
2) Read Numbers From Screen program.