Playing musical notes using Scilab

Did you know that you can play notes in Scilab?

No? Don't worry. You are not alone. Like you, I am also amazed that Scilab can play notes. Actually, this idea of using image processing in order to play musical notes is quite new to me. Interesting. Let's give it a try. :)

First, you have to download a music sheet. For my case, I searched for the famous Twinkle, Twinkle Little Star. Simple sheets for starters.

Here is a cropped portion of the music sheet which I am supposed to play in Scilab. 

First scale of Twinkle, Twinkle Little Star

Before processing the image, this must be first converted to grayscale and then to black and white with a threshold of 0.5. Then, invert the colors.

Image ready for processing

In order to assign values to each note, you need to get rid of unwanted lines in your image. Thus, I used the mask filter below.

Mask Filter

The pixel position of the lines are determined using Paint. When the image is multiplied point per point to the mask filter, the resulting image is one that is composed of only notes.

Resulting Image after applying the filter

As you can see, due to the mask filter, the notes are broken. In order to fix that, the Close morphological operation was applied. The structuring element used was a 3x1 vertical line.

Resulting image after the close operation

Comparing the image above to the original image, the note that still looks broken corresponds to the half-note. The rest of the notes are quarter notes.

Using the MedianFilter function in IPD, the notes were reduced to a size such that the group of 1s will be emphasized. Two median filter were used. The first was a 5x5 binary image followed by a 2x2 median filter. Basically, what the filter does is to check if the logical image inside the filter. If there are more true values, for example,  than false values, then the center of that matrix is replaced to true.

Resulting image after applying a 5x5 median filter and then a 2x2 median filter

NOTE: 

THIS WORK IS STILL IN PROGRESS. THE BLOGGER WILL NOT GIVE UP UNTIL SHE GETS THE CORRECT OUTPUT. THIS IS STILL A 6 OUT OF 10 RATING FOR THE ACTIVITY. STAY TUNED! :)

References:

1. Image Processing with Scilab and Image Processing Design Toolbox Tutorial.

Pre-processing Text using Morphological Operations

Shown below is an image of a scanned receipt. 

A portion of this receipt which contains the handwritten part is cropped. This cropped image is then used for image processing.

Cropped portion of the scanned receipt

The image is converted to grayscale and then to black and white at a threshold of 0.4.

Black and white conversion of the cropped image

Using the imcomplement function of Scilab, the 1's and 0's were inverted so that the value of the handwritten part are now 1's.

Inverted image

In order to get rid of the lines, a mask has been used. The zero values in the mask are determined by the pixel location of the lines. This was done manually. The pixel locations were determined by using Paint.

Mask Filter

After applying the mask filter, a cleaner image was produced.

However, due to the mask filter, the writings look cut. To connect the lines, morphological operations were applied. The first morphological operation applied was the Close operation. The result of which is shown below.

Structuring Element for the Close Operation

Resulting image after the Close Operation

Another morphological operation, the dilation, was also applied after the close operation to connect the remaining broken lines.

Resulting image after dilation

So far, this is the best image I can process. As you can see, the word VGA is already quite readable.

I have applied a combination of techniques learned in class just to process the image right. Unfortunately, I still haven't come up with the correct techniques to correctly produce the expected image.

For this activity, I give myself 8 out of 10. :)

Morphological Operations: Erosion and Dilation

Morphological operations are used to smoothen boundaries, and remove noise and aritfacts. Morphology means shape. So, basically,  the shape of the object in an image changes depending on the type of operation used.

Morphological operations use binary operations. Two of the common morphological operations are dilation and erosion.

Erosion is an operation defined by the equation below.

where A is the image and B is the structuring element. The shape of the image, after the implementing erosion, is defined by the structuring element used. The origin of the structuring element moves across each pixel of the image, turning white pixels into black along the boundaries and holes of the image, thus, resulting to a reduced version of the original image.

The image below is a prediction of the output image if erosion is applied in an image.

Hand-drawn Predictions of Output image using erosion

To check if my predictions are correct, I used the erode function in the SIP toolbox of Scilab 4.1.2. It can be observed that most of my predictions do not match the generated output. There could be something wrong with my code. Looks like the output for the hollow box should not have such images upon applying erosion.

Output Image using Erosion

On the other hand, dilation is defined by the following equation:

 Dilation causes the image to appear bigger or larger than the original. In order to predict what happens to the image after applying dilation, one has to imagine overlapping the center of the structuring element at each pixel of the image. Shown below is my predicted final output upon applying dilation. The topmost row shapes are the structuring element while the leftmost column are the images to be dilated.

Hand-drawn predictions of output image using dilation

 Unlike my predictions when I used erosion, I was able to accurately predict the output when I predicted in the output image for the dialtion. The only output I failed to correctly predict is when the image of a hollow box.

Output Image using Dilation

I really had a hard time predicting the output. It was really quite confusing. So happy I finally understood the concept, not perfectly though. I still have to practice. But, at least. :)

For this activity, I give myself a 10 since accomplishing the tasks at hand and for understanding the concept. :)

thanks to Izay and Mabes for helping me understand the concepts and debugging the code. :)

Personal note:

This activity is really the most stressful one. It was not because of the activity itself but the fact that my Scilab 4.1.2 is not working well. I can't remember how many times I had to restart my laptop because the Scilab console says, "NOT RESPONDING". It's just so weird then that my Scilab was not working well. Only in this activity. Maybe I was just hexed. :|



Reference:
1. Soriano, M. Morphological Operations.