Playing musical notes using Scilab

Did you know that you can play notes in Scilab?

No? Don't worry. You are not alone. Like you, I am also amazed that Scilab can play notes. Actually, this idea of using image processing in order to play musical notes is quite new to me. Interesting. Let's give it a try. :)

First, you have to download a music sheet. For my case, I searched for the famous Twinkle, Twinkle Little Star. Simple sheets for starters.

Here is a cropped portion of the music sheet which I am supposed to play in Scilab. 

First scale of Twinkle, Twinkle Little Star

Before processing the image, this must be first converted to grayscale and then to black and white with a threshold of 0.5. Then, invert the colors.

Image ready for processing

In order to assign values to each note, you need to get rid of unwanted lines in your image. Thus, I used the mask filter below.

Mask Filter

The pixel position of the lines are determined using Paint. When the image is multiplied point per point to the mask filter, the resulting image is one that is composed of only notes.

Resulting Image after applying the filter

As you can see, due to the mask filter, the notes are broken. In order to fix that, the Close morphological operation was applied. The structuring element used was a 3x1 vertical line.

Resulting image after the close operation

Comparing the image above to the original image, the note that still looks broken corresponds to the half-note. The rest of the notes are quarter notes.

Using the MedianFilter function in IPD, the notes were reduced to a size such that the group of 1s will be emphasized. Two median filter were used. The first was a 5x5 binary image followed by a 2x2 median filter. Basically, what the filter does is to check if the logical image inside the filter. If there are more true values, for example,  than false values, then the center of that matrix is replaced to true.

Resulting image after applying a 5x5 median filter and then a 2x2 median filter

NOTE: 

THIS WORK IS STILL IN PROGRESS. THE BLOGGER WILL NOT GIVE UP UNTIL SHE GETS THE CORRECT OUTPUT. THIS IS STILL A 6 OUT OF 10 RATING FOR THE ACTIVITY. STAY TUNED! :)

References:

1. Image Processing with Scilab and Image Processing Design Toolbox Tutorial.