Percussive Staffs in the GPX format, Fades and continued work on Tuplets

Overview of work this week

The main feature that I’ve been playing with this week is support for percussive staffs in the GPX format. Previously, no notes were imported from the percussive staffs at all. This stopped me from determining whether the code to produce tuplets in the GPX format was correct, as segmentation faults occur when the implementation attempts to make tuplets from notes which are not parsed.

To the large extent this problem has now been resolved, and we handle percussive staffs. There are still a few outstanding issue there, and I’ll discuss them here. The state of play at the moment with percussive staffs can be seen below.

gqjtPnd

 

There are some differences here, as Guitar Pro uses some different note heads. Nevertheless, it’s certainly a start. What I have done for the moment is to keep the implementation as it is shown above, at least for the moment. When there’s a good working system in place then it’ll be time to inspect this further, creating new note heads if that becomes necessary. At any rate, this work has been applied to all the staffs that were unsupported in the larger test case that I’ve been working on (Black or White). With this work in place, it was possible to take the work that I had done to parse triplets in the GPX format, modify it to fix corrections that needed to be made, and then insert that into the implementation of GPX import.

The remaining issue that was noticed was where repeated bars (simile markings) were used. As these were present in my test case, the score was not complete. This is now also handled in the GPX format, and such markings are placed in the appropriate places in the score.

With the above changes in place, for the full score I’ve been working on everything looks great. There are no articulations, slurs, slides, bends etc. but are the information that is somehow “core” such as time signatures, modulation, dotted rhythms, tuplets, notes of various lengths from all staffs now seem to be working. One thing in addition that does not work fully correctly is playback. The instrument chosen for all parts is just the default (piano), so that needs to be looked at as well. I expect there are possibly some parts that are supported in Guitar Pro but not Musescore, it might just be best to take the closest matching instrument in this case, but I’ll deal with that problem if and when it arises. Nothing new has been pushed on the GPX format yet but it could use a few hours spent refactoring it, so I’ll do that first next week and then prepare and make a pull request for that.

Some work has also been done looking at the fade symbol again, which I’ve come back around to now the GPX work is settling down a little. I’ve managed to get that symbol to appear in the master palette, but the symbol still cannot be placed on a note. Clicking and dragging the symbol on to the note certainly causes some layout processing (which can be noticed as other articulations move), but the symbol itself is not displayed. I’ve been following this process through to the relevant calls to QPainter as I considered whether an error was being returned from a painting function (which it isn’t – the ones we use are of void type), but when setting the string to be painted on to the score to a constant value, while everything else changes, that constant value does not appear when the fade symbol is added (via a shortcut) to the selected note. It seems likely that somewhere between the function getting called by the shortcut and the QPainter call the request to draw that symbol is discarded. I’ll need to follow that through the Undo/Redo machinery again in order to try and locate where exactly the articulation is thrown away / fails to render, then modify things appropriately. I expect this will be more difficult to find than it will be to fix.

I’ve looked also at bugs #21928 and #21938 which describe issues with beaming and missing stems in Guitar Pro files. These bugs seem to be no longer valid (the bug was filed about a year ago so it’s been fixed inadvertently by somebody, possibly me), but they did highlight a bug to me in the Guitar Pro 5 software where changing the capo has no effect on the fret numbers specified for notes. I’ve added tests to verify this – we display something different from Guitar Pro in this case as we take capo information into account. I’ll be making a pull request for these tests sometime this week.

I’ve changed the details of bug #22436 (where a note in the part was not heard), as it is principally related to the sounds that the instrument used in that part (Baritone Sax) were defined on. As one of the notes in there (D5) seems to be out of the range for a Baritone Sax, that might be closed as ‘wontfix’, but I’ll leave that for one of the main developers to decide upon.

Key tasks that stalled

Support for the fade-in symbol is somewhat stalling, but I seem to be making some headway with it. I’ll look at this a little more next week, I’d quite like to get this feature fixed and out of the way.

Overview of work for next week

Getting the right instruments used for playback in the GPX format would be quite nice, so I’ll take a look at that. Before that though I would like to refactor the code a little and document it, so that is probably what I will start with. There’s the fade symbol which I need to look at a little more also. After I’ve done that I’ll go hunting in the bug tracker again, there’s actually a few existing bugs surrounding percussion import (bugs #21899 and #22124), so some work on something like that would be good also, I’m not sure what kind of documentation Google need from me for the mid-term evaluations yet, so perhaps it’s time to look at that also. I expect it’ll be a short report and some code fragments. If that’s the case then that can be done next week so it’s submitted for the week after.

Advertisements

Bug shooting with tuples

Overview of work this week

The work this week has been focused on two areas. Firstly the GPX format, and secondly on tuplet support.

With respect to tuplet support a few bugs have been fixed. The first involves support for dotted notes were involved (bug #22888). The issue here was was that dotted notes were handed wrong location. This has been fixed and I will make the pull request for that next week.

Bug #22110 has also been fixed with a similar change. I’ll bundle this into the same PR as the above bug fix.

Bug #23074 I’m not sure is relevant any longer, Musescore is displaying something different than what is reported in the bug report and I think that might be good enough. I’ll have to discuss that with another dev and we can decide whether we want to just close this bug as fixed. It’s likely that one of the changes I have done in the last few weeks has fixed this bug inadvertently.

As I was already fixing the aforementioned bugs on tuplets anyway, I have also done some work on adding tuplet support to the GPX format, which was one of the two items which I’ve market out as the next things I want to target. On implementing this I discovered segmentation faults, which after debugging turned out to be highlighting the issue that percussive parts are not supported, as the implementation was attempting to make tuplets with notes on the percussive staff which are not displayed yet. Now that I know this, I have kept the implementation for tuplets on a local branch at the moment and I won’t be making a pull request for that until percussive staffs are implemented (which I intend to take a look at this week).

While developing solutions for the tests for tuplets in previous versions of Guitar Pro, I noticed that there is a bug where dynamics are repeatedly specified. So if, for example, a note is marked as forte, then all following notes will also be marked with the same dynamic. I have filed a bug report for that (bug #25915), and I’ve managed to get the time to fix that up as well, so there’s already a pull request in for that. There are a few tests which are affected by this, and these have all been updated.

Issue #23519 has also been corrected, where key signatures were imported incorrectly. This bug revealed a problem in the implementation where any key signature involving flats, for any Guitar Pro version, was imported wrongly as 7 sharps. It is not the case that -1 = 1 flat, -2 = 2 flats etc., which is what was implemented, but in fact it works like this:

0 = None,
1 = 1 sharp,
2 = 2 sharps,
3 = 3 sharps,

7 = 7 sharps,
255 = 1 flat,
254 = 2 flats,
253 = 3 flats,

249 = 7 flats.

The implementation for this has now been fixed and I’ve added tests for every format, including the GPX format, This also fixes the initial problem stated in the bug report, so that’s also out of the way. The initial support I have written for the GPX format is now sitting in the Musescore master branch so there’s a key signature test for that too. The test has every key signature (major and minor) present in the file, so that should be sufficient to test this accurately. This also has not been pushed yet, but that will be done after the changes to dynamic repetition mentioned above have been merged.

Key tasks that stalled

Percussion support is still not implemented in GPX files, that’s something that I’ll be looking at this week, and then I expect it’ll be time to write a number of tests for that format.

Overview of work for next week

There are quite a few fixes that I have which are unpushed as mentioned above, so I’ll spend a bit of time getting all that sorted out and merged. The first thing I’m going to look at this week though is support for percussive parts in the GPX format. I’ve already written a bit previously about how we have partial support for that, but that’s something that I would like to improve this week so that a) my big score that I’ve been testing this implementation with is complete and b) so that I can push the changes to the GPX tuplet support in there too. From there I’ll be able to write some more tests for the GPX format, at the moment we have one test for key signatures (which will be pushed with those changes mentioned above this week). I would again like to spend some time next week, like I have this week, looking at existing bugs for previous versions of Guitar Pro. There are approximately 17 of these remaining in the bug tracker, some of which I suspect may be not really Guitar Pro problems. I’ll need to investigate each of them. Certainly for the moment I want to be focusing on previous GP versions and then perhaps a week will be spent on GP6 the week after this.

Any issues that are flagged against the GPX format, as teething problems may well be located, is something that I’ll spend a bit of time on too. There have been a few suggested tweaks to that and there’s currently a pull request which fixes those issues, I’ll create pull requests for any new issues too if and when they are flagged.

Aside

Initial GPX support, Tuples, Fades and Dotted notes

Overview of work this week

A number of things worked on this week, and tweaking of the GPX format.

Key signatures are now imported correctly into a Guitar Pro 6 (GPX) file. Modulation that occurs anywhere in the piece should also be detected and displayed correctly, so that should be all completed. There is still an outstanding bug related to key signatures on a different version of Guitar Pro (http://musescore.org/en/node/23519), and that’s something that I would like to clean up next week if possible. The Guitar Pro file submitted with that bug report is pretty large in size, I’ll try to narrow that down a bit if possible before attempting to fix that bug. I certainly prefer the structure of the new Guitar Pro format (GPX) than that of the previous versions, having the score.gpif XML file means that we can see exactly what we’re dealing with when developing.

Dotted rhythms are also supported, for all notes, in the GPX format. I wanted to fix this as it corrected the wrong number of beats in the bar issue that I get on the test case that I’m using during development, though it’s certainly still possible to get that issue due to the lack of support for tuplets. Tuplet support has been started but there’s something that needs to be done before it’s completed. In its current state, ChordRests are initialised when we encounter a new Beat description. Unfortunately, it cannot be done this way in order to support tuples, as each beat specifies some notes and a rhythm, and whenever a tuplet is present in the rhythm, only one note out of that tuplet is specified in the beat. I’m not sure why they have chose to do it this way rather than specifying all the notes in the tuplet, perhaps that adds for greater flexibility in the specification of other information such as dynamics within each note of the tuplet. In any case, I see what they’re doing for this now, I need to make an edit to the way that we handle beats in order to handle this correctly.

I’ve also looked at work on how the clef is specified, and we should see the correct clef now appearing in GPX files for each instrument/part. This includes the percussive staff, which I have also added some support for. I have added back in support for MIDI note specifications that are present in score.gpif now that this is the case. There is still a little work to be done here, as drum notes specified in the Guitar Pro file are not parsed correctly yet.

I have looked at support for bends in the other Guitar Pro formats. There are all kinds of bends that users can specify in their scores and we should have support now for the majority of these. The feature we don’t have support for w.r.t. bends is the “Hold” feature, Musescore doesn’t have a good way to represent this yet so that’s something that I’ll need to add in later. Probably this will be denoted via a dotted line over the held section (likely using a spanner in Musescore terminology). Bends can of course be held on to the next line of the score, so however spanners work when they are spanned on to a new line of the score is something that I’ll have to investigate later. I’m not going to continue further work on bends likely until later in this project, as it’s a relatively minor feature compared with the rest of the things that I need to implement support for.

The issue that I talked about last week with respect to bends has been fixed. For some odd reason, it was possible for a call to Ms::Bend:layout to exist where a score did not exist. This call was present only during execution of mtest (the test framework) and not when run by the user. Once the issue was located it was fixed with a simple conditional statement to check the existence of the score. Support for bends (minor support for holds) is now present in the Musescore repository.

The fade-in issue is something that I’ve also been working on. There’s a slot for this in the articulation & ornaments portion of the side palette, and I’ve implemented a shortcut for that for reasons of faster development, which is something that may or may not stay when I’ve finished doing that, I would think some kind of shortcut will stay. I seem to have hooked up everything that I can see by following the implementation of other articulations, such as marcato, though I’m not seeing the fade in symbol for this yet from the font. There’s likely a line missing somewhere that I’ll need to find. Representing a font symbol for fades is something that Guitar Pro does, but it’s conceivable that one day we might want to draw this with a bezier curve and represent it as a spanner so that it can span across multiple bars.

All of the work on GPX support has been cleaned up a little and merged with the current master, so now I’ve added a pull request for the GPX support this morning so that everything can get merged together and then I can add pull requests on top of that, this support will likely appear in the Musescore repository some time this week.

Key tasks that stalled

Nothing in particular, the primary thing I wanted to hunt down was the bend bug that’s was stalling the last few weeks and this has now been located and fixed.

Overview of work for next week

I’ll make any changes to the GPX pull request that need to be made before it’s merged into the master branch, it’s a fairly large chunk of code so perhaps some of the devs would like edits. Getting the fade symbol displayed in the palette is something that I would like to see done, as well as attempting to locate the source of error in the bug report concerning key signatures in key signatures in previous Guitar Pro versions that I mentioned. Tuplet support in GPX files would also be a good thing to look at, and then perhaps the bugs at http://musescore.org/en/node/22888 , http://musescore.org/en/node/23074 , and http://musescore.org/en/node/22110 too, which is also related to tuplets (in a previous version of Guitar Pro). It occurs to me that these issues with bugs in tuplets might be due to the same reason that my tuplets aren’t working in the GPX format – if ChordRest values are instantiated to rests they can be redefined, losing tuplet information (a possibility, but I’m not certain on that yet, will need some testing). If I get through all of that then I’ll spend the rest of the time adding more support for GPX files.

Continued work on the GPX format, and debugging bends

Overview of work this week

As last week, almost all of my time has been spent on the GPX format. Last week I wrote about the structure of score.gpif in  little more detail, and described how parsing of that had been started, and measures had been created, albeit with no contents.

I decided at the start of the week to first sort out the information that was being parsed. I had some structure already to iterate through all nodes right down to the note/rhythm level, but this was not yet done in a place where notes could be added to the score, without storing this information in some way and then recalling it later. This was sorted out by splitting the parsing of score.gpif among different functions, each of which was called in the right place with respect to the initialization of other Musescore elements such as the creation of measures.

This being done, I proceeded to display the first note that was present in each bar. In the event that the bar started with e.g. a quaver rest, it would simply be ignored and whatever note came next was put in its place, but the length of that note was always created as a minim (as rhythm information was not yet available). If the bar was entirely empty the measure contained no information. This allowed me to check whether note creation was occurring correctly, and exposed a number of different issues. The main issue that it exposed was the tuning of  the guitar strings.

The way that the tuning of the strings is represented in Guitar Pro 6 is in score.gpif as a list of pitches. A good few hours were spent playing with this, and it didn’t work right away, and if it was incorrect usually segmentation faults would occur. This transpired to be because the tuning that was parsed and set in one function had been somehow reset before I got to the part of my implementation that handled note insertion. After some bug hunting, the first note that occurred in a bar was placed at the beginning of a bar for every part, with the correct tuning in the tab part.

The next issue was sorting out parts which did not in fact contain any tuning information. As these parts are automatically created with tab staffs due to the pre-existing implementation, this caused segmentation faults when adding notes to those parts as they contained no tuning information (dummy tuning information had be used up to this point). The implementation was then modified to not create a link tab staff where it was unnecessary (e.g. for vocals parts).

With this in place, Musescore was then stable again and segmentation faults didn’t occur. My next avenue of work was with the rhythm information. This information was parsed correctly, and then the first note displayed in the bars discussed earlier were created with the correct duration. From here, I added support for rests, which in score.gpif are defined as beats with no note information. After that came handling multiple note specifiers, and then multiple beat specifiers. This created problems with the time signature, as time signature changes were not yet detected in the music.

After multiple beat and notes were supported and the time signature change issue was corrected, there were issues with tick to deal with. Playback was not happening correctly, and this was found to be an issue with (a) tick increment and (b) the time signature changes being used for playback. Both of these were corrected and now playback is fixed.

This means that in its current state, GPX files can be opened and notes from most parts (see below) are now displayed. This implementation needs a little polishing, which I’ll do next week, and then I’ll put in a pull request. The most important things that still don’t yet work are:

  • Clefs. The clef of each part is currently set to the G clef. It it slightly concerning that no clef information appears to be present in score.gpif, and I think this might be computed from the Guitar Pro instrument database it has. If this is the case, I’ll need to check for each individual instrument reference in order to create the right clef. This is yet to be confirmed.
  • Key signatures. This is currently set to C major and still needs to be imported correctly.
  • Tuplets and dotted notes. No support for these musical features means that it is possible to have more beats in a bar than should be allowed (as a triplet of crotchets is simply imported as three crotchet notes).
  • Percussive parts. This is related to the clefs issue, as it’s important to know what the clef is of each part. Some information from percussive parts has been imported, as is represented in the standard staff notation for now.

These are the more important issues. Importing features such as dynamics and slur information is for now considered less important as it doesn’t create “invalid” scores like the above too (such as creating 5 crotchet beats in a 4/4 bar). I intend to tackle the above issues first.

Some work has also been spent attempting to deal with the bends test issue that I discussed a bit last week. It looks like the issue is something related to the layout of bend information, as that’s what the stack track from GDB seems to indicate. I had hoped isolating that test from the others would allow me to locate the error, in the event that bad cleanup from previous tests was affecting the bend test, but this is not the case. I’ll be looking at that more this week, it may be the kind of task that is going to eat my time. If I can find the error behind that this week I’ll add a pull request in for it so that it’s out of the way.

Key tasks that stalled

Bends support is still on hold, I managed to get a little time to look into that at the end of last week but it’s something that I’ll be spending a good bit of time on this week if necessary.

Upcoming tasks this week

Cleaning up the implementation surrounding GPX support is something that I’ll be doing this week, as I wan to add a pull request for that and start getting things into the code base. I also intend to spend some time on the bends test bug, which I have the suspicion will take up time. I haven’t yet decided whether additional time this week will be spent implementing the GPX issues that I listed above, or whether I’ll go back and look at previous versions. A little of both might be nice – I’ll start with key signatures for GPX files and then perhaps do tuplets. For older versions of Guitar Pro, getting the fade-in effect supported might be a nice idea, as that symbol is currently not supported, so I’ll take a look at that too.

Implementing initial support for the GPX format

Overview of work this week

Everything I have done this week has been focused on adding support for the GPX format, the format used by Guitar Pro 6. I wrote a little last time about the “score.gpif” file, and part of the time was spent determining precisely how the structure of the file is represented, and how we can extract the information we need into Musescore. I’ll give a discussion on this below.

The top level node of score.gpif is <GPIF> which has several children, namely <GPRevision>, <Score>, <MasterTrack>, <Tracks>, <MasterBars>, <Bars>, <Voices>, <Beats>, <Notes> and <Rhythms>.

  • <GPRevision> simply contains a number which represents version information.
  • <Score> contains meta-information about the score that has been created. Such information in here includes Title, Artist, and Copyright information.
  • <MasterTrack> contains some information that apply to parts. This includes listing unique identifiers that are assigned to parts, the initial tempo, and other miscellaneous information.
  • <Tracks> describes information about a given track with one of the unique identiiers described in <MasterTrack>. Information such as the instrument reference string that is used by Guitar Pro in order to initialise the correct instrument for playback.
  • <MasterBars> lists information about bars, and this information exists in a specific ordering. Each child of <MasterBars> contains three nodes – <Key>, <Time>, and <Bars>. The <Key> node contains information such as a count of the number of accidentals (it is suspected this can be used to detect changes in key signature). <Time> gives the time signature for the bars listed in <Bars>, and can be used to determine changes in time signature. <Bars> gives a list of bar identifiers, all of which apply to a different part in the order they were defined in the <Tracks> node. This means that the number of identifiers specified in <Bars>, should be the number of bars in the music multiplied by the number of parts.
  • <Bars> contains descriptions of individual bars, each of which have unique identifiers. Descriptions include the clef for that bar, and a list of voice identifiers.
  • <Voices> contains a list of the unique <Beat> identifiers for each unique identifier for a voice. The beat identifiers that are listed will be all the beats that will apply to this voice, in the bar that this voice identifier was specified in.
  • <Beats> describes a list of each <Beat>, each of which has a unique identifier, and details information such as the dynamic of that beat (as a string, P for piano, MF for mezzo forte etc.), a <Rhythm> unique identifier, a <Note> unique identifier and some other miscellaneous properties for that note. Note that the <Note> identifier may not be present – if it is not, then that indicates that a rest should take place for the duration described in <Rhythm>.
  • <Rhythms> lists individual <Rhythm> values, each with its own unique identifier. Each contains a description <NoteValue> containing the duration of this <Rhythm> as a string, examples include “Quarter”, “Eighth” etc. In the test case I have, anything subdivided more that “Eighth” is then represented as e.g. “16th”, “32nd”, In addition to this, the tag <AugmentationDot> describes whether this rhythm is dotted, and if so the duration of the note should be multiplied by 1.5. A count of the dots is also present here, so notes can be “double-dotted”.
  • <Notes> gives a list of individual <Note> values, each again with a unique identifier. Descriptions can be given here about whether a note is tied, whether a note should have a bend to apply to it, whether the note is muted etc. String and fret numbers are specified here among various other note-specific properties.

I am not considering absolutely all of the information specified in the file at the same time. My current goals with respect to support of this format is to get something stable and working, such as displaying all the notes in the parts with no additional (e.g. articulation) information and then add a pull request, and build on top of that. At the start of the week my goal was to create the correct number of parts, which some basic meta-information, as they were described in the file, and this was finished. I wrote a little last week about the problems I was encountering at the time with that, where clicking on any given measure would result in a segmentation fault, and I made the hypothesis that this was because there was no information in the measures, and this has indeed turned out to be the case. I verified this initially by creating rests in every measure of every part, and then starting playing around with the parts by adding notes etc. and things seem stable now.

After the issue with measures had been sorted out, I then started to look at the “score.gpif” file and determine how that information was all put together. The connection of bars to parts was a connection that was a little awkward to make, I had expected something perhaps a little more rigorous that simply depending on the order that the parts were defined in. Once this was correctly determined, I implemented functions to complete the parsing of the “score.gpif” file such that:

  • The link from parts, to individual bars, to voices, to beats, and then to rhythms and notes was followed, so information exists in my local copy of the implementation about everything needed to add notes to parts;
  • Warnings were generated for any XML tags that I simply skip right over for the moment (such as information about bends – now is not the right time to consider that level of detail);
  • Information that was parsed was available in the right place in the implementation. This is still a source of work, as I gather up information about individual notes and rhythms at a point before they can be used. I do not wish to parse the file in any way a second time and I don’t want to hold the necessary information in an intermediate data structure, as I believe if things are structured correctly then neither of these will be required. This is going to be my starting point of work for next week.

In addition to this, I have spent some time familiarizing myself with how to add notes to specified measures, and give information such as duration etc, in the implementation. Given that this has been done, and information right down to the rhythm level has already been parsed as described above, after fixing the structure issues and getting the necessary information all in the right place with some kind of efficiency, there should not be much work left remaining before everything can be looped over correctly such that all notes are added to all parts, which is my current goal.

I haven’t spent any time this week on other Guitar Pro formats, such as fixing the issue with the test file for bends as described in my last entry.It might be the case that I’ll do that this week, it will depend on how progress is on the GPX format. It’s likely I’ll prioritize GPX support for the time being as I feel I’m making headway on that task and there is still more to be done.

Key tasks that stalled

The task which took longer than I expected to was understanding entirely the structure of the score.gpif file. Although most connections were immediately clear, such as the connection between bars and notes and rhythms, the connection between parts and bars was not so. Aside from this, nothing took particularly longer than expected. Adding support for other formats, such as fixing up the bends test so that it can be committed into the repository is something that I haven’t looked at this week. It’s possible I’ll get to that this week, if not, then certainly by the week after after some initial GPX support has been completed.

Upcoming tasks this week

As I’ve mentioned, I am going to likely be focusing on the GPX format. I would very much like all the notes in all of the parts to be displayed by the end of the week, and then the most difficult part of the work that’s associated with supporting the GPX format will all be done and out of the way. At that point I’ll be able to clean up and document the implementation so that others can get access to it, and then that will allow me to just focus on individual features, such as ties, bends, voltas, and other information.

Bends and GPX support

Overview of work this week

In the last post I talked about tempos and how the support for those were progressing. I have written tests to verify this feature is working as intended, and submitted a pull request for this feature which has now been merged into the Musescore master branch, so that bug is now closed as fixed. The support for voltas, which I mentioned last time I had made a pull request for, has now also been merged so that’s out of the way.

For the bug concerning the incorrect import of copyright text, tests have been created to verify this is done correctly and they are now also in the master branch of the Musescore repository, so the bug has been closed as fixed.

One of the areas I’ve been looking at this last week is support for importing bends. Bends are a common feature in Guitar Pro files, so support for this is important. There are various different kinds of bends that can be created in Guitar Pro (e.g. Bend/Release, Prebend, etc), all of which are now supported in the version I currently have on my local machine, including arbitrarily complicated bends. There is currently an outstanding issue with this work, and that is to do with the test framework. For some reason, the test files I have created are not processed correctly and the test framework fails – as there is nothing fundamentally different about these tests I presume that the reason for the problem is something specifically related to bends. I need to set aside some time to look into this error further, I expect I’ll do that this week. After I have managed to fix this issue with the test framework, I’ll push that to my fork on Github and add a pull request.

The other main task which I have spent time on is supporting the GPX format. GPX files are files which are compressed with a proprietary compression algorithm, and within that is a file system I have already decompressed the compressed GPX file, and extracted the files in the file system and dumped them to disk so that I can get an idea of what I’m working with. Most of the meta information is now handled (title, artist, etc), though there are some tags which aren’t supported yet as I need to figure out the right place to put them. My goal for the moment is to focus on just the notes themselves – details such as voltas and slurs etc. can be completed afterwards. This will allow me to get something that works to some extent, and is stable, into the master branch of the Musescore repository as early as possible. One of the files in the file system is an XML file named “score.gpif”, and contains the vast majority of the important information that needs to be processed. I have extended the implementation this past week to read instrument descriptions and create parts for them. Measure information is also handled, so we know how many measures there are in the score, and characteristics of those measures (such as clef and key signature), which is also extracted from the score.gpif file. There still is much work to do on the GPX format, and that’s what I’ll continue to look at this week.

I’ve also taken a look at the issue where fade-ins are not supported. I need to create an additional symbol in Musescore in order to do this. I think the best way to go about that is to make some kind of variation on hairpin.cpp, which describes how e.g. crecendos are drawn. Perhaps instead I want to use a bezier curve to describe the fade symbol. I intend to find time to get that issue sorted out this week too.

Key tasks that stalled

The bends feature is taking a little longer than I would like, that’s down to the problems in running the bends tests described earlier. I expect to work on that more this week so that this can get out the way.

There are a few issues also with creating measures in GPX files that’s taking longer than expected. Currently, clicking on measures which have been created makes a segmentation fault in Musescore. I’ve played about with this a fair bit, and I’m starting to think that’s because the measures have been created with no notes or rests, as that information hasn’t been parsed yet so I can’t fill the measures with the right information. If that’s the case, this should just clear up this week as I make progress with the implementation of support for GPX files.

Upcoming tasks this week

This week I primarily intend to continue working on the support for the GPX file format. My hope is that I can parse all the notes, rhythm descriptions etc. in the score.gpif files and display them on the score. I’ll look also at the issue with guitar fade-ins described previously, after the symbol is ready to use it shouldn’t be too complicated to finish off importing the Guitar Pro file correctly. The bends issue I also would like to get out of the way, as the support for that has been completed and I should just need to find a way around this problem that’s currently appearing with the test framework. I’d like to spend some time looking at some of the existing other issues too however, just so I know what’s around the corner…

Voltas, Tempo markings, and Copyright statements

Overview of work this week

I’ve looked at the existing issues reported on the Musescore bug tracker this last week, looking specifically at the below issues. I’ll talk about each one in turn.

#21732 – Multiple tempo text not imported correctly

This bug report describes the current situation of importing tempo markings in Guitar Pro files. At this current time of writing, the master branch largely ignores tempo markings for Guitar Pro related files. In general, initial tempo markings are supported for the beginning of the music but any tempo changes which occur later are ignored. There are two components to this, the first is importing the text marking, and the second is configuring Musescore correctly so that when playback reaches a bar with a tempo change marked it changes the rate of playback to match.

In my local copy of the repository this issue has been fixed for files in the Guitar Pro 3 format. I still need to write a test for that and then verify that Guitar Pro 4 and 5 files operate the same way and apply the work I have completed already to that. Nevertheless, familiarizing myself with the necessary part of Musescore has been completed and I expect that I’ll make a pull request this week which fixes this bug in all available versions.

#22898 – Signs of first and second endings on repeats (voltas) are lost in Guitar Pro files

I had been working on this issue last week, just before the GSoC students were announced, so I have continued work on this feature. Last week, I made a pull request which has been merged into the master branch of the Musescore repository such that voltas are understood in Guitar Pro 5 files. This week I have continued to look at that, and added the work for Guitar Pro 4 and 3 formats.

Note that different versions of Guitar Pro process voltas differently. In Guitar Pro 5 files, the bit sequence present in the file represents which repeat numbers that volta applies to. In the 8-bit sequence of numbers, each 1 or 0 represents whether each digit (1 to 8, reading right to left) should be used as a repeat number in that volta. For example, for the bit sequence 10010, the created voltas should have repeat numbers 2 and 5.

This is not the case in previous versions of Guitar Pro. In previous versions, these are represented as binary numbers. As this is the case, it is not possible to create repeat numbers in voltas which are not sequential. For example, it is not possible to create a volta which has repeat numbers 1, 3, and 4 – it must also include 2. Also note that volta numbers must be remembered, for if say in the GP4 format we create a volta at bar 2 with repeat number 3, and then a volta at bar 3 with repeat number 4, then we should generate repeat numbers 1,2,3 for the volta at bar 2 but just repeat number 4 for bar 3 and we should not reset the counter. There are cases in the GP4 files where the volta counter can be reset. These aren’t immediately obvious, and I’ve described this below.

If an opening repeat sign is present in any given bar, the volta counter is reset on the bar *after* the one containing the opening repeat sign. If a closing repeat symbol is detected in any given bar, the volta counter is reset only if the bar after the closing repeat symbol does not contain a volta. Otherwise, the counter is not reset. In the case where opening and closing repeat symbols are both used in the same bar, then the volta counter is reset on the bar after the one containing the opening repeat sign. The way that voltas operate was discovered with a fair bit of fiddling around with Guitar Pro and exporting and then re-importing the result.

Without any repeat signs, voltas are always numbered consecutively, even if that is not what the user has input. For example if in the Guitar Pro software the user has a volta on bar 2 with repeat numbers 1,2,4, and a volta on bar 3 with repeat numbers 3 and 5, then after the file has been saved and reloaded (guitar pro doesn’t stop the user from creating voltas with these numbers), the volta on bar 2 with have repeat numbers 1,2,3,4 and the volta on bar 3 will have repeat number 5 only. If the user had specified repeat numbers 1,2,5 on the volta in bar 2, as voltas must be numbered consecutively, the volta on bar 3 will be ignored, as the volta in bar 2 has repeat numbers 1 to 5.

Note there are strange occurrences when voltas are ignored. Consider the case where we have a volta which is ignored on the bar after an ending repeat symbol as voltas must be consecutive. Is the volta counter reset or not? In this event, the following takes place in Guitar Pro 4. The volta is ignored, and the counter is not reset as there was a volta after the repeat symbol. However, when the file is next saved, it will no longer contain the volta, and so saving and re-opening the file will mean that the volta counter *will* be reset. This means that in Guitar Pro for some score on the screen A, saving and loading can produce a different score B (where a volta is ignored but the volta counter isn’t reset) and saving and loading a second time will create score C (as the previously ignored volta doesn’t exist any more, the counter is reset)! Naturally, such a property is not something we want in Musescore.

For Musescore, we should just display the same as Guitar Pro. As we don’t have to export guitar pro files, we won’t suffer from the problems with saving and loading the file and getting a different result. That said, we should simply not display voltas with numbers which have already been used in previous voltas where the volta counter has not been reset, but we should not reset the volta counter, as technically a volta still exists and it simply isn’t shown. This will allow us to replicate exactly what is shown in Guitar Pro.

I have added a pull request containing support for voltas for all available versions of Guitar Pro, I imagine that will probably get merged some time this week.

#23077 – Copyright text not imported

It looks like this bug has already been fixed earlier this month, so I have written some tests for that so that we can close the bug report. Should be straightforward, nothing complicated here, I would think that I’ll push the tests for this in the next day or two.

#22436 – Note not heard

This is flagged as a Guitar Pro bug but I think the problem may be in a different area of the code base – the note seems to be imported correctly so I think we don’t have a way of playing that note on that instrument. It’s possible that the problem could simply be that we don’t seem to assign instruments to parts while importing Guitar Pro files. Any note above C_4 don’t seem to play, whether this is something that is indeed due to a bug in the import process is something I I have to check out. I’ll look into this further this week.

Key tasks that stalled

For the work on voltas, volta numbers do not seem to appear when running Guitar Pro through Wine so I lost a little time there. I have instead installed Windows on Virtualbox and and running the Guitar Pro software directly through that, which makes things a little easier for me.

With respect to the different versions of Guitar Pro, I also spent a bit of time looking into supporting formats earlier than GP3, to see if that is possible. From the searching I have done, I think it’s best to stick to the original project specification and support GP3 upwards, as versions of Guitar Pro that can read that format are unavailable for purchase, and there is no formal description of that format. The GTP format (used in Guitar Pro 1 and Guitar Pro 2) seems also to be almost entirely unused, so I shall likely be working on Guitar Pro versions 3 upwords during this project.

Upcoming tasks this week

I’ll be adding the tests for the copyright bug, so that can be removed from the bug tracker. I also plan to push the changes for tempo markings so that bug can be dealt with as well. The bug about playback discussed previously is also something I’d like to look into. If there is time before the middle of the week, which I expect there will be, I should be able to take a look at another couple of bugs.

This week, I also intend to go back to the Guitar Pro 6 (GPX) format and continue work with that, which I haven’t looked at for about a month since I started looking at existing bugs. There is quite a lot of work to be done there, but I would like to see some notes starting to appear on the staff so I can start thinking about making a pull request. I’ll go more into detail on that in the blog next week, by which time I’ll have re-familiarized myself with precisely how GPX files are composed and how we want to handle them in Musescore.

GSoC Introduction

As part of being selected to participate in Google Summer of Code 2014, this blog will describe progress on my chosen project. The aim of this project is to enhance and test the support for Guitar Pro files in Musescore.

Within the last week I submitted a pull request which added support for voltas in the .GP5 (Guitar Pro 5) format. I’m going to start by working further on that area, and add support for voltas in other versions of Guitar Pro. After that is completed I’ll choose another one of the bugs that have been filed against Guitar Pro on the Musescore issue tracker at [2] and work on that. I expect to be working on existing bugs with the currently supported formats for the next week or so. Work on the (currently entirely unsupported) GPX format (the format used by Guitar Pro version 6) has been started already and what I have done for that so far has been pushed to my fork of the Musescore git repository at [1]. Currently, I’m planning to continue working on the GPX format at the beginning of May.

My plan is to update this blog every Sunday so that it’s easy to see what I’m currently working on. That said, I’ll get started!

[1] https://github.com/jpirie/MuseScore
[2] http://musescore.org/en/project/issues