Expanding the Possibilities

Once I realized that my code for generating DX7 sounds (see previous entries here and here) was actually working and is indeed able to create interesting and useful sounds – much to my surprise, I might add – it has become a matter of adding more and more features for refining the sounds once they are downloaded to the synthesizer.

Don’t misunderstand me – the majority of sounds are still useful as-is, but I hate to have to bring them into an editor to fix an output level of 98 or notch up the fine frequency from 1.48 to 1.49 to make things a bit more in tune, so I added a bit of code to enable polishing of sounds that need a tiny bit of care before being truly useful. While doing this, I also started researching parallel avenues for my program. What other synths could be a prime target for this type of patch creation?

As it stands, most other synths today either have a too small pool of created (good) patches to learn from and/or the data format has not been made official. Such is the case for such obvious targets as the Digitone by Elektron (super-secret classified documentation of System Exclusive format) and even older gear such as the ubiquitous Yamaha TX81Z, which according to the lackluster amount of patches online seem to have been destined to only provide Lately Bass to the world ever since.

Next, I looked into DX7-compatible synths. Obviously software emulations such as Dexed will load up my patches without complaints, but that’s no challenge. It would be interesting to test the patches on a Kodamo EssenceFM but that will have to wait until I can bribe my friend into lending me his. What about Yamaha’s own SY77?

As it turns out, I have the module version of the same synth, the TG77. It has been sitting at the lower rung of my rack for a couple of years now since buying it from a good friend, and haven’t found much use – yet. It’s a six operator FM synth engine with vastly superior architecture than the DX7 – surely it must be able to digest DX7 patches?

Unfortunately, Yamaha failed to see the market potential in allowing their new flagship synth be compatible with the vast accumulated sound pool of its predecessor, and from what I could gather through a cursory internet search, most of the converted sounds were made manually, adjusting the levels and envelopes by ear. That’s not what I had in mind.

Fortunately, I was in luck. Again, someone had done work before me, and this time, the giant whose shoulders I could climb is named Derek Cook, and he was actually kind enough to publish his findings in a paper, aptly named Yamaha DX7 to SY77 Conversion Guide which I shall link to here. Armed with this newfound knowledge, as well as the official Yamaha SY77 MIDI Manual, I started work on adding support to generate SY/TG77 sounds as well!

Alas, it was not to be an easy task, and I will share some of the journey here for your enjoyment.

In the Footsteps of Champollion

Let me start by saying: The MIDI System Exclusive format of the Yamaha SY77 will at times make the Rosetta Stone pale in comparison. A patch on the SY77 can take on nine different shapes, each having its own version of the sysex format. This means for instance that the sysex data for different patches in the same bank will be of different length!

It soon became clear that this has historically been a major stumbling block for people wanting to write their own editors or patch generators for the SY77, and I encountered several forum threads where someone would start out with enthusiastic questions about specific details regarding the format, only to become less and less motivated as the discussions went on.

What’s even more problematic is that the documentation is not structured in a completely logical manner. From Yamaha’s end, it is provided in the form of tables, such as the one shown here. While certainly not impossible to decipher (although I had to make educated guesses a couple of times as to how some of the values were bit-encoded into certain bytes) they leave little clues as to in what sequence the aforemented tables should be included into one another. You see, hints are included in the form of indicating that a block of data starting with a volume level goes “here and for thirty bytes” but instead of clearly referring to a table, you are on your own. Are effect settings part of a patch or not? For the longest time I imagined pan envelopes – yes, it has envelopes to control panning, because why wouldn’t you – were a part of the patch, but alas they are not. I finally resorted to comparing two instances of hexadecimal data, one for my debug output and the other from a real patch that would load properly, leaving me cross-eyed for the rest of the day.

The formatting of sysex data is however not the only thing that is slightly cryptic with the way the SY77/TG77 generates sound. Envelopes are really powerful, and have the option to loop. They don’t map 1:1 to DX7 envelopes (as expected), neither does output levels, and algorithms requires a rant of their own. In all of these cases, the conversion guide by mr. Cook came in extremely handy, providing clear and colorful examples of how SY algorithms are mapped to their DX7 counterparts through the equivalent of virtual connections between operators, and even with tables showing the not always linear mapping between the 0-99 values of the DX and 0-127 of the SY. (Apparently, you can even build your own algorithm in the SY77, through sysex trickery!) I am indebted to the author of the conversion guide, and I fear that I would have given up at this point were it not for the information provided. (Clearly, coffee or other brewed drinks is on me, should our paths ever cross! I will however deduct one such beverage for not providing the data in the correct order in the guide, thus tricking me into coding them in the wrong order first time. Ah, well.)

The Sound of Silence

Finally, I managed to shoot a working package of sysex data that seemed to form a coherent voice from my Mac to my TG77. Fortunately, I had already grappled with Yamaha’s checksum system and emerged victorious when learning how to remotely program the DX7, and the same algorithm is used here – sum up all bytes from byte six, modulo 256, and top off by taking the two’s complement of the sum, and insert it last. Why Yamaha even bothered with a checksum is beyond me (and even more unfathomable that some emulators even check it) but if you need guidance, here and here are useful links to examples.

But alas – the patches were quiet. Dumbfounded, I fired up the TG77 editor of Patch Base to inspect the actual data in a more convenient way (by this time, I started to get fed up with the unwanted yoga routine imposed on me by having to crouch down to floor level and illuminate the fading display of my TG77 with my mobile to see what was going on ever so often)

I could immediately see a few glaring errors, but it wasn’t until I inspected the four EG Bias values that the TG came alive. Clearly, they shouldn’t be set to -64, but to zero. (Derek Cook kindly provides guidance on how to roughly map the “key scaling” values to the more sophisticated EG bias, but as I couldn’t get it to work properly, I settled to zeroing the values for now)

Sound! Vaguely reminiscent of FM pianos! Success!

In the next part, the journey into the innards of the SY77/TG77 continues, and I will also provide generated sounds to download and try out for yourself!