Opensource editor for SP/LP (was: sysex reverse-engineering)

Everything Phatty.
Post Reply
jp-ma
Posts: 5
Joined: Wed Oct 21, 2015 11:39 am

Opensource editor for SP/LP (was: sysex reverse-engineering)

Post by jp-ma » Wed Oct 21, 2015 11:56 am

Hello. I have very recently acquired a Slim Phatty and by doing so I am new to the world of Moog (and quite excited as well).

I want to contribute by making opensource editor software to interoperate with Linux and others.
That requires an understanding of the sysex protocol, so I have looked a bit into it. Before doing the big reverse-engineering job I wonder if there is any info or specification somewhere. There is the documentation of the Voyager which I am aware of, but does not appear related.

I will set up online a source repository when I advance on the project, and I will keep the thread updated.
I am curious to see if this idea will generate some interest.

--
Update: I have began to develop an editor software. Linux source code is available.
Last edited by jp-ma on Mon Oct 26, 2015 8:45 am, edited 1 time in total.

denoiser
Posts: 67
Joined: Sun Feb 11, 2007 7:23 am

Re: Specification of sysex format

Post by denoiser » Thu Oct 22, 2015 2:20 pm

jp-ma,

I was thinking exactly the same thing, a Linux editor.

In fact, I thought of officially asking Moog for the sysex implementation now that they discontinued the Phattys. Moreover, It would really be nice if moog could open all the Phatty related docs and even the source code. I read some weeks ago that Moog was going to be sold to someone, so if there is a time to ask for either documentation or source code it is now.

Any words on this, Moog?

Kind regards, Denoiser.

User avatar
BrufordRules
Posts: 101
Joined: Sat Feb 26, 2011 3:03 pm

Re: Specification of sysex format

Post by BrufordRules » Thu Oct 22, 2015 4:26 pm

As far as I know, Moog never public the sysex implementation of their devices.

The Moog MP-201 users like me are still waiting for the sysex info to create an Editor for this device. An Editor that never saw the light.

:cry:

User avatar
Stuka
Posts: 469
Joined: Fri Oct 12, 2012 10:13 am
Location: Knoxville, TN

Re: Specification of sysex format

Post by Stuka » Thu Oct 22, 2015 8:05 pm

There are several pages of MIDI CC commands published for the Little and Slim Phatty -
Just about anything that can be done via the menu system can be done with MIDI CC's, so the lack of SYSEX documentation shouldn't prevent you from developing a decent editor.
(I have an editor I developed for these Phattys using the MIDI Designer app on an iPad - works great!!)

I'm not sure if it's been posted on the Moog site or not, but there's an OS 3.1 operating system addendum that documents all the latest MIDI CC commands.
You can probably request that from Moog tech support.

As for Moog being sold, that already happened this year -
It is now a fully employee-owned company.
Very cool!! 8)

I don't think we have to worry about Moog going anywhere anytime soon!! :)
Little Phatty Solar CV, Minitaur, Slim Phatty (x2), MF-101, MF-102, MF-103, MF-104M, MF-105M, MF-108M, CP-251 (x2), Animoog app, Filtatron app, Model D app, Model 15 app...and some 5U modular and Doepfer stuff...

jp-ma
Posts: 5
Joined: Wed Oct 21, 2015 11:39 am

Re: Specification of sysex format

Post by jp-ma » Fri Oct 23, 2015 1:02 am

All right so I have collected I think 99% of info I need, so I am pretty confident it is possible to make an editor with all features. This is a technical report of the current situation. Code source will follow probably soon, license to be decided.

Sysex presets: Read OK, partial write

I have found all patterns for the name and controls, after a long bit diff'ing session, so reading is fully supported.
Actually the panel format is the same as the preset, so it is effectively possible to receive for the Phatty the panel, as well as the Nth preset.

To modify the values in the sysex actually does not work correctly however, because it messes up the lights of the front panel, misteriously. Beside that, it does sound correct however.

The bank format is not supported. I am pretty sure there is some sort of trivial bit-swapping obfuscation going on in the messages. If you look in the file you can observe preset 1 appears in clear but the next ones are not.
I have not figured it out this coding and I don't think I will bother. I would rather a different format for bank and I am sure I con automate conversions from the original format.

Control Change: To do

I think I will workaround the sysex writing problem with CC as Stuka has suggested. I looked as the OS v3.1 documentation.
In fact the CC messages have a 14 bit variant, as I have found from monitoring the MIDI output, so I will just use that.

The Code

A C library which is currently in focus of developement, and a graphical program in Qt which I would not dare to call yet an "editor". The MIDI interface is programmed in ALSA, but is isolated properly, so surely it can be easily reprogrammable in Windows/Mac/BSD API.

That's about it folks so keep updated for developments.

jp-ma
Posts: 5
Joined: Wed Oct 21, 2015 11:39 am

Re: Specification of sysex format

Post by jp-ma » Fri Oct 23, 2015 1:15 am

Moreover, It would really be nice if moog could open all the Phatty related docs and even the source code. I read some weeks ago that Moog was going to be sold to someone, so if there is a time to ask for either documentation or source code it is now.

The source code would be awesome, I guess can we can always dream.
That would very much be in "Bob's spirit", as Moog like to says. But it will surely never happen.

jp-ma
Posts: 5
Joined: Wed Oct 21, 2015 11:39 am

Re: Specification of sysex format

Post by jp-ma » Mon Oct 26, 2015 8:39 am

I have published source code for my current version (pre-alpha, way incomplete). Currently Linux only and no binary distribution.
https://gitlab.com/jp-ma/phatty-editor

goodweather
Posts: 17
Joined: Sun Jun 21, 2015 10:50 am

Re: Opensource editor for SP/LP (was: sysex reverse-engineer

Post by goodweather » Tue Oct 27, 2015 4:12 pm

Hi JP,
instead of building somehitng from scratch (which is fun of course :D ) maybe that you should look at Ctrlr that enables you to build any kind of panel for Midi devices.
It is based on Juce but uses the Lua language for coding when necessary.
I'm also an owner of the Sub37 and I'm building a panel for it.
Many things are already working fine (look and feel, buttons, EG graph display, current parameter and value display...).
In the coming days I'll add the OSC waves display and also make the EG graph smaller in order to display both enveloppes at the same time.

I was going to start looking at the sysex dumps (buffer content and presets) so I was interested to get a documentation of your sysex analysis.
What I'm looking for is:
- what is the sysex command to request the buffer dump from the Sub37?
- what is the sysex command to request a specific preset dump from the Sub37?
- what is the content and position of each parameter in that dump?

If someone at Moog reads this, any help and info would be appreciated. All synths manufacturers are providing that kind of info publicaly and most of them are also giving editors for free.
When you produce such nice piece of equipment as the Sub37 which is also not so cheap, I think it is a pity not to provide full documentation to your customers.
But of course I also perfectly understand the commercial aspects behind those decisions.

I join you a picture of the panel I'm building.

Thx in advance!
Attachments
Sub37 panel - In work.JPG

jp-ma
Posts: 5
Joined: Wed Oct 21, 2015 11:39 am

Re: Opensource editor for SP/LP (was: sysex reverse-engineer

Post by jp-ma » Wed Oct 28, 2015 4:04 pm

Hi JP,
instead of building somehitng from scratch (which is fun of course :D ) maybe that you should look at Ctrlr that enables you to build any kind of panel for Midi devices.
It is based on Juce but uses the Lua language for coding when necessary.

I have manipulated JUCE a bit already and indeed it is a great piece of software, I did not know about Ctrlr however. I must say good job on your interface it looks pretty neat.
JUCE is king of fancy gui but a bit out of my confort zone, right now I keep with tech I'm familiar and avoid a learning curve. Focus is on the feature set I want to implement.

I was going to start looking at the sysex dumps (buffer content and presets) so I was interested to get a documentation of your sysex analysis.
What I'm looking for is:
- what is the sysex command to request the buffer dump from the Sub37?
- what is the sysex command to request a specific preset dump from the Sub37?
- what is the content and position of each parameter in that dump?


I am working on the Slim Phatty, not Sub37, which I do not own. I have no idea if still my info may be valuable to you but I will explain. (refer to my source code in git)

I use two sysex methods, from phatty.c in source. The first one requests the panel, and second is for presets in range 0-99.

Code: Select all

const uint8_t sysex[] = {
    0xf0, 0x04, 0x05, 0x06, 0x03, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0xf7 };
  const uint8_t sysex[] = {
    0xf0, 0x04, 0x05, 0x06, 0x04, preset_num, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0xf7 };


(If you have an editor on Windows, you can use APImonitor and set your breakpoint on API midiOutLongMsg. Inspect the buffer. It's how I do it. MIDI-OX did not work for me.) If not possible, messages are simple so you may want to try brute force, seeing what generate replies.

About the content of the dump, you must look at tools/codegen-fmt.lisp in source. It has a pretty explicit description of the index and the bit patterns. I do not write the C code to read and write directly, I generate code off lisp language to simplify the work.

For example of how to read it:
("gate_mode" #x17 (#b1 #b11100000))
A parameter of size 4 bits spanning over 2 bytes starting at index 0x17, whose bit positions are given by the masks.

I find the form of the sysex dump is something like : 0xf0 <header> <zero-padding> 0x01 <message> 0xf7
The size of zero-padding varies, so I skip the zeros and position my byte-offset 0 where the 0x01 is. It is the signification of the byte-offset in lisp code above, it is not from start of sysex.

I hope this information is clear and it may help you. Feel free to reuse my source however you like.

Post Reply