Interacting with Live's API: M4L vs Control Surface Scripts

[metastatik]

The following video compares the two methods (M4L and Control Surface Scripts) of interacting with Live's API. There are many misconceptions about the latter method and I hope this video clears at least some of those up. I'd be interested in hearing your thoughts on the comparison and any remaining questions you have on it.

https://www.youtube.com/watch?v=6j4weQ0OSGI

[hoffman2k]

Neato, the thread I've been waiting for
Nice video! I suppose both have their ups and downs. I could never get into the python/js side of it, though since we got dictionaries it has been easier to cut down on objects in favour for a set of instructions.

So seeing your Launchpad example needing the external MIDI output reminded me of a question I had for the python wizards.
Can python access all available MIDI Inputs or only the one that is selected as control surface?
Can you bypass any parsers and just get the raw MIDI values in the 0-255 range?
Can it send from Python to a localhost UDP which can be picked up from MFL and vice versa?

Having had my fill of incomplete/buggy/deprecated solutions to get unrestricted MIDI into MFL, I just made myself a patch that sends Raw MIDI over UDP which goes straight into a MIDIparse in MFL. Which means I can get MPE inside Live. And it works without a hitch.
I wondered if Python could do the same, which would forego the need for an extra patch. You'd lose one control surface slot but be able to control more than 6.

[metastatik]

Can python access all available MIDI Inputs or only the one that is selected as control surface?

Control surface scripts only have a single MIDI input each. That could, of course, be a virtual MIDI port connected to some sort of merging software. There are also MIDI libraries for Python that could potentially access multiple ports from inside of a control surface script. Haven't tried that myself, but it could work.

Can you bypass any parsers and just get the raw MIDI values in the 0-255 range?

You can't bypass Live's filtering of MIDI messages AFAIK, but control surface scripts don't have the same restrictions as tracks do. In particular, they can send/receive SysEx. They can also receive poly AT, but that's currently only used in conjunction with note repeat. There's some discussion about how that could be re-purposed with M4L here.

Can it send from Python to a localhost UDP which can be picked up from MFL and vice versa?

I would think so as Python has a module (socket) for it and OSC works fine, but haven't tried it myself.

[hoffman2k]

Thanks for the replies Stray!
I suppose using a MIDI Library within Python would result in similar problems like with the externals. It'll be based on some open thing like RT MIDI and then won't get updated because the original developer moved on to other stuff.

I tried using Apple's Network MIDI by tricking it into connecting to a bonjour host. But it turns out that Apple's Network MIDI is about the buggiest part of OSX. You essentially need a reboot after every connection and its trying way to hard to stay synced by flooding clock signals...
When I have a free weekend, I'll turn my UDP Patch into an App. Its stable, doesn't drop a single byte and its about as minimal as it gets in terms of code. [midiin] > [UDPsend] and vice versa. The only extra work it'll require is a UDP connection to a [midiinfo] object so you can fill a menu with port names.

Which actually prompts a next question: Does PXT care what port is chosen in Live? Lets say I have an App that gives complete control over whatever message Push sends out, could I add support for custom scripts by just making Push send over a different port? Ie. Push would connect to the App with the User Port and be able to send its data to say the IAC Bus, the App basically being in Thru mode. Giving the user at least 3 modes and even the option to switch between custom scripts.

[metastatik]

I suppose using a MIDI Library within Python would result in similar problems like with the externals. It'll be based on some open thing like RT MIDI and then won't get updated because the original developer moved on to other stuff.

Definitely not comparable at all as the MIDI libraries available for Python are used by lots of people (unlike the externals for M4L you're referring to, which are only useful to a small portion of the M4L user base) and are mostly open source so they can be (and often times are) maintained by multiple contributors.

Which actually prompts a next question: Does PXT care what port is chosen in Live? Lets say I have an App that gives complete control over whatever message Push sends out, could I add support for custom scripts by just making Push send over a different port? Ie. Push would connect to the App with the User Port and be able to send its data to say the IAC Bus, the App basically being in Thru mode. Giving the user at least 3 modes and even the option to switch between custom scripts.

Ports are hard coded in PXT-General, but not in PXT-Live/PXT-Live Plus. So the ports you use don't matter in the latter case. However, the scripts are disabled while Push is in Live (default) Mode and only enabled when Push is in User Mode. Push sends out the mode it's in via a SysEx message. So your app would need to send out that SysEx message to enable/disable the scripts. That message is covered in the Other SysEx section of this document.

[hoffman2k]

Ah, cool. Yeah, I know which message you mean. My app works the same way, makes sense that its the trigger for the script.
Good point about the frequent updates on the many libraries. Thats the issue with the externals really. They're based off an open library which gets updated but the externals don't.

Thanks for the document. I was missing the strings for the touch strip LEDs. Nicely documented!

[chapelier fou]

Cool topic, but from my point of view (cannot code), writing in python or in javascript are both out of reach !

[Schmidi]

There actually is an app for that (easy Python scripting) https://remotify.io/ I've only tried the free version, but it was as easy as can be for a non programmer.

I'd first check out www.nativekontrol.com as Stray's scripts are great if you have a supported controller.

[metastatik]

Cool topic, but from my point of view (cannot code), writing in python or in javascript are both out of reach !

I would think that using standard Max objects for this sort of thing would be much more convoluted compared to using JS, no? Either way, it would be interesting to see how you (or anyone else who prefers Max patching to JS) would implement this functionality with standard Max objects.

[metastatik]

There actually is an app for that (easy Python scripting) https://remotify.io/ I've only tried the free version, but it was as easy as can be for a non programmer.

I'd first check out http://www.nativekontrol.com as Stray's scripts are great if you have a supported controller.

Based on the feature list on their site, the functionality shown in the video (and lots of other functionality you might want to have in a control surface script) is not possible with Remotify. It is, of course, possible with some of our products; notably Arsenal and ClyphX, neither of which requires any programming to use. The point of the video was just to show that control surface script development is not as complicated as many think it is. That said, doing this sort of development (in either control surface scripts or M4L) is not for everyone.

Thanks for the endorsement BTW!

[hoffman2k]

Cool topic, but from my point of view (cannot code), writing in python or in javascript are both out of reach !

I would think that using standard Max objects for this sort of thing would be much more convoluted compared to using JS, no? Either way, it would be interesting to see how you (or anyone else who prefers Max patching to JS) would implement this functionality with standard Max objects.

In what would seem like a convoluted manner to a traditional coder. But I'd still make it work somehow.
The main difference being, don't ask me to modify something I wrote years ago. I'd sooner rebuild it than figure out why I have a wire going from that thing I once fixed but didn't comment the why and how.
Every time I try learning a code, I end up going back to Max with new knowledge and methods, because it still makes more sense in the context of objects and wires opposed to classes and functions.
Its just how my brain is wired. At this point I'm just hoping gen~ keeps evolving so I can port whatever patch to whatever hardware. Nearly there for data, already there for audio. Unless you output your data as audio...

Your video has been very valuable though. For those who are well versed in the API control from Max, its interesting to see the contrast with object based patching. And you show 2 examples. Even with JS being such a big part of Max, I always made due without it. For Arrays, I used to use Jitter. Hell, in some cases I still do. But now we have dictionaries in Max, I can see where traditional coders had the advantage. A dictionary is like a class with subclasses and arrays and you can cycle through them at audio rates with a minimal computing cost. So my patching style evolves again, but now its starting to overlap with the traditional methods. If C74 somehow can get dicts into Gen~...

Anyway, there are at least 3 ways to interact with the API for varied levels of skill. There are things like remotify and clyphx that make it even easier. And its not going to stop evolving. Too late to go back for Ableton, they have 2 products dependent on the API.
Sure, some updates break scripts and MFL devices. But it usually comes with new features to exploit.

Stray, Do you have any experience with the Live file formats? I understand that at some point it switched over to XLS. In the meantime, Korg Electribes and their new online training course have been able to export .als files.
I'm wondering how far manipulating such .als exports is removed from messing with the API. I'm guessing all relevant data is just sets of arrays. It could be nice to generate a set based on a set.

[metastatik]

In what would seem like a convoluted manner to a traditional coder. But I'd still make it work somehow.

FWIW, I started out with stuff like MIDI Translator, Max and SynthMaker many moons ago and could certainly figure out a way to implement what was shown in the video strictly using Max objects. But, at least from my perspective, it would be far more complicated and so wouldn't be worthwhile.

I'd sooner rebuild it than figure out why I have a wire going from that thing I once fixed but didn't comment the why and how.

That's my experience exactly. These days, if I'm going to spend time on something that I know I'll need to update (which is the case with virtually anything that's useful), I ensure that I make that as easy as possible for my future self. I have no ideas how to do that with conventional Max patching, but find it to be easy in traditional programming languages.

Every time I try learning a code, I end up going back to Max with new knowledge and methods, because it still makes more sense in the context of objects and wires opposed to classes and functions.
Its just how my brain is wired.

Understandable and that touches on an important point. Learning how to do things in traditional programming languages translates to virtually every other programming situation including non-traditional programming environments like Max. The converse is not true. I would've never learned how to do some of the advanced things I do these days had I not started learning traditional programming concepts. I really don't think a person could/would learn those concepts using an environment like Max or MIDI Translator.

Stray, Do you have any experience with the Live file formats?

I don't as it's not something I've ever had to muck with.

[JuSchu]

I have quastion for you too.
I started with the Python API with Live 7. Since Live 9 I have to restart Live to reload the script after I edited it. Before Live 9 I only had to go to the settings an deactivate an activate the script.
Do you know if there is a way to reload the script without restarting Live?

The converse is not true. I would've never learned how to do some of the advanced things I do these days had I not started learning traditional programming concepts. I really don't think a person could/would learn those concepts using an environment like Max or MIDI Translator.

I terms of Max (or Reaktor) I agree. But a few months ago I played a little bit with the Unreal Engine 4 and its Blueprint Patches. I know many programing languages and the Blueprints are really close to an actual programming language.

[metastatik]

Yes, just load a new set or reload the current set.

[JuSchu]

Thanks, that will save me a lot of time since I still don't have a SSD in 2017.