Did anyone ever made a whole control surface in Javascript ?

Learn about building and using Max for Live devices.
chapelier fou
Posts: 6002
Joined: Mon May 15, 2006 12:15 pm

Did anyone ever made a whole control surface in Javascript ?

Post by chapelier fou » Fri Jul 01, 2022 1:55 pm

So, I've been studying the conception of control surfaces with JS inside M4L for a few weeks now.
I managed to test more of less all the functionnalities that I need, and everything seems to work. At least at the 'trial' stage.
But things are looking less promising in real world scenarios...
I'm getting lags and poor accuracy. For instance, even with just one fader controlling one track volume, it seems to behave correctly at first, but then, if I move the fader a few times quite fast, it's getting very laggy.
I'm starting to wonder if javascript is slow somehow.
Any thoughts ?
Last edited by chapelier fou on Fri Jul 01, 2022 8:08 pm, edited 1 time in total.
MacBook Pro 13" Retina i7 2.8 GHz OS 10.13, L10.0.1, M4L.
MacStudio M1Max 32Go OS 12.3.1

chapelier fou
Posts: 6002
Joined: Mon May 15, 2006 12:15 pm

Re: Did anyone ever made a whole control surface in Javascript ?

Post by chapelier fou » Fri Jul 01, 2022 4:21 pm

Well,
I made a comparison between JS and M4L both accessing to a control surface. Just 2 faders mapped to track volumes.
The JS part sucks. The plain M4L part works tight.
Damn, I'm disgusted.
MacBook Pro 13" Retina i7 2.8 GHz OS 10.13, L10.0.1, M4L.
MacStudio M1Max 32Go OS 12.3.1

S4racen
Posts: 5821
Joined: Fri Aug 24, 2007 4:08 pm
Location: Dunstable
Contact:

Re: Did anyone ever made a whole control surface in Javascript ?

Post by S4racen » Mon Jul 04, 2022 9:36 am

With M4L running on a low priority thread will it ever have a satisfactory response time, especially when manipulating numerous parameters at the same time?

Cheers
D

chapelier fou
Posts: 6002
Joined: Mon May 15, 2006 12:15 pm

Re: Did anyone ever made a whole control surface in Javascript ?

Post by chapelier fou » Mon Jul 04, 2022 11:03 am

S4racen wrote:
Mon Jul 04, 2022 9:36 am
With M4L running on a low priority thread will it ever have a satisfactory response time, especially when manipulating numerous parameters at the same time?
Well, M4L in itself is indeed a bit slow and unpredictable, especially when accessing Live's API. But Javascript inside M4L is adds much much more slowness.
The same actions (like simply a control surface's control assigned to a track's volume) performed through js are considerably slower than with regular Max objects.
MacBook Pro 13" Retina i7 2.8 GHz OS 10.13, L10.0.1, M4L.
MacStudio M1Max 32Go OS 12.3.1

FrancoBlanko
Posts: 31
Joined: Thu Nov 14, 2013 2:19 pm

Re: Did anyone ever made a whole control surface in Javascript ?

Post by FrancoBlanko » Thu Jul 28, 2022 12:57 pm

Javascript always runs in the low priority thread in Max, so using it for real-time or time-critical applications is not advisable. What you're seeing is that a cascade of very quick calls to [js] will result in strange behaviour and lag, this is due to the scheduler throttling the messages.

It's ok if you use it for things like drawing UI elements (that's what MaxforCats do with Oscillot) or other algorithmic/recursive processes that don't need to be called very frequently.

chapelier fou
Posts: 6002
Joined: Mon May 15, 2006 12:15 pm

Re: Did anyone ever made a whole control surface in Javascript ?

Post by chapelier fou » Thu Jul 28, 2022 2:53 pm

FrancoBlanko wrote:
Thu Jul 28, 2022 12:57 pm
using it for real-time or time-critical applications is not advisable
Thanks I learnt this lesson the hard way.
Now 'learning' Python.
MacBook Pro 13" Retina i7 2.8 GHz OS 10.13, L10.0.1, M4L.
MacStudio M1Max 32Go OS 12.3.1

TimChandler
Posts: 7
Joined: Thu Sep 08, 2022 8:50 am

Re: Did anyone ever made a whole control surface in Javascript ?

Post by TimChandler » Fri Sep 09, 2022 11:26 am

Just to answer this question with "yes" and offer encouragement.

Performance of grabbed or registered controls is sluggish when the Max window is open, but you should find it's perfectly adequate when the max window is closed.

Here is an overview of the integration for Impact LX Mini which is written entirely using the js liveAPI in MaxForLive:
https://www.youtube.com/watch?v=aceHDteeFkM

iainduncan
Posts: 42
Joined: Tue Jun 12, 2012 6:06 pm
Location: BC, Canada
Contact:

Re: Did anyone ever made a whole control surface in Javascript ?

Post by iainduncan » Fri Sep 09, 2022 5:45 pm

I have written them in Python and now write them in Max in Scheme (through my project, Scheme for Max). The disadvantage of using Python is that you can't integrate with all the other things you can do with Max - but doing that kind of stuff does require care and a solid understanding of Max's threading model inside of Live. If you are not careful and you accidentally mix activity from the low and high thread, you can wind up with multiple accidental defers that can really add up to lag, or worse, break your patch in unexpected ways.

One nice thing about doing my work in Scheme for Max is that you can run a scheme interpreter in either the high *or* low thread, so one can have two in a device that send messages to each other.

Project is here in case you are interested in checking it out: https://github.com/iainctduncan/scheme-for-max

jbone1313
Posts: 578
Joined: Wed Dec 12, 2007 2:44 am

Re: Did anyone ever made a whole control surface in Javascript ?

Post by jbone1313 » Thu May 25, 2023 7:35 pm

TimChandler wrote:
Fri Sep 09, 2022 11:26 am
Just to answer this question with "yes" and offer encouragement.

Performance of grabbed or registered controls is sluggish when the Max window is open, but you should find it's perfectly adequate when the max window is closed.

Here is an overview of the integration for Impact LX Mini which is written entirely using the js liveAPI in MaxForLive:
https://www.youtube.com/watch?v=aceHDteeFkM
@chapelier fou did you see this reply from @TimChandler? Did you conduct your tests with and without the Max window open?

I am thinking about messing with jsliveapi for controlling multiple "slave" devices from one "master" device across multiple tracks. But, I would need the timing to be accurate. In the documentation, it says it is NOT possible to configure JavaScript functions to run in the high-priority thread of Max's scheduler, but in practice I am not sure what are the implications of that. I assume that means any audio or MIDI programming done with jsliveapi will not be guaranteed to have accurate timing.

I can see how, in @TimChandler's case, the timing would be fine for just twisting knobs on a synth, but when it comes to sequencing and note placement, I think accurate timing would be really important.

If anyone else can clarify this, then I would really appreciate it.

jbone1313
Posts: 578
Joined: Wed Dec 12, 2007 2:44 am

Re: Did anyone ever made a whole control surface in Javascript ?

Post by jbone1313 » Thu May 25, 2023 10:27 pm

For reference, I found this quote in another thread here from what appears to be an employee of Cycling 74. Emphasis mine.
pukunui wrote:
Mon Dec 14, 2009 11:33 pm
Threading considerations are pretty much out the window when it comes to the API. There's no such thing as running the API in overdrive, as we always have to wait around to get notifications back from the Live process. In fact, running API code at low priority usually makes it perform much better, kinda like Jitter.

It depends what you want to do. On a small scale, there is probably no difference between patching and JS. Once you get out to observing large numbers of properties, and managing a lot of state, the advantages of a procedural text language like JS mean you will save a lot of time and heartache writing your API functionality in that way. I have also observed that JS API code loads faster than its patcher equivalent.

jbone1313
Posts: 578
Joined: Wed Dec 12, 2007 2:44 am

Re: Did anyone ever made a whole control surface in Javascript ?

Post by jbone1313 » Fri May 26, 2023 3:16 pm

Finally, this link in the Max documentation sort of seals the deal for me. From what I can tell, using the Max Javascript API for things that require accurate timing is a no go.

Angstrom
Posts: 14921
Joined: Mon Oct 04, 2004 2:22 pm
Contact:

Re: Did anyone ever made a whole control surface in Javascript ?

Post by Angstrom » Fri May 26, 2023 3:57 pm

I wrote a sequencer where the logic was in JS and some of the graphical elements were too, but all the sequences and time critical features were run by live clips.
Basically how it works is a bunch of dicts getting pulled from a clip (notes etc), the dict is manipulated in Max then the result written back to the clip, or other clips ASAP. Then the notes are played by Live from the midi clip.

it works pretty well because the latency of the logical operations of something like "make all the C notes an octave higher" or "reverse the notes in this range" or "duplicate this sequence onto the other instrument channel and make it 4 bars - is not time critical to the millisecond. The playback handled by live the logic handled by Max. It works ok, apart from my spaghetti code because I was experimenting with all kinds of stuff at the same time.

My laptop is a touchscreen thing, so it was quite good as a performance sequencer.

I've been meaning to rewrite it, but we'll see.

jbone1313
Posts: 578
Joined: Wed Dec 12, 2007 2:44 am

Re: Did anyone ever made a whole control surface in Javascript ?

Post by jbone1313 » Fri May 26, 2023 4:40 pm

Angstrom wrote:
Fri May 26, 2023 3:57 pm
I wrote a sequencer where the logic was in JS...
Cool! Thanks for sharing that example. In my case, I think I need the MIDI processing to be done in "real time," and it seems like the JS thing won't work for that.

Angstrom
Posts: 14921
Joined: Mon Oct 04, 2004 2:22 pm
Contact:

Re: Did anyone ever made a whole control surface in Javascript ?

Post by Angstrom » Fri May 26, 2023 5:52 pm

jbone1313 wrote:
Fri May 26, 2023 4:40 pm
Angstrom wrote:
Fri May 26, 2023 3:57 pm
I wrote a sequencer where the logic was in JS...
Cool! Thanks for sharing that example. In my case, I think I need the MIDI processing to be done in "real time," and it seems like the JS thing won't work for that.
One way you might find is ro let all the max native objects do the time based actions , and provide the data for those actions via JS. For instance if you want to send a modulation curve sequence to a cc its quite likely that the generation and write of the curve can happen with latency ... as long as the read of that data is synchronised.
As I said, I used Dicts in my case. Then used native objects responding to accurate timing pulses to act upon that data.

I think of it like a two tier pipeline system. Only some items have maximum priority, while there are items which can be cached and served. If there are items which require realtime then analyse those use cases in isolation and find common solutions only those. A homogeneous one size fits all pipeline may cause issues. Separate the two tiers and see if that works

jbone1313
Posts: 578
Joined: Wed Dec 12, 2007 2:44 am

Re: Did anyone ever made a whole control surface in Javascript ?

Post by jbone1313 » Fri May 26, 2023 6:39 pm

Angstrom wrote:
Fri May 26, 2023 5:52 pm
its quite likely that the generation and write of the curve can happen with latency
Interesting. I would not mind the latency if it is compensated and the events eventually happen accurately given the compensation. Do you happen to know if the latency in this case is compensated?
Angstrom wrote:
Fri May 26, 2023 5:52 pm
Separate the two tiers and see if that works
I definitely understand the concept, and I appreciate you sharing. But, I am trying to think how that could work with my use case, and I am not seeing how unless, as I wrote above, the latency is compensated and the events eventually happen accurately. Specifically, here is a little bit about what I am trying to do:

I want to have a "master" device which controls a bunch of "slave" devices. I do not want to use the M4L mapping scheme, because a) I do not want to hard code my mappings (because reasons) b) I have found M4L mappings to be flakey.

I have implemented this in a MIDI remote script, and it works great with the Python API, but I am interested in implementing it in Max For Live, so that a) it will work with the new standalone Push b) I might be able to more easily to monetize it down the road. I.e., selling M4L devices seems much easier and better supported than MIDI remote scripts.

Fundamentally, I am wondering if, using M4L Javascript, a dial turn on a "master" device can trigger a dial turn on a "slave" device and have that "slave" device's dial turn be sample accurate and/or latency compensated. I am not looking for help with the implementation, as it seems straightforward enough. Rather, I am looking to understand the behavior given the way M4L Javascript runs with low priority.

I know I can do a POC, and I might, but still it would be nice to collect some more information.

Appreciate the help.

Post Reply