Ableton M4L and Jitter: a modular system for real time video

[[email protected]]

Hi Everybody,

I am building a modular environment using Max/MSP/Jitter on top of the Ableton's host (as M4L) for real time video integration (synthesis but also interaction with audio and video) to be used within Live.
To have an idea of what I do (I mean, an example of results) have a look to: http://vimeo.com/june74 or http://www.myspace.com/june74space.
For instance http://vimeo.com/19392757 is an example of a real-time capture of laptop camera and of audio from live (original music I compose) used to modulate video in OpenGL and audio back in live.

With modular system I mean a framework of M4L instruments/effects which offer all kind of macro/micro functionalities (many directly derived from Jitter's methods, many I created myself combining Jitter modules) and can be FREELY CONNECTED/COMBINED (on the fly, without opening the patch) in any form of chain or more complex derivation, exactly as you would do in Ableton itself for native audio/MIDI devices (so that you can build racks, automate them, etc.).

It all started as an experimentation, but in the last few months it expanded quite far and powerfully.
As now I am quite happy with the results and I think it is worth sharing.
May you may interested in the M4L devices, drop me an email (possibly in the future I will post them here and/or maxforlive.com / maxobjects.com).

More importantly (for me, at least), if you are interested in collaborationg on it (both on the creative and development sides) let me know. My main contact is: [email protected]

PS: I live in the Netherlands (Amsterdam), so if you like to meet, you know the coordinates.

Ciao,
Fabrizio

[[email protected]]

On this Cycling forum link a preview of the patches.

http://cycling74.com/forums/topic.php?i ... ost-156945

Gr.
Fabrizio

[kostrz]

Thanks for excellent work. Uhh. It seems we have now even video synthesis on board. Btw (I'm on mac osx) racks are missing. I can't load them (missing mxpat's in racks).

[[email protected]]

Hi kostrz,

Thanks for trying.
Concerning rack issues, it is just a matter of path.
I am going to fix it for next time.

Thanks.
Fabrizio

[[email protected]]

Hi

For who is interested in V-Module I included a download link (still a beta version, but already with a few fixes) and a manual in PDF on my own web site: http://www.fabriziopoce.com.
Soon I will post the devices at maxforlive.com.

Gr.
Fabrizio

[[email protected]]

Hi,

I just posted the very first release (v0.2) of V-Module which I (hopefully) can label as stable. It is on my web site http://www.fabriziopoce.com AND http://www.maxforlive.com

Relevant notes:
- Installation is now very easy:
1 - Copy Max scripts (JavaScript, Shaders, etc.) in your Max path.
(all stuff a live pack won't copy)
2 - Install the V-Module "live pack"
This includes all devices and seven extensive examples (as livesets).
- Documentation is also quite complete.
- I fixed all bugs I got aware of (thanks to you guys)
- Dependencies from externals should have been reduced to zero.

I think I will sit down for a while, as I actually worked quite a lot on it........ Anyway if you like have a go on the example files and let me know if something breaks (and of course what you think about it).

Kind Regards,
Fabrizio
http://www.fabriziopoce.com

[TriangelKlang]

hai Fabrizio

Thanks a lot. It's great what you achieved with this framework. I'm looking to your work since a week now, so since the first bèta with the doc.manual. Your skills are amazing, thank you so much for sharing. I wish that you can sit down for a while, peacefully and satisfied. Or otherwise make some noise with your kids. Cheers

[[email protected]]

All,

Has anybody tried the example livesets in the live pack of version 0.2?
In particular, any Mac user? Users on Live 8.2.2 / Max 5.1.8?

At this stage I am trying to see if things work properly after installation or if there is some issue I should be aware of.
Thanks if you try.

Gr
Fabrizio

[TriangelKlang]

Hai Fabrizio

Yes, I have tried, works terrific, also in Live 8.2.2/ Max 5.1.8

Vista, 2Quad, 2,33Ghz 3,00GB RAM

Cheers, M.

[[email protected]]

All,

Robert Jarvis (zealousy.com, the author of VIZZable) and I (V-Module's maker) are trying to start up a community on developing modular Jitter-foucused tools within Max4live.

If you are interested have a look to this post and let us know:
http://cycling74.com/forums/topic.php?i ... ost-158308

Ciao
Fabrizio

[[email protected]]

All,

In the meantime (besides working with all the guys at the Google group http://groups.google.com/group/jitterinmax4live-) V-Module has had a couple of relevant updates based on user feedback and tests. Here the details of the latest 0.4 version and of the other recent updates.

Ver 0.4
- Major review of all devices for compatibility against Ableton Max for Live Production Guidelines v0.6:
- All GUI parameters are labeled in a meaningful way (see info view). Yes, you read it right: ALL GUI parameters.
- The most significant GUI parameters (all type float and/or int, according to M4L constraints) can be mapped and automated for clip envelopes.
- Many devices have useful info view text (thanks M._TriangelKlang for the help)
- All devices behave politely and do not spill into Undo history (no loadbang on live.objects, Max umenu managed through pattr instances)
- All devices are frozen and directly compatibile with OSX and Windows OS
- Enhancements and fixes on many devices, in particular for V-Input devices:
- All V-Input devices have rate which can also go into negative values (thanks ChrisG for the tip)
- All V-Input devices are now efficient with memory and use both drag'n'drop and click-to-read ways to open clips (thanks to ChrisG for the "invisible button").
- V-VideoRack supports Thumbnails creation also iffolder is selected while device is ON.
- V-VideoRack, V-SliceBeat, V-Mixer and most V-Effects have full Info View help
- Some OpenGL devices will see improvement in the use of backread technique
- Documentation PDF updated and corrected to reflect changes

Ver 0.3.1
- V-VideoRack and V-SliceBeat support *** absolute *** paths.

Ver 0.3
- V-VideoRack rewritten. Now performance and memory usage are definitively efficient.
- V-VideoRack supports now 16 pads per device
- V-VideoRack supports chaining of multiple V-VideoRack, by using pad MIDI "offset" (see example08 liveset)
- V-VideoRack tested loading 64 clips in 4x V-VideoRacks
- V-VideoRack supports retrigger/continue options for multiple hits on same pad
- V-VideoRack does not abuse undo list

Ver 0.2.1
- V-VideoRack added
- V-Cam made cross platform
- V-Mixer and V-Out devices with window screen support ESC for existing fullscreen
- V-SliceBeat enhanced
- V-FX-GL-Rota-xxx removed theta issue on Live's undo
- V-FX-Feedback2 added

Usual download links:
http://fabriziopoce.com/download.html
http://www.maxforlive.com/library/device.php?id=594

Gr.
Fabrizio
http://fabriziopoce.com/index.html

[[email protected]]

Ver 0.7
- New V-Audio "Instruments" for playback of clips having both video and audio. Audio and video playback is in sync, audio sent through Live's mixer: V-SimAudioPlay, V-SliceAudioBeat, V-VideoAudioRack
- New V-SimAudioLooper an V-SimLooper for Live tempo warp-looping of video clips (respectively with and without audio feed)
- New V-FX-FFWrapper for FreeFrame plugins support
- All previously available V-Instruments (MIDI devices) for QT playback do not spill audio "outside" of Live
- New V-AuBeat based on external op.beatitude~ for Live tempo settings from audio input (even a live feed from external)
- New V-AuBeat2CVEnvelope based on external op.beatitude~ for very tight CV envelope generation from audio input (even a live feed from external)
- All video input devices save file (clips/folder) as part of the liveset
- V-ObjShape expanded for advanced texturing, multi rendering (cloning). It also saves the file as part of the liveset
- New V-ZoomPlay added (for picture zooming)
- Various enhancements to V-GMove3DW devices (3D camera movements) including a toggle for 2d/3d view switch

Ver 0.6
- V-SlabGen instrument introduced for generation of visuals (such as procedural fractals) based on custom OpenGL shaders.
- V-GTexture introduced for texture operations, such as video-to-texture (texture mapping in a 3D object) and texture-to-video (rendering to texture, feedback)
- V-GMulti introduced for efficient cloning of a source 3D object, creating potentially a very large number of 3D clones in space
- Added a V-Mixer version with OpenGL based internal 3D video feedback.
- Added a large number of OpenGL effects and devices for density, proximity, morphing applications
- Added a number of V-AuScope (audio) processing effects for tighter audio to video manipulation.
- Added a range of OpenGL coordinate formula generators (such as V-GCoord3 and V-GCoord4) for more control/customization of 3D objects.
- Added V-Boids instrument with simple fluid simulator.
- Added V-AuEnvelope audio effect for tracking beats and generating envelopes based on Live’s tempo

Ver 0.5
- Solved some bugs with open GL effects and mixer apparently due to video window automatically pop-up. Note: on V-Mixer devices you need to explicitly set "screen ON" now to have a window.
- Added a V-Mixer version (V-GL-RenderMixer4RGB-Out.amxd) for an additional output stream (useful for instance to do recording to disk by V-Rec)
- Several "video" / non 3D devices (V-VideoRack, V-SliceBeat, V-GL-Rota, V-GL-Slab) support ChrisG auto connect mode (i.e. in combination with VIZZable 0.9.2 or later. See device names with "-auto" suffix.
- LiveAPI LFO modulation supported (and LFO parameter bank removed). Each device shows its parameters (shortnames) for modulation and the V-LiveMod new device allows CVsend to LiveAPI modulation mapping
- V-Effects type OpenGL revised: no additional window required and better backread / readback support.
- V-Control device V-DroidSens2CV and V-DroidCtrl2CV added for Android control in combination with Matt Benatan's Max/MSP control app available at http://mattbenatan.com/)
- V-Control device V-Joy2CV added for support of HID usb devices (such as Joystick, Gamepad) etc. as CVsend (using the V-CV2MIDI you can use these to send MIDI informations).
- Several new OpenGL 3D tools (V-GCoord3, V-FX-GLOpScalar, V-GMove3DWJoy, V-GMove3DWPad)

[killingtime]

"- New V-Audio "Instruments" for playback of clips having both video and audio. Audio and video playback is in sync, audio sent through Live's mixer: V-SimAudioPlay, V-SliceAudioBeat, V-VideoAudioRack "

Awesome, thank you very much for this!