This is 2x32 of a planned 32x32 split stereo routing matrix that is turning out to be a total resource hog.
I need 32 copies of subpatch "Input 1&2" in order to complete the patch. The least of my worries is that each duplication of "Input 1&2" adds another apprx 6MB to the patch which is going to make the patch eat up a minimum 180MB of my precious system ram. What is really killing me is that after only half a dozen or so duplications of "Input 1&2" my cpu usage starts to climb absurdly high.
If anyone can help me with any ideas to completely tame the resource useage and make this patch useable I would be most gratefull!
This is being made with Usine 4.1 beta 9.
http://www.savefile.com/files/1991295
Routing Matrix Resource Issues
-
martignasse
- Site Admin
- Posts: 611
- Location: Lyon, FRANCE
- Contact:
Hi gurulogic,
Ouch, what you try to achieve is....a non trivial patch ! :O
Honestly, i don't think the 'brute force patch technique' can work for patch of this level.
It's more a job for a user module (SDK coding), or at least script, or even some external matrix vst like the MoodyMatrixPack in the add-ons section.
Those are basic way to reduce the resources usage that come in mind.
(on my Core 2 duo E6400, 2.13 gHz, your patch eat 5% of the CPU, no need to go further...)
Hope it help
Ouch, what you try to achieve is....a non trivial patch ! :O
Honestly, i don't think the 'brute force patch technique' can work for patch of this level.
It's more a job for a user module (SDK coding), or at least script, or even some external matrix vst like the MoodyMatrixPack in the add-ons section.
Those are basic way to reduce the resources usage that come in mind.
(on my Core 2 duo E6400, 2.13 gHz, your patch eat 5% of the CPU, no need to go further...)
Hope it help
Martin FLEURENT - Usine Developer - SDK maintainer
I'm with Martin here. If I have understood your patch correctly, you're trying to connect 64 inputs to 64 outputs using 4096 switches and then compute the output values by multiplying inputs with switches. Unless I've misunderstood something completely, that would with a block size of 64 amount to 64*4096*64=16777216 multiplications done almost 700 times a second when the sampling rate is 44.1.
Do you really need to be able to send all inputs to all outputs at the same time?
Do you really need to be able to send all inputs to all outputs at the same time?
Bjørn S
I was hoping to have any input made available to any output if that's what you mean?I do not see at any time needing all inputs sent to all outputs but for the sake of recalling routings via presets I though this might be the way to go....bsork wrote:Do you really need to be able to send all inputs to all outputs at the same time?
I guess it seems I've bitten off a little more than I can chew here
-
martignasse
- Site Admin
- Posts: 611
- Location: Lyon, FRANCE
- Contact:
Did you ever saw some matrix of this size ? (vst, or in real...)
I think there is some reasons, the resources involved is one big...
But the interface is another one (apart consuming some resources too, it have to be manageable)
You surely have to be 'more inventive' (no offense here) to achieve this result.
for example , the lightest set of connexions i see for a module like that is :
- 64 audio in
- 64 audio out
- an array in for the list of connection wanted (with same kind of internal coding than the color).
Even with just that, i don't know if a module user can do the job (it's lot of connexions), testing could be interesting.
I think there is some reasons, the resources involved is one big...
But the interface is another one (apart consuming some resources too, it have to be manageable)
You surely have to be 'more inventive' (no offense here) to achieve this result.
for example , the lightest set of connexions i see for a module like that is :
- 64 audio in
- 64 audio out
- an array in for the list of connection wanted (with same kind of internal coding than the color).
Even with just that, i don't know if a module user can do the job (it's lot of connexions), testing could be interesting.
Martin FLEURENT - Usine Developer - SDK maintainer
The largest pre-built matrix I am aware of is in Plogue Bidule.32x32 mono and seems quite light on resources.I have big dreams of being able to put a vst tap on each audio channel in my host and be able freely route each audio channel in and out of some 20 vst's in any order.The split stereo idea was a bonus so I could process left and right with different effects if desired.martignasse wrote:Did you ever saw some matrix of this size ? (vst, or in real...)
The lightest and perhaps easiest way to do this that I have found in Usine is with bus names stored as a preset but there is an unpleasant interruption in the audio when changing a bus's source/destination by changing the name of the bus. If I could find a way around this?
I am just a noob at this so suggestions are more than welcome.
-
martignasse
- Site Admin
- Posts: 611
- Location: Lyon, FRANCE
- Contact:
wow, big mama...
if you want to put 20 VST on top of that (and i'm sure not just simple vst
), you really need optimization on the routing part.
have you tested these VST, on AcousModules site, end of page ?
Hope this make your day
if you want to put 20 VST on top of that (and i'm sure not just simple vst
), you really need optimization on the routing part.have you tested these VST, on AcousModules site, end of page ?
Hope this make your day
Martin FLEURENT - Usine Developer - SDK maintainer
Well, without having done a ton of testing I am assuming that bypassing unused plugins should save my cpu..I have done this lots in Bidule, actually with 2 of each plugin.Lots of ram used but the setup is very workable (but obviously not perfect for my needs or I wouldn't be trying to force Usine to do things my computer doesn't want it to 
)
Thanks for the acousmodules link but the matrix's seem to have similar problems as mine. As soon as I open the UI, the 2Ghz core2 cpu on the laptop I use for testing maxes out on the active core and the 24 i/o actually locks Live/Usine right up...sigh!
)Thanks for the acousmodules link but the matrix's seem to have similar problems as mine. As soon as I open the UI, the 2Ghz core2 cpu on the laptop I use for testing maxes out on the active core and the 24 i/o actually locks Live/Usine right up...sigh!
I think I phrased the question badly. To be more precise: Do you need to be able to send any input to any output at the same time, or is it sufficient that an input can have several destinations, while an output can have only one source at the time, or the other way around. If either of these is the case, maybe it could be made more efficient.gurulogic wrote:I was hoping to have any input made available to any output if that's what you mean?
Your idea of using naming of busses is a clever one, but I have a strong feeling that busses do take some more CPU than a "normal" connection. For a more typical patch with just a handful of busses it's neglible, but not with the monster stuff you're trying to do...
Deleting and creating connections using the IML within a script could be a solution (at least if nothing goes through the wires when you do it), although I suspect that whatever you do, it won't be too CPU friendly.BTW, I think I've understood your matrix a little better after I wrote my previous post. It's not 64*64, more like 2 instances (L/R) of 32*32. At least that halves the number of switches... If you skip the "bonus" part of assigning L and R independently, the interface could at least be made smaller and simpler, for instance using 32 sequenced switch modules.
Bjørn S
yup, ur audio patchbay idea is certainly not easy. I tend to work with something similar, but only 8 stereo inputs -> 8 stereo outputs (L/R swtich)
if 2* (32*32) really does remain u cannot do a brute-force patch - as martignasse says.
1. internal
- conductor: hmmm.. cant think of something right now. a condutor with thousands of states does not seem practical thu.
- user module in C++ (eventually)
- scripts: they are not meant for audio processing
2. external solution:
- audiffex inToneMatrix specialized on exactly that purpose. its a 32 channel vst patchbay. (not cheap, did not try it myself, might be hard to connect to usine)
http://www.kvraudio.com/get/2261.html
- acousmodules: yes, I often found these plugins to be too cpu intensive also. they were made with Synthedit which isnt the proper tool for an audio patchbay I guess.
- hardware audio interface: TC Electronics and others offer audio patchbays that work with the computer. very expensive.
suggestions:
-> A: I believe a clever combination of bidule and usine could rock the job (connect via OSC) ?
-> B: try to think of design constraints like a mastering limiter is always at the end of a processing chain, etc...
-> C: maybe tackle the problem with 8x8 channels first.
if 2* (32*32) really does remain u cannot do a brute-force patch - as martignasse says.
1. internal
- conductor: hmmm.. cant think of something right now. a condutor with thousands of states does not seem practical thu.
- user module in C++ (eventually)
- scripts: they are not meant for audio processing
2. external solution:
- audiffex inToneMatrix specialized on exactly that purpose. its a 32 channel vst patchbay. (not cheap, did not try it myself, might be hard to connect to usine)
http://www.kvraudio.com/get/2261.html
- bidule: I dont think u'll find something more efficient than bidule. its bloody good at audio processing. dont think usine will match bidule's audio engine soon. remark: apart from that usine is better than bidulegurulogic wrote:Well, without having done a ton of testing I am assuming that bypassing unused plugins should save my cpu..I have done this lots in Bidule, actually with 2 of each plugin.Lots of ram used but the setup is very workable (but obviously not perfect for my needs or I wouldn't be trying to force Usine to do things my computer doesn't want it to
- acousmodules: yes, I often found these plugins to be too cpu intensive also. they were made with Synthedit which isnt the proper tool for an audio patchbay I guess.
- hardware audio interface: TC Electronics and others offer audio patchbays that work with the computer. very expensive.
suggestions:
-> A: I believe a clever combination of bidule and usine could rock the job (connect via OSC) ?
-> B: try to think of design constraints like a mastering limiter is always at the end of a processing chain, etc...
-> C: maybe tackle the problem with 8x8 channels first.
I want to be able to send any input to multiple outputs so that I would be able to parallel VST plugins and I don't see needing more than one source at an output and I have definately noticed the extra cpu use of busses...
And yeah, you're right, 32*32 stero split is what I was trying for so there would only be even to even and odd to odd routing.
I came up with another idea which is actually very simple and combats the cpu usage by only processing the routings used:
This sub patch uses apprx. 150 KB per occurance so once again my ram usage would be fairly high if I went with a stereo 32x32 layout but the bonus is that the switch in the matrix grid that controls the patch active module simply turns the single sub patch on to complete the routing while all other routings remain disabled.
The downfall is that I am getting occasional pops when enabling and disabling the patch, presumably from truncuted samples?
I have manged to circumnavigate the pop noise by putting a 100ms data delay between the switch and the patch active module and then running the undelayed switch out of the patch and into a fade in/out out patch that was uploaded here set to a 6 ms fade.
(actually, why that works doesn't even make sense, it seems as though the fade patch is running late!)
The addition of the data delay script to every one of these hundreds ? of sub patches would hugely increase the memory usage so I would rather find another way ...also if I were to use the fade in/out patch, I would need a way for every switch connected to one instance to operate independantly of the others...
As long as there's no fatal flaws in my thought process here I should be able to get something in the way of a workable larger routing matrix ,though I still have to find a better way to deal with the popping audio...?
And yeah, you're right, 32*32 stero split is what I was trying for so there would only be even to even and odd to odd routing.
I came up with another idea which is actually very simple and combats the cpu usage by only processing the routings used:
This sub patch uses apprx. 150 KB per occurance so once again my ram usage would be fairly high if I went with a stereo 32x32 layout but the bonus is that the switch in the matrix grid that controls the patch active module simply turns the single sub patch on to complete the routing while all other routings remain disabled.
The downfall is that I am getting occasional pops when enabling and disabling the patch, presumably from truncuted samples?
I have manged to circumnavigate the pop noise by putting a 100ms data delay between the switch and the patch active module and then running the undelayed switch out of the patch and into a fade in/out out patch that was uploaded here set to a 6 ms fade.
(actually, why that works doesn't even make sense, it seems as though the fade patch is running late!)
The addition of the data delay script to every one of these hundreds ? of sub patches would hugely increase the memory usage so I would rather find another way ...also if I were to use the fade in/out patch, I would need a way for every switch connected to one instance to operate independantly of the others...
As long as there's no fatal flaws in my thought process here I should be able to get something in the way of a workable larger routing matrix ,though I still have to find a better way to deal with the popping audio...?
Ok, so much for optimism,I give up. No matter what I do, this will bring any computer to a crawl.Even just a test 32x32 mono without any audio i/0 or meters eats up a 100MB and saving is painfully slow.
Maybe sometime next decade I'll be able to have the realtime recallable routing I want...?
Maybe sometime next decade I'll be able to have the realtime recallable routing I want...?
Muaahahaha....me give up, lol! Nope, I've got someting cooking....virtually unlimited routing! Just did a live test run(in my basement) and so far so good.
I look forward to sharing if it doesn't go BOOM
I look forward to sharing if it doesn't go BOOM
great !
hey amiga909, I don't know how I missed you're last post but thanks for all the input.
(we musta' posted at the same time or something)
It's funny, I had actually recently downloaded the demo of inToneMatrix and deleted it as soon as it covered my screen in a schlew of cheap looking FX and showed no apparent sign of a hostable vst. Maybe I should have another look as the solution I have put together, despite being extremely versatile is not exactly (as of yet) uber computer efficiency friendly, although I may be able to trim some fat yet...
(we musta' posted at the same time or something)
It's funny, I had actually recently downloaded the demo of inToneMatrix and deleted it as soon as it covered my screen in a schlew of cheap looking FX and showed no apparent sign of a hostable vst. Maybe I should have another look as the solution I have put together, despite being extremely versatile is not exactly (as of yet) uber computer efficiency friendly, although I may be able to trim some fat yet...
something I miss: I understand it can be cool to dispatch any 32 in to any 32 out, but after that each "in" has its own fx, wouldn't this allow only to have "parallel" processing, not true inserts chains? I don't understand the application of having 10 or more parallels effect..?.better then use sends busses no?
I wanted to be able to send a track to any of my FX (in abe live), and I did it via chains, which are kind of selector/dispatcher modules where each chain has it own vst.
I first had the similar clicks when just switching on or off effects and Cpu usage pbs, so the workaround I found was to have a dry/wet fader, that first let me have smooth transition, and that would also turn off the effect if 100 percent dry, and so save CPU without click, (I think it might be better than trigger a fade with delay),
In live when bypass CPU is less than 1 percent and can handlle easy up to 127 different chains, then each chain can "send" to lets say 32 differents sends bus without pb (not all at the same time of course), so might be a workaround to use live as fx process)
but it might certainly be transposable in usine. It could have a bit less possibilities that what you think of, but from what I understand results would be very close, and I find it very efficient "musically"
anyway let us now your searches! interesting
I wanted to be able to send a track to any of my FX (in abe live), and I did it via chains, which are kind of selector/dispatcher modules where each chain has it own vst.
I first had the similar clicks when just switching on or off effects and Cpu usage pbs, so the workaround I found was to have a dry/wet fader, that first let me have smooth transition, and that would also turn off the effect if 100 percent dry, and so save CPU without click, (I think it might be better than trigger a fade with delay),
In live when bypass CPU is less than 1 percent and can handlle easy up to 127 different chains, then each chain can "send" to lets say 32 differents sends bus without pb (not all at the same time of course), so might be a workaround to use live as fx process)
but it might certainly be transposable in usine. It could have a bit less possibilities that what you think of, but from what I understand results would be very close, and I find it very efficient "musically"
anyway let us now your searches! interesting
23fx23, if Ableton Live handled vst "scenes" anything like Usine I would have a solution long ago. Unfortunately, even though Live is very efficient at saving cpu when a plugin is bypassed, each occurance of a plugin in a chain is loaded into ram. This works fine in the context of a single pattern/song or maybe even a dozen pattern/song's if done carefully and even more if using Live's very efficient built in plugins but my goal is to be able to work anywhere up to 128 patterns within a single file load so that I am not limited to a prearranged set during performance.
Perhaps I am being overzealous in my ambitions and the smartest thing for me to do would be to put a modest return FX bus using Usine and for insert fx work within Live's limitations. Also a large hinderance is that every plugin instance in a chain has to have remote controls remapped and I would way prefer to work with global mapping.
My basic setup is eight stereo channels in Live from external hardware devices, an eight channel hardware interfaced loop slicer/sampler/midi sequencer VSTI, six VSTI instances of instruments that respond to program change messages so no external management is needed, anywhere up to 20 VST plugins and one external FX processor (KP3) Also throw in the mix, a Behringer BCR2000 and a BCF2000 each currently with eight pages of potential assignments a really cool tool I built with Usine that displays all current assignments as I switch pages and a great tool built in Bidule with a 3'rd party plugin that stores all current values for every page of mappings and updates the controller visual feedback and the Usine "visualizer" as I switch pages (presets)
I am visualizing two return channels with Usine "racks" in Live that can both be used as send FX from Live and inserts for anywhere up to 30 tracks in Live...
I'm going to do some more work on the "routrix" patch today and hopefully get it debugged enough I can upload it and maybe get some more feedback on ways to approach optomization. Also as a bonus, it is very easily scalable by adding or removing subpatches.
I may be obsessive compulsive with finding better ways of doing things..
Perhaps I am being overzealous in my ambitions and the smartest thing for me to do would be to put a modest return FX bus using Usine and for insert fx work within Live's limitations. Also a large hinderance is that every plugin instance in a chain has to have remote controls remapped and I would way prefer to work with global mapping.
My basic setup is eight stereo channels in Live from external hardware devices, an eight channel hardware interfaced loop slicer/sampler/midi sequencer VSTI, six VSTI instances of instruments that respond to program change messages so no external management is needed, anywhere up to 20 VST plugins and one external FX processor (KP3) Also throw in the mix, a Behringer BCR2000 and a BCF2000 each currently with eight pages of potential assignments a really cool tool I built with Usine that displays all current assignments as I switch pages and a great tool built in Bidule with a 3'rd party plugin that stores all current values for every page of mappings and updates the controller visual feedback and the Usine "visualizer" as I switch pages (presets)
Before I found Usine (a whole week ago) I was using Bidule with limited success across three return channels in Live , but the main potential I am seeing with Usine, apart from wicked vst scene capture and recall, is the ability to route up to 16 stereo Live tracks through a single instance, which with a proper routing facility will allow any of the incoming audio channels to be processed 100% wet if desired through any effect in the "matrix" and then sent back to any 16 tracks in Live.23fx23 wrote:something I miss: I understand it can be cool to dispatch any 32 in to any 32 out, but after that each "in" has its own fx, wouldn't this allow only to have "parallel" processing, not true inserts chains? I don't understand the application of having 10 or more parallels effect..?.better then use sends busses no?
I am visualizing two return channels with Usine "racks" in Live that can both be used as send FX from Live and inserts for anywhere up to 30 tracks in Live...
I'm going to do some more work on the "routrix" patch today and hopefully get it debugged enough I can upload it and maybe get some more feedback on ways to approach optomization. Also as a bonus, it is very easily scalable by adding or removing subpatches.
I may be obsessive compulsive with finding better ways of doing things..
hehe I think we are a few like that...gurulogic wrote:I may be obsessive compulsive with finding better ways of doing things..
good luck in your patch, would love to see how you could create that.
actually I tried to be cool with my laptopgurulogic wrote:... is the ability to route up to 16 stereo Live tracks through a single instance, which with a proper routing facility will allow any of the incoming audio channels to be processed 100% wet if desired through any effect in the "matrix" and then sent back to any 16 tracks in Live.
. Ive got 8 tacks mixdown of my live sessioneach track has a FX rack composed of 16 sub/racks that dry/process the in.
so I can activate up to 16 fx working together on a track (but I find it's not descent asking...), with differents seting for each 8 track.
this works well on a 2core, I think with a quad you could get with "big" Vst a 127fx x 16tracks, enabling let's say 3 or 4 fx per track as median. also you're right about the true boring thing of that technique is you have to remap the parameters used for live session..appart from that I thing you want something more I still can't catch...
but don't bother
wait to see your process,..
Who is online
Users browsing this forum: No registered users and 16 guests