Equivalent VST vs. RE performance test(s)

[antic604]

By now there should be at least few instruments that have both VST and RE format (I'm aware of The Legend, Spire and kHz ONE). Since I have The Legend I thought I'll check how the two formats stack against each other.

Platform: Surface Pro 4 i7-6650 8/256Gb

Testing approach: a Combinator with RE and VST version of The Legend with the same factory preset connected to 6:2 Mixer, with Buttons 1 toggling between sending notes to RE and VST and Rotary 1 & 2 modulating filter's cutof and resonance via automation clips. I turned the Button 1 on (RE active), copied the track repeatedly until it started crackling continuously then moved back (deleting tracks) to get uninterrupted playback. Then I deleted all the duplicates, turned Button 1 off (VST active) and repeated the procedure. I've done this 3 times, to ensure the results are stable.

Test track: Reason 10, uses The Legend VST and RE

Results: the RE version of The Legend played fine 18 copies of the track (so 19 tracks in total), while VST version managed to do 16 copies (17 in total), which means that - at least for this particular pair - VST gets up to 90% of the RE equivalent's performance



I'll do kHz ONE next, when my VST -> RE cross-grade comes through

[antic604]

With all this talk of Reason's "poor VST performance" no one cares to help or even comment?

[Pralijah]

Would be interesting to see tests of more RE vs VST in Reason. But I guess there are not that many devices that are exactly the same versions.
Maybe kHs effects would be accurate, as they are simple enough in look and function, to show a transparent situation on how much difference there is in the VST-wrap thing compared to the RE-wrapper.

I know very little of these things, so my first question is:

Is this result bad that you got? Or is it rather expected that an extra wrap of an VST compared to the RE:s that dont have an extra wrapper?

Would be interesting to hear more opinions.

I think the dilemma of this extra burden, really comes when using a VST here and there, both instruments and effects, and what looks like "ok" in only one VST, soon gets too heavy when having more VST:s, if wanting to use Reason as a VST host, like some do with say Reaper.

Another question is if RE is efficient in itself? Can it ultimately be as efficient as a stock Reason device?

I wonder if there exist VST:s that are equal to their RE counterpart...inside Reason.

[antic604]

Is this result bad that you got? Or is it rather expected that an extra wrap of an VST compared to the RE:s that dont have an extra wrapper?

Reading all the horror-stories I expected the performance of the VST version to be much worse and was surprised that it's "only" ~10%. Obviously with just a single data-point it's difficult to extrapolate any meaningful conclusion from this, hence the topic to maybe encourage others to participate. I even linked the test "song" itself for others to try on their systems...

Anyway - it's a good idea to try kHs' effects as well, as 5 of them are free in both formats, so I can chain them and compare the results. Will do that!

[kooshan]

Ok great
But There is something very important here missing.
In my experience Re’s were never cpu efficient , never .actually there were always quite cpu hungary with the exception of few. So no good comparison i think.
I’m pretty sure if you run the legend vst on ableton live , logic or... you may able to run 30 of them .

[antic604]

Ok great
But There is something very important here missing.
In my experience Re’s were never cpu efficient , never .actually there were always quite cpu hungary with the exception of few. So no good comparison i think.
I’m pretty sure if you run the legend vst on ableton live , logic or... you may able to run 30 of them .

Good call, I'll test that as well

[submonsterz]

If I'm reading your original post right .,...
Your test is a little flawed .
Here's why in my thinking.
First do not mix the re and vst in same song file .
Use each plug separate in Thier own file not together.
Make song file with plug a then delete that plug
And then insert plug b resave as alternate.
Then test each plug seperately and jot down ya results.
Then it's best to also at same time to see the relationship of reason vst use to another say two daws say reaper and FL or what ever daws you can get access to just recreate the same song file it's simple to do . Then you'll have a much better all round on how the vst compares not only in reason but also outside reason .
Ohh and I would strongly suggest if you have access to reason 9.5.1 as well to do the same test on the re and vst too and make Shure all parameters in both plugs match ie oversampling etc and if the re as some do have less of the vst features or effects etc are switched off to match.
Doing as I say you might be surprised at your new results.
Have fun enjoy your findings .....
I gave up being solely dissapointed with the results I was getting .

[antic604]

If I'm reading your original post right .,...
Your test is a little flawed .
Here's why in my thinking.
First do not mix the re and vst in same song file .
Use each plug separate in Thier own file not together.
Make song file with plug a then delete that plug
And then insert plug b resave as alternate.

Yeah, I was thinking about doing it, but didn't really had the time. But since I have both in Combinator it should now indeed be really easy to do as you say.

Regarding the VST / RE settings, I've used exactly the same factory preset and - obviously, due to using Combinator - the same notes and same automation.

I'll re-test like you suggest and post the results

[antic604]

Ohh and I would strongly suggest if you have access to reason 9.5.1 as well to do the same test on the re and vst too

Why is 9.5.1 significant? I'm a new Reason user and v10 is my first version, so I've no access to 9.x

[Oquasec]

Ohh and I would strongly suggest if you have access to reason 9.5.1 as well to do the same test on the re and vst too

Why is 9.5.1 significant? I'm a new Reason user and v10 is my first version, so I've no access to 9.x

Reason used an external midi module for vsts until Reason 9.
Rewire/EMI/ Loopback was the previous ways to get vsts into Reason.
Reason having a native bridge these days is useful, since it gives access to CV wires at the back.

Which has not been done that way before, in anything else.

[antic604]

Why is 9.5.1 significant? I'm a new Reason user and v10 is my first version, so I've no access to 9.x

Reason used an external midi module for vsts until Reason 9.
Rewire/EMI/ Loopback was the previous ways to get vsts into Reason.
Reason having a native bridge these days is useful, since it gives access to CV wires at the back.

Which has not been done that way before, in anything else.

Ok, sorry - I thought he meant I should use 9.5.1 instead of 10 for some reason

[submonsterz]

Reason used an external midi module for vsts until Reason 9.
Rewire/EMI/ Loopback was the previous ways to get vsts into Reason.
Reason having a native bridge these days is useful, since it gives access to CV wires at the back.

Which has not been done that way before, in anything else.

Ok, sorry - I thought he meant I should use 9.5.1 instead of 10 for some reason

No idea what the other guy is on about lol.
Vst was in 9.5.1 but they did a change of some sort from that version to the next and onwards that broke many vst on pc for a lot of people and personally and a couple of little tests with a couple of other people we seem to agree there is a difference in performance from 9.5.1 to version ten not much but we think it's there for us . And I'd suggest do not use combinators etc in your tests just use the re on its own nothing else and then exact same set up with just the vst on its own do not add in nothing else at all to get a real picture of usage and differences .

[antic604]

And I'd suggest do not use combinators etc in your tests just use the re on its own nothing else and then exact same set up with just the vst on its own do not add in nothing else at all to get a real picture of usage and differences

Yes, that's what I do right now. I'm testing the same loop with 4 synths (The Legend, Monark, RePro-5 and Thorn) in 4 DAWs: Reason 10, Bitwig 2, Live 9 and Studio One 3. At the end I'm gonna make Excel from the results and publish here

For a moment I thought to add native synths from those DAWs in the mix, but after 100 instances of Thor I gave up

[Psuper]

You have to do what I did - test the performance of the VST in Reason, and test it in standalone (via midiloopback, or in another DAW). Its an enormous difference, there's no question about it.

[Oquasec]

That might be reason being new to vst support and, or not being optimized for it.
Cuz you could practically use 100+ thors in a combi or like 1500 subtractors in a combi if you wanted to.

[antic604]

You have to do what I did - test the performance of the VST in Reason, and test it in standalone (via midiloopback, or in another DAW). Its an enormous difference, there's no question about it.

Well, I'm - slowly... - in the progress but I don't see that it's "enormous" - more like 10-15% (unless that is considered enormous ), at least against Bitwig 2 which would be the DAW to compare Reason to, since it's also heavily modular. Obviously I expect the difference to be bigger with Studio One 3 or other "linear", traditional DAWs that are built for playing back of stuff that's recorded and not for modulating it in various, often random ways.

I'll post back when I do all the testing against Bitwig, Live & Studio One

[aeox]

Ok, sorry - I thought he meant I should use 9.5.1 instead of 10 for some reason

No idea what the other guy is on about lol.

That's his thing. It's what he does.

[Psuper]

It's definitely an enormous difference: viewtopic.php?f=4&t=7504474&p=368920#p368920

Sum (3 tests):
1. Reason 96k Hosting VST Pianoteq (forced 64 buffer size), playing 30 notes polyphony. CPU on 6 cores at 50%
2. Pianoteq 96k STANDALONE, buffer size 128, playing 50 polyphony, CPU on 6 cores at 12%
3. Reason 96k + MIDILOOPBACK PianoTeq, buffer size 128, playing 50 polyphony, CPU on 6 cores at 18%

I only ran example 2 in standalone to show the VST raw performance with no other factors.

We see that Reason forces 64 buffer size when hosting the VST (example 1), so in order to properly test it we need to midiloopback the app into Reason (example 3).

CPU @ 50% when VST is hosted in Reason, only 30 poly.
CPU @ 18% when VST is midiloopbacked into Reason, upped to 50 poly.

That's absolutely significant, and the only real test worth its salt comparing VST apples-to-apples performance in Reason.

[chaosroyale]

Psuper- what's the result for "3: midiloopback" if the buffer is 64 samples? That would show if the problem is mostly the buffer or some other issue.

[Psuper]

Psuper- what's the result for "3: midiloopback" if the buffer is 64 samples? That would show if the problem is mostly the buffer or some other issue.

Bout the same CPU usage: 19% playing 50 poly for about a min.

However the forced 64 samples is, in my opinion, the main culprit within Reason as I mentioned in the linked thread I posted above. I'm just not sure how much of an affect it will have on the CPU usage, I don't think much if at all. The forced lower buffer size in Reason will definitely affect when you'll get pops and clicks, but shouldn't do much to the CPU.

[chaosroyale]

Well - if nothing else it shows that a the excuse of "VSTs perform badly with 64 sample buffers outside of Reason too" is not entirely true.

[bsp]

CPU @ 50% when VST is hosted in Reason, only 30 poly.
CPU @ 18% when VST is midiloopbacked into Reason, upped to 50 poly.

I think the reason for the high CPU load in Reason is that it is using a "reactor" style threading strategy (not related to Native Instruments) where you run all cores "hot" to be able to better handle low latency playback.

I may be mistaken here (only the PH devs really know) but that would explain why Reason always tries to load all CPU cores even when the project only has a few instruments (I think it even happens with a single combinator).

In other words: You cannot use the CPU load reported by the OS "task manager" to find out about the actual time required for audio rendering.

In my recent benchmarks, Reason was able to run 5*3=15 instances of Pianoteq 6 (5 combinators with 3 instances each, all playing the "Steinway D Prelude" patch with 48 voice polyphony) at a DSP load of 2 bars (my test flooded the plugin with noteons to make sure all voices were in use). The OS reported a load of 34%.

The performance seemed comparable to my own VST host which managed to play 16*3=48 instances, with 74% CPU load reported by the OS (also using 64 sample frame chunks).

Both tests were run @44.1kHz with a 256 sample frame ASIO buffer.

Well - if nothing else it shows that a the excuse of "VSTs perform badly with 64 sample buffers outside of Reason too" is not entirely true.

As far as I can tell, the 64 sample chunks are not the main problem.

At least on my system (and also on my previous PC), the main problem with Reason is that the CPU load spikes every couple of seconds (e.g. the Pianoteq test had underruns every couple of seconds and the DSP meter red-zoned for a split second), plus it does not allow you to spread instrument+effects chains over multiple cores, plus the send fx are also not being run on different cores, i.e. it's not possible to add a (heavy) reverb (e.g. Adaptiverb) to a (heavy) synth (e.g. Diva).

p.s.:
..but all that being said: I still love Reason since VSTs are "hell", from both a user and a developer point of view.
I recently moved to a new PC and while it was a breeze to re-install Reason (including all REs), going through all my VSTs (and I even don't have _that_ many), deauthorizing them, re-authorizing them on the new rig took quite some time.
..and god beware you forget to deauthorize one of them.. (I actually forgot one => one activation out of three is now lost!).
....and don't get me started on all the idiosyncrasies you have to deal with (as a host dev) to get VSTs up and running in the first place (for example, most of the VSTs I have tested simply crash when you multi-thread the initialization, with notable exceptions being the plugins from Waves, FabFilter, AudioThing, Acorn Digital, Waldorf, and UVI)

In an (my) ideal world, PH would produce a VST plugin shell that allows you to run REs in VST hosts (with all the benefits and drawbacks that come with REs).

[antic604]

During my testing - which is still going, I don't really have time - I've came to two conclusions:

1) It really depends on a VST - for example with The Legend, Monark and RePro-5 my results of Reason 10 vs. Bitwig 2.3 were close, i.e up to 10% less for the former. For Thorn VST, the difference was already bigger, i.e. Reason culd run 70% of the instances Bitwig would, whereas for Analog Ultra VA-2 only 30%

2) However, Reason handles high load more "gracefuly". What I mean is that with Bitwig (and indeed Live, I started testing as well) you can add instances and suddenly it starts glitching, but when you remove one instance it still glitches, sometimes even more so you end up removing 3-5 instances to get to stable playback. I never really know what's the precise number. Also, their GUI falls down on its nees - gets totally unresponsible, like 1 frame per second. With Reason, it's much more stable and predictable - if glitches start, it's enough to just take away the last instance that was added and it's back to normal. Also, the GUI - while slowing down - is still functional.