Avatar

Please consider registering
Guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —




— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed sp_TopicIcon
Processes / thread priority and scheduling
No permission to create posts
June 12, 2016
2:10 pm
Avatar
hoegge
Member
Members
Forum Posts: 11
Member Since:
June 12, 2016
sp_UserOfflineSmall Offline

Dear all and/or Will,

Having read the initial chapters of the FX book, which is a great intro, there is still a thing I don't have understood and never really did.

Since modern multitasking still (as far as I known) depends on the CPU (even with several cores) alternating between different processes, what prevents the audio processing thread in the VST's / Plug-ins to be interrupted by other procesesses? Is that solved by changing the priority of these threads and is that done by the DAW? If there are many plug-ins running - what prevents them from "disturbing" each other? Is there a good source of information, where I can read about this, both in general terms e.g. OS scheduling and also specifically about how DAW's in general handle this?

/hoegge

June 12, 2016
11:22 pm
Avatar
W Pirkle
Admin
Forum Posts: 143
Member Since:
January 28, 2017
sp_UserOfflineSmall Offline

The OS deals out slices of CPU time to each thread running in each process. In Windows, as a programmer you can set the thread priority (see the code in App Note 2 http://www.willpirkle.com/app-notes/#AN2) however, you are supposed to "play nicely with other apps" and not try to hog the CPU time. How the OS handles multiple threads is up to the designers.

The book that I learned from back in the 90's, which is pretty much the "Bible" of Win32 programming, is called "Programming Windows" by Charles Petzold and was published by the Microsoft Press - it is no longer in print . (It also explains why people use Hungarian Notation) In that book, Chapter 20 called "Multitasking and Multithreading" gives concrete examples so you might want to try to find that (perhaps in a library)?

- Will

June 14, 2016
7:17 pm
Avatar
hoegge
Member
Members
Forum Posts: 11
Member Since:
June 12, 2016
sp_UserOfflineSmall Offline

Thanks Will.

So the DAW does not really do much but to spawn the threads of the plug-ins when it needs data from them and the leaves it up to the OS to balance the threads and change as needed? That means, nothing really protects a thread - audio-processing function - against being interrupted by another if it is too slow or just if an interrupt occurs? But as long as each plugin are done doing their stuff within approximately 20 ms (typical time slice for a thread if it does not end before) and there is time for all plugins to do that within the time window of the buffer size then you're good?

I'll try to study the Ardour source code to see if I can see how it is done there

-Hoegge

June 14, 2016
7:26 pm
Avatar
W Pirkle
Admin
Forum Posts: 143
Member Since:
January 28, 2017
sp_UserOfflineSmall Offline

Yes, you can max-out the CPU, or create glitches by running too many plugins, or by having a process function that takes too long to get the audio data shipped out before the next buffer arrives. One options is to use multiple threads (as in the FFT example App Note), but this only works for processing that has inherent parallel-processing ability, which many do not.

- Will

June 15, 2016
12:39 am
Avatar
Tom
Admin
Forum Posts: 65
Member Since:
April 3, 2014
sp_UserOfflineSmall Offline

Hi Heogge,
PlugIns are just runtime-loaded libraries. Usually a plugin will not need to spawn threads for UI or the "DSP" part, because those are the parts that the DAW will call within it's own UI and DSP threads. So the DAW will take care to e.g. spawn different plugins across multiple threads to leverage multi core CPUs. But it usually will not spawn a new thread for each plugin. It will also know which plugins can run in parallel and which need to run serial (e.g. the plugins in one channel need to be run in serial usually whereas different channels could be processed in parallel). You should carefully think before spawning threads from a real-world plugin. E.g. consider an EQ which would be used on each track in a 40 track mix. If you did spawn a thread for each instance to do parallel high-res FFT it will kill your performance just from having too many threads around. For an analyzer in an EQ plugin for example you could solve this by only running the FFT thread while the UI is opened.

With ASIO for example, the DAW registers a callback function where the buffers have to be filled/read. Depending on the buffersize and samplerate, this will be called about 100 to 3000 times per second. So you have much less time then the 20ms you mentioned to fill these buffers. To get down to these short intervalls and still process hundreds of plugins you have to do quite some "black magic" inside a DAW Wink.

June 15, 2016
7:50 pm
Avatar
W Pirkle
Admin
Forum Posts: 143
Member Since:
January 28, 2017
sp_UserOfflineSmall Offline

Same thing holds for synth plugins. Spawning threads for each note/voice to try to increase polyphony/multi-timbrality won't work either. It will actually reduce the polyphony/multi-timbrality so don't go down that path.

And, RackAFX does not spawn a new thread each time you load a plugin - there is one and only one signal processing thread which is part of the audio callback system used for shuttling buffers of data to and from the driver (whether it is ASIO or not). Since your processAudioFrame( ) (or process buffer) function is running in this thread, it is crucial that you do not use new/delete or malloc/free within the processing function as this will also kill performance.

- Will

June 16, 2016
10:43 am
Avatar
hoegge
Member
Members
Forum Posts: 11
Member Since:
June 12, 2016
sp_UserOfflineSmall Offline

Ah - ok got it. Do most DAWs then have a higher than normal priority thread running for scheduling (calling in serial) all the plug-in functions (audioprocessing functions) and then a normal priority thread for GUI and non real-time stuff? And how is the audioprocessing thread protected against other software suddenly taking CPU time from it? Do you give it higher priority than the other high and above normal priority processes running on a normal Windows system?

The Bitwig DAW seems to be able to withstand a plug-in crashing. They say they run VSTs in a separate process. But then maybe it is only in a single separate process so all plug-ins crash if one crash, but then the GUI knows how to reaload all the VSTs in a new plugin process I guess.

June 16, 2016
11:32 am
Avatar
hoegge
Member
Members
Forum Posts: 11
Member Since:
June 12, 2016
sp_UserOfflineSmall Offline

Explorer Ableton Live running a bit. Can see it has one high priority thread (prio 24) running in addition to the original process / thread. When you enable multi CPU support it adds more similar threads up til a maximum of 4 as you add plug-ins. After reaching 4 it adds the following plug-ins to the existing 4 threads. Then the sound playing thread (windows drivers or asio4all) have a priority around 15 and the Ableton Live GUI thread is close to normal 12.

Starting to understand 🙂

June 16, 2016
4:23 pm
Avatar
Tom
Admin
Forum Posts: 65
Member Since:
April 3, 2014
sp_UserOfflineSmall Offline

All this stuff has really become complex over the years. Things like multi core processors together with "speed step", "turbo boost" and whatsoever technologies, heat management etc. all have an influence on this and most DAWs I know really had to struggle with these technologies.

One of the difficulties is that there is no "standard" way of handling these things. Windows and MacOS are no "real time" operating systems to start with. But over the years they have added some options to get "quasi real time" capabilities which are good enough for "real time" Audio/Video at least. One of the younger things in windows is called "Multimedia Class Scheduling Service" (MMCSS) - this can give your audio thread that extra-boost to prevent interruptions from other processes even under high system load (to some degree this emulates the behavior of real time operating systems). In general, the OS is king and will decide which thread of which process will be given CPU time (originally, that's the reason why the OS exists in the first place). Without extenstion like MMCSS it actually is/was quite usual to have audio interrupted e.g. just by turning the mouse wheel above a browser window or the like, because the browser may also have applied for "high" priority to allow for a smooth user experience. With the default settings, Windows will in general give a little priority boost to the thread that owns the currently active window.

But there's a caveat even with MMCSS - which thread should apply for this extra boost? When both, DAW and audio driver apply for it, they may fight each other. There are quite some discussions among developers, whether the audio driver should apply for this or the DAW, or even both, and basically there's no one-size-fits-all solution.

Like a lot of things in development, this topic can get really involving on its own Wink.

Here are some resources covering these topics in a more general view:

Windows Scheduler:
https://msdn.microsoft.com/en-us/library/windows/desktop/ms681917(v=vs.85).aspx
MMCSS:
https://msdn.microsoft.com/de-de/library/windows/desktop/ms684247(v=vs.85).aspx

June 16, 2016
7:26 pm
Avatar
W Pirkle
Admin
Forum Posts: 143
Member Since:
January 28, 2017
sp_UserOfflineSmall Offline

"In general, the OS is king and will decide which thread of which process will be given CPU time (originally, that's the reason why the OS exists in the first place)." <-- exactly.

About a year and a half ago, we had a demo of the Yamaha Nuage console at the University. It runs Nuendo and VST3 plugins. The main computer was actually a "farm" of CPUs (I think that one had 18-24 CPUs on it) and after that demo, it became very obvious as to why the VST3 API is setup the way it is in dual-component mode -- distributed computing (perhaps aimed specifically at the Nuage product? not sure...). And that opens up a whole new set of issues regarding threads and scheduling...

- Will

Forum Timezone: America/New_York

Most Users Ever Online: 36

Currently Online:
3 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

Skyler: 47

Peter: 41

Derek: 41

clau_ste: 39

Frodson: 38

Gwen: 32

EZB: 24

lppier: 23

Msaldaña: 18

Jorge: 17

Member Stats:

Guest Posters: 1

Members: 476

Moderators: 1

Admins: 4

Forum Stats:

Groups: 11

Forums: 30

Topics: 482

Posts: 1876

Newest Members:

sam, annaharris, Marie Weaver, kev, Steven, Mr Anderson, mguy, omelc

Moderators: W Pirkle: 143

Administrators: Tom: 65, JD Young: 80, Will Pirkle: 0, W Pirkle: 143