[Media_api] new user / latency, audio, polling, sysex, haskell

[Roger Dannenberg]

Evan Laforge wrote:

> I think this is the "semi-synchronous" thing I was theorizing about.
> In my case, the application has no "wake up" because it never goes to
> sleep.  Think of it as a separate process piping a timestamped midi
> stream into a midi player.  The application doesn't have latency per
> se, since it always runs as fast as it can.  So it doesn't have any
> scheduling either (well, the OS schedules it to run of course, but
> it'll probably be running on its own core anyway).
>   
Terminology is a problem. I generally use "latency" to mean the maximum 
time between input and output, i.e. when a midi message arrives or a 
timer interrupt occurs, how long can it take for the event to be 
delivered to the application, processed, and delivered to the output 
device? (As I type, I see this defn is not quite right, but it's the 
right idea.)
> In my case, I'm using language runtime threads distributed over one OS
> thread for each core, so really the language runtime is doing the
> scheduling, but in practice they just act like fast switching OS
> threads.
>
> The thing that puzzles me, is why would someone *not* do it this way?
> Since I'm new to audio programming stuff I'm perfectly willing to
> believe I've missed something everyone else understands here...
>   
I think what you're saying is that you are polling and busy waiting. The 
main argument against this is that you're using a lot of CPU power to 
determine that there is nothing to do. The simplest alternative is to 
insert a "sleep" in the polling loop. If you can wake up, test for 
input, and suspend in 10us, than a 1ms sleep reduces the wait loop from 
100% to 1% of the CPU. If you don't have a core per thread, this can 
dramatically improve system performance, and even with multiple cores, I 
suspect having them pound on the kernel through polling is not a good 
idea. At a minimum, avoiding 100% utilization keeps my laptop a lot 
cooler and quieter (the fan kicks in when CPU utilization is high.)

If you don't like polling at all, then the standard practice is to 
sleep waiting for an input event or timeout based on the time of the 
next scheduled event. That way, you only wake up when there is work 
to do. This has the advantage that you wake up immediately to handle 
events, but usually at the cost of more overhead.

> In the case of audio, I would say "play at this ...

Audio is essentially an uninterrupted sequence that is clocked in/out by 
the sample clock in the A-toD/D-to-A system. The clock is generally 
independent of the OS system clock and the two are not calibrated with 
each other (e.g. according to the OS, you will never have exactly 44100 
samples per second or any other round number). Therefore, you don't send 
samples with timestamps, you just start the stream and keep computing, 
usually in some sort of callback/demand scheme. If you want to play a 
short sequence of samples, you generally mix them into a stream that is 
already started to avoid glitches on the output and to avoid the 
overhead of setting up the stream and taking it down. Once things are 
running, there is some latency associated with the number of samples 
buffered. On output, this number decreases continuously at the sample 
rate and increases by some integer number every time you write a block 
of samples. Often, the exact number cannot be determined.

> since portmidi doesn't look particularly
> reentrant-safe.  

It's not: as a new user, could you tell me where this important 
information would be noticed and should be written?

> The thing about polling, is if I set it to 1ms, that basically adds
> 1ms to my best case latency (actually, worse, it inserts a 0-1ms
> jitter... 

This is not necessarily true. Your system latency (the time the OS might 
wait before running a thread that is ready to run) is probably at least 
1ms, so whether you sleep for 1ms or 1ns or 0s, you have no guarantee 
that the next instruction will be executed sooner than 1ms from now. By 
inserting a little latency and using timestamps, you can reduce jitter 
independently of whether you sleep or not.

In my opinion, one shouldn't worry much about a couple of ms of jitter 
in MIDI data. Rather than schedule things in the future with timestamps, 
I prefer to just pretend that I'm running infinitely fast and actually 
poll every ms or two. I don't feel this detracts from performance, but 
maybe if I were writing a sequencer (or PortMidi!), I'd put in some 
extra effort to support timestamps and finer time control.

> Interesting... but as a cross platform library, isn't it portmidi's
> job to know what the underlying driver does?  If the OS driver does
> buffering and merging and tests show that it does it "right", then
> it's easy, we just use the OS's implementation.  If not, the library
> emulates that by doing the buffering itself.
>
> I can understand wanting to keep the library as low level and simple
> as possible, but I can only think of one way to do this, and it seems
> like everyone who wants to do things "right" (i.e. support sysex mixed
> with realtime msgs) would be reimplementing the same functionality.
>   
I agree. PortMidi does not help you to send real-time messages while 
transmitting sysex data, certainly not in any system-independent way. I 
wonder how many, if any, sequencers do this "right"? Although it's an 
interesting and challenging problem, I wonder if there is any demand for 
this. I.e., is anyone depending on a drum machine synchronizing to MIDI 
clocks while dumping patches to a synthesizer?

> BTW, I noticed I common lisp binding in the source.  Are you
> interested in other language bindings?  I wrote one for haskell.
>   
We'll soon be adding pyPortMidi to the PortMidi release, and I'd be 
happy to add haskell bindings. The advantage is that if we put them in 
one place, it's possible to say "SVN version 83 was compiled and tested 
with haskell." The disadvantage is that people may expect every change 
to be tested across Common Lisp, C, haskell, python, etc, and in fact 
that's not practical without a lot more effort.

-Roger