From wakefield at mat.ucsb.edu Sat Nov 4 08:55:38 2006 From: wakefield at mat.ucsb.edu (Graham Wakefield) Date: Sat Nov 4 08:55:45 2006 Subject: [Tdg] Hi all - what's the status? Message-ID: <111EED5B-E056-4977-B15F-9AB5F3903E0F@mat.ucsb.edu> HI all, I'll be back in full Santa Barbara action on the 12th of Nov, and I was wondering how things are progressing, whether there's anything that has come out of the meetings that I should be thinking about over the next week, and so on. Anyway, hello all Graham From wesley.hoke at gmail.com Sat Nov 4 11:24:33 2006 From: wesley.hoke at gmail.com (Wesley Smith) Date: Sat Nov 4 11:24:40 2006 Subject: [Tdg] Hi all - what's the status? In-Reply-To: <111EED5B-E056-4977-B15F-9AB5F3903E0F@mat.ucsb.edu> References: <111EED5B-E056-4977-B15F-9AB5F3903E0F@mat.ucsb.edu> Message-ID: <1079b050611041124y7c4fa94eja355b106bbd2a7f9@mail.gmail.com> Well, Not a whole lot. I was hoping to have the documents that Rama has from the spring available somewhere so we could push the design some more, but they're not available right now and probably not until Rama gets back from China about the same time you get back. I've been rading some papers on realtime multimedia systems from ETH in Zurich. There's a group called Corebounce that has produced a piece of software called Soundium that's quite nice. Here's a link: http://www.corebounce.org/wiki/ . In the next week or so I should have access to the source code of Soundium and my plan was to go through it and see how they structured things. Welcome Back Soonish! wes On 11/4/06, Graham Wakefield wrote: > HI all, > > I'll be back in full Santa Barbara action on the 12th of Nov, and I > was wondering how things are progressing, whether there's anything > that has come out of the meetings that I should be thinking about > over the next week, and so on. > > Anyway, hello all > > Graham > _______________________________________________ > Tdg mailing list > Tdg@mat.ucsb.edu > http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg > From rch at umail.ucsb.edu Sat Nov 4 19:25:06 2006 From: rch at umail.ucsb.edu (Rama C. Hoetzlein) Date: Sat Nov 4 19:24:52 2006 Subject: [Tdg] Hi all - what's the status? In-Reply-To: <1079b050611041124y7c4fa94eja355b106bbd2a7f9@mail.gmail.com> References: <111EED5B-E056-4977-B15F-9AB5F3903E0F@mat.ucsb.edu> <1079b050611041124y7c4fa94eja355b106bbd2a7f9@mail.gmail.com> Message-ID: <20061104192506.q3beh7znokc8s8kk@webaccess.umail.ucsb.edu> Hey. Joshua said he would set up a wiki for TDG, but was really busy with ACM last week. Maybe someone could ask him how it's going?.. If wiki is setup, I can post the original design notes. In other news, Sarah and Basak have both expressed interest in developing TDG and have been meeting with us. Several meetings were held to refresh our memories on design. Angus and myself both setup new computers in IGERT trailer to work on TDG. When I get back, I will contribute a system to be reformatted with Linux for cross-platform testing. I'll be back Nov 15th. Regards, Rama Wesley Smith wrote: > Well, > Not a whole lot. I was hoping to have the documents that Rama has > from the spring available somewhere so we could push the design some > more, but they're not available right now and probably not until Rama > gets back from China about the same time you get back. I've been > rading some papers on realtime multimedia systems from ETH in Zurich. > There's a group called Corebounce that has produced a piece of > software called Soundium that's quite nice. Here's a link: > http://www.corebounce.org/wiki/ . In the next week or so I should > have access to the source code of Soundium and my plan was to go > through it and see how they structured things. > > Welcome Back Soonish! > wes > > On 11/4/06, Graham Wakefield wrote: >> HI all, >> >> I'll be back in full Santa Barbara action on the 12th of Nov, and I >> was wondering how things are progressing, whether there's anything >> that has come out of the meetings that I should be thinking about >> over the next week, and so on. >> >> Anyway, hello all >> >> Graham >> _______________________________________________ >> Tdg mailing list >> Tdg@mat.ucsb.edu >> http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg >> > _______________________________________________ > Tdg mailing list > Tdg@mat.ucsb.edu > http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg -- Rama Hoetzlein rch@umail.ucsb.edu From wesley.hoke at gmail.com Sun Nov 5 10:23:05 2006 From: wesley.hoke at gmail.com (Wesley Smith) Date: Sun Nov 5 10:23:12 2006 Subject: [Tdg] Hi all - what's the status? In-Reply-To: <20061104192506.q3beh7znokc8s8kk@webaccess.umail.ucsb.edu> References: <111EED5B-E056-4977-B15F-9AB5F3903E0F@mat.ucsb.edu> <1079b050611041124y7c4fa94eja355b106bbd2a7f9@mail.gmail.com> <20061104192506.q3beh7znokc8s8kk@webaccess.umail.ucsb.edu> Message-ID: <1079b050611051023k1d605a8fpfc4857681e5cc9e7@mail.gmail.com> Alright. I'll get on Joshua about the wiki on Monday. In the mean time do you think you can post the documents Rama? I don't see why not having a wiki is holding up[ posting them on the net. I wasn't at most of the spring meetings, so I have no idea what was accomplished. It's really difficult to know what to do without those docs. best, wes On 11/4/06, Rama C. Hoetzlein wrote: > Hey. Joshua said he would set up a wiki for TDG, but was really busy > with ACM last week. Maybe someone could ask him how it's going?.. If > wiki is setup, I can post the original design notes. In other news, > Sarah and Basak have both expressed interest in developing TDG and > have been meeting with us. Several meetings were held to refresh our > memories on design. Angus and myself both setup new computers in IGERT > trailer to work on TDG. When I get back, I will contribute a system to > be reformatted with Linux for cross-platform testing. I'll be back Nov > 15th. > > Regards, > Rama > > > > Wesley Smith wrote: > > > Well, > > Not a whole lot. I was hoping to have the documents that Rama has > > from the spring available somewhere so we could push the design some > > more, but they're not available right now and probably not until Rama > > gets back from China about the same time you get back. I've been > > rading some papers on realtime multimedia systems from ETH in Zurich. > > There's a group called Corebounce that has produced a piece of > > software called Soundium that's quite nice. Here's a link: > > http://www.corebounce.org/wiki/ . In the next week or so I should > > have access to the source code of Soundium and my plan was to go > > through it and see how they structured things. > > > > Welcome Back Soonish! > > wes > > > > On 11/4/06, Graham Wakefield wrote: > >> HI all, > >> > >> I'll be back in full Santa Barbara action on the 12th of Nov, and I > >> was wondering how things are progressing, whether there's anything > >> that has come out of the meetings that I should be thinking about > >> over the next week, and so on. > >> > >> Anyway, hello all > >> > >> Graham > >> _______________________________________________ > >> Tdg mailing list > >> Tdg@mat.ucsb.edu > >> http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg > >> > > _______________________________________________ > > Tdg mailing list > > Tdg@mat.ucsb.edu > > http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg > > > > -- > Rama Hoetzlein > rch@umail.ucsb.edu > > _______________________________________________ > Tdg mailing list > Tdg@mat.ucsb.edu > http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg > From wesley.hoke at gmail.com Mon Nov 6 15:14:50 2006 From: wesley.hoke at gmail.com (Wesley Smith) Date: Mon Nov 6 15:14:59 2006 Subject: [Tdg] Current State Message-ID: <1079b050611061514t32d1ca44nefd041a311ebcc70@mail.gmail.com> Hi everyone, Rama sent me his docs and I've uploaded them to http://www.mat.ucsb.edu/~whsmith/TDG-Notes.zip . Here's a snippet from one of the docs about the ideas behind TDG: "Vision: Hope to define what it means to do Digital Meida Arts by crystalizing the right tool to enable it." I think this would be a great place to start thinking at a meeting later this week. I don't think we're going to nail it on the head the first time but we have to get started at some point. We all have our specific ideas of what multimedia is and our various ways of composing. There is however enough common ground to start fleshing out some of the concepts that will separate TDG from other systems. A good chunk of this is in the docs. A decent number of people out of town this week, but of those who are in town, who would like to meet later this week on Thursday on Friday. We could take the time between now and then to get acquainted with the docs that Rama has put together and figure out what needs to happen next. What do people think? wes From wakefield at mat.ucsb.edu Tue Nov 14 00:39:39 2006 From: wakefield at mat.ucsb.edu (Graham Wakefield) Date: Tue Nov 14 00:39:45 2006 Subject: [Tdg] Current State In-Reply-To: <1079b050611061514t32d1ca44nefd041a311ebcc70@mail.gmail.com> References: <1079b050611061514t32d1ca44nefd041a311ebcc70@mail.gmail.com> Message-ID: Hi- What time do tdg/minters meet on Tuesdays, and where, if at all? On Nov 6, 2006, at 3:14 PM, Wesley Smith wrote: > Hi everyone, > Rama sent me his docs and I've uploaded them to > http://www.mat.ucsb.edu/~whsmith/TDG-Notes.zip . Here's a snippet > from one of the docs about the ideas behind TDG: > > "Vision: Hope to define what it means to do Digital Meida Arts > by crystalizing the right tool to enable it." > > I think this would be a great place to start thinking at a meeting > later this week. I don't think we're going to nail it on the head the > first time but we have to get started at some point. We all have our > specific ideas of what multimedia is and our various ways of > composing. There is however enough common ground to start fleshing > out some of the concepts that will separate TDG from other systems. > A good chunk of this is in the docs. A decent number of people out of > town this week, but of those who are in town, who would like to meet > later this week on Thursday on Friday. We could take the time between > now and then to get acquainted with the docs that Rama has put > together and figure out what needs to happen next. What do people > think? > > wes > _______________________________________________ > Tdg mailing list > Tdg@mat.ucsb.edu > http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg From rch at umail.ucsb.edu Wed Nov 15 15:04:37 2006 From: rch at umail.ucsb.edu (Rama Hoetzlein) Date: Wed Nov 15 15:04:53 2006 Subject: [Tdg] Current State In-Reply-To: References: <1079b050611061514t32d1ca44nefd041a311ebcc70@mail.gmail.com> Message-ID: <455B9D05.1000808@umail.ucsb.edu> Hi all.. I just got back.. Our current meeting times are Tuesday at 1 and 5pm.. Most people come at 5pm. Anyone who can make one of these time should feel free to attend. 4th Fl. SH. There are no good overlaps for everyone, so I will be there both times as usual. I am hoping one topic of discussion in next meeting(s) will be good alternate times when everyone involved could meet together. Also, next week I intend to make some progress on a wiki/forum we can all post thoughts to (anyone wanna help?). Regards, Rama Graham Wakefield wrote: > Hi- > > What time do tdg/minters meet on Tuesdays, and where, if at all? > > On Nov 6, 2006, at 3:14 PM, Wesley Smith wrote: > >> Hi everyone, >> Rama sent me his docs and I've uploaded them to >> http://www.mat.ucsb.edu/~whsmith/TDG-Notes.zip . Here's a snippet >> from one of the docs about the ideas behind TDG: >> >> "Vision: Hope to define what it means to do Digital Meida Arts >> by crystalizing the right tool to enable it." >> >> I think this would be a great place to start thinking at a meeting >> later this week. I don't think we're going to nail it on the head the >> first time but we have to get started at some point. We all have our >> specific ideas of what multimedia is and our various ways of >> composing. There is however enough common ground to start fleshing >> out some of the concepts that will separate TDG from other systems. >> A good chunk of this is in the docs. A decent number of people out of >> town this week, but of those who are in town, who would like to meet >> later this week on Thursday on Friday. We could take the time between >> now and then to get acquainted with the docs that Rama has put >> together and figure out what needs to happen next. What do people >> think? >> >> wes >> _______________________________________________ >> Tdg mailing list >> Tdg@mat.ucsb.edu >> http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg > > > _______________________________________________ > Tdg mailing list > Tdg@mat.ucsb.edu > http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg From rch at umail.ucsb.edu Wed Nov 15 15:27:50 2006 From: rch at umail.ucsb.edu (Rama Hoetzlein) Date: Wed Nov 15 15:28:06 2006 Subject: [Tdg] Current State (+) In-Reply-To: <455B9D05.1000808@umail.ucsb.edu> References: <1079b050611061514t32d1ca44nefd041a311ebcc70@mail.gmail.com> <455B9D05.1000808@umail.ucsb.edu> Message-ID: <455BA276.6060707@umail.ucsb.edu> Our TDG meetings have been productive for review of past ideas so far, but perhaps a little unfocused lately (in no small part because of ACM and because of my absinthe). Therefore I'd like to proposal a group goal... I think we are ready to start working on a succinct mission statement for this project. What our project aims to do, what are our sub-goals and possible applications, what work we intend to accomplish, and who would like to be involved in real work (ie. programming) and on what parts (sub-systems). We already know a lot of this, but lets formalize it together. Perhaps we can work together to formulate our goals into something presentable in a 2-3 page executive summary (well, faculty summary anyway). There is no real hurry for this, but I'd like to suggest - as Wes did - that we keep working on it until it exists and we're happy with a first draft. rama Rama Hoetzlein wrote: > Hi all.. I just got back.. > > Our current meeting times are Tuesday at 1 and 5pm.. Most people come > at 5pm. > Anyone who can make one of these time should feel free to attend. 4th > Fl. SH. > There are no good overlaps for everyone, so I will be there both times > as usual. > > I am hoping one topic of discussion in next meeting(s) will be good > alternate times > when everyone involved could meet together. > > Also, next week I intend to make some progress on a wiki/forum we can > all post thoughts to (anyone wanna help?). > > Regards, > Rama > > Graham Wakefield wrote: > >> Hi- >> >> What time do tdg/minters meet on Tuesdays, and where, if at all? >> >> On Nov 6, 2006, at 3:14 PM, Wesley Smith wrote: >> >>> Hi everyone, >>> Rama sent me his docs and I've uploaded them to >>> http://www.mat.ucsb.edu/~whsmith/TDG-Notes.zip . Here's a snippet >>> from one of the docs about the ideas behind TDG: >>> >>> "Vision: Hope to define what it means to do Digital Meida Arts >>> by crystalizing the right tool to enable it." >>> >>> I think this would be a great place to start thinking at a meeting >>> later this week. I don't think we're going to nail it on the head the >>> first time but we have to get started at some point. We all have our >>> specific ideas of what multimedia is and our various ways of >>> composing. There is however enough common ground to start fleshing >>> out some of the concepts that will separate TDG from other systems. >>> A good chunk of this is in the docs. A decent number of people out of >>> town this week, but of those who are in town, who would like to meet >>> later this week on Thursday on Friday. We could take the time between >>> now and then to get acquainted with the docs that Rama has put >>> together and figure out what needs to happen next. What do people >>> think? >>> >>> wes >>> _______________________________________________ >>> Tdg mailing list >>> Tdg@mat.ucsb.edu >>> http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg >> >> >> >> _______________________________________________ >> Tdg mailing list >> Tdg@mat.ucsb.edu >> http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg > > > > _______________________________________________ > Tdg mailing list > Tdg@mat.ucsb.edu > http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg From rch at umail.ucsb.edu Thu Nov 16 02:28:16 2006 From: rch at umail.ucsb.edu (Rama Hoetzlein) Date: Thu Nov 16 02:28:33 2006 Subject: [Tdg] Re: TDG In-Reply-To: <455BA276.6060707@umail.ucsb.edu> References: <1079b050611061514t32d1ca44nefd041a311ebcc70@mail.gmail.com> <455B9D05.1000808@umail.ucsb.edu> <455BA276.6060707@umail.ucsb.edu> Message-ID: <455C3D40.1080501@umail.ucsb.edu> Cancel what I said about unfocused : )... I'm very pleased with how things have been going with the group and have come together on this project. I am suprised by the level of interest in this project and not sure what to do about it. Our meetings have been very productive so far. I'm just trying to get back into the swing of things after ACM and traveling (probably need more sleep : ). Anyway, don't mind me..... Stop by the meetings. I'll be there. Regards, Rama From rch at umail.ucsb.edu Thu Nov 16 02:44:49 2006 From: rch at umail.ucsb.edu (Rama Hoetzlein) Date: Thu Nov 16 02:45:07 2006 Subject: [TDG] In-Reply-To: <455C3D40.1080501@umail.ucsb.edu> References: <1079b050611061514t32d1ca44nefd041a311ebcc70@mail.gmail.com> <455B9D05.1000808@umail.ucsb.edu> <455BA276.6060707@umail.ucsb.edu> <455C3D40.1080501@umail.ucsb.edu> Message-ID: <455C4121.4020804@umail.ucsb.edu> Hey again, While in China I met a tech guy from Ars Electronica who told me about this software called VVVV. I've never heard of it, but it looks very cool. Does a lot of graphics stuff fast: http://vvvv.org/tiki-view_blog.php?blogId=4 Also can do sound-driven visuals and real-time 3D. Similar visual interface to Max/MSP but without many annoyances. I haven't played with it yet. Regards, R From wesley.hoke at gmail.com Thu Nov 16 08:11:50 2006 From: wesley.hoke at gmail.com (Wesley Smith) Date: Thu Nov 16 08:11:58 2006 Subject: [TDG] In-Reply-To: <455C4121.4020804@umail.ucsb.edu> References: <1079b050611061514t32d1ca44nefd041a311ebcc70@mail.gmail.com> <455B9D05.1000808@umail.ucsb.edu> <455BA276.6060707@umail.ucsb.edu> <455C3D40.1080501@umail.ucsb.edu> <455C4121.4020804@umail.ucsb.edu> Message-ID: <1079b050611160811i6151524dwde77e970a9c270eb@mail.gmail.com> It's based on DirectX wes On 11/16/06, Rama Hoetzlein wrote: > Hey again, > > While in China I met a tech guy from Ars Electronica who told me about > this software called VVVV. I've never heard of it, but it looks very > cool. Does a lot of graphics stuff fast: > > http://vvvv.org/tiki-view_blog.php?blogId=4 > > Also can do sound-driven visuals and real-time 3D. Similar visual > interface to Max/MSP but without many annoyances. I haven't played with > it yet. > > Regards, > R > > _______________________________________________ > Tdg mailing list > Tdg@mat.ucsb.edu > http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg > From lists at grahamwakefield.net Fri Nov 17 20:07:17 2006 From: lists at grahamwakefield.net (Graham Wakefield) Date: Fri Nov 17 20:46:40 2006 Subject: [Tdg] Re: [glv] Message Passing In-Reply-To: <588ABC59-9261-4772-91A0-D3C36DB1106E@umail.ucsb.edu> References: <1079b050611140017u7f6e066eh2233a09b806ac994@mail.gmail.com> <1079b050611142211w6cd12f2u8f57e7a479e2482@mail.gmail.com> <19CE924D-C0D8-46B2-9B11-AE8CFE66FA44@grahamwakefield.net> <8C410ED8-4E82-410B-8D7B-789B125191E3@grahamwakefield.net> <71EFD6FD-FF64-4EA2-AA5B-E43665820D0F@grahamwakefield.net> <15C6C961-EED3-4C42-98F6-06980E2A55AB@grahamwakefield.net> <588ABC59-9261-4772-91A0-D3C36DB1106E@umail.ucsb.edu> Message-ID: <6828F04A-3B18-4C7E-9AB9-3AB7209FB26D@grahamwakefield.net> After chatting with Rama today about message queues and so on, we came up with a potential model that deals with threads. For example, consider an application that has a central thread running the main loop (running e.g. every 5ms), which has some kind of event manager. The event manager parses an input buffer/queue of event objects on each iteration. Application logic (built-in behaviour, user defined via C++ code, script code, function pointers etc) determines how to respond to each event. Some event may cause other events to be posted; these new events can be placed at the end of the queue (or at the beginning for high priority events) safely, since we are operating within a single thread. Now consider adding a network input facility. Since the network socket exists in a different thread, it will generate new events at interrupt time. Therefore we add a double-buffered event queue to the core application event manager specifically for the network thread. At a specified frequency (some lower division of the main application event loop rate), the event manager will read and process the network input, by swapping the input queues and then reading and processing each event, just as if they were core thread events. Thus different input subsystems can be polled at different rates. These rates could change at runtime. Also, the input buffers should be able to grow at runtime (using e.g. stl::vector or queue). Conversely, if we add audio, we might want to have events generate new notes in the audio thread. Again, we can use a double-buffered queue, but this time have the queue be placed in the audio thread. Any events generated in the core event loop that are specific audio events will be appended to the audio thread's input queue. When the audio thread wakes up (sound card callback) it likewise swaps the input queues and processes any new audio events. Any events that the audio thread generates however will use an input double-buffered queue for audio in the main event loop thread. Crucially, to prevent thread issues, it must be necessary that ANY other thread can only send events to the main event loop thread. So, if receiving a 'note 64' message over the network, the message is turned into an event in the network input queue in the core event loop thread. When the main event loop next looks at the network input, it will find the 'note 64' event and respond by sending a 'make new tone' event to the audio thread's input queue. When the audio callback is next triggered, it should find this event and respond by making sound. This idea can be extended to model heavily threaded applications, and makes remote control of applications very simple to implement. Of course, for a subsystem that already resides in the main thread (e.g. graphics, mouse & keyboard) there is no need to use the double- buffer, they can append directly to the current queue since there is no danger of interrupted access. This idea can be made extensible so that the user could add new subsystems, event types, event handlers etc. On Nov 16, 2006, at 7:18 PM, Jorge Castellanos wrote: > > As I can't attend the meetings, I might be repeating things you > already talked about, but I hope you don't mind. > Here are some thoughts, in case you didn't discuss them already... > > The "event processor" (scheduler ?) would work fine, but keep in > mind that some events might need to run at different rates. OSC, > for example might need to be sent "immediately" while other events > wouldn't mind some latency. > In other words it might be a good idea for the "Event" class to > also have some sort of "rate" or "priority" (a priority queue (?). > Not sure. Something to consider. > > Re: threading... I'm pro double buffering, but considering that > computers are moving to multi-core configurations, and the number > of cores will keep growing, then threads are the way to go. This > way you use a different core. With a single threaded app you would > be wasting the other many cores. The only way to split the work is > by dividing the work into different threads. > That said, double buffering doesn't rule out threads. Meaning a > double buffered priority queue could run in its own thread. > > j > > > On Nov 16, 2006, at 9:58 PM, Graham Wakefield wrote: > > Hmm - threading. > > Should the event queue be double buffered to avoid threading > mishaps (posting an event while another is executing?) > > On Nov 16, 2006, at 5:39 PM, Graham Wakefield wrote: > >> More thoughts on the Event Manager, after a chat today with Eric: >> >> An underlying event engine provides a central event processor, >> which is based around an event queue. The event processor can be >> triggered at a desired rate, which may be the frame rate for example. >> >> Events raised by the operating system (mouse clicks, keyboard >> hits, application level events etc) and from other inputs (OSC >> receivers, MIDI input devices) are placed on the end of the event >> queue. >> >> On each iteration the event processor reads through the queue and >> processes each event in turn, dispatching it to the appropriate >> object by calling the object's generic 'handler' function with the >> event as the argument. >> >> For mouse events, the system will do hit detection and graphic >> tree traversal to find out the appropriate target (mouse events >> will be received from the OS at window-level). Keyboard events >> will be routed to the current TextFocus receiver (not necessarily >> a widget). Other classes of events will be routed to targets in >> other ways, as appropriate. Clearly there is a need to separate >> events by class as well as subtype (see list below). >> >> An object's handler will essentially contain some kind of switch >> based upon the event class and subtype. If the event is not >> handled, it should be passed back to the event manager using >> 'return eventNotHandled' or 'return 1', which can in turn attempt >> to pass the event to the parent item (parent widget for Views, >> other kinds of parents for other objects). Perhaps all objects >> should have a eventParent field? >> >> Anywhere in the application, one could post new events into the >> queue using global functions: >> >> Event::post(event *e); // places at tail of queue. >> Event::immediate(event *e); // places at head of queue. >> >> Events themselves need to be generic enough to handle different >> types of data. A possible structure: >> >> Event { >> int32 class; // e.g. 'mous' >> int32 type; // e.g. 'down' >> double time; // event timestamp - lets us schedule events into >> the future >> int n; // num parameters >> Param ** params; // parameters >> } >> >> Param { >> int32 type; // e.g. 'long' >> size_t size; >> void * data; >> } >> >> Convenience methods could be written for standard event types to >> quickly extract the typical parameters. >> >> The advantage of this generalized system is that a user could also >> define their own event classes and types and parameters. >> >> >> Class type >> Mouse >> down >> up >> drag >> move >> wheel >> Keyboard >> down >> up >> repeat >> modifier (shift etc.) >> Tablet/Joystick >> button >> continuous >> Command/Application >> quit >> save >> copy >> paste >> ...etc >> OSC >> message >> bundle >> MIDI >> noteon >> noteoff >> controller >> sysex >> ...etc. >> ...etc. >> >> >> >> _______________________________________________ >> glv mailing list >> glv@mat.ucsb.edu >> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv > > _______________________________________________ > glv mailing list > glv@mat.ucsb.edu > http://zydeco.mat.ucsb.edu/mailman/listinfo/glv > > _______________________________________________ > glv mailing list > glv@mat.ucsb.edu > http://zydeco.mat.ucsb.edu/mailman/listinfo/glv From lists at grahamwakefield.net Fri Nov 17 20:35:02 2006 From: lists at grahamwakefield.net (Graham Wakefield) Date: Fri Nov 17 20:46:40 2006 Subject: [Tdg] Re: [glv] Message Passing In-Reply-To: <6828F04A-3B18-4C7E-9AB9-3AB7209FB26D@grahamwakefield.net> References: <1079b050611140017u7f6e066eh2233a09b806ac994@mail.gmail.com> <1079b050611142211w6cd12f2u8f57e7a479e2482@mail.gmail.com> <19CE924D-C0D8-46B2-9B11-AE8CFE66FA44@grahamwakefield.net> <8C410ED8-4E82-410B-8D7B-789B125191E3@grahamwakefield.net> <71EFD6FD-FF64-4EA2-AA5B-E43665820D0F@grahamwakefield.net> <15C6C961-EED3-4C42-98F6-06980E2A55AB@grahamwakefield.net> <588ABC59-9261-4772-91A0-D3C36DB1106E@umail.ucsb.edu> <6828F04A-3B18-4C7E-9AB9-3AB7209FB26D@grahamwakefield.net> Message-ID: <1CB7861C-1D3A-4761-971C-F67197F1B095@grahamwakefield.net> And I forgot to mention- Another advantage of this approach is that all application logic resides within a shared memory space and in a single thread, which is a great advantage. Other threads are simply used for low level services. If actions such as creating windows, starting an oscillator etc. are all triggered from a single thread shared memory space, anything can send an event to any other without worrying about locking, critical sections and all that kind of hassle. Also building interfaces (APIs, scripting language interfaces etc) to application logic becomes a lot simpler. What remains to be solved is how to packet events into a standard structure (i.e. of uniform, small size), and how to share arbitrarily large data (blobs) between threads... some kind of automated buffering system is called for I suppose for the latter, possibly using a freelist/pool approach for real time memory management. oao On Nov 17, 2006, at 8:07 PM, Graham Wakefield wrote: > After chatting with Rama today about message queues and so on, we > came up with a potential model that deals with threads. > > For example, consider an application that has a central thread > running the main loop (running e.g. every 5ms), which has some kind > of event manager. The event manager parses an input buffer/queue > of event objects on each iteration. Application logic (built-in > behaviour, user defined via C++ code, script code, function > pointers etc) determines how to respond to each event. Some event > may cause other events to be posted; these new events can be placed > at the end of the queue (or at the beginning for high priority > events) safely, since we are operating within a single thread. > > Now consider adding a network input facility. Since the network > socket exists in a different thread, it will generate new events at > interrupt time. Therefore we add a double-buffered event queue to > the core application event manager specifically for the network > thread. At a specified frequency (some lower division of the main > application event loop rate), the event manager will read and > process the network input, by swapping the input queues and then > reading and processing each event, just as if they were core thread > events. Thus different input subsystems can be polled at different > rates. These rates could change at runtime. Also, the input > buffers should be able to grow at runtime (using e.g. stl::vector > or queue). > > Conversely, if we add audio, we might want to have events generate > new notes in the audio thread. Again, we can use a double-buffered > queue, but this time have the queue be placed in the audio thread. > Any events generated in the core event loop that are specific audio > events will be appended to the audio thread's input queue. When > the audio thread wakes up (sound card callback) it likewise swaps > the input queues and processes any new audio events. Any events > that the audio thread generates however will use an input double- > buffered queue for audio in the main event loop thread. > > Crucially, to prevent thread issues, it must be necessary that ANY > other thread can only send events to the main event loop thread. > So, if receiving a 'note 64' message over the network, the message > is turned into an event in the network input queue in the core > event loop thread. When the main event loop next looks at the > network input, it will find the 'note 64' event and respond by > sending a 'make new tone' event to the audio thread's input queue. > When the audio callback is next triggered, it should find this > event and respond by making sound. > > This idea can be extended to model heavily threaded applications, > and makes remote control of applications very simple to implement. > > Of course, for a subsystem that already resides in the main thread > (e.g. graphics, mouse & keyboard) there is no need to use the > double-buffer, they can append directly to the current queue since > there is no danger of interrupted access. > > This idea can be made extensible so that the user could add new > subsystems, event types, event handlers etc. > > On Nov 16, 2006, at 7:18 PM, Jorge Castellanos wrote: > >> >> As I can't attend the meetings, I might be repeating things you >> already talked about, but I hope you don't mind. >> Here are some thoughts, in case you didn't discuss them already... >> >> The "event processor" (scheduler ?) would work fine, but keep in >> mind that some events might need to run at different rates. OSC, >> for example might need to be sent "immediately" while other events >> wouldn't mind some latency. >> In other words it might be a good idea for the "Event" class to >> also have some sort of "rate" or "priority" (a priority queue (?). >> Not sure. Something to consider. >> >> Re: threading... I'm pro double buffering, but considering that >> computers are moving to multi-core configurations, and the number >> of cores will keep growing, then threads are the way to go. This >> way you use a different core. With a single threaded app you would >> be wasting the other many cores. The only way to split the work is >> by dividing the work into different threads. >> That said, double buffering doesn't rule out threads. Meaning a >> double buffered priority queue could run in its own thread. >> >> j >> >> >> On Nov 16, 2006, at 9:58 PM, Graham Wakefield wrote: >> >> Hmm - threading. >> >> Should the event queue be double buffered to avoid threading >> mishaps (posting an event while another is executing?) >> >> On Nov 16, 2006, at 5:39 PM, Graham Wakefield wrote: >> >>> More thoughts on the Event Manager, after a chat today with Eric: >>> >>> An underlying event engine provides a central event processor, >>> which is based around an event queue. The event processor can be >>> triggered at a desired rate, which may be the frame rate for >>> example. >>> >>> Events raised by the operating system (mouse clicks, keyboard >>> hits, application level events etc) and from other inputs (OSC >>> receivers, MIDI input devices) are placed on the end of the event >>> queue. >>> >>> On each iteration the event processor reads through the queue and >>> processes each event in turn, dispatching it to the appropriate >>> object by calling the object's generic 'handler' function with >>> the event as the argument. >>> >>> For mouse events, the system will do hit detection and graphic >>> tree traversal to find out the appropriate target (mouse events >>> will be received from the OS at window-level). Keyboard events >>> will be routed to the current TextFocus receiver (not necessarily >>> a widget). Other classes of events will be routed to targets in >>> other ways, as appropriate. Clearly there is a need to separate >>> events by class as well as subtype (see list below). >>> >>> An object's handler will essentially contain some kind of switch >>> based upon the event class and subtype. If the event is not >>> handled, it should be passed back to the event manager using >>> 'return eventNotHandled' or 'return 1', which can in turn attempt >>> to pass the event to the parent item (parent widget for Views, >>> other kinds of parents for other objects). Perhaps all objects >>> should have a eventParent field? >>> >>> Anywhere in the application, one could post new events into the >>> queue using global functions: >>> >>> Event::post(event *e); // places at tail of queue. >>> Event::immediate(event *e); // places at head of queue. >>> >>> Events themselves need to be generic enough to handle different >>> types of data. A possible structure: >>> >>> Event { >>> int32 class; // e.g. 'mous' >>> int32 type; // e.g. 'down' >>> double time; // event timestamp - lets us schedule events into >>> the future >>> int n; // num parameters >>> Param ** params; // parameters >>> } >>> >>> Param { >>> int32 type; // e.g. 'long' >>> size_t size; >>> void * data; >>> } >>> >>> Convenience methods could be written for standard event types to >>> quickly extract the typical parameters. >>> >>> The advantage of this generalized system is that a user could >>> also define their own event classes and types and parameters. >>> >>> >>> Class type >>> Mouse >>> down >>> up >>> drag >>> move >>> wheel >>> Keyboard >>> down >>> up >>> repeat >>> modifier (shift etc.) >>> Tablet/Joystick >>> button >>> continuous >>> Command/Application >>> quit >>> save >>> copy >>> paste >>> ...etc >>> OSC >>> message >>> bundle >>> MIDI >>> noteon >>> noteoff >>> controller >>> sysex >>> ...etc. >>> ...etc. >>> >>> >>> >>> _______________________________________________ >>> glv mailing list >>> glv@mat.ucsb.edu >>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >> >> _______________________________________________ >> glv mailing list >> glv@mat.ucsb.edu >> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >> >> _______________________________________________ >> glv mailing list >> glv@mat.ucsb.edu >> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv > > _______________________________________________ > glv mailing list > glv@mat.ucsb.edu > http://zydeco.mat.ucsb.edu/mailman/listinfo/glv From ericnewman at umail.ucsb.edu Fri Nov 17 21:09:54 2006 From: ericnewman at umail.ucsb.edu (Eric Newman) Date: Fri Nov 17 21:10:12 2006 Subject: [Tdg] Re: [glv] Message Passing In-Reply-To: <1CB7861C-1D3A-4761-971C-F67197F1B095@grahamwakefield.net> References: <1079b050611140017u7f6e066eh2233a09b806ac994@mail.gmail.com> <1079b050611142211w6cd12f2u8f57e7a479e2482@mail.gmail.com> <19CE924D-C0D8-46B2-9B11-AE8CFE66FA44@grahamwakefield.net> <8C410ED8-4E82-410B-8D7B-789B125191E3@grahamwakefield.net> <71EFD6FD-FF64-4EA2-AA5B-E43665820D0F@grahamwakefield.net> <15C6C961-EED3-4C42-98F6-06980E2A55AB@grahamwakefield.net> <588ABC59-9261-4772-91A0-D3C36DB1106E@umail.ucsb.edu> <6828F04A-3B18-4C7E-9AB9-3AB7209FB26D@grahamwakefield.net> <1CB7861C-1D3A-4761-971C-F67197F1B095@grahamwakefield.net> Message-ID: Why use a double-buffered scheme when you can just lock a mutex? You wouldn't have to lock it for very long; just long enough to extract the next event from the head of the queue; then you unlock it again; then you pass the event into a switch. So, pthread_mutex_lock(&eventQueue_mutex); Event nextEvent = eventQueue.front(); eventQueue.pop(); pthread_mutex_unlock(&eventQueue_mutex); switch (nextEvent.id) { // handle the event // ... } You'd have to use a mutex lock in a double-buffered scheme anyway (when you swap the buffers). So really, a double buffer is redundant. Plus, events wouldn't be handled as quickly. On Fri, 17 Nov 2006, Graham Wakefield wrote: > And I forgot to mention- > > Another advantage of this approach is that all application logic resides > within a shared memory space and in a single thread, which is a great > advantage. Other threads are simply used for low level services. If actions > such as creating windows, starting an oscillator etc. are all triggered from > a single thread shared memory space, anything can send an event to any other > without worrying about locking, critical sections and all that kind of > hassle. Also building interfaces (APIs, scripting language interfaces etc) > to application logic becomes a lot simpler. > > What remains to be solved is how to packet events into a standard structure > (i.e. of uniform, small size), and how to share arbitrarily large data > (blobs) between threads... some kind of automated buffering system is called > for I suppose for the latter, possibly using a freelist/pool approach for > real time memory management. > > oao > > On Nov 17, 2006, at 8:07 PM, Graham Wakefield wrote: > >> After chatting with Rama today about message queues and so on, we came up >> with a potential model that deals with threads. >> >> For example, consider an application that has a central thread running the >> main loop (running e.g. every 5ms), which has some kind of event manager. >> The event manager parses an input buffer/queue of event objects on each >> iteration. Application logic (built-in behaviour, user defined via C++ >> code, script code, function pointers etc) determines how to respond to each >> event. Some event may cause other events to be posted; these new events >> can be placed at the end of the queue (or at the beginning for high >> priority events) safely, since we are operating within a single thread. >> >> Now consider adding a network input facility. Since the network socket >> exists in a different thread, it will generate new events at interrupt >> time. Therefore we add a double-buffered event queue to the core >> application event manager specifically for the network thread. At a >> specified frequency (some lower division of the main application event loop >> rate), the event manager will read and process the network input, by >> swapping the input queues and then reading and processing each event, just >> as if they were core thread events. Thus different input subsystems can be >> polled at different rates. These rates could change at runtime. Also, the >> input buffers should be able to grow at runtime (using e.g. stl::vector or >> queue). >> >> Conversely, if we add audio, we might want to have events generate new >> notes in the audio thread. Again, we can use a double-buffered queue, but >> this time have the queue be placed in the audio thread. Any events >> generated in the core event loop that are specific audio events will be >> appended to the audio thread's input queue. When the audio thread wakes up >> (sound card callback) it likewise swaps the input queues and processes any >> new audio events. Any events that the audio thread generates however will >> use an input double-buffered queue for audio in the main event loop thread. >> >> Crucially, to prevent thread issues, it must be necessary that ANY other >> thread can only send events to the main event loop thread. So, if >> receiving a 'note 64' message over the network, the message is turned into >> an event in the network input queue in the core event loop thread. When >> the main event loop next looks at the network input, it will find the 'note >> 64' event and respond by sending a 'make new tone' event to the audio >> thread's input queue. When the audio callback is next triggered, it should >> find this event and respond by making sound. >> >> This idea can be extended to model heavily threaded applications, and makes >> remote control of applications very simple to implement. >> >> Of course, for a subsystem that already resides in the main thread (e.g. >> graphics, mouse & keyboard) there is no need to use the double-buffer, they >> can append directly to the current queue since there is no danger of >> interrupted access. >> >> This idea can be made extensible so that the user could add new subsystems, >> event types, event handlers etc. >> >> On Nov 16, 2006, at 7:18 PM, Jorge Castellanos wrote: >> >>> >>> As I can't attend the meetings, I might be repeating things you already >>> talked about, but I hope you don't mind. >>> Here are some thoughts, in case you didn't discuss them already... >>> >>> The "event processor" (scheduler ?) would work fine, but keep in mind that >>> some events might need to run at different rates. OSC, for example might >>> need to be sent "immediately" while other events wouldn't mind some >>> latency. >>> In other words it might be a good idea for the "Event" class to also have >>> some sort of "rate" or "priority" (a priority queue (?). >>> Not sure. Something to consider. >>> >>> Re: threading... I'm pro double buffering, but considering that computers >>> are moving to multi-core configurations, and the number of cores will keep >>> growing, then threads are the way to go. This way you use a different >>> core. With a single threaded app you would be wasting the other many >>> cores. The only way to split the work is by dividing the work into >>> different threads. >>> That said, double buffering doesn't rule out threads. Meaning a double >>> buffered priority queue could run in its own thread. >>> >>> j >>> >>> >>> On Nov 16, 2006, at 9:58 PM, Graham Wakefield wrote: >>> >>> Hmm - threading. >>> >>> Should the event queue be double buffered to avoid threading mishaps >>> (posting an event while another is executing?) >>> >>> On Nov 16, 2006, at 5:39 PM, Graham Wakefield wrote: >>> >>>> More thoughts on the Event Manager, after a chat today with Eric: >>>> >>>> An underlying event engine provides a central event processor, which is >>>> based around an event queue. The event processor can be triggered at a >>>> desired rate, which may be the frame rate for example. >>>> >>>> Events raised by the operating system (mouse clicks, keyboard hits, >>>> application level events etc) and from other inputs (OSC receivers, MIDI >>>> input devices) are placed on the end of the event queue. >>>> >>>> On each iteration the event processor reads through the queue and >>>> processes each event in turn, dispatching it to the appropriate object by >>>> calling the object's generic 'handler' function with the event as the >>>> argument. >>>> >>>> For mouse events, the system will do hit detection and graphic tree >>>> traversal to find out the appropriate target (mouse events will be >>>> received from the OS at window-level). Keyboard events will be routed to >>>> the current TextFocus receiver (not necessarily a widget). Other classes >>>> of events will be routed to targets in other ways, as appropriate. >>>> Clearly there is a need to separate events by class as well as subtype >>>> (see list below). >>>> >>>> An object's handler will essentially contain some kind of switch based >>>> upon the event class and subtype. If the event is not handled, it should >>>> be passed back to the event manager using 'return eventNotHandled' or >>>> 'return 1', which can in turn attempt to pass the event to the parent >>>> item (parent widget for Views, other kinds of parents for other objects). >>>> Perhaps all objects should have a eventParent field? >>>> >>>> Anywhere in the application, one could post new events into the queue >>>> using global functions: >>>> >>>> Event::post(event *e); // places at tail of queue. >>>> Event::immediate(event *e); // places at head of queue. >>>> >>>> Events themselves need to be generic enough to handle different types of >>>> data. A possible structure: >>>> >>>> Event { >>>> int32 class; // e.g. 'mous' >>>> int32 type; // e.g. 'down' >>>> double time; // event timestamp - lets us schedule events into the >>>> future >>>> int n; // num parameters >>>> Param ** params; // parameters >>>> } >>>> >>>> Param { >>>> int32 type; // e.g. 'long' >>>> size_t size; >>>> void * data; >>>> } >>>> >>>> Convenience methods could be written for standard event types to quickly >>>> extract the typical parameters. >>>> >>>> The advantage of this generalized system is that a user could also define >>>> their own event classes and types and parameters. >>>> >>>> >>>> Class type >>>> Mouse down >>>> up >>>> drag >>>> move >>>> wheel >>>> Keyboard >>>> down >>>> up >>>> repeat >>>> modifier (shift etc.) >>>> Tablet/Joystick >>>> button >>>> continuous >>>> Command/Application >>>> quit >>>> save >>>> copy >>>> paste >>>> ...etc >>>> OSC message >>>> bundle >>>> MIDI >>>> noteon >>>> noteoff >>>> controller >>>> sysex >>>> ...etc. >>>> ...etc. >>>> >>>> >>>> >>>> _______________________________________________ >>>> glv mailing list >>>> glv@mat.ucsb.edu >>>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >>> >>> _______________________________________________ >>> glv mailing list >>> glv@mat.ucsb.edu >>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >>> >>> _______________________________________________ >>> glv mailing list >>> glv@mat.ucsb.edu >>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >> >> _______________________________________________ >> glv mailing list >> glv@mat.ucsb.edu >> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv > > _______________________________________________ > glv mailing list > glv@mat.ucsb.edu > http://zydeco.mat.ucsb.edu/mailman/listinfo/glv From rch at umail.ucsb.edu Fri Nov 17 22:02:01 2006 From: rch at umail.ucsb.edu (Rama Hoetzlein) Date: Fri Nov 17 22:02:15 2006 Subject: [Tdg] Re: [glv] Message Passing In-Reply-To: References: <1079b050611140017u7f6e066eh2233a09b806ac994@mail.gmail.com> <1079b050611142211w6cd12f2u8f57e7a479e2482@mail.gmail.com> <19CE924D-C0D8-46B2-9B11-AE8CFE66FA44@grahamwakefield.net> <8C410ED8-4E82-410B-8D7B-789B125191E3@grahamwakefield.net> <71EFD6FD-FF64-4EA2-AA5B-E43665820D0F@grahamwakefield.net> <15C6C961-EED3-4C42-98F6-06980E2A55AB@grahamwakefield.net> <588ABC59-9261-4772-91A0-D3C36DB1106E@umail.ucsb.edu> <6828F04A-3B18-4C7E-9AB9-3AB7209FB26D@grahamwakefield.net> <1CB7861C-1D3A-4761-971C-F67197F1B095@grahamwakefield.net> Message-ID: <455EA1D9.8080101@umail.ucsb.edu> Good point. I think the final solution will depend on where the lock happens though. One problem with mutexes is that if you lock during event handling, then if another thread sends a message during lock it will have to be rejected since there is no double-buffering and the queue is locked. This could be a critical message, such as a keyboard press. Graham and I discussed the possibility of locking based on messages generation instead, which has the advantage that all messages are still passed into the queue (the generation of a message briefly locks that event handling sub-system). The disadvantage is that the event handler, which is a renderer or an audio subsystem, may skip frames or decide to drop events. However, this should be less noticable than missing critical events. So although it may be delayed a timing cycle, at least the sub-system still gets to decide what to do for every event generated. I'm still not sure about the double-buffering or not. I know in rendering I need double-buffering because I need to distinguish draw calls with current timestamps versus calls made for a future time, but I don't know about other sub-systems. Rama Eric Newman wrote: > > Why use a double-buffered scheme when you can just lock a mutex? You > wouldn't have to lock it for very long; just long enough to extract > the next event from the head of the queue; then you unlock it again; > then you pass the event into a switch. > > So, > > pthread_mutex_lock(&eventQueue_mutex); > Event nextEvent = eventQueue.front(); > eventQueue.pop(); > pthread_mutex_unlock(&eventQueue_mutex); > > switch (nextEvent.id) > { > // handle the event > // ... > } > > You'd have to use a mutex lock in a double-buffered scheme anyway > (when you swap the buffers). So really, a double buffer is > redundant. Plus, events wouldn't be handled as quickly. > > > On Fri, 17 Nov 2006, Graham Wakefield wrote: > >> And I forgot to mention- >> >> Another advantage of this approach is that all application logic >> resides within a shared memory space and in a single thread, which is >> a great advantage. Other threads are simply used for low level >> services. If actions such as creating windows, starting an >> oscillator etc. are all triggered from a single thread shared memory >> space, anything can send an event to any other without worrying about >> locking, critical sections and all that kind of hassle. Also >> building interfaces (APIs, scripting language interfaces etc) to >> application logic becomes a lot simpler. >> >> What remains to be solved is how to packet events into a standard >> structure (i.e. of uniform, small size), and how to share arbitrarily >> large data (blobs) between threads... some kind of automated >> buffering system is called for I suppose for the latter, possibly >> using a freelist/pool approach for real time memory management. >> >> oao >> >> On Nov 17, 2006, at 8:07 PM, Graham Wakefield wrote: >> >>> After chatting with Rama today about message queues and so on, we >>> came up with a potential model that deals with threads. >>> >>> For example, consider an application that has a central thread >>> running the main loop (running e.g. every 5ms), which has some kind >>> of event manager. The event manager parses an input buffer/queue of >>> event objects on each iteration. Application logic (built-in >>> behaviour, user defined via C++ code, script code, function pointers >>> etc) determines how to respond to each event. Some event may cause >>> other events to be posted; these new events can be placed at the end >>> of the queue (or at the beginning for high priority events) safely, >>> since we are operating within a single thread. >>> >>> Now consider adding a network input facility. Since the network >>> socket exists in a different thread, it will generate new events at >>> interrupt time. Therefore we add a double-buffered event queue to >>> the core application event manager specifically for the network >>> thread. At a specified frequency (some lower division of the main >>> application event loop rate), the event manager will read and >>> process the network input, by swapping the input queues and then >>> reading and processing each event, just as if they were core thread >>> events. Thus different input subsystems can be polled at different >>> rates. These rates could change at runtime. Also, the input >>> buffers should be able to grow at runtime (using e.g. stl::vector or >>> queue). >>> >>> Conversely, if we add audio, we might want to have events generate >>> new notes in the audio thread. Again, we can use a double-buffered >>> queue, but this time have the queue be placed in the audio thread. >>> Any events generated in the core event loop that are specific audio >>> events will be appended to the audio thread's input queue. When the >>> audio thread wakes up (sound card callback) it likewise swaps the >>> input queues and processes any new audio events. Any events that >>> the audio thread generates however will use an input double-buffered >>> queue for audio in the main event loop thread. >>> >>> Crucially, to prevent thread issues, it must be necessary that ANY >>> other thread can only send events to the main event loop thread. >>> So, if receiving a 'note 64' message over the network, the message >>> is turned into an event in the network input queue in the core event >>> loop thread. When the main event loop next looks at the network >>> input, it will find the 'note 64' event and respond by sending a >>> 'make new tone' event to the audio thread's input queue. When the >>> audio callback is next triggered, it should find this event and >>> respond by making sound. >>> >>> This idea can be extended to model heavily threaded applications, >>> and makes remote control of applications very simple to implement. >>> >>> Of course, for a subsystem that already resides in the main thread >>> (e.g. graphics, mouse & keyboard) there is no need to use the >>> double-buffer, they can append directly to the current queue since >>> there is no danger of interrupted access. >>> >>> This idea can be made extensible so that the user could add new >>> subsystems, event types, event handlers etc. >>> >>> On Nov 16, 2006, at 7:18 PM, Jorge Castellanos wrote: >>> >>>> >>>> As I can't attend the meetings, I might be repeating things you >>>> already talked about, but I hope you don't mind. >>>> Here are some thoughts, in case you didn't discuss them already... >>>> >>>> The "event processor" (scheduler ?) would work fine, but keep in >>>> mind that some events might need to run at different rates. OSC, >>>> for example might need to be sent "immediately" while other events >>>> wouldn't mind some latency. >>>> In other words it might be a good idea for the "Event" class to >>>> also have some sort of "rate" or "priority" (a priority queue (?). >>>> Not sure. Something to consider. >>>> >>>> Re: threading... I'm pro double buffering, but considering that >>>> computers are moving to multi-core configurations, and the number >>>> of cores will keep growing, then threads are the way to go. This >>>> way you use a different core. With a single threaded app you would >>>> be wasting the other many cores. The only way to split the work is >>>> by dividing the work into different threads. >>>> That said, double buffering doesn't rule out threads. Meaning a >>>> double buffered priority queue could run in its own thread. >>>> >>>> j >>>> >>>> >>>> On Nov 16, 2006, at 9:58 PM, Graham Wakefield wrote: >>>> >>>> Hmm - threading. >>>> >>>> Should the event queue be double buffered to avoid threading >>>> mishaps (posting an event while another is executing?) >>>> >>>> On Nov 16, 2006, at 5:39 PM, Graham Wakefield wrote: >>>> >>>>> More thoughts on the Event Manager, after a chat today with Eric: >>>>> >>>>> An underlying event engine provides a central event processor, >>>>> which is based around an event queue. The event processor can be >>>>> triggered at a desired rate, which may be the frame rate for example. >>>>> >>>>> Events raised by the operating system (mouse clicks, keyboard >>>>> hits, application level events etc) and from other inputs (OSC >>>>> receivers, MIDI input devices) are placed on the end of the event >>>>> queue. >>>>> >>>>> On each iteration the event processor reads through the queue and >>>>> processes each event in turn, dispatching it to the appropriate >>>>> object by calling the object's generic 'handler' function with the >>>>> event as the argument. >>>>> >>>>> For mouse events, the system will do hit detection and graphic >>>>> tree traversal to find out the appropriate target (mouse events >>>>> will be received from the OS at window-level). Keyboard events >>>>> will be routed to the current TextFocus receiver (not necessarily >>>>> a widget). Other classes of events will be routed to targets in >>>>> other ways, as appropriate. Clearly there is a need to separate >>>>> events by class as well as subtype (see list below). >>>>> >>>>> An object's handler will essentially contain some kind of switch >>>>> based upon the event class and subtype. If the event is not >>>>> handled, it should be passed back to the event manager using >>>>> 'return eventNotHandled' or 'return 1', which can in turn attempt >>>>> to pass the event to the parent item (parent widget for Views, >>>>> other kinds of parents for other objects). Perhaps all objects >>>>> should have a eventParent field? >>>>> >>>>> Anywhere in the application, one could post new events into the >>>>> queue using global functions: >>>>> >>>>> Event::post(event *e); // places at tail of queue. >>>>> Event::immediate(event *e); // places at head of queue. >>>>> >>>>> Events themselves need to be generic enough to handle different >>>>> types of data. A possible structure: >>>>> >>>>> Event { >>>>> int32 class; // e.g. 'mous' >>>>> int32 type; // e.g. 'down' >>>>> double time; // event timestamp - lets us schedule >>>>> events into the future >>>>> int n; // num parameters >>>>> Param ** params; // parameters >>>>> } >>>>> >>>>> Param { >>>>> int32 type; // e.g. 'long' >>>>> size_t size; >>>>> void * data; >>>>> } >>>>> >>>>> Convenience methods could be written for standard event types to >>>>> quickly extract the typical parameters. >>>>> >>>>> The advantage of this generalized system is that a user could also >>>>> define their own event classes and types and parameters. >>>>> >>>>> >>>>> Class type >>>>> Mouse down >>>>> up >>>>> drag >>>>> move >>>>> wheel >>>>> Keyboard >>>>> down >>>>> up >>>>> repeat >>>>> modifier (shift etc.) >>>>> Tablet/Joystick >>>>> button >>>>> continuous >>>>> Command/Application >>>>> quit >>>>> save >>>>> copy >>>>> paste >>>>> ...etc >>>>> OSC message >>>>> bundle >>>>> MIDI >>>>> noteon >>>>> noteoff >>>>> controller >>>>> sysex >>>>> ...etc. >>>>> ...etc. >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> glv mailing list >>>>> glv@mat.ucsb.edu >>>>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >>>> >>>> >>>> _______________________________________________ >>>> glv mailing list >>>> glv@mat.ucsb.edu >>>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >>>> >>>> _______________________________________________ >>>> glv mailing list >>>> glv@mat.ucsb.edu >>>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >>> >>> >>> _______________________________________________ >>> glv mailing list >>> glv@mat.ucsb.edu >>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >> >> >> _______________________________________________ >> glv mailing list >> glv@mat.ucsb.edu >> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv > > _______________________________________________ > Tdg mailing list > Tdg@mat.ucsb.edu > http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg From alex at neisis.net Fri Nov 17 22:54:29 2006 From: alex at neisis.net (Alex Norman) Date: Fri Nov 17 22:54:39 2006 Subject: [Tdg] Re: [glv] Message Passing In-Reply-To: <455EA1D9.8080101@umail.ucsb.edu> References: <19CE924D-C0D8-46B2-9B11-AE8CFE66FA44@grahamwakefield.net> <8C410ED8-4E82-410B-8D7B-789B125191E3@grahamwakefield.net> <71EFD6FD-FF64-4EA2-AA5B-E43665820D0F@grahamwakefield.net> <15C6C961-EED3-4C42-98F6-06980E2A55AB@grahamwakefield.net> <588ABC59-9261-4772-91A0-D3C36DB1106E@umail.ucsb.edu> <6828F04A-3B18-4C7E-9AB9-3AB7209FB26D@grahamwakefield.net> <1CB7861C-1D3A-4761-971C-F67197F1B095@grahamwakefield.net> <455EA1D9.8080101@umail.ucsb.edu> Message-ID: <20061118065429.GX21215@silverninja.net> On 0, Rama Hoetzlein wrote: > Good point. I think the final solution will depend on where the lock > happens though. > > One problem with mutexes is that if you lock during event handling, then > if another thread sends a message during lock it will have to be > rejected since there is no double-buffering and the queue is locked. > This could be a critical message, such as a keyboard press. I guess you're talking about some sort of "try_lock" function here? Otherwise the thread that is writing on the queue really has no idea if it locked or waited for the lock, unless you add some b.s. after every mutex_lock call. So this doesn't result in a dropped message, it simply results in a [if you've written it correctly], very tiny delay time. > > Graham and I discussed the possibility of locking based on messages > generation instead, which has the advantage that all messages are still > passed into the queue (the generation of a message briefly locks that > event handling sub-system). The disadvantage is that the event handler, > which is a renderer or an audio subsystem, may skip frames or decide to > drop events. However, this should be less noticable than missing > critical events. So although it may be delayed a timing cycle, at least > the sub-system still gets to decide what to do for every event generated. I don't quite understand what you mean by locking based on generation. To be safe, using mutexes, you have to lock when you put stuff on the queue and when you take stuff off, or, if you know you only have one reader and one writer, at least when you update or read the indices. > > I'm still not sure about the double-buffering or not. I know in > rendering I need double-buffering because I need to distinguish draw > calls with current timestamps versus calls made for a future time, but I > don't know about other sub-systems. I am also unsure of double buffering. It seems that if you write your queue and dequeueing functions well and set the priorities of threads intelligently then using mutexes shouldn't really be a problem. It also makes it so that the event handler doesn't have to process the whole queue when it gets sees that there is data on it. finally what if this happens: queue0, queue1 start empty threadA { flip queue selection bit == 1 put x,y,z on queue 1 } threadB{ queue1 has data, processes item x CONTEXT SWITCH [queue0 is empty, queue1 has y, z] threadA { flip queue selection bit == 1 put j,k on the queue } [queue0 is j,k, queue1 has y, z] threadA { flip queue selection bit == 1 put a, b on the queue CONTEXT SWITCH [queue0 is j,k, queue1 has y, z, a, b] threadB { well, so the queue it is pointing to has y,z,a,b but j,k are older than a,b.. shouldn't it process j,k first? } basically, without some mutex or some lock-free data structure, some test&set or something more you still cannot guarantee safety with this method [i don't think]. -Alex > > Rama > > Eric Newman wrote: > > > > >Why use a double-buffered scheme when you can just lock a mutex? You > >wouldn't have to lock it for very long; just long enough to extract > >the next event from the head of the queue; then you unlock it again; > >then you pass the event into a switch. > > > >So, > > > >pthread_mutex_lock(&eventQueue_mutex); > >Event nextEvent = eventQueue.front(); > >eventQueue.pop(); > >pthread_mutex_unlock(&eventQueue_mutex); > > > >switch (nextEvent.id) > >{ > > // handle the event > > // ... > >} > > > >You'd have to use a mutex lock in a double-buffered scheme anyway > >(when you swap the buffers). So really, a double buffer is > >redundant. Plus, events wouldn't be handled as quickly. > > > > > >On Fri, 17 Nov 2006, Graham Wakefield wrote: > > > >>And I forgot to mention- > >> > >>Another advantage of this approach is that all application logic > >>resides within a shared memory space and in a single thread, which is > >>a great advantage. Other threads are simply used for low level > >>services. If actions such as creating windows, starting an > >>oscillator etc. are all triggered from a single thread shared memory > >>space, anything can send an event to any other without worrying about > >>locking, critical sections and all that kind of hassle. Also > >>building interfaces (APIs, scripting language interfaces etc) to > >>application logic becomes a lot simpler. > >> > >>What remains to be solved is how to packet events into a standard > >>structure (i.e. of uniform, small size), and how to share arbitrarily > >>large data (blobs) between threads... some kind of automated > >>buffering system is called for I suppose for the latter, possibly > >>using a freelist/pool approach for real time memory management. > >> > >>oao > >> > >>On Nov 17, 2006, at 8:07 PM, Graham Wakefield wrote: > >> > >>>After chatting with Rama today about message queues and so on, we > >>>came up with a potential model that deals with threads. > >>> > >>>For example, consider an application that has a central thread > >>>running the main loop (running e.g. every 5ms), which has some kind > >>>of event manager. The event manager parses an input buffer/queue of > >>>event objects on each iteration. Application logic (built-in > >>>behaviour, user defined via C++ code, script code, function pointers > >>>etc) determines how to respond to each event. Some event may cause > >>>other events to be posted; these new events can be placed at the end > >>>of the queue (or at the beginning for high priority events) safely, > >>>since we are operating within a single thread. > >>> > >>>Now consider adding a network input facility. Since the network > >>>socket exists in a different thread, it will generate new events at > >>>interrupt time. Therefore we add a double-buffered event queue to > >>>the core application event manager specifically for the network > >>>thread. At a specified frequency (some lower division of the main > >>>application event loop rate), the event manager will read and > >>>process the network input, by swapping the input queues and then > >>>reading and processing each event, just as if they were core thread > >>>events. Thus different input subsystems can be polled at different > >>>rates. These rates could change at runtime. Also, the input > >>>buffers should be able to grow at runtime (using e.g. stl::vector or > >>>queue). > >>> > >>>Conversely, if we add audio, we might want to have events generate > >>>new notes in the audio thread. Again, we can use a double-buffered > >>>queue, but this time have the queue be placed in the audio thread. > >>>Any events generated in the core event loop that are specific audio > >>>events will be appended to the audio thread's input queue. When the > >>>audio thread wakes up (sound card callback) it likewise swaps the > >>>input queues and processes any new audio events. Any events that > >>>the audio thread generates however will use an input double-buffered > >>>queue for audio in the main event loop thread. > >>> > >>>Crucially, to prevent thread issues, it must be necessary that ANY > >>>other thread can only send events to the main event loop thread. > >>>So, if receiving a 'note 64' message over the network, the message > >>>is turned into an event in the network input queue in the core event > >>>loop thread. When the main event loop next looks at the network > >>>input, it will find the 'note 64' event and respond by sending a > >>>'make new tone' event to the audio thread's input queue. When the > >>>audio callback is next triggered, it should find this event and > >>>respond by making sound. > >>> > >>>This idea can be extended to model heavily threaded applications, > >>>and makes remote control of applications very simple to implement. > >>> > >>>Of course, for a subsystem that already resides in the main thread > >>>(e.g. graphics, mouse & keyboard) there is no need to use the > >>>double-buffer, they can append directly to the current queue since > >>>there is no danger of interrupted access. > >>> > >>>This idea can be made extensible so that the user could add new > >>>subsystems, event types, event handlers etc. > >>> > >>>On Nov 16, 2006, at 7:18 PM, Jorge Castellanos wrote: > >>> > >>>> > >>>>As I can't attend the meetings, I might be repeating things you > >>>>already talked about, but I hope you don't mind. > >>>>Here are some thoughts, in case you didn't discuss them already... > >>>> > >>>>The "event processor" (scheduler ?) would work fine, but keep in > >>>>mind that some events might need to run at different rates. OSC, > >>>>for example might need to be sent "immediately" while other events > >>>>wouldn't mind some latency. > >>>>In other words it might be a good idea for the "Event" class to > >>>>also have some sort of "rate" or "priority" (a priority queue (?). > >>>>Not sure. Something to consider. > >>>> > >>>>Re: threading... I'm pro double buffering, but considering that > >>>>computers are moving to multi-core configurations, and the number > >>>>of cores will keep growing, then threads are the way to go. This > >>>>way you use a different core. With a single threaded app you would > >>>>be wasting the other many cores. The only way to split the work is > >>>>by dividing the work into different threads. > >>>>That said, double buffering doesn't rule out threads. Meaning a > >>>>double buffered priority queue could run in its own thread. > >>>> > >>>>j > >>>> > >>>> > >>>>On Nov 16, 2006, at 9:58 PM, Graham Wakefield wrote: > >>>> > >>>>Hmm - threading. > >>>> > >>>>Should the event queue be double buffered to avoid threading > >>>>mishaps (posting an event while another is executing?) > >>>> > >>>>On Nov 16, 2006, at 5:39 PM, Graham Wakefield wrote: > >>>> > >>>>>More thoughts on the Event Manager, after a chat today with Eric: > >>>>> > >>>>>An underlying event engine provides a central event processor, > >>>>>which is based around an event queue. The event processor can be > >>>>>triggered at a desired rate, which may be the frame rate for example. > >>>>> > >>>>>Events raised by the operating system (mouse clicks, keyboard > >>>>>hits, application level events etc) and from other inputs (OSC > >>>>>receivers, MIDI input devices) are placed on the end of the event > >>>>>queue. > >>>>> > >>>>>On each iteration the event processor reads through the queue and > >>>>>processes each event in turn, dispatching it to the appropriate > >>>>>object by calling the object's generic 'handler' function with the > >>>>>event as the argument. > >>>>> > >>>>>For mouse events, the system will do hit detection and graphic > >>>>>tree traversal to find out the appropriate target (mouse events > >>>>>will be received from the OS at window-level). Keyboard events > >>>>>will be routed to the current TextFocus receiver (not necessarily > >>>>>a widget). Other classes of events will be routed to targets in > >>>>>other ways, as appropriate. Clearly there is a need to separate > >>>>>events by class as well as subtype (see list below). > >>>>> > >>>>>An object's handler will essentially contain some kind of switch > >>>>>based upon the event class and subtype. If the event is not > >>>>>handled, it should be passed back to the event manager using > >>>>>'return eventNotHandled' or 'return 1', which can in turn attempt > >>>>>to pass the event to the parent item (parent widget for Views, > >>>>>other kinds of parents for other objects). Perhaps all objects > >>>>>should have a eventParent field? > >>>>> > >>>>>Anywhere in the application, one could post new events into the > >>>>>queue using global functions: > >>>>> > >>>>>Event::post(event *e); // places at tail of queue. > >>>>>Event::immediate(event *e); // places at head of queue. > >>>>> > >>>>>Events themselves need to be generic enough to handle different > >>>>>types of data. A possible structure: > >>>>> > >>>>>Event { > >>>>> int32 class; // e.g. 'mous' > >>>>> int32 type; // e.g. 'down' > >>>>> double time; // event timestamp - lets us schedule > >>>>>events into the future > >>>>> int n; // num parameters > >>>>> Param ** params; // parameters > >>>>>} > >>>>> > >>>>>Param { > >>>>> int32 type; // e.g. 'long' > >>>>> size_t size; > >>>>> void * data; > >>>>>} > >>>>> > >>>>>Convenience methods could be written for standard event types to > >>>>>quickly extract the typical parameters. > >>>>> > >>>>>The advantage of this generalized system is that a user could also > >>>>>define their own event classes and types and parameters. > >>>>> > >>>>> > >>>>>Class type > >>>>>Mouse down > >>>>> up > >>>>> drag > >>>>> move > >>>>> wheel > >>>>>Keyboard > >>>>> down > >>>>> up > >>>>> repeat > >>>>> modifier (shift etc.) > >>>>>Tablet/Joystick > >>>>> button > >>>>> continuous > >>>>>Command/Application > >>>>> quit > >>>>> save > >>>>> copy > >>>>> paste > >>>>> ...etc > >>>>>OSC message > >>>>> bundle > >>>>>MIDI > >>>>> noteon > >>>>> noteoff > >>>>> controller > >>>>> sysex > >>>>> ...etc. > >>>>>...etc. > >>>>> > >>>>> > >>>>> > >>>>>_______________________________________________ > >>>>>glv mailing list > >>>>>glv@mat.ucsb.edu > >>>>>http://zydeco.mat.ucsb.edu/mailman/listinfo/glv > >>>> > >>>> > >>>>_______________________________________________ > >>>>glv mailing list > >>>>glv@mat.ucsb.edu > >>>>http://zydeco.mat.ucsb.edu/mailman/listinfo/glv > >>>> > >>>>_______________________________________________ > >>>>glv mailing list > >>>>glv@mat.ucsb.edu > >>>>http://zydeco.mat.ucsb.edu/mailman/listinfo/glv > >>> > >>> > >>>_______________________________________________ > >>>glv mailing list > >>>glv@mat.ucsb.edu > >>>http://zydeco.mat.ucsb.edu/mailman/listinfo/glv > >> > >> > >>_______________________________________________ > >>glv mailing list > >>glv@mat.ucsb.edu > >>http://zydeco.mat.ucsb.edu/mailman/listinfo/glv > > > >_______________________________________________ > >Tdg mailing list > >Tdg@mat.ucsb.edu > >http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg > > > _______________________________________________ > Tdg mailing list > Tdg@mat.ucsb.edu > http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg From ericnewman at umail.ucsb.edu Fri Nov 17 23:42:46 2006 From: ericnewman at umail.ucsb.edu (Eric Newman) Date: Fri Nov 17 23:43:06 2006 Subject: [Tdg] Re: [glv] Message Passing In-Reply-To: <20061118065429.GX21215@silverninja.net> References: <19CE924D-C0D8-46B2-9B11-AE8CFE66FA44@grahamwakefield.net> <8C410ED8-4E82-410B-8D7B-789B125191E3@grahamwakefield.net> <71EFD6FD-FF64-4EA2-AA5B-E43665820D0F@grahamwakefield.net> <15C6C961-EED3-4C42-98F6-06980E2A55AB@grahamwakefield.net> <588ABC59-9261-4772-91A0-D3C36DB1106E@umail.ucsb.edu> <6828F04A-3B18-4C7E-9AB9-3AB7209FB26D@grahamwakefield.net> <1CB7861C-1D3A-4761-971C-F67197F1B095@grahamwakefield.net> <455EA1D9.8080101@umail.ucsb.edu> <20061118065429.GX21215@silverninja.net> Message-ID: if i understood alex correctly, i agree with what he is saying. a double-buffer scheme might process out of order (or in the worst infinite case, never flip back to the other buffer). so it might cause more problems than it solves. like alex said, you'd never lose a message with a single-buffer mutex scenario. if the queue was locked, a thread trying to add a new event would just sleep (0% cpu) until the lock was opened again to allow queue pushing. so there would be a tiny delay. but, it would be imperceptible. there is no 'rejecting' with mutexes, only delays. also like alex said, not only does the event handler have to lock a mutex when it reads the front of the queue, but any incoming event has to be locked when it is added to the end of the queue. so you have a static Application member function that any other part of the program (i.e. any subsystem) can use, like this: static void Application::addEvent(Event e) { pthread_mutex_lock(&eventQueue_mutex); eventQueue->push(e); pthread_mutex_unlock(&eventQueue_mutex); } the mutex and the queue are also static members of Application. so, obviously, it's the same mutex as the main event handler loop uses. On Fri, 17 Nov 2006, Alex Norman wrote: > On 0, Rama Hoetzlein wrote: >> Good point. I think the final solution will depend on where the lock >> happens though. >> >> One problem with mutexes is that if you lock during event handling, then >> if another thread sends a message during lock it will have to be >> rejected since there is no double-buffering and the queue is locked. >> This could be a critical message, such as a keyboard press. > > I guess you're talking about some sort of "try_lock" function here? Otherwise > the thread that is writing on the queue really has no idea if it locked or > waited for the lock, unless you add some b.s. after every mutex_lock call. So > this doesn't result in a dropped message, it simply results in a [if you've > written it correctly], very tiny delay time. > >> >> Graham and I discussed the possibility of locking based on messages >> generation instead, which has the advantage that all messages are still >> passed into the queue (the generation of a message briefly locks that >> event handling sub-system). The disadvantage is that the event handler, >> which is a renderer or an audio subsystem, may skip frames or decide to >> drop events. However, this should be less noticable than missing >> critical events. So although it may be delayed a timing cycle, at least >> the sub-system still gets to decide what to do for every event generated. > > I don't quite understand what you mean by locking based on generation. To be > safe, using mutexes, you have to lock when you put stuff on the queue and when > you take stuff off, or, if you know you only have one reader and one writer, at > least when you update or read the indices. > >> >> I'm still not sure about the double-buffering or not. I know in >> rendering I need double-buffering because I need to distinguish draw >> calls with current timestamps versus calls made for a future time, but I >> don't know about other sub-systems. > > I am also unsure of double buffering. It seems that if you write your queue and > dequeueing functions well and set the priorities of threads intelligently then > using mutexes shouldn't really be a problem. It also makes it so that the > event handler doesn't have to process the whole queue when it gets sees that > there is data on it. > > finally what if this happens: > > queue0, queue1 start empty > > threadA { > flip queue selection bit == 1 > put x,y,z on queue 1 > } > > threadB{ > queue1 has data, processes item x > CONTEXT SWITCH > > [queue0 is empty, queue1 has y, z] > > threadA { > flip queue selection bit == 1 > put j,k on the queue > } > > [queue0 is j,k, queue1 has y, z] > > threadA { > flip queue selection bit == 1 > put a, b on the queue > CONTEXT SWITCH > > [queue0 is j,k, queue1 has y, z, a, b] > > threadB { > well, so the queue it is pointing to has y,z,a,b > but j,k are older than a,b.. shouldn't it process j,k first? > } > > basically, without some mutex or some lock-free data structure, some test&set or > something more you still cannot guarantee safety with this method [i don't > think]. > > -Alex > >> >> Rama >> >> Eric Newman wrote: >> >>> >>> Why use a double-buffered scheme when you can just lock a mutex? You >>> wouldn't have to lock it for very long; just long enough to extract >>> the next event from the head of the queue; then you unlock it again; >>> then you pass the event into a switch. >>> >>> So, >>> >>> pthread_mutex_lock(&eventQueue_mutex); >>> Event nextEvent = eventQueue.front(); >>> eventQueue.pop(); >>> pthread_mutex_unlock(&eventQueue_mutex); >>> >>> switch (nextEvent.id) >>> { >>> // handle the event >>> // ... >>> } >>> >>> You'd have to use a mutex lock in a double-buffered scheme anyway >>> (when you swap the buffers). So really, a double buffer is >>> redundant. Plus, events wouldn't be handled as quickly. >>> >>> >>> On Fri, 17 Nov 2006, Graham Wakefield wrote: >>> >>>> And I forgot to mention- >>>> >>>> Another advantage of this approach is that all application logic >>>> resides within a shared memory space and in a single thread, which is >>>> a great advantage. Other threads are simply used for low level >>>> services. If actions such as creating windows, starting an >>>> oscillator etc. are all triggered from a single thread shared memory >>>> space, anything can send an event to any other without worrying about >>>> locking, critical sections and all that kind of hassle. Also >>>> building interfaces (APIs, scripting language interfaces etc) to >>>> application logic becomes a lot simpler. >>>> >>>> What remains to be solved is how to packet events into a standard >>>> structure (i.e. of uniform, small size), and how to share arbitrarily >>>> large data (blobs) between threads... some kind of automated >>>> buffering system is called for I suppose for the latter, possibly >>>> using a freelist/pool approach for real time memory management. >>>> >>>> oao >>>> >>>> On Nov 17, 2006, at 8:07 PM, Graham Wakefield wrote: >>>> >>>>> After chatting with Rama today about message queues and so on, we >>>>> came up with a potential model that deals with threads. >>>>> >>>>> For example, consider an application that has a central thread >>>>> running the main loop (running e.g. every 5ms), which has some kind >>>>> of event manager. The event manager parses an input buffer/queue of >>>>> event objects on each iteration. Application logic (built-in >>>>> behaviour, user defined via C++ code, script code, function pointers >>>>> etc) determines how to respond to each event. Some event may cause >>>>> other events to be posted; these new events can be placed at the end >>>>> of the queue (or at the beginning for high priority events) safely, >>>>> since we are operating within a single thread. >>>>> >>>>> Now consider adding a network input facility. Since the network >>>>> socket exists in a different thread, it will generate new events at >>>>> interrupt time. Therefore we add a double-buffered event queue to >>>>> the core application event manager specifically for the network >>>>> thread. At a specified frequency (some lower division of the main >>>>> application event loop rate), the event manager will read and >>>>> process the network input, by swapping the input queues and then >>>>> reading and processing each event, just as if they were core thread >>>>> events. Thus different input subsystems can be polled at different >>>>> rates. These rates could change at runtime. Also, the input >>>>> buffers should be able to grow at runtime (using e.g. stl::vector or >>>>> queue). >>>>> >>>>> Conversely, if we add audio, we might want to have events generate >>>>> new notes in the audio thread. Again, we can use a double-buffered >>>>> queue, but this time have the queue be placed in the audio thread. >>>>> Any events generated in the core event loop that are specific audio >>>>> events will be appended to the audio thread's input queue. When the >>>>> audio thread wakes up (sound card callback) it likewise swaps the >>>>> input queues and processes any new audio events. Any events that >>>>> the audio thread generates however will use an input double-buffered >>>>> queue for audio in the main event loop thread. >>>>> >>>>> Crucially, to prevent thread issues, it must be necessary that ANY >>>>> other thread can only send events to the main event loop thread. >>>>> So, if receiving a 'note 64' message over the network, the message >>>>> is turned into an event in the network input queue in the core event >>>>> loop thread. When the main event loop next looks at the network >>>>> input, it will find the 'note 64' event and respond by sending a >>>>> 'make new tone' event to the audio thread's input queue. When the >>>>> audio callback is next triggered, it should find this event and >>>>> respond by making sound. >>>>> >>>>> This idea can be extended to model heavily threaded applications, >>>>> and makes remote control of applications very simple to implement. >>>>> >>>>> Of course, for a subsystem that already resides in the main thread >>>>> (e.g. graphics, mouse & keyboard) there is no need to use the >>>>> double-buffer, they can append directly to the current queue since >>>>> there is no danger of interrupted access. >>>>> >>>>> This idea can be made extensible so that the user could add new >>>>> subsystems, event types, event handlers etc. >>>>> >>>>> On Nov 16, 2006, at 7:18 PM, Jorge Castellanos wrote: >>>>> >>>>>> >>>>>> As I can't attend the meetings, I might be repeating things you >>>>>> already talked about, but I hope you don't mind. >>>>>> Here are some thoughts, in case you didn't discuss them already... >>>>>> >>>>>> The "event processor" (scheduler ?) would work fine, but keep in >>>>>> mind that some events might need to run at different rates. OSC, >>>>>> for example might need to be sent "immediately" while other events >>>>>> wouldn't mind some latency. >>>>>> In other words it might be a good idea for the "Event" class to >>>>>> also have some sort of "rate" or "priority" (a priority queue (?). >>>>>> Not sure. Something to consider. >>>>>> >>>>>> Re: threading... I'm pro double buffering, but considering that >>>>>> computers are moving to multi-core configurations, and the number >>>>>> of cores will keep growing, then threads are the way to go. This >>>>>> way you use a different core. With a single threaded app you would >>>>>> be wasting the other many cores. The only way to split the work is >>>>>> by dividing the work into different threads. >>>>>> That said, double buffering doesn't rule out threads. Meaning a >>>>>> double buffered priority queue could run in its own thread. >>>>>> >>>>>> j >>>>>> >>>>>> >>>>>> On Nov 16, 2006, at 9:58 PM, Graham Wakefield wrote: >>>>>> >>>>>> Hmm - threading. >>>>>> >>>>>> Should the event queue be double buffered to avoid threading >>>>>> mishaps (posting an event while another is executing?) >>>>>> >>>>>> On Nov 16, 2006, at 5:39 PM, Graham Wakefield wrote: >>>>>> >>>>>>> More thoughts on the Event Manager, after a chat today with Eric: >>>>>>> >>>>>>> An underlying event engine provides a central event processor, >>>>>>> which is based around an event queue. The event processor can be >>>>>>> triggered at a desired rate, which may be the frame rate for example. >>>>>>> >>>>>>> Events raised by the operating system (mouse clicks, keyboard >>>>>>> hits, application level events etc) and from other inputs (OSC >>>>>>> receivers, MIDI input devices) are placed on the end of the event >>>>>>> queue. >>>>>>> >>>>>>> On each iteration the event processor reads through the queue and >>>>>>> processes each event in turn, dispatching it to the appropriate >>>>>>> object by calling the object's generic 'handler' function with the >>>>>>> event as the argument. >>>>>>> >>>>>>> For mouse events, the system will do hit detection and graphic >>>>>>> tree traversal to find out the appropriate target (mouse events >>>>>>> will be received from the OS at window-level). Keyboard events >>>>>>> will be routed to the current TextFocus receiver (not necessarily >>>>>>> a widget). Other classes of events will be routed to targets in >>>>>>> other ways, as appropriate. Clearly there is a need to separate >>>>>>> events by class as well as subtype (see list below). >>>>>>> >>>>>>> An object's handler will essentially contain some kind of switch >>>>>>> based upon the event class and subtype. If the event is not >>>>>>> handled, it should be passed back to the event manager using >>>>>>> 'return eventNotHandled' or 'return 1', which can in turn attempt >>>>>>> to pass the event to the parent item (parent widget for Views, >>>>>>> other kinds of parents for other objects). Perhaps all objects >>>>>>> should have a eventParent field? >>>>>>> >>>>>>> Anywhere in the application, one could post new events into the >>>>>>> queue using global functions: >>>>>>> >>>>>>> Event::post(event *e); // places at tail of queue. >>>>>>> Event::immediate(event *e); // places at head of queue. >>>>>>> >>>>>>> Events themselves need to be generic enough to handle different >>>>>>> types of data. A possible structure: >>>>>>> >>>>>>> Event { >>>>>>> int32 class; // e.g. 'mous' >>>>>>> int32 type; // e.g. 'down' >>>>>>> double time; // event timestamp - lets us schedule >>>>>>> events into the future >>>>>>> int n; // num parameters >>>>>>> Param ** params; // parameters >>>>>>> } >>>>>>> >>>>>>> Param { >>>>>>> int32 type; // e.g. 'long' >>>>>>> size_t size; >>>>>>> void * data; >>>>>>> } >>>>>>> >>>>>>> Convenience methods could be written for standard event types to >>>>>>> quickly extract the typical parameters. >>>>>>> >>>>>>> The advantage of this generalized system is that a user could also >>>>>>> define their own event classes and types and parameters. >>>>>>> >>>>>>> >>>>>>> Class type >>>>>>> Mouse down >>>>>>> up >>>>>>> drag >>>>>>> move >>>>>>> wheel >>>>>>> Keyboard >>>>>>> down >>>>>>> up >>>>>>> repeat >>>>>>> modifier (shift etc.) >>>>>>> Tablet/Joystick >>>>>>> button >>>>>>> continuous >>>>>>> Command/Application >>>>>>> quit >>>>>>> save >>>>>>> copy >>>>>>> paste >>>>>>> ...etc >>>>>>> OSC message >>>>>>> bundle >>>>>>> MIDI >>>>>>> noteon >>>>>>> noteoff >>>>>>> controller >>>>>>> sysex >>>>>>> ...etc. >>>>>>> ...etc. >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> glv mailing list >>>>>>> glv@mat.ucsb.edu >>>>>>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> glv mailing list >>>>>> glv@mat.ucsb.edu >>>>>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >>>>>> >>>>>> _______________________________________________ >>>>>> glv mailing list >>>>>> glv@mat.ucsb.edu >>>>>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >>>>> >>>>> >>>>> _______________________________________________ >>>>> glv mailing list >>>>> glv@mat.ucsb.edu >>>>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >>>> >>>> >>>> _______________________________________________ >>>> glv mailing list >>>> glv@mat.ucsb.edu >>>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >>> >>> _______________________________________________ >>> Tdg mailing list >>> Tdg@mat.ucsb.edu >>> http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg >> >> >> _______________________________________________ >> Tdg mailing list >> Tdg@mat.ucsb.edu >> http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg > _______________________________________________ > Tdg mailing list > Tdg@mat.ucsb.edu > http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg > From wakefield at mat.ucsb.edu Sat Nov 18 01:16:22 2006 From: wakefield at mat.ucsb.edu (Graham Wakefield) Date: Sat Nov 18 01:16:32 2006 Subject: [Tdg] Re: [glv] Message Passing In-Reply-To: <20061118065429.GX21215@silverninja.net> References: <19CE924D-C0D8-46B2-9B11-AE8CFE66FA44@grahamwakefield.net> <8C410ED8-4E82-410B-8D7B-789B125191E3@grahamwakefield.net> <71EFD6FD-FF64-4EA2-AA5B-E43665820D0F@grahamwakefield.net> <15C6C961-EED3-4C42-98F6-06980E2A55AB@grahamwakefield.net> <588ABC59-9261-4772-91A0-D3C36DB1106E@umail.ucsb.edu> <6828F04A-3B18-4C7E-9AB9-3AB7209FB26D@grahamwakefield.net> <1CB7861C-1D3A-4761-971C-F67197F1B095@grahamwakefield.net> <455EA1D9.8080101@umail.ucsb.edu> <20061118065429.GX21215@silverninja.net> Message-ID: The audio thread is almost universally a higher priority. On Nov 17, 2006, at 10:54 PM, Alex Norman wrote: > > I am also unsure of double buffering. It seems that if you write > your queue and > dequeueing functions well and set the priorities of threads > intelligently then > using mutexes shouldn't really be a problem. It also makes it so > that the > event handler doesn't have to process the whole queue when it gets > sees that > there is data on it. > > -Alex From wakefield at mat.ucsb.edu Sat Nov 18 01:17:36 2006 From: wakefield at mat.ucsb.edu (Graham Wakefield) Date: Sat Nov 18 01:17:43 2006 Subject: [Tdg] Re: [glv] Message Passing In-Reply-To: References: <1079b050611140017u7f6e066eh2233a09b806ac994@mail.gmail.com> <1079b050611142211w6cd12f2u8f57e7a479e2482@mail.gmail.com> <19CE924D-C0D8-46B2-9B11-AE8CFE66FA44@grahamwakefield.net> <8C410ED8-4E82-410B-8D7B-789B125191E3@grahamwakefield.net> <71EFD6FD-FF64-4EA2-AA5B-E43665820D0F@grahamwakefield.net> <15C6C961-EED3-4C42-98F6-06980E2A55AB@grahamwakefield.net> <588ABC59-9261-4772-91A0-D3C36DB1106E@umail.ucsb.edu> <6828F04A-3B18-4C7E-9AB9-3AB7209FB26D@grahamwakefield.net> <1CB7861C-1D3A-4761-971C-F67197F1B095@grahamwakefield.net> Message-ID: <9272C082-1400-40F6-BBAC-FEEE3237AF6C@mat.ucsb.edu> Because you only need to lock it for the duration of the buffer swap, which is exchanging 2 pointers. On Nov 17, 2006, at 9:09 PM, Eric Newman wrote: > > Why use a double-buffered scheme when you can just lock a mutex? > You wouldn't have to lock it for very long; just long enough to > extract the next event from the head of the queue; then you unlock > it again; then you pass the event into a switch. From wakefield at mat.ucsb.edu Sat Nov 18 01:19:50 2006 From: wakefield at mat.ucsb.edu (Graham Wakefield) Date: Sat Nov 18 01:19:58 2006 Subject: [Tdg] Re: [glv] Message Passing In-Reply-To: <455EA1D9.8080101@umail.ucsb.edu> References: <1079b050611140017u7f6e066eh2233a09b806ac994@mail.gmail.com> <1079b050611142211w6cd12f2u8f57e7a479e2482@mail.gmail.com> <19CE924D-C0D8-46B2-9B11-AE8CFE66FA44@grahamwakefield.net> <8C410ED8-4E82-410B-8D7B-789B125191E3@grahamwakefield.net> <71EFD6FD-FF64-4EA2-AA5B-E43665820D0F@grahamwakefield.net> <15C6C961-EED3-4C42-98F6-06980E2A55AB@grahamwakefield.net> <588ABC59-9261-4772-91A0-D3C36DB1106E@umail.ucsb.edu> <6828F04A-3B18-4C7E-9AB9-3AB7209FB26D@grahamwakefield.net> <1CB7861C-1D3A-4761-971C-F67197F1B095@grahamwakefield.net> <455EA1D9.8080101@umail.ucsb.edu> Message-ID: <30E59C2E-47D8-4E84-8A43-DBEB14CD12A9@mat.ucsb.edu> On Nov 17, 2006, at 10:02 PM, Rama Hoetzlein wrote: > Good point. I think the final solution will depend on where the > lock happens though. > > One problem with mutexes is that if you lock during event handling, > then if another thread sends a message during lock it will have to > be rejected since there is no double-buffering and the queue is > locked. This could be a critical message, such as a keyboard press. Or for an audio thread, it can't wait at all. A double-buffered queue shouldn't create much delay, since it is the reader that will swap the buffers. From wakefield at mat.ucsb.edu Sat Nov 18 01:25:49 2006 From: wakefield at mat.ucsb.edu (Graham Wakefield) Date: Sat Nov 18 01:25:56 2006 Subject: [Tdg] Re: [glv] Message Passing In-Reply-To: <20061118065429.GX21215@silverninja.net> References: <19CE924D-C0D8-46B2-9B11-AE8CFE66FA44@grahamwakefield.net> <8C410ED8-4E82-410B-8D7B-789B125191E3@grahamwakefield.net> <71EFD6FD-FF64-4EA2-AA5B-E43665820D0F@grahamwakefield.net> <15C6C961-EED3-4C42-98F6-06980E2A55AB@grahamwakefield.net> <588ABC59-9261-4772-91A0-D3C36DB1106E@umail.ucsb.edu> <6828F04A-3B18-4C7E-9AB9-3AB7209FB26D@grahamwakefield.net> <1CB7861C-1D3A-4761-971C-F67197F1B095@grahamwakefield.net> <455EA1D9.8080101@umail.ucsb.edu> <20061118065429.GX21215@silverninja.net> Message-ID: <55B2D5B3-2C48-43C3-AFD7-28738AB3B839@mat.ucsb.edu> > > I don't quite understand what you mean by locking based on > generation. To be > safe, using mutexes, you have to lock when you put stuff on the > queue and when > you take stuff off, or, if you know you only have one reader and > one writer, at > least when you update or read the indices. > What I was suggesting is a single reader, single writer scenario. The writer doesn't need to check any locks, since it always has a buffer to write to. It will want to lock while it writes. The reader will swap buffers (swap 2 pointers or set a bit) IF the buffer isn't locked. Otherwise, it will try again on the next event loop. It might not be the ideal method, but it seems pretty safe and minimizes locking/copying I think. >> >> I'm still not sure about the double-buffering or not. I know in >> rendering I need double-buffering because I need to distinguish draw >> calls with current timestamps versus calls made for a future time, >> but I >> don't know about other sub-systems. > > I am also unsure of double buffering. It seems that if you write > your queue and > dequeueing functions well and set the priorities of threads > intelligently then > using mutexes shouldn't really be a problem. It also makes it so > that the > event handler doesn't have to process the whole queue when it gets > sees that > there is data on it. Why wouldn't it want to process the whole queue? The whole queue is everything that should be happening right now. > > finally what if this happens: > > queue0, queue1 start empty > > threadA { > flip queue selection bit == 1 > put x,y,z on queue 1 > } > > threadB{ > queue1 has data, processes item x > CONTEXT SWITCH > > [queue0 is empty, queue1 has y, z] > > threadA { > flip queue selection bit == 1 > put j,k on the queue > } > > [queue0 is j,k, queue1 has y, z] > > threadA { > flip queue selection bit == 1 > put a, b on the queue > CONTEXT SWITCH > > [queue0 is j,k, queue1 has y, z, a, b] > > threadB { > well, so the queue it is pointing to has y,z,a,b > but j,k are older than a,b.. shouldn't it process j,k first? > } > The flip queue selection bit should be triggered by the event reader, not the event writer. From wakefield at mat.ucsb.edu Sat Nov 18 01:28:44 2006 From: wakefield at mat.ucsb.edu (Graham Wakefield) Date: Sat Nov 18 01:28:53 2006 Subject: [Tdg] Re: [glv] Message Passing In-Reply-To: References: <19CE924D-C0D8-46B2-9B11-AE8CFE66FA44@grahamwakefield.net> <8C410ED8-4E82-410B-8D7B-789B125191E3@grahamwakefield.net> <71EFD6FD-FF64-4EA2-AA5B-E43665820D0F@grahamwakefield.net> <15C6C961-EED3-4C42-98F6-06980E2A55AB@grahamwakefield.net> <588ABC59-9261-4772-91A0-D3C36DB1106E@umail.ucsb.edu> <6828F04A-3B18-4C7E-9AB9-3AB7209FB26D@grahamwakefield.net> <1CB7861C-1D3A-4761-971C-F67197F1B095@grahamwakefield.net> <455EA1D9.8080101@umail.ucsb.edu> <20061118065429.GX21215@silverninja.net> Message-ID: <061DEBD2-C967-4C9D-B732-D830FFB71CD4@mat.ucsb.edu> What I'm suggesting wouldn't reject any events either, nor would they be processed out of order, nor is there any sleep call required in the thread that posts an event. The only potential delay would be if the event manager wakes up and finds the queue locked, so it can't swap the buffer. In that case, it would try again ASAP. Putting a sleep requirement into an event triggered by an audio callback seems like a bad idea to me. Though this is based on hearsay, I have no evidence to support it! On Nov 17, 2006, at 11:42 PM, Eric Newman wrote: > > if i understood alex correctly, i agree with what he is saying. a > double-buffer scheme might process out of order (or in the worst > infinite > case, never flip back to the other buffer). so it might cause more > problems than it solves. > > like alex said, you'd never lose a message with a single-buffer mutex > scenario. if the queue was locked, a thread trying to add a new event > would just sleep (0% cpu) until the lock was opened again to allow > queue > pushing. so there would be a tiny delay. but, it would be > imperceptible. > there is no 'rejecting' with mutexes, only delays. > > also like alex said, not only does the event handler have to lock a > mutex when it reads the front of the queue, but any incoming event > has to be locked when it is added to the end of the queue. so you > have a static Application member function that any other part of > the program (i.e. any subsystem) can use, like this: > > static void Application::addEvent(Event e) > { > pthread_mutex_lock(&eventQueue_mutex); > eventQueue->push(e); > pthread_mutex_unlock(&eventQueue_mutex); > } > > the mutex and the queue are also static members of Application. so, > obviously, it's the same mutex as the main event handler loop uses. > > On Fri, 17 Nov 2006, Alex Norman wrote: > >> On 0, Rama Hoetzlein wrote: >>> Good point. I think the final solution will depend on where the lock >>> happens though. >>> >>> One problem with mutexes is that if you lock during event >>> handling, then >>> if another thread sends a message during lock it will have to be >>> rejected since there is no double-buffering and the queue is locked. >>> This could be a critical message, such as a keyboard press. >> >> I guess you're talking about some sort of "try_lock" function >> here? Otherwise >> the thread that is writing on the queue really has no idea if it >> locked or >> waited for the lock, unless you add some b.s. after every >> mutex_lock call. So >> this doesn't result in a dropped message, it simply results in a >> [if you've >> written it correctly], very tiny delay time. >> >>> >>> Graham and I discussed the possibility of locking based on messages >>> generation instead, which has the advantage that all messages are >>> still >>> passed into the queue (the generation of a message briefly locks >>> that >>> event handling sub-system). The disadvantage is that the event >>> handler, >>> which is a renderer or an audio subsystem, may skip frames or >>> decide to >>> drop events. However, this should be less noticable than missing >>> critical events. So although it may be delayed a timing cycle, at >>> least >>> the sub-system still gets to decide what to do for every event >>> generated. >> >> I don't quite understand what you mean by locking based on >> generation. To be >> safe, using mutexes, you have to lock when you put stuff on the >> queue and when >> you take stuff off, or, if you know you only have one reader and >> one writer, at >> least when you update or read the indices. >> >>> >>> I'm still not sure about the double-buffering or not. I know in >>> rendering I need double-buffering because I need to distinguish draw >>> calls with current timestamps versus calls made for a future >>> time, but I >>> don't know about other sub-systems. >> >> I am also unsure of double buffering. It seems that if you write >> your queue and >> dequeueing functions well and set the priorities of threads >> intelligently then >> using mutexes shouldn't really be a problem. It also makes it so >> that the >> event handler doesn't have to process the whole queue when it gets >> sees that >> there is data on it. >> >> finally what if this happens: >> >> queue0, queue1 start empty >> >> threadA { >> flip queue selection bit == 1 >> put x,y,z on queue 1 >> } >> >> threadB{ >> queue1 has data, processes item x >> CONTEXT SWITCH >> >> [queue0 is empty, queue1 has y, z] >> >> threadA { >> flip queue selection bit == 1 >> put j,k on the queue >> } >> >> [queue0 is j,k, queue1 has y, z] >> >> threadA { >> flip queue selection bit == 1 >> put a, b on the queue >> CONTEXT SWITCH >> >> [queue0 is j,k, queue1 has y, z, a, b] >> >> threadB { >> well, so the queue it is pointing to has y,z,a,b >> but j,k are older than a,b.. shouldn't it process j,k first? >> } >> >> basically, without some mutex or some lock-free data structure, >> some test&set or >> something more you still cannot guarantee safety with this method >> [i don't >> think]. >> >> -Alex >> >>> >>> Rama >>> >>> Eric Newman wrote: >>> >>>> >>>> Why use a double-buffered scheme when you can just lock a >>>> mutex? You >>>> wouldn't have to lock it for very long; just long enough to extract >>>> the next event from the head of the queue; then you unlock it >>>> again; >>>> then you pass the event into a switch. >>>> >>>> So, >>>> >>>> pthread_mutex_lock(&eventQueue_mutex); >>>> Event nextEvent = eventQueue.front(); >>>> eventQueue.pop(); >>>> pthread_mutex_unlock(&eventQueue_mutex); >>>> >>>> switch (nextEvent.id) >>>> { >>>> // handle the event >>>> // ... >>>> } >>>> >>>> You'd have to use a mutex lock in a double-buffered scheme anyway >>>> (when you swap the buffers). So really, a double buffer is >>>> redundant. Plus, events wouldn't be handled as quickly. >>>> >>>> >>>> On Fri, 17 Nov 2006, Graham Wakefield wrote: >>>> >>>>> And I forgot to mention- >>>>> >>>>> Another advantage of this approach is that all application logic >>>>> resides within a shared memory space and in a single thread, >>>>> which is >>>>> a great advantage. Other threads are simply used for low level >>>>> services. If actions such as creating windows, starting an >>>>> oscillator etc. are all triggered from a single thread shared >>>>> memory >>>>> space, anything can send an event to any other without worrying >>>>> about >>>>> locking, critical sections and all that kind of hassle. Also >>>>> building interfaces (APIs, scripting language interfaces etc) to >>>>> application logic becomes a lot simpler. >>>>> >>>>> What remains to be solved is how to packet events into a standard >>>>> structure (i.e. of uniform, small size), and how to share >>>>> arbitrarily >>>>> large data (blobs) between threads... some kind of automated >>>>> buffering system is called for I suppose for the latter, possibly >>>>> using a freelist/pool approach for real time memory management. >>>>> >>>>> oao >>>>> >>>>> On Nov 17, 2006, at 8:07 PM, Graham Wakefield wrote: >>>>> >>>>>> After chatting with Rama today about message queues and so on, we >>>>>> came up with a potential model that deals with threads. >>>>>> >>>>>> For example, consider an application that has a central thread >>>>>> running the main loop (running e.g. every 5ms), which has some >>>>>> kind >>>>>> of event manager. The event manager parses an input buffer/ >>>>>> queue of >>>>>> event objects on each iteration. Application logic (built-in >>>>>> behaviour, user defined via C++ code, script code, function >>>>>> pointers >>>>>> etc) determines how to respond to each event. Some event may >>>>>> cause >>>>>> other events to be posted; these new events can be placed at >>>>>> the end >>>>>> of the queue (or at the beginning for high priority events) >>>>>> safely, >>>>>> since we are operating within a single thread. >>>>>> >>>>>> Now consider adding a network input facility. Since the network >>>>>> socket exists in a different thread, it will generate new >>>>>> events at >>>>>> interrupt time. Therefore we add a double-buffered event >>>>>> queue to >>>>>> the core application event manager specifically for the network >>>>>> thread. At a specified frequency (some lower division of the >>>>>> main >>>>>> application event loop rate), the event manager will read and >>>>>> process the network input, by swapping the input queues and then >>>>>> reading and processing each event, just as if they were core >>>>>> thread >>>>>> events. Thus different input subsystems can be polled at >>>>>> different >>>>>> rates. These rates could change at runtime. Also, the input >>>>>> buffers should be able to grow at runtime (using e.g. >>>>>> stl::vector or >>>>>> queue). >>>>>> >>>>>> Conversely, if we add audio, we might want to have events >>>>>> generate >>>>>> new notes in the audio thread. Again, we can use a double- >>>>>> buffered >>>>>> queue, but this time have the queue be placed in the audio >>>>>> thread. >>>>>> Any events generated in the core event loop that are specific >>>>>> audio >>>>>> events will be appended to the audio thread's input queue. >>>>>> When the >>>>>> audio thread wakes up (sound card callback) it likewise swaps the >>>>>> input queues and processes any new audio events. Any events that >>>>>> the audio thread generates however will use an input double- >>>>>> buffered >>>>>> queue for audio in the main event loop thread. >>>>>> >>>>>> Crucially, to prevent thread issues, it must be necessary that >>>>>> ANY >>>>>> other thread can only send events to the main event loop thread. >>>>>> So, if receiving a 'note 64' message over the network, the >>>>>> message >>>>>> is turned into an event in the network input queue in the core >>>>>> event >>>>>> loop thread. When the main event loop next looks at the network >>>>>> input, it will find the 'note 64' event and respond by sending a >>>>>> 'make new tone' event to the audio thread's input queue. When >>>>>> the >>>>>> audio callback is next triggered, it should find this event and >>>>>> respond by making sound. >>>>>> >>>>>> This idea can be extended to model heavily threaded applications, >>>>>> and makes remote control of applications very simple to >>>>>> implement. >>>>>> >>>>>> Of course, for a subsystem that already resides in the main >>>>>> thread >>>>>> (e.g. graphics, mouse & keyboard) there is no need to use the >>>>>> double-buffer, they can append directly to the current queue >>>>>> since >>>>>> there is no danger of interrupted access. >>>>>> >>>>>> This idea can be made extensible so that the user could add new >>>>>> subsystems, event types, event handlers etc. >>>>>> >>>>>> On Nov 16, 2006, at 7:18 PM, Jorge Castellanos wrote: >>>>>> >>>>>>> >>>>>>> As I can't attend the meetings, I might be repeating things you >>>>>>> already talked about, but I hope you don't mind. >>>>>>> Here are some thoughts, in case you didn't discuss them >>>>>>> already... >>>>>>> >>>>>>> The "event processor" (scheduler ?) would work fine, but keep in >>>>>>> mind that some events might need to run at different rates. OSC, >>>>>>> for example might need to be sent "immediately" while other >>>>>>> events >>>>>>> wouldn't mind some latency. >>>>>>> In other words it might be a good idea for the "Event" class to >>>>>>> also have some sort of "rate" or "priority" (a priority queue >>>>>>> (?). >>>>>>> Not sure. Something to consider. >>>>>>> >>>>>>> Re: threading... I'm pro double buffering, but considering that >>>>>>> computers are moving to multi-core configurations, and the >>>>>>> number >>>>>>> of cores will keep growing, then threads are the way to go. This >>>>>>> way you use a different core. With a single threaded app you >>>>>>> would >>>>>>> be wasting the other many cores. The only way to split the >>>>>>> work is >>>>>>> by dividing the work into different threads. >>>>>>> That said, double buffering doesn't rule out threads. Meaning a >>>>>>> double buffered priority queue could run in its own thread. >>>>>>> >>>>>>> j >>>>>>> >>>>>>> >>>>>>> On Nov 16, 2006, at 9:58 PM, Graham Wakefield wrote: >>>>>>> >>>>>>> Hmm - threading. >>>>>>> >>>>>>> Should the event queue be double buffered to avoid threading >>>>>>> mishaps (posting an event while another is executing?) >>>>>>> >>>>>>> On Nov 16, 2006, at 5:39 PM, Graham Wakefield wrote: >>>>>>> >>>>>>>> More thoughts on the Event Manager, after a chat today with >>>>>>>> Eric: >>>>>>>> >>>>>>>> An underlying event engine provides a central event processor, >>>>>>>> which is based around an event queue. The event processor >>>>>>>> can be >>>>>>>> triggered at a desired rate, which may be the frame rate for >>>>>>>> example. >>>>>>>> >>>>>>>> Events raised by the operating system (mouse clicks, keyboard >>>>>>>> hits, application level events etc) and from other inputs (OSC >>>>>>>> receivers, MIDI input devices) are placed on the end of the >>>>>>>> event >>>>>>>> queue. >>>>>>>> >>>>>>>> On each iteration the event processor reads through the >>>>>>>> queue and >>>>>>>> processes each event in turn, dispatching it to the appropriate >>>>>>>> object by calling the object's generic 'handler' function >>>>>>>> with the >>>>>>>> event as the argument. >>>>>>>> >>>>>>>> For mouse events, the system will do hit detection and graphic >>>>>>>> tree traversal to find out the appropriate target (mouse events >>>>>>>> will be received from the OS at window-level). Keyboard events >>>>>>>> will be routed to the current TextFocus receiver (not >>>>>>>> necessarily >>>>>>>> a widget). Other classes of events will be routed to >>>>>>>> targets in >>>>>>>> other ways, as appropriate. Clearly there is a need to separate >>>>>>>> events by class as well as subtype (see list below). >>>>>>>> >>>>>>>> An object's handler will essentially contain some kind of >>>>>>>> switch >>>>>>>> based upon the event class and subtype. If the event is not >>>>>>>> handled, it should be passed back to the event manager using >>>>>>>> 'return eventNotHandled' or 'return 1', which can in turn >>>>>>>> attempt >>>>>>>> to pass the event to the parent item (parent widget for Views, >>>>>>>> other kinds of parents for other objects). Perhaps all objects >>>>>>>> should have a eventParent field? >>>>>>>> >>>>>>>> Anywhere in the application, one could post new events into the >>>>>>>> queue using global functions: >>>>>>>> >>>>>>>> Event::post(event *e); // places at tail of queue. >>>>>>>> Event::immediate(event *e); // places at head of queue. >>>>>>>> >>>>>>>> Events themselves need to be generic enough to handle different >>>>>>>> types of data. A possible structure: >>>>>>>> >>>>>>>> Event { >>>>>>>> int32 class; // e.g. 'mous' >>>>>>>> int32 type; // e.g. 'down' >>>>>>>> double time; // event timestamp - lets us schedule >>>>>>>> events into the future >>>>>>>> int n; // num parameters >>>>>>>> Param ** params; // parameters >>>>>>>> } >>>>>>>> >>>>>>>> Param { >>>>>>>> int32 type; // e.g. 'long' >>>>>>>> size_t size; >>>>>>>> void * data; >>>>>>>> } >>>>>>>> >>>>>>>> Convenience methods could be written for standard event >>>>>>>> types to >>>>>>>> quickly extract the typical parameters. >>>>>>>> >>>>>>>> The advantage of this generalized system is that a user >>>>>>>> could also >>>>>>>> define their own event classes and types and parameters. >>>>>>>> >>>>>>>> >>>>>>>> Class type >>>>>>>> Mouse down >>>>>>>> up >>>>>>>> drag >>>>>>>> move >>>>>>>> wheel >>>>>>>> Keyboard >>>>>>>> down >>>>>>>> up >>>>>>>> repeat >>>>>>>> modifier (shift etc.) >>>>>>>> Tablet/Joystick >>>>>>>> button >>>>>>>> continuous >>>>>>>> Command/Application >>>>>>>> quit >>>>>>>> save >>>>>>>> copy >>>>>>>> paste >>>>>>>> ...etc >>>>>>>> OSC message >>>>>>>> bundle >>>>>>>> MIDI >>>>>>>> noteon >>>>>>>> noteoff >>>>>>>> controller >>>>>>>> sysex >>>>>>>> ...etc. >>>>>>>> ...etc. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> glv mailing list >>>>>>>> glv@mat.ucsb.edu >>>>>>>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> glv mailing list >>>>>>> glv@mat.ucsb.edu >>>>>>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >>>>>>> >>>>>>> _______________________________________________ >>>>>>> glv mailing list >>>>>>> glv@mat.ucsb.edu >>>>>>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> glv mailing list >>>>>> glv@mat.ucsb.edu >>>>>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >>>>> >>>>> >>>>> _______________________________________________ >>>>> glv mailing list >>>>> glv@mat.ucsb.edu >>>>> http://zydeco.mat.ucsb.edu/mailman/listinfo/glv >>>> >>>> _______________________________________________ >>>> Tdg mailing list >>>> Tdg@mat.ucsb.edu >>>> http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg >>> >>> >>> _______________________________________________ >>> Tdg mailing list >>> Tdg@mat.ucsb.edu >>> http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg >> _______________________________________________ >> Tdg mailing list >> Tdg@mat.ucsb.edu >> http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg >> > _______________________________________________ > Tdg mailing list > Tdg@mat.ucsb.edu > http://zydeco.mat.ucsb.edu/mailman/listinfo/tdg From wesley.hoke at gmail.com Sat Nov 18 01:41:33 2006 From: wesley.hoke at gmail.com (Wesley Smith) Date: Sat Nov 18 01:41:41 2006 Subject: [Tdg] Re: [glv] Message Passing In-Reply-To: <061DEBD2-C967-4C9D-B732-D830FFB71CD4@mat.ucsb.edu> References: <19CE924D-C0D8-46B2-9B11-AE8CFE66FA44@grahamwakefield.net> <15C6C961-EED3-4C42-98F6-06980E2A55AB@grahamwakefield.net> <588ABC59-9261-4772-91A0-D3C36DB1106E@umail.ucsb.edu> <6828F04A-3B18-4C7E-9AB9-3AB7209FB26D@grahamwakefield.net> <1CB7861C-1D3A-4761-971C-F67197F1B095@grahamwakefield.net> <455EA1D9.8080101@umail.ucsb.edu> <20061118065429.GX21215@silverninja.net> <061DEBD2-C967-4C9D-B732-D830FFB71CD4@mat.ucsb.edu> Message-ID: <1079b050611180141k5625aa12qc9a8dba30aae6ef6@mail.gmail.com> Something that may be releveant to this is a PhD thesis Xavier gave to me today. I took photos of it and am uploading a PDF right now. It's called "Object-Oriented Time Synchronization of Audio-Visual Data in a Multimedia Application Framework" by Philipp Ackermann from ETH Zurich. Particularly Chapter 4 on the Time Synchronization Framework. He goes into detail on his C++ structures for dealing with this issue. It's p79 of the thesis and p. 47 of the PDF. It's all JPEG images, so the doc is 200 MB. Here's the link: http://www.mat.ucsb.edu/~whsmith/Docs/ObjectOriented_Time_Synchronization.pdf It will be uploaded in about an hour or so. wes From rch at umail.ucsb.edu Sat Nov 18 08:08:26 2006 From: rch at umail.ucsb.edu (Rama Hoetzlein) Date: Sat Nov 18 08:08:38 2006 Subject: [Tdg] Re: [glv] Message Passing In-Reply-To: <20061118065429.GX21215@silverninja.net> References: <19CE924D-C0D8-46B2-9B11-AE8CFE66FA44@grahamwakefield.net> <8C410ED8-4E82-410B-8D7B-789B125191E3@grahamwakefield.net> <71EFD6FD-FF64-4EA2-AA5B-E43665820D0F@grahamwakefield.net> <15C6C961-EED3-4C42-98F6-06980E2A55AB@grahamwakefield.net> <588ABC59-9261-4772-91A0-D3C36DB1106E@umail.ucsb.edu> <6828F04A-3B18-4C7E-9AB9-3AB7209FB26D@grahamwakefield.net> <1CB7861C-1D3A-4761-971C-F67197F1B095@grahamwakefield.net> <455EA1D9.8080101@umail.ucsb.edu> <20061118065429.GX21215@silverninja.net> Message-ID: <455F2FFA.8010606@umail.ucsb.edu> >I guess you're talking about some sort of "try_lock" function here? Otherwise >the thread that is writing on the queue really has no idea if it locked or >waited for the lock, unless you add some b.s. after every mutex_lock call. So >this doesn't result in a dropped message, it simply results in a [if you've >written it correctly], very tiny delay time. > > I'm not convinced it will be a tiny delay. You're talking about delaying a device-level thread (such as the audio thread) potentially on all outgoing messages because the primary event queue is currently locked. It seems much better to lock the queue when the device-level thread uses it so the primary event handler is delayed instead. In fact, in latter discussions I think we're doing the same thing. >I don't quite understand what you mean by locking based on generation. To be >safe, using mutexes, you have to lock when you put stuff on the queue and when >you take stuff off, or, if you know you only have one reader and one writer, at >least when you update or read the indices. > > What I mean is, you definitely lock when you update the indicies (as discussed above). The read lock is where the double-buffering comes in that Graham and I discussed. There is no need to lock when you read them because of the double-buffering as it would ideally just be a thread-safe pointer swap (that may ultimately require doing some locking). Again, as far as locking, I think we're saying the same thing ultimately: 1) The update lock is not so timing-critical so long as every message gets into the queue 2) The read lock is timing-critical as it blocks the other device-level threads, so the amount of locking by the reader must be very short. A pointer swap using a double-buffer is one option (while the double-buffer may have other purposes also) as the reader can processing events while the 'other' buffer is filling. >am also unsure of double buffering. It seems that if you write your queue and >dequeueing functions well and set the priorities of threads intelligently then >using mutexes shouldn't really be a problem. It also makes it so that the >event handler doesn't have to process the whole queue when it gets sees that >there is data on it. > > I agree with Graham, why not process whole queue? In the renderer, if draw events are coming over a network in any order, then the only way to get all drawing events for the current timestamp is to scan the whole queue. You can't guarantee they are coming in the correct order. Rama From alex at neisis.net Sat Nov 18 09:11:23 2006 From: alex at neisis.net (Alex Norman) Date: Sat Nov 18 09:11:33 2006 Subject: [Tdg] Re: [glv] Message Passing In-Reply-To: <55B2D5B3-2C48-43C3-AFD7-28738AB3B839@mat.ucsb.edu> References: <8C410ED8-4E82-410B-8D7B-789B125191E3@grahamwakefield.net> <71EFD6FD-FF64-4EA2-AA5B-E43665820D0F@grahamwakefield.net> <15C6C961-EED3-4C42-98F6-06980E2A55AB@grahamwakefield.net> <588ABC59-9261-4772-91A0-D3C36DB1106E@umail.ucsb.edu> <6828F04A-3B18-4C7E-9AB9-3AB7209FB26D@grahamwakefield.net> <1CB7861C-1D3A-4761-971C-F67197F1B095@grahamwakefield.net> <455EA1D9.8080101@umail.ucsb.edu> <20061118065429.GX21215@silverninja.net> <55B2D5B3-2C48-43C3-AFD7-28738AB3B839@mat.ucsb.edu> Message-ID: <20061118171123.GA21215@silverninja.net> I think you have the same problem with single writer single reader with this double buffering method. queue1 has data a,b,c,d queue0 is empty Reader{ flip bit [select queue 1 for read] read from queue 1.. } Writer{ index = queue 0 for writing write into x,y,z queue[index] CONTEXT_SWITCH } Reader{ flip bit [select queue 0 for read] read from queue 0.. } back to writer ... write into queue[index] //this is still queue 0 //now we're writing into the queue the reader is reading from because we started writing to this queue before the context switch. Okay, so this is the case if you store the index of which buffer to use at the beginning of the read/write block. If you don't do this then you have to lock EVERY time you write to the queue because the write-vs-read index could change on you. You could set some bit to indicate if a writer is accessing the data or not, and you don't flip the bit in that case... but then the reader has to wait or drop events. I guess this "lock on every write" case is what you were talking about by generation based locking. I guess that does make more sense, though it does involve A LOT of locking. I think there is a way [since you only have single reader/writer] to use one queue and have less locking because: reader{ lock.. get range of valid reads unlock read that shit lock.. update read index unlock } writer{ lock.. get range of valid writes [if you're using some ring buffer or something] or get the last elem on the list so that you can update it's pointers if you're using linked lists. unlock write to that shit lock update the write index unlock } after reading Graham's text again, it seems the "lock on every write" is exactly what you're talking about, and I agree now, that would probably work.. Which to use I think depends on the system. If writers generally push one event on the queue at a time then this double buffering system seems like it would work well, but if writers create lots of events at a time, then maybe this method I've decribed above could work better? -Alex On 0, Graham Wakefield wrote: > > > >I don't quite understand what you mean by locking based on > >generation. To be > >safe, using mutexes, you have to lock when you put stuff on the > >queue and when > >you take stuff off, or, if you know you only have one reader and > >one writer, at > >least when you update or read the indices. > > > > What I was suggesting is a single reader, single writer scenario. > The writer doesn't need to check any locks, since it always has a > buffer to write to. It will want to lock while it writes. The reader > will swap buffers (swap 2 pointers or set a bit) IF the buffer isn't > locked. Otherwise, it will try again on the next event loop. > > It might not be the ideal method, but it seems pretty safe and > minimizes locking/copying I think. > > >> > >>I'm still not sure about the double-buffering or not. I know in > >>rendering I need double-buffering because I need to distinguish draw > >>calls with