|
tech
newsgroups
|
|||||||||||||||||||||||
|
|||||||||||||||||||||||
Asynchronous I/O completion ports vs multi-threadingHi everyone,
I am facing a problem where Asynchronous I/O completion ports using it might help me but I'm still debating whether to start on this method of development. I have to write a data (video) to a file, one frame at a time. Currently I grab the frames on one thread, and perform all of the file writes on a background thread. But - the writes still slow down the whole machine. Do you think that asynchronous I/O completion ports would help here? if so, could you explain why? for me it seemed that they probably just doing the same thing - doing the I/O operation on a different thread. Or am I missing anything? thanks SO much for any idea Lior On Nov 23, 3:50 pm, liormessin***@gmail.com wrote:
Show quote > Hi everyone, Hi,> > I am facing a problem where Asynchronous I/O completion ports using it > might help me > but I'm still debating whether to start on this method of development. > > I have to write a data (video) to a file, one frame at a > time. Currently I grab the frames on one thread, and perform all of > the file writes on a background thread. But - the writes still > slow down the whole machine. > > Do you think that asynchronous I/O completion ports would help here? > if so, could you explain why? for me it seemed that they probably just > doing the same thing - doing the I/O operation on a different thread. > Or am I missing anything? > > thanks SO much for any idea > Lior Well, if you need to Create/Handle a large number of threads, it is more efficient to consider using I/O Completion Ports. Also, the stack's default size for each thread is 1 megabyte, which would mean you've got a large amount of memory tied up just in stacks before adding in any programs or data. The following weblink should shed light on the subject: http://msdn2.microsoft.com/en-us/library/Aa365198.aspx Kellie. liormessin***@gmail.com wrote:
Show quote > Hi everyone, In a word, no. It's unlikely that using IO Completion ports would help > > I am facing a problem where Asynchronous I/O completion ports using it > might help me > but I'm still debating whether to start on this method of development. > > I have to write a data (video) to a file, one frame at a > time. Currently I grab the frames on one thread, and perform all of > the file writes on a background thread. But - the writes still > slow down the whole machine. > > Do you think that asynchronous I/O completion ports would help here? > if so, could you explain why? for me it seemed that they probably just > doing the same thing - doing the I/O operation on a different thread. > Or am I missing anything? given that you have a single producer/single consumer model. What generally does help, is to use FILE_FLAG_NO_BUFFERING, which requires that you write disk-sector-aligned data, and do those writes using async I/O (but not IO completion ports - they wouldn't help). If your data is not naturally disk-sector blocked, then you'll prabably have to implement a re-blocking scheme that copies parts of frames into large, sector-aligned buffers (e.g. 64K or larger), starting writes on buffers as they become full while simultaneously filling the next buffer with new data. You may simply be at the limit of your disk system as well, in which case no amount of clever async programming is going to save you - you may simply need a faster disk. How large are the frames and what's the frame rate? For example, fuill resolution NTSC video requires 27Mb/s write speeds - unless you have a fast RAID controller with multiple fast SAS/SATA/SCSI drives, you won't get anywhere near that write speed for more than a fraction of a second (long enough to fill up all the write buffers on the disk(s)). -cd Actually, with modern IDE disks, one can easily have sustained speed at 50
MB/s. This might require writing with large pieces, as extending a file may become a bottleneck. Though NTFS applies an optimization to that. Show quote "Carl Daniel [VC++ MVP]" <cpdaniel_remove_this_and_nospam@mvps.org.nospam> wrote in message news:OrWuynkLIHA.4948@TK2MSFTNGP02.phx.gbl... > liormessin***@gmail.com wrote: >> Hi everyone, >> >> I am facing a problem where Asynchronous I/O completion ports using it >> might help me >> but I'm still debating whether to start on this method of development. >> >> I have to write a data (video) to a file, one frame at a >> time. Currently I grab the frames on one thread, and perform all of >> the file writes on a background thread. But - the writes still >> slow down the whole machine. >> >> Do you think that asynchronous I/O completion ports would help here? >> if so, could you explain why? for me it seemed that they probably just >> doing the same thing - doing the I/O operation on a different thread. >> Or am I missing anything? > > In a word, no. It's unlikely that using IO Completion ports would help > given that you have a single producer/single consumer model. > > What generally does help, is to use FILE_FLAG_NO_BUFFERING, which requires > that you write disk-sector-aligned data, and do those writes using async > I/O (but not IO completion ports - they wouldn't help). If your data is > not naturally disk-sector blocked, then you'll prabably have to implement > a re-blocking scheme that copies parts of frames into large, > sector-aligned buffers (e.g. 64K or larger), starting writes on buffers as > they become full while simultaneously filling the next buffer with new > data. > > You may simply be at the limit of your disk system as well, in which case > no amount of clever async programming is going to save you - you may > simply need a faster disk. How large are the frames and what's the frame > rate? For example, fuill resolution NTSC video requires 27Mb/s write > speeds - unless you have a fast RAID controller with multiple fast > SAS/SATA/SCSI drives, you won't get anywhere near that write speed for > more than a fraction of a second (long enough to fill up all the write > buffers on the disk(s)). > > -cd >  |
|||||||||||||||||||||||
Other interesting topics