Reverse Playback of Audio Streams in iOS

link管理

链接快照平台

输入网页链接，自动生成快照
标签化管理网页链接

The Demo Project

If you’d like to follow along, download the starter demo project and open it up. (Alternatively, a completed version of the demo can be gotten here , on Github.)

This is essentially the same player we set up last time around, but with a few modifications:

Swift: This time we’re using Swift as our project language (but note we’ve retained the use of Objective-C for our reading and rendering code, for reasons we touched on here .)

FileReader Class:

we’ve moved our file reading methods (


    openFileAtURL:

and


    readFrames:audioBufferList:bufferSize:

) into their own


    FileReader

class, where they belong;

we’ve refactored those methods by breaking them into smaller, more tightly-focused sub-methods; and

we’ve given our view controller the necessary property and method for creating a fresh reader object each time a new file is loaded from the iPod Library;

Bridging Header: We’ve added a bridging header our project, in order to make our Objective-C classes available to Swift.

Scrubbing: we’ve added


    UISlider

for scrubbing through the audio file.


    seekToFrame:

method to


    FileReader

(so it can perform the scrubbing);


    IBAction

to our view controller (so our slider can talk to


    FileReader

); and


    FileReaderDelegate

protocol (in


    FileReader

) to which our view controller conforms, so it can be notified of changes to the current playback location and update the slider control accordingly.)

Continuous Looping of the File: We’ve set things up so when playback reaches the end of the file, it jumps back to the beginning and starts all over again.

UI: Finally, we’ve gussied up the UI (well, just a smidgen) with:

the aforementioned UISlider for scrubbing through the audio file;

a UISegmentedControl for toggling playback direction;

a pair of labels for displaying title/artist information for the currently loaded track;

a handful of descriptive labels for each of the above.

a little splash of color for some added stimulation!

Like so:

Run the project (on a device, not the simulator) and you’ll see it behaves much as before: we can browse the device’s iPod Library, choose a file, play it and pause it. In addition, we can now scrub to random locations within the file, and when playback reaches the end of the file, it loops back to the beginning.

As we pointed out last time, if you check the app’s memory footprint in the Debug navigator you’ll find that it is in fact quite small, and for good reason: we’re streaming our audio file directly from disk rather than loading the entire thing into memory.

The Reverse segmented control doesn’t do anything, of course, since we haven’t implemented reverse playback yet – that’s what we’re here to talk about!

Reversing The Audio Stream

Like I said, we’re going to tackle this in two steps: first, by attempting to read our forwardly-arranged sample blocks in reverse order , and second, by attempting to reverse the order of samples within each block.

The first thing we need is a way to indicate what the current playback direction should be. Add a new property to FileReader.h :

@property (assign, nonatomic) BOOL reversePlayback;

Then, in FileReader.m , add the following to initWithFileURL: , below the existing _isReading = NO; line:

_reversePlayback = NO;

Next we need an @IBAction for setting the reversePlayback flag when the UISegmentedControl is toggled, so let's add an action method to ViewController :

@IBAction func reversePlayback(sender: UISegmentedControl) if reader != nil { if sender.selectedSegmentIndex == 0 { reader.reversePlayback = true } else { reader.reversePlayback = false

Don't forget to wire this action up to the UISegmentedControl 's "Value Changed" event in the Storyboard!

Finally, head down to the readFrames: method and replace the line _frameIndex += frames; with the following:

// Reverse Playback if (self.reversePlayback) { NSLog(@"Reversing playback"); for (int buff = 0; buff < audioBufferList->mNumberBuffers; buff++) { Float32* revBuff = audioBufferList->mBuffers[buff].mData; [self reverseContentsOfBuffer:revBuff numberOfFrames:frames]; _frameIndex -= frames; // decrement frame index } else { _frameIndex += frames; // increment frame index

This is actually fairly straightforward:

First we check if our


    reversePlayback

flag is set. If its not - i.e. we're currently performing normal playback - the only thing we do is what we were doing before: we allow our buffers to continue unmolested on their journey back to the render callback, and we increment


    _frameIndex

for the next read cycle.


    reversePlayback

is set, we loop through our incoming buffers (we're using stereo interleaved here, so there's only going to be one), set up a new pointer variable and pass that to something called


    reverseContentsOfBuffer

. We'll write that method in a moment.

If, at this point, you were to comment out everything in that code block except for the decrement of _frameIndex , you'd get a nice demonstration of why merely reversing the index is only a partial solution: the sequence of reads does indeed move backwards through the file, but the sample frames in each read block remain arranged in forward sequential order. (If you decide to try this, be sure to first scrub to some location other than the very beginning of the file, before reversing playback.)

Let's take care of that now by adding the following method to FileReader.m :

- (Float32*)reverseContentsOfBuffer:(Float32*)audioBuffer numberOfFrames:(UInt32)frames Float32* reversedBuffer = audioBuffer; Float32 tmp; int i = 0; int j = frames - 1; while (j > i) { tmp = reversedBuffer[j]; reversedBuffer[j] = reversedBuffer[i]; reversedBuffer[i] = tmp; return reversedBuffer;

So much power in so few lines of code:

First, we're passing in a pointer to our buffer - the one we declared a few lines back - along with the size of that buffer. Remember, this pointer is really pointing to our render callback's outgoing buffer(s), which


    ExtAudioFileRead

has just filled with fresh audio samples from the file on disk.

Next, we set up a pair of

int

counter variables, one pointing to the first element in the buffer and another pointing to the last element.

We also set up a third variable,

tmp

, to help us play Three Card Monte with the buffer's samples.

Finally, we set up a


    while

loop to perform some magic: on each iteration of the loop, the counters

and

increment in opposite directions -

going up and

going down. When they meet in the middle -


    j > i

- all the samples in the buffer will have been shuffled into reverse order - which is how the render callback will now ship them out to the hardware.

The last thing we need to do is modify checkCurrentFrameAgainstLoopMarkers - which currently handles looping around when playback reaches the end of the file - to perform a similar check when playback is reversed and we reach the beginning of the file:

- (void)checkCurrentFrameAgainstLoopMarkers:(SInt64)currentFrame inFrames:(UInt32)frames if (self.reversePlayback) { // If we're approaching startOfFile and have fewer than a buffer's worth of samples to go, reduce the size of the read if (currentFrame - (SInt64)frames < self.lMarker) { frames = (UInt32)currentFrame; // If we're *exactly* at startOfFile, seek to (endOfFile - frames) if (currentFrame - (SInt64)frames == self.lMarker) { [self seekToFrame:self.rMarker - frames]; } else { // forward // If we've reached endOfFile, seek to startOfFile if (currentFrame > self.rMarker) { [self seekToFrame:self.lMarker];

Try running the app, but with the following caveat: depending on the file you've loaded for playback - whether its compressed or not and, if so, how it was encoded and at what bit rate - you may (or may not) hear disappointing results when you reverse playback.

This means we have one more challenge to overcome.

Dealing With Compressed Formats

Reversing playback on the fly like this works great for PCM formats where each sample frame consists of a single set of samples, spread across all channels at that point in time. It even works well for robust compressed formats like AAC and well-encoded, high bit rate MP3s.

But compressed formats present an inherent challenge: unlike uncompressed formats, the frames of a compressed file consist of blocks of encoded samples plus a header (at the beginning of each block) indicating how the samples in that particular block were encoded - information a decoder needs in order to decode!

So what happens ordinarily, when you're playing a compressed file and the next read lands in the middle of a block, without the header? The MP3 codec has a mechanism for retaining the header and samples of incomplete blocks, so they can be reunited with the rest of the block on the next read. Clever!

But you can see where this is going: read those sample blocks in reverse order and you've clearly subverted the entire mechanism! The result: buffers coming back from ExtAudioFileRead with entire swaths of their samples set to zero. Nasty!

A Solution

What to do? One strategy is to seek to a location in the file prior to the data you're actually interested in, read in a greater number of samples (in the hopes of capturing the complete block of target samples including the header) and, after conversion, simply drop the extra frames.

This actually works surprisingly well, primarily because ExtAudioFileRead is optimized to be fast . Still, given the nature of compressed formats, it involves something of a gamble, which is: how can you be sure you've gone back far enough in the file to get the header data you need for the frames you actually want to read?

You can't. But by reading a fairly conservative number of extra frames, we can get pretty far with this idea. I'm only going to focus on detecting MP3 files because, honestly, I haven't had much trouble reversing other compressed formats.

Begin by adding a define macro to the top of FileReader.m

#define kExpandedBufferSize 4096

Typically, reading is done in blocks of 512 or 1024 sample frames so 4096 makes for a nice, comfortable multiple. If it seems excessive to be grabbing 4-8x as many samples as we would normally need, keep in mind that not only is ExtAudioFileRead very fast, but the way we're going to 'crop' our buffer is by simply reassigning its pointer.

Next, there should be a flag we can set when we're dealing with an .MP3 file. Add another property to FileReader.m :

@property (assign, nonatomic) BOOL isMP3;

And add this line to initWithFileURL , so we always begin life with isMP3 turned OFF:

_isMP3 = NO;

For this to be useful, we need some way of ascertaining if the current file is an MP3 or not. Add the following method, which decodes the file's formatID flag (and then tests whether that flag indicates an MP3) to the bottom of FileReader.m :

- (BOOL)fileIsMP3:(UInt32)formatID char formatIDString[5]; UInt32 ID = CFSwapInt32HostToBig (formatID); bcopy (&ID, formatIDString, 4); formatIDString[4] = '\0'; // NSLog (@"Format ID: %10s", formatIDString); NSString* fileExtension = [NSString stringWithCString:formatIDString encoding:NSUTF8StringEncoding]; if ([fileExtension isEqual: @".mp3"]) { return YES; return NO;

Now, in openFileAtURL we can check ( after we've retrieved the file's native format, of course) if we've got an MP3 and, if we have, set the flag accordingly::

self.isMP3 = [self fileIsMP3:_fileFormat.mFormatID] ? YES : NO;

With our flag and helper method in place, we can now add the necessary conditional code for reversing an MP3 file stream. Start by modifying createFileReadingBufferList as follows:

- (void)createFileReadingBufferList // AudioBufferList defines a single buffer, so we need to allocate additional buffers ourselves fileReadingBufferList = (AudioBufferList*)malloc(sizeof(AudioBufferList) + (sizeof(AudioBuffer) * (_clientFormat.mChannelsPerFrame))); fileReadingBufferList->mNumberBuffers = _clientFormat.mChannelsPerFrame; for ( int i=0; i < fileReadingBufferList->mNumberBuffers; i++ ) { fileReadingBufferList->mBuffers[i].mNumberChannels = 1; UInt32 bufferSize; if (self.isMP3 == YES) { bufferSize = kExpandedBufferSize; // for reversed MP3 files } else { bufferSize = 1024; fileReadingBufferList->mBuffers[i].mDataByteSize = bufferSize * sizeof(float); fileReadingBufferList->mBuffers[i].mData = malloc(bufferSize * sizeof(float));

This sets a larger buffer size (using our define macro) for reading in the extra frames. Next, add the following to readFrames: , above the code that currently handles reversing:

UInt32 framesToRead; if (self.isMP3 && self.playbackIsReversed) { // Larger reads for reversed MP3s framesToRead = kExpandedBufferSize; CheckError(ExtAudioFileSeek(_audioFile, _frameIndex - (framesToRead - frames)), "MP3 ExtAudioFileSeek FAILED"); CheckError(ExtAudioFileRead(_audioFile, &framesToRead, fileReadingBufferList), "Failed to read audio data from audio file"); for (int buff = 0; buff < fileReadingBufferList->mNumberBuffers; buff++) { Float32* croppedBuffer = fileReadingBufferList->mBuffers[buff].mData; croppedBuffer = &croppedBuffer[kExpandedBufferSize - frames]; audioBufferList->mBuffers[buff].mData = croppedBuffer; } else { // Normal reads otherwise framesToRead = frames; CheckError(ExtAudioFileSeek(_audioFile, _frameIndex), "Non-MP3 ExtAudioFileSeek FAILED"); CheckError(ExtAudioFileRead(_audioFile, &framesToRead, audioBufferList), "Failed to read audio data from audio file");

All we're doing here is checking if we're currently reversing playback and, if we are, is the file we're reversing is an MP3? If we are (and it is), we (a) set our read size ( framesToRead ) to the kExpandedBufferSize macro we set up earlier; (b) have ExtAudioFileRead read framesToRead number of sample frames to a new float buffer; and (c) 'crop' the returned (and filled) buffer by resetting its pointer (which normally points to the first element in the buffer) to point instead to kExpandedBufferSize minus frames - which gives us the sample frames we were looking for in the first place. Thanks to the expanded read size, however, ExtAudioFileRead will have been able to completely fill those frames with properly decoded samples!

With that, we should give it a test run... check it out ...

And there you have it: On-the-fly reversible playback, streamed right out of your iPod Library!

The Demo Project

Reversing The Audio Stream

Dealing With Compressed Formats

A Solution

Post navigation