What is Linear PCM

Linear PCM is an uncompressed audio format. It can have up to 8 channels of audio at 48khz or 96khz sampling frequency and 16, 20 or 24 bits per sample. It has a maximum bitrate of a huge 6.144 MB/s. It’s also known as LPCM.

Supported Audio Formats in Playback & Recording

Supported Audio Playback formats:

• AAC
• HE-AAC
• AMR (Adaptive Multi-Rate, a format for speech)
• ALAC (Apple Lossless)
• iLBC (internet Low Bitrate Codec, another format for speech)
• IMA4 (IMA/ADPCM)
• linear PCM (uncompressed)
• µ-law and a-law
• MP3 (MPEG-1 audio layer 3

Supported Audio Recording Formats:

• ALAC (Apple Lossless)
• iLBC (internet Low Bitrate Codec, for speech)
• IMA/ADPCM (IMA4)
• linear PCM
• µ-law and a-law

AAC, MP3, and ALAC (Apple Lossless) Playback for AAC, MP3, and ALAC sounds can use efficient hardware-based decoding on iPhone OS–based devices, but these codecs all share a single hardware path. The device can play only a single instance of one of these formats at a time through hardware. If the user is playing a sound in one of these three formats in the iPod application, then your application—to play along over that audio—will employ software decoding. AAC, ALAC, MP3 can employ hardware decoding for playback.

Linear PCM and IMA4 (IMA/ADPCM You can play multiple linear PCM or IMA4 sounds simultaneously in iPhone OS without incurring CPU resource problems. It is same for the AMR and iLBC speech-quality formats, and for the µ-law and a-law compressed formats.

The device can play only a single instance of one of these formats at a time through hardware. For example, if you are playing a stereo MP3 sound, a second simultaneous MP3 sound will use software decoding. Similarly, you cannot simultaneously play an AAC and an ALAC sound using hardware. If the iPod application is playing an AAC sound in the background, your application plays AAC, ALAC, and MP3 audio using software decoding.

To play multiple sounds with best performance, or to efficiently play sounds while the iPod is playing in the background, use linear PCM (uncompressed) or IMA4 (compressed) audio.

Measurement of Audio Latency

Latency = buffer duration + inherent hardware latency

In other words, latency is to get the last sample of audio in its buffer out of the headphones or speaker.

Reference here.

Audio References:
Subfruther.com
Michael Tyson

Audio Framework OpenAL

OpenAL is similar to OpenGL. It is straightforward with 3 main entities: the Listener, the Source, and the Buffer.

The Listener is you. OpenAL allows you to specify where the listenser is in relation to the sources. You can keep the minimum static sound.

The Source is similar to a speaker. It generates sound which the listener can hear. You can move the sources around and get positional effects. Or you keep this simple.

The Buffer is sound that will be played. The buffer holds the raw audio data.

The device is hardware that will be playing the sound and context is the current session for audio.

Minimum Process

* get the device
* make a context with the device
* put data into the buffer
* attach the buffer to a source
* play the source

Why you should use CAF instead of AIF & WAV?

iPhone OS have a native audio file format, the Core Audio Format (or CAF) file format. It is available in iphone OS 2.0 +. it can contain any audio data format supported on a platform. CAF files have no size restrictions—unlike .aif and .wav files—and can support a wide range of metadata, such as channel information and text annotations.

This is command line to convert audio into Little-Endian 16-it 44,100 sample rate format in .caf.

/usr/bin/afconvert -f caff -d LEI16@44100 inputSoundFile.aiff outputSoundFile.caf

It seems like Audio Toolbox methods handle multiple formats. However, if you use the audio in the right format then iphone won’t have to do it at play time.

Audio Toolkit is fine if you have a button and simple UI interaction. It doesn’t play immediately. You can’t match specific frame with specific sound effects. It is late by many frames or the entire audio will pause and wait for audio toolbox to load the sound into the buffer. It is just not good for game or music app.

SDK 3.0 Upgrades on AVFoundation Framework

SDK 3.0 has added the following APIs in AVFoundation framework

• AVAudioRecorder.h
• AVAudioSession.h
• AVAudioSettings.h

Existing APIs in SDK 2.2.1:

• AVAudioPlayer.h
• AVFoundation.h

Objective-C AVAudioRecorder SDK 3.0

AVAudioRecorder class is only available on OS3.0.

// Properties
@property (readonly) NSTimeInterval currentTime;
@property (assign) id <AVAudioRecorderDelegate> delegate;
@property (getter=isMeteringEnabled) BOOL meteringEnabled;
@property (readonly, getter=isRecording) BOOL recording;
@property (readonly) NSDictionary *settings;
@property (readonly) NSURL *url;

// Returns the average power for a given channel, in decibels, for the sound being recorded
– (float)averagePowerForChannel:(NSUInteger)channelNumber

// Deletes a recorded audio file
– (BOOL)deleteRecording

// Initializes and returns an audio recorder
– (id)initWithURL:(NSURL *)url settings:(NSDictionary *)settings error:(NSError **)outError

// Pauses a recording
– (void)pause

// Returns the peak power for a given channel, in decibels, for the sound being recorded
– (float)peakPowerForChannel:(NSUInteger)channelNumber

// Creates an audio file and prepares the system for recording
– (BOOL)prepareToRecord

// Starts or resumes recording
– (BOOL)record

// Records for a specified duration of time
– (BOOL)recordForDuration:(NSTimeInterval)duration

// Stops recording and closes the audio file
– (void)stop

// Refreshes the average and peak power values for all channels of an audio recorder
– (void)updateMeters

Audio Format Settings

NSString *const AVFormatIDKey;
NSString *const AVSampleRateKey;
NSString *const AVNumberOfChannelsKey;

AVFormatIDKey
A format identifier.

AVSampleRateKey
A sample rate, in hertz, expressed as an NSNumber floating point value.

AVNumberOfChannelsKey
The number of channels expressed as an NSNumber integer value.

Linear PCM Format Settings

NSString *const AVLinearPCMBitDepthKey;
NSString *const AVLinearPCMIsBigEndianKey;
NSString *const AVLinearPCMIsFloatKey;

AVLinearPCMBitDepthKey
An NSNumber integer that indicates the bit depth for a linear PCM audio format—one of 8, 16, 24, or 32.

AVLinearPCMIsBigEndianKey
A Boolean value that indicates whether the audio format is big endian (YES) or little endian (NO).

AVLinearPCMIsFloatKey
A Boolean value that indicates that the audio format is floating point (YES) or fixed point (NO).

Audio Encoder Settings

NSString *const AVEncoderAudioQualityKey;
NSString *const AVEncoderBitRateKey;
NSString *const AVEncoderBitDepthHintKey;

Sample Rate Conversion Settings

NSString *const AVSampleRateConverterAudioQualityKey;

Audio Quality Flags

Keys that specify sample rate conversion quality, used for the AVSampleRateConverterAudioQualityKey property.
enum {
   AVAudioQualityMin       = 0,
   AVAudioQualityLow       = 0x20,
   AVAudioQualityMedium    = 0x40,
   AVAudioQualityHigh      = 0x60,
   AVAudioQualityMax       = 0x7F
};
typedef NSInteger AVAudioQuality;

									

Quick Audio Framework Comparison in SDK 3.0

Audio Toolkit (AudioToolbox Framework)
It plays audio but it can’t change pitch or loop.

AVAudioPlayer (AVFoundation Framework)
File-based playback of DRM-free audio in Apple-supported codecs.

Audio Queue Services (AudioToolbox Framework)
It can loop audio but it is painful to re-read the data from the file to adjust the playback frequency. C-based API for buffered recording and playback of audio. It would work for a net radio player, or your own formats and/or DRM/encryption schemes (decrypt in memory before handing off to the queue). Inherent latency due to the use of buffers.

OpenAL
Sample can be pre-loaded. It is easy to adjust the frequency and looping. However, quality is suffered when you stop the looping. The built-in 3D positional audio mixer is too CPU-intensive.

Look for example codes “SoundEngine.h”, “CrashLanding” or “TouchFighter” that use OpenAL.

AudioUnits (aka RemoteIO)
RemoteIO is poorly documented and extremely difficult to implement, similar to AudioQueue Services. Low-level C-based API. Very low latency, as little as 29 milliseconds. Mixing, effects, near-direct access to input and output hardware.

SDK 3.0 AVAudioRecorder

AVAudioRecorder class in 3.0 SDK allow you to record audio in .caf & .aif. If you like to record it in .mp3, you would have to include your own encorder. You can’t change the encorder.

Only changable encorder settings:

• AVEncoderAudioQualityKey;
• AVEncoderBitRateKey;
• AVEncoderBitDepthHintKey;

Memory Leak on AudioServicesCreateSystemSoundID from bad conversion habits

Audio Toolbox Framework & AVFoundation Framework are the most popular audio frameworks to play audio. AudioQueue, part of Audio Toolbox Framework, requires you to manage buffers and byte level.

Mistakes you don’t want to make:

If you think you can convert the file format from .wav to .aif and then rename the extension from .aif to .caf, it may be just a hack and it’s not a good practice at all. This hack will cause a memory leak on the fuction of AudioServicesCreateSystemSoundID. This is something you don’t want to do in your code.

Proper way to convert files to .caf

afconvert -f caff -d ima4 input.wav output.caf

Audio Toolbox Framework

NSString *sound = @"sound";
NSString *path = [[NSBundle mainBundle] pathForResource:sound ofType:@"caf"];
SystemSoundID soundID;
AudioServicesCreateSystemSoundID((CFURLRef)[NSURL fileURLWithPathath], &soundID);
AudioServicesPlaySystemSound(soundID);

AVFoundation Framework

NSString *sound = [[NSBundle mainBundle] pathForResource:@"sound" ofType:@"mp3"];
Player =[[AVAudioPlayer alloc] initWithContentsOfURL:[NSURL fileURLWithPath:sound] error:nil];
[Player prepareToPlay];
[Player play];