iOS Multimedia Framework Introduction

Media layer

The media layer contains graphics, audio, and video technologies that combine to bring the best multimedia experience to mobile devices and, more importantly, make it easier to create apps that are good for external guanyin. You can use the Advanced framework of iOS to create advanced graphics and animations more quickly, or you can access the necessary tools through the underlying framework to accomplish some sort of task in a certain way.

Graphic technology

High-quality graphics are an important part of iOS applications. The simplest and most efficient way to create an application is to use a pre-rendered picture, with a standard view and a control of the Uikit framework, and then hand the drawing task to the system for execution. In some cases, however, you may need some functionality that Uikit does not have, and you need to customize some behavior. In this case, you can use the following techniques to manage the graphical content of your application:

• Core Graphics (also known as quartz) for handling local 2D vector rendering and picture rendering.

• Core Animation (part of the Quartz Core framework) provides a higher level of support for animated views and other content.

• OpenGL ES, which provides support for 2D and 3D rendering using hardware-accelerated interfaces.

• Core text, providing a sophisticated layout and rendering engine.

• Image I/O, which provides interfaces for reading and writing most graphics formats.

• The Asset Library framework (Assets Library framework), which can be used to access photos and videos in the user photo gallery.

Most applications should not need to be changed, or can be run on devices with high-resolution screens with minimal modifications. Because when you draw or manipulate the view, the coordinates you specify are mapped to the logical coordinate system, which is not associated with the underlying screen resolution. And the drawn content is automatically scaled as needed to support high-resolution screens. For code that is based on vectors, the system framework automatically uses extra pixels to improve the content of the drawing and make it clearer. If you use a picture in your application, you can use Uikit to automatically load the high-resolution version of an existing picture. For more information on how to support high-resolution screens, see "supporting high-resolution screens"in the iOS application Programming Guide .

To learn more about image-related frames, see the appropriate content in the media layer framework.

Audio technology

iOS audio technology helps you provide a rich audio experience for your users. You can use audio technology to play or record high-quality audio, or to trigger the vibration of a hair device (a device with a vibrating function).

The iOS system provides several ways to play or record audio for your use. When choosing audio technology, keep in mind that you want to choose the high-level framework as much as possible, as they simplify the work required to play audio. The frameworks listed below are arranged from high to low, and the Media Player Framework (Framwork) provides the most advanced interfaces:

• Media Player framework. The framework makes it easy to access the user's Itume library and supports playing tracks and playlists.

• AV Foundation Framework. It provides a set of easy-to-use OBJECTIVE-C interfaces that you can use to manage the playback or recording of audio.

• OpenAL frame. It provides a set of interfaces for distributing azimuth audio across platforms.

• Core Audio Framework. It provides an interface that is simple and precise and can be used to play or record audio content. You can use these interfaces to play the system's alarm sounds, to trigger a vibration function, to manage multi-channel buffering and playback, and to stream audio content.

iOS audio technology supports the following audio formats:

• Aac

• Apple Lossless (ALAC)

• A-law

• IMA/ADPCM (IMA4)

• Linear PCM

• Μ-law

• Dvi/intel IMA ADPCM

• Microsoft GSM 6.10

• aes3-2003

To learn more about the above audio framework, see the appropriate content in the media layer framework.

Video

iOS has several technologies that can be used to play the movie files of the application package as well as the data flow content from the network. If your device has the right video hardware, these technologies can also be used to capture video and integrate the captured video into your application.

The system also offers several ways to play or record video content, which you can select as needed. When selecting video technology, choose the high-level framework as much as possible, because the advanced framework simplifies the work required to provide support for a feature. The frameworks listed below are arranged from advanced to low level. Among them, the Media Player framework provides the highest level of interface:

• The media Player framework, which provides a set of easy-to-use interfaces that can be used to play a full-screen or partial-screen movie in an application.

• AV Foundation Framework, which provides a set of OBJECTIVE-C interfaces that can be used to manage the capture and playback of movies.

• Core Media Framework, which describes the underlying types used by higher-level frameworks, and also provides some underlying interfaces that are used to process media.

iOS video technology supports playback of movie files that should have .mov , .mp4 ,, .m4v and .3gp file extensions, and that the file should use the following compression criteria:

• The video, up to 1.5 Mbps, 640x480 pixels, 30 frames per second. The low-complexity version of H. Baseline Profile supports AAC-LC audio (and .m4v .mp4 .mov stereo audio up to 160Kbps and 48KHz in file formats).

• The video, up to kbps,320x240 pixels, 30 frames per second. The Baseline profile of level 1.3 supports AAC-LC audio ( .m4v .mp4 as well as .mov up to 160Kbps, 48KHz stereo audio in file formats).

• MPEG-4 video, up to 2.5 Mbps, 640 x 480 pixels, 30 frames per second. Simple profiles Support AAC-LC Audio ( .m4v .mp4 as well as .mov up to 160Kbps, 48KHz of stereo audio in file formats).

• Various audio formats, including the list of "Audio Technologies" listed.

To learn more about the video framework above, see the appropriate content in the media layer framework.

The media layer contains the framework

The following sections describe the services provided by the media Layer framework and framework.

Asset Library Framework

IOS 4.0 introduces the Asset Library framework ( AssetsLibrary.framework ), which provides a query interface through which you can find user photos and data. By using this framework, you can access the assets managed by photos, including the albums saved by the user and the pictures or videos that are imported into the device. You can also save photos or videos to your users ' albums.

To learn more about the framework, see the Asset Library framework reference .

AV Foundation Framework

IOS 2.2 Introduces the AV Foundation Framework ( AVFoundation.framework ), which contains the Objective-c class that can be used to play audio content. By using this framework, you can play sound files or play audio data in memory, play multiple sounds at the same time, and control the playback of individual sounds.

In IOS 4.0 and later, the services provided by the framework are greatly expanded, and the following services are now included in the framework:

• Media Asset Management

• Media editor

• Movie Capture

• Movie Playback

• Track Management

• Meta-data management for media items

• Stereo selection

• Precise synchronization of different sounds

• A objective-c interface for judging the details of a sound file, such as determining the data format, sample rate, and number of channels.

The AV Foundation framework is the only framework for recording audio and video playback in iOS, which also supports the management and processing of media items.

To learn more about the AV Foundation framework, see the AV Foundation Framework Reference .

Core Audio

The core Audio framework family, listed in table 3-1, provides local support for audios. The COre audio Framework provides a C-language interface that can be used to manipulate stereo audio. With the iOS system Core Audio framework, you can generate, record, mix, or play audio in your application, and you can also access the device's vibration capabilities (vibration-enabled devices).

Table 3-1 Core Audio Framework

Framework	Service
CoreAudio.framework	Defines the type of audio data used by the core Audio framework family.
AudioToolbox.framework	Play or record audio files or data streams, and can also be used to manage audio files, play system warning sounds, and trigger vibrations on some devices.
AudioUnit.framework	For the built-in Audio unit service, the built-in audio unit refers to the audio processing module.

For more information on Core Audio, please refer to Core Audio overview . For information on how to play sounds using the Audio Toolbox framework, check out the Programming Guide for voice queuing Services , R, and an audio Toolbox framework Reference .

Core Graphics Frame

Core Graphics Frame ( CoreGraphics.framework ) contains the Quartz 2D drawing API interface. Quartz is a vector drawing engine used by Mac OS x systems that supports path drawing, antialiasing rendering, gradients, pictures, colors, coordinate space transformations, creation, display, and parsing of PDF files. Although the API is based on the C language, it uses object-based abstractions to represent basic drawing objects, which makes it easier for developers to save and reuse image content.

For more information on how to draw content with Quartz, see the Quartz 2D Programming Guide and the Core Graphics framework Reference.

Core Text Frame

IOS 3.2 introduces the core text framework ( CoreText.framework ), which contains a simple and efficient set of C interfaces that can be used to lay out text and manipulate fonts. The Core text Framework provides a complete textual layout engine that allows you to manage the placement of text on the screen. The text you manage can also use different fonts and rendering properties.

The framework is designed for applications such as word-processing programs that require sophisticated text-processing capabilities. If your application requires only one text input and display, you should use the classes already in the Uikit framework.

For more information on how the core text interface is used, refer to the Core Text Programming Guide and the core text reference set .

Core Video Framework

IOS 4.0 introduces the core video framework ( CoreVideo.framework ), which provides support for the cache and cache pools for Core media. Most applications should not use the framework directly.

Image I/O framework

IOS 4.0 introduces the image I/O framework ( ImageIO.framework ), which interfaces can be used to import or export image data and image metadata. The framework is built on top of the Core graphics data types and functions to support all standard image types on iOS.

To learn more about the framework's data types and functions, see the mage I/O reference set .

Media Player Framework

Media Player Framework ( MediaPlayer.framework ) provides advanced support for the application to play video and audio content. With this framework, you can play videos using the standard system interface. IOS 3.0 adds support for accessing the user's itune library. As a result, you can use the framework to play music tracks, playlists, search songs, and display the media selection interface to your users.

In the IOS 3.2 system, the framework has changed to support the playback of video in a view that can be changed in size (full screen is supported previously). Several additional interfaces are also available to support configuring and managing movie playback.

To learn more about the classes in the Media Player framework, see the Media Player Framework reference . To learn how to use the framework to access the user's itunes library, see the IPod Library Access Programming Guide .

OpenAL Frame

In addition to core audio, IOS supports the Open Audio Library (OpenAL). The OpenAL interface is a cross-platform standard for publishing azimuth audio in an application. By using this framework, you can achieve high-performance, high-quality audio in a game or program that requires azimuth audio output. OpenAL is a cross-platform standard, and the iOS platform can be ported to many other platforms using code modules written by OpenAL.

For OpenAL and how to use it, see http://www.openal.org.

OpenGL ES Framework

OpenGL ES Framework ( OpenGLES.framework ) tools are available to draw 2D and 3D content. Based on the C language, the framework works closely with device hardware to provide high frame rates for full-screen game-type applications.

The OpenGL framework needs to be used in conjunction with the Eagl interface. These interfaces are part of the OpenGL ES Framework and are interfaces to OpenGL ES drawing Code and window objects in applications.

In IOS 3.0 and subsequent versions of the system, the OpenGL ES Framework supports both the OpenGL ES 2.0 and OpenGL ES 1.1 interface specifications. The 2.0 specification supports segmentation and point coloring and only supports 2.0 for devices running iOS 3.0 and later. All versions of iOS and iOS devices support the OpenGL ES 1.1 specification.

To learn how to use OpenGL ES in your application, see the IOS OpenGL es Programming Guide . If you need reference information, see the OpenGL ES Framework reference .

Quartz Core Framework

Quartz Core Framework ( QuartzCore.framework ) contains the core animation interface. Core Animation is a high-level animation and blending technology that uses optimized render paths for complex animations and visual effects. It provides an advanced OBJECTIVE-C interface to configure the animation effect and then render it in the device hardware to improve program performance. The Core animation framework is integrated into many parts of iOS, including many classes (such as) in the Uikit framework UIView , and can animate a variety of system behaviors. You can also use the Objective-c interface in the framework to create custom animations directly.

For more information on how to use core Animation in your application, see Core Animation Programming Guide  and  Core Animation Reference set .