Analysis on the Application of 3D graphics technology on mobile phones

The boom in the Asian market for graphic electronic games is stimulating a huge demand for the next generation of mobile phones with complex 3D graphics Functions. This article describes in detail the technical challenges of 3D graphics technology in mobile phones, the 3D graphics API standards, and the technical competition that developers will face to help engineers understand the application status of 3D graphics technology, analyze existing technical challenges and possible technical competition.

Just two years ago, mobile phones only provided basic 2D bitmap graphics games as additional functions, just as mobile phones that can send text messages appeared a few years ago. Mobile gaming has become popular in Asian countries, and consumers are easily attracted by the next generation of cell phones. Users soon began to want to play higher-performance games on their mobile phones. Mobile phone manufacturers are also taking advantage of advanced graphics features (16-bit color per pixel and qvga resolution color screens with limited graphic acceleration capabilities) and more complex and Interactive Gaming mobile phones. With the improvement in performance, the development speed of graphic mobile phones in Asia is faster than that of similar mobile phones anywhere in the world.

The 3D graphics function is now available. The first stage is to introduce the 3D graphics function through software. This requires a great increase in processor performance, but some operations still have great performance limitations, and highly complex interactive features are not feasible. Last year, especially in the second half of last year, mobile devices with hardware-based and real 3D graphics Functions began to appear on the market. But in many other parts of the world, people still have doubts: for a common voice communication device, why do they need a color screen and add 2D/3D graphics to it? Playing advanced electronic games with mobile phones may soon become popular in Europe and will become popular in the United States in two years. In Western countries, high-end users interested in Internet connection, wireless Internet access, GPS services, built-in cameras, multimedia messages, mobile videos, MP3 playback, and games are increasing.

However, an understandable concept of prudence began to spread in the mobile phone industry: although games rely on 3D graphics or, in fact, 3D graphics to make mobile games with higher performance possible and promote the future mobile phone replacement market, this is not absolutely sure, after all, the game is only an additional feature of the mobile phone voice communication device. Currently, there are several first-generation mobile phones with different architectures and 3D graphics Functions on the market, some of which claim to have low power consumption, others promote better visual effects or specific functional features. But in general, they do not have much difference in performance or functionality.

Technical challenges

Creating demand for advanced features is only one of the challenges mobile device manufacturers face. The task of processing and operating static and active image data with more powerful processing capabilities, coupled with additional storage resources without increasing the volume and weight of devices, this is not technically easy. The technical challenges will become extremely severe if at least or ideally the battery's standby time is increased.

At this time, graphics technology in the dedicated video game market has been greatly improved. For such devices, especially console-based devices, there is not much pressure to put powerful processing capabilities and dedicated game engines into a narrow space. Power consumption is still a problem with portable gaming devices, although they often have enough space to accommodate other hardware, such as complex 3D graphics chipsets and accelerators. However, dual-screen, touch screen, interactivity, and the ability to process a large number of pixels will make the phone a new threat to game consoles such as Nintendo and Sony.

To integrate anything close to these performance into a mobile phone, you need to consider the mobile phone architecture completely. About six hours of battery time is acceptable to the gaming machine at least, but for mobile phones, it is not enough even if the time is doubled. In view of this, most mobile phone designers adopt solutions that can reduce the power consumption of devices, such as adding processing capabilities and operating core CPUs separately.

However, mobile phones are still under pressure from space and must be added with minimal silicon (hardware) overhead. Even with the first color screen, processing and transferring pixel tasks still require coprocessors or dedicated DMA engines that understand pixel space. Today, although many DSP-based chips and kernels, Graphics accelerators, dedicated coprocessors, graphics engines, and software solutions are available, how to implement them remains critical. Architecture, software and hardware division, and system-level chip (SOC) or chipset decision-making must be developed and critical at the early stage of the design process. Manufacturing costs can only be shared by mass production. Later design changes greatly increase costs. Project delays caused by changes may lead to loss of major market and profit opportunities.

The graphic function is not a post-added function, but a complete component of all operations related to the display. Integrating and verifying such complex electronic circuits requires a lot of work. The emergence of a dedicated platform is of great significance to mobile phone developers because it facilitates design reuse and saves a lot of design time and verification work. 3D graphics is the most successful implementation of application subsystems, especially dedicated accelerators that can execute some special functions (not easily implemented through a general processing platform.

3D graphics API standard-open GL-ES

The mobile phone industry needs the support of game developers to ensure that there are affordable games for users to choose from. One potential challenge is the need for an application programming interface (API) communication between processing hardware, software, and graphics engines. The mobile phone industry has been far away from the detours in the early days of the PC 3D game market. At that time, various specialized APIs caused unnecessary competition and conflicts. Major mobile phone manufacturers, chip suppliers, graphics engines and software providers, game developers, and basic equipment development enterprises all quickly see this potential problem, and extensive cooperation/collaboration to jointly ensure the adoption of consistent APIs from the very beginning.

As the mobile 3D graphics industry standard open GL-ES, avoiding mobile phone manufacturers due to incompatible game software format and lead to unnecessary market competition. Open GL-ES comes from the desktop market and is especially easy to scale, and has stripped away some of the original unnecessary features, so it can be achieved with a smaller footprint.

Open GL-ES has received a wide range of industry support, not only by mobile phone manufacturers and devices and kernel providers support, but also by graphics engine, game developers and operating system providers support. More importantly, although open GL-ES is a low-level API, it can be used as a supplement to the more advanced JSR 184 API in Java game environments.

Now, industry experts are developing open GL-ES 2.0, who are some people who understand market and hardware limitations and can correctly assess the resources required to develop software and hardware for future applications. The second-generation mobile phone may enter the design phase this year, and a new product will be launched one year later-although this largely depends on the user's acceptance of the first-generation product. By then these phones will be listed in large quantities, and is expected to release Open GL-ES 2.0 to prepare for the development of third generation mobile phones with 3D graphics features.

Technical Competition

The real "technology competition" will start with the second generation of mobile phones with 3D graphics Functions. Vendors will face unprecedented competition in terms of pure technical performance, especially once the API standards are fully established, there will be almost no difference. Some people believe that mobile phone manufacturers should avoid expanding their platform too quickly and creating some proprietary extensions, and the industry must ensure that open GL-ES standard APIs develop along with the market.

In fact, the open GL-ES road map has been established and people have begun to work on developing open GL-ES 2.0. Although the current API is based on the state machine, for third-generation mobile phones, it must develop into a standard based on the shader (coloring engine. Currently, the API is based on a fixed feature pipeline that enables or disables certain features based on the current rendering conditions, it allows manufacturers to create different mobile phones based on throughput, number of pixels, and similar features.

With open GL-ES 2.0, people can program certain elements in a graphic pipeline so that content developers can precisely define how vertex or pixels are processed. This can use procedural algorithms to make code performance dependent on implementation. Therefore, it not only provides suppliers with greater space for feature set and performance innovation, but also provides greater differentiation (especially in terms of visual quality and effect ), it can also reserve a public platform for developers.

Compared with the desktop or console game machine market that uses a large display, the graphics subsystem developer must review the mobile phone market. The pixel density of the 2.2 inch mobile phone display is much higher than that of the laptop screen. The new 2.5 inch display is likely to have a density of up to 400 pixels per inch, which is twice the current pixel density. Therefore, the traditional method of using a large number of polygon and fast polygon speed to obtain high-resolution images is almost not applicable. SMART designers will focus on the more advanced (but based on) technology for lighting and display. The difference lies in the pixel quality and the number of high-end processing that can be used for these pixels (its metric is the number of operations per vertex per second and the number of operations per pixel per second ). Developers plan to use this method not only to provide the image quality that was previously achieved only through millions of polygon, but also not to break through the processor performance, memory capacity and power consumption indicators.

The planned development of open GL-ES standards not only provides a clear roadmap and steady growth for the entire mobile 3D graphics market, but also provides a healthy competitive market for manufacturers. It is expected that with the development of open standards in the next few years, mobile 3D graphics technology will expand to many platforms and markets, including automobiles and avionics devices.

Compact 3D graphics Solution

Today, the most popular 3D graphics solution for mobile device platforms is the powervr mbx Graphics Accelerator series developed by arm. This product was jointly developed by arm and imagination technologies. Its kernel is used by application processors to provide 3D images with PC and game console quality in mobile phones, wireless games, and other embedded devices.

The graphics accelerator has two versions (mbx R-S and mbx HR-S) that provide qvga and svga graphics performance respectively ). The mbx R-S version has a smaller bare size and is mainly used for small wireless devices. While the mbx HR-S version has higher performance. Both kernels have the option of adding a vertex geometric processor, which can free the master CPU from the geometric processing task for advanced illumination.

Powervr mbx solution that supports full 2D and 3D feature sets and is compatible with open GL-ES, uses tile-based rendering to implement full-bandwidth-intensive pixel processing on the chip. By canceling the Z buffer between the 3D kernel and external memory and ensuring that only visible pixels are written to the frame buffer, the memory bandwidth requirements can be reduced. Block rendering not only supports high-precision rendering and synthesis (even on a system with a 16-bit frame buffer), but also implements delayed textures that can cancel all redundant texture operations. This scheme facilitates the adoption of a Unified Storage Architecture. In this architecture, the graphic subsystem shares the system SDRAM with the CPU, so that the cost-saving and space-saving SOC can be used.

The mbx kernel is easy to integrate with ARM926EJ-S and ARM1136J-S as well as the arm primexsys platform, reducing product time to market. The kernel architecture is compact and saves space and silicon costs. By adopting mature power management technology (including module-and register-level clock selection), the power consumption is minimized.

Other integration support can be provided through the amba ahb (Advanced High Performance Bus) interface of ARM corporation and a variety of arm primecell peripheral devices. These pre-verified and hardware-proven Soft IP macro units can be easily integrated with the 3D graphics acceleration solution through the multi-port storage controller interface of the primecell peripheral, because these interfaces are optimized for mbx R-S and HR-S.

Evolving OpenGL-es

The khronos workgroup is a consortium composed of multiple member organizations. It focuses on developing licensed and cross-platform open standard graphics APIs to create and play dynamic media on various platforms and devices. All khronos members can participate in the development of khronos API specifications, vote at various stages prior to public deployment, and accelerate advanced 3D platforms and applications through early access to draft specifications and compliance tests..

Khronos API specifications include: openml for capturing, transmitting, processing, displaying, and synchronizing digital media (including 2D/3d and audio/video streams; openvg can provide low-level hardware acceleration interfaces for vector graphics libraries such as flash and SVG; openmax for media processing standard access to previously widely used video codecs such as graphics, audio and image libraries, and MPEG-4; and open gl-es.

Open GL-ES is a license-free, cross-platform API for embedded systems (including handheld devices, instruments, and vehicles) with full-featured 2D and 3D graphics. It is a set of well-defined desktop OpenGL that allows you to create flexible and powerful low-level interfaces between software and graphic acceleration. OpenGL ES 1.0 includes the common profile (public class) and common-lite profile (shared simplified class) for floating point and fixed point systems, and EGL specifications for handheld connections to local window systems. OpenGL-ES 2.0 is currently under development.

In addition, a "security key" working group has been established to help transplant standards to other mobile platforms that use 3D graphics, especially automotive and avionics devices.

Author: Ed Plowman
3D graphics Product Manager
Arm