At Intel™The processor transplanted OpenGL * games to Android * (Part 1)

There is a huge opportunity to migrate games and other applications that use a large number of 3D graphics from OpenGL to Google Android devices, including devices built on Intel's atom micro-architecture, because OpenGL-based games, game engines, and other traditional software are easy to obtain, OpenGL is easy to transplant, and Android can provide OpenGL ES and C/C ++ support. Even, the availability of many OpenGL-based games and engines is the same as that of open-source Software, such as the Quake series of Id Software. This article consists of two parts. This article describes how to start such a project by detailing the obstacles in porting the rendering components of applications built by earlier versions of OpenGL to Android on Intel's Atom processor. These applications can be games, game engines, or any software that uses OpenGL to build 3D scenes or graphical user interfaces (guis. In addition, it also includes porting OpenGL code from desktop operating systems (such as Windows * and Linux *) and applications using the embedded version of OpenGL ES (whether including or not including the external wing system.

The first part of this article describes how to use OpenGL ES on the Android platform through the software development kit (SDK) or Android Native Development Tool Kit (NDK), and how to determine the method to choose. This section also describes the OpenGL ES sample applications in various sdks and NDK, as well as Java * native interfaces (JNI). This allows you to use Java and C/C ++ components in combination. In addition, we also discussed how to determine whether to use OpenGL ES version 1.1 or 2.0.

The second part will discuss the obstacles that you should understand before starting such a project to transplant OpenGL games, including the differences between OpenGL extensions, floating point support, texture compression formats, and the GLU library. It also describes how to use OpenGL ES to set up an Android development system for the Intel Atom processor and how to obtain the optimal performance of the Android virtual device simulation tool.

Images on Android

You can render images on the Android platform in four different ways. Each method has its strengths and weaknesses (see table 1 ). This document does not describe any of the four methods. There are only two ways to transplant OpenGL code from other platforms: the SDK package class for OpenGL ES and the NDK that can be used to develop OpenGL ES in C/C ++. For the other two methods, the SDK Canvas application programming interface (API) is a powerful 2D API that can be used in combination with OpenGL ES, but is limited to 2D and requires new code.

Android's Renderscript API initially does not support OpenGL ES and has been deprecated in API level 16 (Jelly Bean), so it cannot be used in new projects. Currently, Renderscript is most suitable for applications that can improve the performance of computing-intensive algorithms and do not need to allocate a large amount of memory or transmit a large amount of data, such as simulation computing of the game physical engine.

Table 1. Four ways to render images on Android

Method	Rules
SDK Canvas API	Only Java and 2D images are supported.
SDK package class for OpenGL ES	OpenGL ES 1.1 and 2.0 can be called from Java (with JNI overhead)
NDK OpenGL ES	OpenGL ES 1.1 and 2.0 (including calling native C/C ++ from Java)
Renderscript for OpenGL ES	OpenGL ES support has been deprecated in API level 16

It is difficult to transplant OpenGL applications to earlier versions of Android, because most traditional OpenGL code is written in C or C ++, in addition, Android only supports NDK (Cupcake) before NDK is released in Android 1.5 ). OpenGL ES 1.0 and 1.1 are supported from the beginning, but the performance is inconsistent because acceleration is optional. However, Android has made significant progress in recent years. Added OpenGL ES 2.2 Support to the SDK in Android 2.0 (Froyo) and NDK in Revision 3, and added support for extended OpenGL ES extension in NDK Revision 7. Now, on all new Android devices, the accelerated OpenGL ES 1.1 and 2.0 are essential configurations-especially as the screen size increases. Today, Android provides consistent and reliable performance for 3D intensive applications built on OpenGL ES 1.1 or 2.0 in Java or C/C ++, developers can also choose multiple ways to make the implant process easier.

Android framework SDK Using OpenGL ES package

The Android SDK framework provides a package class for three Android versions of OpenGL ES (1.0, 1.1, and 2.0. These classes support Java code to easily call the OpenGL ES driver in the Android system-even if the driver is executed locally. If you are creating a new OpenGL ES Android game, or are willing to convert traditional C/C ++ code to Java, this is probably the easiest way. Although Java design is portable, it is difficult to transplant Java applications, because Android cannot support a full set of existing Java platforms, Standard Edition (Java SE), Java platform, Micro Edition (Java ME) category, library, or API. Although the Canvas API of Android supports 2D APIs, it can only be used on Android and cannot be compatible with traditional code.

Other categories provided by the Android SDK make it easier to use OpenGL ES, suchGLSurfaceViewAndTextureView.GLSurfaceViewIt is similar to the SurfaceView class used with the Canvas API, but it also has other features-especially for OpenGL ES. It can process the initialization of the required embedded system graphics library (EGL), and can allocate rendering interfaces for Android to display and render at a fixed position on the screen. In addition, it provides some useful features for tracking and debugging OpenGL ES calls. Through execution orientedGLSurfaceView.Renderer()You can quickly create a new OpenGL ES application, as shown in table 2.

Table 2. Basic Methods for GLSurfaceView. Renderer

Method	Description
OnSurfaceCreated ()	Called once when the application starts Initialization
OnSurfaceChanged ()	Call when the size or direction of GLSurfaceView changes
OnDrawFrame ()	Repeat the call to render the scene of each frame

If you start from Android 4.0, you can useTextureViewClass, do not useGLSurfaceViewTo provide rendering interfaces for OpenGL ES with additional functions, but more code is required.TextureViewThe interface runs in the same way as common Views and can be used to render the interface outside the screen. WhenTextureViewsWhen merging to a screen, you can use this function to migrate, convert, implement animation effects or mix them. You can also useTextureViewUse OpenGL ES rendering and Canvas API in combination.

TheBitmapAndGLUtilsClass. You can use the Android Canvas API to create textures for OpenGL ES, or load textures from PNG, JPG, or GIF files. Bitmap can be used to allocate rendering interfaces to the Canvas API of Android.GLUtilsClass can convert the effect from Bitmap to OpenGL ES texture. This integration allows you to use the Canvas API to render 2D images and then use them as textures used in combination with OpenGL ES. This is especially useful for creating graphical elements not provided by OpenGL ES, such as GUI widgets and text fonts. But of course, new Java code is needed to take advantage of these features.

BitmapClass is mainly used with the Canvas API. When it is used to load textures for OpenGL ES, there are some serious restrictions. The Canvas API complies with the Porter-Duff specification for alpha-value Hybrid Processing. Bitmap uses the premultiplied format (A, R * A, G * A, B *) store the image to optimize the alpha value of each pixel. This is suitable for Porter-Duff, but not OpenGL, which uses a non-premultiplied (ARGB) format. This means that the bitmap class can only be used with textures that are completely opaque (or do not have an alpha value for each pixel. Generally, 3D games require textures with the alpha value of each pixel. In this case, you must not use bitmaps to load textures from byte arrays or through NDK.

Another problem isBitmapOnly images can be loaded from PNG, JPG, or GIF formats. However, in general, OpenGL Games use a compressed texture format decoded by the GPU and are generally only dedicated to the GPU architecture, for example, ETC1 and PVRTC.BitmapAndGLUtilsNo dedicated compression texture format or mipmapping is supported. These textures are frequently used by most 3D games, which poses a serious obstacle to porting traditional OpenGL games to Android using sdks. Until Google solves these problems, the best solution is to avoid usingBitmapAndGLUtilsClass to load the texture. This article will further discuss the texture format, "texture compression format ."

Android ApiDemos containsStaticTriangleRendererTo demonstrate how to useGLES10Encapsulation,GLSurfaceView,BitmapAndGLUtilsClass loads opaque textures from PNG resources. NameGLES20TriangleRendererSimilar versionsGLES20Encapsulation class. If you are using an encapsulation class to process the Android framework SDK, these sample applications can lay a good foundation for developing OpenGL ES games. Do not use the nameTriangleRendererBecause it is namedjavax.microedition.khronos.openglesJava uses the encapsulation bound to older versions of OpenGL ES. Google has created a new binding to provide static interfaces for OpenGL ES for Android. These static bindings provide better performance and more OpenGL ES features, it also provides a programming model similar to that used by OpenGL ES in combination with C/C ++-which is beneficial for code reuse.

The Android framework SDK supports OpenGL ES through multiple Java-oriented bindings provided by Google and Khronos, as shown in table 3.

Table 3. encapsulation class summary and example applications for OpenGL ES

Java binding for OpenGL ES	Description	Example Application
Javax. microedition. khronos. egl	Khronos standard definition	-
Javax. microedition. khronos. opengles	Khronos standard definition	TriangleRenderer, Kube, CubeMap
Android. opengl	Static interfaces for Android	-

Android-specific static binding provides better performance. If feasible, use it instead of Khronos binding. Static binding provides encapsulation classes for all versions of OpenGL ES available for Android applications. Table 4 summarizes these classes.

Table 4. Android encapsulation class summary and example applications for OpenGL ES

API version	Java class	Example Application
OpenGL ES 1.0	Android. opengl. GLES10	StaticTriangleRenderer, CompressedTexture
OpenGL ES 1.0	Android. opengl. GLES10Ext	-
OpenGL ES 1.1	Android. opengl. GLES11	-
OpenGL ES 1.1	Android. opengl. GLES11Ext	-
OpenGL ES 2.0	Android. opengl. GLES20	GLES20TriangleRenderer, BasicGLSurfaceView, HelloEffects

These encapsulation classes support the conversion of most OpenGL ES calls in traditional C/C ++ code to Java only by using the appropriate API version of the encapsulation class OpenGL ES function and symbol name prefix. See the example in table 5.

Table 5. Example of editing OpenGL ES calls from C to Java

C Language	Java language
GlDrawArrays (GL_TRIANGLES, 0, count)	GLES11.glDrawArrays (GLES11.GL _ TRIANGLES, 0, count)
GlDrawArrays (GL_TRIANGLES, 0, count)	Gles1_gldrawarrays (gles1_gl _ TRIANGLES, 0, count)

There are three major limitations to porting OpenGL games to Android using the above encapsulation classes: the overhead and operations of a large number of Java virtual machines and JNI are required to convert traditional C/C ++ code to Java. Java is an interpreted language. All Java code on Android runs on the Dalvik virtual machine, so it is slower to run than the compiled C/C ++ code. Because the OpenGL ES driver is always running locally, calling OpenGL ES functions through these encapsulation results in JNI overhead, which limits the performance of game graphics rendering. The more OpenGL ES calls your application sends, the greater the impact on the JNI overhead. Fortunately, the OpenGL ES design can help minimize the number of calls typically required in a performance-critical rendering loop. If performance is important, you can choose to use NDK to migrate performance-critical code to C/C ++ at any time. However, if your code starts with C/C ++, you 'd better use NDK first.

Whether to convert C/C ++ code to Java depends on the specific situation of your project. If the amount of C/C ++ code is relatively small and easy to understand, and you do not want to improve performance and power consumption, you can convert it to Java. If the C/C ++ code is large and hard to understand, and you do not want to increase the power consumption, consider using NDK.

Use Android NDK

Google added the NDK in June 2009 to allow applications to use local running C/C ++ code, which provides higher performance than Java. Because most traditional OpenGL code is written in C or C ++, NDK can provide a simpler path to transplant applications, especially when the amount of C/C ++ code is so large that it is not practical to convert all of it into Java. This is the main reason why Google decided to release the NDK publicly, which makes it easier to transplant OpenGL games to Android. Due to these advantages, NDK has become the main method for applications that require fast graphics speed on Android.

With NDK, you can write C/C ++ code to the Linux shared object library, which is statically connected to your Android application. Library is built using the GNU tool. These toolkit are included in the NDK release package provided by Google. You can use Eclipse * integrated development environment or command line interface to run the tool on Windows, Mac OS * X, or Linux development system. The toolchain supports three processor architectures: ARM, Intel lingdong (x86), and MIPS. Although all the performance of C/C ++ is available, most Linux APIs are unavailable. In fact, the directly supported APIs only include OpenGL ES, OpenMAX * AL, OpenSL ES *, zlib, and Linux File I/O. Google calls them stable APIs. However, documents about how to port other Linux libraries to your NDK project will be provided as needed.

NDK allows you to flexibly partition code between Java and C/C ++ based on application conditions. NDK supports Calling C/C ++ code from Java called Java Native Interface. However, JNI calls incur a large amount of overhead. Therefore, it is important to use native code to partition the application so as to minimize the call volume through JNI. Generally, most OpenGL ES should be written in C/C ++ for optimal performance and ease of transplantation,GLSurfaceViewAnd other SDK classes to manage application lifecycle events and support other game functions, you can write new Java code. Android started to support JNI for Intel's Atom processor, starting with the NDK version 6b.

NDK supports OpenGL ES 1.1 and 2.0 and provides sample applications for both versions. These sample applications also demonstrate how to use JNI to combine C functions with Java. The difference between these applications lies in the Code partitioning and threading between Java and C. Both use NDK and local C Code. However,native-mediaAll the OpenGL ES code of the sample application is completed in Java, andsan-angelesAndnative-activityAll OpenGL ES code of is completed in C,hello-gl2The EGL and OpenGL ES codes are separated between Java and C. We should avoid usinghello-gl2Sample, not only because of the preceding split, but also because it cannot be configured preferentially for the OpenGL ES 2.0 interfaceGLSurfaceViewIt is responsible for callingsetEGLContextClientVersion(2). See table 6.

Table 6. Summary of OpenGL ES sample application in NDK

APIs used	Example Application	SDK/NDK Partition
OpenGL ES 1.1	San-angeles	All EGL and OpenGL ES codes are C.
OpenGL ES 1.1	Native-activity	All codes are C and use the NativeActivity class.
OpenGL ES 2.0	Hello-gl2	The EGL settings are in Java, and the OpenGL ES code is C.
OpenGL ES 2.0	Native-media	All EGL and OpenGL ES codes are Java.

Although OpenGL ES is not usedbitmap-plasmaThe sample is also very interesting because it demonstrates how to usejnigraphicsLibrary to execute local functions and directly access the pixels of Android bitmap.

Note: You can download Android NDK from the http://developer.android.com/tools/sdk/ndk/index.html.

Activity lifecycle events and thread

Android requires that all calls to OpenGL ES be executed from a single thread, because the EGL environment can only be associated with a single thread, it is not recommended to render graphics on the main UI thread. Therefore, the best way is to create a separate thread for all OpenGL ES code and keep executing it through this thread. If your application usesGLSurfaceViewIt automatically creates this dedicated OpenGL ES rendering thread. In other cases, your application must create a rendering thread on its own.

san-angelesAndnative-activityAll OpenGL ES code of the sample application is in C,san-angelesSome Java andGLSurfaceViewTo create rendering threads and manage activity lifecycles.native-activityThe sample does not use any Java code. Do not useGLSurfaceViewBecause it manages the activity lifecycle in C. UseNativeActivityClass.NativeActivityIs a convenient class provided by NDK. It allows you to use local code to execute activity lifecycle handlers, suchonCreate(),onPause()AndonResume(). Some Android services and content providers cannot directly access from local code, but they can be obtained through JNI.

native-activityThe sample is suitable for importing OpenGL ES games because it demonstrates how to useNativeActivityClass andandroid_native_app_glueThe static library uses local code to process lifecycle activities. This class provides a separate rendering thread for OpenGL ES code, a rendering interface, and a window on the interface. Therefore, you do not need to useGLSurfaceViewOrTextureViewClass. The main entry point of the local application isandroid_main()It runs in its own thread and has its own event loop for retrieving input events. Unfortunately, NDK cannot provide a sample version for OpenGL ES 2.0, but you can use 2.0 code to replace all 1.1 code in this sample.

UseNativeActivityThe application must be used on Android 2.3 or later, and a special statement is issued in its configuration file. For details, see the second part of this article.

Java Native Interface

If you choose to implement most applications in C/C ++, it is difficult to avoid using Java classes for larger and more professional projects. For example, Android AssetManager and Resources APIs can only be used in sdks. This is the preferred method for international processing and different page sizes. However, JNI provides a solution because it not only allows Java code to call C/C ++ functions, but also allows C/C ++ code to call Java classes. Therefore, although JNI may produce some overhead, do not avoid using it completely. It is the best way to access important system functions that can only be used in the SDK-especially when these functions are not performance-critical. This article does not provide a complete introduction to using JNI, but below lists the three basic steps required to call JNI from Java to C/C ++:

1. Add a declaration to the C/C ++ function in the Java class file as the local type.
2. Add a static initialization tool for the shared object library that contains local functions.
3. Add the corresponding name to the local source file according to the specific naming scheme.

Note: For more information about using JNI with NDK, see http://developer.android.com/training/articles/perf-jni.html.

Select OpenGL ES 1.1 or 2.0?

Which version of OpenGL ES should you use on Android? Version 1.0 of OpenGL ES has been replaced by version 1.1. Therefore, you must select version 1.1 and version 2.0. Khronos and Google may support two editions without limit, but in most cases, OpenGL ES 2.0 is better than 1.1. With its OpenGL coloring language (GLSL) ES coloring tool programming features, it features more comprehensive and provides higher performance. Even, it may require less code and memory to process textures. However, the reason Khronos and Google continue to support version 1.1 is that it is more like the initial OpenGL 1.xthat has been used in the desktop and console gaming world for decades. Therefore, porting old games to OpenGL ES 1.1 is easier than 2.0, and the older the game version, the more suitable this situation is.

If the transplanted game does not have any shadow code, you can choose OpenGL ES 1.1 or 2.0, but it may be easier to use version 1.1. If your game already has the shadow code, you should definitely choose OpenGL ES 2.0, especially considering that the Android version has been widely used in 2.0. According to Google, as of October 2012, more than 90% of Android devices accessing the Google Play website supported OpenGL ES 1.1 and 2.0.

Note: For more information, see http://developer.android.com/about/dashboards/index.html.

Conclusion

You can implement graphics rendering on Android with OpenGL ES through the Android SDK, the NDK, or a combination of both using the JNI. the SDK method needs to be encoded in Java and is most suitable for the development of new applications. However, it is more practical to transplant traditional OpenGL with C/C ++ in NDK. Most game porting projects need to combine the SDK and NDK components. For new projects, select OpenGL ES 2.0 instead of Version 1.1-unless your traditional OpenGL code is too old to use any GLSL shader code.

The second part of the series will discuss the obstacles to porting OpenGL games that you must understand before starting such a project, including differences in OpenGL extensions, floating point support, texture compression formats, and the GLU library. It also describes how to use OpenGL ES to set up an Android development system for the Intel Atom processor and how to obtain the optimal performance of the Android virtual device simulation tool.

About the author

Clay D. Montgomery is the main developer who develops OpenGL-oriented drivers and applications on embedded systems. He has worked as an independent consultant in the design of cross-platform Graphics Accelerator hardware, graphics drivers, APIs, and OpenGL applications in STB systems, VLSI technology, PHILIPS semiconductors, Nokia, Texas Instruments, and AMX. He has participated in the development of the first OpenGL ES, OpenVG * and SVG drivers and applications on the Freescale I. MX and TI OMAP * platforms and Vivante, AMD, and PowerVR. He developed OpenGL ES on Embedded Linux and opened seminars for professors. He is also a representative of many companies in Khronos Group.

More information

• Google OpenGL ES guide, "Show graphics cards with OpenGL ES": http://developer.android.com/training/graphics/opengl/index.html
• Vladimir Silva, professional Android games: http://www.apress.com/9781430226475
• Dan Ginsburg, "Finding 3D applications for mobile devices based on OpenGL and OpenGL ES": http://software.intel.com/en-us/articles/targeting-3d-applications-for-mobile-devices-powered-by-opengl-and-opengl-es
• Chris Pruett, "Android-oriented game development: Quick Start": http://android-developers.blogspot.com/2010/06/game-development-for-android-quick.html
• Tewdew Software, "Android texture decision": http://blog.tewdew.com/post/7362195285/the-android-texture-decision
• Jack Palevich, "porting GLES Quake-Quake to Android": http://grammerjack.blogspot.com/2009/10/gles-quake-port-of-quake-to-android.html
• Romain Guy and Chet Haase, "Android 4.0 graphics and Animation": http://android-developers.blogspot.com/2011/11/android-40-graphics-and-animations.html