Klayge resource Loading System

Reprinted please indicate the source for the klayge game engine, the permanent link of this article is http://www.klayge.org /? P = 2336

A year and a half ago, when developing klayge 4.0, Guo Peng designed a resource loading system to solve asynchronous loading and repeated loading problems. However, I just implemented a rough asynchronous load. As the complexity of scenarios increases, a perfect resource loading system becomes necessary. Following the previous design ideas, I have finally completed the implementation.

Target

The resource Loading System of klayge has several design requirements:

1. Supports at least five types of resources: texture, model, effect, Font, and post processor, and supports rapid scaling of new resources.
2. You can select synchronous or asynchronous loading.
3. Do not load resources repeatedly to reduce waste.
4. Automatic Management. You do not need to manually specify the resource ID and try not to explicitly Delete the resource.
5. Resources do not require a single inheritance.
6. Asynchronous load without blocking.

Previously, resloader supported 2, 4, and 5, partially supported 1, and 3 in a bad way, not 6 at all. The new resloader finally successfully solved these problems.

Compared with traditional Resource Manager

First, make a horizontal comparison, compared with the traditional resource manager that is common in other game engines. Here, t indicates the type of the resource pointer, and resmgr indicates the resource manager.

	Traditional Resource Manager	Klayge resource Loading System
Resource inheritance	Different types of resources must be inherited from the Resource class.	Each resource is completely independent and does not require any inheritance.
Resource Manager inheritance	Write different managers based on different resources, such as shadermanager and texturemanager. These managers also inherit from a ResourceManager class	A unified resource loader does not require any inheritance.
Asynchronous loading	Int id = resmgr. asyncquery (res_name ,...);	Function <t ()> RL = resloader. asyncquery (res_name ,...);
Obtain the loaded asynchronous Resource	T p = resmgr. getres (ID );	T p = RL (); If the resource has not been loaded, null is returned.
Uninstall Resources	Resmgr. Unload (ID );	P. Reset ();

It can be seen from this that apart from inheritance, there is also a major difference that the operations such as resource acquisition of traditional resource managers need to deal with the global resource manager itself. In the resource Loading System of klayge, these operations are only partial and do not need to interact with resloader.

Compared with earlier versions

Vertical ratio after horizontal comparison. The old version of resloader does not check the repeated loading of resources. If texture, model, and postprocessor are loaded twice, the two resources are retained in the memory. Only texture of the same model can be loaded without repeated loading. While effect and font perform repeated detection in renderfactory. In the new version, these are all concentrated in resloader, which does not differentiate resource types and are managed in the same way.

In earlier versions, asynchronous loading is blocked. If you try to obtain a resource pointer that has not been completed, the system will block it and return it until the loading is complete. In addition, the old version will allocate a thread (through the Global thread pool) for each loaded resource, and the new version will only allocate one thread to complete the entire resource loading queue.

Method

One of the keys to achieving these requirements lies in the Resource description. The reason for a single inheritance of resources is that the resource manager needs to call this base class to know how to read and write files and how to put data into resources. However, what is really needed in the resource loading process isResource DescriptionTo describe the resource loading process. The coupling between the resource description and the resource is very weak and only exists in the loading stage. After loading, You can discard the resource description. If it is a traditional inheritance structure, this coupling cannot be eliminated.

With the Resource Description, You need to design the Resource Description Interface in detail based on the function. Resource loading can be divided into two phases.The first is the sub-thread-independent phase.Mainly file read/write and format parsing.The second step must be in the main thread and device-related phase.It is mainly used to call a specific API to create resources. At the same time, because the Resource Description of the same resource is independently created, the resource description also needs to compare whether two resource descriptions represent the same resource. Otherwise, repeated loading cannot be avoided. This is generally compared by the resource type, Resource Name, and created parameters.

If you find that the resource to be loaded has been loaded, you need to reuse the resource. Note that resources should be dividedStateless and stateful. Stateless resources, such as texture and font, are identical for each load. Resource pointers can be reused directly.Copydata. Stateful resources, such as effect and postprocess, are modified during use. Therefore, if resource pointers are reused, the two resources cannot work independently. Therefore, only the first stage of data can be reusedCloneresourceYou also need to execute the second stage separately to establish resources. It should be noted that the model is also stateful, because the model contains an effect, but the model's Vb/IB can be reused, so it should be differentiated when cloneresource is used.

For different types of resources, you need to derive an independent resource description from the base class of the Resource Description to implement the features described above. Resources are not limited to modification. With the help of Resource Description, resloader can manage different types of resources in the same way, and resources do not need to involve the inheritance system. To expand to a new resource, you only need to write a new resource description. The coupling degree is very low. The resloader itself contains two queues,LoadedThe queue stores the loaded resources,LoadingThe queue stores the resources being loaded. After a resource is loaded, resloader moves the resource description from the loading queue to the loaded queue and retains the weak_ptr of a resource. by regularly checking this weak pointer, resloader can know whether the resource has been deleted. If yes, it can also delete the resource from the loaded queue of resloader. At the same time, resloader also has the same unload function as the traditional one, which is used to force detach a specific resource.

The state machine for loading resources is like this. Synchronous loading is nothing to say. Here we will look at asynchronous loading.

 

Note that the reason for repeated check in the loaded queue is that if an asynchronous resource is executed synchronously when it is waiting for the sub-thread to execute, it will appear in the loaded queue.

Summary

With this new resource loading system, resource waste has been eliminated, and the resource loading speed has also improved. For example, in the deferred rendering example, the memory consumption is reduced by 10%, and the loading speed is increased by 20%. In addition, since the entire system is not congested, the rendering thread will continue to execute during the loading process, so we can see the loading of the scenario.