Add more control over resources loading

[arthuro555]

Description

When GDevelop loads up, it takes much time preloading everything. Sometimes though, you may want to have a game that loads fast initially and then have some loading screens to load the missing stuff.

Solution suggested

Instead of forcing preloading resources to the user, we could take a similar approach to what was done in #2006 :

1. Add a property on the resources to choose if it should be preloaded or not
2. Add actions to preload and unload resources
3. Modify the loaded resources getters to load the resource if it hasn't been loaded yet or has been unloaded

That way, the game never breaks, but it gives a way for users to control what is loaded or not to have a better memory management, and to make their own "loading screens" by initially loading only the loading screen assets so that the gdevelop splash screen can be hidden as it is too fast to see anyways, and using a GDevelop scene using the actions to load what they need and display the on screen feedback they want in the meantime.

[4ian]

In short: yes! :)

That's definitely something I want to have at some point: be able to disable the initial loading of some/almost all resources, then do the loading whenever it's needed (and potentially also have a way to unload, or at least specify that something can be unloaded).
See this card: https://trello.com/c/TjYFGhQV/302-improve-loading-by-only-loading-resources-of-the-first-scene-at-startup-and-load-the-rest-when-needed

Modify the loaded resources getters to load the resource if it hasn't been loaded yet or has been unloaded

This may create some weird behavior in the sense that for example, if you set a sprite image to be not loaded at the beginning, then it should be loaded whenever an instance of an object that use it is created in the runtime scene.
The problem is that loading a resource is not something that is immediate: it will have at least a few frames during which the image is not loaded. What should we do at this moment:

• Block the game while the resource is loaded? (if you don't want to block the game, then take care of preloading it in a scene before using the action)
• Don't block the game, so that the object won't show anything until the image is loaded at which point it will "pop" on the screen. This is what we do for the scene editor... but it's because the scene editor, we don't care about this effect and value the simplicity first. Not sure for a game it's a good idea because this means for example that while the image is loading, the object width and height will be 0... can be a problem for collisions and even for events/logic of the game.

unload resources

If you want to investigate a bit, it would be interesting to see what Pixi.js offers for this. Could be a good idea to have a way to monitor the usage in terms of memory/GPU memory too. That's something we could maybe expose using some Electron apis (at least just for developers) (I was also thinking of having a dialog in the editor that would also show the usage in terms of memory, to see how well the editor handles big games)

[arthuro555]

I am also thinking about some methods to make loading easier:

1. We could introduce resource groups. Resource groups can be used to group resources, and the user can simply tell load resources "resource group" and tell to unload them. For pixi this is very simple to do, we can simply use a separate loader for every group and call loader.destroy(), if I understood correctly this will destroy all webgl context of the textures, and remove them from the textures cache. All that would be left to do is removing them from the image manager so that they can be garbage collected. There would be a default resource group "all resources" and one per scene that contains resources used by scenes so that users can make their own loading screen by disabling the default one and loading all resources in their scene or loading the scene resources before switching to any scene. The groups that contain multiple resources would be internally split in two groups, so that only resources of one type are grouped together to make loading of a resources group easier.
2. Add a way to load a scene in the background (make the loading of scene async and load it's used resources asynchronously), with callback variables so that user can make a simple loading scene (load the scene in the background, get percentage done via callback variables, and once the callback variables says it is done switch to the newly loaded scene).

Both of those methods would have a callback variable with the amount of things to load, the amount of things loaded and if it finished loading.

The problem is that loading a resource is not something that is immediate: it will have at least a few frames during which the image is not loaded. What should we do at this moment:

Imo we should do both. For that, we can either use a project property or an action "block until all resources are loaded" or a condition "all resources loaded". The default should be not blocking though, as it is up to the user to implement a loading screen to hide the loading process I think.

the object width and height will be 0... can be a problem for collisions and even for events/logic of the game.

We could pregenerate a transparent image of the size of the image to load to put there in the meantime maybe?

[4ian]

We could introduce resource groups

Speaking of this, there is already a notion of resource groups/resource folders in the codebase, which is not exposed in the editor. Though it requires a lot of painful manual work to organize your resources. Might be better to focus on loading resources for a scene - or autogenerate groups for a scene :)

get percentage done via callback variables, and once the callback variables says it is done switch to the newly loaded scene)

We might not need "callback" variables, just an expression giving the percentage of the progress of the currently loaded resources, and a condition checking if the resources have just finished loading.

We could pregenerate a transparent image of the size of the image to load to put there in the meantime maybe?

I would avoid this because of confusion that this can make ("why is my object not showing", "actually it is showing", "but the animation is messed up because it started not at the right time because it was loading").

Maybe to get things rolling on this, we could start with an experiment:

• See how calling destroy on the Pixi loader impacts a running game. We could imagine using one loader per scene, so that when we switch scene we destroy the old one and load the "resource group" of the next scene with a new loader (see the discussion here: https://www.html5gamedevs.com/topic/42153-pixiloader-how-to-clear-resources-up/).
• See how calling destroy on a texture individually works.

I think overall the biggest challenge is to do the work of identifying which resources are used in the scenes. Fortunately, it should be doable with gd::Project::ExposeResources method (we should probably make one for each scene), though it will need a bit of C++ and testing to make it correct. We can then persist in the exported/preview game file the list of resources of each scene.

Then, we it should be fairly easy when loading a new scene/unloading a scene/changing to a new scene to compute the difference (the set of new resources minus the set of old resources), and load the new resources/unload the ones that are now useless :)

If you want to experiment but don't feel like doing the ExposeResources part, just ignore this and I'll do this later myself.
It would be good to check how the Pixi texture cache reacts when we call destroy - to verify things are unloaded. Also all of this discussion is about images. For audio, we would need probably your refactoring PR to go ahead first. For JSON, it's easy to just drop the reference if asked to unload.

[VanCastar]

Anything happening around this feature request lately?

[Bouh]

The team is working on a better way to load the assets from the scenes.
Doing this first allow a better way to manage resources, and may open the door to a new and finest actions to manages resources.

#5572

[2655251820]

I have to say that this is a very important feature, for open large maps yes, I have always wanted to do Ori and the Blind Forest like the game, I learned the documentation, ready the material, only to find that the engine can not support dynamic loading, I am really disappointed, after a long time to prepare it is like this,I really like this engine and hope to make this feature soon

[panteliskassotis]

Just my two cents here. In Unreal's World Partition feature the scene is split in cubes that all load or unload their contents depending on the player's position. 4ian proposed we can access our game's resources and command them each to load or unload at will.

What if we placed invisible square sprites inside each scene that would resemble this feature and gave each resource a command to load or unload when the main player touches their invisible square? You could then resize the sprites manually based on your preference to give the engine enough room to load the parts of the scene you'd want without the player noticing the assets popping awkwardly in to existence. It's a crude solution I know but maybe it could work?

Down the road this feature could be a resizable square grid (which GDevelop already kinda has inside the levels of the external layouts) where each square allows us to write the command:

"When player is inside this particular square(insert square size,position to identify the square you want) ---> Load x,y,z resources"

and

"When player stops being inside ---> Unload x,y,z resources".

I know it's a pain and it requires getting inside the guts of the engine but I believe it's worth a shot. Allowing users to create large and complicated levels is the key to success for a product like this (especially in 3D which is already overcrowded with high-end solutions) and I'm surprised this thread dates back to 2020 with no concrete update yet.

[4ian]

This thread is indeed old and missing part of the big picture and the progress made since then.

Progress made since the thread:

• Resources are now downloaded/loaded progressively. Only the first scene resources are needed first (first loader), which means that if the first scene is very lightweight, it can be shown almost instantly.
• Resources needed by other scenes are then downloaded/loaded in the background, in the order of the scenes.
• This allows to also create scenes that act as loaders. See this documentation page for a bit more information: https://wiki.gdevelop.io/gdevelop5/all-features/resources-loading/#optimize-resource-loading.

The big picture is that there are three things you want for a "large world" game:

1. resource loading/unloading (to keep "memory" and VRAM usage low)
2. rendering only a portion of the level (to keep GPU fast)
3. physics/logic only on a portion of the level (to keep CPU fast)

Point 3, physics/logic only on a portion of the level is typically handled by the physics engine ("Box2D" for 2D, or the upcoming 3D physics Engine, "Jolt Physics", are both made to scale to very large world thanks to spatial data structures). For a large world 3D game, using the 3D Physics Engine Jolt or the 2D Physics engine will be way way way more scalable than using GDevelop basic collision handling.
For this point, it's also important to make sure events are only applied to a portion of the game level, which could be partially automated by GDevelop or by conditions that would allow to filter objects to only use the ones "near the player".

Point 2 is slightly related to point 3. While the 2D and 3D rendering do "culling" to avoid asking the GPU to render things outside of the view, the engine still needs to do this "culling". It could be made faster by using information from the physics engine, which does this in a scalable way thanks to spatial data structures.

Point 1 could be divided in two things:

• First approach: in addition to automated, progressive loading of resources in background, we could also add options to entirely prevent loading certain scene resources, or unload certain scene resources. This means that your game could scale as long as different scenes are using different resources.
• Second approach: something more akin to "world streaming" or other fancy words to say that even in a scene, resources can be unloaded or loaded according to what is needed. This is more complex because for now objects assume that all their resources are available immediately in a scene. The scene are for now the boundaries.

tl;dr: Progressive loading is already a huge improvement for web games (slashing first loading time and so reducing new player bounce rate). We could go further by preventing resource load of subsequent scenes until needed.
Anything more involved would probably involve using information from the 2D/3D physics engine, or another spatial data structure, to reduce rendering and logic time, and finally make resource loading/unloading adapted to this (but needs some rework on how objects interact with their resources).

[4ian]

Note that my answer is maybe missing the most important point for you: it's highly unlikely that low poly 3D models of your game (as seens in the other screenshot) is what is making your game performance poor - they will fit easily in your GPU memory. Solving this will probably solve nothing.

It's much more likely the events are trying to handle too many instances (collision between hundreds of instances, iteration/distance check on hundreds of instances at each frame, update of thousands of static instances, etc...)
You could give a try by chunking your game into multiple external layouts and create only one: the game should perform well, indicating it's not that much about resources but about the game doing too much work for the logic (events, behaviors).

[panteliskassotis]

We need a feature that allows us to resize this grid and identifies the resources inside each square to help with the load/unload command.