Flixel lacks of rendering optimisations

[Sword352]

I'd love to see automatic rendering optimisations implemented into Flixel.
Many other game frameworks and/or engines often implements mechanisms such as multi-texture batching which greatly improves performance overall, and draw things with the least amount of draw calls possible.
Making things such as UI is often complicated with flixel due to the fact it doesn't have any of those. While it will be functional, it doesn't run great most of the time.

I've heard flixel has FlxAtlas to pack multiple graphics into a single one. Is it possible for this process to be automatic? This would be a great improvement to the framework

[Geokureli]

This is just a vague aimless rant, I want real specific targets and goals. You mention:

1. Automatically adding everything to an atlas

• but thats not a good idea, just manually add them when it benefits you

2. Putting ui in a sheet

• I was actually working on that now, via haxeui-flixel, but it's a long process with many steps involved

3. Multi-texture batching

• No idea on this one, never looked into it, not familiar with it. If it is achievable, elaborate

If you wanna make actual proposals, they will be appreciated! It's also really important that you start with actual problems, not theoretical problems.

Good example: This relatively simple code has perf issues when it (or similar systems) would perform well in other frameworks, the bottleneck seems to be in this code where it does [thing] when [better thing] is available

Bad Example: Doing this would probably be faster, though I have no actual case of having to work around the current system due to performance issues, or if I do, I didn't look into why it was happening and how that would perform in my vaguely proposed system

[Sword352]

Sorry if I was rude or violent, it wasn't my intention at all. I just started this issue so we could discuss about potential solutions, because I personally experienced flixel struggling to keep stability when rendering a lot of objects (even when they have the same graphics, due to ordering, and only way to solve this is to write spaghetti code). HaxeUI is also a good example, flixel doesn't keep up well with huge and complex UIs, and often causes a lot of GC pressure. Note that I'm not a professional on low level rendering, I just thought packing automatically would be a viable solution even though I do agree it's not the most efficient method

[Geokureli]

Sorry if I was rude or violent, it wasn't my intention at all.

There's no problem I'm just telling you what type of conversations are helpful

I personally experienced flixel struggling to keep stability when rendering a lot of objects

HaxeUI is also a good example, flixel doesn't keep up well with huge and complex UIs, and often causes a lot of GC pressure

isn't dynamic slower

Generalized stuff like this can be the basis that gets you started in looking for specific actionable changes and proposals, but on their own they just vague complaints

[CodeLazier]

This could refer to "Maturing the auto-batcher in the OpenGL renderer" in OpenFL9. I'm not very familiar with the rendering implementation in HaxeFlixel. Is it possible to take advantage of this rendering acceleration in OpenFL9?
https://community.openfl.org/t/development-update-openfl-9/12347

[Sword352]

I figured it would be better if I were to illustrate my ideas.

package;

import flixel.FlxG;
import flixel.FlxSprite;
import flixel.FlxState;
import flixel.group.FlxContainer;
import flixel.util.FlxColor;

class PlayState extends FlxState
{
	override public function create()
	{
		super.create();

		var graphic = FlxG.bitmap.add("flixel/images/logo/default.png");
		var y = 0.0;

		// var normalContainer:FlxContainer = cast add(new FlxContainer());
		// var redContainer:FlxContainer = cast add(new FlxContainer());

		while (y < FlxG.height)
		{
			var x = 0.0;
			while (x < FlxG.width)
			{
				var normalSprite = new FlxSprite(x, y, graphic);
				normalSprite.active = false;

				var redSprite = new FlxSprite(x, y, graphic);
				redSprite.color = FlxColor.RED;
				redSprite.active = false;

				// normalContainer.add(normalSprite);
				// redContainer.add(redSprite);

				add(normalSprite);
				add(redSprite);

				x += graphic.width;
			}
			y += graphic.height;
		}
	}

	override public function update(elapsed:Float)
	{
		super.update(elapsed);
	}
}

The sample above creates and adds bunch of sprites, with some of them being colored.
We can notice different behaviors when those sprites are added into different containers.
In the C++ target, when the colored and normal sprites shares the same container, the game has absymal performance. The framerate is extremely low and there's a lot of memory being allocated then cleaned by the GC afterwards.

In the same target, when the sprites are this time added into different containers, the game runs much better. The framerate is acceptable and stable, as well as memory.

We can notice how the DrawQuads amount is different between those 2 screenshots. They represent the actual amount of draw calls that has been operated (how many times we called drawQuads from OpenFL's graphics api, not the amount of time we called draw on sprites). With these informations, we can conclude that with the least amount of draw calls, the game operates much better.
Similar behaviors can be observed in HTML5 and Hashlink (haven't tried Flash though). While I tried this on debug, it doesn't change the fact that it doesn't give a good outlook at all, and the differences between using different containers or not aren't negligible.
While these behaviors differences might look normal, we could replace coloring with anything that breaks batching (such as using a different graphic), and the game would still run awful. Other high level game engines and frameworks implements mechanisms that actually prevents the game from running awful in these situations. Indeed, most of the time the framework/engine takes the care, it shouldn't be the role of the developer to adopt practices like packing graphics.
This is why it's hard to make anything complex with flixel, such as UI. These kind of practices (using different containers, graphic packing...) is extremely complicated in these contexts. Even in simpler contexts, they would result in spaghetti code.

[Geokureli]

It's great having concrete examples like this, and something to strive for, but what comes next? How would Flixel need to change in order to prevent this? I tried making the quad renderer batch unlike colored sprites by using colored vertices but either I did something wrong or theres some limitation I'm not aware of.

it shouldn't be the role of the developer to adopt practices like packing graphics

This is kind of a blanket statement that really ignores a lot of the nuance in the role of a dev. Sure, if we find a one-size-fits-all solution that doesn't have side effects that slow down other situations, we should absolutely work towards implementing it. But we shouldn't implement an ad-hoc fix in the framework for this specific issue assuming that this is how everyone is using Flixel, especially when it's far easier to make an ad-hoc fix for the project, like you just did.

It's Flixel's role to improve the framework but it will always be the devs role to work within the limitations given. We should never fully put that responsibility on one or the other

[Geokureli]

Related to: #3354
and cheems has a fix for this specific case, but I'm trying to learn more about this before I merge

[Sword352]

But we shouldn't implement an ad-hoc fix in the framework for this specific issue

Once again, this was just an example illustrating the underlying issue, I did say we could replace coloring with anything that breaks batching. I also don't see why mentioning #3354, it is not an issue entirely related to coloring.

it will always be the devs role to work within the limitations given

Maybe, but is it really a choice when a lot of other frameworks performs much better because they have things Flixel lacks? Even OpenFL attemps to resolves these issues by:

• Not redrawing DisplayObjects when it is not needed (although it might not have significant impacts in the case of games)
• Having multitexture batching implemented into the Tilemap API (for 9.5, iirc)

Not only that, but I did mention implementing these ad-hoc fixes is not very joyful during production because of how it either results in messy code, or is extremely complicated to do for more complex environments.

How would Flixel need to change in order to prevent this?

Some ideas I did find around (although this doesn't imply I know how they work):

• Occlusion culling, which consists of not rendering objects hidden by other objects that are on top of them. This would actually be an improvement to the sample I sent
• Multitexture batching, which consists of batching multiple objects even if they have different graphics. This would be a really major improvement to the framework for things like huge worlds or UI
• Maybe sorting the draw queue smartly before rendering so that objects with the same graphics and same "layer" batches? I have no idea if this is even possible nor if people already did that, just an idea I got on top of my head

[Geokureli]

• Occlusion culling, which consists of not rendering objects hidden by other objects that are on top of them. This would actually be an improvement to the sample I sent

This seems like an entirely separate feature, unrelated to the problem at hand

• Multitexture batching, which consists of batching multiple objects even if they have different graphics. This would be a really major improvement to the framework for things like huge worlds or UI

sure, but how?

• Maybe sorting the draw queue smartly before rendering so that objects with the same graphics and same "layer" batches? I have no idea if this is even possible nor if people already did that, just an idea I got on top of my head

This would affect the draw order of overlapping sprites, inadvertently, so I don't see it as a valid framework solution. If anything we could make some new container type that does this by default

Having multitexture batching implemented into the Tilemap API (for 9.5, iirc)

Are you saying this is something OpenFL has fixed in the next minor release, or something they've planned to implement in the future? Do you have a plan for what we would need to do to utilize it?

[Geokureli]

I feel like I'm still being misunderstood here. I'm down to discuss broad strokes and help find a plan of action, but if you would like me to actually implement these suggestions I need a concrete, low-level, nitty-gritty, proposal of exactly how I would do this, not an abstract idea, which is all I've seen in this thread. Even then I'll need plenty of spare time to implement it and I would probably need to learn a lot more about rendering.

If you want these things done faster, the best way is to just dive in and try making those changes yourself and report back with the results. For instance, if you think multi-texture batching is possible, implement it yourself in a branch and prove it. I lack the knowledge to implement that myself, I also failed to implement multi-color batching.

[Sword352]

This seems like an entirely separate feature, unrelated to the problem at hand

Maybe you're right, it's just an extra thing we could do as well.

sure, but how?

I have 2 potential solutions for this, although it still needs verification as I'm not too experienced in the domain of rendering. First solution would be to just pack graphics automatically at runtime, although I wonder how we could achieve this with an efficient manner. Perhaps this could maybe done at compile-time for already known graphics. Second solution, which I think OpenFL actually uses for it's tilemap multitexturing last time I've been told, is having a shader hold multiple sampler2D bitmap inputs and switch between them as needed during rendering. We can take a look here: https://github.com/openfl/openfl/tree/feature/tilemap-multitexture

This would affect the draw order of overlapping sprites

Not really. I did mention sprites with the same visual "layer".
I did a very simple sketch to illustrate it:

For example, the tree for this could be

- FlxSpriteContainer
  - FlxSprite (black)
  - FlxSprite (red)
- FlxSpriteContainer
  - FlxSprite (black)
  - FlxSprite (red)

but when rendering, we could render the 2 black sprites together, then the 2 red sprites, as doing so wouldn't break the visual order. Of course, the framework has to ensure it wouldn't cause visual disordering.

or something they've planned to implement in the future

Yes, it is an upcoming improvement for the next OpenFL minor version release as far as I know

Do you have a plan for what we would need to do to utilize it

I'm not too sure... I was just mentioning one of OpenFL's practice, I didn't think this API could be used in flixel, and my intention wasn't to make flixel use it either.

the best way is to just dive in and try making those changes yourself

Of course! I opened this issue so we could collectively think of solutions before jumping into it. I'm down to integrate some of these to flixel.