(One of a series of posts about converting Unity3D into a 1st-class Entity Component System. Read others here)
How to store game data: GameObject? MonoBehaviour? ScriptableObject?
Those three pillars of Unity’s architecture are the curse of game implementation and performance. Architecturally, they’re great: they’re a big part of what makes Unity easy for newcomers to pick up and understand within hours, instead of taking weeks.
On large Unity game projects, they’re hated for being slow and hard to work with when you get to hundreds of thousands (or millions, tens of millions, …) in a single scene. Unity provides zero tools and support for this situation – it just gets harder and harder to work with, and slower and slower to run.
In Entity Systems world, we hate them because they are inherently slow and buggy – no matter how great Unity’s coders, they’ll never make them work as well as our preferred approach (See below).
But … replacing them forces you to replace most of the Unity Editor!
Most of the features of our Entity System need new Editor GUI anyway. If we’re going to be adding big chunks to the editor, replacing GO/MB/SO is going to be much cheaper than normal. And it potentially has huge benefits that are independent of the Entity System.
If we’re going to replace them, what will we replace them with? Custom versions that have almost the same name and work the same way? Or something very different?
What is Unity’s ‘GameObject’ anyway?
As far as I can tell, the real meaning of GameObject is:
def. GameObject: Something that exists inside a Scene, and has a Position, Rotation, and Scale. It can be “embedded” inside any other GameObject, meaning that any change to the other object’s Position, Rotation, or Scale gets merged onto this one. It’s the ONLY thing that can have Unity “Components” (MonoBehaviours) attached to it. Anything that is not a GameObject, and is not attached to a GO, gets deleted every time you load/reload/play/resume a Scene.
From a code perspective, looking at the backend systems, there’s a lot more to it. The GameObject (GO) has a slightly weird name, when you think about it. You use it so much during Unity dev that you stop noticing it. “Game … Object” … wha?
Other purposes GameObject serves
Here’s a few obvious ones (I’ll edit later if I think of more):
- Guarantees everything has the GetInstanceID() method
- Needed for Unity’s Serialization system to work (because of how it’s been designed, not because serializing requires this – C# has built-ins that would have done the same job)
- Enables the Editor to “select” anything in any tab/window, and that thing is ALSO selected in all other tabs/windows (e.g. click in Scene, and the Inspector updates to show the components on that thing)
- …probably loads of other things. It’s so commonly used I hardly notice it
- When doing in-Editor things, Unity can often “ignore” classes that are “not part of the game, because they don’t extend GameObject”
- Reading between the lines of official docs, and simplifying vastly, this is what happens behind the scenes. Especially with Serialization, I suspect.
- This can be a huge pain. It’s hard to debug code when “GameObject subclasses” magically behave differently to “all other classes” with no indication of why
- Since every GO has a Transform … you can do automatic, general-purpose, high-quality, Occlusion Culling
- NB: for this to work perfectly, you’d also want every GameObject to have a .bounds variable. Why did Unity’s original architects leave that out? Dunno, sorry.
How to store game-data: the Entity Systems ideal way
We know this, it’s been discussed to death: if your programming language supports it, use structs.
By definition, structs are the most memory-efficient, CPU-efficient, multi-thread-safe, typesafe way of storing data, bar none. There is no better way to do this in mainstream languages. C# and C++ both support structs; I think we have a winner!
There are probably some awesomely clever advanced Math ways of doing things better, or by using raw assembler and code optimized for each individual CPU on the market. Or, or … but: in terms of what’s ‘normal’, and commonly used, this is the best you’re going to get
But how will this work in practice? What about all the “other” roles of GO etc in Unity?
Structs in … Identity (GetInstanceID)
Simple ECS implementations often hand-around raw pointers to their Components; that gets messy with Identity.
But most (all?) advanced implementations have some kind of indirection – a smart pointer, or an integer “index” that the Component Storage / Manager maps internally to the actual point in memory where that lives.
If we turn that “advanced” feature into a requirement, we should be fine.
Structs in … Serialization
- Start writing your own version of the Unity Serialization system that only uses structs
- Finish it the same day
- Discover it runs 10 times faster than Unity’s built-in one
- …with fewer bugs
- PROFIT!
Structs are the easiest thing you could possibly try to serialize and deserialize. Unity coders probably wish that they could restrict us all to structs – it would make their live much easier in some ways.
User-written structs: special rules?
Do we need to add special “rules” to user-written structs that appear in the game?
Nope! It turns out that C#’s built-in Reflection system is able to inspect every struct, shows us every field/property/etc, and lets us easily read and write to them. Both private and public.
CAVEAT: structs break C# .SetValue()
Yeah, this sucks – and it’s nothing to do with Unity, it’s core C#.
A little care is needed when we implement the Entity System internals. But it has no effect on user code / game code.
Structs in … SceneView (Transforms)
No! Just … NO!
This is one of the architectural mistakes of Unity we’re going to fix in passing: we won’t require everything to have a Transform.
However, it foreshadows a more subtle problem that we’ll eventually have to deal with:
Entity Systems are a table-based/Relational/RDBMS way to store and access data; such systems are very inefficient at storing and retrieving tree-structured data (e.g. OOP classes + objects)
The transform hierarchy in a Unity scene is a massive tree. As it turns out, it’s bigger than it needs to be (because of that law of Unity that everything must have a Transform) – but we uses trees all the damn time in game-development. So it’s a problem we’ll have to come back to.
Bigger storage: where do the structs live?
Fine; at the lowest level, the individual ECS Components, we’ll use structs.
But how do we store those?
Choosing the right aggregate data-structures for your ECS is a major topic in itself. C# gives us (almost) total control of how we’d like to do it – plenty of rope to hang ourselves with.
At this point: I’m not sure.
As a temporary solution, I’m going to focus on making the storage-system swappable, so that I can implement a crappy version quickly, and then easily swap-it-out with something much better – without having to rewrite anything else.
Does it work?
Yes. I tried it – I wrote a complete replacement for the Unity Inspector tab that only works on Entities-with-Components, where “Component” is any struct. Ugly, but it works:
Everything there is auto-generated (including the colour; every struct gets a globally unique background colour, which I find makes it much easier when you have hundreds or thousands of Component types).
The only tricky bit was the problem with C# FieldInfo.SetValue(…), as mentioned above.
Further Reading
…some things I might want to pick up on in future posts: