The root dir of our repository has grown quite a lot the past few months.
I'd like to make it more clear where the bulk of the engine lives (`src/`) and
also make it more clear which Mach libraries are consumable as standalone projects.
As for the name of this directory, `libs` was my first choice but there's a bit of
a convention of that being external libraries in Zig projects _today_, while these
are libraries maintained as part of Mach in this repository - not external ones.
We will name this directory `libs`, and if we have a need for external libraries
we will use `external` or `deps` for that directory name. I considered other names
such as `components`, `systems`, `modules` (which are bad as they overlap with
major ECS / engine concepts), and it seems likely the official Zig package manager
will break the convention of using a `libs` dir anyway.
Performed via:
```sh
mkdir libs/
git mv freetype libs/
git mv basisu libs/
git mv gamemode libs/
git mv glfw libs/
git mv gpu libs/
git mv gpu-dawn libs/
git mv sysaudio libs/
git mv sysjs libs/
git mv ecs libs/
```
git-subtree-dir: glfw
git-subtree-mainline: 0d5b853443
git-subtree-split: 572d1144f11b353abdb64fff828b25a4f0fbb7ca
Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
git mv ecs libs/
Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
This change makes the the Iterator generic over the components being
queried. This restores the api to the previous style with components
given to the query() function and next() having no parameters (other
that the iterator itself).
The reference to the old archetype is invalidated by getOrPut() calls of
std.ArrayHashMap. The implementation of std.ArrayHashMap means that
pointers can be invalidated on getOrPut() calls even if the key exists
in the map. This means that the reference to the old archetype needs to
be refreshed unconditionally (i.e. not only if the new archetype didn't
exist previously).
This switches our ECS over to manually managed memory (1 `[]u8` per archetype table,
with multiple column arrays packed into it - dealing with padding/alignment ourselves)
rather than the prior 1 `ArrayList(Component)` per component in an archetype table.
This idea was discussed in depth in [#ecs:matrix.org](https://matrix.to/#/#ecs:matrix.org)
(thanks Levy!) Notable advantages from my POV:
1. This means we don't need an `ErasedComponentStorage` interface, which is nice.
2. It means component storage does not have to have a Zig type. It could e.g. in theory enable
the ECS to be usable from other languages (C, WebAssembly plugins, etc.) with component types
defined in those languages in the future.
3. It reduces some overhead `ArrayList` has: slice ptr+len+capacity integers per component
array per table
4. It guarantees component arrays are contiguous memory, rather than relying on the allocator
to hopefully provide that (may not hold true in multi-threaded large-allocation situations.)
5. It means we could easily optimize for tables that have very few components by allocating exact
memory for them (could've done this with `ArrayList` too, but now it's more likely the
allocation are larger and thus more reusable by future archetype tables.) This could be quite
important because one can imagine ending up with many small archetype tables.
Overall seems like the right thing to do, so we're doing it.
Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
In the past:
* hexops/mach#156 was the initial ECS implementation detailed in https://devlog.hexops.com/2022/lets-build-ecs-part-1
* hexops/mach#157 was the second major redesign in which we:
* Eliminated major limitations (e.g. inability to add/remove components at runtime)
* Investigated sparse sets
* Began thinking in terms of databases
* Enabled runtime introspection
Our second revision of the ECS, however, still had _archetypes_ exposed as a public-facing
user concern. When a new component was added to an entity, say a weapon, the table storing
entities of that archetype changed to effectively have a new column `?Weapon` with a null
value for _all existing entities of that archetype_. We can say that our ECS had archetypes
as a user-facing concern AND this made performance worse: when iterating all entities with
a weapon, we needed to check if the component value was `null` or not because every column
was `?Weapon` instead of a guaranteed non-null value like `Weapon`. This was a key learning
that I got from [discussing ECS tradeoffs with the Bevy team](https://github.com/hexops/mach/pull/157#issuecomment-1022916117).
This third revision of our ECS has some big benefits:
* Entities are now just IDs proper, you can add/remove arbitrary components at runtime.
* You don't have an "entity which always belongs to one archetype table which changes"
* Rather, you have an "entity of one archetype" and adding a component means that entity _moves_ from one archetype table to another.
* Archetypes are now an implementation detail, not something you worry about as a consumer of the API.
* Performance
* We benefit from the fact that we no longer need check if a component on an entity is `null` or not.
* Introspection
* Previously iterating the component names/values an entity had was not possible, now it is.
* Querying & multi-threading
* Very very early stages into this, but we now have a general plan for how querying will work and multi-threading.
* Effectively, it will look much like interfacing with a database: you have a connection (we call it an adapter)
and you can ask for information through that. More work to be done here.
* Systems, we now have a (very) basic starting point for how systems will work.
Some examples of how the API looks today:
* 979240135b/ecs/src/main.zig (L49)
* 979240135b/ecs/src/entities.zig (L625-L656)
Much more work to do, I will do a blog post detailing this step-by-step first though.
Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
