zig/mach
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

ecs: major rethink & database-aligned design

Browse source 
:: Limitations of our ECS

Previously, we had thought about our ECS in terms of archetypes defined at compile time (effectively arrays
of archetype structs with comptime defined fields as components.) I believe that this is likely *the most
efficient* way that one could ever represent entities. However, it comes with many limitations, namely that:

You have to define which components your entity will have _at compile time_: with our implementation,
adding/removing components to an entity at runtime was not possible (although declaring components at comptime
that had optional _values_ at runtime was). This is contradictory with some goals that we have:

* The ability to add/remove components at runtime:
    * In an editor for the game engine, e.g. adding a Physics component or similar to see how it behaves.
    * In a code file as part of Zig hot code swapping in the future, adding an arbitrary component to an entity
      while your game is running.
    * In more obscure cases: adding components at runtime as part of loading a config file, in response to network
      operations, etc.

:: Investigating sparse sets

To find the best way to solve this, I did begin to investigate sparse sets which I saw mentioned in various contexts
with ECS implementations. My understanding is that many ECS implementations utilize sparse sets to store a relation
between an entity ID and the dense arrays of components associated with it. My understanding is that sparse sets
often imply storing components as distinct dense arrays (e.g. an array of physics component values, an array of weapon
component values, etc.) and then using the sparse set to map entity IDs -> indexes within those dense component arrays,
`weapon_components[weapons_sparse_set[entityID]]` is effectively used to lookup an entity's weapon component value,
because not every entity is guaranteed to have the same components and so `weapon_components[entityID]` is not possible.

This of course introduces overhead, not only due to two arrays needed to lookup a component's value, but also because
you may now be accessing `weapon_components` values non-sequentially which can easily introduce CPU cache misses. And
so I began to think about how to reconcile the comptime-component-definition archetype approach I had written before
and this sparse set approach that seems to be popular among other ECS implementations.

:: Thinking in terms of databases

What helped me was thinking about an ECS in terms of databases, where tables represent a rather arbitrary "type" of
entity, rows represent entities (of that type) themselves, and the columns represent component values. This makes a lot
of sense to me, and can be implemented at runtime easily to allow adding/removing "columns" (components) to an entity.

The drawback of this database model made the benefit of sparse sets obvious: If I have a table representing monster
entities, and add a Weapon component to one monster - every monster must now pay the cost of storing such a component
as we've introduced a column, whether they intend to store a value there or not. In this context, having a way to
separately store components and associate them with an entity via a sparse set is nice: you pay a bit more to iterate
over such components (because they are not stored as dense arrays), but you only pay the cost of storing them for
entities that actually intend to use them. In fact, iteration could be faster due to not having to skip over "empty"
column values.

So this was the approach I implemented here:

* `Entities` is a database of tables.
    * It's a hashmap of table names (entity type names) to tables (`EntityTypeStorage`).
    * An "entity type" is some arbitrary type of entity _likely to have the same components_. It's optimized for that.
      But unlike an "archetype", adding/removing ocmponents does not change the type - it just adds/removes a new column
      (array) of data.
    * You would use just one set of these for any entities that would pass through the same system. e.g. one of these
      for all 3D objects, one for all 2D objects, one for UI components. Or one for all three.
* `EntityTypeStorage` is a table, whose rows are entities and columns are components.
    * It's a hashmap of component names -> `ComponentStorage(T)`
    * Adding/removing a component is as simple as adding/removing a hashmap entry.
* `ComponentStorage(T)` is one of two things:
    * (default) a dense array of component values, making it quite optimal for iterating over.
    * (optional) a sparsely stored map of (row ID) -> (component value).
* `EntityID` thus becomes a simple 32-bit row ID + a 16-bit table ID, and it's globally unique within a set of `Entities`.
    * Also enables O(1) entity ID lookups, effectively `entities.tables[tableID].rows[rowID]`

:: Benefits

::: Faster "give me all entities with components (T, U, V) queries"

One nice thing about this approach is that to answer a query like "give me all entities with a 'weapon' component", we can
reduce the search space dramatically right off the bat due to the entity types: an `EntityTypeStorage` has fast access to
the set of components all entities within it may have set. Now, not all of them will have such a component, but _most of
them will_. We just "know" that without doing any computations, our data is structured to hint this to us. And this makes
sense logically, because most entities are similar: buttons, ogre monsters, players, etc. are often minor variations of
something, not a truly unique type of entity with 100% random components.

::: Shared component values

In addition to having sparse storage for `entity ID -> component value` relations, we can _also_ offer a third type of
storage: shared storage. Because we allow the user to arbitrarily define entity types, we can offer to store components
at the entity type (table) level: pay to store the component only once, not per-entity. This seems quite useful (and perhaps
even unique to our ECS? I'd be curious to hear if others offer this!)

For example, if you want to have all entities of type "monster" share the same `Renderer` component value for example,
we simply elevate the storage of that component value to the `EntityTypeStorage` / as part of the table itself, not as a column
or sparse relation. This is a mere `component name -> component value` map. There is no `entity ID -> component value`
relationship involved here, we just "know" that every entity of the "monster" entity type has that component value.

::: Runtime/editor introspection

This is not a benefit of thinking in terms of databases, but this implementation opens the possibility for runtime (future editor)
manipulation & introspection:

* Adding/removing components to an entity at runtime
* Iterating all entity types within a world
    * Iterating all entities of a given type
        * Iterating all possibly-stored components for entities of this type
        * Iterating all entities of this type
            * Iterating all components of this entity (future)
* Converting from sparse -> dense storage at runtime

:: A note about Bevy/EnTT

After writing this, and the above commit message, I got curious how Bevy/EnTT handle this. Do they do something similar?

I found [Bevy has hybrid component storage (pick between dense and sparse)](https://bevyengine.org/news/bevy-0-5/#hybrid-component-storage-the-solution)
which appears to be more clearly specified in [this linked PR](https://github.com/bevyengine/bevy/pull/1525) which also indicates:

> hecs, legion, flec, and Unity DOTS are all "archetypal ecs-es".
> Shipyard and EnTT are "sparse set ecs-es".

:: Is our archetypal memory layout better than other ECS implementations?

One notable difference is that Bevy states about Archetypal ECS:

> Comes at the cost of more expensive add/remove operations for an Entity's components, because all components need
> to be copied to the new archetype's "table"

I've seen this stated elsewhere, outside of Bevy, too. I've had folks tell me that archetypal ECS implementations
use an AoS memory layout in order to make iteration faster (where `A`, `B`, and `C` are component values):

```
ABCABCABCABC
```

I have no doubt a sparse set is worse for iteration, as it involves accessing non-sequentially into the underlying dense
arrays of the sparse set (from what I understand.) However, I find the archetypal storage pattern most have settled on
(AoS memory layout) to be a strange choice. The other choice is an SoA memory layout:

```
AAAA
BBBB
CCCC
```

My understanding from data oriented design (primarily from Andrew Kelley's talk) is that due to struct padding and alignment
SoA is in fact better as it reduces the size of data (up to nearly half, IIRC) and that ensures more actually ends up in CPU
cache despite accessing distinct arrays (which apparently CPUs are quite efficient at.)

Obviously, I have no benchmarks, and so making such a claim is super naive. However, if true, it means that our memory layout
is not just more CPU cache efficient but also largely eliminates the typically increased cost of adding/removing components
with archetypal storage: others pay to copy every single entity when adding/removing a component, we don't. We only pay to
allocate space for the new component. We don't pay to copy anything. Of course, in our case adding/removing a component to
sparse storage is still cheaper: effectively a hashmap insert for affected entities only, rather than allocating an entire
array of size `len(entities)`.

An additional advantage of this, is that even when iterating over every entity your intent is often not to access every component.
For example, a physics system may access multiple components but will not be interested in rendering/game-logic components and
those will "push" data we care about out of the limited cache space.

:: Future

Major things still not implemented here include:

* Multi-threading
* Querying, iterating
* "Indexes"
    * Graph relations index: e.g. parent-child entity relations for a DOM / UI / scene graph.
    * Spatial index: "give me all entities within 5 units distance from (x, y, z)"
    * Generic index: "give me all entities where arbitraryFunction(e) returns true"

Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
This commit is contained in:
Stephen Gutekanst 2022-01-26 18:28:10 -07:00 • committed by Stephen Gutekanst
parent a75b599279
commit c2c4335ff2
2 changed files with 501 additions and 282 deletions
Download patch file Download diff file Expand all files Collapse all files

22
ecs/README.md
View file

@ -27,23 +27,17 @@ There are plenty of known issues, and things that just aren't implemented yet. A
* Currently only handles entity management, no world management or scheduling. No global data, etc.
* Lack of API documentation (see "example" test)
* Missing hooks that would enable visualizing memory usage, # of entities, components, etc. and otherwise enable integration of editors/visualizers/profilers/etc.
* TypedEntities does not allow for storage of sparse data yet, and also specifically comptime defined sparse data - both are needed. e.g. if only a handful of entities need a component but there are hundreds of thousands.
* If many entities are deleted, TypedEntities iteration becomes slower due to needing to skip over entities in the free_slots set, we should add a .compact() method that allows for remediating this as well as exposing .
* If *tons* of entities are deleted, even with .compact(), memory would not be free'd / returned to the OS by the underlying MultiArrayList. We could add a .compactAndFree() method to correct this.
* We have dense and sparse data, but no shared data yet.
* If many entities are deleted, iteration becomes slower due to needing to skip over entities in the free_slots set, we should add a .compact() method that allows for remediating this.
* If *tons* of entities are deleted, even with .compact(), memory would not be free'd / returned to the OS by the underlying components arrays. We could add a .compactAndFree() method to correct this.
* It would be nicer if there were configuration options for performing .compactAndFree() automatically, e.g. if the number of free entity slots is particularly high or something.
* Currently we do not expose an API for pre-allocating entities (i.e. allocating capacity up front) but that's very important for perf and memory usage in the real world.
* When TypedEntities is deinit'd, entity Archetypes - or maybe systems via an event/callback, need a way to be notified of destruction.
* Entities.get() currently operates on the archetype type name, but we should perhaps enable also getting a TypedEntities instance via passing a unique string name. e.g. if you want to store two separate team's Player entities in distinct TypedEntities collections.
* There should exist a Merge/Combine function for composing Archetypes from components and other Archetypes without explicitly listing every single component out in a new struct.
* When entity is deleted, maybe via systems / an event/callback, need a way to be notified of destruction. Same with updated maybe.
See also the numerous TODOs in main.zig.
## Copyright & patent mitigation
The initial implementation was a clean-room implementation by Stephen Gutekanst without having
read other ECS implementations' code, but with speaking to people familiar with other ECS
implementations. Contributions past the initial implementation may be made by individuals in
non-clean-room settings (familiar with open source implementations only.)
The initial implementation was a clean-room implementation by Stephen Gutekanst without having read other ECS implementations' code, but with speaking to people familiar with other ECS implementations. Contributions past the initial implementation may be made by individuals in non-clean-room settings (familiar with open source implementations only.)
Critically, this entity component system stores components for a classified archetype using both
a multi array list (independent arrays allocated per component) as well as hashmaps for sparse
component data for optimization. This is a novel and fundamentally different process than what
is described Unity Software Inc's patent US 10,599,560. This is not legal advice.
Critically, this entity component system stores components for a classified archetype using independent arrays allocated per component as well as hashmaps for sparse component data as an optimization. This is a novel and fundamentally different process than what is described in Unity Software Inc's patent US 10,599,560. This is not legal advice.

717
ecs/src/main.zig
View file

@ -16,252 +16,465 @@
//! implementations. Contributions past the initial implementation may be made by individuals in
//! non-clean-room settings.
//!
//! Critically, this entity component system stores components for a classified archetype using both
//! a multi array list (independent arrays allocated per component) as well as hashmaps for sparse
//! component data for optimization. This is a novel and fundamentally different process than what
//! is described Unity Software Inc's patent US 10,599,560. This is not legal advice.
//! Critically, this entity component system stores components for a classified archetype using
//! independent arrays allocated per component as well as hashmaps for sparse component data as an
//! optimization. This is a novel and fundamentally different process than what is described in
//! Unity Software Inc's patent US 10,599,560. This is not legal advice.
//!
const std = @import("std");
const mem = std.mem;
const Allocator = mem.Allocator;
const testing = std.testing;
const builtin = @import("builtin");
const assert = std.debug.assert;
const is_debug = builtin.mode == .Debug;
/// An entity ID uniquely identifies an entity globally within an Entities set.
///
/// It stores the type of entity, as well the index of the entity within EntityTypeStorage, in only
/// 48 bits.
///
/// Database equivalent: a row within a table
pub const EntityID = packed struct {
/// Entity type ("table ID")
type_id: u16,
const Entity = u32;
/// Entity ID ("row index")
id: u32,
};
pub fn TypedEntities(comptime Archetype: type) type {
/// Entity is a thin wrapper over an entity ID that makes interacting with a specific entity nicer.
///
/// Database equivalent: a row within a table
pub const Entity = struct {
/// The ID of the entity.
id: EntityID,
/// The entity type corresponding to id.type_id. You can look this up using Entities.byID()
///
/// Database equivalent: table of entities
entity_type: *EntityTypeStorage,
/// Adds or updates a component for this entity.
///
/// Optimized for *most* entities (of this type) having this type of component. If only a few
/// entities will have it, use `.setSparse()` instead.
pub inline fn set(entity: Entity, name: []const u8, component: anytype) !void {
var entity_type = entity.entity_type;
var storage = try entity_type.get(name, @TypeOf(component));
try storage.set(entity_type.allocator, entity.id.id, component);
}
/// Adds or updates a component for this entity.
///
/// Optimized for *few* entities (of this type) having this type of component. If most entities
/// will have it, use `.set()` instead.
pub inline fn setSparse(entity: Entity, component_name: []const u8, component: anytype) !void {
var entity_type = entity.entity_type;
var storage = try entity_type.get(component_name, @TypeOf(component));
try storage.setSparse(entity_type.allocator, entity.id.id, component);
}
/// Gets a component for this entity, returns null if that component is not set on this entity.
pub inline fn get(entity: Entity, component_name: []const u8, comptime Component: anytype) ?Component {
var entity_type = entity.entity_type;
var storage = entity_type.getIfExists(component_name, Component) orelse return null;
return storage.get(entity.id.id);
}
/// Removes the given component from this entity, returning a boolean indicating if it did
/// exist on the entity.
pub inline fn remove(entity: Entity, component_name: []const u8) bool {
var entity_type = entity.entity_type;
var storage = entity_type.getErasedIfExists(component_name) orelse return false;
return storage.remove(storage.ptr, entity.id.id);
}
// Deletes this entity.
pub inline fn delete(entity: Entity) !void {
var entity_type = entity.entity_type;
try entity_type.delete(entity.id);
}
// TODO: iterator over all components for the entity
};
/// Represents the storage for a single type of component within a single type of entity.
///
/// Database equivalent: a column within a table.
pub fn ComponentStorage(comptime Component: type) type {
return struct {
allocator: Allocator,
components: std.MultiArrayList(Archetype),
free_slots: std.AutoHashMap(Entity, void),
/// A reference to the total number of entities with the same type as is being stored here.
total_entities: *u32,
/// The actual component data. This starts as empty, and then based on the first call to
/// .set() or .setDense() is initialized as dense storage (an array) or sparse storage (a
/// hashmap.)
///
/// Sparse storage may turn to dense storage if someone later calls .set(), see that method
/// for details.
data: union(StorageType) {
empty: void,
dense: std.ArrayListUnmanaged(?Component),
sparse: std.AutoArrayHashMapUnmanaged(u32, Component),
} = .{ .empty = {} },
pub const StorageType = enum {
empty,
dense,
sparse,
};
pub const Archetype = Archetype;
const Self = @This();
pub const Iterator = struct {
e: *Self,
index: Entity = 0,
pub fn next(it: *Iterator) ?Archetype {
std.debug.assert(it.index <= it.e.components.len);
if (it.e.components.len == 0) return null;
while (it.index < it.e.components.len) {
if (it.e.contains(it.index)) {
const current = it.index;
it.index += 1;
return it.e.get(current);
}
it.index += 1;
}
return null;
}
};
pub fn init(allocator: Allocator) Self {
return .{
.allocator = allocator,
.components = std.MultiArrayList(Archetype){},
.free_slots = std.AutoHashMap(Entity, void).init(allocator),
};
}
pub fn add(self: *Self, components: Archetype) error{OutOfMemory}!Entity {
// Reuse a free slot, if available.
if (self.free_slots.count() > 0) {
var iter = self.free_slots.keyIterator();
const taken = iter.next().?.*;
_ = self.free_slots.remove(taken);
self.components.set(taken, components);
return taken;
}
// Create a new slot, potentially allocating.
try self.components.append(self.allocator, components);
return @intCast(Entity, self.components.len-1);
}
// In debug builds, panics if the entity does not exist.
pub fn update(self: *Self, entity: Entity, partial_components: anytype) void {
if (is_debug and !self.contains(entity)) @panic("no such entity");
inline for (@typeInfo(@TypeOf(partial_components)).Struct.fields) |field, i| {
const fieldEnum = @intToEnum(std.MultiArrayList(Archetype).Field, i);
self.components.items(fieldEnum)[entity] = @field(partial_components, field.name);
pub fn deinit(storage: *Self, allocator: Allocator) void {
switch (storage.data) {
.empty => {},
.dense => storage.data.dense.deinit(allocator),
.sparse => storage.data.sparse.deinit(allocator),
}
}
// In debug builds, panics if the entity does not exist.
pub fn set(self: *Self, entity: Entity, components: Archetype) void {
if (is_debug and !self.contains(entity)) @panic("no such entity");
self.components.set(entity, components);
}
// In debug builds, panics if the entity does not exist.
pub fn remove(self: *Self, entity: Entity) error{OutOfMemory}!void {
if (is_debug and !self.contains(entity)) @panic("no such entity");
try self.free_slots.put(entity, {});
}
pub fn contains(self: *Self, entity: Entity) bool {
if (entity > self.components.len-1) return false;
return !self.free_slots.contains(entity);
}
// In debug builds, panics if the entity does not exist.
pub fn get(self: *Self, entity: Entity) Archetype {
if (is_debug and !self.contains(entity)) @panic("no such entity");
return self.components.get(entity);
}
/// Creates a copy using the same allocator
pub inline fn clone(self: Self) !Self {
return self.cloneWithAllocator(self.allocator);
}
/// Creates a copy using a specified allocator
pub inline fn cloneWithAllocator(self: Self, new_allocator: Allocator) !Self {
return Self{
.allocator = new_allocator,
.components = try self.components.clone(new_allocator),
.free_slots = try self.free_slots.cloneWithAllocator(new_allocator),
};
}
pub fn iterator(self: *Self) Iterator {
return .{ .e = self };
}
pub fn deinit(self: *Self) void {
self.components.deinit(self.allocator);
self.free_slots.deinit();
}
};
}
pub fn Entities(comptime archetypes: anytype) type {
comptime var fields: []const std.builtin.TypeInfo.StructField = &.{};
inline for (archetypes) |Archetype| {
fields = fields ++ [_]std.builtin.TypeInfo.StructField{.{
.name = @typeName(Archetype),
.field_type = TypedEntities(Archetype),
.is_comptime = false,
.default_value = null,
.alignment = 0,
}};
}
const ComptimeEntities = @Type(.{
.Struct = .{
.layout = .Auto,
.is_tuple = false,
.decls = &.{},
.fields = fields,
// If the storage of this component is sparse, it is turned dense as calling this method
// indicates that the caller expects to set this component for most entities rather than
// sparsely.
pub fn set(storage: *Self, allocator: Allocator, row: u32, component: ?Component) !void {
switch (storage.data) {
.empty => if (component) |c| {
var new_dense = std.ArrayListUnmanaged(?Component){};
try new_dense.ensureTotalCapacityPrecise(allocator, storage.total_entities.*);
try new_dense.appendNTimes(allocator, null, storage.total_entities.*);
new_dense.items[row] = c;
storage.data = .{ .dense = new_dense };
} else return,
.dense => |dense| {
if (dense.items.len >= row) try storage.data.dense.appendNTimes(allocator, null, dense.items.len + 1 - row);
dense.items[row] = component;
},
});
.sparse => |sparse| {
// Turn sparse storage into dense storage.
defer storage.data.sparse.deinit(allocator);
const RuntimeEntities = struct {
erased: *anyopaque,
deinit: fn(Allocator, *anyopaque) void,
clone: fn(Allocator, *anyopaque) error{OutOfMemory}!*anyopaque,
};
return struct {
allocator: Allocator,
comptime_entities: ComptimeEntities,
runtime_entities: std.StringHashMap(RuntimeEntities),
const Self = @This();
pub fn init(allocator: Allocator) Self {
var comptime_entities: ComptimeEntities = undefined;
inline for (archetypes) |Archetype| {
@field(comptime_entities, @typeName(Archetype)) = TypedEntities(Archetype).init(allocator);
var new_dense = std.ArrayListUnmanaged(?Component){};
try new_dense.ensureTotalCapacityPrecise(allocator, storage.total_entities.*);
var i: u32 = 0;
while (i < storage.total_entities.*) : (i += 1) {
new_dense.appendAssumeCapacity(sparse.get(i));
}
new_dense.items[row] = component;
storage.data = .{ .dense = new_dense };
},
}
}
// If the storage of this component is dense, it remains dense.
pub fn setSparse(storage: *Self, allocator: Allocator, row: u32, component: ?Component) !void {
switch (storage.data) {
.empty => if (component) |c| {
var new_sparse = std.AutoArrayHashMapUnmanaged(u32, Component){};
try new_sparse.put(allocator, row, c);
storage.data = .{ .sparse = new_sparse };
} else return,
.dense => |dense| {
if (dense.items.len >= row) try storage.data.dense.appendNTimes(allocator, null, dense.items.len + 1 - row);
dense.items[row] = component;
},
.sparse => if (component) |c| try storage.data.sparse.put(allocator, row, c) else {
_ = storage.data.sparse.swapRemove(row);
},
}
}
/// Removes the given entity ID.
pub fn remove(storage: *Self, row: u32) bool {
return switch (storage.data) {
.empty => false,
.dense => |dense| if (dense.items.len > row and dense.items[row] != null) {
dense.items[row] = null;
return true;
} else false,
.sparse => storage.data.sparse.swapRemove(row),
};
}
/// Gets the component value for the given entity ID.
pub inline fn get(storage: Self, row: u32) ?Component {
return switch (storage.data) {
.empty => null,
.dense => |dense| if (dense.items.len > row) dense.items[row] else null,
.sparse => |sparse| sparse.get(row),
};
}
};
}
/// A type-erased representation of ComponentStorage(T) (where T is unknown).
///
/// This is useful as it allows us to store all of the typed ComponentStorage as values in a hashmap
/// despite having different types, and allows us to still deinitialize them without knowing the
/// underlying type.
pub const ErasedComponentStorage = struct {
ptr: *anyopaque,
deinit: fn (erased: *anyopaque, allocator: Allocator) void,
remove: fn (erased: *anyopaque, row: u32) bool,
pub fn cast(ptr: *anyopaque, comptime Component: type) *ComponentStorage(Component) {
var aligned = @alignCast(@alignOf(*ComponentStorage(Component)), ptr);
return @ptrCast(*ComponentStorage(Component), aligned);
}
};
/// Represents a single type of entity, e.g. a player, monster, or some other arbitrary entity type.
///
/// See the `Entities` documentation for more information about entity types and how they enable
/// performance.
///
/// Database equivalent: a table where rows are entities and columns are components (dense storage)
/// or a secondary table with entity ID -> component value relations (sparse storage.)
pub const EntityTypeStorage = struct {
allocator: Allocator,
/// This entity type storage identifier. This is used to uniquely identify this entity type
/// within the global set of Entities, and is identical to the EntityID.type_id value.
id: u16,
/// The number of entities that have been allocated within this entity type. This is identical
/// to the EntityID.id value.
count: u32 = 0,
/// A string hashmap of component_name -> type-erased *ComponentStorage(Component)
components: std.StringArrayHashMapUnmanaged(ErasedComponentStorage) = .{},
/// Free entity slots. When an entity is deleted, it is added to this map and recycled the next
/// time a new entity is requested.
free_slots: std.AutoArrayHashMapUnmanaged(u32, void) = .{},
pub fn init(allocator: Allocator, type_id: u16) EntityTypeStorage {
return .{
.allocator = allocator,
.comptime_entities = comptime_entities,
.runtime_entities = std.StringHashMap(RuntimeEntities).init(allocator),
.id = type_id,
};
}
pub fn get(self: *Self, comptime Archetype: type) !*TypedEntities(Archetype) {
// Comptime archetype lookup
if (@hasField(ComptimeEntities, @typeName(Archetype))) {
return &@field(self.comptime_entities, @typeName(Archetype));
pub fn deinit(storage: *EntityTypeStorage) void {
for (storage.components.values()) |erased| {
erased.deinit(erased.ptr, storage.allocator);
}
storage.components.deinit(storage.allocator);
storage.free_slots.deinit(storage.allocator);
}
// Runtime archetype lookup
var v = try self.runtime_entities.getOrPut(@typeName(Archetype));
/// Creates a new entity of this type.
pub fn new(storage: *EntityTypeStorage) !Entity {
return Entity{
.id = try storage.newID(),
.entity_type = storage,
};
}
// TODO: bulk allocation of entities
/// Creates a new entity of this type.
pub fn newID(storage: *EntityTypeStorage) !EntityID {
// If there is a previously deleted entity, recycle it's ID.
// TODO: add some "debug" mode which catches use-after-delete of entities (could be super
// confusing if one system deletes it and another creates it and you don't notice!)
const free_slot = storage.free_slots.popOrNull();
if (free_slot) |recycled| return EntityID{ .type_id = storage.id, .id = recycled.key };
// Create a new entity ID and space to store it in each component array.
const new_id = storage.count;
storage.count += 1;
return EntityID{ .type_id = storage.id, .id = new_id };
}
/// Deletes the specified entity. See also the `Entity.delete()` helper.
///
/// This merely marks the entity as deleted, the same ID will be recycled the next time a new
/// entity is created.
pub fn delete(storage: *EntityTypeStorage, id: EntityID) !void {
assert(id.type_id == storage.id);
try storage.free_slots.put(storage.allocator, id.id, .{});
}
/// Returns the component storage for the given component. Creates storage for this type of
/// component if it does not exist.
///
/// Note: This is a low-level API, you probably want to use `Entity.get()` instead.
pub fn get(storage: *EntityTypeStorage, component_name: []const u8, comptime Component: type) !*ComponentStorage(Component) {
var v = try storage.components.getOrPut(storage.allocator, component_name);
if (!v.found_existing) {
const new = try self.allocator.create(TypedEntities(Archetype));
new.* = TypedEntities(Archetype).init(self.allocator);
v.value_ptr.* = RuntimeEntities{
.erased = new,
var new_ptr = try storage.allocator.create(ComponentStorage(Component));
new_ptr.* = ComponentStorage(Component){
.total_entities = &storage.count,
};
v.value_ptr.* = ErasedComponentStorage{
.ptr = new_ptr,
.deinit = (struct {
pub fn deinit(allocator: Allocator, erased: *anyopaque) void {
const aligned = @alignCast(@alignOf(*TypedEntities(Archetype)), erased);
const entities = @ptrCast(*TypedEntities(Archetype), aligned);
entities.deinit();
allocator.destroy(entities);
pub fn deinit(erased: *anyopaque, allocator: Allocator) void {
var ptr = ErasedComponentStorage.cast(erased, Component);
ptr.deinit(allocator);
allocator.destroy(ptr);
}
}.deinit),
.clone = (struct {
pub fn clone(new_allocator: Allocator, erased: *anyopaque) error{OutOfMemory}!*anyopaque {
const aligned = @alignCast(@alignOf(*TypedEntities(Archetype)), erased);
const entities = @ptrCast(*TypedEntities(Archetype), aligned);
const new_erased = try new_allocator.create(TypedEntities(Archetype));
new_erased.* = try entities.cloneWithAllocator(new_allocator);
return new_erased;
}).deinit,
.remove = (struct {
pub fn remove(erased: *anyopaque, row: u32) bool {
var ptr = ErasedComponentStorage.cast(erased, Component);
return ptr.remove(row);
}
}).clone,
}).remove,
};
}
const aligned = @alignCast(@alignOf(*TypedEntities(Archetype)), v.value_ptr.erased);
return @ptrCast(*TypedEntities(Archetype), aligned);
return ErasedComponentStorage.cast(v.value_ptr.ptr, Component);
}
/// Creates a copy using the same allocator
pub inline fn clone(self: Self) !Self {
return self.cloneWithAllocator(self.allocator);
/// Returns the component storage for the given component, returning null if it does not exist.
///
/// Note: This is a low-level API, you probably want to use `Entity.get()` instead.
pub fn getIfExists(storage: *EntityTypeStorage, component_name: []const u8, comptime Component: type) ?*ComponentStorage(Component) {
var v = storage.components.get(component_name);
if (v == null) return null;
return ErasedComponentStorage.cast(v.?.ptr, Component);
}
/// Creates a copy using a specified allocator
pub fn cloneWithAllocator(self: Self, new_allocator: Allocator) !Self {
var comptime_entities: ComptimeEntities = undefined;
inline for (archetypes) |Archetype| {
const field = @field(self.comptime_entities, @typeName(Archetype));
@field(comptime_entities, @typeName(Archetype)) = try field.cloneWithAllocator(new_allocator);
/// Returns the type-erased component storage for the given component, returning null if it does
/// not exist.
///
/// Note: This is a low-level API, you probably want to use `Entity.get()` instead.
pub inline fn getErasedIfExists(storage: *EntityTypeStorage, component_name: []const u8) ?ErasedComponentStorage {
return storage.components.get(component_name);
}
};
var runtime_entities = try self.runtime_entities.cloneWithAllocator(new_allocator);
var iter = runtime_entities.valueIterator();
while (iter.next()) |entities| {
entities.erased = try entities.clone(new_allocator, entities.erased);
}
/// A database of entities. For example, all player, monster, etc. entities in a game world.
///
/// Entities are divided into "entity types", arbitrary named groups of entities that are likely to
/// have the same components. If you are used to archetypes from other ECS systems, know that these
/// are NOT the same as archetypes: you can add or remove components from an entity type at will
/// without getting a new type of entity. You can get an entity type using e.g.:
///
/// ```
/// const world = Entities.init(allocator); // all entities in our world
/// const players = world.get("player"); // the player entities
///
/// const player1 = players.new(); // a new entity of type "player"
/// const player2 = players.new(); // a new entity of type "player"
/// ```
///
/// Storage is optimized around the idea that all entities of the same type *generally* have the
/// same type of components. Storing entities by type also enables quickly iterating over all
/// entities with some logical type without any sorting needed (e.g. iterating over all "player"
/// entities but not "monster" entities.) This also reduces the search area for more complex queries
/// and makes filtering entities by e.g. "all entities with a Renderer component" more efficient
/// as we just *know* that if player1 has that component, then player2 almost certainly does too.
///
/// You can have 65,535 entity types in total.
///
/// Although storage is *generally* optimized for all entities within a given type having the same
/// components, you may set/remove components on an entity at will via e.g. `player1.set(component)`
/// and `player1.remove(Component)`. `player1` and `player2` may not both have a Renderer component,
/// for example.
///
/// If you use `player1.set(myRenderer);` then dense storage will be used: we will optimize for
/// *every* entity of type "player" having a Renderer component. In this case, every "player" entity
/// will pay the cost of storing a Renderer component even if they do not have one.
///
/// If you use `player1.setSparse(myRenderer);` then sparse storage will be used: we will optimize
/// for *most* entities of type "player" not having a Renderer component. In this case, only the
/// "player" entities which have a Renderer component pay a storage cost. If most entities have a
/// Renderer component, this would be the wrong type of storage and less efficient.
///
/// Database equivalents:
/// * Entities is a database of tables, where each table represents a type of entity.
/// * EntityTypeStorage is a table, whose rows are entities.
/// * EntityID is a 32-bit row ID and a 16-bit table ID, and so globally unique.
/// * ComponentStorage(T) is a column of data in a table for a specific component type
/// * Densely stored as an array of component values.
/// * Sparsely stored as a map of (row ID -> component value).
pub const Entities = struct {
allocator: Allocator,
return Self{
.allocator = new_allocator,
.comptime_entities = comptime_entities,
.runtime_entities = runtime_entities,
/// A mapping of entity type names to their storage.
///
/// Database equivalent: table name -> tables representing entities.
types: std.StringArrayHashMapUnmanaged(EntityTypeStorage),
pub fn init(allocator: Allocator) Entities {
return .{
.allocator = allocator,
.types = std.StringArrayHashMapUnmanaged(EntityTypeStorage){},
};
}
pub fn deinit(self: *Self) void {
inline for (archetypes) |Archetype| {
@field(self.comptime_entities, @typeName(Archetype)).deinit();
pub fn deinit(entities: *Entities) void {
var iter = entities.types.iterator();
while (iter.next()) |entry| {
entry.value_ptr.deinit();
}
var runtime_iter = self.runtime_entities.valueIterator();
while (runtime_iter.next()) |runtime_entities| {
runtime_entities.deinit(self.allocator, runtime_entities.erased);
}
self.runtime_entities.deinit();
entities.types.deinit(entities.allocator);
}
// TODO: iteration over all entities
// TODO: iteration over all entities with components (U, V, ...)
// TODO: iteration over all entities with type T
// TODO: iteration over all entities with type T and components (U, V, ...)
// TODO: "indexes" - a few ideas we could express either within a single entity type or across
// all entities:
//
// * Graph relations index: e.g. parent-child entity relations for a DOM / UI / scene graph.
// * Spatial index: "give me all entities within 5 units distance from (x, y, z)"
// * Generic index: "give me all entities where arbitraryFunction(e) returns true"
//
/// Returns a nice helper for interfacing with the specified entity.
///
/// This is a mere O(1) array access and so is very cheap.
pub inline fn byID(entities: *const Entities, id: EntityID) Entity {
return .{
.id = id,
// TODO: entity type lookup `entities.types.entries.get(id.type_id).value`
// would not give us a pointer to the entry, which is required. I am 99% sure we can do this
// in O(1) time, but MultiArrayList (`entries`) doesn't currently expose a getPtr method.
//
// For now this is actually not O(1), but still very fast.
.entity_type = entities.types.getPtr(entities.typeName(id)).?,
};
}
/// Returns the entity type name of the entity given its ID.
///
/// This is a mere O(1) array access and so is very cheap.
pub inline fn typeName(entities: *const Entities, id: EntityID) []const u8 {
return entities.types.entries.get(id.type_id).key;
}
// Returns the storage for the given entity type name, creating it if necessary.
// TODO: copy name?
pub fn get(entities: *Entities, entity_type_name: []const u8) !*EntityTypeStorage {
const num_types = entities.types.count();
var v = try entities.types.getOrPut(entities.allocator, entity_type_name);
if (!v.found_existing) {
v.value_ptr.* = EntityTypeStorage.init(entities.allocator, @intCast(u16, num_types));
}
return v.value_ptr;
}
// TODO: ability to remove entity type entirely, deleting all entities in it
// TODO: ability to remove entity types with no entities (garbage collect)
};
test "example" {
const allocator = testing.allocator;
//-------------------------------------------------------------------------
// Create a world.
var world = Entities.init(allocator);
defer world.deinit();
//-------------------------------------------------------------------------
// Define component types, any Zig type will do!
// A location component.
const Location = struct {
x: f32 = 0,
@ -272,64 +485,76 @@ test "example" {
// A name component.
const Name = []const u8;
// A player archetype.
const Player = struct {
name: Name,
location: Location = .{},
};
//-------------------------------------------------------------------------
// Create a player entity type. Every entity with the same type ("player" here)
// will pay to store the same set of components, whether they use them or not.
var players = try world.get("player");
const allocator = testing.allocator;
// Create first player entity.
var player1 = try players.new();
try player1.set("name", @as(Name, "jane")); // add Name component
try player1.set("location", Location{}); // add Location component
// Entities for e.g. a world. Stores multiple archetypes!
var entities = Entities(.{
// Predeclare your archetypes up front here and you get Archetype lookups at comptime / for free!
Player,
}).init(allocator);
defer entities.deinit();
// Create second player entity. Note that it pays the cost of storing a Name and Location
// component regardless of whether or not we use it: all entities in the same type ("players")
// pays to store the same set of components.
var player2 = try players.new();
try testing.expect(player2.get("location", Location) == null);
try testing.expect(player2.get("name", Name) == null);
// Get the player entities
var players = try entities.get(Player);
//-------------------------------------------------------------------------
// We can add new components at will. Now every player entity will pay to store a Rotation
// component.
const Rotation = struct { degrees: f32 };
try player2.set("rotation", Rotation{ .degrees = 90 });
try testing.expect(player1.get("rotation", Rotation) == null); // player1 has no rotation
// A monster archetype
const Monster = struct {
name: Name,
};
//-------------------------------------------------------------------------
// Most of your entities don't have a component, but a few do? Use setSparse instead!
// This is optimized for some entities having the component, but most not having it.
const Weapon = struct { name: []const u8 };
try player1.setSparse("weapon", Weapon{ .name = "sword" });
try testing.expectEqualStrings("sword", player1.get("weapon", Weapon).?.name); // lookup is the same regardless of storage type
try testing.expect(player2.get("weapon", Weapon) == null); // player2 has no weapon
// Get the monster entities - note that we didn't declare this archetype up front in Entities.init!
// This archetype lookup will be done at runtime via a type name hashmap
var monsters = try entities.get(Monster);
//-------------------------------------------------------------------------
// Remove a component from any entity at will. We'll still pay the cost of storing it for each
// component, it's just set to `null` now.
// TODO: add a way to "cleanup" truly unused components.
_ = player1.remove("location"); // remove Location component
_ = player1.remove("weapon"); // remove Weapon component
// Let's add some entities!
const carrot = try monsters.add(.{.name = "carrot"});
const tomato = try monsters.add(.{.name = "tomato"});
const potato = try monsters.add(.{.name = "potato"});
//-------------------------------------------------------------------------
// At runtime we can query the type of any entity.
try testing.expectEqualStrings("player", world.typeName(player1.id));
// Remove some of our entities
try monsters.remove(carrot);
try monsters.remove(tomato);
//-------------------------------------------------------------------------
// Entity IDs are all you need to store, they're 48 bits. You can always look up an entity by ID
// in O(1) time (mere array access):
const player1_by_id = world.byID(player1.id);
// Change an entity's components
monsters.set(potato, .{.name = "totally real potato"});
//-------------------------------------------------------------------------
// Introspect things.
// Entity types
var entity_types = world.types.keys();
try testing.expectEqual(@as(usize, 1), entity_types.len);
try testing.expectEqualStrings("player", entity_types[0]);
// Don't want to set all the components of an entity? i.e. just want to set one field? This can
// be more efficient:
monsters.update(potato, .{.name = "secretly tomato"});
// Component types for a given entity type "player"
var component_names = (try world.get("player")).components.keys();
try testing.expectEqual(@as(usize, 4), component_names.len);
try testing.expectEqualStrings("name", component_names[0]);
try testing.expectEqualStrings("location", component_names[1]);
try testing.expectEqualStrings("rotation", component_names[2]);
try testing.expectEqualStrings("weapon", component_names[3]);
// Get all components of an entity
try testing.expectEqual(Monster{.name = "secretly tomato"}, monsters.get(potato));
// TODO: iterating components an entity has not currently supported.
// Iterate entities.
_ = try players.add(.{.name = "jane"});
_ = try players.add(.{.name = "bob"});
var iter = players.iterator();
try testing.expectEqualStrings("jane", iter.next().?.name);
try testing.expectEqualStrings("bob", iter.next().?.name);
// We can clone sets of entities, if needed.
var players2 = try players.clone();
defer players2.deinit();
// Or maybe clone ALL entities!
var entities2 = try entities.clone();
defer entities2.deinit();
//-------------------------------------------------------------------------
// Delete an entity whenever you wish. Just be sure not to try and use it later!
try player1_by_id.delete();
}
test "entity ID size" {
try testing.expectEqual(6, @sizeOf(EntityID));
}