Writing an Entity-Component System in Rust, Part 1: Components and Queries

For a long time I've had an idea for a kind of abstract relational system, where entities and all their properties could be tracked, queried, and manipulated to synthesize logical and useful data. After burning a hole in the back of my head for years, I've decided draft a series of posts to document my exploration of this idea!

My primary inspiration for this system came from mentally modelling the data files for the game Dwarf Fortress. Those who have played a significant amount of the enigmatic but excellent Dwarf Fortress (which recently had a Steam Release!) might recognize this kind of data as similar to the game's raw file data format. This text-based format tags everything in the game with all their properties, including weapons, toys, instruments, foods, and every living thing. For example, the raws describe every creature's body plan, what their flesh and blood is made of, what foods they can eat, and what languages they can learn based on their bodily anatomy. An extremely condensed sample of a raw file taken directly from the game looks something like this:

[OBJECT:CREATURE]

[CREATURE:DWARF]
	[DESCRIPTION:A short, sturdy creature fond of drink and industry.]
	[NAME:dwarf:dwarves:dwarven]
	[CASTE_NAME:dwarf:dwarves:dwarven]
	[CREATURE_CLASS:MAMMAL]

	[INTELLIGENT]
	[BENIGN]
	[CANOPENDOORS]

	[BODY:HUMANOID_NECK:2EYES:2EARS:NOSE:2LUNGS:HEART:GUTS:ORGANS:HUMANOID_JOINTS:THROAT:NECK:SPINE:BRAIN:SKULL:5FINGERS:5TOES:MOUTH:TONGUE:FACIAL_FEATURES:TEETH:RIBCAGE]
    
	[TISSUE_LAYER:BY_CATEGORY:FINGER:NAIL:FRONT]
	[TISSUE_LAYER:BY_CATEGORY:TOE:NAIL:FRONT]

	[SELECT_TISSUE_LAYER:HEART:BY_CATEGORY:HEART]
	 [PLUS_TISSUE_LAYER:SKIN:BY_CATEGORY:THROAT]
		[TL_MAJOR_ARTERIES]

	[HAS_NERVES]

	... lots more body plan information ...

	[CAN_DO_INTERACTION:PET_ANIMAL]
	[CAN_DO_INTERACTION:MATERIAL_EMISSION]
		[CDI:ADV_NAME:Spit]
        
    ... over 5,000 lines ...

After lots of time playing and inspecting the raws, I realized that this information maps almost perfectly to the use case for an Entity-Component System! Each item along with its metadata could be its own Component attached to a "Creature::Dwarf" entity in the system. A system that could query for semantic data out of a persistent list of only loosely related entities sounds at the most quite useful, and at least a neat tool to play with. So without further ado, let's get started!


My ideal API for this ECS is type-based, and is very much inspired by Bevy's excellent ECS module. Someone who wanted to query the database would request entities matching a set of components as Rust types:

let results = <(&mut TA, Option<&TB>, Without<TC>, &TD)>::query(&mut World);
for (componentA, componentB, _, componentD) in results {
	...
}

Queries constructed at runtime should be allowed, too:

let results: HashMap<Uuid, ComponentUnchecked> = DynamicQuery::new()
    .with_mut(TA::COMPONENTID)
    .without(TC::COMPONENTID)
    .option(TB::COMPONENTID)
    .with(
    .execute(&mut World);

let mut componentA : TA = results.get(TA::COMPONENTID)?.into(); // or try_into();

Finally, defining new components should be simple...

#[derive(Component, Serialize, DeserializeOwned)]
#[Component("289fa888-a7e2-424f-a2a0-41e218bc0108")
struct MyComponentA;

#[derive(Component, Serialize, DeserializeOwned)]
#[Component("c6373892-7437-45df-b928-2e79e1ab9418")
struct MyComponentB {
    .. Any contents
}

... and users should be able to implement Component themselves, too.

#[derive(Serialize, DeserializeOwned)]
struct MyComponentCustom;

impl Component for MyComponentCustom {
    const COMPONENTID: uuid::Uuid;

    fn into_bytes(self) -> Result<Vec<u8>> {
        // Super duper complicated serialization routine
    }

    fn from_bytes(b: &[u8]) -> Result<Self> {
        // Super duper complicated deserialization routine
    }
}
While it may be decidedly un-ergonomic to require users to specify their type's UUID manually, it saves us a lot of error-prone type checking code while developing. Ultimately, having UUIDs defined at compile time will spare a lot of headache in the long run. In the future we'll design some extra modules to better support runtime components and queries.

But enough specification speculation! Let's get started with a project and some dependencies:

cargo new --lib ecsdb
# Cargo.toml
[package]
name = "ecsdb"
version = "0.1.0"
edition = "2021"

[dependencies]
bson = { version = "2.5.0", features = ["uuid-1", "serde_with"]}
diesel = { version = "^2.0", features = ["r2d2", "postgres", "postgres_backend", "uuid"] }
diesel_migrations = "^2.0"
dotenvy = "0.15"
eyre = "^0.6"
hashbrown = "^0.13"
num-bigint = ">=0.2.0, <0.5.0"
serde = { version = "1.0", features = ["derive"] }
tracing = "^0.1"
tracing-subscriber = "^0.3"
uuid = { version = "=1.2.2", features = ["v4", "serde"] }
uuid-simd = "0.8.0"

We'll use diesel to manage our PostgreSQL backend, saving us lots of boilerplate. Our components will be identified by UUIDs, and the data they represent will be serialized into the database in bson format.

Note: Code for diagnostics such as logging and error management have been removed and/or simplified from pieces of the source for brevity. The full and unedited source is available on github.

The World

The starting point for our ECS is the World. This will be the data structure that keeps a database connection around for use when querying for entities as well as committing those entities back to the database.

// src/lib.rs
#[derive(Clone)]
pub struct World {
    db: Pool<ConnectionManager<PgConnection>>,
}

pub type WorldConnection = PooledConnection<ConnectionManager<PgConnection>>;

As far as complexity, this is as simple as it gets. Our world will keep a connection pool to a PostgreSQL database as long as it's alive, and when it is dropped the connection will be automatically closed. Because we're using a connection pool instead of a single connection, we can move around and clone our World and still maintain thread safety (because Pool is Send+ Sync, World will be too). Finally, we define a type alias for an individual world connection, since we'll be passing it around a lot.

Let's move on to some API structure:

impl World {
    pub fn connect<'a>(database_url: impl Into<&'a str>) -> Result<World> {
        let url = database_url.into();
        let mut conn = ConnectionManager::<PgConnection>::new(url);
        let manager = Pool::builder()
            .test_on_check_out(true)
            .build(conn)?;

        const MIGRATIONS: EmbeddedMigrations = embed_migrations!();
        if let Err(e) = manager.get()?.run_pending_migrations(MIGRATIONS) {
            bail!(e);
        }

        Ok(World { db: manager })
    }

    pub(crate) fn get(&mut self) -> Result<WorldConnection> {
        Ok(self.db.get()?)
    }
}

These methods are used to create new worlds and connections. New worlds (pools into the database) can be created with new, and individual connections can be requested using get. Importantly, the get method is only accessible to our crate - we don't want just anyone to be able to query the database directly, that's what our API will be for!

And that's it for our World API, at least for now. There's a lot more functionality we'll want to add as we expand on the API in future posts, but for now let's move on to some more mechanical features we can really sink our teeth into.

Components and Queries

Our next stop will be the user-facing query system. Once we have that built, the backend can be plugged in (or even swapped out!) later.

Let's make a new module and write up a Component trait:

// src/component.rs
pub trait Component: Send + Sync + Serialize + DeserializeOwned {
    const COMPONENTID: uuid::Uuid;

	fn from_unchecked(from: UncheckedEntity) -> Result<Self>;
    
    fn into_bytes(self) -> Result<Vec<u8>> {
        Ok(bson::to_vec(&self)?)
    }

    fn from_bytes(b: &[u8]) -> Result<Self> {
        Ok(bson::from_slice(b)?)
    }
}

Our Component trait is fairly simple, but it does a lot of heavy lifting for us!

  • Every component has its own ID which allows the library to identify which database object corresponds to which data structure.
  • We require it to be Send + Sync, which automatically makes the C in our ECS thread-safe by default!
  • We also require it to be Serialize + DeserializeOwned. Both these traits are provided by Serde, and mean any component can be seamlessly be converted to and from binary format.
  • Finally, we provide some methods to quickly serialize and de-serialize the component using the aforementioned trait constraints.

The associated constant COMPONENTID has a few implications. By representing a component's ID as an associated constant (rather than, say, the result of a trait method) we implicitly deny any library functionality that takes Components as an input from using Component-like objects created at runtime. This keeps our serialized data strongly tied to our program's type system, thus avoiding many potential issues with ambiguity in determining the underlying type of some component.

Finally, we have a from_unchecked function that takes in an EntityUnchecked and returns an instance of the component. EntityUnchecked will be an important intermediate component used to bridge the gap between raw database results and concrete Component instances, and it looks like this:

pub struct EntityUnchecked {
    id: Uuid,
    components: HashMap<Uuid, Vec<u8>>,
}

For now, this will be the most we delve into dynamic type serialization. We'll eventually get to exploring runtime components in another post, but for now we'll stay in the safe and comfortable land of strongly-typed values.


Our query system is conceptually more complex than the Component definition, so let's break it down bit by bit before we dive into the deep end. First, let's take a look at the WorldQuery trait, which will leverage Rust's powerful type system to build our queries:

pub trait WorldQuery {
    type Item<'a>;

    fn build(b: &mut QueryBuilder) -> &mut QueryBuilder;

    fn parse(r: &EntityUnchecked) -> Result<Self::Item<'static>>;

    fn query(w: &mut World) -> Result<Vec<Self::Item<'static>>> {
        // -- snipped out --
    }
}

Any type that implements this trait will know how to query itself from the database as well as serialize itself into some concrete data the user can act upon. It does this with a few members:

  • Every WorldQuery will have a type they will try to return (type Item<'a>;). Importantly, it need not be the type it's implemented on! This will be useful later when we implement Type Guards for mutable query results - mutably queried components will need to be committed back to the database when they're dropped.
  • The build function allow implementors to decide how to filter query results based on their associated type. By separating this from the parse and execute functions we can actually compile the SQL query with its filters built-in, thus saving bandwidth and memory by not asking for payloads we're not interested in getting.
  • Parse asks implementors to transform a single EntityUnchecked (representing a single row from a query result) into their concrete type. Diesel's query functionality doesn't allow for runtime query return types - every query must have a statically defined in-software format to deserialize into. While this is very good for memory safety, it's not so great for our ECS which may request an arbitrary amount of different types; parse is our solution to get around this limitation.

The only remaining item is Query, which has its own default implementation:

fn query(w: &mut World) -> Result<Vec<Self::Item<'static>>> {
    let results = diesel::sql_query(Self::build(&mut QueryBuilder::new()).to_sql())
    .load::<RawEntity>(&mut w.get()?)?;
    let len = results.len();
    results.into_iter()
    .try_fold::<Vec<Self::Item<'static>>, _, Result<Vec<Self::Item<'static>>>>(
        Vec::with_capacity(len), 
        |mut acc, r| { 
            acc.push(Self::parse(&EntityUnchecked::new(r)?, w)?);
            Ok(acc)
        }
    )
}

This implementation leverages build and parse to assemble and run the query, and then return the Items for matching entities in a vector.

  • First it lends an exclusive reference to an instance of QueryBuilder into the implementer's build method to construct a SQL query string.
  • Then it performs a direct query into PostgreSQL using diesel's sql_query function. Diesel marshals the raw query data into a struct named RawEntity, which we'll be defining in a later post alongside the guts of the database implementation.
  • It converts each resultant RawEntity into an EntityUnchecked, which it folds into a vector that is then returned. The vector is preallocated with the number of entities, which ensures only one allocation is needed per call to query.
Why not use map? I specifically chose to use try_fold here instead of map primarily because try_fold fast-exits if its closure returns an error, which is desirable in this context. If the user gets a set of values, we want to make sure they are absolutely correct, so lazy loading (which we'd achieve by returning an Iterator instead of a Vec!) is undesirable and fast-exiting on an error increases efficiency.

Before implementing our new trait, we should define the QueryBuilder so we can understand how to use it properly in our implementations.

// src/query/builder.rs

#[derive(Default)]
pub(crate) struct QueryBuilder {
    components: Vec<(Uuid, QueryCombinator)>,
}

Our QueryBuilder will be little more than a simple list of component requests! A few methods will complete our builder, and successfully abstract away any SQL we'll be interacting with in later posts.

impl QueryBuilder {
    pub fn new() -> Self { QueryBuilder::default() }

    fn register<T: Component>(&mut self, combo: QueryCombinator) -> &mut Self {
        let component = <T as Component>::COMPONENTID;
        self.components.push((component, combo)); 
        self
    }

    pub fn with<T: Component>(&mut self) -> &mut Self {
        self.register::<T>(QueryCombinator::PRESENT)
    }

    pub fn without<T: Component>(&mut self) -> &mut Self {
        self.register::<T>(QueryCombinator::ABSENT)
    }

    pub fn option<T: Component>(&mut self) -> &mut Self {
        self.register::<T>(QueryCombinator::OPTIONAL)
    }

    pub fn to_sql(&self) -> String {
        todo!();
    }
}

enum QueryCombinator {
    PRESENT,
    ABSENT,
    OPTIONAL,
}

Here we have three public builder functions that record how a particular queryable type should be represented in a query that may have multiple terms – the query might only want entities which have the component, only ones which don't have it, or it might not care whether or not it has the component, but if it does it wants its payload. All of these functions are routed through a single helper function, and there is also a stubbed method to consume the builder and produce a SQL query string.


With our builder now built, we can finally start implementing WorldQuery. Our first candidate is perhaps the simplest kind of value: A single, immutable component.

impl<T: Component> WorldQuery for &T {
    type Item<'_world> = &'_world T;

    fn build(b: &mut QueryBuilder) -> &mut QueryBuilder {
        b.with::<T>()
    }

    fn parse(r: &QueryOutput) -> Result<Self::Item<'static>> {
        if let Some((_, bytes)) = r.get(&T::COMPONENTID) {
            T::from_bytes(bytes)
        }
        else {
            bail!("Single-component WorldQuery response did not contain the correct id")
        }
    }
}

This blanket implementation covers any and every type that implements Component! Let's look at it member by member:

  • type Item<'_world> = &'_world T; – If we're providing a reference to data that is stored in the World (even if only conceptually), that data should only live as long as the world that handed it out lives.
  • b.with::<T>() – If the user is querying for the presence of a component, we should request that component's associated data.
  • fn parse – Here we're getting the binary payload (Vec<u8>) corresponding to the implementing component's ID. If the response didn't have any entities with the component, we use eyre's bail! macro to return early with an error message.

Unfortunately, if we try to compile we get an error:

error[E0308]: mismatched types
  --> src\query\mod.rs:71:13
   |
62 | impl<T: Component> WorldQuery for &T {
   |      - this type parameter
...
69 |     fn parse(r: &HashMap<Uuid, Vec<u8>>) -> Result<Self::Item<'static>> {
   |                                             --------------------------- expected `Result<&'static T, ErrReport>` because of return type
70 |         if let Some((_, bytes)) = r.get_key_value(&T::COMPONENTID) {
71 |             T::from_bytes(bytes)
   |             ^^^^^^^^^^^^^^^^^^^^ expected `&T`, found type parameter `T`
   |
   = note: expected enum `Result<&'static T, _>`
              found enum `Result<T, _>`

The issue lies in our return value; Our function promises Self::Item, which is a shared reference to T, but we're trying to return an owned T! If we try to naively fix this by attempting to return a reference to the value, we predictably get an ownership error:

T::from_bytes(bytes).map(|t| &t)
error[E0515]: cannot return reference to function parameter `t`
  --> src\query\mod.rs:71:42
   |
71 |             T::from_bytes(bytes).map(|t| &t)
   |                                          ^^ returns a reference to data owned by the current function

In this case, parse still owns the constructed object it created once it returns. If we try to return a reference to it, the reference will immediately outlive the lifetime of the object it's referencing and thus violate Rust's memory safety rules.

However, this only a temporary setback. We can solve this handily by introducing a small struct that owns the returned data and hands out only immutable references using the nifty Borrow trait:

// src/query/references.rs
pub struct Ref<T: Component + ?Sized> {
    value: T
}

impl<T> Ref<T> where T: Component + ?Sized {
    pub fn new(value: T) -> Self {
        Ref{ value }
    }
}

impl<T> Borrow<T> for Ref<T> where T: Component + ?Sized {
    fn borrow(&self) -> &T {
        &self.value
    }
} 

Astute readers will notice that our new type is similar to Box<T> in that it allows itself to be borrowed as its internal value. Unlike Box however, it only allows immutable borrows, and cannot be dereferenced to its internal value either. Wrapping a value in this type guarantees at compile time that no shared references to the underlying object will exist (barring the use of unsafe and undefined behaviors).

With this in our toolbox we can amend our above implementation to the following:

impl<T: Component> WorldQuery for &T {
    type Item<'_world> = Ref<T>;      // <------ CHANGED

    fn build(b: &mut QueryBuilder) -> &mut QueryBuilder {
        b.with::<T>()
    }

    fn parse(r: &QueryOutput) -> Result<Self::Item<'static>> {
        if let Some(bytes) = r.get_key_value(&T::COMPONENTID) {
            T::from_bytes(bytes).map(|r| Ref::new(r))      // <------ CHANGED
        }
        else {
            bail!("Single-component WorldQuery response did not contain the correct id")
        }
    }
}

Now the compiler is satisfied and our desired behavior is maintained.

let mut object: Ref<MyQueryComponent> = MyQueryComponent::parse(&result)?;
let reference: &MyQueryComponent = &object;
let mut_ref = &mut object; // Still works, but no &mut methods will be available.

While we're at it, let's implement the same for Option<T>, where the requester wants the component if it exists, but still wants to include entities that don't have the component. The impl block is identical to our previous block for &T, except we wrap Ref<T> into Option<Ref<T>> and substitute b.with::<T>() for b.option::<T>():

impl<T: Component> WorldQuery for Option<&T>  {
    type Item<'_world> = Option<Ref<T>>;
    
    fn build(b: &mut QueryBuilder) -> &mut QueryBuilder {
        b.option::<T>()
    }

    fn parse(r: &QueryOutput) -> Result<Self::Item<'static>> {
        match r.get_key_value(&T::COMPONENTID) {
            Some((_, bytes)) => T::from_bytes(bytes).map(|t| Some(Ref::new(t))),
            None => Ok(None),
        }
    }
}

Next let's implement the reverse, a Without query that specifies the given component should be excluded from the search rather than included. This one is even simpler than the last:

pub struct Without<T: for<'_world> Component>(PhantomData<T>);

impl<T: for<'_world> Component> WorldQuery for Without<T> {
    type Item<'_world> = ();

    fn build(b: &mut QueryBuilder) -> &mut QueryBuilder {
        b.without::<T>()
    }

    fn parse(r: &QueryOutput) -> Result<Self::Item<'static>> {
        if let Some(_) = r.get(&T::COMPONENTID) {
            bail!("WITHOUT Query returned SOME")
        }
        else {
            Ok(())
        }
    }
}

Here, we specify the item is (), which is rust's unit type, and it means Item can never have any meaningful value. After all, if we want components from entities that do not have component T, then there's no way we could get an instance of T from that entity! This streamlines our implementation since there's no return value management to worry about.


The last kind of query we have to implement is a mutable reference to a component. When a user mutates a queried component, that object in memory will contain the change, but subsequent queries into the backing database won't! In order to persist our changes to all future queries we need to inform the database that we've changed some data and it needs to be updated.

In order to implement this, we'll need two things: a way to detect when changes are made and a way to write those changes back to the database. Let's go ahead and make a new struct alongside Ref<T>:

// src/query/references.rs
pub struct RefMut<'_world, '_world + Component + ?Sized> {
    entity: Uuid,
    value: T,
    world: WorldConnection, 
}

impl<'_world, T> Borrow<T> for RefMut<'_world, T> where T: '_world + Component + ?Sized {
    fn borrow(&self) -> &T {
        &self.value
    }
} 

impl<'_world, T> BorrowMut<T> for RefMut<'_world, T> where T: '_world + Component + ?Sized {
    fn borrow_mut(&mut self) -> &mut T {
        &mut self.value
    }
}

RefMut is conceptually similar to Ref, but with a few extra bells and whistles. First of all we're keeping track of the ID corresponding to the entity that the component is on, as well as a reference to the World it came from. With that, we have all the information we'll need to commit the component's contents back to the correct fields of the database.

Now for the more difficult task of detecting changes in the component data. A wise man once said, "Premature optimization is the root of all evil," so while I can certainly imagine many complex yet ergonomic solutions that will likely be difficult to test, I will instead opt for a simple solution and come back to it later if it performs poorly.

Instead of trying to detect change in internal state, we'll just assume that any resource requested as mutable will have changed at some point in the lifetime of the query result. Therefore, we can simply commit the component back to the database when the guard is dropped:

impl<T: Component + ?Sized> RefMut<T> {
    pub(crate) fn new(entity: Uuid, value: T, world: WorldConnection) -> Self {
        RefMut{ entity, value, world }
    }
    
    fn commit(value: &mut Self) -> Result<()> {
        todo!()
    }
}

impl<T> Drop for RefMut<T> where T: Component + ?Sized {
    fn drop(&mut self) {
        Self::commit(self).expect("Failed to commit on drop!");
    }
}

Since we've bound RefMut to the lifetime '_world, we can ensure at compile time that no RefMut instances will live longer than the database connection itself (of course, barring connection issues outside of our control). If the user wishes to handle any possible connection errors that arise, they can call commit directly.

Now that we have RefMut we can go ahead and write our implementation of WorldQuery for mutable components:

impl<'__w, T: Component> WorldQuery for &'__w mut T  {
    type Item<'_world> = references::RefMut<T>;

    fn build(b: &mut QueryBuilder) -> &mut QueryBuilder {
        b.with::<T>()
    }

    fn parse(r: &EntityUnchecked) -> Result<Self::Item<'static>> {
        if let Some(bytes) = r.component(&T::COMPONENTID) {
            T::from_bytes(bytes).map(|t| RefMut::new(
                r.id(),
                t,
                w, // how do we get access to a WorldConnection???
            ))
        }
        else {
            bail!("Single-component WorldQuery response did not contain the correct id")
        }
    }
}

Again, our code is the same as our implementation for &T which used Ref<T>, but there's already a snag: our Parse method doesn't provide us with World access! The simplest solution is to provide it access to a WorldConnection by modifying the trait item:

// in src/query/mod.rs::WorldQuery
fn parse(r: &EntityUnchecked, w: &mut World) -> Result<Self::Item<'static>>;

impl<'__w, T: Component> WorldQuery for &'__w mut T  {
    type Item<'_world> = references::RefMut<T>;

    fn build(b: &mut QueryBuilder) -> &mut QueryBuilder {
        b.with::<T>()
    }

    fn parse(r: &EntityUnchecked, w: &mut World) -> Result<Self::Item<'static>> {
        if let Some(bytes) = r.component(&T::COMPONENTID) {
            let w = w.get()?; //      <----- NEW
            T::from_bytes(bytes).map(|t| RefMut::new(
                r.id(),
                t,
                w,
            ))
        }
        else {
            bail!("Single-component WorldQuery response did not contain the correct id")
        }
    }
}

Now our RefMut instance has a handle to a database connection in a sane, thread-safe way.


Compound Queries

In the example API at the top of this post, I posited that my ideal usage of the query system which looks like this:

let results = <(&mut TA, Option<&TB>, Without<TC>, &TD)>::query(&mut World);
for (componentA, componentB, _, componentD) in results {
	...
}

... which can also be written like this:

for (ca, cb, _, cd) in <(&mut TA, Option<&TB>, Without<TC>, &TD)>::query(&mut World) {
     ...
}

These are tuples of types, which are actually their own types under rust's type system! We can use this to work generically over known-length tuples:

trait Transpose {
    type Output;
    fn transpose(self) -> Self::Output;
}

impl<T1, T2> Transpose for (T1, T2) {
    type Output = (T2, T1);
    fn transpose(self) -> Self::Output {
        (self.1, self.0)
    }
}

fn do_transpose() {
    let tup = (1usize, "hello".to_owned());
    let (a, b) = tup.transpose();
    assert_eq!(a, "hello");
    assert_eq!(b, 1usize);
}

This example implements the transpose method for every tuple of any two types!

In our application, we can combine this with a recursive trait implementation to abstract over tuples of any WorldQuery-able object. For example, take a tuple of two Queryables:

impl<T1, T2> WorldQuery for (T1, T2)
where
    T1: WorldQuery,
    T2: WorldQuery,
    {
        type Item<'_world> = (
            <T1 as WorldQuery>::Item<'_world>,
            <T2 as WorldQuery>::Item<'_world>,
        );
        fn build(b: &mut QueryBuilder) -> &mut QueryBuilder {
            <T1 as WorldQuery>::build(b);
            <T2 as WorldQuery>::build(b)
        }
        fn parse(r: &EntityUnchecked, w: &mut World) -> Result<Self::Item<'static>> {
            Ok((T1::parse(r, w)?, T2::parse(r, w)?))
        }
    }

This may look jarring and dense, so let's break it down:

impl<T1, T2> WorldQuery for (T1, T2)
where
    T1: WorldQuery,
    T2: WorldQuery,

Here, we declare we are implementing WorldQuery the same as before, except for a tuple of any two types. However, those two types must both implement WorldQuery themselves.

Next up let's look at what our Item will be.

type Item<'_world> = (
    <T1 as WorldQuery>::Item<'_world>,
    <T2 as WorldQuery>::Item<'_world>,
);

This declaration specifies that the output of querying this tuple is a tuple containing the respective outputs of each type in the original tuple. To fill in the (generic) blanks, if our tuple were (&usize, Option<String>) then our declaration would resolve to:

type Item<'_world> = (&usize, Option<String>);

Finally, moving on to build and parse:

fn build(b: &mut QueryBuilder) -> &mut QueryBuilder {
    <T1 as WorldQuery>::build(b);
    <T2 as WorldQuery>::build(b)
}

Because we require that T1 and T2 also implement WorldQuery, we don't need to do any special logic to build our query – all we do is chain our query builder into each sub-item's builder and implicitly return it at the end.

fn parse(r: &EntityUnchecked, w: &mut World) -> Result<Self::Item<'static>> {
    Ok((T1::parse(r, w)?, T2::parse(r, w)?))
}

Just like build, all we need to do in parse is successively call each member type's parse function and return those values assembled into a tuple. We also throw in some Try question mark operators to fast-exit if there's a problem.


That seems like a lot to type out though, and how many tuples do we even want to support? Even just 5-tuples would involve a lot of boilerplate writing!

Fortunately, Rust's macro system will come to our rescue:

macro_rules! impl_worldquery {
    ($($t:ident),+) => {
        impl< $($t),+ > WorldQuery for ($($t),+)
        where
        $($t: WorldQuery),+ {
            type Item<'_world> = ($(<$t as WorldQuery>::Item<'_world>),+);

            fn build(b: &mut QueryBuilder) -> &mut QueryBuilder {
                $(< $t as WorldQuery >::build(b));+
            }

            fn parse(r: &EntityUnchecked, w: &mut World) -> Result<Self::Item<'static>> {
                Ok(($(
                  $t::parse(r, w)?  
                ),+))
            }
        }
    };
}
📙
An explanation of macros is outside the scope of this post, and in fact could be its own post! If you want to learn more about macro_rules!, check out the Rust Documentation or Rust by Example.

Each time it is invoked, this macro will take that repetitive template we explored just now and applies it to one tuple of pre-defined length. We can call it 31 times (and manually implement the 1-tuple (T1,)) by repeating it several times:

impl<T1: WorldQuery> WorldQuery for (T1,) {
    type Item<'_world> = (<T1 as WorldQuery>::Item<'_world>,);

    fn build(b: &mut QueryBuilder) -> &mut QueryBuilder {
        T1::build(b)
    }

    fn parse(r: &EntityUnchecked, w: &mut World) -> Result<Self::Item<'static>> {
        Ok((T1::parse(r, w)?,))
    }
}

impl_worldquery!(T1, T2);
impl_worldquery!(T1, T2, T3);
impl_worldquery!(T1, T2, T3, T4);
impl_worldquery!(T1, T2, T3, T4, T5);
/* ... and so on, until (T1, .. T32) ... */

With this, we've implemented arbitrarily long queries, even if the amount of types exceed the maximum number of types in a single tuple! Users can simply query for tuples of tuples ( for example, ((T1, T2), (T3, T4)) ) thanks to our recursive query trait implementation.


Next Steps

Now that we've got the user-facing interface done, we can see that the compiler already accepts our library's API as valid rust! The following example snippet will compile successfully, but won't run since our backend isn't implemented.

fn test() -> Result<()> {
    use crate::*;
    let mut world = World::connect("postgres://diesel:superormpassword@localhost/codecells-dev")?;

    #[derive(serde::Serialize, serde::Deserialize)]
    struct CA;
    impl Component for CA {
        const COMPONENTID: uuid::Uuid = uuid::uuid!("4ae65ba3-2ed7-45d4-9de8-42396f845695");
    }

    #[derive(serde::Serialize, serde::Deserialize)]
    struct CB;
    impl Component for CB {
        const COMPONENTID: uuid::Uuid = uuid::uuid!("23877646-101c-4025-9d90-588d0d7644c6");
    }

    use crate::query::WorldQuery;
    for (a, b, c) in <(&CA, Without<CB>, &mut CB)>::query(&mut world)? {

    }

    Ok(())
}

In future posts, we'll begin exploring the backend for our ECS to get actual data back and forth from PostgreSQL. In a separate post, we'll also investigate some ergonomics additions to add to this library using Rust's procedural macros.

Until next time!
~Monty

Subscribe to monty.sh

Sign up now to get access to the library of members-only issues.
Jamie Larson
Subscribe