Spatial Intelligence
Almost every bug in a spatial application is a pose bug. Content in the wrong place. Content that slides. A robot that thinks it is one aisle over. A result that is right on one device and wrong on another.
And almost every pose bug is a space bug. The number is usually correct. It is just expressed in a coordinate space nobody checked.
There are three of them. A coordinate is meaningless until you know which one it belongs to.
Session space is what an agent gets for free from its own motion estimate. A phone gets it from ARKit or ARCore. A robot gets it from wheel or visual odometry, the odom frame in ROS. A drone gets it from visual-inertial odometry.
These look like different systems and they are the same kind of frame: local, relative, and private to one run.
The origin is wherever that particular session happened to start. So a phone and a robot standing side by side in the same room hold two completely different sets of coordinates for the same physical spot. Neither is wrong. They are not comparable.
Session space is precise near the origin and less trustworthy the further the agent travels, because every motion estimate works by measuring change from the last known state and adding it up. Each measurement carries a small error and those errors compound. That is drift.
Nothing durable should ever be stored in session space. A session coordinate is valid for one run on one device and is meaningless the moment it restarts.
Map space is the frame of a map, a MapSet, or a map version. Its origin is tied to the physical space that was scanned, so it is the same today and next year, on a phone and on a robot.
This is the frame you author in, store in, and share in.
It is also what makes shared experiences work without syncing. Two people in the same room, localized against the same map, see the same object in the same physical place. A robot sent to those coordinates arrives at the same spot. Nobody negotiated anything. They are all reading one frame.
Global space is WGS 84: latitude, longitude, altitude and heading, optionally packaged as GeoPose.
It only exists for a map that has been georeferenced. It is the right frame for handing a position to a GIS, a fleet manager, or another platform. It is the wrong frame for placing anything precisely, because GPS on its own is accurate to metres rather than centimetres.
A transform converts a pose from one space to another, and each one has exactly one source.
The first row is the one that matters most. Before localizing, an agent knows how it has moved but has no idea where it is. A localization query answers that: send a camera frame, get back the camera’s pose in map space. From that moment the session frame is pinned to the map frame and map coordinates become directly usable.
That is also why localization is not a one-off setup step. Drift keeps accumulating, so you re-localize on a schedule to reset it.
This one costs people an afternoon, so it is worth stating plainly.
MultiSet returns poses in a left-handed, Y-up frame by default, the Unity convention. Set isRightHanded: true and you get the right-handed Y-up frame that ARKit, ARCore, WebXR and Three.js use.
But hint parameters are always supplied in the map’s native left-handed frame, whatever you set isRightHanded to.
So passing a right-handed hintPosition straight back from a previous result is the most common cause of a query that mysteriously stops finding a pose. Negate x first.
Nothing errors. The query just quietly stops working, which is the worst kind of bug.
Two rules fall out of all this, and they are the whole point.
Author in map space, always. An anchor is a pose in map space that something of yours hangs from: a model, a label, a waypoint, an inspection point, a trigger zone. Because map space is stable, an anchor authored once is valid in every future session, on every device, for every agent.
Store coordinates in map space, never in session space. This is the rule that separates a demo from a deployment. A demo can hold everything in session space and look perfect for ten minutes. A deployment cannot.
Physical spaces change. Furniture moves, a wing gets refitted, a scan ages.
Rescanning would normally produce a brand new map with a brand new origin, invalidating everything authored against the old one. Map versioning exists to prevent exactly that: a new scan is aligned to the base map and stored as a version of it rather than as a replacement, so the coordinate frame carries over and anchors, routes and overlays keep working without re-authoring.
A pairing worth remembering:
Use a MapSet to make a space bigger. Use a map version to make a space newer.
Session space is where an agent thinks it is. Map space is where it actually is. Global space is where that sits on Earth.
Localization is the bridge from the first to the second, and it is the only thing that stops drift accumulating forever.
Full detail, including the query shapes and per-agent integration patterns, is in How It Works in the docs. The vocabulary used here is defined in the glossary, and camera pose estimation covers what a pose is from first principles.
