VPS
Put a robot and a headset in the same room. Both are localizing. Both are confident. Neither one agrees on where "here" is. The robot set its origin the moment it powered on. The headset picked a different one when you put it on. Ask them to meet at the same point and they walk to two different places, each certain it's right.
I've watched a version of this break a demo in real time. Two devices, one space, two private worlds. Everything worked in isolation. Nothing worked together.
Here's the other version. A handheld barcode scanner reads ten thousand codes a shift, flawlessly. Ask it where in the building any single scan happened and it has nothing. The read is perfect. The location is missing.
Both failures come down to the same missing piece: camera pose, and whether it's expressed in a frame anyone else can read. This post walks it from the ground up, from the intrinsics and extrinsics most engineers first hit during calibration, through 6-DoF pose, tracking, and positioning, to the shared map that lets a fleet of devices, and eventually AI agents, agree on one space. I'm writing for CV and ML engineers, robotics and XR developers, and the enterprise architects who have to make localization work across many devices, not one device on a bench.
Practical first, technical second, and still worth reading if you came for the math.
Before the arguments, the map of terms. Keep this handy. Most confusion in localization comes from mixing up which of these lives on the device and which lives in a shared frame.
Notice the split. The first two rows and camera tracking are things a device can do alone. Camera positioning and relocalization need something outside the device: a map that persists and that other devices can read. That gap is the whole story.
You can't talk about pose without intrinsics and extrinsics, so let's be precise about both. This is the part developers usually meet first, and it's where a surprising number of downstream problems are actually born.
Camera intrinsics are the internal parameters of a camera: focal length, principal point, and lens distortion. They describe how the camera projects a 3D point in front of it onto a 2D pixel. Get them wrong and every geometric step after them inherits the error. Bad intrinsics poison every pose downstream.
Camera extrinsics describe where the camera is and how it's oriented, expressed as a rotation and a translation relative to some reference frame. Intrinsics are about the camera in isolation. Extrinsics are about the camera in a world.
Here's where this stops being a textbook exercise. We've had developers reach out with exactly this problem, phrased almost word for word: they need to determine the extrinsics for a camera array. Several fixed cameras, each individually calibrated, and they still can't get the rig to place itself correctly in the building.
That's not a calibration bug. It's a frame problem. You can calibrate each camera's extrinsics relative to the rig perfectly, and the array as a whole still has no idea where it sits in the world, because nothing tied that rig to a coordinate frame the rest of your system shares. Per-camera extrinsics answer "where is camera 3 relative to camera 1." They don't answer "where is this array in the facility, in the same frame my headset and my robot use."
Resolving that is a localization problem, not a bench-calibration one. Each camera, once it can see mapped space, gets its 6-DoF pose against one shared world frame. For this post, hold the takeaway: extrinsics are only as useful as the frame they resolve into.
Camera pose is a camera's full position and orientation in space: six degrees of freedom. Three for translation (where you are: x, y, z) and three for rotation (which way you're facing: roll, pitch, yaw). That's what "6-DoF" means. Six independent numbers that together pin a camera down completely.
Camera bearing is just the rotation half. It tells you the direction the camera faces. Bearing is useful, but on its own it's the answer to half the question. You know which way you're looking. You don't know where you're standing.
A camera pose isn't a location. It's an agreement. Six numbers are meaningless until you say which frame they're in. My pose in the frame my device invented on startup can't be compared to your pose in the frame your device invented on startup. Same room, incompatible answers. The moment more than one device is involved, the frame matters more than the numbers.
How do you actually recover pose from a camera? A few method families, and where each one shines and struggles. I'll keep this honest rather than exhaustive.
The pattern across the top four rows: they tell a device how it moved. They give a private, drifting, relative frame. That's not a criticism. It's what they're designed to do. The bottom row is different in kind, not degree. Relocalizing against a prior map is the only one that hands you an absolute pose that another device, or the same device tomorrow, can share.
This is the distinction that trips up most teams, and it's the one worth internalizing.
Camera tracking follows how a camera moves through space, frame to frame. It's on-device, relative, and it drifts. Start at an origin, integrate motion, and small errors compound the further you travel. Tracking is excellent at telling one device how it moved from where it started.
Camera positioning, done with a visual positioning system (VPS), tells the camera where it is in an environment by matching what it sees against a prebuilt map. The output is an absolute pose in a shared, persistent frame. Not "you moved 4 meters from your origin," but "you are here, in the building, in the same coordinates everyone else uses."
VIO propagates. VPS corrects.
Take the handheld scanner from the opening. We were introduced to a team trying to estimate the pose of handheld barcode scanners to track their movement through a space. On-device tracking gets you part of the way: the scanner can sense it's moving. But over a shift, across aisles, through pickups and set-downs, relative tracking drifts and every reset loses the thread. What that team actually needs is the scanner's 6-DoF pose in the facility's shared frame. Give it that and every scan becomes a located event, not just a code read. You know not only what was scanned but where the operator was standing when they scanned it, in the same map your other devices use.
Tracking device movement in a space, reliably, takes three things a tracker alone can't give you: a persistent map to localize into, relocalization when the device is picked up or loses tracking, and drift correction mid-motion. That's a positioning problem.
If you only remember one thing from this article: on-device tracking tells a device how it moved. It cannot tell two devices they're in the same place. That takes a shared map.
Before I sound like I'm dismissing tracking, I'm not. SLAM and VIO are some of the best engineering in the field, and for a large class of problems they're exactly right.
If you need a single device to understand its own motion in real time, with no prior setup and no connectivity, on-device tracking is the answer. It's fast, it's self-contained, and it handles motion that a pure image-matching approach would choke on. A drone holding position, a headset smoothing head motion between corrections, a robot dead-reckoning through a gap in coverage. Tracking isn't legacy and it isn't a compromise. It's a production tool with a clear job.
The gap it doesn't close is shared, persistent, cross-device state. Tracking gives every device its own excellent private answer. It never makes those answers agree. The right architecture isn't tracking or positioning. It's tracking corrected by positioning: the device propagates its own motion and periodically relocalizes against the shared map to kill drift and stay in the common frame. Both, doing what each is good at.
Now push this one rung up, because it's where the whole field is heading.
The AI agents and world models getting funded right now are racing to make machines that reason and act in the physical world. They're getting genuinely good at understanding a space and deciding what to do in it. But an agent that reasons perfectly still fails if it doesn't know where it is. Cognition and grounding are two different problems, and almost everyone is solving the first while assuming the second.
An agent pose estimate is an embodied agent's position and orientation in a shared world frame. Reasoning doesn't produce it. The agent needs to retrieve it, the same way a language model retrieves facts.
That comparison is exact, and it's worth sitting with. Enterprises run many language models and never retrain one every time knowledge changes. Too slow, too costly. Instead they put a retrieval layer underneath, feeding each model the right context at the moment it answers. The model brings intelligence. The layer brings memory and context.
Physical AI hits the same wall for the same reason. You won't run one world model or one robot brain. You'll run several, from different vendors, across phones, headsets, and robots. The moment there's more than one, you need a neutral grounding layer none of them owns, so no device has to re-map a space another device already knows. Language models retrieve context so they don't retrain. Agents query a shared map so they don't re-map.
That shared map is the destination this whole post has been walking toward. Camera pose, expressed in one frame every machine can read, is what lets a fleet of devices, and the agents running on them, operate in the same reality instead of a thousand private ones.
No system is magic, and a pillar post that pretends otherwise isn't worth your time.
Positioning against a map depends on the map. In feature-poor spaces, long blank corridors, uniform warehouse racking, freshly painted walls, there's less for the system to match against, and accuracy suffers. Heavy lighting swings between a sunny afternoon and a dark night shift make matching harder. And a map is a snapshot: if a space changes and the map doesn't, localization degrades until the map catches up.
The right operational response to all of this is not denial. It's map currency and honest coverage. Scan the space well, keep the map versioned as the environment changes, and validate accuracy under the conditions you'll actually run in, not just the demo conditions. We built map versioning precisely because real facilities move, and a stale map is a slow failure rather than an obvious one.
Two honest boundaries first, because they set expectations correctly.
We don't ship a bench calibration toolkit. If your task is calibrating a rig on a table with a checkerboard, that's a different tool. What we do is resolve each device's pose against one shared world frame through localization. And we don't read your barcode. The scanner reads the code. We tell the scanner where it was standing when it did.
With that clear, here's what we are. MultiSet is an independent, scan-agnostic visual positioning system. Scan-agnostic means we ingest whatever reality-capture format you already have, across more than ten of them, with no proprietary capture pipeline you're forced to adopt. Independent means we belong to no platform: we're the layer beneath your devices, not a competitor to any of them. And deploy-anywhere means the operator owns the frame and the data, from public cloud to private VPC to self-hosted to on-device to fully air-gapped.
That combination isn't a feature list. It's an architectural commitment: one map, many devices, one shared frame, owned by you.
The proof:
That last point matters because plenty of companies now say "shared coordinate system" and "ground truth." The difference is we have it in production, ranked independently, with no crowd-sourced capture to depend on.
No. SLAM builds and tracks a private map as a device moves, giving it a relative frame that's specific to that session. Camera pose estimation against a prebuilt map relocalizes a device into a shared, absolute frame that other devices and later sessions can use. They're complementary: SLAM handles self-motion, positioning handles shared ground truth.
Yes. Once each camera can see mapped space, it localizes against a common map and receives its 6-DoF pose in one shared world frame. This is different from bench extrinsic calibration, which only relates cameras to each other, not to the environment your other devices operate in.
No. One map serves many devices across platforms. A phone, a headset, a robot, and a handheld scanner can all localize into the same frame. You only update the map when the physical space changes, and map versioning handles that without a full re-map.
Camera pose looks like a math problem until you put a second device in the room. Then it becomes an agreement problem, and agreements need something shared to point at. Intrinsics and extrinsics get one camera honest. A shared map gets your whole fleet, and the agents running on it, into the same reality.
If you're working on any of this, from camera arrays to handheld tracking to grounding an AI agent, start free and localize your first space, or book a demo and we'll talk through your deployment.
