PinPoint is a multi-platform spatial intelligence tool designed for the showroom. Built on Snap Spectacles and Snap Cloud, it lets customers attach voice notes to real products as they browse, then turns those briefs into a structured preference profile, surfaced to the salesperson through a live web portal before the conversation even starts.
I formed the team and led the ideation in March. As a result, PinPoint won 2nd place at XR Creator Con (XRCC) 2026. As the product manager and full-stack engineer in the team, I led the team in defining the features and scope, and wiring the data flow from the AR frontend through the backend to the web frontend.
As an enthusiast and practitioner working actively in the smart glasses space, I’d like to write this as a record of my hands-on thinking and practice. So this essay is less about the hackathon win and more about what building for smart glasses taught me about designing for the form factor itself. If you are building applications for glasses, read along. Your feedback is welcome.
The starting point of PinPoint is a problem that already exists: in reality, language is fragmented and often stripped of context. We speak in pieces, pointing at things that are only clear in the moment, and most of that meaning is gone as soon as the moment passes.
This becomes a real pain point in a place that’s dense with objects and heavy with verbal feedback, like a kitchen showroom. When a customer says "I love this handle but the color is too cold," the words alone carry almost nothing. Which handle? Cold compared to what? Dr. Fei-Fei Li describes language as a purely generated signal that is one-directional, sequential, and lossy. Strip away the shared context and what remains flattens a 3D, embodied experience into a flat string of words.
Smart glasses, with vision and audio modalities working together can restore the missing dimensions. They are able to capture various sources of references at the moment of speech: what you are looking at, where you are standing, what you are saying, and when it happened, all bound to the words in real time.
PinPoint does not simply transcribe speech. It binds language to the real-world context it came from, turning something fleeting and fragmented into data that is retrievable and organised.
This points to the real power of multimodal capture on smart glasses, in a consented setting of course: vision, voice, gaze, and position, all bundled hands-free at the point of attention, without breaking the browsing flow. A phone could take a photo with a voice memo, or a video, but you would still have to stitch it together later, never at the point of gaze, hands-free. The binding has to happen as the words are spoken, otherwise the context is already gone.
PinPoint was designed around a simple conviction: give customers something lightweight if that genuinely helps, for people who would like to engage with new technology but definitely not a heavy headset. Naturally, smart glasses are the perfect fit. We did consider letting the salesperson wear the glasses, but two things ruled it out. First, what would they actually need them for? Second, we believe the key still lies in human communication, where eye contact from the salesperson matters, and eyewear with cameras on, pointed at customers, can easily make them feel uncomfortable.
Once we accept this product setting, the form factor starts dictating the architecture. We designed the glasses to be a sensing and display layer, so the AR frontend processes as little as possible. Input flows from the device's native modalities through an intelligent layer of independent edge functions in Snap Cloud, where product detection, intent extraction, and catalog matching happen, and lands on a web portal where the relevant specialist views and analyzes it.
I’ve always believed that AR applications are not an isolated island. They can partner with other conventional technologies. PinPoint is built on that belief: each surface carries what it is natively good at. The AR part in glasses are spatial, personal, and in-the-moment. The database and cloud is where raw modality data becomes structured intent. The web portal is a screen, because the salesperson's job is analysis and triage across many customers, not immersion.
The value lives in the flow between these surfaces, and together they close a loop: the customer's reactions in AR become structured preferences in the cloud, and matched products flow back from the web portal to be visualised in AR right in front of the customer. The system is the core product, and that is what we are most proud of.
For fellow builders, I would suggest this: let the product and the form factor shape each other, and match every surface to its job and its social context. Smart glasses are still early and many people, businesses and customers alike, will meet them for the first time through products like this one. That puts a quiet responsibility on us as builders: design experiences that respect consent, disappear into the moment, and keep the human conversation at the center rather than replacing it. If PinPoint makes one case, it is that the best use of glasses is not to put more in front of your eyes, but to make more of what you already see and say.
At last, I would like to shout out my teammates, who trusted the idea and dedicated themselves to bringing it to people and validating it in front of the public in my absence: Jiayao Yu, Felicity Chen, Seonjeong Park, and Anna Zielinska. So proud of women's power. And a shoutout to the mentors, judges, and organisers who gave us feedback and the opportunity to showcase our strength.