Camera-Free Object Discovery and Interaction with GazeTags

Smart glasses, AI pins, earbuds and other head-worn devices offer the promise of always-available and proactive assistance. To achieve their full potential, these systems must understand what in the environment is relevant to the user. Gaze is one of the strongest signals for this purpose, often revealing a user's intent or task. This can be fine-grained eye-vector gaze (what specific object are they looking at) or coarser-grained head-vector gaze (what part of a scene are they facing towards). Systems with this information can adapt their user interfaces and offer timely assistance, such as retrieving information about appliances, surfacing instructions for tools, or reacting to changes in attended objects.

Today, however, gaze-aware wearables generally rely on cameras and computer vision. While powerful, this is especially problematic for small, battery-constrained wearable platforms. Cameras are relatively power-hungry sensors, and vision pipelines are also energy expensive, either when run locally or requiring transmission elsewhere for processing. As a result, continuous operation remains an outstanding challenge for compact devices such as glasses, earbuds, and pins. In practice, this most often means that vision-based sensing is either explicitly triggered by wearers for short periods, or launched periodically with a low duty cycle, limiting its usefulness for always-available contextual assistance.

A second major challenge of vision-powered gaze for wearables is privacy. Even when such systems are technically effective, their social acceptability can be limited by discomfort around image capture, uncertainty about what is being recorded, and concern over downstream analysis. This tension is particularly acute for wearable AI, where the ideal technical solution is persistent vision sensing. This could include video capture in sensitive contexts like bathrooms, where AI assistance could be invaluable for some users. However, active cameras in these contexts are presently non-starters for many users and bystanders.

In this paper, we explore a different path: achieving head-oriented gaze sensing without cameras or computer vision, sidestepping many significant power and privacy issues. Our proposed solution is GazeTags, a low-cost tagging approach for instrumenting environments so that head-worn devices can discover and interact with attended objects. A user-worn device periodically emits a narrow field-of-view infrared signal that is naturally aligned with the user’s head direction. Only tags within that head-gaze-correlated infrared "spotlight" wake and respond. In reply, tags announce both identity and dynamic payloads, such as object state, sensor readings, and contextual metadata. In this way, GazeTags can even move beyond computer-vision sensing, exposing non-visual states of objects, such as temperature, moisture, and power draw.

Of course, in order to be practically deployed across e.g., scores of objects in a home, GazeTags must be ultra-low-cost, small, and maintenance-free over long periods. Our implementation achieves these three properties. GazeTags cost around $2 USD each, including batteries, and could fall below $1 in high volumes. This makes dense instrumentation of everyday environments feasible (e.g., a home kitchen with 15 appliances would cost around $30 to augment). Second, GazeTags are small pucks, measuring 30 mm in diameter and 18 mm thick. They can be discreetly attached to objects of interest with magnets or tape. Third, GazeTags run for more than one year on a coin cell battery; when depleted, the battery can be replaced in seconds.

At a high level, our approach offers several advantages. Foremost, GazeTags do not require cameras, image processing, or computer vision models, and they are far better matched to the energy and compute budgets of small wearable devices. They also avoid many of the privacy concerns associated with always-on visual sensing, since no imagery needs to be captured or analyzed. Yet, GazeTags preserve a form of gaze-aware object discovery and interaction that is highly useful for wearable interfaces. We detail our implementation, followed by a multi-part evaluation. Our results show our system works up to ranges of 8 m and with many GazeTags operating simultaneously.

Download

Paper PDF

Reference

Mollyn, V., and Harrison, C. 2026. GazeTags: Head-Gaze-Aware, Dynamic-Payload, Low-Cost Tags to Inexpensively and Densely Activate Environments. In Proceedings of the 39th Annual ACM Symposium on User Interface Software and Technology (November 2 – 5, 2026). UIST '26. ACM, New York, NY.

Chris Harrison	Research Curriculum Vitae Fun Projects Travel
Camera-Free Object Discovery and Interaction with GazeTags Smart glasses, AI pins, earbuds and other head-worn devices offer the promise of always-available and proactive assistance. To achieve their full potential, these systems must understand what in the environment is relevant to the user. Gaze is one of the strongest signals for this purpose, often revealing a user's intent or task. This can be fine-grained eye-vector gaze (what specific object are they looking at) or coarser-grained head-vector gaze (what part of a scene are they facing towards). Systems with this information can adapt their user interfaces and offer timely assistance, such as retrieving information about appliances, surfacing instructions for tools, or reacting to changes in attended objects. Today, however, gaze-aware wearables generally rely on cameras and computer vision. While powerful, this is especially problematic for small, battery-constrained wearable platforms. Cameras are relatively power-hungry sensors, and vision pipelines are also energy expensive, either when run locally or requiring transmission elsewhere for processing. As a result, continuous operation remains an outstanding challenge for compact devices such as glasses, earbuds, and pins. In practice, this most often means that vision-based sensing is either explicitly triggered by wearers for short periods, or launched periodically with a low duty cycle, limiting its usefulness for always-available contextual assistance. A second major challenge of vision-powered gaze for wearables is privacy. Even when such systems are technically effective, their social acceptability can be limited by discomfort around image capture, uncertainty about what is being recorded, and concern over downstream analysis. This tension is particularly acute for wearable AI, where the ideal technical solution is persistent vision sensing. This could include video capture in sensitive contexts like bathrooms, where AI assistance could be invaluable for some users. However, active cameras in these contexts are presently non-starters for many users and bystanders. In this paper, we explore a different path: achieving head-oriented gaze sensing without cameras or computer vision, sidestepping many significant power and privacy issues. Our proposed solution is GazeTags, a low-cost tagging approach for instrumenting environments so that head-worn devices can discover and interact with attended objects. A user-worn device periodically emits a narrow field-of-view infrared signal that is naturally aligned with the user’s head direction. Only tags within that head-gaze-correlated infrared "spotlight" wake and respond. In reply, tags announce both identity and dynamic payloads, such as object state, sensor readings, and contextual metadata. In this way, GazeTags can even move beyond computer-vision sensing, exposing non-visual states of objects, such as temperature, moisture, and power draw. Of course, in order to be practically deployed across e.g., scores of objects in a home, GazeTags must be ultra-low-cost, small, and maintenance-free over long periods. Our implementation achieves these three properties. GazeTags cost around $2 USD each, including batteries, and could fall below $1 in high volumes. This makes dense instrumentation of everyday environments feasible (e.g., a home kitchen with 15 appliances would cost around $30 to augment). Second, GazeTags are small pucks, measuring 30 mm in diameter and 18 mm thick. They can be discreetly attached to objects of interest with magnets or tape. Third, GazeTags run for more than one year on a coin cell battery; when depleted, the battery can be replaced in seconds. At a high level, our approach offers several advantages. Foremost, GazeTags do not require cameras, image processing, or computer vision models, and they are far better matched to the energy and compute budgets of small wearable devices. They also avoid many of the privacy concerns associated with always-on visual sensing, since no imagery needs to be captured or analyzed. Yet, GazeTags preserve a form of gaze-aware object discovery and interaction that is highly useful for wearable interfaces. We detail our implementation, followed by a multi-part evaluation. Our results show our system works up to ranges of 8 m and with many GazeTags operating simultaneously. Download Paper PDF Reference Mollyn, V., and Harrison, C. 2026. GazeTags: Head-Gaze-Aware, Dynamic-Payload, Low-Cost Tags to Inexpensively and Densely Activate Environments. In Proceedings of the 39th Annual ACM Symposium on User Interface Software and Technology (November 2 – 5, 2026). UIST '26. ACM, New York, NY.
© Chris Harrison