We present Air+Touch, a new class of interactions that interweave touch events with in-air gestures, offering a unified input modality with expressiveness greater than each input modality alone. We demonstrate how air and touch are highly complementary: touch is used to designate targets and segment in-air gestures, while in-air gestures add expressivity to touch events. For example, a user can draw a circle in the air and tap to trigger a context menu, do a finger 'high jump' between two touches to select a region of text, or drag and in-air ‘pigtail’ to copy text to the clipboard. Through an observational study, we devised a basic taxonomy of Air+Touch interactions, based on whether the in-air component occurs before, between or after touches. To illustrate the potential of our approach, we built four applications that showcase seven exemplar Air+Touch interactions we created.
For more media, see Anthony Chen's webpage.
Chen, X., Schwarz, J. Harrison, C., Mankoff, J. and Hudson, S. 2014. Air+Touch: Interweaving Touch & In-Air Gestures. In Proceedings of the 27th Annual ACM Symposium on User interface Software and Technology (Honolulu, Hawaii, October 5 - 8, 2014). UIST '14. ACM, New York, NY. 519-525.