--- title: A Brief Look at eXtended Reality Interaction Methods description: Let's explore some of the most common devices & methods to facilitate 3D eXtended Reality interaction methods utilizing cameras & handheld controllers. image: https://info.keylimeinteractive.com/hubfs/XR%20methods%20pic%201.png --- [![Key Lime Logo](https://info.keylimeinteractive.com/hs-fs/hubfs/Key%20Lime%20Logo.png?width=200&height=39&name=Key%20Lime%20Logo.png "Key Lime Logo")](https://keylimeinteractive.com/) - SERVICES - [All Services](https://keylimeinteractive.com/services) - [UX Strategy](https://keylimeinteractive.com/ux-strategy-design/) - [User Research](https://keylimeinteractive.com/user-research/) - [UX/UI Design](https://keylimeinteractive.com/ux-design-strategy/) - [EmTech Strategy](https://keylimeinteractive.com/emerging-technology/) - OUR WORK - [Our Approach](https://dev.keylimeinteractive.com/our-approach/) - [Enterprise Clients](https://keylimeinteractive.com/work/) - [Public Sector](https://keylimeinteractive.com/public-sector-gsa-schedule/) - [ABOUT US](https://keylimeinteractive.com/about-us/) - [Who We Are](https://keylimeinteractive.com/about-us/) - [People](https://keylimeinteractive.com/our-team/) - [Careers](https://workforcenow.adp.com/mascsr/default/mdf/recruitment/recruitment.html?cid=1eb079d3-bb50-438a-a731-eaa18a982193&ccId=19000101_000001&lang=en_US) - INSIGHTS - [CX Blog](https://info.keylimeinteractive.com) - [Events](https://keylimeinteractive.com/upcoming-events/) - [Competitive Indexes](https://keylimeinteractive.com/reports/#rankings) - [White Papers](https://keylimeinteractive.com/reports/) - [CONTACT US](https://keylimeinteractive.com/contact/) [![](https://keylimeinteractive.com/wp-content/uploads/2024/02/key-lime-interactive.svg)](https://info.keylimeinteractive.com/) - [What We Do](https://keylimeinteractive.com/ux-research-services/) - [Proven Process](https://keylimeinteractive.com/proven-process/) - [About Us](https://keylimeinteractive.com/about-us/) - [Meet Our Team](https://keylimeinteractive.com/our-team/) - [Careers](https://keylimeinteractive.com/careers/) - [News](https://keylimeinteractive.com/news/) - [Blog](https://info.keylimeinteractive.com/) - Resources - [Case Studies](https://keylimeinteractive.com/case-studies/) - [Events](https://info.keylimeinteractive.com/events) - [What We Do](https://keylimeinteractive.com/ux-research-services/) - [Proven Process](https://keylimeinteractive.com/proven-process/) - [About Us](https://keylimeinteractive.com/about-us/) - [Meet Our Team](https://keylimeinteractive.com/our-team/) - [Careers](https://keylimeinteractive.com/careers/) - [News](https://keylimeinteractive.com/news/) - [Blog](https://info.keylimeinteractive.com/) - Resources - [Case Studies](https://keylimeinteractive.com/case-studies/) - [Events](https://info.keylimeinteractive.com/events) [Contact us](https://info.keylimeinteractive.com/contact) [![](https://keylimeinteractive.com/wp-content/uploads/2024/02/key-lime-interactive.svg)](https://info.keylimeinteractive.com/) - [What We Do](https://keylimeinteractive.com/ux-research-services/) - [Proven Process](https://keylimeinteractive.com/proven-process/) - [About Us](https://keylimeinteractive.com/about-us/) - [Meet Our Team](https://keylimeinteractive.com/our-team/) - [Careers](https://keylimeinteractive.com/careers/) - [News](https://keylimeinteractive.com/news/) - [Blog](https://info.keylimeinteractive.com/) - Resources - [Case Studies](https://keylimeinteractive.com/case-studies/) - [Events](https://info.keylimeinteractive.com/events) - [What We Do](https://keylimeinteractive.com/ux-research-services/) - [Proven Process](https://keylimeinteractive.com/proven-process/) - [About Us](https://keylimeinteractive.com/about-us/) - [Meet Our Team](https://keylimeinteractive.com/our-team/) - [Careers](https://keylimeinteractive.com/careers/) - [News](https://keylimeinteractive.com/news/) - [Blog](https://info.keylimeinteractive.com/) - Resources - [Case Studies](https://keylimeinteractive.com/case-studies/) - [Events](https://info.keylimeinteractive.com/events) [Contact us](https://info.keylimeinteractive.com/contact) - - - # A Brief Look at eXtended Reality Interaction Methods [By Key Lime Interactive](https://info.keylimeinteractive.com/author/key-lime-interactive) March 04, 2021 [VR](https://info.keylimeinteractive.com/topic/vr), [XR](https://info.keylimeinteractive.com/topic/xr) - [Tweet](https://twitter.com/share) ![3D XR Interaction methods](https://info.keylimeinteractive.com/hubfs/XR%20methods%20pic%201.png) Recent advancements in the field of [eXtended Reality (XR)](https://keylimeinteractive.com/digital-reality-for-the-enterprise/) have given rise to a new era of interaction devices and techniques. Exploration and experimentation with eXtended Reality interaction methods has yielded some very interesting ways to interact in immersive environments with the goal of making the user experience (UX) more immersive. Some examples include [tracking gaze with eye tracking technology](https://imotions.com/blog/vr-eye-tracking/), using [neural interfaces to track gestures](https://venturebeat.com/2019/11/20/ctrl-labs-ceo-well-have-neural-interfaces-in-less-than-5-years/), camera based hand tracking, and obviously, handheld controllers. Although [eye tracking](https://info.keylimeinteractive.com/why-you-should-use-eye-tracking-for-website-optimization) and neural interfaces might require the least amount of physical effort and seem like the ideal choice, they are both limited in their abilities and often require a good bit of training before you can master them. Hand tracking using cameras and controllers, on the other hand, are far more natural and intuitive as they are similar to using our hands or handheld tools to do things in the real world. For example, grasping a cup of coffee or using a spatula to scramble eggs. In short, hand tracking and controllers allow us to interact with the virtual environment in a way that is more familiar to us. In this article we are going to look at some of the most common devices and methods to facilitate 3D interaction in [virtual reality (VR) and augmented reality (AR)](https://info.keylimeinteractive.com/whats-the-difference-between-vr-ar-and-mr)utilizing cameras and handheld controllers. ## Handheld Controllers To start off, let’s talk about the most common form of XR interactions in virtual worlds - handheld controllers. Since the early days of gaming, we have been using gamepads to control events in virtual worlds and it is only natural to port that over to immersive technology. With massive improvements to tracking technology in the last decade, we saw the rise of handheld controllers for use with VR. Some of the most commonly used controllers include the Wand controllers by HTC, Touch controllers by Oculus, the PlayStation Move controllers and the Valve Index controllers. All four of these provide both positional and rotational tracking, several buttons to control events and interact with virtual objects in the virtual world, and some level of haptic feedback. They provide a very compelling sense of presence and agency positively impacting the immersiveness of the experience. The Valve Index and Oculus Touch controllers also afford some level of finger tracking that further aids natural interaction. ![handheld controllers for immersive interactions](https://info.keylimeinteractive.com/hs-fs/hubfs/XR%20methods%20pic%202.png?width=936&name=XR%20methods%20pic%202.png) Virtual Reality Interaction and Augmented Reality Devices Popular handheld controllers for immersive interactions Although handheld controllers seem familiar based on years of gamepad usage, they are mostly prevalent in VR and often require some level of training. Moreover, using them can also get very complex and increase a user’s cognitive load depending on the interactions designed, virtual representation of the controller in VR, i.e. seeing an animated virtual hand instead of the controller, and past experience with unfamiliar devices. ## Hand Tracking The next set of interaction techniques that facilitate natural interaction in virtual worlds is the use of cameras to track hand movements. This is arguably more natural as this is the modality we are most used in everyday life. The technology has been on the rise lately and usually involves specialized cameras like depth and RGB cameras capturing images of your hands as you move them. These images are used to recognize how your finger joints and the palm are moving and how these movements are mimicked in virtual settings in real time. The level of movements tracked depends on the task a user needs to accomplish in the VR or AR. For example, a lot of AR applications only need to track finger position and pinching gestures as they often only require selection operations. On the other hands, VR applications employ more complex interactions and often include simple gestures and meticulous tracking of each finger joint as well. ![xr methods pic 3](https://info.keylimeinteractive.com/hs-fs/hubfs/xr%20methods%20pic%203.gif?width=400&name=xr%20methods%20pic%203.gif) Augmented Reality Interaction Simple pinch selection in Augmented Reality Source: [Crunchfish](https://www.youtube.com/watch?v=_K9iCCx8zNg&feature=youtu.be) One of the first devices to track hands for interaction in VR was the [Leap Motion sensor](https://www.ultraleap.com/product/leap-motion-controller/). The sensor had an array of sensors that would capture several types of images of your hands and used image recognition to recognize poses and gestures. The device was extremely versatile and the company has since used this technology to create more complex devices. A major contributing factor to hand tracking becoming increasingly popular is the use of [inside-out tracking](https://xinreality.com/wiki/Inside-out_tracking) with the help of cameras in several AR and VR headsets like the Microsoft HoloLens and Oculus Quest headsets. With the help of an array of onboard cameras, these headsets can also track our hand gestures and movements in real-time just like the Leap Motion sensor. However, one of the biggest drawbacks of this technology is that you lose tracking the moment your hands are outside the camera's field of view. This can be frustrating if the experience requires a lot of hand movement. The other limitation of this technology is the complete lack of haptics which can be jarring for complex interactions. Some experiences rely on proprioceptive and kinesthetic information to perform effectively like surgery training simulations. ![xr pic 4](https://info.keylimeinteractive.com/hs-fs/hubfs/xr%20pic%204.gif?width=400&name=xr%20pic%204.gif) Hand tracking in Virtual Reality Source: Oculus ![xr methods pic 5](https://info.keylimeinteractive.com/hs-fs/hubfs/xr%20methods%20pic%205.gif?width=400&name=xr%20methods%20pic%205.gif) Hand tracking in Augmented Reality Source: Microsoft ## Experimental Devices We are also seeing a rise in more customized devices that aim to afford more natural interaction with proprioceptive and kinesthetic information in the form of directed force feedback, haptic sensations from multiple sources, and shapes that closely resemble the virtual subject of the interaction. Most of these devices are not commercially available yet and are often byproducts of research explorations. Some good examples of these types of devices include the haptic revolves that aims to provide haptic feedback for touching surfaces in VR and this [3D printed fishing rod for VR](https://ayushbhargava.com/tangibles-in-vr-fishing/). Such devices are often created by combining different technologies like lighthouse tracking, 3D printing, actuators and smart tangibles. The end result is always impressive and highly immersive. Big tech companies have also invested in research relating to such devices and a recent byproduct of this interest are the [haptic gloves presented at CES 2021](https://www.forbes.com/sites/jenniferhicks/2021/01/11/at-ces-2021-these-haptic-gloves-could-change-virtual-reality-training-forever/?sh=ef36c667d4cd). I think it is safe to say that the future of interaction in virtual worlds is going to be very exciting and I for one am very excited. Moreover, I can’t wait to try some experiences that employ some of these interaction techniques in conjunction with other immersive technologies like haptic backpacks and olfactory senses. It might take a hot minute for such devices to be easily accessible but believe me it’ll be worth it! ###### More by this Author [Key Lime Interactive](https://info.keylimeinteractive.com/author/key-lime-interactive) ##### You might also like ![Emerging CX Trends in the Tech Industry](https://info.keylimeinteractive.com/hubfs/shutterstock_1955663605.jpg) [Emerging CX Trends in the Tech Industry](https://info.keylimeinteractive.com/emerging-cx-trends-in-the-tech-industry) March 04, 2021 ![Getting Started with Training in VR](https://info.keylimeinteractive.com/hubfs/Screen%20Shot%202020-02-10%20at%2011.28.15%20AM.png) [Getting Started with Training in VR](https://info.keylimeinteractive.com/getting-started-with-training-in-vr) March 04, 2021 #### Comments [Add Comment](https://info.keylimeinteractive.com/extended-reality-interaction-methods#comment-form) ABOUT KLI [Key Lime Interactive](https://keylimeinteractive.com/about-us)is a user experience research and service design agency, with a sweet spot for emerging technology. As UX experts, our goal is to make your life easier, optimize user experiences, and make the world a better place. - [Recent](https://info.keylimeinteractive.com/extended-reality-interaction-methods#tabs-0) - [Popular](https://info.keylimeinteractive.com/extended-reality-interaction-methods#tabs-1) ![Upsell with Purpose: Fueling SaaS Growth Through User Insight](https://info.keylimeinteractive.com/hubfs/Multiethnic%20group%20of%20young%20people%20sitting%20in%20conference%20room%20and%20brainstorming%20on%20business%20meeting.jpeg) [Upsell with Purpose: Fueling SaaS Growth Through User Insight](https://info.keylimeinteractive.com/upselling-in-a-saas-environment-with-a-user-centric-approach) Aug 21, 2024 ![5 Ways to Get the Most Out of Working with a UX Research Consultant](https://info.keylimeinteractive.com/hubfs/Attractive%20businessman%20speaking%20with%20fellow%20coworkers.jpeg) [5 Ways to Get the Most Out of Working with a UX Research Consultant](https://info.keylimeinteractive.com/5-ways-to-get-the-most-out-of-working-with-a-ux-research-consultant) Aug 21, 2024 ![Designing Engagement Through Micro-Interactions](https://info.keylimeinteractive.com/hubfs/AI-Generated%20Media/Images/A%20closeup%20shot%20of%20a%20finger%20gently%20tapping%20a%20digital%20button%20on%20a%20sleek%2c%20modern%20interface.jpeg) [Designing Engagement Through Micro-Interactions](https://info.keylimeinteractive.com/the-significance-of-micro-interactions-in-shaping-user-engagement-and-experience) Aug 21, 2024 ![What is the Keystroke-Level Model?](https://info.keylimeinteractive.com/hubfs/Keystroke.jpg) [What is the Keystroke-Level Model?](https://info.keylimeinteractive.com/what-is-the-keystroke-level-model) Aug 21, 2024 ![Current-State vs. Future-State Journey Mapping](https://info.keylimeinteractive.com/hubfs/destination-2386255_1280.jpg) [Current-State vs. Future-State Journey Mapping](https://info.keylimeinteractive.com/current-state-vs.-future-state-journey-mapping) Aug 21, 2024 ![A History of Voice Technology](https://info.keylimeinteractive.com/hubfs/Screen%20Shot%202020-08-11%20at%2012.18.34%20PM.png) [A History of Voice Technology](https://info.keylimeinteractive.com/history-of-voice-technology) Aug 21, 2024 Create a UX Strategy [![key-lime-interactive-ux](https://info.keylimeinteractive.com/hs-fs/hubfs/key-lime-interactive-ux.png?width=635&height=634&name=key-lime-interactive-ux.png)](https://keylimeinteractive.com/ux-strategy-design) ### Post By Topics - [Research Methods](https://info.keylimeinteractive.com/topic/research-methods) - [UX Strategy](https://info.keylimeinteractive.com/topic/ux-strategy) - [Events & News](https://info.keylimeinteractive.com/topic/events-news) - [UX Research](https://info.keylimeinteractive.com/topic/ux-research) - [User Experience](https://info.keylimeinteractive.com/topic/user-experience) - [User Interface](https://info.keylimeinteractive.com/topic/user-interface) - [UX](https://info.keylimeinteractive.com/topic/ux) - [News](https://info.keylimeinteractive.com/topic/news) - [Mobile UX](https://info.keylimeinteractive.com/topic/mobile-ux) - [Usability Testing](https://info.keylimeinteractive.com/topic/usability-testing) - [Competitive Research](https://info.keylimeinteractive.com/topic/competitive-research) - [UX Design](https://info.keylimeinteractive.com/topic/ux-design) - [Eye tracking & Biometrics](https://info.keylimeinteractive.com/topic/eye-tracking-biometrics) - [Qualitative Research](https://info.keylimeinteractive.com/topic/qualitative-research) - [Customer Experience](https://info.keylimeinteractive.com/topic/customer-experience) - [Journey Mapping](https://info.keylimeinteractive.com/topic/journey-mapping) - [DEI](https://info.keylimeinteractive.com/topic/dei) - [Remote Testing](https://info.keylimeinteractive.com/topic/remote-testing) - [User Experience Consultant](https://info.keylimeinteractive.com/topic/user-experience-consultant) - [VR](https://info.keylimeinteractive.com/topic/vr) - [XR](https://info.keylimeinteractive.com/topic/xr) - [Multicultural UX](https://info.keylimeinteractive.com/topic/multicultural-ux) - [Mobile Banking](https://info.keylimeinteractive.com/topic/mobile-banking) - [Quantitative Research](https://info.keylimeinteractive.com/topic/quantitative-research) - [App Development](https://info.keylimeinteractive.com/topic/app-development) - [User Experience Research Studies](https://info.keylimeinteractive.com/topic/user-experience-research-studies) - [Diversity Equity Inclusion](https://info.keylimeinteractive.com/topic/diversity-equity-inclusion) - [UI](https://info.keylimeinteractive.com/topic/ui) - [Voice Technology](https://info.keylimeinteractive.com/topic/voice-technology) - [Accessibility](https://info.keylimeinteractive.com/topic/accessibility) - [Behavioral Personas](https://info.keylimeinteractive.com/topic/behavioral-personas) - [Diary Study & Ethnography](https://info.keylimeinteractive.com/topic/diary-study-ethnography) - [Global Research](https://info.keylimeinteractive.com/topic/global-research) - [Inclusivity Index](https://info.keylimeinteractive.com/topic/inclusivity-index) - [User Testing](https://info.keylimeinteractive.com/topic/user-testing) - [Card Sort & Tree Test](https://info.keylimeinteractive.com/topic/card-sort-tree-test) - [Human-first Approach](https://info.keylimeinteractive.com/topic/human-first-approach) - [Personalization](https://info.keylimeinteractive.com/topic/personalization) - [CX](https://info.keylimeinteractive.com/topic/cx) - [Diary Study](https://info.keylimeinteractive.com/topic/diary-study) - [In-Depth Interviews](https://info.keylimeinteractive.com/topic/in-depth-interviews) - [Inclusivity](https://info.keylimeinteractive.com/topic/inclusivity) - [A/B Testing](https://info.keylimeinteractive.com/topic/a-b-testing) - [CX Research](https://info.keylimeinteractive.com/topic/cx-research) - [IDIs](https://info.keylimeinteractive.com/topic/idis) - [Bias](https://info.keylimeinteractive.com/topic/bias) - [Personas](https://info.keylimeinteractive.com/topic/personas) - [Research Participant Recruitment](https://info.keylimeinteractive.com/topic/research-participant-recruitment) - [Usability](https://info.keylimeinteractive.com/topic/usability) - [Women in UX](https://info.keylimeinteractive.com/topic/women-in-ux) - [Workshops](https://info.keylimeinteractive.com/topic/workshops) - [Customer Journey Mapping](https://info.keylimeinteractive.com/topic/customer-journey-mapping) - [Remote Research](https://info.keylimeinteractive.com/topic/remote-research) - [Research Analysis](https://info.keylimeinteractive.com/topic/research-analysis) - [UX Methods](https://info.keylimeinteractive.com/topic/ux-methods) - [Webinars](https://info.keylimeinteractive.com/topic/webinars) - [Anthropology](https://info.keylimeinteractive.com/topic/anthropology) - [Competitive Benchmarking](https://info.keylimeinteractive.com/topic/competitive-benchmarking) - [Competitive Insights](https://info.keylimeinteractive.com/topic/competitive-insights) - [Competitive Intelligence](https://info.keylimeinteractive.com/topic/competitive-intelligence) - [Confirmation Bias](https://info.keylimeinteractive.com/topic/confirmation-bias) - [Ethnography](https://info.keylimeinteractive.com/topic/ethnography) - [Gaming](https://info.keylimeinteractive.com/topic/gaming) - [Healthcare CX Design](https://info.keylimeinteractive.com/topic/healthcare-cx-design) - [Healthcare Customer Experience Design](https://info.keylimeinteractive.com/topic/healthcare-customer-experience-design) - [Heuristics](https://info.keylimeinteractive.com/topic/heuristics) - [Unmoderated](https://info.keylimeinteractive.com/topic/unmoderated) - [sample size calculation](https://info.keylimeinteractive.com/topic/sample-size-calculation) - [AI development](https://info.keylimeinteractive.com/topic/ai-development) - [Biometrics](https://info.keylimeinteractive.com/topic/biometrics) - [Communication](https://info.keylimeinteractive.com/topic/communication) - [Competitive Intelligence Analysis](https://info.keylimeinteractive.com/topic/competitive-intelligence-analysis) - [Concept Testing](https://info.keylimeinteractive.com/topic/concept-testing) - [Consultant](https://info.keylimeinteractive.com/topic/consultant) - [Custom Projects](https://info.keylimeinteractive.com/topic/custom-projects) - [Dedicated Services](https://info.keylimeinteractive.com/topic/dedicated-services) - [Digital Divide](https://info.keylimeinteractive.com/topic/digital-divide) - [Impact](https://info.keylimeinteractive.com/topic/impact) - [Keystroke Level Model](https://info.keylimeinteractive.com/topic/keystroke-level-model) - [Literature Review](https://info.keylimeinteractive.com/topic/literature-review) - [Negotiation in UX](https://info.keylimeinteractive.com/topic/negotiation-in-ux) - [Nonprofit UX](https://info.keylimeinteractive.com/topic/nonprofit-ux) - [Presentation](https://info.keylimeinteractive.com/topic/presentation) - [Rapid Research](https://info.keylimeinteractive.com/topic/rapid-research) - [Report](https://info.keylimeinteractive.com/topic/report) - [SaaS](https://info.keylimeinteractive.com/topic/saas) - [UX Templates](https://info.keylimeinteractive.com/topic/ux-templates) - [World Usability Day](https://info.keylimeinteractive.com/topic/world-usability-day) - [digital design](https://info.keylimeinteractive.com/topic/digital-design) - [persona](https://info.keylimeinteractive.com/topic/persona) - [user engagement](https://info.keylimeinteractive.com/topic/user-engagement) ![ ](https://keylimeinteractive.com/wp-content/uploads/2024/05/aicpa-soc-certified.png) ![Logo of the National Minority Supplier Development Council with colorful abstract shapes and text.](https://keylimeinteractive.com/wp-content/uploads/2024/05/NMSDC-Logo-NATIONAL-1024x716.jpg) ![Logo with the text "Insights Association" and colorful squares forming a pattern above the text.](https://keylimeinteractive.com/wp-content/uploads/2024/02/insights-association-logo-v2.png) ![](https://keylimeinteractive.com/wp-content/uploads/2024/02/key-lime-interactive.svg) Your Trusted Partner for UX Research Key Lime Interactive - [What We Do](https://keylimeinteractive.com/ux-research-services/) - [Proven Process](https://keylimeinteractive.com/proven-process/) Resources - [Case Studies](https://info.keylimeinteractive.com/case-studies) - [Events + Workshops](https://info.keylimeinteractive.com/events) - [Blog](https://info.keylimeinteractive.com/) Company - [About Us](https://keylimeinteractive.com/about-us/) - [Meet Our Team](https://keylimeinteractive.com/our-team/) - [Careers](https://keylimeinteractive.com/careers/) - [News](https://keylimeinteractive.com/news/) - [Security & Compliance](https://app.vanta.com/keylimeinteractive.com/trust/cylnxu23wx6pzvej7bloh9) - [Public Sector GSA](https://keylimeinteractive.com/public-sector-gsa-schedule/) Get in touch - [Contact Us](https://keylimeinteractive.com/contact) © 2025 All Rights Reserved. - [Privacy Policy](https://keylimeinteractive.com/privacy-policy/) - [Terms of Use](https://keylimeinteractive.com/terms/)