朝菌学园ZhaoDesign
朝菌学园是一家专注前沿、未来、智能设计领域的品牌。包含人机交互设计、服务设计、用户体验设计、互动媒体装置、智能产品设计等学习方向，已经帮助数百名学员取得考研、保研与留学申请的名校录取offer
930篇原创内容
Official Account
，
01 摘要
Abstract
用于AR/VR较为广泛的触觉反馈装置可分为两类：带有单个振动驱动器的简单手持式控制器，或带有多个嵌入式驱动器的复杂手套系统。
在此项目中，我们探索了手腕上用以与虚拟物体互动的触觉反馈。我们使用了Tasbi——一款小巧的手镯装置，能够呈现复杂的多感官挤压和振动触觉反馈。
通过Tasbi的触觉渲染、头戴式显示器的标准视觉和音频渲染，我们展示了将感官替代触觉与视觉和音频线索紧密结合在一起的几种交互，包括按下/松开按钮、旋转旋钮、纹理、刚性体重和惯性，以及一些自定义的双手操作，例如射箭。
这些演示表明，与无触觉反馈相比，基于手腕的触觉反馈极大地改善了AR/VR中的虚拟手势交互。
Most widespread haptic feedback devices for augmented and virtual reality (AR/VR) fall into one of two categories: simple hand-held controllers with a single vibration actuator, or complex glove systems with several embedded actuators. In this work, we explore haptic feedback on the wrist for interacting with virtual objects. We use Tasbi, a compact bracelet device capable of rendering complex multisensory squeeze and vibrotactile feedback. Leveraging Tasbi’s haptic rendering, and using standard visual and audio rendering of a head mounted display, we present several interactions that tightly integrate sensory substitutive haptics with visual and audio cues. Interactions include push/pull buttons, rotary knobs, textures, rigid body weight and inertia, and several custom bimanual manipulations such as shooting an arrow from a bow. These demonstrations suggest that wrist-based haptic feedback substantially improves virtual hand-based interactions in AR/VR compared to no haptic feedback.
02 关键词
Author Keywords
触觉；可穿戴设备；多感觉的；手腕；虚拟现实
haptics; wearables; multisensory; bracelet; virtual reality
03 CCS分类
CCS Concepts
• 以人为本的计算→虚拟现实
• 硬件→触觉设备
•Human-centered computing → Virtual reality; •Hardware → Haptic devices;
04 介绍
Introduction
通过使用便携、全天候可穿戴的眼镜或头戴式显示器来呈现复杂的三维视觉效果和空间化音频内容，增强现实和虚拟现实界面将革命性地改变人机交互。
AR/VR界面与传统二维屏幕之间的显著差异带来了一些问题——过去的交互设备(如键盘、鼠标和控制器)可能不再合适或有效。基于与三维内容交互的标准方法，我们如果使用视觉、听觉等多感觉搭配的触觉反馈，可以改善与虚拟物体间的交互。Tasib可以提供丰富的触觉刺激加以证明、探索、和说明以上假设。
Augmented and virtual reality interfaces are positioned to revolutionize human-computer interaction by rendering complex three-dimensional visuals and spatialized audio content using portable, all-day-wearable, glasses or headmounted displays. Stark differences between AR/VR interfaces and conventional two-dimensional screens present problems where interaction devices of the past (e.g. keyboards, mice, and controllers) may no longer be suitable or effcient. Building upon the standard methods of interacting with three-dimensional content [1], we hypothesize that multisensory haptic feedback [2], carefully paired with audio/visual stimuli may improve virtual object interactions. These demonstrations aim to explore and inform this hypothesis using the rich haptic stimuli provided by Tasbi.
05 多感觉触觉手环
Tasibi
Tasbi(图1)是一款多感觉触觉手环，包含两种触觉模式:振动和挤压。振动通过六个径向和均匀间隔的线性谐振驱动器(LRAs)传递，而响应式挤压则由位于背侧的张力机制提供。
这一机制的特点是一个新颖的绳索缠绕系统，该系统可以在一个相对较小的包装尺寸中提供给Tasbi较高的力输出。设计和实现的详细信息可以在[3]中找到。虽然Tasbi有许多潜在的应用，但我们主要关注的是它作为增强和虚拟现实的触觉反馈设备的实用性。
实际上，我们最感兴趣的是研究基于腕力的反馈，这种反馈是对感受力的感官的替代，否则这些力就会发生在手和脚尖上。类似的概念已经在截肢者的假肢反馈背景下进行了探索，但在AR/VR背景下进行的工作相对较少。此展示与摘要展示了我们对各种交互的早期探索。
Tasbi (Fig. 1) is a multisensory haptic bracelet that incorporates two haptic modalities: vibration and squeeze. Vibration is delivered though six radially and uniformly spaced linear resonant actuators (LRAs), while responsive squeeze is provided by a dorsally located tensioning mechanism. This mechanism features a novel cord winding system that affords Tasbi high force output in a relatively small package size. Design and implementation details can be found in [3]. While Tasbi has a number of potential applications, our primary focus has been on its utility as a haptic feedback device for augmented and virtual reality. Specifcally, we are most interested in investigating wrist-based feedback for sensory substitution of forces that would otherwise occur at the hands and fngertips. Similar concepts have been explored in the context of prosthetic feedback for amputees, but relatively little work has been done in the context of AR/VR. This abstract and demonstration presents our early explorations across a variety of interactions.
图1:Tasbi
(a)是一种多感觉触觉手环，它能在手腕周围产生强大的、有反应的挤压
(b)以及六个等间距径向位置的振动
(c)本演示探索了将Tasbi的基于手腕的触觉技术与视觉和音频刺激相结合，以在VR中替代手和手指的感官交互。
先前的演示
Previous Demonstration
Tasbi之前在2019年世界触觉大会上展示过——通过虚拟按钮和一个独立的弓箭交互提供的刚度反馈。之后，通过在相对的手腕上引入第二腕带，可以实现双手反馈。
为了探索这一领域，我们已经开发了大量新的引人注目的双手交互。此外，我们还设计了涉及旋转、扭曲和拉伸运动的交互反馈，以及物体表面、物体重量和惯性渲染。
Tasbi was previously demonstrated at the 2019 World Haptics Conference. This demo featured unimanual feedback for stiffness rendering via virtual push buttons and a separate bow and arrow interaction. Since then, bimanual feedback has been enabled through the introduction of a second bracelet on the opposite wrist. A number of new compelling two-handed interactions have been developed to explore this space. In addition, we have designed feedback for interactions involving twisting and pulling motions, as well surface and object weight and inertia rendering.
06 多感觉的反馈模式
Multisensory Feedback Paradigm
大多数互动都遵循一个简单的三部分“公式”。
首先，触觉振动呈现离散事件，如手碰撞、冲击和高频内容，通常与音频效果一致。通常情况下，Tasbi中的所有六个振动器都会同时被更强的刺激所驱动，但在适当的时候，我们会使用一个子集来传递方向性(参见力场)。
第二部分为连续的相互作用力利用比例挤压。例如，当按下虚拟按钮时增加挤压以渲染其硬度，或者当拿起物体时增加挤压以传达其质量。
第三个也是最后一个元素是视觉。在许多最新的虚拟现实游戏中，我们采用了“上帝对象”原则[4]，既可以防止虚拟手穿透物体，又可以计算交互作用力。我们通过伪触觉的概念来扩展这一观点，即在真实(控制)的手和虚拟(显示)的手之间添加人工差异，以表明某些动作需要更多的力。
之前的研究表明，通过控制显示(C/D)与挤压相结合的比例，可以提高用户对虚拟“刚度”[3]的感知。将所有这三种感官刺激结合起来，可以产生比单感官方法更有说服力的互动。
We adhere to a simple three-part “formula” for most interactions. First, haptic vibration renders discrete events such as hand collisions, impacts, and high frequency content, often in concert with audio effects. Usually, all six vibrotactors in Tasbi are simultaneously actuated for a stronger stimulus, but when appropriate we use a subset to convey directionality (see Force Fields). The second part leverages proportional squeeze for continuous interaction forces. For example, increasing squeeze as a virtual button is pressed to render its stiffness, or squeezing when objects are picked up to convey their mass. The third and fnal element is visual. Like many of the latest VR games, we employ the “god-object” principle [4] to both prevent virtual hands from penetrating objects, and to compute interaction forces. We extend this idea through a concept known as psuedohaptics, where artifcial discrepancies between the real (control) hand, and the virtual (display) hand are added to suggest that certain actions require more physical effort. We have previously shown that manipulating the controlto-display (C/D) ratio congruently with squeeze improves users’ perception of virtual “stiffness” [3]. Combining all three sensory stimuli can lead to more compelling interactions than would be possible with a unisensory approach.
07 原始的交互
Primitive Interactions
通过虚拟按钮的例子，我们的多感官方法很容易理解(图2)。当用户的虚拟手指第一次接触其表面时，就会呈现低振幅振动。按下按钮时，挤压的大小与它的位移成正比。通过计算C/D比来增加用户在某个按钮位置上必须实现位移的数量，会给人一种按钮更难按或“更硬”的印象。额外的振动可以沿着按钮的运动范围和旅行结束，以提高真实感。
我们把这个按钮看作是推动对象的基本交互的一个例子。我们也探索了其他原始的运动（指在现实生活中的实物运动），例如以旋钮的形式扭动，以及以拉把手的形式拉动(图3)。尽管每种交互的触觉反馈都是完全相同的，但我们发现，用户对每种原始交互的理解通常是不同的，这与相关的视觉环境有关。因此，Tasbi是足够灵活的，它能够适应多种类型的交互。
Our multisensory approach is easily understood through the example of a virtual push button (Fig. 2). When the user’s virtual fnger frst makes contact with its surface, a low amplitude vibration is rendered. As the button is pressed, squeeze increases proportional to its displacement. Changing the C/D ratio to increase the amount users must extend for a given button displacement gives the impression that the button is harder to press, or “stiffer”. Additional vibrations can be played along the button’s range of motion and end of travel for heightened realism. We consider the button an example for the primitive interaction of pushing objects. We have also explored other primitive motions, such as twisting in the form of knobs, and pulling in the form of pull handles (Fig. 3). Although the haptic feedback for each of these is exactly the same, we have found that users generally interpret each primitive interaction differently given the context of the associated visuals. As such, Tasbi is fexible enough to accommodate many types of interactions.
图2：我们的一般反馈策略结合了手腕振动(用于离散接触和高频事件)、挤压(用于连续力量)和视觉伪触觉错觉(即控制/显示操作)。
图3：旋转旋钮(a)和拉把手(b)使用的方法与图2中的按钮范例类似，分别用于扭转和拉动运动。
08 表面和纹理
Surfaces and Textures
通过振动执行器对虚拟表面和纹理进行绘制是一个深入探索的领域。由于Tasbi配备了6个LRA，它可以实现许多现有的纹理渲染算法。然而，一般来说，振动主要用于渲染精细的表面细节——凸起、裂缝、脊等。与几何相关的粗糙细节绘制表面通常保留给接地的动觉类型设备，在物理上限制或引导手部运动，Tasbi作为一个不接地的可穿戴设备，它不能施加净力/扭矩，然而，我们发现挤压提供了一个令人惊讶且令人信服的方式来渲染表面法线。
考虑图4中的块。所有的块都采用了前面提到的视觉和触觉“上帝对象”渲染技术，其中挤压与代理手的穿透深度成正比。当用户的手扫过表面时，穿透深度和挤压力会发生变化。
对于较大的振幅块,这种变化在挤压传达表面高度相当不错。对于较小的振幅块体，在每个峰值处则采用典型的振动效应。中间的块代表了挤压和振动表面渲染的平衡，可以说是三者中最引人注目的。
Rendering virtual surfaces and textures through vibrating actuators is a thoroughly explored area. Because Tasbi is equipped with six LRAs, it could implement many existing texture rendering algorithms. Vibration in general, however, is mostly confned to rendering fne surface details – bumps, cracks, ridges, etc. Rendering surfaces with coarse details related to geometry is typically reserved to grounded kinesthetic-type devices that physically restrict or guide hand motion. Tasbi, as an ungrounded wearable, cannot exert net forces/torques, yet, we fnd squeeze provides a surprisingly compelling way to render surface normals. Consider the blocks in Fig. 4. All blocks employ the previously mentioned visual and haptic "god-object" rendering technique, where squeeze is proportional to the proxy hand penetration depth. As the user sweeps their hand across the surface, penetration depth and thus squeeze force change. For the larger amplitude blocks, this change in squeeze conveys surface height reasonably well. For the smaller amplitude blocks, typical vibration effects are employed at each peak. The middle block represents a balance of both squeeze and vibration based surface rendering, and is arguably the most compelling of all three.
图4：定量用于渲染fne细节纹理(a)，而squeeze提供了渲染球场、基于几何的表面的能力(c)。两者可以和(b)结合。
09 物体的重量与惯性
Object Weight and Inertia
对于所有不接地的触觉设备来说，如何真实地传达物体的重量和惯性感是非常具有挑战性的。当然，如果不与物理世界联系起来，就不可能呈现向下的重力，或线性惯性/角惯量的力/力矩。
一些手套型设备可能会试图通过向手指施加一种与重力方向或加速度相反方向的力来呈现一种伪重量。这里的挑战是，施加的力/力矩必须接地到手的其他部分，同时我们不希望的反作用力可能导致知觉的不连续。相反，径向挤压提供了一种有趣的方法，因为它是自接地的，而且力只被感觉为局部的内向压力。
一种简单的方法是将物体重量和空隙的范围映射到挤压水平的范围。我们发现这种静态方法是相当无效的。相反，渲染动态效果会产生更引人注目的交互。
在图5中，当玩家拿起网球或球拍时，会施加一个基本的挤压量，但除此之外，当用户在半空中挥动这些物体时，这些物体的惯性会通过挤压的小函数呈现。对于网球拍，我们也增加了与倾斜角度成比例的挤压，传递一种力矩手臂扭矩的感觉并传给手腕。然而，具有挑战的是适当地衡量每种现象对总挤压的贡献，因为只有自由度是可用的。
A challenge for all ungrounded haptic devices is realistically conveying the sense of object weight and inertia. Of course, it is impossible to render the downward force of weight, or the forces/torques of linear/angular inertia, without being connected to the physical world. Some glove-type devices may attempt rendering a type of psuedo-weight by applying forces to the fngers in the direction of gravity or in the opposite direction of acceleration. The challenge here is that the applied forces/torques must be grounded to other parts of the hand, where undesired reaction forces may result in perceptual discontinuities. In contrast, radial squeeze offers an interesting approach because it is self grounding, and forces are only felt as localized inward pressure. A simple approach would map a range of object weight and intertia to a range of squeeze levels. We have found this static approach to be rather ineffective. Instead, rendering dynamic effects result in a much more compelling interaction. In Fig. 5, a base amount of squeeze is applied when the tennis ball or racket is picked up, but additionally, the inertia of these objects is rendered through small fuctuations in squeeze as the user waves them in mid-air. For the tennis racket, we also increase squeeze proportional to the tilt angle, conveying a sense of the moment arm torque it would impart to the wrist. One challenge, however, is appropriately scaling each phenomenon’s contribution to total squeeze since only on degree of freedom is available.
图5：刚体的重量和惯量可以通过挤压函数来表示。
10 力场
Force Fields
我们对没有物理交互点时力场的渲染进行了探索。图6中的风扇说明了这一点。用户的手在气流前面移动时，挤压增加来传递空气压力的感觉，并且与风扇与手之间的距离成比例变化。光的振动呈现在面对风扇的振动器上，以传达局部的风力效应。
We have also explored rendering force felds for phenomena that have no physical points of interaction. The fan in Fig. 6 demonstrates this. When the users’ hand moves in front of the air stream, squeeze increases to convey air pressure, and changes proportional to the distance between the fan and hand. Light vibrations are rendered on vibrotactors facing the fan to convey localized wind effects.
图6：与物理接触无关的力，如来自风扇的气流，手腕会产生挤压和振动。
11 用户移动时的感觉
User Movement and Proprioception
许多VR游戏通过让玩家抓住周围的环境并将自己拉向所需的方向来解决移动问题。如果玩家不专注于自己的目标，或者专注于其他任务，本体感受就会被打破，失去位置感。
挤压可以通过传送沿着路径行进的距离来减轻这种影响。在图7所示的梯子例子中，每次抓取都会呈现出振动，当用户将自己拉上每个横木时，挤压也会增加，释放时复位，从而呈现出一种攀爬的感觉。
Many VR games solve the locomotion problem by allowing players to grab nearby surroundings and pull themselves in a desired direction. However, if the player is not fxated on their target, or is also focused on other tasks, proprioception can break down and the sense of locality will be lost. Squeeze can mitigate this by conveying distance traveled along a path. In the ladder example of Fig. 7, vibrations are rendered with each grab, and squeeze increases as users pull themselves up each rung, resetting on release, and thereby giving a sensual presentation of climbing.
图7：梯子的交互，挤压会随着用户提升而增加，探索运动和增强本体感受。
12 双手交互
Bimanual Interactions
Tasbi最引人注目的交互作用是在两个手腕上提供反馈。当交互作用模拟双手之间的反应力时，这是最明显的。
举个例子，使用弓和箭(图8)：在弓手上渲染挤压来模拟稳定力，而增加弦手的挤压来模拟弓张力的建立。在大多数的例子中，通过微妙的振动和C/D的操纵，体验得到了极大的增强。
更复杂的交互作用也是可能的，如图9所示范的手枪。它具有单手触感效果,如扳机拉动时的挤压和振动,以及用双手的效果,例如当玩家用另一只手扳动滑块或拧下消音器时的挤压力。
The most compelling Tasbi interactions are those that render feedback on both wrists. This is most apparent when the interaction simulates reactive forces between hands. For example, using a bow and arrow (Fig. 8): squeeze is rendered on the bow hand to simulate stabilizing forces, while increasing squeeze on the string hand simulates the build-up of bow tension. As in most examples, the experience is substantially enhanced with subtle vibration and C/D manipulation. Even more complex interactions are possible, as shown in the pistol demonstration (Fig. 9). It features unimanual haptic effects, such as squeeze for trigger pull and vibration for fring, and bimanual effects such as squeeze forces when the player uses the opposite hand to cock the slide or twist off the silencer.
图8：用弓和箭演示了双手的相互作用。当弓被拉出时，两个手腕会受到越来越大的挤压。微妙的振动给体验带来更多的真实感。
图9：手枪展示了一个更复杂的双手物体，它具有多个互动点，如拧下消音器或倾斜滑动。
13 演示格式
Demonstration Format
这里讨论的每个交互都可以进行演示。交互被组织和包含在独立的虚拟孤岛上，用户可以根据自己的兴趣在虚拟孤岛上移动。
有两个独立的演示站可用。演示参与者将戴上商用VR头显。因为我们目前依靠VR控制器来追踪手和手指，所以理想情况下，用户应该熟悉基本的基于控制器的抓取和手势机制。在Tasbi穿戴过程中需要短时间校准。
Each of the interactions discussed here are available to demo. The interactions are organized and contained on separate virtual islands which users can move between depending on their interests. Two separate demo stations are available. Demo participants are expected to wear a commercial VR headset. Since we currently rely on VR controllers for hand and fnger tracking, users should ideally have some familiarity with basic controller-based grab and gesture mechanics. A short calibration is required during the Tasbi donning process.
14 参考文献
REFERENCES
[1] Doug A. Bowman, Ernst Kruijff, Joseph J. LaViola, and Ivan Poupyrev. 2004. 3D User Interfaces: Theory and Practice. Addison Wesley Longman Publishing Co., Inc., USA.[2] Karon E. MacLean, Oliver S. Schneider, and Hasti Seif. 2017. The Handbook of Multimodal-Multisensor Interfaces. Association for Computing Machinery and Morgan & Claypool, New York, NY, USA, Chapter Multisensory Haptic Interactions: Understanding the Sense and Designing for It, 97–142. DOI: 查看链接 ↗
[3] Evan Pezent, Ali Israr, Majed Samad, Shea Robinson, Priyanshu Agarwal, Hrvoje Benko, and Nick Colonnese. 2019. Tasbi: Multisensory Squeeze and Vibrotactile Wrist Haptics for Augmented and Virtual Reality. In IEEE World Haptics Conference. 1–6. DOI: 查看链接 ↗
[4] Craig B. Zilles and J. Kenneth Salisbury. 1995. A Constraint-based God-object Method for Haptic Display. In IEEE International Conference on Intelligent Robots and Systems (IROS), Vol. 3. 146–151 vol.3. DOI: 查看链接 ↗