Creation of Haptic Virtual Reality

Although recent progress of information technology has remarkably augmented the quality of communication, haptic (tactile and force) information is not incorporated in the contemporary communication media. The information content handled in the communication system is essentially for humans so that it should be optimized to the form humans can receive efficiently. All the sensation channels they have, the five senses, need to be engaged in communication to fully exploit the information channels of humans. Communication system for haptic channel has long been under development for several decades to extend mostly the function of our ’hand.’ However, haptic communication system is not necessarily available yet for ordinary people who work in a broad field since it is not applicable to general tasks.

Human haptic sensation is a sensor of mechanical information around the body and it enables manipulation of physical environment based on obtained information. The sensing is directed to both environmental mechanical interaction and inner status of own body. The haptic sensation consists of tactile and proprioceptive sensations each of which has own receptors. Information acquired by these receptors is integrated in central nerve system to consistently perceive physical status of both the body and the environment. Then the perceived information is provided to perform physical functionality of manipulation of environment based on actuation of the body. Exerted force, motion induced, and interaction at the surface are sensed and controlled based on operator’s intention. Thus the haptic sensation inevitably involves the control of body and environment. It is generated with a complex construct based on diverse interaction.

Communication occurs everywhere in the living organism and it is indispensable for its control. The communication related to the living organism is classified into two; an inner communication and an outer communication where the density of information flow differs. The former is for control of the body and the latter is for physical environment and interpersonal interaction. Communication technology for haptics usually focuses on the latter.

Communication is viewed as multilayered structure in its functional levels such as semantic (symbolic) interaction, outer manipulation, inner control, and basic signal levels. Haptic information is in more control side than symbolic side. It is received basically, then sent out to make a dynamic loop of cybernetic organization. Haptic information discussed in the later section is more in basic reception level since high quality haptic information has not been presented yet on a communication terminal specifically in the sense of tactile one.

Contact status and force exerted between the body and the environment obtained by mechanoreceptors. It is a raw data for other levels. Not only tactile sensation at the contact surface but proprioception due to the force exerted at the surface.

Inner control level

Kinetic status of the human body monitored by somatic sensation. The body is controlled by real-time feedback of this status information. It is a basis of manipulation of environmental objects. Some parts of this information are obtained by removing environmental status from the interaction between the body and the environment. The environmental status is considered to be removed based on the consistency of information of the operator’s body.

Outer manipulation level

Information on the continuous manipulation of environmental objects based on inner control. It is obtained by removing the information of inner control level. It involves the information of the body so that the boundary of inner/outer information is ambiguous. Accuracy of the information depends also on the model of the object the operator has.

Semantic level

Symbolic treatment of lower level information to other people as in a language. Haptic interactions such as a tap, a push, a pull, a scratch, a stroke, a twist, etc. have their own meaning in the specific context and in more general cultural conventions. Both language equivalent and proper representation are involved. Lexical range is small beyond comparison with a general language.

3. Implementation and use of haptic display

Haptic information terminal is not established even in the sense of the basic signal level. It is still in the early stage towards high quality presentation. Some developments in this course specifically focused on tactile sensation are presented in this and later sections.

Haptic display is to be built to represent both tactile and proprioceptive sensations. Either channel is not easy to implement since spatiotemporal and kinetic range of haptic display is very large. Tactile sensation arouses based on a very small stimulus; an order of mm change is easily detected. Receptors in the skin cover all parts of the body that has a complicated shape which changes its geometry very rapidly. The range and velocity of kinetic motion are broad in the three-dimensional real space. So far many types of haptic display have been proposed, almost all of which are intended to impart limited range stimulation to a single channel.

Three displays introduced in the present paper focus mainly on producing a realistic tactile sensation to the fingertip that is the most sensitive part of the body surface and the principal channel in manipulation of objects. Integration of tactile and proprioceptive presentations was examined with the tactile and force(proprioceptive) compound haptic displays. Both displays are among those with the highest resolution and reality ever developed. They are introduced in the next section.

Research prototypes built by the author are in the first stage to create tactile sensation and its consistent rendering with a force reflection. The next section describes three basic designs of haptic display.

The skin sensation is generated chiefly by the stimulus added from the environment on the surface of a skin. The stimulation can be applied through either mechanical or electrical stimulation. Mechanical stimulation produces more localized and natural sensation of touch than electrical stimulation which activates peripheral nerves under the skin and often causes a pain. The authors have developed mechanical stimulation haptic displays which have a vibratory pin array contacted to the fingertip. Each pin is independently controlled in its amplitude and frequency of vibration. This design is targeted to produce texture sensation that we receive when we touch the surfaces with relatively small geometrical variation. Roughness (or smoothness) is the typical sensation that we experience with such a variation.

Three displays, a high density desktop type (TextureDisplay2R), and force reflection types (TextureExplorer, TD44FX) are described in the following sections.

TextureDisplay2R shown in Fig. 1 is designed specifically for high definition tactile sensation. The display has a dense window in which fifty vibratory pins are arranged in a 5 x 10 array with a 2-mm inter-pin distance [1]. Each pin is driven by a piezoelectric actuator. The amplitude is controlled in 256 levels by changing the duty ratio of the PWM power output. The peak amplitude as a function of vibration frequency is shown in Fig. 2. Around 280 Hz is there a resonant frequency that is set there to optimize the amplitude at 250 Hz where skin sensation indicates highest sensitivity [2]. The highest output frequency is 410 Hz where peak amplitude maintains above 10 mm, and the lowest frequency is 40 Hz where another sort of mechanoreceptor exhibits high sensitivity [3].

The TextureExplorer, shown in Fig. 5 [4], is a compound haptic display consisting of a vibratory pin-array and the Phantom (1.5AG) force display that is able to produce both cutaneous and proprioceptive stimulations. During exploration on a surface of a virtual object, the user can perceive stimulus variation within a surface at the fingerpad as well as reflection force from the object at the hand. This unique design simulates a real exploration of a surface by the finger, in which the skin contacts the object at multiple points in a surface area with different intensity of stimuli under a constraint force from the surface.

The system hardware consists of three parts, a visual display, a tactile stimulator and a force display, respectively. For visual presentation, a stereoscopic view is provided by an 80-inch rear projection screen using shutter glasses. A 3-D motion sensor (FASTRAK, Polhemus Inc.) is attached to the glasses to enable head tracking. For tactile stimulation, a vibratory pin-array display (Fig. 6) is attached to the stylus of the PHANToM with a handle to grip by the palm. The 3-degree of freedom (DOF) rotation of the pin-array is measured by another motion sensor. Ten pins synchronously deliver the three-dimensional distribution of stimulus at a limited area of the fingerpad. The force display imparts the information related to shape, texture and stiffness of the object based on the mode to the finger and palm simultaneously with cutaneous pin stimulation.

4.3. TD44FX (TextureDisplay44 force extension)

One more degree of freedom of force presentation was added in design of TD44FX shown in Fig. 7. Two small force displays (Phantom Omni) were incorporated to drive a pin-array that has forty-four contact pins for a fingertip. Each force display has three-DOF positioning. The position of the pin stimulator is controlled up to five DOF which produces moment of force during touch to the object surface. A three dimensional surface can be explored with more natural force feedback including a torque occurs from friction between the virtual finger and objects.

Part of the present research was performed under the support of the special research fund of TMIT and the Grant-in-Aid for Scientific Research. The author would like to thank contributors of this study in the lab: Mariko Yamada, Shigeki Arata, Masashi Shiratori, and project support members.

Fig. 1. TextureDisplay2R (TD2R)

Fig. 3. Equal sensation magnitude curves

Fig. 4. Texture data displayed on TD2R

Fig. 7. TD44FX: Texture Display with 44-pin stimulator driven by two force displays.