WO2025028139A1 - Information processing system, information processing method, and force sense presentation device - Google Patents

Abstract

An information processing system according to the present disclosure comprises: a grip detection unit that, regarding a force sense presentation device having a force sense presentation unit that presents a predetermined force sense by varying the center of gravity of a housing with respect to a user gripping the housing, detects the grip state of the housing by the user; and a control unit that controls the operation of the force sense presentation unit on the basis of the detected grip state.

Description

Information processing system, information processing method, and force feedback device

This disclosure relates to an information processing system, an information processing method, and a force feedback device.

Technology that allows for a realistic virtual reality (VR) experience has become widespread. A known example of such VR-related conventional technology is a controller that allows a user to feel the weight (also known as force sensation) of various tools held within a VR space.

International Publication No. 2023/095475

However, the above-mentioned conventional technology has the problem that it is difficult to determine how the user is holding the controller. As a result, depending on the way the user holds the controller, it may not be possible to present the correct force feedback.

The present disclosure provides an information processing system, an information processing method, and a force feedback device that can present a force feedback according to the user's gripping state.

Note that the above problem or objective is merely one of several problems or objectives that can be solved or achieved by the multiple embodiments disclosed in this specification.

The information processing system disclosed herein is a force-feedback presentation device having a force-feedback presentation unit that presents a predetermined force sensation to a user holding the housing by varying the center of gravity of the housing, and includes a grip detection unit that detects the state in which the housing is being held by the user, and a control unit that controls the operation of the force-feedback presentation unit based on the detected grip state.

1 is a block diagram showing a configuration example of an information processing system according to an embodiment of the present disclosure. FIG. 2 is a perspective view showing the appearance of a controller. FIG. 2 is a schematic diagram showing a pressure distribution sensor deployed. 11 is an explanatory diagram illustrating an example of attachment of a pressure distribution sensor to a controller. FIG. FIG. 4 is an explanatory diagram illustrating variable center of gravity of the controller. FIG. 11 is an explanatory diagram illustrating estimation of the position of a hand holding a controller. 11 is an explanatory diagram illustrating the operation control of a center-of-gravity variable portion. FIG. FIG. 13 is an explanatory diagram for explaining calculation of the position of a center-of-gravity variable part. 10A and 10B are explanatory diagrams illustrating an example of the operation of a center of gravity variable unit. 10A and 10B are explanatory diagrams illustrating an example of the operation of a center of gravity variable unit. 10A and 10B are explanatory diagrams illustrating an example of the operation of a center of gravity variable unit. 10A and 10B are explanatory diagrams illustrating examples of vibration of a vibration presentation unit. FIG. 11 is an explanatory diagram illustrating an example of moment control in a controller. FIG. 11 is an explanatory diagram illustrating an example of moment control in a controller. FIG. 11 is an explanatory diagram illustrating an example of a feedback loop of moment control. 11 is an explanatory diagram illustrating an example of adjusting the strength of a force presented by the controller. FIG. FIG. 11 is an explanatory diagram for explaining estimation of a grip angle of the controller. FIG. 11 is an explanatory diagram illustrating estimation of a swing speed of a controller. FIG. 11 is an explanatory diagram for explaining estimation of the end of a controller swing. 11 is an explanatory diagram illustrating an example of an interaction according to a force with which the controller is gripped. FIG. 11 is an explanatory diagram illustrating an example of an interaction according to a force with which the controller is gripped. FIG. 11 is an explanatory diagram illustrating an example of an interaction according to the position of the hand holding the controller. FIG. 11 is an explanatory diagram illustrating an example of an interaction according to the position of the hand holding the controller. FIG. FIG. 13 is an explanatory diagram illustrating an example of application to a virtual musical instrument. FIG. 4 is an explanatory diagram illustrating an example of a button operation on a controller. FIG. 4 is an explanatory diagram illustrating an example of a button operation on a controller. FIG. 4 is an explanatory diagram illustrating an example of a pressure detection method in a controller. FIG. 2 is an explanatory diagram illustrating an example of operation buttons on a controller. 10 is a flowchart illustrating an operation example of the information processing system according to an embodiment of the present disclosure. FIG. 2 is a hardware configuration diagram illustrating an example of a computer.

The following describes in detail the embodiments of the present disclosure with reference to the accompanying drawings. Note that in this specification and drawings, components that have substantially the same functional configuration are designated by the same reference numerals to avoid redundant description.

In addition, in this specification and drawings, similar components of the embodiments may be distinguished by adding at least one different alphabet and number after the same reference numeral. However, if there is no particular need to distinguish between each of the similar components, the same reference numeral will be used.

One or more of the embodiments (including examples, variations, and application examples) described below can be implemented independently. However, at least a portion of the embodiments described below may be implemented in appropriate combination with at least a portion of another embodiment. These embodiments may include novel features that are different from one another. Thus, these embodiments may contribute to solving different purposes or problems and may provide different effects.

FIG. 1 is a block diagram showing an example configuration of an information processing system according to an embodiment of the present disclosure.

As shown in FIG. 1, an information processing system 1 according to an embodiment of the present disclosure includes a controller 10, a control device 81, a camera 82, and a VR device (display device) 83. The control device 81 is connected to each of the controller 10, the camera 82, and the VR device 83 by wire or wirelessly.

The control device 81 is realized by a computer. The control device 81 holds an App (application) 812 in a storage unit (not shown), and a processing unit can be realized by executing the App 812 by a processor (not shown).

Typically, App 812 may be a game application. However, the type of App 812 is not limited to a game application. For example, App 812 may be an application other than a game application.

App 812 progresses based on information transmitted from the controller 10, information transmitted from the VR device 83, or images captured by the camera 82. App 812 then controls the output from the VR device 83 based on its own progress. For example, the output from the VR device 83 may include a screen display from a display device or audio output from a speaker.

Here, the information transmitted from the controller 10 may correspond to information for controlling the execution of App 812. For example, the information transmitted from the controller 10 may include information indicating that a button provided on the controller 10 has been pressed, or may include information detected by a sensor provided on the controller 10. Similarly, the information transmitted from the VR device 83 may include information indicating that a button provided on the VR device 83 has been pressed, or may include information detected by a sensor provided on the VR device 83.

More specifically, App 812 constructs a VR space. An avatar corresponding to the user exists in the VR space. App 812 controls the movement of the avatar within the VR space based on information transmitted from the controller 10, information transmitted from the VR device 83, or an image captured by the camera 82. At this time, an image of the VR space is provided from App 812 to the VR device 83, and is displayed by the VR device 83, allowing the VR space to be visually recognized by the user.

Also, App 812 transmits various control signals to the controller 10. Furthermore, the control device 81 stores an SDK (Software Development Kit) 813 in a storage unit (not shown). The SDK 813 is a component necessary for executing App 812.

Typically, the control device 81 may be a game machine. However, the control device 81 is not limited to a game machine. For example, the control device 81 may be a PC (Personal Computer) or the like.

The camera 82 has an image sensor, and obtains an image by capturing an image of the imaging range with the image sensor. More specifically, the camera 82 obtains a plurality of frames (i.e., moving images) by capturing images of the imaging range continuously in time series with the image sensor. For example, the controller 10 and the VR device 83 may be present within the imaging range of the camera 82.

It is assumed that the camera 82 is typically installed independently of other devices (e.g., the controller 10, the control device 81, and the VR device 83). However, the camera 82 may be integrated with other devices. For example, the camera 82 may be provided in the VR device 83 or in the control device 81.

The VR device 83 performs output according to the control of App 812. For example, the VR device 83 includes a display device, and performs screen display on the display device according to the control of App 812. Alternatively, the VR device 83 includes a speaker, and performs audio output from the speaker according to the control of App 812.

Typically, the VR device 83 can be worn on the user's head when in use. However, the VR device 83 does not necessarily have to be worn on the user's head.

In the example shown in FIG. 1, the VR device 83 is connected to the control device 81. However, the form of the VR device 83 is not limited thereto. For example, the VR device 83 may be used by setting a smartphone in goggles. Alternatively, the VR device 83 may be in a form that operates as a headset alone.

The controller 10 is an example of a force feedback device that provides a predetermined force feedback to a user holding the housing. For example, the controller 10 is used as a housing for a virtual tool in a VR space. Examples of virtual tools include a gun (e.g., a shotgun), a sword (e.g., a Japanese sword), and a wind instrument (e.g., a flute or a saxophone).

Under the control of the control device 81, the controller 10 presents a predetermined force sensation to the user holding the housing, providing the user with a sense of realism as if they were actually operating a virtual tool (gun, sword, wind instrument, etc.) in the VR space.

The controller 10 includes a pressure detection unit (pressure distribution sensor) 110, a microcomputer 120 (control unit), a center of gravity variable unit (movable unit) 130, a vibration presentation unit 140, and a battery 150.

In addition to the above configuration, the controller 10 is equipped with various sensors, such as a gravity sensor for detecting the attitude of the controller 10 relative to the vertical direction (device angle), a position sensor using a GPS (Global Positioning System) or the like, and a temperature sensor. The gravity sensor in the controller 10 is an example of an angle detection unit.

The pressure detection unit 110 is a pressure sensor provided on the surface of the housing, and detects the gripping position and gripping force when the user grips the housing.

FIG. 2 is a perspective view showing the external appearance of the controller 10. As shown in FIG. 2, the housing E1 of the controller 10 is, as one example, a cylindrical body. The shape of the housing E1 is not limited to a cylindrical body. For example, the housing E1 of the controller 10 can take various shapes, such as a pillar-like body such as a hexagonal prism, or a sphere.

Pressure distribution sensors 110a and 110b are attached to the surface of the housing E1. As an example, the pressure distribution sensor 110a is attached to the upper side of the illustrated example and the pressure distribution sensor 110b is attached to the lower side of the surface of the housing E1 from the center toward both ends.

By attaching multiple pressure distribution sensors 110a, 110b to the surface of the housing E1 in this manner, it is possible to detect the gripping position and gripping force when the user grips the housing E1 across the entire surface of the housing E1.

FIG. 3 is a schematic diagram of the pressure distribution sensors 110a and 110b deployed. As shown in FIG. 3, the pressure distribution sensors 110a and 110b have, for example, 10×16 detection points and detect pressure values at each detection point.

In addition, the pressure distribution sensors 110a and 110b may be equipped with functions to detect not only pressure values but also shear stress, temperature, acceleration, and the like. For example, shear stress can be detected by a strain sensor, temperature by a thermistor, and acceleration by an acceleration sensor.

Figure 4 is an explanatory diagram illustrating an example of attaching a pressure distribution sensor to a controller. As shown in case a of Figure 4, the controller 10a has a vertically long shape. For this reason, the pressure distribution sensor attached to the surface of the housing E1 of the controller 10a may be divided into multiple parts, such as pressure distribution sensors 110c, 110d, and 110e. In this case, the wiring and flexible cables (not shown) of the pressure distribution sensors 110c, 110d, and 110e may be embedded in the housing E1.

Also, as shown in case b, the pressure distribution sensors 110c and 110e of the controller 10b may be attached to areas that are likely to be gripped (held) by the user, such as both ends.

Also, as shown in case c, the pressure distribution sensor attached to the controller 10c may have different resolution depending on the position, such as high-resolution pressure distribution sensors 110f, 110h and low-resolution pressure distribution sensor 110g. As an example, the high-resolution pressure distribution sensor 110f is attached to a portion that is likely to be gripped by the user (e.g., both ends), and the low-resolution pressure distribution sensor 110g is attached to a portion that is unlikely to be gripped by the user (e.g., the center).

Also, as shown in case d, in the controller 10d, the pressure distribution sensor 110i does not have to be attached to the entire circumference of the housing E1, but may be attached to only a portion of it.

Returning to FIG. 1, the microcontroller 120 corresponds to an MCU (Micro-Control Unit) and can be configured as an integrated circuit that implements a processor, memory (recording medium), input/output circuits, etc.

The functions of the microcomputer 120 can be realized by a processor executing a program stored in a memory. If a microcomputer is present outside the controller 10, the functions of the microcomputer 120 may be incorporated into the microcomputer present outside the controller 10. For example, the functions of the microcomputer 120 may be incorporated into the control device 81.

The microcomputer 120 detects the user's gripping state based on the detection information of the pressure detection unit 110. In other words, the microcomputer 120 is an example of a grip detection unit.

The microcomputer 120 controls the operation of the center of gravity variable unit 130 and the vibration presentation unit 140 based on the detected gripping state. Note that the control device 81 may have the function of detecting the user's gripping state based on the detection information of the pressure detection unit 110 and the function of controlling the operation of the center of gravity variable unit 130 and the vibration presentation unit 140.

Under the control of the microcomputer 120, the center of gravity variable unit 130 varies the center of gravity of the controller 10 (makes the center of gravity variable) by moving a weight provided inside the housing E1 using a motor or the like.

The vibration presentation unit 140 is a vibrator located within the housing E1 that operates under the control of the microcomputer 120.

FIG. 5 is an explanatory diagram explaining the variable center of gravity of the controller 10. As shown in FIG. 5, a center of gravity variable part 130 (movable part) that can be moved in the tube length direction inside the housing E1 by a motor or the like is provided inside the housing E1.

In addition to the motor, the movable part is provided with a position sensor that detects the position of the movable part in the tube length direction inside the housing E1, a vibration presentation unit 140, and the like. The movable part may also be equipped with a temperature sensor, an IMU (Inertial Measurement Unit) sensor, and the like.

In the controller 10, the movable parts (the center of gravity variable part 130, the vibration presentation part 140) are operated to present various force sensations to the user through the user's hand H1 holding the housing E1.

For example, when the center of gravity variable unit 130 is located above the hand H1 that is holding the bottom of the housing E1, the user feels as if the center of gravity of the controller 10 is located above, and as if they are holding a long virtual object in the VR space (presentation of shape through force sensation).

In addition, by quickly moving the center of gravity variable unit 130, the user can feel as if they have received an impact from the controller 10 (presentation of an impact).

In addition, by quickly vibrating the vibration presentation unit 140 in the movable part, the user can feel as if the controller 10 is vibrating (presentation of vibration).

In this way, the center of gravity variable unit 130 and the vibration presentation unit 140 are an example of a force sense presentation unit that presents a predetermined force sense to a user holding the housing E1.

Returning to FIG. 1, the battery 150 supplies power to each component of the controller 10. For example, the battery 150 supplies power to the pressure detection unit 110, the microcomputer 120, the center of gravity variable unit 130, and the vibration presentation unit 140.

In the example shown in FIG. 1, the vibration presentation unit 140 is provided outside the center of gravity variable unit 130 (movable unit), but it may be included in the center of gravity variable unit 130. In this case, the center of gravity variable unit 130 can use the vibration presentation unit 140 as a weight.

Next, we will explain in detail the function of detecting the user's gripping state based on the detection information of the pressure detection unit 110, and the function of controlling the operation of the center of gravity variable unit 130 and the vibration presentation unit 140 based on the detected gripping state.

In this embodiment, the above functions are performed by the microcomputer 120, but it goes without saying that the control device 81 may also perform the above functions via the microcomputer 120.

FIG. 6 is an explanatory diagram for explaining the estimation of the position of the hand H1 holding the controller 10. In the example of FIG. 6, it is assumed that the user is holding the lower part of the housing E1 with the hand H1. Therefore, the pressure distribution sensor 110b attached to the lower part of the housing E1 detects pressure values at 10 x 16 detection points that correspond to the shape of the hand H1 as shown.

As shown in FIG. 6, the microcontroller 120 estimates the gripping position of the hand H1 and the shape of the hand H1 based on the pressure values (pressure distribution) sensed by the detection points of the pressure distribution sensor 110b.

For example, the microcontroller 120 performs the above estimation using a learning model that has been machine-learned to output the gripping position and shape of the hand H1 when pressure values at each detection point of the pressure distribution sensor 110b are input.

In addition, the microcontroller 120 estimates whether the gripping hand H1 is the right hand, the left hand, or both hands that have a different pressure distribution from hand H1, based on the pressure value (shape of pressure distribution) sensed by the detection point of the pressure distribution sensor 110b.

For example, the microcomputer 120 makes the above estimation using a learning model that has been machine-learned to output that the gripping hand H1 is the right hand, the left hand, or both hands when pressure values from each detection point of the pressure distribution sensor 110b are input.

The microcontroller 120 may also estimate the center G2 of the hand based on the pressure values (pressure distribution) sensed by the detection points of the pressure distribution sensor 110b.

As an example, the microcontroller 120 estimates the center of the position where a certain pressure value is detected as the center of the hand G2. At this time, the microcontroller 120 may exclude values that are significantly off the position.

The microcontroller 120 may also detect whether the gripping hand H1 is the right hand or the left hand based on whether the shape of the pressure distribution sensed by the detection point of the pressure distribution sensor 110b shows a characteristic shape for gripping with the right hand or the left hand.

The microcontroller 120 may also detect the hand H1 based on the context of App 812. For example, if a virtual object corresponding to the controller 10 in the VR space presented by App 812 is held in the right hand, the microcontroller 120 assumes that it is being held in the right hand and regards the hand center G2 as the center position of the right hand.

The microcomputer 120 controls the operation of the movable part (center of gravity variable part 130) based on the detected gripping position of the hand H1 (center of hand G2). For example, the microcomputer 120 changes the position of the center of gravity variable part 130 based on the detected center of hand G2.

The detection of the gripping state of the housing E1 by the user's hand H1 described above may be performed not only by the microcomputer 120 but also by the control device 81 that acquires detection information from the controller 10. The control device 81 (including the microcomputer 120) may also detect the gripping state of the housing E1 by the user's hand H1 based on an image captured by the camera 82. For example, the control device 81 performs the above detection by using a learning model that has been machine-learned to output the user's gripping state when an image captured by the camera 82 is input.

FIG. 7 is an explanatory diagram explaining the operation control of the center of gravity variable unit 130. As shown in case C1 in FIG. 7, when the user holds the controller 10 with the hand H1, the user feels a moment (rotational moment) due to the deviation of the center of gravity G1 of the controller 10 from the center G2 of the holding hand H1.

In order to prevent the user from feeling unnecessary rotational moment, the microcomputer 120 changes the position of the center of gravity variable unit 130 based on the center G2 of the hand. Specifically, the microcomputer 120 controls the position of the center of gravity variable unit 130 so that the center of gravity G1 is a predetermined value relative to the center of gravity G2 of the hand by referring to table data or the like that indicates the position of the center of gravity G1 that corresponds to the position of the center of gravity variable unit 130.

As an example, as shown in cases C2 and C3, the microcomputer 120 controls the position of the center of gravity variable unit 130 so that the center of gravity G1 relative to the center of the hand G2 remains constant even if the gripping position of the hand H1 (center of the hand G2) changes.

FIG. 8 is an explanatory diagram for explaining the calculation of the position of the center of gravity variable unit 130. As shown in FIG. 8, the calculation of the position of the center of gravity variable unit 130 in the cylinder length direction of the controller 10 may be performed by converting the position into a range from 1 to -1.

For example, the microcontroller 120 may set the position where the center of gravity G1 and the center of the hand G2 overlap as 0, and may treat "center of the hand G2 < center of gravity G1" as positive, and "center of the hand G2 > center of gravity G1" as negative. In addition, the reference coordinate system (positive direction) may treat the direction defined as "upward" in designing the controller 10 as positive.

In addition, the microcomputer 120 may move the center of gravity variable unit 130 and then return it to its original position based on the gripping position of the gripping hand H1 (center G2 of the hand). Figure 9 is an explanatory diagram for explaining an example of the operation of the center of gravity variable unit 130.

As shown in FIG. 9, the controller 10 is treated as a virtual shotgun V1 in the VR space. The microcomputer 120 can provide the user's hand H1 with a force sensation (impact) corresponding to the firing of the virtual shotgun V1 by quickly moving the center of gravity variable unit 130.

At this time, the microcomputer 120 returns the position of the center of gravity variable unit 130 so that the center of gravity G1 returns to its original position, based on the detected center G2 of the hand. This allows the controller 10 to provide a sense of force so that the shape (rotational moment) of the virtual shotgun V1 does not change before and after the firing of the virtual shotgun V1.

FIG. 10 is an explanatory diagram illustrating an example of the operation of the center of gravity variable unit 130. As shown in cases C4 and C5 in FIG. 10, when the user is holding the controller 10 with both hands H1 and H2, the microcomputer 120 may treat the center of the holding positions of both hands as the origin (0).

The microcomputer 120 may also control the position of the center of gravity variable unit 130 based on the strength of the detected pressure value (gripping force).

For example, as shown in case C6, the microcontroller 120 may change the origin position (initial position of the center of gravity variable unit 130) based on the strength of the pressure value detected at the position of hand H1 and the strength of the pressure value detected at the position of hand H2.

As an example, the microcontroller 120 may determine the origin (0) by taking a weighted average of the grip strength of hand H1 and the grip strength of hand H2 detected by the pressure detection unit 110.

The microcontroller 120 may also control the vibration of the movable part (vibration presentation part 140) depending on whether the detected hand H1 is a right hand, a left hand, or both hands H1 and H2. For example, the microcontroller 120 may weaken the vibration in the case of one hand (right or left hand) and strengthen the vibration in the case of both hands. The microcontroller 120 may also adjust the position of the center of gravity variable part 130 depending on whether it is a right hand, a left hand, or both hands.

The microcomputer 120 may also control the vibration of the movable part (vibration presentation unit 140) based on the detected gripping position of the hand H1 (center G2 of the hand). Figure 11 is an explanatory diagram that explains an example of the operation of the center of gravity variable unit 130. Figure 12 is an explanatory diagram that explains an example of the vibration of the vibration presentation unit 140.

As shown on the left side of Figure 11, if the center G2 of the hand and the center of gravity G1 are close to each other, it becomes physically difficult to vibrate, and the vibration when the center of gravity variable unit 130 (vibration presentation unit 140) is operated becomes weaker. Therefore, the microcontroller 120 moves the center of gravity variable unit 130 based on the detected gripping position of the hand H1 (center of hand G2) so that the center position of vibration (center of gravity G1) does not overlap with the center of hand G2. Here, the microcontroller 120 may generate vibration by quickly moving 130 up and down, or may generate vibration by operating the vibration presentation unit 140 provided in the movable part.

The microcomputer 120 may also control the vibration to reproduce a vibration waveform based on the detected gripping position of the hand H1 (center G2 of the hand). For example, as shown in FIG. 12, when the vibration presentation unit 140 vibrates far from the gripping position of the hand H1, the vibration will be attenuated compared to when it vibrates close to the gripping position. The microcomputer 120 controls the vibration to reproduce a vibration waveform that takes into account such vibration attenuation.

For example, when the gripping position of the hand H1 is o, the position of the movable part (vibration presentation unit 140) is p, and the vibration damping characteristics are F(o, p), the microcontroller 120 reproduces a vibration waveform in which the vibration to be reproduced has been passed through a filter that is an inverse function of F. The microcontroller 120 may also use the above filter as a filter that takes into account the damping characteristics F(o, p, f) for each vibration frequency f.

The microcontroller 120 may also perform position control (moment control) of the center of gravity variable unit 130 to present the intended amount of moment (force sensation) to the user.

Specifically, the microcomputer 120 determines the position of the movable part (center of gravity variable part 130) based on the detected gripping position of the hand H1 (hand center G2) and the attitude of the controller 10 relative to the vertical direction (device angle).

FIGS. 13 and 14 are explanatory diagrams illustrating an example of moment control in the controller 10. As shown in FIG. 13, a moment M to be presented to a hand H1 holding the controller 10 is determined. In addition, the attitude (device angle) of the controller 10 relative to the vertical direction detected using a gravity sensor or the like is θ. The mass (device mass) of the controller 10 is m, and the gravitational acceleration is g.

The microcontroller 120 determines the center of gravity position p corresponding to the presented moment M based on the detected grip position o, device angle θ, device mass m, and gravitational acceleration g as center of gravity position p = o + M/(m * g * sin(θ)).

The microcontroller 120 controls the position of the center of gravity variable unit 130 so that the center of gravity is at the above-mentioned position p, even while the user is moving the controller 10 in various positions, thereby presenting a constant moment M to the user.

The microcomputer 120 may also determine the moment acting on the hand H1 from the pressure distribution of the gripping hand H1 detected by the pressure detection unit 110. As shown in FIG. 14, when a user grips the controller 10 with the hand H1, as one example, the controller 10 is subjected to an upward normal force N1 from the thumb of the hand H1 and a downward normal force N2 from the little finger of the hand H1.

These mutually opposite normal forces N1 and N2 generate a moment M centered on the center G2 of the hand. The microcomputer 120 calculates the moment M from the integral of the pressure distribution (F) detected by the pressure detection unit 110.

FIG. 15 is an explanatory diagram illustrating an example of a feedback loop for moment control. As shown in FIG. 15, the microcomputer 120 may perform moment control by inputting a moment M calculated from the pressure distribution of the hand H1 into a feedback loop L1.

The microcomputer 120 may also estimate the size (area) of the hand H1 based on the range of pressure distribution in which the pressure value detected by the pressure detection unit 110 is equal to or greater than a predetermined value.

The microcomputer 120 may control the operation of the movable parts (the center of gravity variable part 130 and the vibration presentation part 140) based on the estimated size (area) of the hand H1. Specifically, the microcomputer 120 controls the strength of the force sensation (e.g., impact, vibration) presented to the user by the operation of the movable parts.

FIG. 16 is an explanatory diagram illustrating an example of adjusting the strength of the force presented by the controller 10. As shown in FIG. 16, case C7 illustrates a case where the range of pressure distribution 111 where the pressure value is equal to or greater than a predetermined value in the pressure detection unit 110 is wide, and the hand H1 holding the controller 10 is large. In contrast, case C8 illustrates a case where the range of pressure distribution 111 where the pressure value is equal to or greater than a predetermined value in the pressure detection unit 110 is narrow, and the hand H1 holding the controller 10 is small.

As shown in case C7, when the range of pressure distribution 111 is wide and the hand H1 holding the controller 10 is large, the microcontroller 120 adjusts the position of the movable part (vibration presentation unit 140) (to the left end in the illustrated example) so that the center of gravity G1 is away from the center G2 of the hand. Then, the microcontroller 120 causes the vibration presentation unit 140 to vibrate relatively strongly.

Also, as shown in case C6, when the range of pressure distribution 111 is narrow and the hand H1 holding controller 10 is small, microcontroller 120 adjusts the position of the movable part (vibration presentation unit 140) (near the center in the illustrated example) so that center of gravity G1 approaches center G2 of the hand. Then, microcontroller 120 causes vibration presentation unit 140 to vibrate relatively weakly.

In this way, the microcontroller 120 can control the strength of the force sensation (e.g., impact, vibration) presented to the user according to the detected size of the hand H1, thereby adjusting the force sensation to be smaller for someone with a small hand H1, for example, and preventing fatigue.

The microcontroller 120 may also estimate the angle (grip angle) of the controller 10 when held by the user based on the detection information of the pressure detection unit 110 when the movable unit (center of gravity variable unit 130, vibration presentation unit 140) is operated.

FIG. 17 is an explanatory diagram for explaining estimation of the grip angle of the controller 10. As shown in FIG. 17, case C9 is when the controller 10 is held vertically, and case C10 is when the controller 10 is held horizontally.

As shown in case C9, when the controller 10 is held vertically, even if the center of gravity variable unit 130 is operated in the tube length direction, no rotational moment is generated because the movement is in the vertical direction, and there is no change in the pressure distribution 111 of the pressure detection unit 110.

In contrast, as shown in case C10, when the controller 10 is held horizontally, moving the center of gravity variable unit 130 in the tube length direction generates a rotational moment according to the inclination of the controller 10 with respect to the vertical direction. This causes a change in the force of the hand H1 gripping the controller 10 to counteract the rotational moment, and the pressure distribution 111 of the pressure detection unit 110 changes.

The microcontroller 120 estimates the angle (grip angle) of the controller 10 when held by the user based on the fluctuations in pressure distribution 111 that occur in this manner.

For example, the microcomputer 120 performs the above estimation using a learning model that has been machine-learned to output a grip angle when a change in pressure distribution 111 detected during a specified operation of the center of gravity variable unit 130 in the tube length direction is input.

The microcontroller 120 may also estimate the swing state of the controller 10 held by the user based on the detection information of the pressure detection unit 110 when the movable units (the center of gravity variable unit 130 and the vibration presentation unit 140) are stopped.

FIG. 18 is an explanatory diagram for explaining the estimation of the swing speed of the controller 10. FIG. 19 is an explanatory diagram for explaining the estimation of the end of the swing of the controller 10.

As shown in FIG. 18, case C11 is a case where the controller 10 is slowly swung from vertical to horizontal, and case C12 is a case where the controller 10 is swung faster than in case C11.

As is clear from a comparison of cases C11 and C12, the pressure distributions 111a, 111b at the point where the controller 10 is gripped by the hand H1 differ greatly depending on the speed at which the controller 10 is swung.

For example, the pressure distributions 111a and 111b that occur when the controller 10 is swung down and stopped sideways become larger and the pressure value becomes higher as the speed of the swing increases.

The microcomputer 120 estimates the swing speed of the controller 10 by the user based on the fluctuations in the pressure distributions 111a, 111b that occur in this manner. For example, the microcomputer 120 performs the above estimation using a learning model that has been machine-learned to output a swing down speed when the fluctuations in the pressure distributions 111a, 111b are input.

As shown in FIG. 19, case C13 is when the controller 10 is being swung from vertical to horizontal, and case C14 is when the controller 10 is stopped (ended) while being held horizontally.

As is clear from comparing cases C13 and C14, the pressure distributions 111a and 111b are significantly different when the controller 10 is being swung and at the end of the swing.

For example, in contrast to case C13, in which the controller 10 is being swung, in case C14, at the end of the swing, the gripping force of the hand H1 increases in order to stop the momentum of the controller 10, and the pressure distributions 111a and 111b become larger and the pressure value also becomes higher.

The microcomputer 120 estimates the swing state of the controller 10 (during swing or at the end of swing) based on the fluctuations in pressure distributions 111a, 111b that occur in this way.

For example, the microcomputer 120 performs the above estimation using a learning model that has been machine-learned to output a swing state when a variation in the pressure distributions 111a and 111b is input. In addition, the microcomputer 120 may operate the vibration presentation unit 140 to present a vibration when it detects the end of the swing.

The microcontroller 120 may also operate the movable parts (the center of gravity variable part 130, the vibration presentation part 140) based on the detection information of the pressure detection part 110 to present a predetermined force sensation (interaction).

For example, in the VR space provided by App812, the user is holding the controller 10 as an extendable beam sword. In such a case, the microcomputer 120 changes the position of the center of gravity variable unit 130 according to the force with which the user holds it (the magnitude of the pressure value of the pressure detection unit 110).

FIG. 20 is an explanatory diagram illustrating an example of an interaction according to the force with which the controller 10 is gripped. In FIG. 20, the magnitude of the force with which the user grips the controller 10 with the hand H1 increases in the order of cases C15, C16, and C17.

As shown in FIG. 20, the greater the user's gripping force (the magnitude of the pressure value of the pressure detection unit 110), the more the microcomputer 120 moves the position of the center of gravity variable unit 130 so as to move the center of gravity G1 away from the center of the hand G2.

As a result, the stronger the grip on the controller 10, the more the user feels the sensation of a beam-like sword extending.

Also, for example, in the VR space provided by App812, the user is holding the controller 10 as a hose spraying water.

In such a case, the microcontroller 120 may present an interaction that changes the amount of water sprayed by operating the center of gravity variable unit 130 according to the user's gripping force (the magnitude of the pressure value of the pressure detection unit 110).

FIG. 21 is an explanatory diagram illustrating an example of an interaction according to the force with which the controller 10 is gripped. In FIG. 21, the force with which the user grips the controller 10 (virtual hose V2) with hand H1 is greater in case C19 than in case C18.

As shown in FIG. 21, for example, when the gripping force is increased, the microcomputer 120 operates to move the position of the center of gravity variable unit 130 closer to the position of the user's hand H1, and when the gripping force is decreased, the microcomputer 120 operates to move the position of the center of gravity variable unit 130 away from the position of the user's hand H1.

In addition, the microcontroller 120 may increase the vibration of the vibration presentation unit 140 and the vibration frequency when the gripping force is increased, and decrease the vibration of the vibration presentation unit 140 and the vibration frequency when the gripping force is decreased.

By doing this, the user can get the sense that water is being released forcefully from the hand when the user grips the virtual hose V2 (controller 10) with force, and the user can get the sense that water is being released slowly when the user releases the force.

Also, for example, assume that in the VR space provided by App812, the user is holding the controller 10 as a shotgun (virtual gun).

In such a case, the microcontroller 120 may operate the center of gravity variable unit 130 according to the position of the user's hand.

FIG. 22 is an explanatory diagram illustrating an example of an interaction according to the position of the hands holding the controller 10. As shown in FIG. 22, the user is holding the controller 10 as a virtual shotgun V1 with hands H1 and H2.

As shown in FIG. 22, the microcomputer 120 adjusts the position of the center of gravity variable unit 130 in accordance with the sliding position of the hand H2.

For example, when the hand H2 is slid along the barrel length direction of the controller 10 to perform the pump action of the virtual shotgun V1, the microcomputer 120 moves the center of gravity variable unit 130 to match the position of the hand H2.

This allows the controller 10 to present the user with a force sensation corresponding to the pump action of the virtual shotgun V1. The microcomputer 120 may also vibrate the vibration presentation unit 140 when the hand H2 reaches a predetermined position, thereby presenting the user with a force sensation corresponding to the loading of a bullet into the virtual shotgun V1.

Also, for example, assume that in the VR space provided by App812, the user is holding the controller 10 as a virtual sword.

In such a case, the microcomputer 120 may operate the center of gravity variable unit 130 according to the position of the user's hand holding the sword, thereby providing a sense of force when the sword is unsheathed or sheathed.

FIG. 23 is an explanatory diagram illustrating an example of an interaction according to the position of the hands holding the controller 10. As shown in FIG. 22, the user is holding the controller 10 as a virtual sword V3 with the hands H1 and H2.

As shown in FIG. 23, when the hand H2 slides on the surface of the controller 10, the microcomputer 120 maintains the position of the center of gravity variable unit 130 in accordance with the position of the hand H1 corresponding to the handle of the virtual sword V3.

At this time, the microcomputer 120 determines the speed of unsheathing or sheathing the sword from the change in the sliding position of the hand H2, and vibrates the vibration presentation unit 140 in accordance with that speed, thereby providing the user with a sense of vibration corresponding to unsheathing or sheathing the sword.

In the illustrated example, hand H1 is shown gripping the hilt of virtual sword V3, but hand H2 may grip the hilt of virtual sword V3.

Also, for example, it is assumed that in the VR space provided by App812, the user is holding the controller 10 as a virtual instrument.

In such a case, the microcomputer 120 may detect the position of the user's fingers using the pressure detection unit 110, and assign the detected finger positions to the operation of the instrument.

FIG. 24 is an explanatory diagram illustrating an example of application to a virtual instrument V4. As shown in FIG. 24, the user treats the controller 10 as a virtual instrument V4 such as a saxophone, and operates the fingers on the controller 10 as key operations on the virtual instrument V4.

The microcomputer 120 detects the user's finger operation from the pressure detection unit 110 and notifies the control device 81. As a result, the control device 81 produces sound from the virtual musical instrument V4 in the VR space.

At this time, the microcomputer 120 changes the position of the center of gravity variable unit 130 according to the weight of the virtual musical instrument V4 being operated, or vibrates the vibration presentation unit 140 according to the vibration (sound) of the virtual musical instrument V4.

For example, the microcomputer 120 may move the position of the center of gravity variable unit 130 downward (in the direction of gravity) as the weight of the virtual musical instrument V4 increases. The microcomputer 120 may also change the position of the center of gravity variable unit 130 and the vibration frequency of the vibration presentation unit 140 in accordance with the pitch of the sound produced by the virtual musical instrument V4.

For example, the microcontroller 120 may move the center of gravity variable unit 130 downward when the sound is low, and move the center of gravity variable unit 130 upward when the sound is high.

Also, if the user temporarily moves the controller 10 in sync with the performance, the microcomputer 120 does not need to move the position of the center of gravity variable unit 130.

Also, if the virtual instrument V4 is held horizontally, like a violin or trumpet, the "downward direction" may be considered to be the direction away from the user.

In addition, in a VR space, the controller 10 can be assigned to the stick of a weapon or musical instrument, and the impact felt when the weapon or musical instrument collides with an object or drum in the VR space can be presented as a force sensation.

Specifically, when a collision occurs with an object, a drum, or the like in the VR space, the microcomputer 120 receives a notification from the control device 81 and presents an impact or vibration through high-speed operation of the center of gravity variable unit 130, or a vibration from the vibration presentation unit 140.

At this time, the microcontroller 120 may change the strength of the impact or vibration caused by the operation of the center of gravity variable unit 130 and the vibration presentation unit 140 depending on the speed at which the user shakes the controller 10.

The microcontroller 120 may also change the strength of the impact or vibration caused by the operation of the center of gravity variable unit 130 and the vibration presentation unit 140 depending on how tightly the user grips the controller 10.

For example, the faster the shaking speed, the stronger the impact and vibration will be, and the stronger the user grips the controller 10, the stronger the impact and vibration will be.

In addition, the microcontroller 120 may not only increase the strength of the impact or vibration, but also change the frequency of the vibration depending on how strongly the user grips the controller 10. For example, the microcontroller 120 may increase the vibration frequency the stronger the user grips the controller 10, and decrease the vibration frequency the weaker the user grips the controller 10.

In addition, if the virtual musical instrument V4 is a violin in the VR space, the controller 10 can be regarded as the violin itself, and the violin bow can be replaced with a wooden stick or pencil separate from the controller 10.

At this time, the contact between the controller 10 and the wooden stick is detected by the pressure detection unit 110, and the control device 81 generates sound from the virtual instrument V4 in the VR space according to the detection value (contact state) and changes the sound. The microcomputer 120 may control the movable parts (center of gravity variable unit 130, vibration presentation unit 140) according to the generation of sound and the contact position in the virtual instrument V4.

When recreating a saw in a VR space, the controller 10 can be regarded as the saw itself, and the thing to be cut can be substituted with a physical object other than the controller 10 (for example, a desk).

At this time, the virtual object is placed in the VR space so that it is in the same position as the real object, and when the controller 10 comes into contact with the real object, the virtual object also comes into contact with the saw. When the user tries to cut this virtual object with the saw (controller 10), the microcomputer 120 can reproduce the feeling of cutting by vibrating the movable parts (center of gravity variable unit 130, vibration presentation unit 140) depending on the material of the virtual object.

Note that the above example is merely one example, and it can also be used as an interaction between the controller 10 and real objects only, without a VR space.

The twisting motion (sliding motion) of the hand detected by the pressure detection unit 110 may be assigned to operating the accelerator of a motorcycle or to operating a pepper or salt mill in a VR space.

At this time, if a twisting motion is made, it is determined that the accelerator of the motorcycle is turning, and the microcomputer 120 vibrates the movable parts (the center of gravity variable part 130, the vibration presentation part 140). This allows the user to feel the force of the accelerator operation.

In addition, when operating a pepper or salt mill, the image in the VR space may show pepper or salt falling from the tip of the mill, and the force sensation of the pepper or salt being ground may be expressed by the vibration of the moving parts (center of gravity variable unit 130, vibration presentation unit 140), etc.

In addition, an operation button (operation instruction) may be assigned to a specific position of the pressure distribution sensor on the surface of the controller 10, and the microcontroller 120 may detect the pressing operation (presence or absence of an operation instruction) at that specific position.

25 and 26 are explanatory diagrams illustrating an example of button operations on a controller. As shown in FIG. 25, in the VR space, a virtual operation button 112 is placed at a predetermined position of the pressure distribution sensor 110b of the controller 10.

App812 of the control device 81 acquires detection information of the pressure distribution sensor 110b from the microcomputer 120 and determines whether or not a pressing operation has been performed at the position where the virtual operation button 112 is placed. When App812 detects a pressing operation of the virtual operation button 112, it generates an event corresponding to the virtual operation button 112 in the VR space. The VR space may be an AR (Augmented Reality) space.

As shown in FIG. 26, App 812 may treat the controller 10 as a virtual gun V5 in the VR space and place the virtual operation button 112 at the trigger position of the virtual gun V5.

App812 acquires detection information from the pressure distribution sensor 110b from the microcomputer 120 and determines whether or not there is a sliding operation of the fingers of the hand H1 on the virtual operation button 112. If App812 detects a sliding operation of the virtual operation button 112 (for example, an amount of sliding equal to or greater than a threshold (approximately equivalent to pulling the trigger)), it generates an event corresponding to that virtual operation button 112 (firing the virtual gun V5) in the VR space.

FIG. 27 is an explanatory diagram illustrating an example of a pressure detection method in a controller. As shown in the controller 10e in FIG. 27, the pressure distribution sensor in the pressure detection unit 110 may be a capacitive touch pad 110j.

When the contact area of the capacitive touchpad 110j when held by the user can be detected, the contact area may be converted into a pressure value.

Also, as shown in the controller 10f in FIG. 27, the pressure distribution sensor in the pressure detection unit 110 may be a resistive touch pad 110k-110n divided into multiple sections in the tube length direction.

Also, as shown in controller 10f, the pressure distribution sensor in pressure detection unit 110 may be a plurality of resistive touch pads 110o-110q arranged along the tube length direction.

In addition, for resistive touchpads 110k to 110q, if they are of a type that can obtain pressure values, the detected pressure values may be used. In the case of analog resistive touchpads, multiple pressure sensors may be attached together to increase the number of pressure points that can be detected.

In the case of a sensor such as a touchpad, it may function only as a contact position sensor, or a pressure sensor such as a strain gauge may be provided separately from the contact position sensor, and the values of the two sensors may be used in combination. (There are no limitations on the type of contact position sensor.)

In addition, in the controller 10, an operation button may be formed on the pressure distribution sensor of the pressure detection unit 110, and the pressing operation (pressure) of the operation button may be detected. FIG. 28 is an explanatory diagram explaining an example of an operation button on a controller.

As shown in FIG. 28, the operation buttons on the pressure distribution sensor of the pressure detection unit 110 are explicitly formed with a shape that is different from other parts on the surface of the housing E1.

Specifically, the operation button on the pressure distribution sensor of the pressure detection unit 110 may be an operation button 113a with texture, or an operation button 113b with protrusions.

In this way, operation buttons 113a and 113b may be provided explicitly to the user.

Here, an example of the operation will be described in which a user operates the controller 10 as a virtual shotgun V1 in a VR space. FIG. 29 is a flowchart showing an example of the operation of an information processing system according to an embodiment of the present disclosure.

As shown in FIG. 29, when processing starts, the microcomputer 120 waits for input from the pressure detection unit 110 (waits for pressure detection) (S1). Next, it is assumed that the user has pressed the virtual button on the controller 10 that corresponds to the trigger (S2).

When the pressure detection unit 110 detects pressure at the position of the virtual button, the microcomputer 120 notifies App812 of the detection result. As a result, App812 renders the firing of the gun (virtual shotgun V1) in the VR space (S3). The microcomputer 120 also drives the weight (center of gravity variable unit 130) to present the impact (force sensation) of firing the gun to the user (S4).

Then, the microcomputer 120 waits for input from the pressure detection unit 110 (waits for pressure detection) (S5). Next, it is assumed that the user performs an action of rubbing the cylinder of the controller 10 (pump action) on the controller 10 (S6).

When the microcomputer 120 detects a sliding motion corresponding to a pump action in the pressure detection unit 110, it notifies App812 of the detection result. As a result, App812 renders the pump action in the VR space in accordance with the movement of the user's hand (S7).

The microcomputer 120 also drives the weight (the variable center of gravity unit 130) in real time in accordance with the movement of the user's hand, and presents the user with a force sensation corresponding to the pump action (S8). Next, the microcomputer 120 ends the process when the action related to the virtual shotgun V1 in the VR space ends (S9).

As described above, the information processing system detects the state in which the housing E1 is being held by the user using the microcomputer 120 or the control device 81 for the force feedback device (controller 10) that has a center of gravity variable unit 130, vibration feedback unit 140 and presents a predetermined force feedback to the user holding the housing E1. The microcomputer 120 controls the operation of the force feedback unit based on the detected gripping state. This allows the information processing system to present a force feedback according to the user's gripping state.

The microcomputer 120 also detects the user's gripping position based on the pressure detection positions of multiple pressure sensors provided on the surface of the housing E1. This allows the information processing system to detect the user's gripping position as the user's gripping state.

The microcomputer 120 also operates the force feedback unit based on the detected grip position of the user to provide a force feedback corresponding to the grip position. This allows the information processing system to provide a force feedback according to the user's grip position.

The microcomputer 120 also operates the force feedback unit based on the amount of change in the detected user's grip position to provide a force feedback corresponding to the change in grip position. This allows the information processing system to provide a force feedback according to the change in the user's grip position.

The microcomputer 120 also detects the user's gripping force based on the detection value of the pressure sensor. This allows the information processing system to detect the user's gripping force as the user's gripping state.

The microcomputer 120 also detects the user's gripping force based on the pressure detection ranges of the multiple pressure sensors. This allows the information processing system to detect the user's gripping force as the user's gripping state.

The microcomputer 120 also operates the force sense presentation unit based on the detected gripping force to present a force sense corresponding to the gripping force. This allows the information processing system to present a force sense according to the gripping force of the user.

The microcomputer 120 also detects the range that the user is gripping based on the detection range in which the pressure values of the multiple pressure sensors are equal to or greater than a predetermined value. This allows the information processing system to detect the range that the user is gripping as the user's grip state.

The microcomputer 120 also operates the force feedback unit based on the detected range of the user's grip to provide a force feedback corresponding to the range of the user's grip. This allows the information processing system to provide a force feedback according to the range of the user's grip.

The microcomputer 120 or the control device 81 also detects whether the user is holding the housing E1 in his/her right hand, left hand, or both hands. This allows the information processing system to detect whether the user is holding the housing E1 in his/her right hand, left hand, or both hands.

The microcomputer 120 or the control device 81 also causes the force sense presentation unit to operate based on the detected hand (right hand, left hand, or both hands) that the user is holding, to present a force sense corresponding to the gripping force. This allows the information processing system to present a force sense according to whether the user is holding the housing E1 with the right hand, left hand, or both hands.

The force feedback unit also varies the center of gravity G1 of the housing E1 to provide a force feedback. This allows the information processing system to provide a force feedback corresponding to the variation in the center of gravity G1.

The force sense presentation unit also vibrates the housing E1 to present a force sense. This allows the information processing system to present a force sense corresponding to the vibration.

The controller 10 also includes an angle detection unit that detects the angle of the housing E1. The microcomputer 120 controls the operation of the force feedback unit based on the detected angle of the housing. This allows the information processing system to present a force feedback corresponding to the angle of the housing.

The microcomputer 120 also detects the angle of the housing based on the amount of variation in the detection information when the force feedback unit presents a predetermined force feedback. This allows the information processing system to detect the angle of the housing.

The microcomputer 120 also detects a pressing operation at a predetermined position on the surface of the housing E1 based on the pressure detection positions of multiple pressure sensors provided on the surface of the housing E1. The microcomputer 120 determines whether or not an operation instruction has been assigned to a predetermined position based on whether or not a pressing operation has been detected at the predetermined position. This allows the information processing system to receive an operation instruction from the user that corresponds to a pressing operation on the surface of the housing E1.

The microcomputer 120 also estimates the user's swinging motion of the housing E1 based on the amount of fluctuation in the detection information when the operation of the force feedback unit is stopped. This allows the information processing system to obtain the swinging motion of the housing E1 when it is being held by the user.

The microcomputer 120 also causes the force sense presentation unit to operate based on the estimated swinging motion of the housing E1 to present a force sense corresponding to the swinging motion. This allows the information processing system to present a force sense corresponding to the swinging motion of the housing E1.

The control device 81 (App812) also controls the display of the display device that presents the virtual space to the user based on the detected gripping state. This allows the information processing system to display in the virtual space corresponding to the user's gripping state.

(Hardware configuration example)
Information devices such as the controller 10, the control device 81, and the VR device 83 in the information processing system according to the above-described embodiment are realized by a computer 1000 having a configuration as shown in Fig. 30, for example. Fig. 30 is a hardware configuration diagram showing an example of the computer 1000. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a SSD (Solid State Drive) 1400, a communication interface 1500, and an input/output interface 1600. The components of the computer 1000 are connected by a bus 1050.

The CPU 1100 operates based on the programs stored in the ROM 1300 or the SSD 1400, and controls each component. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the SSD 1400 into the RAM 1200, and executes processes corresponding to the various programs.

The ROM 1300 stores boot programs such as the Basic Input Output System (BIOS) that is executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the hardware of the computer 1000.

SSD1400 is a recording medium readable by computer 1000 that non-temporarily records programs executed by CPU1100 and data used by such programs. Specifically, SSD1400 is a recording medium that records the communication program according to the present disclosure, which is an example of program data 1450. SSD1400 may be another non-temporary recording medium, such as a HDD (Hard Disk Drive).

The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

The input/output interface 1600 is an interface for connecting the input/output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a touch panel, keyboard, mouse, microphone, or camera via the input/output interface 1600. The CPU 1100 also transmits data to an output device such as a display, speaker, or printer via the input/output interface 1600. The input/output interface 1600 may also function as a media interface that reads programs and the like recorded on a specific recording medium. Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase change rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, or semiconductor memories.

For example, when the computer 1000 functions as the controller 10, the control device 81, and the VR device 83 in the information processing system according to the embodiment, the CPU 1100 of the computer 1000 executes a communication program loaded onto the RAM 1200 to realize various functional configurations. The SSD 1400 stores the program according to the present disclosure and data in the RAM 1200. The CPU 1100 reads and executes the program data 1450 from the SSD 1400, but as another example, the CPU 1100 may obtain these programs from other devices via the external network 1550.

The present technology can also be configured as follows.
(1)
a gripping detection unit that detects a gripping state of a housing by a user, the gripping detection unit detecting a gripping state of the housing by a user, the gripping detection unit having a gripping state of the housing and a gripping detection unit that detects a gripping state of the housing by a user, the gripping detection unit detecting ...
A control unit that controls an operation of the force feedback unit based on the detected grip state.
Information processing system.
(2)
the grip detection unit detects a grip position of the user based on pressure detection positions of a plurality of pressure sensors provided on a surface of the housing;
The information processing system according to (1) above.
(3)
the control unit causes the force sense presentation unit to present a force sense corresponding to the gripping position by an operation of the force sense presentation unit based on the detected gripping position of the user.
The information processing system according to (2) above.
(4)
the control unit causes the force sense presentation unit to operate based on the detected amount of change in the grip position of the user, to present a force sense corresponding to the change in the grip position.
The information processing system according to (2) above.
(5)
The grip detection unit detects a gripping force of the user based on a detection value of a pressure sensor provided on a surface of the housing.
The information processing system according to any one of (1) to (4).
(6)
The grip detection unit detects a gripping force of the user based on pressure detection ranges of a plurality of pressure sensors provided on a surface of the housing.
The information processing system according to any one of (1) to (5).
(7)
the control unit causes the force sense presentation unit to present a force sense corresponding to the gripping force by an operation of the force sense presentation unit based on the detected gripping force of the user.
The information processing system according to (5) or (6).
(8)
the grip detection unit detects a gripping range of the user based on a detection range in which a pressure value of a plurality of pressure sensors provided on a surface of the housing is equal to or greater than a predetermined value;
The information processing system according to any one of (1) to (7).
(9)
the control unit causes the force sense presentation unit to present a force sense corresponding to the range grasped by the user through an operation based on the detected range grasped by the user.
The information processing system according to (8) above.
(10)
The grip detection unit detects whether the user grips the housing with his/her right hand, his/her left hand, or his/her both hands.
The information processing system according to any one of (1) to (9).
(11)
The control unit causes the force sense presentation unit to present a force sense corresponding to the gripping force by an operation based on the detected hand of the user gripping the object.
The information processing system according to (10) above.
(12)
The force sense presentation unit presents the force sense by vibrating the housing.
The information processing system according to any one of (1) to (11).
(13)
An angle detection unit that detects an angle of the housing is further provided,
The control unit controls an operation of the force sense presentation unit based on the detected angle of the housing.
The information processing system according to any one of (1) to (12).
(14)
the grip detection unit detects a pressing operation at a predetermined position on a surface of the housing based on pressure detection positions of the plurality of pressure sensors;
the control unit determines whether or not an operation instruction assigned to the predetermined position is present based on whether or not a pressing operation is detected at the predetermined position.
The information processing system according to (2) above.
(15)
the control unit estimates a shaking motion of the housing by the user based on an amount of change in a detection value of a pressure sensor provided on a surface of the housing while the operation of the force sense presentation unit is stopped.
The information processing system according to any one of (1) to (14).
(16)
the control unit causes the force sense presentation unit to present a force sense corresponding to the swinging motion by an operation based on the estimated swinging motion of the housing;
The information processing system according to (15) above.
(17)
The control unit performs display control of a display device that presents a virtual space to the user based on the detected grip state.
The information processing system according to any one of (1) to (16).
(18)
The computer
A force sense presentation device includes a force sense presentation unit that presents a predetermined force sense to a user holding a housing by varying a center of gravity of the housing, and detects a state in which the housing is held by the user;
controlling an operation of the force sense presentation unit based on the detected gripping state;
An information processing method for performing processing.
(19)
a force feedback unit that changes a center of gravity of a housing to provide a predetermined force feedback to a user holding the housing;
a detection unit that detects a gripping state of the user based on detection information from a pressure sensor provided on a surface of the housing;
A control unit that controls an operation of the force feedback unit based on the detected grip state.
Force feedback device.

10, 10a to 10g...Controller 81...Control device 82...Camera 83...VR device 110...Pressure detection unit 110a to 110e, 110i...Pressure distribution sensor 110f, 110h...High-resolution pressure distribution sensor 110g...Low-resolution pressure distribution sensor 110j...Capacitive touch pad 110k to 110q...Resistive touch pad 111, 111a, 111b...Pressure distribution 112...Virtual operation button 113, 113a, 113b...Operation button 120...Microcomputer 130...Center of gravity variable unit 140...Vibration presentation unit 150...Battery 812...App
813…SDK
1000... Computer 1050... Bus 1100... CPU
1200...RAM
1300...ROM
1400...SSD
1450...program data 1500...communication interface 1550...external network 1600...input/output interface 1650...input/output devices a to d, C1 to C19...case E1...housing G1...center of gravity G2...center of hand H1, H2...hand L1...feedback loop V1...virtual shotgun V2...virtual hose V3...virtual sword V4...virtual musical instrument V5...virtual pistol

Claims (19)

a gripping detection unit that detects a gripping state of a housing by a user, the gripping detection unit detecting a gripping state of the housing by a user, the gripping detection unit having a gripping state of the housing and a gripping detection unit that detects a gripping state of the housing by a user, the gripping detection unit detecting ...
A control unit that controls an operation of the force feedback unit based on the detected grip state.
Information processing system. the grip detection unit detects a grip position of the user based on pressure detection positions of a plurality of pressure sensors provided on a surface of the housing;
The information processing system according to claim 1 . the control unit causes the force sense presentation unit to present a force sense corresponding to the gripping position by an operation of the force sense presentation unit based on the detected gripping position of the user.
The information processing system according to claim 2 . the control unit causes the force sense presentation unit to operate based on the detected amount of change in the grip position of the user, to present a force sense corresponding to the change in the grip position.
The information processing system according to claim 2 . The grip detection unit detects a gripping force of the user based on a detection value of a pressure sensor provided on a surface of the housing.
The information processing system according to claim 1 . The grip detection unit detects a gripping force of the user based on pressure detection ranges of a plurality of pressure sensors provided on a surface of the housing.
The information processing system according to claim 1 . the control unit causes the force sense presentation unit to present a force sense corresponding to the gripping force by an operation of the force sense presentation unit based on the detected gripping force of the user.
7. The information processing system according to claim 5 or 6. the grip detection unit detects a gripping range of the user based on a detection range in which a pressure value of a plurality of pressure sensors provided on a surface of the housing is equal to or greater than a predetermined value;
The information processing system according to claim 1 . the control unit causes the force sense presentation unit to present a force sense corresponding to the range grasped by the user through an operation based on the detected range grasped by the user.
The information processing system according to claim 8. The grip detection unit detects whether the user grips the housing with his/her right hand, his/her left hand, or his/her both hands.
The information processing system according to claim 1 . The control unit causes the force sense presentation unit to present a force sense corresponding to the gripping force by an operation based on the detected hand of the user gripping the object.
The information processing system according to claim 10. The force sense presentation unit presents the force sense by vibrating the housing.
The information processing system according to claim 1 . An angle detection unit that detects an angle of the housing is further provided,
The control unit controls an operation of the force sense presentation unit based on the detected angle of the housing.
The information processing system according to claim 1 . the grip detection unit detects a pressing operation at a predetermined position on a surface of the housing based on pressure detection positions of the plurality of pressure sensors;
the control unit determines whether or not an operation instruction assigned to the predetermined position is present based on whether or not a pressing operation is detected at the predetermined position.
The information processing system according to claim 2 . the control unit estimates a shaking motion of the housing by the user based on an amount of change in a detection value of a pressure sensor provided on a surface of the housing while the operation of the force sense presentation unit is stopped.
The information processing system according to claim 1 . the control unit causes the force sense presentation unit to present a force sense corresponding to the swinging motion by an operation based on the estimated swinging motion of the housing;
16. The information processing system according to claim 15. The control unit performs display control of a display device that presents a virtual space to the user based on the detected grip state.
The information processing system according to claim 1 . The computer
A force sense presentation device includes a force sense presentation unit that presents a predetermined force sense to a user holding a housing by varying a center of gravity of the housing, and detects a state in which the housing is held by the user;
controlling an operation of the force sense presentation unit based on the detected gripping state;
An information processing method for performing a process. a force feedback unit that changes a center of gravity of a housing to provide a predetermined force feedback to a user holding the housing;
a detection unit that detects a gripping state of the user based on detection information from a pressure sensor provided on a surface of the housing;
A control unit that controls an operation of the force feedback unit based on the detected grip state.
Force feedback device.

Images