JP2006005618A - Sound source environment restoration system, automatic sound source environment restoring device, and method and program for sound source environment restoration - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To grasp operation states of respective bodies which are viewed from the whole execution area with output sounds from a plurality of speakers when body images in operation areas which are displayed at respective monitor parts of a plurality of information processors are moving in an execution area of common software. <P>SOLUTION: Various information regarding a body image displayed at a monitor part 13 of an information processor 10 is analyzed and sent as image analytic information to an automatic sound source environment restoring device 30. An image information restoration part 33 of the automatic sound source environment restoring device 30 integrates pieces of audio information included in pieces of image analytic information sent from the plurality of information processors 10 to restore a sound signal based upon the position relation between a body etc., that body position information indicates and a reference point of an audible space, and the restored sound signal is sent to a plurality of speakers 20 to output restored sounds. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a sound source environment restoration system, an automatic sound source environment restoration device, and a sound source environment that output sound related to an object image to a plurality of speakers according to the operation of a multidimensional object image displayed on a monitor unit of an information processing device. In particular, a sound source environment restoration system, an automatic sound source environment restoration device, a sound source environment restoration method, and a sound source environment restoration program that perform different control for each speaker for output sound from a plurality of speakers. About.

  In recent years, personal computers (personal computers) have become popular not only in corporations but also in general households due to their low price and high performance. As for software, a wide variety of products have been developed and sold to the general public. In particular, there are many types of game software, from what is played on a single computer to what is recently called online games or network games (games played on multiple computers via a network). Are on the market.

By the way, when the execution of the game software is started on the personal computer, the screen for the game is usually displayed on the monitor unit, and sound is output from a speaker provided in the personal computer.
Examples of the sound include BGM (Back Ground Music) and sound effects (for example, sound when a piece is placed in a board game, sound when an attack is received in a fighting game, etc.). , Voice was selected and output according to the progress of the game (such as the action of the displayed object).

However, when a plurality of speakers are provided, it may sound that the operation of the object displayed on the monitor unit does not match the output sound from each speaker.
For example, when a scene in which an object (airplane) moves (flys) from the right side of the screen to the left side is projected, if the same flight sound is output from both speakers installed on the left and right sides of the monitor unit, From the output sound, the operator can know that the airplane is flying, but cannot know that it is flying from right to left.
In view of this, a technique has been proposed in which different sounds are output from a plurality of speakers according to the content of an image displayed on the screen to give the operator a sense of realism.

For example, if speakers are installed on the left and right sides of the monitor unit, and the object displayed on the monitor unit moves from the right side to the left side, the moving sound is output from the right speaker first, and then gradually output from the left speaker. A technique has been proposed in which control is performed so that the moving sound increases and then the moving sound is output only from the left speaker (see, for example, Patent Document 1).
According to this technique, the operator of the personal computer can know the moving direction of the object displayed on the monitor unit from these output sounds.

As another conventional technique, for example, a sensor unit that calculates the distance and angle between each speaker and the operator, and the frequency band and intensity of sound appropriate to the operator are calculated based on information from the sensor unit. And a classification device unit that selects pre-stored sound frequency band data and intensity-frequency characteristic data that apply to the processing result in the calculation device unit, and sends them to the personal computer unit (speaker) as sound source data. There has been proposed a technology having a configuration (see, for example, Patent Document 2).
According to this technique, even if the position of the operator deviates from the center of the range surrounded by the monitor unit, the speaker, etc., the operator can respond to the movement of the three-dimensional object displayed on the monitor unit. Sound can be heard properly.
Japanese Patent Application Laid-Open No. 07-072875 JP 2001-197600 A

  However, in each of the above prior arts, although the output sounds of a plurality of speakers can be controlled in accordance with the operation of the object image displayed on one monitor unit, the plurality of monitor units can display different object images. When operating on the execution area of software that is displayed and has the same object image, each operator or a person other than each operator knows the operation status of the entire execution area from the output sound of each speaker. I couldn't.

For example, it is assumed that the object image a1 is displayed on the monitor unit m1, the object image a2 is displayed on the monitor unit m2, and the object image a3 is displayed on the monitor unit m3. It is assumed that the sound based on the operation of the object a1 is output from the speaker s1, the sound of the object a2 is output from the speaker s2, and the sound of the object a3 is output from the speaker s3.
In this case, a person who can hear the output sounds of the speakers s1, s2, and s3 (listener) can know the operation contents of the objects a1, a2, and a3, but viewed from the entire execution region. There has been a problem that it is impossible to grasp the operation status of each of the objects a1, a2, and a3.

  A specific example will be described. For example, the sound of the object a1 is output from the speaker s1 installed in front of the listener, and the sound of the object a2 is output from the speaker s2 installed behind the listener. . Here, it is assumed that the object a1 displayed on the monitor unit m1 moves in a direction approaching the object a2 displayed on the monitor unit m2. In this case, the object a1 moves from the front to the rear of the listener when viewed from the entire execution area of the common software. However, the moving sound of the object a1 is output from the speaker s1, and even if the object a1 approaches the object a2, the moving sound is still output only from the speaker s1 (speaker installed in front of the listener). Not come. For this reason, the listener feels the illusion that the object a1 has only moved from the back to the front within a limited area in front. Thus, the listener cannot accurately grasp the operation state of the object a1 viewed from the entire execution area from the output sound of the speaker.

  When the object a1 is so close that the object a2 is displayed on the displayed monitor unit m2 at the same time, the sounds of the object a2 and the object a1 are output from the speaker s2 behind the listener. At this time, since the object a1 and the object a2 are also displayed on the monitor unit m1, both the sounds of the object a1 and the object a2 are output from the speaker s1 in front of the listener similarly to the speaker s2. End up. Then, the listener can hear both the sound of the object a1 and the object a2 both from the front and the rear, and the positions of the object a1 and the object a2 in the entire execution region (the object a1 and the object a1 as viewed from the entire execution region). In which position the object a2 is present) could not be correctly captured.

  The present invention has been made to solve the above-described problem, and in the case where each object image displayed on a plurality of monitor units is operating on the execution area of a common software, the entire execution area is used. It is an object of the present invention to provide a sound source environment restoration system, an automatic sound source environment restoration device, a sound source environment restoration method, and a sound source environment restoration program that make it possible to grasp the operation status of each object that has been viewed by sounds output from a plurality of speakers.

  In order to achieve this object, a sound source environment restoration system according to the present invention includes a plurality of information processing apparatuses having a monitor unit that displays object images, and outputs sound related to each object image to form an audible space. Two or more speakers and sound information about each object image are received from a plurality of information processing apparatuses, and the received plurality of sound information are integrated and restored as sound output in an audible space. An automatic sound source environment restoration device for outputting to a speaker is provided.

When the sound source environment restoration system has such a configuration, the automatic sound source environment restoration device integrates a plurality of pieces of audio information received from the respective information processing devices and restores them as sounds output in an audible space. The restored sound can be output to the listener.
For this reason, for example, the sound of the object image displayed on each monitor unit is integrated as sound captured from the entire execution area of the common software executed by each information processing apparatus. In addition, assuming that the audible space is an execution area of the software, the sound is restored according to the position of each object existing in the execution area. For this reason, the listener can accurately grasp the operation status of each object in the execution area by listening to the restored sound in the audible space.

  The sound source environment restoration system according to the present invention performs a process in which each information processing apparatus executes common software and operates an object image using an operation area as a part of the execution area of the software. The automatic sound source environment restoration apparatus receives one location in the execution area as a reference point, receives position information related to the object and / or information related to the operation area from each information processing apparatus together with the audio information related to the object image, and receives these information The restored voice is generated based on the positional relationship between the object and / or the operation area indicated by each piece of information and the reference point.

If the sound source environment restoration system has such a configuration, a restored voice can be generated based on the positional relationship between each object or operation area in the execution area and the reference point. As a result, the positional relationship between the objects in the execution area is clarified, and the positions and ways of transmission of the sounds of these objects can be accurately restored and reproduced as restored sounds.
Moreover, by assuming the reference point as a position where the listener listens to the restored voice, the restored voice capable of grasping the operation status of each object in the entire execution area can be provided to the listener.

  The sound source environment restoration system according to the present invention is configured such that the automatic sound source environment restoration device sets an execution area based on information about each operation area received from each information processing apparatus.

  When the sound source environment restoration system has such a configuration, an execution area can be set including all the operation areas of each information processing apparatus. Therefore, the listener can grasp the operation status of all object images displayed on each information processing apparatus.

  In the sound source environment restoration system of the present invention, each information processing device generates information on one or more objects displayed on the monitor unit, and the automatic sound source environment restoration device receives one or more objects. On the basis of the information on the information, the reflected sound in one or two or more objects is calculated to generate a restored sound.

  If the sound source environment restoration system has such a configuration, a more realistic restoration sound can be provided to the listener. For example, when the sound from the object a1 displayed on the monitor unit m1 reaches the object a2 displayed on the monitor unit m2 and is reflected, this reflected sound can also be restored. For this reason, the listener can hear more realistic restored sound.

  In addition, the automatic sound source environment restoration device of the present invention integrates a plurality of audio information with a receiving unit that receives audio information regarding an object image displayed on a monitor unit from a plurality of information processing devices, and one or more speakers. The sound restoration unit restores the output sound in the audible space formed by the output sound from the sound and the sound output unit that sends the restored sound to each speaker and outputs it.

  When the automatic sound source environment restoration device has such a configuration, the sound of each object image displayed on a plurality of monitor units is integrated and output as a restored sound from a plurality of speakers. By listening to the restored voice, it is possible to accurately grasp the operation state of the object image displayed on the monitor unit of each information processing apparatus.

  Also, the sound source environment restoration method of the present invention integrates a reception process for receiving audio information related to an object image displayed on a monitor unit from a plurality of information processing devices and a plurality of audio information, and outputs from one or more speakers. There is a method including a sound restoration process for restoring the output sound in the audible space formed by the output sound and a sound output process for sending the restored sound to each speaker for output.

  If the sound source environment restoration method is such a method, the sound information on each object image collected from a plurality of information processing devices is integrated and restored as sound output in a hearing-capable space. The situation can be grasped by listening to the restored voice.

  In addition, the sound source environment restoration program of the present invention integrates a reception process for receiving audio information regarding an object image displayed on a monitor unit from a plurality of information processing apparatuses and a plurality of audio information, and outputs from one or more speakers. The automatic sound source environment restoration apparatus executes a sound restoration process for restoring the output sound in the audible space formed by the output sound and a sound output process for sending the restored sound to each speaker for output.

  The sound source environment restoration program having such a configuration is read and executed by the automatic sound source environment restoration device, so that the operation status of each object operating in the common software execution area is output from a plurality of speakers. It can be grasped by the voice that has been made.

  According to the present invention, when each object image displayed on a plurality of monitor units is operating on the execution area of the common software, the audio information of these object images is integrated by the automatic sound source environment restoration device, It is restored and reproduced as a voice generated at the position of each object in the execution area. Therefore, by listening to the restored voice, it is possible to grasp and understand the operation status of each object image and the progress status of each information processing device in the execution area of the common software.

  Hereinafter, preferred embodiments of a sound source environment restoration system, an automatic sound source environment restoration device, a sound source environment restoration method, and a sound source environment restoration program according to the present invention will be described with reference to the drawings.

First, an embodiment of a sound source environment restoration system and an automatic sound source environment restoration device according to the present invention will be described with reference to FIG.
FIG. 2 is a block diagram showing the configuration of the sound source environment restoration system of this embodiment.
As shown in the figure, the sound source environment restoration system 1 includes information processing apparatuses 10-1 to 10-n, speakers 20-1 to 20-n, an automatic sound source environment restoration apparatus 30, and a network 40. Yes.

Here, the information processing apparatuses 10-1 to 10-n (hereinafter collectively referred to as “information processing apparatus 10”) are apparatuses that operate a predetermined function by executing a program. PC), notebook computer, mobile phone, PHS, PDA and the like.
Each information processing apparatus 10 can be connected via a network 40.
In FIG. 1, four information processing apparatuses 10 are provided in the sound source environment restoration system 1. However, the information processing apparatus 10 is not limited to four, and may include one, two, three, or five or more. it can.

As illustrated in FIG. 2, the information processing apparatus 10 includes a storage unit 11, a personal computer unit 12, a monitor unit 13, an image information analysis unit 14, and a communication unit 15.
The storage unit 11 can be configured by a hard disk, an external storage device, or the like, and stores programs (including software installed or downloaded) for executing various functions of the information processing apparatus 10, various information, and the like. Remember.
As a specific example, for example, in the storage unit 11, software for generating an image of a three-dimensional object or displaying it on the monitor unit 12, a program for causing the image information analysis unit 14 to execute analysis processing, an object Image information, image analysis information, and the like are stored.

The personal computer unit 12 is a main body portion of the information processing apparatus 10 having at least processing means of the information processing apparatus 10 (a part including a control means and a calculation means and configured by a CPU or the like). The personal computer unit 12 executes a program stored in the storage unit 11 to send commands to each component of the information processing apparatus 10 to perform predetermined processing.
The personal computer unit 12 generates a three-dimensional object image (object image) by reading and executing a predetermined program. Then, the generated object image is sent to the monitor unit 13 for display.
The generated object image is a three-dimensional object image in the present embodiment, but is not limited to three dimensions, and may be two-dimensional.

In addition, the personal computer unit 12 extracts information (object image information) related to the object image from the storage unit 11 and sends the information to the image information analysis unit 14. Then, the image analysis information from the image information analysis unit 14 is sent to and stored in the storage unit 11.
Further, the personal computer unit 12 sends the image analysis information extracted from the storage unit 11 (or the image analysis information from the image information analysis unit 14) to the automatic sound source environment restoration device 30 via the communication unit 15.

The monitor unit 13 can be composed of a CRT (CRT), LCD (Liquid Crystal Display), plasma display, or the like. For example, an object image from the personal computer unit 12 can be displayed.
When a part or all of an area (execution area) executable by software is displayed on the monitor unit 13, this area is referred to as an “operation area”.
As illustrated in FIG. 3, each information processing apparatus 10 can display object images in different operation areas in the common execution area.

For example, as shown in the figure, the information processing apparatus 10-1 is the upper half and lower right area of the execution area, the information processing apparatus 10-2 is the lower half area of the execution area, and the information processing apparatus 10-3. Can be the lower half of the execution area, and the information processing apparatus 10-4 can be the right half of the execution area.
In addition, the object image displayed in each operation area can be displayed by each information processing apparatus 10 as a common object image, or a different object image can be displayed.

The image information analysis unit 14 analyzes information related to an audio signal generated by the three-dimensional object generated by the personal computer unit 12. That is, the image information analysis unit 14 analyzes the object image information from the personal computer unit 12 to obtain various information about the three-dimensional object currently displayed on the monitor unit 13.
Examples of information (image analysis information) that can be obtained by the image information analysis unit 14 include the following.
Object identification information, object position information, object movement start position information, object movement direction information, object movement speed information, object movement end position information, operator position information, audio information, operation area information, other object related information, etc.

The object specifying information is information indicating what is the object currently displayed on the monitor unit 13. This object specifying information includes information indicating the type of object.
The object position information (position information related to the object) is information indicating where the object is currently located. Note that the position indicated by the object position information may be a position relative to the operation area, or may be a position relative to the execution area.

The object movement start position information is information indicating a position where the object starts moving.
The object moving direction information is information indicating in which direction the object is moving.
The object moving speed information is information indicating how fast the object is moving.
The object movement end position information is information indicating a position where the object has finished moving.
The operator position information refers to information indicating at which point (area) the operator is located in the execution area of the common software.

Audio information (audio information related to an object image) refers to information indicating what type of audio is produced in accordance with the operation of a three-dimensional object, and the type of the audio.
The operation area information (information related to the operation area) refers to information indicating in which part of the area (execution area) the operation area displayed on the monitor unit 13 can be executed by software.
The other object relationship information is information indicating how the relationship (for example, distance) between the object a1 and the object a2 is present when the other object a2 is present in the operation region where the object a1 exists. . This other object related information includes, for example, information indicating the presence / absence of reflected sound.

The image analysis information is acquired by the image information analysis unit 14 and then sent to the personal computer unit 12 (or sent to the storage unit 11 for storage).
Of the image analysis information, for example, the object position information, the object movement start position information, the object movement direction information, the object movement end position information, the operator position information, the operation area information, and the other object related information are shown in FIG. As shown, it can be expressed in coordinates.

The communication unit (transmission unit) 15 can be configured by a network board, for example, and transmits the image analysis information from the personal computer unit 12 to the automatic sound source environment restoration apparatus 30 via the network 40.
The input unit 16 can be configured by an MMI (Man Machine Interface), a pointing device, or the like, and specifically includes a mouse, a trackball, a trackpad, a joystick, a keyboard, a controller, and the like. The input unit 16 is operated by an operator. Thereby, the three-dimensional object displayed on the monitor unit 13 performs a predetermined operation according to the operation.

Speakers 20-1 to 20-n (hereinafter collectively referred to as “speaker 20”) output restored sound based on the restored sound signal sent from automatic sound source environment restoring device 30.
As shown in FIG. 1, four speakers 20 can be provided as a front speaker 20-1, a left speaker 20-2, a right speaker 20-3, and a rear speaker 20-4, for example.
In addition, although the four speakers 20 are provided in the same figure, it is not restricted to four, For example, one, two, three, or five or more can be provided.

Further, the plurality of speakers 20 are arranged so that the output directions of the sounds are gathered at almost one point (reference point (described later)). As a result, a person (listener) located at the reference point listens to the sound output from each speaker 20, thereby executing the software execution status (for example, game software) in each information processing apparatus 10. You can know the progress etc.).
A space / range in which the sound output from each speaker 20 can be heard is referred to as “audible space (sound source environment)”.

Examples of the listener include a visitor who confirms the progress of the network game.
The sound source environment restoration system of the present invention can be provided, for example, at a game software exhibition hall. In this case, when multiple PCs are installed in one or more booths in the venue, and those computers are running common software, there is an application such as outputting restored audio in the booth or venue. Is possible.

The automatic sound source environment restoration device 30 includes a communication unit 31, a storage unit 32, an image information restoration unit 33, an audio control unit 34, and an audio output unit 35.
The communication unit (reception unit) 31 can be configured by a network board, for example, and receives image analysis information transmitted from the communication unit 15 of each information processing apparatus 10.

The storage unit 32 can be configured with a hard disk, an external storage device, or the like, and stores programs for executing various functions of the automatic sound source environment restoration device 30, various information, and the like.
As a specific example, for example, the storage unit 32 stores a program for extracting information about a three-dimensional object from image analysis information, a program for generating a restored audio signal, image analysis information, a restored audio signal, and the like.

The storage unit 32 stores a table (image analysis information table) including image analysis information as shown in FIG.
The information stored as the image analysis information table includes, for example, information for specifying the information processing device 10, information for specifying the personal computer unit 12, and information indicating the type of object image displayed on the monitor unit 13 (type of object). , Information indicating the movement start position of the object, information indicating the movement end position of the object, information indicating the position of the operator, information indicating the operation region, information indicating the types of surrounding objects, and the like can be included.
The image analysis information table can include a part or all of various information included in the image analysis information transmitted from the information processing apparatus 10 in addition to the information illustrated in FIG.

When receiving the image analysis information from each information processing apparatus 10 from the communication unit 31 (or referring to the image analysis information table stored in the storage unit 32), the image information restoration unit (sound restoration unit) 33 receives the image analysis information. Each (or referred) image analysis information is analyzed, and each piece of sound information included in the image analysis information is integrated and restored as sound output in the audible space.
A specific example of the audio restoration process in the image information restoration unit 33 will be described in “Procedure of audio restoration process (restored audio signal generation process) in the image information restoration unit of the automatic sound source environment restoration apparatus” described later.

The restored audio control unit 34 can be configured by an amplifier circuit (for example, an amplifier), and amplifies the audio signal intensity of the restored audio signal from the image information restoring unit 33 (or taken out from the storage unit 32).
The audio output unit 35 sends the restored audio signal from the restored audio control unit 34 to each speaker 20. Thereby, in each speaker 20, the audio | voice based on the decompression | restoration audio | voice signal is output.

The network 40 can use any publicly known public line, commercial line, or dedicated line known in the art, and can be composed of one or more lines.
Specifically, for example, the Internet, intranet, LAN (Local Area Network), WAN (Wide Area Network), analog telephone network, digital telephone network (for example, ISDN (Integral Service Digital Network) and wireless communication network (for example, portable network) A telephone network, PHS (Personal Handyphone System), satellite broadcasting, etc.) can be used.

Next, the operation (sound source environment restoration method) of the sound source environment restoration system of this embodiment will be described with reference to FIG.
This figure is an operation procedure diagram showing the processing procedure of the sound source environment restoration method.
As shown in the figure, common software (for example, a common 3D display-compatible game) is started in each personal computer unit 12 of the plurality of information processing apparatuses 10 (step 10). Thereby, an image of the three-dimensional object is displayed on each monitor unit 13 of the information processing apparatus 10 (step 11). Here, the operator can operate the three-dimensional object by performing an input operation with the input unit 16 while viewing the display screen of the monitor unit 13.

In each personal computer unit 12, object image information is generated in accordance with the operation of the three-dimensional object (step 12) and sent to the image information analysis unit 14.
The received object image information is analyzed in the image information analysis unit 14 (step 13), and the image analysis information as a result of this analysis is sent to the storage unit 11 and stored therein. Further, the image analysis information is transmitted from the image information analysis unit 14 to the automatic sound source environment restoration device 30 via the communication unit 15 and the network 40 (step 14). The transmitted image analysis information includes at least audio information (audio information related to the object image). For example, object specifying information, object position information, object movement start position information, object movement direction information, object Movement speed information, object movement end position information, operator position information, operation area information, other object related information, and the like can be included.
Further, the transmission of the image analysis information is performed in real time according to the operation of the three-dimensional object displayed on the monitor unit 13 (according to the input operation at the input unit 16).

The communication unit 31 of the automatic sound source environment restoration apparatus 30 receives the image analysis information sent from each information processing apparatus 10 (reception process, step 15). The received image analysis information is sent to and stored in the storage unit 32 and is also sent to the image information restoration unit 33.
In the image information restoration unit 33, the sound information included in each image analysis information is integrated and restored as sound output in the audible space (sound source environment) (restored sound signal generation processing, sound restoration processing, Step 16).

  Here, in the image information restoration unit 33, operation area information is extracted from each image analysis information, and an integrated area, that is, an execution area of common software is set based on the operation area information. The integrated area (execution area) is set as an area including all operation areas displayed on the monitor unit 13 of each information processing apparatus 10. However, the integrated area set by the image information restoring unit 33 and the execution area where the common software can actually be executed do not necessarily match, and the integrated area is set as an area including all the operation areas. It only has to be. In other words, the image information restoration unit 33 assumes that the execution area is an integrated area.

Further, in the set integration area (execution area), as shown in FIG. 7, reference points are set at arbitrary locations.
The reference point is a location where the listener listens to the composite sound in the integrated region (execution region).
For example, the reference point can be set at the center position of the integrated region, or can be set at a position other than the center position. In terms of the relationship with each speaker 20, the reference point may be set at a position where the distance from each speaker 20 is the same, or may be set at a different position. However, it is desirable to set the reference point at the center position of the integrated region.
Moreover, although the reference point has one set location in the present embodiment, the reference point is not limited to one location, and may be a plurality of locations. In this case, a restored sound is generated for each of the set reference points.

Then, in the image information restoration unit 33, object position information, operation area information, and the like are extracted from the image analysis information, and a restored audio signal is generated based on the positional relationship between the object or operation area indicated by these pieces of information and the reference point. Is done.
That is, the image information restoration unit 33 calculates how the sound emitted from the object is heard at the reference point, that is, how the sound is heard by the listener located at the reference point. This calculated sound becomes the restored sound output in the audible space (sound source environment).
The restored audio signal generation process will be described later in “Procedure of audio restoration process (restored audio signal generation process) in the image information restoration unit of the automatic sound source environment restoration apparatus”.

The restored audio signal generated by the image information restoration unit 33 is sent to the storage unit 32 and stored therein. Further, the restored audio signal is amplified in the audio signal intensity by the audio control unit 34 and is output to each speaker 20 via the audio output unit 35 (audio output processing, step 17).
In each speaker 20, a restored voice signal is input (step 18), and a restored voice is output based on the restored voice signal (step 19).
By outputting the restored sound from each speaker 20 in such a processing procedure, the listener can stand at the reference point in the audible space (sound source environment) formed by the output restored sound, It is possible to grasp the operation state of each object image viewed from the entire execution area of the common software.

Next, the procedure of audio restoration processing (restored audio signal generation processing) in the image information restoration unit of the automatic sound source environment restoration device will be described with reference to FIG.
After the image analysis information from each information processing apparatus 10 is received by the communication unit 31 of the automatic sound source environment restoration device 30, the image information restoration unit 33 receives the image analysis information from the communication unit 31 (or stores it). The image analysis information table stored in the unit 32 is referred to).
Audio information, operation area information, object position information, object position information, object movement start information, object movement end information, operator position information, etc. are extracted from the received (or referred) image analysis information. (Alternatively, the information is extracted from the image analysis information table) (step 30).

Next, the position or the like of the object indicated by the extracted (or extracted) information is determined and grasped (step 31). Specifically, the following determination is made.
From the object position information, the position of the object currently displayed on the monitor unit 13 of each information processing apparatus 10 is determined.
From the object identification information, the type of the object is determined.
From the object movement start information, the position where the movement of the object is started is determined.
From the object movement end information, the position where the movement of the object is completed is determined.
From the operator position information, it is determined in which area of the software execution area each operator is located in each currently activated information processing apparatus 10.

Subsequently, an integrated area (execution area) is set based on each of the operation area information sent from the plurality of information processing apparatuses 10 (step 32).
Then, one place on the integrated area is set as a reference point (step 33).
Further, based on the relationship between the reference point and the position of the object or the like indicated by the above information, it is determined how the sound is heard at the reference point, and the sound is subjected to arithmetic processing to generate a restored sound signal. (Step 34).

As a specific example of the voice calculation process, for example, the following example is given.
(Example 1) It is assumed that an image of a vehicle is displayed on the monitor unit 13-1 and a state in which the vehicle moves is displayed by the control of the personal computer unit 12-1 of the information processing apparatus 10-1. On the other hand, it is assumed that a state where a tree is standing is displayed on the monitor unit 13-2 by the control of the personal computer unit 12-2 of the information processing apparatus 10-2. Assuming that the operation area displayed on the monitor unit 13-1 and the operation area displayed on the monitor unit 13-2 are close (or adjacent to each other), the sound emitted by the vehicle displayed on the monitor unit 13-1 Is reflected by the tree displayed on the monitor unit 13-2. This reflected sound is calculated by the image information restoration unit 33.
As described above, information on one or more objects displayed on each monitor unit 13 is generated by the personal computer unit 12 of the information processing apparatus 10, and based on the information on the object, the image information restoration unit 33 The reflected sound in one or more objects can be calculated to generate a restored sound.

  (Example 2) It is assumed that the monitor unit 13 of the information processing apparatus 10 displays a moving object (automobile) moving from the near side to the far side (from the front to the back). It is assumed that the listener located at the reference point hears a sound in which the moving object gradually moves farther forward from the near side, like the display content. The image information restoration unit 33 restores a sound assumed to be heard by the listener. When the operation area displayed on the monitor unit 13 is located in front of the listener located at the reference point, the restored sound is output from the speaker 20 in front of the listener.

  (Example 3) It is assumed that the monitor unit 13 of the information processing apparatus 10 displays a state in which a moving object (automobile) approaches from the far side (the back in the front) to the near side and further farther away. The listener located at the reference point is assumed to be able to hear the sound that the moving body approaches from the far side (the far back in the front) and goes far behind. The image information restoration unit 33 restores a sound assumed to be heard by the listener. When the operation area displayed on the monitor unit 13 is located in front of the listener located at the reference point, the moving body is far away from the speaker 20 located in front of the listener (the far rear in the front). ), The restored sound approaching the near side is output from the rear speaker 20, and the restored sound that the moving body approaches and further rearward is output.

Here, the generation of the restored sound accompanying the movement of the object image has been mainly described, but in addition, for example, the restored voice is selected according to the moving speed of the object, the type of the moving object, the nature of the sound, the environment in which the sound propagates, etc. Can be generated.
Also, as shown in Example 3, different restored sounds can be output from each speaker 20.

[Sound source environment restoration program]
Next, the sound source environment restoration program will be described.
The sound source environment restoration function (function for generating a restored voice signal and outputting the restored voice) of the computer (information processing apparatus, automatic sound source environment restoration apparatus, sound source environment restoration system) in the above-described embodiment is stored in each storage unit (for example, , ROM, etc.) and realized by a sound source environment restoration program.

The sound source environment restoration program is read into the computer to send instructions to each component of the computer, and performs predetermined processing, for example, software activation processing of the personal computer unit in the information processing apparatus, image generation processing, object image information generation processing Object image display processing of the monitor unit, object image information analysis processing of the image information analysis unit, image analysis information transmission processing of the communication unit, image analysis information reception processing of the communication unit in the automatic sound source environment restoration device, restoration of the image information restoration unit An audio signal generation process, a restored voice amplification process of the voice control unit, a restored voice signal output process of the voice output unit, and the like are performed.
As a result, the sound source environment restoration function is realized by an information processing device, an automatic sound source environment restoration device, and a sound source environment restoration system including them, in which a sound source environment restoration program that is software and a computer that is a hardware resource cooperate. The

A sound source environment restoration program for realizing the sound source environment restoration function is stored in a computer-readable recording medium, such as an external storage device and a portable recording medium, in addition to being stored in a ROM or a hard disk of a computer. be able to.
The external storage device is a memory expansion device that incorporates a storage medium such as a CD-ROM and is externally connected to the automatic sound source environment restoration device. On the other hand, the portable recording medium is a recording medium that can be mounted on a recording medium driving device (drive device) and is portable, and refers to, for example, a flexible disk, a memory card, a magneto-optical disk, and the like.

The program recorded on the recording medium is loaded into the RAM of the computer and executed by the CPU. By this execution, the function of the automatic sound source environment restoration apparatus of the present embodiment described above is realized.
Furthermore, when the sound source environment restoration program is loaded by the computer, the sound source environment restoration program held by another computer can be downloaded to its own RAM or an external storage device using a communication line. The downloaded sound source environment restoration program is also executed by the CPU to realize the sound source environment restoration function of the automatic sound source environment restoration device of the present embodiment.

  The preferred embodiments of the sound source environment restoration system, automatic sound source environment restoration device, sound source environment restoration method, and sound source environment restoration program of the present invention have been described above, but the sound source environment restoration system, automatic sound source environment restoration device, and sound source according to the present invention have been described. It goes without saying that the environment restoration method and the sound source environment restoration program are not limited to the above-described embodiments, and various modifications can be made within the scope of the present invention.

For example, in the embodiment described above, a plurality of speakers are provided in the audible space. However, the present invention is not limited to the installation of a plurality of speakers. For example, the monitor unit may be provided to display an image of the integrated region. it can. As a result, the listener who is located in the audible space can see each object image viewed from the entire integrated area (execution area) not only from the restored sound output from each speaker but also from the image displayed on the monitor unit. You can grasp the operating status of.
In this case, the image information restoration unit 33 of the automatic sound source environment restoration device 30 performs an interpolation process that connects the operation regions displayed on the monitor unit 13 of each information processing device 10 to form a continuous region. Are output to the monitor. The monitor unit displays an image in an arbitrary direction by shifting the installed controller in an arbitrary direction.

  In the above embodiment, the automatic sound source environment restoration device receives and collects various information included in object image information from each information processing device, but is limited to receiving and collecting from each information processing device. For example, the automatic sound source environment restoration device itself can download software via a network and extract voice information, an execution area, and the like.

  Since the present invention relates to the control of sound output in accordance with the operation of an object image displayed on the screen by execution of software, the present invention can be used for a system or apparatus having image display means and sound output means. is there.

It is a block diagram which shows the structure of the sound source environment restoration system of this invention. It is a block diagram which shows the structure of information processing apparatus. It is a block diagram which shows the relationship between the operation area | region of each information processing apparatus, and the execution area of software. It is a block diagram which shows the structure of the automatic sound source environment restoration apparatus of this invention. It is a table block diagram which shows the structure of the image analysis information table memorize | stored in the automatic sound source environment restoration apparatus of this invention. It is an operation | movement procedure figure which shows the process sequence of the sound source environment restoration method in the sound source environment restoration system of this invention. It is explanatory drawing which shows the execution area and reference | standard point of software. It is a flowchart which shows the procedure of a decompression | restoration audio | voice signal production | generation process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Sound source environment restoration system 10 Information processing apparatus 12 Personal computer part 13 Monitor part 14 Image information analysis part 15 Communication part 20 Speaker 30 Automatic sound source environment restoration apparatus 31 Image information restoration part 35 Audio | voice output part

Claims (7)

A plurality of information processing devices having monitor units for displaying object images;
One or two or more speakers that output sound related to each of the object images to form an audible space;
Audio information related to each object image is received from the plurality of information processing devices, the received audio information is integrated, restored as audio output in the audible space, and the restored audio is sent to each speaker. A sound source environment restoration system comprising an automatic sound source environment restoration device for outputting. Each of the information processing apparatuses executes common software, and performs a process of operating the object image using an operation area as a partial area of the execution area of the software,
The automatic sound source environment restoration device is
One point in the execution area as a reference point,
The position information and / or the operation area information related to the object is received from each of the information processing devices together with the audio information related to the object image, and the object and / or the operation area and the reference indicated by the received information are received. The sound source environment restoration system according to claim 1, wherein the restoration sound is generated based on a positional relationship with a point. The sound source environment restoration system according to claim 2, wherein the automatic sound source environment restoration device sets the execution area based on information on each operation area received from each information processing apparatus. Each of the information processing devices generates information on one or more objects displayed on the monitor unit,
2. The automatic sound source environment restoration device calculates a reflected sound from the one or more objects based on the received information on the one or more objects, and generates the restored sound. The sound source environment restoration system according to any one of? A receiving unit that receives audio information about the object image displayed on the monitor unit from a plurality of information processing devices;
A sound restoration unit that integrates a plurality of the sound information and restores the sound as an output sound in an audible space formed by output sound from one or more speakers;
An automatic sound source environment restoration apparatus comprising: an audio output unit that sends restored audio to each speaker and outputs the audio. A reception process for receiving audio information related to the object image displayed on the monitor unit from a plurality of information processing devices;
A sound restoration process for integrating a plurality of the sound information and restoring the sound as an output sound in an audible space formed by output sound from one or more speakers;
A sound source environment restoring method comprising: sound output processing for sending restored sound to each of the speakers and outputting the same. A reception process for receiving audio information related to the object image displayed on the monitor unit from a plurality of information processing devices;
A sound restoration process for integrating a plurality of the sound information and restoring the sound as an output sound in an audible space formed by output sound from one or more speakers;
A sound source environment restoration program characterized by causing an automatic sound source environment restoration device to execute sound output processing for sending restored sound to each speaker for output.

Images