JP2019176229A - Signal synchronization method - Google Patents

Abstract

To improve a synchronization accuracy in the case where an object signal is synchronized by using a sound signal acquired in a device moving to a sound source.SOLUTION: A signal synchronization method includes: a step of acquiring a first sound signal acquired in a first device moving to a sound source; a step of acquiring a first object signal of which a time difference from the first sound signal is known; a step of acquiring a second sound signal acquired in a second device different from the first device; a step of acquiring a second object signal of which a time difference from the second sound signal is known; and a step of synchronizing the first and second object signals by using a signal obtained by performing time stretch of at least a section of the first sound signal on the basis of the movement and using the second sound signal.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a technique for synchronizing a plurality of signals.

  A technique for synchronizing various signals such as video or audio is known. Patent Document 1 describes a video processing device that receives material data by wireless communication from a terminal device that records material data including sound and moving images, and generates content by editing a plurality of material data.

JP 2017-17423

  In the technique described in Patent Document 1, when moving images are recorded while the apparatus is moving, the pitch (pitch) changes due to Doppler shift, and there is a possibility that the signals cannot be accurately synchronized.

  In contrast, the present invention provides a technique for improving the accuracy of synchronization when a target signal is synchronized using a sound signal acquired in a device moving with respect to a sound source.

  The present invention includes a step of acquiring a first sound signal acquired in a first device moving with respect to a sound source, a step of acquiring a first target signal whose time difference from the first sound signal is known, Obtaining a second sound signal obtained in a second device different from the first device, obtaining a second target signal whose time difference from the second sound signal is known, and based on the movement A signal synchronization method comprising: a signal obtained by time-stretching at least a part of one sound signal; and a step of synchronizing the first target signal and the second target signal using the second sound signal.

  The signal synchronization method may include a step of dividing the first sound signal into a plurality of sections, and time-stretching each of the plurality of sections with an individually set expansion rate.

  In this signal synchronization method, for a signal that has been synchronized after time-stretching, a step of changing the length of the division and dividing it again into a plurality of sections, and for each of the newly divided sections, Time stretch with a stretch rate set to, a result synchronized using a time stretched signal using multiple sections divided by a certain length, and the length and another length A step of adopting a result having a higher degree of synchronization among results synchronized using a signal that has been time stretched using a plurality of sections.

  The step of synchronizing the first target signal and the second target signal includes, for at least one parameter related to the time stretch, the first time stretched using each of the parameters quantized at a first interval. Using the sound signal and the second sound signal to synchronize the first target signal and the second target signal, and after the time stretch, the parameter is more detailed than the first interval. Synchronizing the first target signal and the second target signal using the first sound signal and the second sound signal time-stretched using each of the parameters quantized at intervals of two; You may have.

  ADVANTAGE OF THE INVENTION According to this invention, when synchronizing a target signal using the sound signal acquired in the apparatus which is moving with respect to a sound source, the precision of a synchronization can be improved.

The figure which illustrates the outline | summary of the signal synchronization system 1 which concerns on one Embodiment. The figure which illustrates the function structure of the signal synchronizer 10. FIG. The figure which illustrates the hardware constitutions of the signal synchronizer 10. 3 is a diagram illustrating a functional configuration of the device 20. FIG. FIG. 3 is a diagram illustrating a hardware configuration of the device 20. The flowchart which illustrates operation | movement (signal synchronization method) of the signal synchronizer 10. FIG. The figure which illustrates the time stretch process of signal Sr.

1. Configuration FIG. 1 is a diagram illustrating an overview of a signal synchronization system 1 according to an embodiment. The signal synchronization system 1 is a system for synchronizing the signal St [1] and the signal St [2]. The signal St [1] and the signal St [2] are acquired in the device 20 [1] and the device 20 [2], respectively. The signal St [1] and the signal St [2] are examples of the first target signal and the second target signal to be synchronized, and the device 20 [1] and the device 20 [2] are the first device and the second target signal. 2 is an example of an apparatus. The device 20 [1] acquires the signal Sr [1] in addition to the signal St [1]. The device 20 [2] acquires the signal Sr [2] in addition to the signal St [2]. The signal Sr [1] and the signal Sr [2] are sound signals whose time differences from the signal St [1] and the signal St [2] are respectively known (for example, simultaneously). The signal synchronizer 10 receives input of the signal St [1] and the signal Sr [1] from the apparatus 20 [1], and receives input of the signal St [2] and the signal Sr [2] from the apparatus 20 [2]. The signal synchronizer 10 synchronizes the signal St [1] and the signal St [2] based on the cross-correlation between the signal Sr [1] and the signal Sr [2] after time stretching described later. When the signals St [1] and St [2] are not distinguished from each other, they are simply expressed as a signal St. The same applies to the signal Sr and the device 20.

  Each of the signal St and the signal Sr is a signal output from a sensor, for example. The sensor is, for example, a video sensor (imaging device), a sound sensor (microphone), an acceleration sensor, or a temperature sensor. The device 20 [1] and the device 20 [2] are devices on which these sensors are mounted, for example, a smartphone, a camera, or a voice recorder. In one example, the signal St is a video signal output from the video sensor, and the signal Sr is a sound signal output from the microphone. The signal Sr [1] and the signal Sr [2] are examples of the first sound signal and the second sound signal. In this example, at least one of the device 20 [1] and the device 20 [2] outputs a signal St and a signal Sr while moving with respect to a sensing object (for example, a subject or a sound source).

  FIG. 2 is a diagram illustrating a functional configuration of the signal synchronization apparatus 10. The signal synchronization device 10 includes a signal acquisition unit 11, a time stretch unit 12, a synchronization unit 13, a storage unit 14, a control unit 15, and a communication unit 16. The signal acquisition unit 11 acquires the signal St [1] and the signal Sr [1] from the device 20 [1], and acquires the signal St [2] and the signal Sr [2] from the device 20 [2]. The signal acquisition unit 11 may acquire the signal St [1] and the signal Sr [1] at the same time, or may acquire them at different timings. The same applies to the signal St [2] and the signal Sr [2]. The time stretcher 12 time stretches at least a part of at least one of the signals Sr [1] and Sr [2]. Here, time stretching refers to a process of extending a sound signal in the time axis direction, and specifically, for example, a resampling process. Details of the time stretch will be described later. A signal obtained by time-stretching the signal Sr is referred to as a signal Sr_ts. When the device 20 [1] moves and outputs the signal St and the signal Sr, the time stretch unit 12 time stretches the signal Sr [1] to obtain the signal St_ts [1]. The synchronization unit 13 synchronizes the signal St [1] and the signal St [2] based on the cross-correlation between the signal Sr_ts [1] and the signal Sr [2]. The storage unit 14 stores various data. The control unit 15 performs various controls. The communication unit 16 communicates with other devices.

  FIG. 3 is a diagram illustrating a hardware configuration of the signal synchronization apparatus 10. In this example, the signal synchronization device 10 is a computer device having a CPU (Central Processing Unit) 101, a memory 102, a storage 103, and a communication IF 104. The CPU 101 is a control device that performs various calculations according to a program and controls other hardware elements of the signal synchronization device 10. The memory 102 is a main storage device that functions as a work area when the CPU 101 executes processing, and includes, for example, a ROM (Read Only Memory) and a RAM (Random Access Memory). The storage 103 is an auxiliary storage device that stores various data and programs, and includes, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive). The communication IF 104 is a device for communicating with other devices in accordance with a predetermined communication standard, and includes, for example, a NIC (Network Interface Card).

  In one example, the signal synchronization apparatus 10 is a server apparatus that operates on a network (for example, the Internet). The storage 103 stores a program for causing the computer apparatus to function as the signal synchronization apparatus 10 in the signal synchronization system 1 (hereinafter referred to as “synchronization program”). The CPU 101 reads the synchronization program from the storage 103 and executes processing in the memory 102, whereby the functions shown in FIG. In a state where the CPU 101 is executing the synchronization program, the CPU 101 is an example of the signal acquisition unit 11, the time stretch unit 12, the synchronization unit 13, and the control unit 15. At least one of the memory 102 and the storage 103 is an example of the storage unit 14. The communication IF 104 is an example of the communication unit 16.

  FIG. 4 is a diagram illustrating a functional configuration of the device 20. The apparatus 20 includes a generation unit 21, an acquisition unit 22, a storage unit 23, a transmission unit 24, a UI unit 25, and a control unit 26. The generation unit 21 generates a signal St. The acquisition unit 22 acquires the signal Sr. The storage unit 23 stores various data. The data stored in the storage unit 23 includes data obtained by sampling the signal St and the signal Sr. Hereinafter, data obtained by sampling the signal St and the signal Sr are referred to as data Dt and data Dr, respectively. The transmission unit 24 transmits the data Dt and the data Dr to the signal synchronization device 10. The UI unit 25 outputs information to the user of the device 20 and inputs an instruction from the user. The control unit 26 performs various controls.

  FIG. 5 is a diagram illustrating a hardware configuration of the device 20. The device 20 is a computer device having a CPU 201, a memory 202, a storage 203, a mobile communication unit 204, a wireless LAN communication unit 205, a camera 206, a microphone 207, and a touch screen 208. The CPU 201 is a control device that performs various calculations according to a program and controls other hardware elements of the device 20. The memory 202 is a main storage device that functions as a work area when the CPU 201 executes processing, and includes, for example, a ROM and a RAM. The storage 203 is an auxiliary storage device that stores various data and programs, and includes, for example, an SSD or an HDD. The mobile communication unit 204 is a device for communicating with other devices in accordance with a predetermined mobile communication standard (for example, LTE), and includes, for example, an antenna and a chip set that comply with the communication standard. The wireless LAN communication unit 205 is a device for communicating with other devices in accordance with a predetermined wireless LAN communication standard (for example, WiFi (registered trademark)), and includes, for example, an antenna and a chip set conforming to the communication standard. The camera 206 outputs a video signal obtained by photographing the target. The microphone 207 outputs a sound signal obtained from the target. The camera 206 and the microphone 207 are an example of a sensor that detects a target situation and outputs a signal indicating the result. The touch screen 208 includes a display device (for example, an LCD) that displays information and a touch sensor formed on the display device.

  In one example, the device 20 is a device having a function of accessing a network to which the signal synchronization device 10 is connected, for example, a smartphone. The signal St is a video signal indicating a moving image shot with a smartphone. The storage 203 stores a program (data acquisition program) for causing a computer device to function as the device 20 in the signal synchronization system 1. In a situation where the CPU 201 is executing a data acquisition program, the camera 206 is an example of the generation unit 21. The microphone 207 is an example of the acquisition unit 22. The CPU 201 is an example of the control unit 26. At least one of the memory 202 and the storage 203 is an example of the storage unit 23. The mobile communication unit 204 or the wireless LAN communication unit 205 is an example of the transmission unit 24. The touch screen 208 is an example of the UI unit 25.

  In one example, the device 20 executing the data acquisition program displays a UI screen (not shown) including a recording button on the touch screen 208. The recording button is a UI element for receiving an instruction to start recording. When the user touches a position corresponding to the recording button on the touch screen 208, the CPU 201 starts recording the signal St and the signal Sr. During recording of the signal St and the signal Sr, the device 20 displays a UI screen (not shown) including a stop button on the touch screen. The stop button is a UI element for receiving an instruction to start recording. When the user touches a position corresponding to the stop button on the touch screen 208, the CPU 201 gives attribute information (or tag information) such as a time stamp to the data Dm including the data indicating the signal St and the data indicating the signal Sr. And stored in the storage 203. The CPU 201 starts recording the signal St and the signal Sr at the same time and ends them simultaneously. The timing for recording the signal St and the signal Sr only needs to be known in time, and does not need to be recorded simultaneously.

2. Operation FIG. 6 is a flowchart illustrating the operation (signal synchronization method) of the signal synchronization apparatus 10. Before starting the flow of FIG. 6, the device 20 [1] acquires data Dt [1] and data Dr [1], and the device 20 [2] acquires data Dt [2] and data Dr [2]. It is done. In the following description, a functional element such as the signal acquisition unit 11 may be described as a subject of processing. This is because a hardware element such as the CPU 101 that executes software such as a synchronization program is replaced with other hardware. This means that the process is executed in cooperation with the element.

  In step S101, the signal acquisition unit 11 acquires data Dt [1] from the device 20 [1]. For example, the data Dt [1] is transmitted from the device 20 [1] to the signal synchronization device 10 in response to an event that the user inputs a data transmission instruction in the device 20 [1]. The device 20 [1] adds the attribute information to the data Dt [1] and sends it. The attribute information is information accompanying the data D [1]. The attribute information includes, for example, at least one of an identifier of the device 20, an identifier of the user of the device 20, a description of the subject, a time stamp, and position information (latitude and longitude of the device 20 when the image is captured). . Attribute information is similarly added to other data such as data Dr [2]. In step S102, the signal acquisition unit 11 acquires data Dr [1] from the device 20 [1]. The data Dr [1] is transmitted from the device 20 [1] to the signal synchronization device 10 triggered by the same event as the data Dt [1], for example. Alternatively, the data Dr [1] may be transmitted from the device 20 [1] to the signal synchronization device 10 triggered by an event different from the data Dt [1]. In this case, the relationship with the data Dt [1] is specified by the identifier included in the attribute information of the data Dr [1]. The storage unit 14 stores the data Dt [1] and the data Dr [1] acquired by the signal acquisition unit 11.

  In step S103, the signal acquisition unit 11 acquires data Dt [2] from the device 20 [2]. In step S104, the signal acquisition unit 11 acquires data Dr [2] from the device 20 [2]. Details of steps S103 and S104 are the same as steps S101 and S102. The storage unit 14 stores the data Dt [2] and the data Dr [2] acquired by the signal acquisition unit 11.

  In step S105, the control unit 15 receives an input of an instruction to synchronize two data (hereinafter referred to as “synchronization instruction”). The data synchronization here means the synchronization of the signal waveform indicated by the data and is synonymous with the signal synchronization. This instruction is input from, for example, a terminal device (not shown; for example, a smartphone or a PC that can access the Internet) that can access the signal synchronization device 10. This instruction includes information specifying data Dt [1] and data Dt [2] to be subjected to synchronization processing.

  In step S106, the time stretch unit 12 performs time stretch processing on the target section of the time stretch target signal and outputs a signal Sr_ts. The time stretch target signal is a signal to be subjected to time stretch processing among the signals Sr [1] and Sr [2]. The target of the time stretch process is, for example, a signal acquired in the device 20 moving with respect to the sound source. When the signal Sr is acquired, if the device 20 [1] is moved with respect to the sound source, the signal Sr [1] is a target of time stretch processing. When the signal Sr is acquired, if the device 20 [2] is moved with respect to the sound source, the signal Sr [2] is a target of time stretch processing. For example, when the device 20 [1] and the device 20 [2] that are moving with respect to the sound source are known, the signal Sr acquired in the device 20 is the target of the time stretch process. When it is not known which of the devices 20 [1] and 20 [2] is moving with respect to the sound source, the devices 20 [1] and 20 [2] perform time stretch processing one by one in order. It becomes a target.

  The target section refers to a section that is a target of time stretch processing in the time stretch target signal. The target section is a part or all of the time domain of the time stretch target signal. In one example, the time stretch unit 12 equally divides the time stretch target signal into a plurality of sections (for example, 10 sections). The time stretch unit 12 sets one of the ten sections as a target section in order. For example, the time stretch unit 12 first sets the first section as the target section. After the process to be described later is completed, the time stretch unit 12 next sets the second section as the target section. Hereinafter, the time stretch unit 12 sets the target sections in order up to the tenth section.

Next, the resampling process will be described as an example of the time stretch process. Now, consider an example in which the observer 20 [1] moves at a speed v1 and the sound source moves at a speed v2. The frequency of the sound emitted by the sound source is f. As a result of the Doppler shift, the observer hears sound of frequency fp.
fp = {(c−v2) / (c−v1)} f = αf (1)
α≡ (c−v2) / (c−v1) (2)
Here, when both the observer and the sound source are moving at a constant velocity, α is constant, so the observer may resample at a time interval of Δt / α. Here, Δt is the (original) sampling period of the signal Sr [1] (for example, Δt = 1 / fs (fs is the sampling frequency)). On the other hand, if either the observer or the sound source is accelerating, if the temporal change of the velocity v1 or the velocity v2 or the acceleration β is known, the observer sequentially Re-sampling may be performed at a time interval of Δt / β. When the speed of the observer or the sound source is unknown, the time stretch unit 12 performs resampling processing by changing the resampling period, and searches for an optimum condition.

In step S107, the synchronization unit 13 synchronizes the signal specified by the synchronization instruction. Here, synchronizing the signals means specifying a positional shift (difference) between the two signals in the time domain. In one example, the signal St [1] and the signal St [2] are synchronized based on the cross-correlation of the signal Sr [1] and the signal Sr [2]. Specifically, the synchronization unit 13 calculates a time difference τ that maximizes the absolute value | Cij (τ) | of the cross-correlation Cij (τ) expressed by the following equation (1).
Here, the signal yi is a signal indicated by the data Dr [1], that is, the signal Sr [1], and the signal yj is a signal indicated by the data Dr [2], that is, the signal Sr [2]. At least one of the signal Sr [1] and the signal Sr [2] is time-stretched in step S106. Equation (1) is obtained by matching the start point of the signal yi (t) and the start point of the signal yj (t) in the time domain, and then the time difference (shift on the time axis) of the signal yj (t) with respect to the signal yi (t) A numerical string indicating the degree of correlation of the signal waveform between the two using the amount τ as a variable. The time difference τ can be a positive value or a negative value. For example, when the signal Sr [2] is located behind the signal Sr [1] in the time domain, the time difference τ is positive, and the signal Sr [2] is located ahead of the signal Sr [1] in the time domain. The time difference τ is negative. The cross-correlation is not limited to that calculated by the equation (1), and may be calculated by the following equation (2), for example.

  In step S108, the synchronization unit 13 stores the absolute value | Cij (τ) | of the cross-correlation Cij (τ) in the storage unit 14 as a value indicating the degree of synchronization.

  In step S109, the synchronization unit 13 determines whether all time stretch conditions are considered. The time stretch condition is a condition related to the time stretch process, and is, for example, a time stretch target signal, a target section, and the number of divisions of the target section. When both the signal Sr [1] and the signal Sr [2] are time stretch target signals, the synchronization unit 13 determines the absolute value of the cross-correlation Cij (τ) for both the signal Sr [1] and the signal Sr [2]. It is determined whether the value | Cij (τ) | Further, the synchronization unit 13 determines whether all sections of the time stretch target signal are considered as target sections. Further, the synchronization unit 13 determines whether all the division numbers to be considered have been considered. For example, when 10 divisions, 5 divisions, and 3 divisions are objects to be considered, the synchronization unit 13 performs the first to the tenth sections when the target signal is divided into ten, the first to the fifth sections when the division is performed, Then, it is determined whether or not the absolute value | Cij (τ) | of the cross-correlation Cij (τ) has been calculated for all of the first to third sections obtained by dividing into three. The number of divisions of the time stretch target signal is a parameter related to the time stretch process and is an example of a quantized parameter. When there is a combination of conditions that are not considered (S109: NO), the synchronization unit 13 updates the target condition, and the process proceeds to step S106 again. When it is determined that all the time stretch conditions have been taken into account (S109: YES), the synchronization unit 13 proceeds to step S110.

  FIG. 7 is a diagram illustrating time stretching processing of the signal Sr. In this example, the signal Sr [1] is a time stretch target signal. Of the signal Sr [1], the device 20 [1] took an image while moving relative to the sound source in the section Dv (FIG. 7A). The time stretcher 12 divides the signal Sr [1] into four sections D1 to D4 (FIG. 7B). For example, for the section D2, the time stretch unit 12 performs the time stretch process by changing the expansion rate to various values. At this time, the time stretcher 12 keeps the time length of the section D2 constant with the original signal. For example, when ten types of values are employed as the expansion rate, the signal Sr [1] subjected to time stretch processing using each expansion rate is represented as signal Sr [1,1] to signal Sr [1,10]. The synchronizer 13 calculates a cross-correlation with the signal Sr [2] for each of these ten signals Sr [1,1] to [1,10]. The synchronization unit 13 obtains the time difference τ from the calculation result between the mutual layers using the signal Sr [1, x] having the maximum cross-correlation among these ten calculation results.

  As shown in this example, the boundaries of the sections D1 to D4 do not have to coincide with the boundaries of the section Dv. Even if the time stretch process is performed in a state where the position of the section Dv is unknown, the synchronization accuracy can be improved as compared with the case where the time stretch process is not performed at all.

  Refer to FIG. 6 again. In step S110, the synchronization unit 13 determines a time stretch condition that maximizes the absolute value | Cij (τ) | of the cross-correlation Cij (τ) from among a plurality of time stretch conditions. The synchronization unit 13 determines the time difference τ calculated under the time stretch condition as the time difference τ between the signal St [1] and the signal St [2].

  In step S111, the control unit 15 generates data Ds indicating a signal Ss obtained by combining the synchronized signal St [1] and the signal St [2]. In step S112, the control unit 15 outputs data Ds. For example, the control unit 15 outputs the data Ds to, for example, the terminal device that has output the synchronization instruction. Alternatively, the control unit 15 may store the data Ds in the storage unit 14.

  In the case where the signal St is a video signal, the data Ds indicates a video including an area for displaying the video indicated by the signal St [1] and an area for displaying the video indicated by the signal St [2] in one screen. It is data. The data Ds includes at least one of the sound indicated by the signal Sr [1] and the sound indicated by the signal Sr [2] as audio synchronized with the video. In one example, the data Ds includes a signal Sr that has not been time-stretched under a time-stretch condition in which the absolute value | Cij (τ) | of the cross-correlation Cij (τ) is maximum, and includes a signal Sr that has been time-stretched. There is no (that is, only the sound signal acquired in the device 20 not moving relative to the sound source is included). Alternatively, the data Ds may include the signal Sr [1] in the first channel and the signal Sr [2] in the second channel.

  According to the present embodiment, in the example in which the signal St is synchronized using the sound signal acquired in the apparatus moving with respect to the sound source as the signal Sr, the synchronization accuracy is improved as compared with the case where the time stretch process is not used. Can be made.

  This embodiment can be used for the following scenes as an example. The devices 20 [1] and 20 [2] are both video cameras with built-in microphones. The object of shooting is, for example, a swimming competition or an athletics competition. A rail is installed in advance along the lane of the pool or track, and the device 20 [1] photographs the player while moving on the rail. The device 20 [2] is fixed at the start point or the goal point. The sound source is BGM (Background Music) flowing in the venue. Alternatively, the sound source may be a player to be photographed (in this case, the signal Sr is a signal of a splashing sound or a sound of kicking the ground with a foot). The use scene of the present invention is not limited to this. For example, the present invention may be used in a scene where a performer in a concert is photographed using a fixed camera and a moving camera. Alternatively, a vehicle such as an automobile may be the subject.

3. Modifications The present invention is not limited to the above-described embodiments, and various modifications can be made. Hereinafter, some modifications will be described. Two or more of the following modifications may be used in combination.

3-1. Modification 1
When the time stretch target signal is divided into a plurality of sections, the time stretch unit 12 may perform time stretch processing on two or more sections. In this case, the time stretch unit 12 may perform the time stretch process using the expansion rate set individually for each section.

3-2. Modification 2
When searching for the time stretch condition that maximizes the cross-correlation, the time stretch unit 12 first narrows down the optimum range with coarse accuracy for at least one parameter related to the time stretch condition, and then optimizes the condition with more detailed accuracy. May also be performed. In one example, the time stretch unit 12 divides the time stretch target signal into, for example, five, and obtains an optimum expansion rate for each section. Thereafter, the time stretcher 12 further divides a section having the largest absolute value of the difference in expansion rate from the original signal into a plurality of sections, and searches for a condition that maximizes the cross-correlation.

3-3. Modification 3
For example, when the frequency of the sound emitted from the sound source is known, the time stretcher 12 may search for the time stretch condition so that the frequency of the signal Sr matches the frequency of the sound source.

3-4. Other Modifications A specific method for synchronizing two signals is not limited to using cross-correlation. For example, the signals St [1] and St [2] may be synchronized by pattern matching between the signal Sr [1] and the signal Sr [2].

  The hardware configuration for realizing the function of the signal synchronization system 1 is not limited to the one exemplified in the embodiment. The signal synchronization system 1 may have any hardware configuration as long as the required function can be realized. In addition, the correspondence relationship between functions and hardware is not limited to that illustrated in the embodiment. For example, the functions implemented in the signal synchronization apparatus 10 in the embodiment may be distributed and implemented in two or more apparatuses. In addition, at least a part of the functions implemented in the signal synchronization apparatus 10 in the embodiment or the modification may be implemented in the apparatus 20. For example, the device 20 itself may have a function as the signal synchronization device 10. In this case, for example, the device 20 [1] acquires the data Dt [2] and the data Dr [2] from the device 20 [2], and uses the data Dt [1] and the data Dr [1] stored by itself. Use to synchronize the signal St [1] and the signal St [2].

  In the embodiment, an example in which two signals are synchronized has been described, but the number of signals to be synchronized is not limited to two. Three or more signals may be synchronized.

  The program executed in the signal synchronization apparatus 10 and the apparatus 20 may be provided by a storage medium such as an optical disk, a magnetic disk, or a semiconductor memory, or may be downloaded via a communication line such as the Internet. Further, this program need not include the steps exemplified in the embodiment. Some of these steps may be omitted. Further, the order of some of these steps may be changed.

DESCRIPTION OF SYMBOLS 1 ... Signal synchronization system, 10 ... Signal synchronization apparatus, 11 ... Signal acquisition part, 12 ... Time stretch part, 13 ... Synchronization part, 14 ... Memory | storage part, 15 ... Control part, 16 ... Communication part, 20 ... Apparatus, 21 ... Generation unit, 22 ... acquisition unit, 23 ... storage unit, 24 ... transmission unit, 25 ... UI unit, 26 ... control unit, 101 ... CPU, 102 ... memory, 103 ... storage, 104 ... communication IF, 201 ... CPU, 202 ... Memory, 203 ... Storage, 204 ... Mobile communication unit, 205 ... Wireless LAN communication unit, 206 ... Camera, 207 ... Microphone, 208 ... Touch screen

Claims (4)

Obtaining a first sound signal obtained in a first device moving relative to a sound source;
Obtaining a first target signal having a known time difference from the first sound signal;
Obtaining a second sound signal obtained in a second device different from the first device;
Obtaining a second target signal having a known time difference from the second sound signal;
A step of time-stretching at least a part of the first sound signal based on the movement, and a step of synchronizing the first target signal and the second target signal using the second sound signal. Signal synchronization method. Dividing the first sound signal into a plurality of sections;
The signal synchronization method according to claim 1, wherein each of the plurality of sections is time-stretched at an expansion rate set individually. For a signal synchronized after being time stretched, the step of changing the length of the division and dividing it again into a plurality of sections;
For each of a plurality of newly divided sections, a step of time stretching with an individually set expansion rate;
A result of synchronization using a signal that has been time-stretched using a plurality of sections divided by a certain length, and a signal that has been time-stretched by using a plurality of sections that are divided by a length different from that result. The signal synchronization method according to claim 2, further comprising a step of adopting a result having a higher degree of synchronization among results synchronized using. Synchronizing the first target signal and the second target signal includes:
For at least one parameter related to the time stretch, the first target signal using the first sound signal and the second sound signal time-stretched using each of the parameters quantized at a first interval. And synchronizing the second target signal;
After the time stretching, the first sound signal and the second sound time-stretched using each of the parameters quantized at a second interval that is more detailed than the first interval. The signal synchronization method according to any one of claims 1 to 3, further comprising: synchronizing the first target signal and the second target signal using a signal.