JP2017123918A - Respiration monitoring device, respiration rate measurement device, respiration rate measurement method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To measure a respiratory rate of a subject at a higher speed from a moving image of the subject. SOLUTION: A respiratory rate monitoring device is a continuous N image group composed of N (N: integers of 2 or more) frame images of continuous moving images, and S frame images shifted. The second image group composed of a frame image is acquired, the correlation coefficient between the image feature amount of the first image group and the image feature amount of the second image group is calculated, and the first image group and the second image are calculated. The number S of frame images, which is the difference from the group, is changed, and it is determined whether or not the respiratory cycle can be specified by the transition of the correlation coefficient with respect to the change. If the respiratory cycle cannot be specified, the respiratory rate monitoring device resets the number of frames S. [Selection diagram] FIG. 3B

Description

  The present invention relates to a technique for measuring the respiration rate of a subject from a moving image obtained by photographing the subject.

  Patent Document 1 is known in the technical field of calculating a human respiratory rate from a moving image obtained by photographing a human. The apparatus of Patent Literature 1 generates time-series data from fluctuations in image density in a human moving image captured by a camera. And this apparatus calculates | requires the power spectrum of time series data, and calculates the respiration rate using the peak frequency.

JP 2014-171574 A

  The technique described in Patent Document 1 has room for improvement with respect to followability to changes in respiratory rate. Specifically, since the system described in Patent Document 1 calculates the respiration rate based on the peak frequency of the power spectrum, it matches the minimum respiration rate (respiration with the lowest frequency) in the respiration rate measurement range. It was necessary to enlarge the time window for calculating the power spectrum. Since the power spectrum calculation analyzes the frequency of data in the entire time window, it is difficult to analyze changes that occur in a short time when the time window becomes large. In addition, when the time window becomes large, it takes time for data changes to appear in the analysis result. The technique described in Patent Document 1 has room for further improving the followability to changes in respiration.

  The present invention has been made to solve the above-described problems, and provides a technique for measuring the respiration rate of a subject with good follow-up from a moving image obtained by photographing the subject.

  The respiration rate monitoring apparatus according to an exemplary embodiment of the present invention is a series of N (N: an integer greater than or equal to 2) frame images consecutive from a specific frame image of a moving image in which a subject to breathe is captured. A first image group acquisition processing unit configured to acquire data of the configured first image group, and N consecutive frames starting from the previous frame image by S (S: an integer of 1 or more) from the specific frame image A second image group acquisition processing unit that acquires data of the second image group composed of the frame images, and an image feature amount of each frame image of the first image group is calculated, and is a set of the image feature amounts. A first image feature quantity group creation processing unit for creating a first image feature quantity group; and an image feature quantity of each frame image of the second image group is calculated, and a second image feature quantity that is a set of the image feature quantities Second image feature group creation processing for creating a group A correlation coefficient calculation processing unit that calculates a correlation coefficient between the first image feature quantity group and the second image feature quantity group, and a frame that is a difference between the first image group and the second image group A parameter for resetting the number of frames S when the respiratory cycle cannot be specified by the respiratory cycle specification processing unit that attempts to specify the respiratory cycle and the respiratory cycle specification determination processing unit based on the relationship between the number S of images and the correlation coefficient. A reset processing unit.

  In one embodiment, the respiratory rate monitoring apparatus further includes an interface that receives region designation information for designating a partial region in the frame image, and the first image feature quantity group creation processing unit includes the partial image information processing unit. The image feature quantity of the area is calculated to create the first image feature quantity group, and the second image feature quantity group creation processing unit calculates the image feature quantity of the partial area to calculate the second image feature quantity. Create a quantity group.

  In one embodiment, the parameter resetting processing unit sets the S frame images so as not to be larger than the N frame images.

  In one embodiment, the parameter resetting processing unit sets the S frame images and the N frame images so as not to be larger than the N frame images.

  In one embodiment, the respiratory cycle specifying processing unit specifies a period corresponding to the S frame images as a respiratory cycle when the correlation coefficient is greater than or equal to a predetermined threshold. To do.

  In one embodiment, the parameter resetting processing unit increases the specific frame number S by K (K: an integer of 1 or more).

  In one embodiment, the parameter resetting processing unit holds a ratio of frame images included in common in the first image group and the second image group at or above a predetermined minimum data duplication rate. Increase the value of N equally.

  In one embodiment, the respiratory cycle specifying processing unit makes the correlation coefficient negative, positive and negative as the number S of frame images, which is a difference between the first image group and the second image group, increases. If the positive value is larger than or greater than the predetermined threshold when sequentially changing, the time corresponding to the S frame images is specified as the respiratory cycle of the subject.

  In one embodiment, the respiratory cycle specifying processing unit makes the correlation coefficient negative, positive and negative as the number S of frame images, which is a difference between the first image group and the second image group, increases. When sequentially changing, if the first negative value is less than or equal to the first threshold value and the positive value is greater than or equal to the predetermined threshold value, it corresponds to the S frame images. Time is specified as the subject's breathing cycle.

  In one embodiment, the image feature amount is a value based on a pixel value.

  In one embodiment, the image feature amount is a value based on a luminance value.

  In one embodiment, the respiration rate monitoring device applies a least square method to the image feature amount of the first image group and the image feature amount of the second image group, thereby causing a disturbance superimposed on the respiration waveform. An approximate data generation processing unit that obtains a primary approximate line of the trend, and a trend removal processing unit that subtracts the primary approximate line from the respiratory waveform to remove the trend from the respiratory waveform.

  A respiratory rate monitoring method according to another exemplary embodiment of the present invention includes N (N: an integer greater than or equal to 2) consecutive frames starting from a specific frame image of a moving image in which a subject that breathes is captured. A first acquisition step of acquiring data of a first image group composed of frame images, and N consecutive frames starting from a past frame image by S (S: an integer of 1 or more) from the specific frame image A second acquisition step of acquiring data of the second image group composed of the frame images, and calculating the image feature amount of each frame image of the first image group, and the first image which is a set of the image feature amounts A first creation step of creating a feature quantity group; a second creation step of calculating an image feature quantity of each frame image of the second image group and creating a second image feature quantity group that is a set of the image feature quantities And the first image feature A step of calculating a correlation coefficient between a group and the second image feature quantity group, and a relationship between the number of frame images S, which is a difference between the first image group and the second image group, and the correlation coefficient, Including the step of attempting to specify the respiratory cycle and the step of resetting the number of frames S if the respiratory cycle cannot be specified.

  A computer program according to still another exemplary embodiment of the present invention is a computer program executed by a computer, and the computer program specifies a moving image in which a subject to be breathed is captured with respect to the computer. A first acquisition step of acquiring data of a first image group composed of N (N: an integer greater than or equal to 2) frame images counted from a frame image, and S (S: 1 from the specific frame image) A second acquisition step of acquiring data of the second image group composed of consecutive N frame images, counting from the past frame images of the integer), and each frame image of the first image group A first creation step of creating a first image feature quantity group that is a set of the image feature quantities, and each frame of the second image group. A second creation step of calculating a second image feature amount group that is a set of the image feature amounts, calculating the image feature amount of the image image, and the first image feature amount group and the second image feature amount group; A correlation coefficient of the first image group and the second image group, and a step of attempting to specify a respiratory cycle according to a relationship between the correlation coefficient and the number S of frame images, which is a difference between the first image group and the second image group; If the respiratory cycle cannot be specified, the step of resetting the number S of frames is executed.

  In one embodiment, the computer program is stored on a recording medium.

  The respiration rate monitoring apparatus according to the present invention reads two sets of image groups in which the first frame images are shifted by S frame images from consecutive frame images constituting a moving image, and image characteristics of each image group Calculate the correlation coefficient of quantities. When the correlation coefficient is equal to or greater than a predetermined threshold, the time corresponding to the S frame images is specified as the breathing cycle of the subject. Since the correlation coefficient is used to represent a waveform range in which the number of frame images S is shifted by one breath, one breathing cycle can be specified.

  For example, when the correlation coefficient is less than the threshold, the frame shift S between the first frame images is increased by K (K: an integer of 1 or more), two sets of images are read, and a new correlation coefficient is calculated. . Further, the number of frames used for calculating the correlation coefficient is made variable. Thereby, the followability to a respiratory change can be improved. Since it is not necessary to always set the time window in accordance with the assumed minimum respiration rate (respiration with the lowest frequency), the respiration rate calculation can be performed with a shorter sample number than the conventional one on average. Since the temporal change in respiration is relatively finely understood, the followability to the change can be improved.

It is a figure which shows the structure of the monitoring system 100 containing the respiration rate monitoring apparatus 30 by Embodiment 1. FIG. 2 is a diagram illustrating a hardware configuration of a respiration rate monitoring apparatus 30. FIG. 2 is a diagram mainly illustrating a configuration example of a respiratory rate monitoring apparatus 30. FIG. 2 is a diagram mainly showing a detailed configuration example of a respiratory rate monitoring apparatus 30. FIG. 4 is a flowchart showing a processing procedure of the respiration rate monitoring apparatus 30. It is a figure which shows the respiration area | region 202 designated to the certain frame image 200. FIG. (A)-(d) is a figure which shows an example of the time-sequential change of the image in the respiration area | region 202 over several frame images. It is a figure which shows an example of the acquired respiratory waveform. A first image group that includes N frame images; a second image group that includes N frame images that are consecutive, starting from a frame image at a position shifted by S sheets from the first frame image of the first image group; It is a figure which shows the relationship with a respiration waveform. It is a figure which shows the example which changed the number of frame images of each image group, and the overlapping frame image so that it may not fall below the minimum data duplication rate P (%) between the 1st image group and the 2nd image group. It is a figure which shows typically the threshold value conditions which calculate respiration rate. It is a figure which shows the example of a respiration rate calculation process. It is a figure which shows typically the threshold value conditions which calculate the respiration rate in Embodiment 2. FIG. It is a figure which shows the example of a respiration waveform in which the correlation coefficient coef (S) is not below -coefTh. It is a figure which shows the example of a respiration waveform in which the conditions corresponding to S = S14 in FIG. 12 are not satisfy | filled. It is a figure which shows typically the threshold value conditions which calculate the respiration rate in Embodiment 3. FIG. (A) And (b) is a figure which shows the respiration waveform obtained from the 2nd image group and the 1st image group, respectively. (A) And (b) is a figure which shows each respiration waveform and the average value (solid line of a horizontal direction). It is a figure which shows the functional structure of the respiration rate measuring apparatus 502 by Embodiment 4. FIG. It is a flowchart which shows the procedure of the trend removal process by the respiration rate measuring apparatus. It is a figure which shows the structure of the monitoring system 101 which has the respiration rate monitoring apparatus 31 which acquires a respiration area | region automatically. It is a figure which mainly shows the functional structure of the respiration rate monitoring apparatus 31 which acquires a respiration area | region automatically. It is a flowchart which shows the process sequence of the respiration rate monitoring apparatus 30 regarding the process which acquires a respiration area | region automatically. (A) And (b) is a figure which shows the example of the partial area | region Q in the two frame images image | photographed at different time, (c) is a figure which shows the change of the luminance value of the small area Q. It is a figure which shows the structure of the monitoring system 100 by which a moving image is preserve | saved at a hard disk drive (HDD) and the respiration rate monitoring apparatus 30 acquires image data from the HDD.

  Hereinafter, embodiments of a respiratory rate measuring device and a respiratory rate monitoring device having the respiratory rate measuring device according to the present invention will be described with reference to the accompanying drawings.

(Embodiment 1)
FIG. 1 shows a configuration of a monitoring system 100 including a respiration rate monitoring apparatus 30 according to the present embodiment.

  The monitoring system 100 includes a camera 10, a mouse 20, a respiratory rate monitoring device 30, and a display device 40. Although the subject 1 is shown in FIG. 1, the subject 1 is not included in the monitoring system 100.

  The monitoring system 100 is used to measure the respiration rate of the subject 1 from a moving image obtained by photographing the subject 1. In each of the following embodiments, the respiration rate means the number of respirations actually performed by the subject 1 within a certain period. In the present embodiment, the time required for one breath is obtained, and the number of breaths per unit time is obtained from the relationship between the time and the frame rate of the moving image.

  The camera 10 shoots the subject 1 and generates and outputs digital data of a moving image. For convenience of explanation, it is assumed that data is generated and output in units of one frame image. Note that the data format is arbitrary as long as it can be separated into frame images. For example, H.M. The digital data of moving images may be generated and output using moving image compression processing such as H.264 standard.

  The mouse 20 is an input device used for an operator of the monitoring system 100 to input an instruction to the respiratory rate monitoring device 30. More specifically, the mouse 20 is used for designating which region of the moving image is used when the respiratory rate monitoring device 30 measures the respiratory rate. The mouse 20 is an example of an input device, and may be another known input device such as a trackball, a keyboard, a touch pen, a touch screen panel, or a digitizer.

  The input device typified by the mouse 20 is typically hardware, but may include a mode for receiving an instruction in software. For example, it is assumed that the respiratory rate monitoring device 30 is a computer. Then, it is assumed that an image processing program operating on the computer receives a moving image, identifies a region where the luminance change of the moving image is large, and outputs the region as a region for measuring the respiration rate. Instead of the mouse 20 or the like, the result of the processing of such an image processing program can be used as an input (instruction) to the subsequent processing on the respiratory rate monitoring device 30. Such an interface for data transfer is called an application program interface (API). That is, the API can function as an input interface that changes to an input device as hardware such as a mouse. An example of such a configuration is shown in FIG. When the above-described region identification processing is performed outside the respiration rate monitoring device 30, a communication terminal used for communication with an external device functions as an input device, or the region identification result obtained from the communication terminal is displayed. The API for transferring to the main processing program of the respiratory rate monitoring device 30 functions as a software input device.

  The respiration rate monitoring device 30 is typically a computer. The respiration rate monitoring device 30 receives a signal of a moving image taken by the camera 10 and information on a specified area using the mouse 20, and uses the moving image data in the specified area. The respiration rate of subject 1 is measured. Details of the processing will be described later.

  The display device 40 is, for example, a liquid crystal display, and outputs a processing result of the respiration rate monitoring device 30, for example, information on the measured respiration rate of the subject 1 or information indicating that the respiration rate could not be acquired. The liquid crystal display is an example, and may be another known display device such as a plasma display, an organic EL display, or a projector. Note that the display device 40 may include a portable device such as a tablet or a smartphone provided with a liquid crystal display or the like.

  The display device 40 is typically hardware, but may essentially be an information presentation unit that can present information to an operator, a subject, or the like. For example, a virtual screen that can be used in Windows (registered trademark) or the like is also an example of an information presentation unit. In addition, as information indicating the respiratory rate or information indicating that the respiratory rate could not be acquired, a voice that outputs the voice itself or a voice is also an example of the information presentation unit.

  FIG. 2 shows a hardware configuration of the respiration rate monitoring apparatus 30. In the present embodiment, the respiratory rate monitoring device 30 is connected to the camera 10, the mouse 20, and the display device 40 as described above. In this specification, the interface device is abbreviated as “I / F”. As shown in FIG. 2, the I / F is typically hardware such as a connection terminal (connector), but a mechanism for transmitting and receiving data in software (for example, a socket, a port, a data structure, and a protocol) is also used. It is a category of I / F.

  The respiratory rate monitoring device 30 receives data of the captured moving image from the camera 10. Moreover, the display 40 displays the measurement result of the respiration rate of the subject 1 as a result of the processing.

  The respiratory rate monitoring device 30 includes a CPU 301, a ROM 302, a RAM 303, a hard disk drive (HDD) 304, an image processing circuit 305, a video input I / F 306, a USB I / F 307, and an output I / F 308. .

  The CPU 301 controls the operation of the respiratory rate monitoring device 30. The ROM 302 stores a computer program. The computer program is a group of instructions for causing the CPU 301 or the image processing circuit 305 to perform processing according to, for example, a flowchart described later or processing according to a modification thereof. The computer program may be recorded on an optical recording medium such as a CD-ROM, may be stored on a semiconductor recording medium such as a memory card, or may be recorded on a magnetic recording medium. The computer program can be transmitted over a telecommunication line.

  A RAM 303 is a work memory for developing a computer program when executed by the CPU 301. The HDD 304 is a storage device that stores moving image data received from the camera 10 or measured respiratory rate data of the subject 1.

  The video input I / F 306 is an interface for the respiratory rate monitoring device 30 to receive moving image data from the camera 10. As the video input I / F 306, for example, a terminal of the HDMI (registered trademark) standard, the DVI-D standard, or the DisplayPort (registered trademark) standard can be used.

  The USB I / F 307 is an example of a terminal that connects the mouse 20 and the respiratory rate monitoring device 30. As another example, PS / 2 I / F may be used.

  The output I / F 308 is an interface for outputting data for the respiratory rate monitoring device 30 to display on the display device 40. As the output I / F 308, for example, a terminal of HDMI (registered trademark) standard or DVI standard can be used. The output I / F 308 may output only text data displayed on the display device 40.

  Note that the video input I / F 306 and / or the output I / F 308 do not have to be dedicated video I / Fs. For example, if the camera 10 or another device (not shown) that has received video data from the camera 10 can perform communication using the TCP / IP protocol, the video input I / F 306 is an Ethernet (registered trademark) terminal. It may be. Alternatively, when transmitting and / or receiving moving image data by wireless communication, the video input I / F 306 and / or the output I / F 308 are communication that performs communication based on, for example, the Wi-Fi (registered trademark) standard. It can be realized as a circuit.

  The image processing circuit 305 is a so-called graphics processor that analyzes moving image data. The image processing circuit 305 calculates the respiratory rate of the subject 1 using moving image data read from the HDD 304. In the present embodiment, description will be made mainly assuming that the image processing circuit 305 performs calculation for calculating the respiration rate. However, this is an example. The processing of the image processing circuit 305 may be performed by the CPU 301. In this specification, the CPU 301 and the image processing circuit 305 may be collectively referred to as “arithmetic circuit”. FIG. 2 shows an arithmetic circuit 309.

  The outline of the operation of the arithmetic circuit 309 of the respiration rate monitoring apparatus 30 is as follows. That is, the arithmetic circuit 309 receives from the storage device such as the HDD 304 a first image group composed of N (N: an integer greater than or equal to 2) frame images of moving images, and a first frame image ( A second image group composed of consecutive N frame images, which is shifted by S (S: integer greater than or equal to 1) frame images from the most recent frame image) is read out. The arithmetic circuit 309 calculates a correlation coefficient between the image feature amount of the first image group and the image feature amount of the second image group. In the present embodiment, the number of frame images in the first image group and the number of frame images in the second image group are both N.

  As an example, the “image feature amount” is calculated based on a pixel value in a region where a change due to the breathing of the subject 1 exists in the image (hereinafter referred to as “breathing region”). For example, the average value or median value of the luminance value of the image frame and the pixel value of the green component. In the present embodiment, the breathing area is specified by the operator of the respiration rate monitoring device 30, but in other embodiments to be described later, the breathing area is recognized by a device that has performed predetermined image processing. . When the correlation coefficient is equal to or greater than a predetermined threshold, the arithmetic circuit 309 specifies the time corresponding to the S frame images as the breathing cycle of the subject.

  Unlike the conventional technique, the respiration rate monitoring apparatus 30 according to the present embodiment sets the frame image number N of the first and second image groups according to the assumed minimum respiration rate (respiration with the lowest frequency). Good. A specific setting method will be described later.

  If the value of N is set to a relatively small value, the process of calculating the correlation coefficient between the image feature quantity of the first image group and the image feature quantity of the second image group and the detection process of the subject's breathing cycle are relative. Get faster. Therefore, even when the respiratory rate fluctuates, the fluctuation can be quickly followed. However, when the respiratory cycle is longer than the above-described N images, it is considered difficult to detect such one respiratory cycle.

  On the other hand, if the value of N is set to be relatively large, a process for calculating the correlation coefficient between the image feature quantity of the first image group and the image feature quantity of the second image group, and a process for detecting the breathing cycle of the subject It takes time relatively. This means that when the respiratory rate fluctuates, it takes time to follow the fluctuation. However, by increasing the value of N, it is possible to follow even if the respiratory cycle becomes relatively long.

  Here, an advantage obtained by determining the number N of frame images of the first and second image groups will be described. When it is determined that there is a correlation between the image groups using the frame images of the first and second image groups, the respiration rate monitoring apparatus 30 can abort the process at that time. This is advantageous over known techniques that measure respiratory rate, for example, by fast Fourier transform using a computer. When using the Fast Fourier Transform, set the frequency range corresponding to all possible respiratory cycles, determine the power spectrum for all the frequency ranges, and based on the calculated power spectrum, the frequency corresponding to respiration Need to be identified. For example, even if the subject's breathing cycle is fast and the breathing frequency is high, it is necessary to perform calculations for lower frequencies.

  On the other hand, according to the present embodiment, if the correlation coefficient meets a threshold condition described later, the process can be terminated at that time. It is not necessary to perform all predetermined processes every time, and the process can be suppressed to the minimum necessary process. Therefore, the respiratory rate can be calculated more efficiently than the conventional one on average.

  FIG. 3A mainly shows a configuration example of the respiration rate monitoring apparatus 30.

  The respiratory rate monitoring device 30 includes a respiratory rate measuring device 102, a frame image data reception I / F 103, a respiratory region designation data reception I / F 104, a frame image data buffer 106, a respiratory region designation data buffer 107, and a respiratory waveform. A data generation processing unit 108. The respiratory rate measuring apparatus 102 includes a respiratory waveform data buffer 109, a respiratory rate calculation processing unit 110, and a respiratory rate data buffer 111. The respiration rate calculation processing unit 110 has a buffer 112 that temporarily stores data used for calculation processing.

  The correspondence between these components and each component shown in FIG. 2 is as follows. The respiration waveform data generation processing unit 108 of the respiration rate monitoring device 30 and the respiration rate calculation processing unit 110 of the respiration rate measurement device 102 correspond to the arithmetic circuit 309 (the CPU 301 and / or the image processing circuit 305). The frame image data reception I / F 103 corresponds to the video input I / F 306. The breathing area designation data reception I / F 104 corresponds to the USB I / F 307. Alternatively, a software interface (API) may be used. The frame image data buffer 106, the respiratory region designation data buffer 107, the respiratory waveform data buffer 109, and the respiratory rate data buffer 111 correspond to the RAM 303 or the HDD 304. The CPU 301 and / or the image processing circuit 305 often incorporate a storage device called a cache memory. Such a cache memory may be used as all or part of each of the buffers described above.

  FIG. 3B shows a detailed configuration example of the respiration rate monitoring apparatus 30 of FIG. 3A. More specifically, FIG. 3B shows a detailed configuration example of the frame image data buffer 106, the respiration waveform data generation processing unit 108, and the respiration rate calculation processing unit 110 in FIG. 3A.

  The frame image data buffer 106 includes a first image group acquisition processing unit 106a and a second image group acquisition processing unit 106b. The first and second image group acquisition processing units 106a and 106b respectively acquire the data of the first image group and the second image group described above. “Acquire” as used herein typically refers to receiving the data of the first image group and the second image group described above via the frame image data reception I / F 103.

  Note that the first image group acquisition processing unit 106 a and the second image group acquisition processing unit 106 b are not necessarily provided as part of the frame image data buffer 106. For example, when a moving image including the first and second image groups is already stored in the HDD 304 (FIG. 2), the first and second image group acquisition processing units 106a and 106b receive the first and second images from the HDD 304. A processing circuit for reading data of two image groups may be used.

  The respiratory waveform data generation processing unit 108 includes a first image feature quantity group creation processing unit 108a and a second image feature quantity group creation processing unit 108b. The first image feature value group creation processing unit 108a calculates the image feature value of each frame image of the first image group, and creates a first image feature value group that is a set of the image feature values. Similarly, the second image feature quantity group creation processing unit 108a calculates the image feature quantity of each frame image of the second image group, and creates a second image feature quantity group that is a set of the image feature quantities. Examples of image feature amounts are as described above.

  The respiration rate calculation processing unit 110 includes a correlation coefficient calculation processing unit 110a, a respiration cycle specifying processing unit 110b, and a parameter resetting processing unit 110c.

  The correlation coefficient calculation processing unit 110a calculates a correlation coefficient between the first image feature quantity group and the second image feature quantity group. The respiratory cycle identification processing unit 110b determines whether the respiratory cycle can be identified based on the transition of the correlation coefficient when the number S of frame images, which is the difference between the first image group and the second image group, is changed. . The parameter reset processing unit 110c resets the value of the number of frames S when the respiratory cycle cannot be specified. Further, the parameter resetting processing unit 110c can change the number N of frame images in the first and second image groups. “Resetting” means, for example, acquiring data of the first and second image groups based on the value of the number N of frame images constituting each image group and / or the value of the number of frames S after the change. Instructing the first image group acquisition processing unit 106a and the second image group acquisition processing unit 106b.

  Below, based on the structure of FIG. 3A, operation | movement of the respiration rate monitoring apparatus 30 is demonstrated. Where appropriate, the detailed configuration of FIG. 3B is also referred to. 2, the “respiration waveform data generation processing unit 108” or “respiration rate calculation processing unit 110” in the following description is referred to as “the calculation circuit 309” as it is. It may be read as “CPU 301” or “image processing circuit 305”.

  FIG. 4 is a flowchart showing a processing procedure of the respiration rate monitoring apparatus 30. In the following description, each process will be described with reference to other drawings as appropriate.

  First, in step S <b> 1, the frame image data reception I / F 103 receives moving image data transmitted from the camera 10. The frame image data reception I / F 103 stores the moving image data in the frame image data buffer 106. With this processing, the frame image data buffer 106 stores at least as many frame image data as can be processed as described later.

  Even when the compressed and encoded moving image data is received from the camera 10, it is only necessary that the frame image data buffer 106 holds at least the number of frame images that can be processed later. The compression-coded moving image data includes, for example, data (I picture data) in which a complete picture can be constructed using only the compressed data, and a complete picture using only the compressed data. However, there is data (P, B picture data) in which a complete picture can be constructed by referring to data of other pictures. The image processing circuit 305 can store the necessary number of frame images in the frame image data buffer 106 by decoding the I, P, and B picture data.

  In relation to the example of FIG. 3B, in step S1, the first image group acquisition processing unit 106a and the second image group acquisition processing unit 106b acquire moving image data of the first image group and the second image group, respectively.

  In step S <b> 2, the breathing area designation data reception I / F 104 receives information (area designation information) that specifies a part of the area in the frame image designated by the operator. The area designation information is information for specifying a breathing area. The respiratory region designation data reception I / F 104 stores the information in the respiratory region designation data buffer 107.

  FIG. 5 shows a breathing area 202 designated in a certain frame image 200. A plurality of straight lines extending in the vertical direction and the horizontal direction and a rectangular small area group surrounded by them are virtually shown on the frame image 200 on the display device 40 in order to allow the operator to specify a specific small area. It ’s just that. In the present specification, a small area including a change in body movement accompanying the breathing of the subject 1 is designated as a breathing area. The user designates the breathing area 202 using the mouse 20 from among them. FIG. 5 shows an example in which two small areas adjacent in the vertical direction are designated as one breathing area. The breathing area is an absolute position in the frame image. The image feature quantity of the breathing region at the same position is obtained over a plurality of frame images.

  FIGS. 6A to 6D show an example of a time-series change of an image in the breathing region 202 over a plurality of frame images. Referring to FIG. 6A, there are two objects in the breathing area 202. A subject object 204 including the subject 1 and a background object 206. The object 204 is relatively bright due to the influence of the color of the subject's clothes and the like. That is, the luminance value is large. One background object 206 is relatively dark. That is, the luminance value is small.

  As shown in FIGS. 6A to 6D, the area of the subject object 204 occupying the fixed breathing region 202 changes in synchronization with the breathing of the subject 1. Such a small region can be designated as the breathing region 202 by the operator.

  6A to 6D are not necessarily continuous frame images. For example, when the subject 1 requires 2 seconds for one breath, FIGS. 6A to 6D are four frame images at intervals of 0.5 seconds, for example.

  Next, in step S <b> 3 of FIG. 4, the respiratory waveform data generation processing unit 108 reads frame image data from the frame image data buffer 106, reads respiratory region designation data from the breathing region designation data buffer 107, and Respiratory waveform data is generated. The respiratory waveform data generation processing unit 108 stores the generated respiratory waveform data in the respiratory waveform data buffer 109.

  The process of generating respiration waveform data is a process for obtaining time-series data of average values of luminance values in the respiration region 202. As shown in FIGS. 6A to 6D, since the area of the subject object 204 and the area of the background object 206 change in synchronization with respiration, the average luminance value of the entire respiration region 202 is also temporally changed. fluctuate. The respiratory waveform data generation processing unit 108 can obtain information that varies with the breathing of the subject 1, that is, a respiratory waveform, by obtaining the average value of the luminance values for each frame image.

  Note that, according to FIGS. 1 and 5, the camera 10 is installed between an obliquely upper position of the subject 1 and a position near horizontal. However, this arrangement is an example. The angle at which the camera 10 captures the subject 1 is arbitrary. For example, it may be installed directly above the subject 1. When the camera 10 photographs the subject 1 from directly above the subject 1, the background object 206 shown in FIG. 6A or the like may not be included. However, even in such a case, the respiratory waveform of the subject 1 can be obtained. This is due to the fact that the reflection direction of light changes every moment as the subject 1 moves with breathing. By continuously measuring the luminance value, a respiratory waveform can be obtained.

  FIG. 7 shows an example of the acquired respiratory waveform. In FIG. 7, the change of the luminance value is shown as a continuous function, but this is for convenience of understanding. Note that it is actually obtained as a discrete value. The image processing circuit 305 can also obtain a finer waveform by interpolating the average value of the luminance values of the respective frame images obtained as discrete values with, for example, a spline curve.

  In relation to the example of FIG. 3B, in step S3, the first and second image feature quantity group creation processing units 108a and 108b create an image feature quantity group (respiration waveform) that is a set of image feature quantities.

  In step S4, the respiratory waveform data buffer 109 stores the obtained respiratory waveform data.

In step S5, the respiration rate calculation processing unit 110 reads the respiration waveform data from the respiration waveform data buffer 109, and obtains the respiration rate of the subject 1 from the respiration waveform data. The respiratory rate of the subject 1 is stored in the respiratory rate data buffer 111 in step S6.
The respiration rate calculation process is performed using the following data (1) to (3).

(1) Respiration waveform data generated from consecutive past N (N: integer greater than or equal to 2) frame images (first image group) from a specific frame image.
(2) Generated from the past N frame images (second image group) including the frame image S (S: integer greater than or equal to 1) previous frames as the first frame image from a specific frame image Respiration waveform data.
(3) A correlation coefficient indicating the strength of correlation between the respiratory waveform data obtained from the first image group and the respiratory waveform data obtained from the second image group.

  An example of the “specific frame image” in the above (1) and (2) is the latest frame image.

  With respect to this correlation coefficient, a value that can be determined to have a significantly high correlation between the first image group and the second image group is set in advance. That is, a value that can be determined to have a high correlation such that it can be determined that the second image group was captured one breath before the first image group is set in advance. The respiration rate calculation processing unit 110 determines whether or not the correlation coefficient is greater than or equal to such a set value. When the correlation coefficient is greater than or equal to the set value, the first image group and the second image group at that time are determined. A value S corresponding to the interval is specified as a breathing interval. For example, assuming that the frame rate is f per second, the respiratory rate calculation processing unit 110 obtains the time T required for one breath as T = (S / f) seconds. And the respiration rate calculation processing part 110 calculates | requires that the respiration rate per minute is 60 / T times. The respiration rate calculation processing unit 110 stores the respiration rate information in the respiration rate data buffer 111 as respiration rate data.

  Details of the respiration rate calculation process will be described below.

  FIG. 8 shows a first image group including N frame images, and a second image including N consecutive frame images starting from a frame image at a position shifted by S sheets from the first frame image of the first image group. The relationship between an image group and a respiration waveform is shown. (1) and (2) in FIG. 8 correspond to the above-described first image group and second image group, respectively.

  The inventors of the present application decided to operate the respiratory rate calculation processing unit 110 based on the following knowledge.

  First, the value of N related to the first image group needs to be determined by the length of the respiratory waveform cycle to be detected. It is necessary to increase the value of N if the period is long, and decrease the value of N if the period is short. If the value of N is determined based on a long period of the assumed respiration waveform period, if the respiration becomes faster, a plurality of respirations are performed during the imaging period corresponding to the N frame images. Will be done. This is the same when breathing becomes faster when breathing becomes faster or slower. When the value of N is determined based on a long period, the correlation coefficient obtained from the N first image groups and the N second image groups is difficult to meet a threshold condition described later, and it is determined that there is a correlation. It takes time to be done. Thus, it has been found that the problem of followability can occur.

  On the other hand, the present inventors have also found that, when obtaining the correlation of a waveform whose period changes like respiration, if the value of N is constant, an appropriate correlation may not be obtained.

  Therefore, the present inventors set the value (N0) initially set as the N frame based on the case where breathing is fast (the cycle is short) so that the correlation can be calculated. Further, the inventors of the present application increase (a) the value of the interval S between the first image group and the second image group as a response to the case where breathing is slow (the breathing cycle is long), (b) In addition to the above (a), the number N of the first image group and the number N of the second image group are also changed. Thereby, even if breathing slows, it can be determined whether or not there is a correlation between the first image group and the second image group.

  In either case (a) or (b), a rule may be provided in advance as to how the change is made. For example, the overlap rate. “Overlap ratio” means, for example, the number of frame images included in common in the first image group and the second image group, the total number N of frame images included in the first image group (or the second image group), It is a relationship. The “duplication rate” may be defined as the relationship between the number of frame images included in common in the first image group and the second image group and the number of frames from the latest frame image to the oldest frame image. . The latter “latest frame image” is the first frame image of the first frame image group ((1) in FIG. 8), and the “oldest frame image” is the second frame image group (( This is the last frame image of 2)).

  Hereinafter, a specific calculation method of the correlation coefficient will be described. However, the following description is an example, and even if a different method is used, that method may be used as long as the correlation can be obtained.

As described above, the correlation coefficient can be obtained based on the following equation 1 using the number of first image groups and the number of second image groups (both are N).

  The function f (i) included in Equation 1 is the value of the respiratory waveform corresponding to the i-th frame image.

  The respiration rate calculation processing unit 110 sets S = S0 as an initial value, and obtains a correlation coefficient coef (S) by Equation 1. For example, when 30 breaths / minute is a short breath target, the initial value S0 may be set to a value equal to or greater than the number of frames for 2 seconds. That is, the number of frames may be set longer than the time required for one breath.

  In relation to the example of FIG. 3B, the correlation coefficient calculation processing unit 110a calculates the above-described correlation coefficient.

  The respiration rate calculation processing unit 110 compares the correlation coefficient coef (S) with a preset correlation coefficient threshold value coefTh, and under a predetermined threshold condition, the correlation coefficient coef (S) Is determined to be greater than or equal to the correlation coefficient threshold value coefTh. This means that the respiration rate calculation processing unit 110 determines whether the respiration cycle can be specified. This process corresponds to the operation of the respiratory cycle identification processing unit 110b in the example of FIG. 3B.

  When the correlation coefficient coef (S) is equal to or greater than the correlation coefficient threshold value coefTh, the respiration rate calculation processing unit 110 calculates a time corresponding to the S frame image (the above S / f) as a time required for one breath. Note that the correlation coefficient threshold value coefTh may be any constant between 0 and 1. In this specification, the correlation coefficient threshold value coefTh = 0.7.

  When the time required for one breath is calculated, the process of incrementing the value S at that time ends. The respiration rate calculation processing unit 110 calculates the respiration rate from the time required for one respiration.

  The respiratory rate calculation processing unit 110 operates so that the number of data used for calculating the correlation coefficient is large when the respiratory cycle is long (frequency is low) and small when the respiratory cycle is short (frequency is high). Change. The reason for dynamically changing the number of data used for the calculation of the correlation coefficient is to improve followability to respiratory rate fluctuations. Here, the number of data is the number of frame images constituting the first image group and the number of frame images constituting the second image group, both of which are N.

In order to evaluate the respiration rate, respiration waveform data corresponding to one respiration (that is, data corresponding to one respiration) is required. If the number of frame images for obtaining the respiratory waveform data is too large compared to the size of the respiratory waveform, and the respiratory rate is evaluated using all the frame images, the respiratory rate is There is a possibility that the fluctuation cannot be followed quickly. Obviously, the more respiratory waveforms that are included, the longer it takes to determine that there is a correlation between the overall waveforms. Therefore, it is appropriate to dynamically change the number of data used for calculating the correlation coefficient as described above. Further, as shown in FIG. 9, a maximum value of N (N _max ) may be set.

  For example, FIG. 9 shows an example in which the number of frame images and overlapping frame images of each image group are changed between the first image group and the second image group so as not to fall below the minimum data duplication rate P (%). Indicates. The number of frame images constituting the first image group and the number of frame images constituting the second image group are each N.

  As shown in FIG. 9, the respiration rate calculation processing unit 110 first calculates N0 using N as the initial value N0 and the minimum data duplication rate. Then, the respiration rate calculation processing unit 110 sets N larger as S increases so that the data duplication rate does not fall below the minimum data duplication rate. The minimum data duplication rate here is predetermined in the ratio (%) of the number of frame images common to two image groups for calculating the correlation coefficient to the number of respiratory waveform data used for calculating the correlation coefficient. The minimum value given. The minimum data duplication rate is 30%, for example.

  The overlap rate will be further described. Since evaluation requires data of a period or more, when the evaluation is to be performed with the smallest number of samples, the duplication rate may be set to 0%. This means that the sample before shifting (first image group) and the sample after shifting (second image group) are continuous without any gap. An image group setting method in which the first image group and the second image group do not overlap and one or more frame images exist between them is not appropriate. This is because only data having a shorter time than the period to be evaluated exists.

  The above-described processing for changing the value of the interval S and processing for setting N large correspond to the operation of the parameter resetting processing unit 110c in FIG. 3B.

  Next, a comparison process between the correlation coefficient coef (S) and a preset correlation coefficient threshold value coefTh will be described in more detail. The “respiration rate calculation processing unit 110” in the following description corresponds to the processing of the correlation coefficient calculation processing unit 110a (FIG. 3B) with respect to the process for obtaining the correlation coefficient, and the respiratory cycle based on the comparison between the correlation coefficient and the threshold value. The determination processing corresponds to the breathing cycle specifying processing unit 110b (FIG. 3B), and the adjustment processing of the parameters S and / or N corresponds to the parameter resetting processing unit 110c (FIG. 3B).

  FIG. 10 schematically shows threshold conditions for calculating the respiration rate. The horizontal axis of FIG. 10 indicates the number of shifted frames S, and the vertical axis indicates the correlation coefficient value. FIG. 10 shows a preset correlation coefficient threshold value coefTh (> 0).

  In the present embodiment, the respiration rate calculation processing unit 110 determines that the correlation coefficient coef (S) is equal to or greater than a preset correlation coefficient threshold coefTh when the following threshold condition is satisfied. That is, as shown in FIG. 10, the respiration rate calculation processing unit 110 sequentially obtains correlation coefficients while sequentially increasing the number of frames S of the correlation coefficient coef (S). When S is small, the correlation coefficient is positive. After that, after increasing S, the correlation coefficient coef (S) becomes negative (S = S1), then transitions to positive (S = S2), and then becomes negative (S = S3). Increase the number of frames S. The respiratory rate calculation processing unit 110 determines that the threshold condition is satisfied when the correlation coefficient coef (S) is equal to or greater than the threshold value coefTh while the frame number S is between S2 and S3.

 Note that “negative” is a number less than 0, and “positive” is a number greater than 0. However, it is not necessary to exclude 0. For example, “negative” may be interpreted as a number of 0 or less, and “positive” may be interpreted as a number greater than 0. Alternatively, “negative” may be interpreted as a number less than 0, and “positive” may be interpreted as a number greater than 0. In the present specification, when “negative (less than 0 or less than 0)” and “positive (greater than 0 or greater than 0)” are listed, “negative” is a number less than 0 and “positive” Means a number greater than or equal to 0, or “negative” means a number less than 0, and “positive” means a number greater than 0.

  Attention is now paid to the portions 400 and 402 where the correlation coefficient coef (S) is equal to or greater than a preset correlation coefficient threshold value coefTh. Due to the threshold condition described above, the respiration rate calculation processing unit 110 does not use the frame number S obtained in the portion 400. More specifically, although the coefficient portion 400 is equal to or greater than the correlation coefficient threshold value coefTh, the above-described correlation coefficient coef (S) becomes negative (S = S1), and then transitions to positive (S = S2), then not going negative (S = S3). Therefore, the number of frames S obtained in the portion 400 is not used.

  On the other hand, the respiration rate calculation processing unit 110 uses the number of frames S at which the correlation coefficient value coef (S) obtained in the portion 402 has a maximum value as the number of frames corresponding to the time required for one breath.

  Note that the number of frames corresponding to the time required for one breath may be any frame number S as long as the correlation coefficient value of the portion 402 is continuously within the threshold or more. For example, the number of frames corresponding to the center of the section or the number of frames at the beginning or end of the section may be used. Alternatively, the number of frames S in which the correlation coefficient value coef (S) in the section becomes the maximum may be used.

  It should be noted that the number of frames S that takes the maximum value of the correlation coefficient value coef (S) that appears in this specification is a section where the target position (corresponding to the portion 402 in this embodiment) is continuously equal to or greater than the threshold value. Any number of frames can be used. For example, the number of frames corresponding to the center of the section or the number of frames at the beginning or end of the section may be used. Alternatively, the number of frames S in which the correlation coefficient value coef (S) in the section becomes the maximum may be used.

  In the above-described example, the case where the correlation coefficient value is equal to or greater than the correlation coefficient threshold value coefTh after the correlation coefficient value is reversed three times has been described as the threshold condition. However, other threshold conditions may be employed. For example, before and after each frame number S1 and S2 shown in FIG. 10, the sign of the correlation coefficient value is inverted twice, and then, when the correlation coefficient value becomes equal to or greater than the correlation coefficient threshold value coefTh, the shifted frame number S (> S2) may be specified.

  Note that when the above threshold condition is not satisfied, the respiration rate calculation processing unit 110 cannot calculate the respiration rate. This state means that the respiratory rate cannot be acquired. The respiration rate calculation processing unit 110 may determine this period as an apnea state.

  FIG. 11 shows an example of the respiration rate calculation process. This process is performed by the respiratory rate calculation processing unit 110.

  In step S11, the respiratory rate calculation processing unit 110 extracts the first image group and the second image group. The initial value at this time is a predetermined value (N = N0, S = S0).

  In step S <b> 12, the respiration rate calculation processing unit 110 calculates a correlation coefficient coef (S) based on the equation (1).

  In step S13, the respiratory rate calculation processing unit 110 increases the difference (number of difference frame images) S between the first frame images of the first image group and the second image group, and the number N of frame images constituting each image group. The repetitive correlation coefficient coef (S) is calculated. At this time, as described with reference to FIG. 9, the respiration rate calculation processing unit 110 adjusts the parameters S and N so as not to fall below a predetermined minimum data duplication rate.

  In step S14, the respiration rate calculation processing unit 110 determines whether or not the obtained plurality of correlation coefficients coef (S) satisfy a threshold condition. If the threshold condition is met, the process proceeds to step S15. If not, the process proceeds to step S16.

  In step S15, the respiration rate calculation processing unit 110 calculates the time required for one breath from the number of frame images S corresponding to the difference between the first image group and the second image group, and further calculates the respiration rate.

  In step S16, the respiratory rate calculation processing unit 110 determines that the subject 1 is in an apnea state, and sets the respiratory rate to 0.

  In step S17, the respiration rate calculation processing unit 110 stores the obtained respiration rate data in the respiration rate data buffer 111, and ends the processing.

  According to the above processing, the number of data used for calculating the correlation coefficient increases when the respiratory cycle is long (frequency is low), and when the respiratory cycle is short (frequency is high), The period can be specified. Thereby, even when the respiratory rate fluctuates, followability to the fluctuation is enhanced.

(Embodiment 2)
In the present embodiment, an example in which a threshold condition different from the threshold condition in the first embodiment is used will be described. The configuration of each device included in the monitoring system 100 is the same. Therefore, description of Embodiment 1 is used and description for the second time is omitted. In particular, in the flowchart shown in FIG. 11, only the specific contents of the process shown in step S14 are changed, and the other processes are the same. That is, only the processing of the respiratory cycle identification processing unit 110b (FIG. 3B) of the respiratory rate calculation processing unit 110 is changed.

  The threshold condition described in the first embodiment is an example in which one correlation coefficient threshold value coefTh is used (FIG. 10).

  In this embodiment, two threshold conditions are used. Specifically, a second threshold value (-coefTh) is set in addition to the first threshold value coefTh.

  FIG. 12 schematically shows threshold conditions for calculating the respiration rate in the present embodiment. The horizontal axis of FIG. 12 indicates the number of difference frames S shifted, and the vertical axis indicates the correlation coefficient value. FIG. 10 shows a correlation coefficient threshold value coefTh (> 0) and a correlation coefficient threshold value −coefTh (<0) set in advance.

  The respiration rate calculation processing unit 110 determines that the correlation coefficient coef (S) is equal to or greater than a preset correlation coefficient threshold coefTh when the following threshold condition is satisfied, and from the frame image number S, 1 respiration Calculate the time required for. This will be specifically described. First, the respiration rate calculation processing unit 110 obtains the correlation coefficient coef (S) while increasing the number of frames S. Then, after the correlation coefficient coef (S) becomes negative for the first time (S = S11), the correlation coefficient coef (S) keeps a negative state and becomes -coefTh or less (S = S12). When coef (S) becomes positive (S = S13), the correlation coefficient threshold value coefTh becomes equal to or greater than the coefTh while maintaining a positive state (S = S14), and then becomes negative (S = S15). Judge that the condition is satisfied.

  On the other hand, FIG. 13 and FIG. 14 show examples of respiratory waveforms in which the above threshold condition is not satisfied. For example, in FIG. 13, the correlation coefficient coef (S) is not less than or equal to −coefTh. Further, in FIG. 14, the condition corresponding to S = S14 in FIG. 12 is not satisfied.

  When the above threshold condition is not satisfied, the respiration rate calculation processing unit 110 cannot calculate the respiration rate. This state means that the respiratory rate cannot be acquired. The respiration rate calculation processing unit 110 may determine this period as an apnea state.

  Hereinafter, two modifications of the present embodiment will be described.

(Modification 1 of Embodiment 2)
Please refer to FIG. The parameter reset processing unit 110c (FIG. 3B) of the respiration rate calculation processing unit 110 changes the difference frame number S so that the correlation coefficient calculation processing unit 110a (FIG. 3B) correlates with the change. The number coef (S) is obtained. After the correlation coefficient coef (S) becomes negative for the first time (S = S11) by the determination process of the respiratory cycle identification processing unit 110b (FIG. 3B), the correlation coefficient coef (S) maintains a negative state. However, it is assumed that the value is -coefTh or less (S = S12), and the minimum value of the correlation coefficient coef (S) is obtained in the number of frames S = S21. At this time, the respiratory cycle identification processing unit 110b (FIG. 3B) may estimate that the difference frame number S is a half cycle of respiration and a value obtained by doubling the frame number S is one cycle. .

  Note that the number of frames S from which the minimum value of the correlation coefficient coef (S) can be obtained is any number of frames as long as the correlation coefficient value is continuously greater than or equal to the threshold value at the position of interest (S11 to S13 in this embodiment). S may be used. For example, the number of frames S corresponding to the center of the section or the number of frames at the beginning or end of the section may be used. Alternatively, the number of frames S that maximizes the correlation coefficient value coef (S) in the section may be used.

  Note that the number of frames S taking the minimum value of the correlation coefficient value coef (S) referred to in this specification may be any frame number as long as it is within the interval that is continuously below the threshold value of the target position. For example, the number S of frames corresponding to the center of the section, or the number of frames at the beginning or end of the section may be used. Further, the number of frames S that minimizes the correlation coefficient value coef (S) in the section may be used.

  According to this processing, since the number of difference frames S corresponding to one cycle is estimated by actively using the periodicity of respiration, the number of data used for the calculation of the correlation coefficient is reduced, and the processing can be speeded up. It becomes possible.

(Modification 2 of Embodiment 2)
Please refer to FIG. The parameter reset processing unit 110c of the respiration rate calculation processing unit 110 changes the difference frame number S so that the correlation coefficient calculation processing unit 110a obtains the correlation coefficient coef (S) in synchronization with the change. . Then, after the correlation coefficient coef (S) becomes negative for the first time by the determination process of the respiratory cycle identification processing unit 110b (S = S11), while the correlation coefficient coef (S) remains in a negative state, -coefTh The following is obtained (S = S12), and the minimum value of the correlation coefficient coef (S) is obtained. The respiratory cycle identification processing unit 110b holds the number of frames S21 corresponding to this minimum value.

  Then, while the parameter reset processing unit 110c continues to increase the difference frame number S, the correlation coefficient calculation processing unit 110a obtains the correlation coefficient coef (S). Then, after coef (S) becomes positive (S = S13), the correlation coefficient threshold coefTh is not less than the correlation coefficient coefTh while maintaining a positive state (S = S14), and then the correlation coefficient coef (S) is maximized. A value is obtained. If the difference frame number S21 corresponding to the minimum value is within the range of 50 ± 10% of the frame number S22 corresponding to the maximum value, the respiratory cycle identification processing unit 110b determines that the threshold condition is satisfied. If not, the data is not used for calculating the respiration rate. This numerical range is only an example. The range can be changed as appropriate.

  According to this process, it is possible to obtain the number of frames S corresponding to one period with higher accuracy using the periodicity of respiration.

(Embodiment 3)
In this embodiment, an example in which a threshold condition different from the threshold conditions in the first and second embodiments is used will be described. The configuration of each device included in the monitoring system 100 is the same. Therefore, description of Embodiment 1 is used and description for the second time is omitted. In particular, in the flowchart shown in FIG. 11, only the specific contents of the process shown in step S14 are changed, and the other processes are substantially the same. Mainly, the processing of the respiratory cycle specifying processing unit 110b of the respiratory rate calculation processing unit 110 is changed.

  The threshold condition described in the second embodiment is an example in which two correlation coefficient threshold values coefTh and (−coefTh) are used (FIG. 12).

  In the present embodiment, four correlation coefficient threshold values are used. Specifically, a first correlation coefficient threshold value coefTh1, a second correlation coefficient threshold value (-coefTh1), a third correlation coefficient threshold value coefTh2, and a second correlation coefficient threshold value (-coefTh2) are set.

  FIG. 15 schematically shows threshold conditions for calculating the respiration rate in the present embodiment. The horizontal axis in FIG. 15 indicates the number of shifted frames S, and the vertical axis indicates the correlation coefficient value. FIG. 15 shows preset correlation coefficient threshold values coefTh1 (> 0), coefTh2 (> 0), -coefTh1 (<0), and -coefTh2 (<0). Also, coefTh1> coefTh2 and -coefTh2> -coefTh1. These use coefTh1 and (-coefTh1) as one set (first combination), and coefTh2 and (-coefTh2) as another set (second combination).

  In the first embodiment, it has been described that the correlation is strong when the value of the correlation coefficient is greater than 0.7. Subsequently, the correlation is said to be moderately strong when the correlation coefficient is in the range of 0.4 to 0.7. Therefore, although the correlation coefficient threshold value coefTh1 is set to 0.7 and coefTh2 is set to 0.6, any value between 0 and 1 may be set.

  The higher the correlation coefficient threshold value, the stronger the correlation required between the first image group and the second image group. That is, regarding correlation recognition, it can be said that the correlation coefficient threshold value according to the first combination is a stricter condition than the correlation coefficient threshold value according to the second combination.

  The respiratory cycle identification processing unit 110b determines that the correlation coefficient coef (S) is equal to or greater than a preset correlation coefficient threshold coefTh when the following threshold condition is satisfied. First, the parameter reset processing unit 110c increases the number of frames S, and the correlation coefficient calculation processing unit 110a obtains the correlation coefficient coef (S) in synchronization therewith.

  Threshold condition (a): After the correlation coefficient coef (S) becomes negative for the first time (S = S11), the correlation coefficient coef (S) maintains a negative state and becomes −coefTh1 or less (S = S12). ) Next, after coef (S) becomes positive (S = S13), it is assumed that the positive state is maintained and the correlation coefficient threshold value coefTh1 is exceeded (S = S14). At this time, the respiratory cycle identification processing unit 110b determines that the threshold condition is satisfied, and calculates the time required for one breath from the number S of frame images. The process at this time is completed in the first cycle.

  On the other hand, when the above-described threshold condition (a) is not satisfied, the respiratory cycle identification processing unit 110b determines whether or not the next threshold condition (b) is satisfied. This means that even if the strict threshold condition (a) is not satisfied, the respiratory cycle identification processing unit 110b does not determine that it is an apnea state.

  Threshold condition (b): The respiratory cycle identification processing unit 110b switches the correlation coefficient threshold value related to the first combination to the correlation coefficient threshold value related to the second combination. For the threshold condition (b), the correlation coefficient threshold values coefTh and (-coefTh) in the second embodiment may be read as coefTh2 and (-coefTh2), respectively. Since the processing related to the correlation coefficient threshold value coefTh1 has already been completed, for example, the respiratory cycle identification processing unit 110b holds the correlation coefficient value obtained by changing the number S of frame images in the buffer 112. If the correlation coefficient value obtained so far and further data for determining whether or not the threshold condition (b) is satisfied are additionally acquired, the respiratory cycle identification processing unit 110b can detect the threshold condition (b ) Is satisfied. As a result, whether or not the threshold condition (b) is satisfied is generally completed in the 1.5th cycle.

  Furthermore, when the above-described threshold condition (b) is not satisfied, the respiratory cycle specifying processing unit 110b determines whether or not the next threshold condition (c) is further satisfied. This means that even if the threshold condition (b) that is more lenient than the threshold condition (a) is not satisfied, the respiratory cycle identification processing unit 110b does not determine that it is an apnea state.

  Threshold condition (c): The respiratory cycle identification processing unit 110b switches the correlation coefficient threshold values coefTh1 and (−coefTh1) related to the second combination to only the correlation coefficient threshold value coefTh1. For the threshold condition (c), the correlation coefficient threshold value coefTh in the first embodiment may be read as coefTh2. That is, in the threshold condition (c), the respiratory cycle identification processing unit 110b uses only the correlation coefficient threshold value coefTh without using the correlation coefficient threshold value (-coefTh).

  Even in determining the threshold condition (c), the parameter resetting processing unit 110c can use the correlation coefficient value obtained so far by changing the number S of frame images. This is because the processing for the correlation coefficient threshold value coefTh2 has already been completed, and the correlation coefficient value is held in the buffer 112. As a result, whether or not the threshold condition (c) is satisfied is also completed in approximately 1.5th cycle.

  In addition, when the above-described threshold conditions (a) to (c) are not satisfied, the respiratory cycle identification processing unit 110b cannot calculate the respiration rate. This state means that the respiratory rate cannot be acquired. The respiratory cycle identification processing unit 110b may determine that this period is an apnea state.

(Modification of Embodiment 3)
A modification of the above process will be described.

  As shown in FIG. 15, the respiratory cycle identification processing unit 110b may determine whether or not S = S21 that gives a minimum value is included in the vicinity of the middle between the maximum values in S = S20 and S22. More severe conditions can be imposed by weighting this condition to the threshold condition (a).

  Alternatively, the respiratory cycle identification processing unit 110b determines that the maximum value (> 0) at S = S20 and / or S = S22 and the absolute value of the minimum value at S = S21 are within a predetermined range (for example, ± 10%). It may be determined whether or not it is in. By weighting this condition to the threshold condition (a), more severe conditions can be imposed as well.

  The conditions according to the above two modifications may be weighted simultaneously to the threshold condition (a).

  In Embodiments 2 and 3 (including modifications) described above, ± coefTh and ± coefThx (x: 1 or 2) are given as correlation coefficient threshold values, but it is essential that the absolute values are equal. Absent. For example, instead of ± coefTh, two positive values and negative values having different absolute values may be set independently.

(Embodiment 4)
In the first to third embodiments, the respiratory waveform (FIG. 7) is obtained, and the correlation coefficient is calculated using the luminance value of each frame image of the first image group and the second image group.

  Even if the inventors set the first image group and the second image group so as not to fall below the minimum data duplication rate, there may be an event that adversely affects the correlation coefficient due to external factors or the like. I found.

  For example, the second image group includes the effect of gradually increasing light, while the first image group includes the effect of gradually decreasing light as the door opens and closes and the amount of solar radiation inserted into the room changes. Consider the case. FIGS. 16A and 16B show respiratory waveforms obtained from the second image group and the first image group, respectively.

  A gradually increasing trend 500a is superimposed on the respiration waveform shown in FIG. 16 (a), and a gradually decreasing trend 500b is superimposed on the respiration waveform shown in FIG. 16 (b). It can be said that the trend is the influence of disturbance.

  When the correlation coefficient is obtained using the waveforms shown in FIGS. 16A and 16B, it is easily estimated that the value is difficult to satisfy the threshold condition described above. This is because the correlation coefficient calculation formula shown in Equation 1 calculates the vertical change from the average value of each waveform. FIGS. 17A and 17B show each respiratory waveform and its average value (solid line in the horizontal direction).

  When the trend is superimposed on the respiration waveform, it is difficult to calculate the respiration rate quickly and more accurately. Therefore, the inventors of the present application have examined the countermeasure for removing the trend from the respiratory waveform before obtaining the correlation coefficient in order to calculate an appropriate correlation coefficient even when the trend is superimposed.

  FIG. 18 shows a functional configuration of the respiration rate measuring apparatus 502 according to the present embodiment. For example, in FIG. 3A, the respiration rate measuring device 502 can be provided in the respiration rate monitoring device 30 instead of the respiration rate measuring device 102. The respiration rate measuring device 502 can be realized by the hardware configuration shown in FIG. Other functions and operations in the monitoring system 100 including the respiratory rate monitoring device 30 and the respiratory rate monitoring device 30 other than the respiratory rate measuring device 502 are the same. Therefore, the description regarding Embodiments 1-3 is used, and the description again in this Embodiment is abbreviate | omitted.

  The difference between the respiratory rate measuring device 502 and the respiratory rate measuring device 102 is that the respiratory rate measuring device 502 is provided with a new component for removing the trend of the respiratory waveform. Specifically, the respiration rate measuring apparatus 502 includes, as new components, a partial waveform extraction processing unit 510, first and second partial waveform buffers 512-1 and 512-2, an approximate data generation processing unit 514, First and second data buffers 516-1 and 516-2, a trend removal processing unit 518, and first and second respiratory waveform data buffers 520-1 and 520-2 are provided.

  First, the correspondence between these components and the components shown in FIG. 2 is as follows. The partial waveform extraction processing unit 510, the approximate data generation processing unit 514, and the trend removal processing unit 518 of the respiration rate measuring device 502 correspond to the arithmetic circuit 309 (the CPU 301 and / or the image processing circuit 305). The first and second partial waveform buffers 512-1 and 512-2, the first and second data buffers 516-1 and 516-2, and the first and second respiratory waveform data buffers 520-1 and 520-2 are stored in the RAM 303. Or the HDD 304. The CPU 301 and / or the image processing circuit 305 often incorporate a storage device called a cache memory. Such a cache memory may be used as all or part of each of the buffers described above.

  Hereinafter, the operation of each component will be described with reference to FIG.

  FIG. 19 is a flowchart showing a procedure of trend removal processing by the respiration rate measuring apparatus 502. This process is, for example, a process performed between steps S4 and S5 in FIG.

  In step S21, the partial waveform extraction processing unit 510 extracts the nth partial waveform data corresponding to the nth image group from the respiratory waveform data stored in the respiratory waveform data buffer 109 (n: 1 or 2, and so on). ).

  In step S22, the partial waveform extraction processing unit 510 stores the nth partial waveform data corresponding to the extracted nth image group in the nth partial waveform buffer 512-n.

  In step S23, the approximate data generation processing unit 514 calculates a coefficient of a primary approximate line from the nth partial waveform data. In the present embodiment, the approximate data generation processing unit 514 calculates the coefficient of the primary approximate line using the least square method.

Now, it is assumed that K luminance values are included in the nth partial waveform data. The frame image number S and its value IS (S: an integer from 1 to K) are represented by coordinates (S, IS). At this time, assuming that the distribution of the K luminance values follows I (S) = a · S + b, coefficients a and b that minimize J shown in the following equation 2 may be obtained. The addition is performed from S = 1 to K.
(Equation 2)
J = Σ (IS-I (S)) ²

  From the coefficients a and b obtained in this way, the first-order approximate straight line I (S) = a · S + b, which is a trend, can be specified. In order to speed up the calculation process, when K luminance values are included in the nth partial waveform data, it is not necessary to use all K values.

  In step S24, the approximate data generation processing unit 514 stores the coefficients a and b generated from the nth partial waveform in the nth data buffer 516-n.

  In step S25, the trend removal processing unit 518 subtracts the primary approximate line from the respiration waveform data corresponding to the nth image group. As a result, the nth respiratory waveform data corresponding to the nth image group from which the trend has been removed is obtained.

  In step S26, the trend removal processing unit 518 stores the calculated nth respiratory waveform data in the nth respiratory waveform data buffer 520-n.

  Through the above processing, respiratory waveform data from which the trend has been removed is obtained. If the processing shown in FIG. 4 is performed using this respiration waveform data, the respiration rate can be calculated quickly and more accurately.

(Embodiment 5)
In the first to fourth embodiments, the operator of the respiration rate monitoring apparatus designates a respiration region using an input device (for example, the mouse 20) (FIG. 5).

  In the present embodiment, a respiratory rate monitoring apparatus that recognizes a respiratory region and automatically sets the respiratory region will be described. The processing other than automating the breathing region is the same as that in any one of the first to fourth embodiments. Below, the detail of the process which sets a respiration area | region automatically is demonstrated.

  FIG. 20 shows a configuration of a monitoring system 101 having a respiratory rate monitoring device 31 that automatically acquires a respiratory region.

  In the monitoring system 101 shown in FIG. 20, the light source 50 emits infrared light 51. The optical axis of the light source 50 and the optical axis of the camera 10 are arranged close to each other. A reflector 60 is installed around the chest of subject 1. The reflector 60 is made of a retroreflecting material, and at least reflects infrared light retroreflectively.

  The retroreflecting material has an optical characteristic of reflecting incident light toward the incident direction. That is, the incident angle of light incident on the retroreflecting material is equal to the emission angle of light reflected by the retroreflecting material. However, this property is ideal and can actually be reflected in a direction different from the incident direction.

  In the present embodiment, the optical axis of the light source 50 and the optical axis of the camera 10 are arranged close to each other. As a result, the infrared light 51 emitted from the light source 50 is reflected by the retroreflecting material of the reflector 60, and most of the light enters the camera 10 as infrared light 52. Therefore, the camera 10 can photograph the subject 1 with a sufficient amount of light. In the present embodiment, a cloth coated with glass beads is used as the retroreflecting material.

  By using the retroreflective material, the disturbance light 53 incident on the retroreflective material is reflected as reflected light 54 in the incident direction. Since the reflected light 54 does not substantially enter the camera 10, the moving image captured by the camera 10 is less susceptible to disturbance light. However, the disturbance light 53 includes light that is directly incident on the camera 10 and light that is reflected by other than the reflector 60 and indirectly incident on the camera 10. Therefore, it is difficult to completely remove the influence of the disturbance light 53.

  In the present embodiment, a retroreflecting material is used for the camera 10 to capture an image with a sufficient amount of light, but this is an example. It is not essential to use a retroreflecting material.

  FIG. 21 mainly shows a functional configuration of a respiration rate monitoring apparatus 31 that automatically acquires a respiration region.

  The difference between the respiratory rate monitoring device 31 and the respiratory rate monitoring device 30 (FIG. 3A) is that, instead of the respiratory region designation data reception I / F 104, a respiratory region selection unit 120 that automatically specifies a respiratory region is provided. is there. The breathing area selection unit 120 corresponds to the arithmetic circuit 309 (the CPU 301 and / or the image processing circuit 305). Hereinafter, the operation of the breathing region selection unit 120 will be described.

  FIG. 22 is a flowchart illustrating a processing procedure of the respiration rate monitoring apparatus 30 regarding processing for automatically acquiring a respiration region.

  In step S31, the breathing region selection unit 120 reads data of a plurality of frame images stored in the frame image data buffer 106, and divides each frame image into a plurality of small regions. The plurality of small regions mean regions partitioned in a mesh shape in FIG. Each small area has a size of, for example, 64 pixels horizontally and 64 pixels vertically. Note that “divide” does not require division as an actual operation. For example, the operation of setting the size of the partial area as a unit for cutting out an image or a unit for performing processing may be included in the “dividing” operation described here.

  In step S <b> 32, the breathing region selection unit 120 specifies a region where the retroreflective material is present and a region including a body movement location due to a biological reaction based on the luminance value of each small region.

  Hereinafter, it demonstrates in detail, referring FIG. 23 (a)-23 (c).

  FIGS. 23A and 23B show examples of partial areas Q in two frame images taken at different times. It is assumed that a region R shown in FIGS. 23A and 23B is a high luminance region in which reflected light from the reflector is detected. The partial area Q may or may not become a high luminance area due to body movement accompanying breathing.

  FIG. 23C shows a change in the luminance value of the partial region Q at this time. When the luminance of the partial area Q is observed in time series over a plurality of frame images, a group of frame images exceeding a certain threshold T and a group of frame images not exceeding exist alternately. The image processing circuit 305 specifies a partial region group in which such a change appears using a plurality of frame images. The reason for specifying the partial region group is that, in the present embodiment, it is necessary to calculate the difference in luminance value.

  The partial area where the reflector 60 exists can be specified in each frame image. On the other hand, the partial region including the body movement location can be specified over a plurality of frame images, that is, between the plurality of frame images.

  The partial area where the reflector exists is specified by the following process. For example, the breathing area selection unit 120 holds in advance in the ROM 302 information on luminance values of small areas observed when a reflector is present. Using this information as a threshold value of the luminance value, a partial region having a luminance value equal to or higher than the threshold value is specified as a partial region where the reflector exists.

  The luminance value at this time may be the sum of the luminance values of the pixels included in the small region, or may be an average value. Depending on whether the sum of luminance values or the average value is adopted, the threshold may also change. Since the calculation process for calculating the sum is smaller in calculation load and faster than the calculation process for calculating the average value, it is the sum of the luminance values of the pixels included in the small region in this embodiment. And

  On the other hand, a small region including a body movement location is specified by the following process. As described above, the region where the reflected light from the reflector is observed fluctuates due to body movement due to a biological reaction (respiration). With regard to each frame image, attention is paid to a small area Q existing at a certain common coordinate position. FIGS. 23A and 23B show examples of small regions Q in two frame images taken at different times as a more general example. It is assumed that a region R shown in FIGS. 23A and 23B is a high luminance region in which reflected light from the reflector is detected. The small area Q may or may not become a high luminance area due to body movement accompanying breathing.

  FIG. 23C shows a change in the luminance value of the small region Q at this time. When the luminance of the small region Q is observed in a time series over a plurality of frame images, a group of frame images exceeding a certain threshold T and a group of frame images not exceeding exist alternately. The breathing region selection unit 120 specifies a small region group in which such a change appears using a plurality of frame images. The reason why the small region group is specified is that it is necessary for calculating the difference between the luminance values in the present embodiment.

  Through the above processing, the small region Q including the body movement location can be specified. Thereafter, the processing performed by the respiration rate monitoring device 31 is the same as the processing performed by the respiration rate monitoring device 30 according to the first embodiment after the respiration region is designated via the respiration region designation data reception I / F 104.

  The embodiment has been described above. In the above-described first to fifth embodiments, it is assumed that the respiration rate monitoring apparatus receives and processes a moving image signal sequentially transmitted from the camera 10. However, the respiratory rate monitoring device does not necessarily need to receive a moving image signal in real time.

  FIG. 24 shows a configuration of the monitoring system 100 in which moving images are stored in a hard disk drive (HDD), and the respiratory rate monitoring device 30 acquires image data from the HDD. It is also possible to temporarily store the moving image data in the HDD, and the respiration rate monitoring device 30 can read out the necessary moving image information from the HDD for processing.

  In the above embodiment, the respiration rate means the number of respirations actually performed by the subject 1 within a certain period. However, this is an example. As another example, the number of breaths per minute of the subject 1 estimated by measuring the number of breaths actually performed by the subject 1 during a certain period and estimated from the number of breaths may be used. For example, when it is measured that the subject 1 took 3 breaths in 10 seconds, the respiratory rate monitoring device 30 may output the respiratory rate as 18 times / minute. In the measurement of the respiration of 3 times, the process of each embodiment mentioned above should just be performed.

  In the above-mentioned embodiment, the example which monitors the respiration rate of the test subject who is a person was demonstrated. However, as long as body movement is caused by respiration, the above-described embodiments can be used to monitor the respiration rate of any animal. For example, each embodiment is applicable when monitoring the respiration rate of pet animals such as dogs and cats, and laboratory animals such as pigs and mice. In this specification, humans and animals other than humans may be collectively referred to as “subjects” as subjects to be monitored for respiration.

  In this specification, it was determined whether or not the respiratory cycle could be specified based on whether or not the correlation coefficient was equal to or greater than the correlation coefficient threshold value. However, whether or not to include a boundary value is not strict, and can be selected as appropriate. For example, “below a certain value” may be read as “less than a certain value”. “More than a certain value” may be read as “greater than a certain value”. Therefore, based on whether or not the correlation coefficient is larger than the correlation coefficient threshold value, it may be determined whether or not the respiratory cycle can be specified.

  This specification discloses a respiratory monitoring device, a respiratory rate measuring device, a respiratory rate measuring method, and a computer program described in the following items.

[Item 1]
A first image group for acquiring data of a first image group composed of N (N: integer greater than or equal to 2) frame images counted from a specific frame image of a moving image in which a subject that breathes is captured An acquisition processing unit;
Second image group acquisition for acquiring data of a second image group composed of consecutive N frame images, starting from S (S: integer greater than or equal to) S past frame images from the specific frame image A processing unit;
A first image feature quantity group creation processing unit that calculates an image feature quantity of each frame image of the first image group and creates a first image feature quantity group that is a set of the image feature quantities;
A second image feature quantity group creation processing unit that calculates an image feature quantity of each frame image of the second image group and creates a second image feature quantity group that is a set of the image feature quantities;
A correlation coefficient calculation processing unit for calculating a correlation coefficient between the first image feature quantity group and the second image feature quantity group;
A respiratory cycle specifying processing unit that attempts to specify a respiratory cycle according to the relationship between the correlation coefficient and the number S of frame images that is the difference between the first image group and the second image group;
A respiratory rate monitoring device comprising: a parameter resetting processing unit that resets the number S of frames when the respiratory cycle cannot be specified by the respiratory cycle specification determination processing unit.

[Item 2]
An interface for receiving area designation information for designating a part of the area in the frame image;
The first image feature amount group creation processing unit calculates the image feature amount of the partial area and creates the first image feature amount group,
The respiration rate monitoring apparatus according to Item 1, wherein the second image feature quantity group creation processing unit creates the second image feature quantity group by calculating image feature quantities of the partial region.

[Item 3]
The respiratory rate monitoring apparatus according to item 1 or 2, wherein the parameter resetting processing unit sets the S frame images so as not to be larger than the N frame images.

[Item 4]
3. The respiratory rate monitoring apparatus according to item 1 or 2, wherein the parameter resetting processing unit sets the S frame images and the N frame images so as not to be larger than the N frame images.

[Item 5]
From the item 1, the respiratory cycle specifying processing unit specifies a period corresponding to the S frame images as a respiratory cycle when the correlation coefficient is greater than or equal to a predetermined threshold. 5. The respiratory rate monitoring apparatus according to any one of 4.

[Item 6]
6. The respiratory rate monitoring apparatus according to any one of items 1 to 5, wherein the parameter resetting processing unit increases the specific frame number S by K (K: an integer of 1 or more).

[Item 7]
The parameter resetting processing unit is configured to hold the ratio of frame images included in common in the first image group and the second image group larger than or greater than a predetermined minimum data duplication rate. The respiration rate monitoring apparatus according to any one of items 1 to 6, wherein the value of N is increased equally.

[Item 8]
The respiratory cycle specifying processing unit, when the correlation coefficient sequentially changes negative, positive and negative as the number of frame images S, which is the difference between the first image group and the second image group, increases. When the positive value is greater than or equal to the predetermined threshold value, the time corresponding to the S frame images is specified as the respiratory cycle of the subject. The respiratory rate monitoring apparatus in any one.

[Item 9]
The respiratory cycle specifying processing unit, when the correlation coefficient sequentially changes negative, positive and negative as the number of frame images S, which is the difference between the first image group and the second image group, increases. When the first negative value is less than or less than the first threshold and the positive value is greater than or greater than the predetermined threshold, it corresponds to the S frame images 8. The respiratory rate monitoring apparatus according to any one of items 1 to 7, wherein time is specified as a respiratory cycle of the subject.

[Item 10]
10. The respiratory rate monitoring apparatus according to any one of items 1 to 9, wherein the image feature amount is a value based on a pixel value.

[Item 11]
The respiratory rate monitoring apparatus according to any one of items 1 to 10, wherein the image feature amount is a value based on a luminance value.

[Item 12]
Approximate data generation for obtaining a first-order approximation line of a trend that is a disturbance superimposed on the respiratory waveform by applying a least square method to the image feature quantity of the first image group and the image feature quantity of the second image group A processing unit;
The respiratory rate monitoring apparatus according to any one of items 1 to 11, further comprising: a trend removal processing unit that removes the trend from the respiratory waveform by subtracting the first-order approximate straight line from the respiratory waveform.

[Item 13]
A first acquisition step of acquiring data of a first image group composed of N (N: an integer greater than or equal to 2) frame images counted from a specific frame image of a moving image in which a subject to be breathed is captured. When,
A second acquisition step of acquiring data of a second image group composed of consecutive N frame images by counting from S (S: integer of 1 or more) past frame images from the specific frame image; ,
A first creation step of calculating an image feature quantity of each frame image of the first image group and creating a first image feature quantity group that is a set of the image feature quantities;
A second creation step of calculating an image feature quantity of each frame image of the second image group and creating a second image feature quantity group that is a set of the image feature quantities;
Calculating a correlation coefficient between the first image feature quantity group and the second image feature quantity group;
Trying to specify the respiratory cycle according to the relationship between the correlation coefficient and the number S of frame images that is the difference between the first image group and the second image group;
A respiration rate monitoring method, comprising: resetting the number of frames S when the respiration cycle cannot be specified.

[Item 14]
A computer program executed by a computer, the computer program being
A first acquisition step of acquiring data of a first image group composed of N (N: an integer greater than or equal to 2) frame images counted from a specific frame image of a moving image in which a subject to be breathed is captured. When,
A second acquisition step of acquiring data of a second image group composed of consecutive N frame images by counting from S (S: integer of 1 or more) past frame images from the specific frame image; ,
A first creation step of calculating an image feature quantity of each frame image of the first image group and creating a first image feature quantity group that is a set of the image feature quantities;
A second creation step of calculating an image feature quantity of each frame image of the second image group and creating a second image feature quantity group that is a set of the image feature quantities;
Calculating a correlation coefficient between the first image feature quantity group and the second image feature quantity group;
Trying to specify the respiratory cycle according to the relationship between the correlation coefficient and the number S of frame images that is the difference between the first image group and the second image group;
And a step of resetting the number of frames S when the respiratory cycle cannot be specified.

[Item 15]
A recording medium storing the computer program according to Item 14.

  The present invention can be used as a method for measuring the respiration rate of a subject in a non-contact manner from a moving image obtained by photographing the subject. The present invention can also be used as an apparatus, a system, and a computer program for measuring such respiration rate.

30, 31 Respiration rate monitoring device 301 CPU
305 Image processing circuit 306 Video input interface (I / F)
307 USB I / F
308 Output I / F
309 Arithmetic circuit 100, 101 Monitoring system 102, 502 Respiration rate measuring device

Claims (15)

A first image group for acquiring data of a first image group composed of N (N: integer greater than or equal to 2) frame images counted from a specific frame image of a moving image in which a subject that breathes is captured An acquisition processing unit;
Second image group acquisition for acquiring data of a second image group composed of consecutive N frame images, starting from S (S: integer greater than or equal to) S past frame images from the specific frame image A processing unit;
A first image feature quantity group creation processing unit that calculates an image feature quantity of each frame image of the first image group and creates a first image feature quantity group that is a set of the image feature quantities;
A second image feature quantity group creation processing unit that calculates an image feature quantity of each frame image of the second image group and creates a second image feature quantity group that is a set of the image feature quantities;
A correlation coefficient calculation processing unit for calculating a correlation coefficient between the first image feature quantity group and the second image feature quantity group;
A respiratory cycle specifying processing unit that attempts to specify a respiratory cycle according to the relationship between the correlation coefficient and the number S of frame images that is the difference between the first image group and the second image group;
A respiratory rate monitoring device comprising: a parameter resetting processing unit that resets the number S of frames when the respiratory cycle cannot be specified by the respiratory cycle specification determination processing unit. An interface for receiving area designation information for designating a part of the area in the frame image;
The first image feature amount group creation processing unit calculates the image feature amount of the partial area and creates the first image feature amount group,
The respiration rate monitoring apparatus according to claim 1, wherein the second image feature quantity group creation processing unit creates the second image feature quantity group by calculating an image feature quantity of the partial region.   The respiration rate monitoring apparatus according to claim 1 or 2, wherein the parameter resetting processing unit sets the S frame images so as not to be larger than the N frame images.   The respiration rate monitoring apparatus according to claim 1 or 2, wherein the parameter resetting processing unit sets the S frame images and the N frame images so as not to be larger than the N frame images.   The respiratory cycle specifying processing unit specifies a period corresponding to the S frame images as a respiratory cycle when the correlation coefficient is greater than or equal to a predetermined threshold. To 4. The respiratory rate monitoring apparatus according to any one of 4 to 4.   6. The respiratory rate monitoring apparatus according to claim 1, wherein the parameter reset processing unit increases the specific frame number S by K (K: an integer of 1 or more).   The parameter resetting processing unit is configured to hold the ratio of frame images included in common in the first image group and the second image group larger than or greater than a predetermined minimum data duplication rate. The respiration rate monitoring apparatus according to any one of claims 1 to 6, wherein the value of N is increased equally.   The respiration cycle specifying processing unit first becomes negative (less than 0 or less than 0) as the number of frame images S, which is the difference between the first image group and the second image group, increases. After that, when sequentially transitioning to positive (0 or more or greater than 0) and negative (less than 0 or less than 0), when the positive value is greater than or equal to the predetermined threshold, The respiration rate monitoring apparatus according to claim 1, wherein a time corresponding to the S frame images is specified as a respiration cycle of the subject.   The respiration cycle specifying processing unit is positive and negative after the correlation coefficient becomes negative for the first time as the number S of frame images, which is the difference between the first image group and the second image group, increases. When sequentially changing, if the first negative value is less than or equal to the first threshold value and the positive value is greater than or equal to the predetermined threshold value, The respiratory rate monitoring apparatus according to claim 1, wherein a time corresponding to a frame image is specified as a breathing cycle of the subject.   The respiratory rate monitoring apparatus according to claim 1, wherein the image feature amount is a value based on a pixel value.   The respiration rate monitoring apparatus according to claim 1, wherein the image feature amount is a value based on a luminance value. Approximate data generation for obtaining a first-order approximation line of a trend that is a disturbance superimposed on the respiratory waveform by applying a least square method to the image feature quantity of the first image group and the image feature quantity of the second image group A processing unit;
The respiratory rate monitoring apparatus according to claim 1, further comprising: a trend removal processing unit that removes the trend from the respiratory waveform by subtracting the first-order approximate straight line from the respiratory waveform. A first acquisition step of acquiring data of a first image group composed of N (N: an integer greater than or equal to 2) frame images counted from a specific frame image of a moving image in which a subject to be breathed is captured. When,
A second acquisition step of acquiring data of a second image group composed of consecutive N frame images by counting from S (S: integer of 1 or more) past frame images from the specific frame image; ,
A first creation step of calculating an image feature quantity of each frame image of the first image group and creating a first image feature quantity group that is a set of the image feature quantities;
A second creation step of calculating an image feature quantity of each frame image of the second image group and creating a second image feature quantity group that is a set of the image feature quantities;
Calculating a correlation coefficient between the first image feature quantity group and the second image feature quantity group;
Trying to specify the respiratory cycle according to the relationship between the correlation coefficient and the number S of frame images that is the difference between the first image group and the second image group;
A respiration rate monitoring method, comprising: resetting the number of frames S when the respiration cycle cannot be specified. A computer program executed by a computer, the computer program being
A first acquisition step of acquiring data of a first image group composed of N (N: an integer greater than or equal to 2) frame images counted from a specific frame image of a moving image in which a subject to be breathed is captured. When,
A second acquisition step of acquiring data of a second image group composed of consecutive N frame images by counting from S (S: integer of 1 or more) past frame images from the specific frame image; ,
A first creation step of calculating an image feature quantity of each frame image of the first image group and creating a first image feature quantity group that is a set of the image feature quantities;
A second creation step of calculating an image feature quantity of each frame image of the second image group and creating a second image feature quantity group that is a set of the image feature quantities;
Calculating a correlation coefficient between the first image feature quantity group and the second image feature quantity group;
Trying to specify the respiratory cycle according to the relationship between the correlation coefficient and the number S of frame images that is the difference between the first image group and the second image group;
And a step of resetting the number of frames S when the respiratory cycle cannot be specified.   A recording medium storing the computer program according to claim 14.

Images