JP2018005612A - Image processing device, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable an inspector to efficiently review an inspection target in an image. An image processing apparatus 200 sets a partial area in an image based on an acquisition means for acquiring information about an object to be inspected extracted from an image and information about an object acquired by the acquisition means. It includes a setting means and a determining means for determining the priority of the partial area set by the setting means based on the information about the object acquired by the acquisition means. [Selection diagram] Fig. 1

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

As part of the inspection of the structure, in order to grasp the distribution of deformation such as cracks and rust and the transition from the past, the deformation is analyzed based on the image captured by the camera. In order to improve the efficiency of analysis work, various techniques for extracting deformation by image processing have been proposed, but there are many cases where erroneous extraction results are generated. Therefore, a review work is performed for the purpose of correction by human visual judgment by taking the deformation extracted by image processing as a candidate.
In Patent Document 1, an inspection target part is extracted from an inspection video in which a building is imaged, and an importance level indicating the level of necessity of inspection by an inspector is expressed based on the distribution of the extracted inspection target part in a frame ( A point to be calculated in units of (image) is disclosed. In this Patent Document 1, an important section in an inspection video is determined based on the calculated importance, and reproduction of the inspection video is controlled.

JP2013-134190A

In a review operation related to a structure inspection application, the inspection image naturally increases, and the number of objects to be inspected in the inspection image also becomes enormous. The technique described in Patent Literature 1 determines an important section in an inspection video, and efficiently reviews the inspection target parts in a unit suitable for the visual judgment of the inspector in the inspection image. could not.
Accordingly, an object of the present invention is to enable an inspector to efficiently review an object to be inspected in an image.

  In order to solve the above problems, an aspect of the image processing apparatus according to the present invention includes an acquisition unit that acquires information about an object to be inspected extracted from an image, and information about the object acquired by the acquisition unit. Setting means for setting a partial area in the image, and determining means for determining the priority of the partial area set by the setting means based on information on the object acquired by the acquiring means; .

  According to the present invention, an inspector can efficiently review an object to be inspected in an image.

It is a block diagram of the image processing system in 1st embodiment. It is a hardware block diagram of an image processing apparatus. It is a flowchart explaining the flow of an image process. It is a figure explaining the data structure in 1st embodiment. It is a flowchart explaining an area | region setting process. It is a flowchart explaining an area | region feature-value calculation process. It is a flowchart explaining area | region priority determination processing. It is a figure which shows the example of a display screen. It is a block diagram of the image processing system in 2nd embodiment. It is a flowchart explaining the area | region setting process of 2nd embodiment. It is a figure explaining the data structure in 2nd embodiment. It is a block diagram of the image processing system in 3rd embodiment. It is a flowchart explaining the area | region feature-value calculation process of 3rd embodiment. It is a figure explaining the data structure in 3rd embodiment. It is a flowchart explaining the area | region feature-value calculation process of 4th embodiment. It is a flowchart explaining the area | region priority determination process of 4th embodiment. It is a figure explaining the data structure in 4th embodiment.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. The embodiment described below is an example as means for realizing the present invention, and should be appropriately modified or changed depending on the configuration and various conditions of the apparatus to which the present invention is applied. It is not limited to the embodiment.
FIG. 1 is a configuration diagram of an image processing system 100 in the present embodiment. The image processing system 100 is a system that extracts an object to be inspected based on an image, and sets and presents an area suitable for review of the object as a partial area in the image. In the present embodiment, a case will be described in which an object to be inspected is a deformation of a structure (such as cracks or rust). The object to be inspected may be a part constituting a structure or a human face.
The image processing system 100 includes an image processing device 200 and an image display device 300. The image processing apparatus 200 is an apparatus that extracts a deformation candidate from an image and outputs a review area and a priority of the review area. Here, the review area is a partial area in the image suitable for the review of deformation, and the priority is defined as an index to be preferentially reviewed as the value increases.

The image processing apparatus 200 includes an image input unit 201, a candidate extraction unit 202, an area setting unit 203, a feature amount calculation unit 204, and a priority determination unit 205.
The image input unit 201 inputs an image to be processed. In the present embodiment, the image input by the image input unit 201 is an image obtained by capturing a structure. The candidate extraction unit 202 extracts deformation candidates from the image input by the image input unit 201. The area setting unit 203 sets the review area as a partial area in the image based on the information on the deformation candidates extracted by the candidate extraction unit 202. When setting the partial area, the area setting unit 203 sets the review area so as to cover the entire image. The feature amount calculation unit 204 calculates the feature amount of the review region set by the region setting unit 203. The priority determination unit 205 determines the priority of the review region based on the feature amount of the review region calculated by the feature amount calculation unit 204.

The image display device 300 is a device that presents an image based on the information regarding the review area and priority output from the image processing device 200. The image display device 300 includes an area presentation unit 301 and an image display unit 302.
The area presentation unit 301 presents a display form related to the review area and priority output from the image processing apparatus 200. The image display unit 302 displays an image on a display device (not shown) according to the display form presented from the region presentation unit 301. Here, the display device includes a CRT (Cathode Ray Tube) display, a liquid crystal display, or the like.
In the present embodiment, the case where the image processing device 200 and the image display device 300 are different devices will be described. However, the image processing device 200 and the image display device 300 may be a single device. In addition, the image processing apparatus 200 may include only a part of the functions of the image display apparatus 300 (for example, the area presentation unit 301).

FIG. 2 is a hardware configuration diagram of the image processing apparatus 200.
The image processing apparatus 200 includes a CPU 21, a RAM 22, a ROM 23, a NetIF 24, an input device 25, an HDD 26, and a system bus 27. The CPU 21 controls the operation of the image processing apparatus 200 in an integrated manner, and controls each component (22 to 26) via the system bus 27. The RAM 22 functions as a main memory of the CPU 21 or functions as a work memory necessary for loading an execution program and executing the program.
The ROM 23 records a control program that defines the operation processing procedure of the CPU 21. The ROM 23 includes a program ROM that records basic software (OS), which is a system program that controls the equipment of the computer system, and a data ROM that records information necessary for operating the system. The function of the ROM 23 can also be realized by the HDD 26 described later. The CPU 21 implements various functional operations by loading a necessary program from the ROM 23 into the RAM 22 when executing the processing, and executing the loaded program.

  The NetIF 24 is a network interface that performs input / output control of data such as an image transmitted / received via a network. The input device 25 is configured by a pointing device such as a touch panel, a keyboard, and a mouse, and receives an operation instruction from a user. The HDD 26 is a hard disk drive. The HDD 26 can be used for storing application programs and data such as images. The system bus 27 is an input / output bus (address bus, data bus, and control bus) for connecting the above-described units.

The functions of each unit of the image processing apparatus 200 shown in FIG. 1 can be realized by the CPU 21 executing a program. However, at least some of the units of the image processing apparatus 200 illustrated in FIG. 1 may operate as dedicated hardware. In this case, the dedicated hardware operates based on the control of the CPU 21.
In addition, in the case of the image display apparatus 300, in addition to the components shown in FIG. The function of each part of the image display apparatus 300 shown in FIG. 1 can be realized by the CPU of the image display apparatus executing a program. However, at least some of the units of the image display apparatus 300 illustrated in FIG. 1 may operate as dedicated hardware. In this case, the dedicated hardware operates based on the control of the CPU of the image display apparatus 300.

FIG. 3 is a flowchart for explaining the operation of the image processing apparatus 200.
The process of FIG. 3 is started in response to an instruction input by the user, for example. However, the start timing of the process in FIG. 3 is not limited to the above timing. The image processing apparatus 200 can implement each process illustrated in FIG. 3 by the CPU 21 reading and executing a necessary program. However, as described above, at least a part of each element of the image processing apparatus 200 illustrated in FIG. 1 may operate as dedicated hardware so that the processing in FIG. 3 may be realized. In this case, the dedicated hardware operates based on the control of the CPU 21. Hereinafter, the alphabet S means a step in the flowchart.

First, in S1, the image input unit 201 reads an image designated by the user. Next, in S2, the candidate extraction unit 202 extracts deformation candidates included in the image read in S1. As the deformation candidate, for example, there is a crack candidate such as concrete. As a technique for extracting such deformation candidates, a technique for extracting a contour by edge detection processing using a Sobel filter can be used. The candidate extraction unit 202 can detect a closed area surrounded by the extracted contour and extract a vectorized shape by elliptic approximation or the like as a deformation candidate.
The candidate extraction unit 202 outputs information indicating the extracted modification candidates as modification candidate information. As shown in FIG. 4A, the deformation candidate information 410 includes, for example, an ID 411 that uniquely indicates a deformation candidate, a start point coordinate position 412 of the deformation candidate for the entire image, an end point coordinate position 413, and the like. And a width 414. Furthermore, the deformation candidate information 410 may include a reliability 415 indicating the likelihood of deformation (for example, the likelihood). The reliability 415 can be expressed by an average of values obtained as a result of performing the Sobel filter processing for pixels belonging to the vicinity of the vicinity 8 of the side.

Note that the method for extracting the deformation candidates and the deformation candidate information are not limited to the above. In this embodiment, the case where the image processing apparatus 200 includes the candidate extraction unit 202 will be described. However, another apparatus may perform the process of extracting an object to be inspected (deformation candidate) from an image. . That is, the image processing apparatus 200 may be configured to acquire information on an object (deformation candidate) to be inspected extracted from an image from another apparatus.
Furthermore, in this embodiment, in order to extract linear deformation candidates such as cracks on the surface of the structure, the deformation candidate information 410 includes a start point coordinate position 412, an end point coordinate position 413, and a width 414. However, the information included in the deformation candidate information 410 may be information indicating the position, size, shape, and the like in the deformation candidate image, and is not limited to the above.

In S <b> 3, the area setting unit 203 sets a deformation candidate review area. Specifically, the area setting unit 203 executes the area setting process shown in FIG. 5 in S3, and sets review area information that is information indicating the review area. As shown in FIG. 4B, the review area information 420 can include an ID 421 that uniquely indicates the review area, a center point coordinate 422 and a size 423 of the review area, and a registered deformation candidate 424.
Here, the registered deformation candidate 424 is a deformation candidate registered corresponding to each review area, and is a deformation candidate included in the corresponding review area. Each deformation candidate is registered only in one review area and is not registered in a plurality of review areas. Hereinafter, the region setting process will be described in detail with reference to FIG.

In S31, the area setting unit 203 selects one deformation candidate that is not registered in any review area. For example, a deformation candidate that is not registered in the review area and has the lowest reliability 415 is selected. In S32, the region setting unit 203 sets a review region based on the information on the deformation candidate selected in S31. For example, the region setting unit 203 sets a region centered on the deformation candidate selected in S31 as a review region. The size of the review area may be a predetermined value related to the display resolution, or may be variably set according to the reliability 415 of the selected deformation candidate. In the latter case, the size of the review area is reduced as the reliability 415 of the selected deformation candidate is lower.
In S33, the area setting unit 203 selects a deformation candidate included in the review area set in S32. In S33, the area setting unit 203 may select a deformation candidate that is completely included in the review area, or may select a deformation candidate in which a predetermined ratio or more falls within the review area. In S34, the area setting unit 203 registers the deformation candidate selected in S33 in the review area set in S32.

In S35, the area setting unit 203 checks whether there is a deformation candidate that is not registered in any review area. If there is a deformation candidate, the process returns to S31, and if not, the process proceeds to S36. In S36, the area setting unit 203 sets a review area to the remaining area for the entire image. Specifically, the area setting unit 203 detects an area that is not covered by the review area group determined in S32 with respect to an area representing the entire image, and sets a new review area for each area. If the area not covered by the review area group is not divided for each rectangular area, the area is appropriately divided into a plurality of rectangular areas, and each divided area is set as a review area.
With the area setting process of FIG. 5, the area setting unit 203 outputs the review area information 420 shown in FIG. That is, the area setting unit 203 outputs review area information 425 for reviewing an area in which no deformation candidate is detected, in addition to information on a review area for reviewing an area including the deformation candidate.

Returning to FIG. 3, in S4, the feature amount calculation unit 204 calculates a feature amount for each review area. Specifically, the feature amount calculation unit 204 calculates the feature amount for each review region by executing the region feature amount calculation process shown in FIG. 6, and outputs feature amount information that is information indicating the feature amount. In the present embodiment, the case where the feature amount calculation unit 204 outputs the statistics (average and standard deviation) of the reliability 415 of the deformation candidate associated with the review region as the feature amount of the review region is described. To do.
As illustrated in FIG. 4C, the feature amount information 430 can include an ID 431 that uniquely indicates a review region, a reliability average 432 and a reliability standard deviation 433 that are feature amounts of the review region. Hereinafter, the region feature amount calculation processing will be described in detail with reference to FIG.

In S <b> 41, the feature amount calculation unit 204 determines whether or not the number of deformation candidates registered in the focused review area is equal to or greater than N in advance. When the number of deformation candidates is N or more, the process proceeds to S42, and when the number of deformation candidates is less than N, the process proceeds to S43.
In S42, the feature amount calculation unit 204 calculates a statistical amount of the reliability 415 of the deformation candidate associated with the review area, and outputs it as a feature amount. For example, the feature amount calculation unit 204 calculates the average θμ and the standard deviation θσ of the reliability 415 of the deformation candidate registered in the review area as the feature amount. On the other hand, in S43, the feature amount calculation unit 204 outputs a predetermined value as a feature amount. For example, when the average θμ and the standard deviation θσ of the reliability 415 of the deformation candidate are output as feature amounts, the feature amount calculation unit 204 has features having an average of 0 and a standard deviation of 0 as shown in FIG. Quantity information 434 is output.
By the region feature amount calculation process of FIG. 6, the feature amount calculation unit 204 outputs the feature amount information 430 shown in FIG. In the present embodiment, the feature amount of the review area is the average and standard deviation of the reliability 415. However, the feature amount is, for example, variance, median, quartile, maximum value, minimum value, and the like. Such a statistic may be used.

Returning to FIG. 3, in S <b> 5, the priority determination unit 205 determines the priority of the review area based on the feature amount information 430. Specifically, the priority determination unit 205 executes the region priority determination process shown in FIG. 7 to determine the priority of the review region, and outputs priority information indicating the priority. As shown in FIG. 4D, the priority information 440 can include an ID 441 that uniquely indicates the review area and a priority 442 of the review area. Hereinafter, the region priority determination process will be described in detail with reference to FIG.
In S51, the priority determination unit 205 calculates the priority by adding the feature quantities of the review area with weights. Specifically, the priority determination unit 205 calculates the priority P ′ _i based on the following equation (1). Here, i is a subscript representing each review area.

In the above equation (1), k is a subscript representing the type of feature quantity θ. W _k is a weight determined for each feature quantity θ _k . In the present embodiment, the weight wμ = −1 for the average θμ and the weight wσ = 1 for the standard deviation θσ. In this embodiment, the priority is calculated by weighted addition of feature amounts. However, the priority may be calculated by weighted addition of the difference between the feature amount and a preset reference value. Good.

In S52, the priority determination unit 205 normalizes the priority P ′ _i calculated in S51. For example, the priority determining unit 205 normalizes the priority based on the following equation (2) based on the maximum value P ′ _max and the minimum value P ′ _{min of} the priority P ′ _i calculated in S51. The degree P _i is calculated.
P _i = (P ′ _i −P ′ _min ) / (P ′ _max −P ′ _min ) (2)
The normalized priority P _i is a value of 0 ≦ P _i ≦ 1. As described above, the normalized priority P _i is converted so as to be intuitively understandable when presented to the user. Therefore, when the priority value is not presented directly to the user, the normalization process of S52 can be omitted.

By the region priority determination process of FIG. 7, the priority determination unit 205 outputs the priority information 440 shown in FIG. In the present embodiment, as indicated by reference numeral 443 in FIG. 4D in FIG. 7, the priority of the review area (ID = AN0001) not including the deformation candidate is the maximum value = 1.0, which is preferentially Set to be reviewed.
Returning to FIG. 3, in S6, the image processing apparatus 200 outputs the review area and the priority to the image display apparatus 300, and ends the process.

Next, an example of a display form of the review area and priority in the image display device 300 will be described.
FIG. 8A is an example of presentation of a review area. The presentation screen 500 is a screen displayed on the display device of the image display apparatus 300, and has an area 501 for displaying an image corresponding to the review area. Area 501 displays a partial image represented by center point coordinates 422 and size 423 included in review area information 420 shown in FIG.
In the area 501, the deformation candidate represented by the deformation candidate information 410 shown in FIG. 4A is superimposed and displayed as vector data on the partial image. The modification candidates displayed in the area 501 include modification candidates 502 included in the registered modification candidate 424 and modification candidates 503 not included in the registered modification candidate 424. The deformation candidate 502 and the deformation candidate 503 may be displayed by different display methods. For example, the deformation candidate 502 may be highlighted by changing the color, changing the brightness, blinking, or the like.

  Further, the area 501 may be displayed by switching the review areas in descending order of priority. In this case, the image display apparatus 300 first selects the review area with the highest priority based on the priority information 440 shown in FIG. 4D, and performs display related to the review area. Thereafter, the image display apparatus 300 selects a review area with the next highest priority in accordance with a user instruction, and performs display related to the review area. The display of the review area may be switched at the timing when the user presses the button 504 displayed on the presentation screen 500, or may be switched automatically every predetermined time.

  The display form of the review area is not limited to the form shown in FIG. As another display form, as shown in FIG. 8B, a plurality of review areas may be displayed in an area 511 of the presentation screen 500. In this case, the area 511 may display a plurality of review areas arranged in order of priority. FIG. 8B shows an example in which two review areas are displayed side by side in order of priority from left to right on the presentation screen 500. When all the review areas cannot be displayed in the area 511, the display area may be switched to the next highest priority review area group according to a user instruction (for example, pressing the button 512).

Furthermore, as illustrated in FIG. 8C, an area 521 for displaying a review area and an area 522 for displaying a link of the review area may be provided on the presentation screen 500. In this case, in the area 522, the links to the review area may be arranged and displayed in descending order of priority, and the review area corresponding to the link designated by the user may be displayed in the area 521.
As another display form, as shown in FIG. 8D, the entire input image may be displayed in the area 531 of the presentation screen, and the review area 532 and the priority 533 may be superimposed on the input image. In this case, you may make it display only the review area | region more than the priority selected by the user using the priority selection part 534. FIG. That is, when the user selects the priority 0.8 using the priority selection unit 534, for example, as shown in the right diagram of FIG. 8D, the review areas 535 and 536 having a priority of 0.8 or more. Only may be displayed.

As described above, the image processing apparatus 200 according to the present embodiment acquires information related to an object to be inspected (in this embodiment, a deformation candidate for a structure) extracted from an image to be processed. Here, the image processing apparatus 200 may acquire information on the deformation candidate by inputting the image to be processed and extracting the deformation candidate from the input image, or another apparatus may extract the image from the image. Information regarding the deformed candidate may be acquired.
The image processing apparatus 200 sets a plurality of partial areas in the image as review areas based on the acquired information regarding the deformation candidates. Then, the image processing apparatus 200 determines a priority for each review area (for each partial area) based on information on the deformation candidate. At that time, the image processing apparatus 200 derives a feature amount for each review region (for each partial region) based on information about the deformation candidate, and for each review region (for each partial region) based on the derived feature amount. Determine the priority.

As described above, the image processing apparatus 200 sets a plurality of partial areas in the image as review areas with reference to the deformation candidates, and prioritizes each set partial area. As a result, the image processing apparatus 200 can appropriately set a unit area suitable for the review of the deformation candidate. Therefore, by performing display control in which the image processing apparatus 200 displays an image corresponding to the review area, the inspector can review in units of areas summarized to a degree suitable for visual judgment, and an efficient review can be performed. It becomes possible.
Furthermore, the image processing apparatus 200 can perform display control to display an image corresponding to the review area according to the priority of the review area. For example, the image processing device 200 displays the display by switching the review areas in descending order of priority, arranging the review areas in the order of high priority, or displaying only the review areas having a predetermined priority or higher. Control can be performed. Thereby, the inspector can review the deformation candidates more efficiently.

Further, the image processing apparatus 200 can perform display control for displaying an image corresponding to the review area together with the priority of the review area. Thus, by displaying the priority of the review area, the inspector can review the deformation candidate while recognizing the necessity (importance) of the inspection of the review area that he / she is paying attention to. .
Furthermore, the image processing apparatus 200 can also perform display control for highlighting the deformation candidates included in the review area (deformation candidates associated with the review area). Thereby, the inspector can more appropriately review the deformation candidates.
Further, the image processing apparatus 200 sets a review area based on the reliability of the deformation candidate. For example, by setting a review area in which a candidate for deformation with low reliability is used as a reference and the candidate for deformation is located in the center, the candidate for deformation with low reliability can be easily caught by the inspector. Review is possible. Thus, a more appropriate review can be performed by setting the review area in accordance with the reliability of the deformation candidate.

The image processing apparatus 200 sets a region including the deformation candidate extracted from the image as a review region. This makes it possible to review the deformation candidate without fail. When an image to be processed is divided into predetermined small sections, the deformation candidate may straddle the small sections. Therefore, if the divided small sections are set as review areas and an image is displayed for each review area, appropriate reviews cannot be performed. As in the present embodiment, by setting the review area based on the extracted information on the deformation candidate, it is possible to avoid that the modification candidate cannot be appropriately reviewed across the review area.
Further, as a result of setting the area including the deformation candidate as the review area, the image processing apparatus 200 sets the remaining area that is not set as the review area in the image as the review area. That is, the image processing apparatus 200 sets a review area even for an area where no deformation candidate has been detected. Thereby, it is possible to cope with omission of extraction of deformation candidates by image processing, and more appropriate review is possible.

Further, the image processing apparatus 200 derives the statistical quantity of reliability of the deformation candidate associated with the review area as the feature quantity of the review area. Thereby, the priority of a review area | region can be determined according to the reliability of a deformation | transformation candidate. For example, the lower the reliability of the deformation candidate, the higher the priority, and the visual review by the inspector can be preferentially performed. In this way, it is possible to make an appropriate setting according to the need for visual inspection by the inspector.
Further, the image processing apparatus 200 may determine the weighted addition of the feature amount of the review region as the priority of the review region, or weight the difference between the feature amount of the review region and a predetermined reference value. The addition may be determined as the priority of the review area. Thereby, the priority of a review area | region can be determined appropriately.

  As described above, the image processing apparatus 200 according to the present embodiment can appropriately determine a region for efficiently reviewing an object to be inspected extracted from an image and the priority of the region. It becomes. Therefore, the image processing apparatus 200 can perform display control that makes it easy to review the object using these pieces of information. As a result, the inspector can appropriately review the validity of the object to be inspected extracted from the image.

(Second embodiment)
Next, a second embodiment of the present invention will be described.
In the first embodiment described above, the case where the review area is set based on the deformation candidate extracted from the image has been described. In the second embodiment, an example will be described in which region information in an image input by a user is also used for setting a review region.
FIG. 9 is a configuration example of the image processing system 100 according to the second embodiment. The image processing system 100 shown in FIG. 9 is the same as the configuration of FIG. 1 except that the image processing apparatus 200A of this embodiment additionally includes a region input unit 206. Accordingly, the following description will focus on the different parts.

The area input unit 206 acquires area information input by the user, and outputs the acquired area information to the area setting unit 203. Here, the region information is information indicating a predetermined region in the image, and may be a region coordinate value representing a range in the image, or grid information for dividing the entire image.
The overall processing flow executed by the image processing apparatus 200A is the same as that in the first embodiment described above. That is, the image processing apparatus 200A executes the process shown in FIG. However, the process in S3 of FIG. 3 is different from that of the first embodiment.

FIG. 10 is a flowchart showing the procedure of the area setting process executed by the area setting unit 203 in this embodiment. The process of FIG. 10 is the same as the process of FIG. 5 except that S37 and S38 are added. Therefore, the same step numbers are assigned to the parts that perform the same processing as in FIG.
In S37, the area setting unit 203 acquires area information representing a predetermined area in the image specified by the user via the area input unit 206, and selects the predetermined area specified by the user based on the acquired area information. Set as a review area. In S38, the deformation candidates included in the review area determined in S37 are registered. The method for registering deformation candidates is the same as S33 and S34.

By the area setting process of FIG. 10, the area setting unit 203 outputs review area information 420 ′ as shown in FIG. That is, in addition to the review area information 420 described in the first embodiment, review area information 426 set based on area information specified by the user can be set.
In the review area information 420 ′ shown in FIG. 11, the review areas are all the same size. However, the review area size set based on the area information specified by the user is arbitrarily specified by the user. Also good.

As described above, the image processing apparatus 200A according to the present embodiment inputs information related to a predetermined area in an image designated by the user, and sets the predetermined area as a review area based on the input information. As a result, it is possible to set a review area reflecting the area information input by the user and register deformation candidates.
The user can designate a region in the image after knowing in advance the previous review results and the distribution tendency of the deformation of the target structure. Therefore, the image processing apparatus 200A can easily and appropriately set the review area that reflects the priority area.

(Third embodiment)
In the first embodiment and the second embodiment described above, the case has been described in which the statistical amount of reliability of the deformation candidate associated with the review region is calculated as the feature amount of the review region. In the third embodiment, an example will be described in which a feature amount is calculated based on deformed distribution information acquired in advance.
FIG. 12 is a configuration example of the image processing system 100 according to the third embodiment. The image processing system 100 shown in FIG. 12 is the same as the configuration of FIG. 1 except that the image processing apparatus 200B of this embodiment additionally includes a prior distribution input unit 207. Accordingly, the following description will focus on the different parts.

The prior distribution input unit 207 acquires distribution information (prior distribution information) as prior knowledge such as the distribution information of deformation acquired after the previous review and the distribution trend of deformation of the target structure. Distribution information is output to the feature amount calculation unit 204. The prior distribution information may be input in advance to the image processing apparatus 200B by the user, or may be held in advance by the image processing apparatus 200B.
The overall processing flow executed by the image processing apparatus 200B is the same as that of the first embodiment described above. That is, the image processing device 200B executes the process shown in FIG. However, the process in S4 of FIG. 3 is different from the first embodiment.

FIG. 13 is a flowchart illustrating a procedure of region feature amount calculation processing executed by the feature amount calculation unit 204 in the present embodiment.
First, in step S44, the feature amount calculation unit 204 acquires deformed prior distribution information. As shown in FIG. 14A, the prior distribution information 450 includes an ID 451 uniquely indicating a review area, and sample numbers 452 to 456 obtained by classifying the deformations included in each review area according to the deformation width. Can be included. That is, in the present embodiment, the prior distribution information is information indicating the distribution of the deformation width in the review area.

In S <b> 45, the feature amount calculation unit 204 calculates distribution information (candidate distribution information) of deformation candidates included in the review region set by the region setting unit 203. As shown in FIG. 14B, the candidate distribution information 460 includes an ID 461 that uniquely indicates a review area, and sample numbers 462 to 466 obtained by classifying the deformation candidates included in each review area according to the deformation width. Can be included. That is, in the present embodiment, the candidate distribution information is information indicating the distribution of the widths of the deformation candidates in the review area.
In S46, the feature amount calculation unit 204 calculates the distribution divergence between the prior distribution information 450 obtained in S44 and the candidate distribution information 460 obtained in S45 as the feature amount of the review area. For example, the feature amount calculation unit 204 calculates a chi-square value as the degree of divergence between the two distribution information.
Through the region feature amount calculation processing of FIG. 13, the feature amount calculation unit 204 outputs feature amount information 430A as shown in FIG. The feature amount information 430A can include an ID 434 that uniquely indicates the review area and a distribution divergence degree 435.

As described above, the image processing apparatus 200B according to the present embodiment inputs prior distribution information that is distribution information (first distribution information) in the deformation candidate image. Further, the image processing apparatus 200B calculates candidate distribution information that is distribution information (second distribution information) in the deformation candidate image based on the information regarding the deformation candidate extracted from the image. Then, the image processing device 200B derives the distribution divergence degree between the first distribution information and the second distribution information as a feature amount of the review area.
Thereby, the feature amount of the review area reflecting the prior distribution information can be derived. Therefore, the review work can be performed with a more reliable priority compared to the previously known deformation distribution such as the previous review result and the deformation distribution tendency of the target structure.

In the present embodiment, the distribution information has been described as information indicating the distribution of the deformation width, but is not limited to the above. For example, the distribution information may be information indicating the distribution of the deformation length, or information indicating the distribution of the deformation type if the candidate extraction unit 202 can identify the deformation type. There may be. In the present embodiment, the feature amount of the review area has been described as the distribution divergence degree, but may be a combination with the statistical amount of reliability in the first embodiment.
Furthermore, the image processing apparatus 200B of the present embodiment has a configuration in which a prior distribution input unit 207 is added to the image processing apparatus 200 of the first embodiment illustrated in FIG. However, the image processing apparatus 200B may have a configuration in which the prior distribution input unit 207 is added to the image processing apparatus 200A in the second embodiment illustrated in FIG.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described.
In the first embodiment described above, the case has been described where the priority is determined by weighted addition of the feature amounts of the review area. In the fourth embodiment, an example in which priority is determined in consideration of continuity of review areas will be described.
The configuration of the image processing apparatus 200 of the present embodiment and the overall processing flow executed by the image processing apparatus 200 are the same as those in the first embodiment described above. That is, the image processing apparatus 200 has the configuration shown in FIG. 1 and executes the process shown in FIG. However, the region feature amount calculation processing in S4 of FIG. 3 and the region priority determination processing in S5 are different from the first embodiment.

FIG. 15 is a flowchart illustrating a procedure of region feature amount calculation processing executed by the feature amount calculation unit 204 according to the present embodiment.
In S47, the feature amount calculation unit 204 calculates the feature amount of the review area. Specifically, relative position information for the entire image is calculated for each review area set by the area setting unit 203. For example, the feature amount calculation unit 204 calculates, as the review region feature amount, a value obtained by normalizing the center point coordinates of the entire review region image with the image width and height.
By the region feature amount calculation process of FIG. 15, the feature amount calculation unit 204 outputs feature amount information 430B as shown in FIG. The feature amount information 430B can include an ID 436 that uniquely indicates a review area, and a center point X coordinate 437 and a center point Y coordinate 438 of the review area.

FIG. 17 is a flowchart showing a procedure of region priority determination processing executed by the priority determination unit 205 in the present embodiment.
In step S53, the priority determination unit 205 calculates the shortest path around all the review areas in the feature amount space of the review area. For example, each review area is a node on the feature amount space, and the shortest route around all the review areas is searched from the Euclidean distance of the feature amount. Specifically, the shortest path between the center points of each review area is searched using the center point coordinates that are the feature quantities of each review area. As the shortest path search solution, for example, a method of obtaining an approximate solution using a genetic algorithm can be used. As a result, the priority determination unit 205 outputs review area order information 470 as shown in FIG. The order information 470 can include an ID 471 that uniquely indicates the review area and an order 472 that follows the shortest path.

In S54, the priority determination unit 205 determines the priority based on the order 472 of the shortest path obtained in S53. For example, the priority determination unit 205 calculates the priority P _i based on the following equation (3), where M is the total number of review areas and Rank _i is the order of the shortest paths.
P _i = (M−Rank _i ) / M (3)
By the region priority determination process of FIG. 17, the priority determination unit 205 outputs priority information 440A as shown in FIG. The priority information 440A can include an ID 443 that uniquely indicates the review area and a priority 444.

As described above, the image processing apparatus 200 according to the present embodiment derives the coordinates of the center point of the review area as the feature quantity of the review area, and calculates the shortest path between the center points in the space of the derived feature quantity. Search and prioritize review areas according to the shortest path. Thereby, it becomes possible to determine the priority in consideration of the continuity of the review area. Therefore, the visibility in the form of continuously presenting review areas as shown in FIG. 8A can be enhanced.
In this embodiment, the feature amount of the review area is the relative position information with respect to the entire image of the review area. However, the feature amount in the first to third embodiments may be used, or a combination of these. It may be.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a recording medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  DESCRIPTION OF SYMBOLS 100 ... Image processing system, 200 ... Image processing apparatus, 201 ... Image input part, 202 ... Candidate extraction part, 203 ... Area setting part, 204 ... Feature-value calculation part, 205 ... Priority determination part, 300 ... Image display apparatus, 301... Region presentation unit, 302... Image display unit

Claims (19)

An acquisition means for acquiring information about an object to be inspected extracted from an image;
Setting means for setting a partial area in the image based on information about the object acquired by the acquisition means;
An image processing apparatus comprising: a determining unit that determines a priority of the partial area set by the setting unit based on information about the object acquired by the acquiring unit. The setting means includes
The image processing apparatus according to claim 1, wherein the partial area is set based on a reliability of the object. The setting means includes
The image processing apparatus according to claim 1, wherein an area including the object extracted from the image is set as the partial area. The setting means includes
4. The remaining area that is not set as the partial area in the image is set as the partial area as a result of setting the area including the object extracted from the image as the partial area. The image processing apparatus described. A derivation unit for deriving a feature amount of the partial region set by the setting unit based on information on the object acquired by the acquisition unit;
5. The image processing apparatus according to claim 1, wherein the determining unit determines the priority of the partial region based on the feature amount derived by the deriving unit. The derivation means includes
The image processing apparatus according to claim 5, wherein a statistic of reliability of the object associated with the partial region is derived as a feature amount of the partial region. The derivation means derives a plurality of feature amounts for each partial region,
The image processing apparatus according to claim 5, wherein the determining unit determines, as the priority, weighted addition of a plurality of feature amounts derived by the deriving unit. The derivation means derives a plurality of feature amounts for each partial region,
6. The determination unit according to claim 5, wherein the determination unit determines, as the priority, a weighted addition of differences between the plurality of feature amounts derived by the deriving unit and respective predetermined reference values. 6. The image processing apparatus according to 6. The derivation means includes
The image processing apparatus according to claim 5, wherein coordinates of a center point of the partial area set by the setting unit are derived as the feature amount. The determining means includes
The image processing apparatus according to claim 9, wherein a search is made for a shortest path between the center points on the feature amount space derived by the deriving unit, and the priority is determined according to the shortest path. First input means for inputting in advance first distribution information indicating the distribution of the object in the image;
Calculating means for calculating second distribution information indicating the distribution of the object in the image based on the information of the object acquired by the acquiring means;
The derivation means derives a distribution divergence degree between the first distribution information and the second distribution information respectively corresponding to the partial area as a feature amount of the partial area. The image processing apparatus according to any one of 10. A second input means for inputting information on a predetermined area in the image;
The image processing apparatus according to claim 1, wherein the setting unit sets the predetermined area as the partial area.   The image processing apparatus according to claim 1, further comprising a display control unit that displays the image corresponding to the partial area set by the setting unit. The display control means includes
The image processing apparatus according to claim 13, wherein an image corresponding to the partial area is displayed in accordance with the priority determined by the determination unit. The display control means includes
The image processing apparatus according to claim 13, wherein an image corresponding to the partial area set by the setting unit is displayed together with the priority determined by the determination unit. The display control means includes
The image processing apparatus according to claim 13, wherein the object associated with the partial area set by the setting unit is highlighted.   The image processing apparatus according to claim 1, wherein the object is a deformation of a structure. Obtaining information about the object to be inspected extracted from the image;
Setting a partial region in the image based on the acquired information about the object;
And determining a priority of the partial area set by the setting means based on the acquired information on the object.   The program for functioning a computer as each means of the image processing apparatus of any one of Claim 1 to 17.