WO2026009759A1 - Information processing device, information processing method, information processing system, program, and recording medium - Google Patents

Abstract

An information processing device according to the present invention includes an acquisition means for acquiring an image that has been captured of an evaluation object, a detection means for detecting the outline of the evaluation object and a rectangular region that circumscribes the outline of the evaluation object from the image, an index computation means that derives an area ratio for the area inside the outline of the evaluation object and the area of the rectangular region, and an assessment means that makes an assessment regarding quality evaluation of the evaluation object on the basis of the derived area ratio and outputs the assessment results.

Description

Information processing device, information processing method, information processing system, program, and recording medium

The present invention relates to an information processing device, information processing method, information processing system, program, and recording medium that support evaluation work related to objects.

Quality assessment has traditionally been based on the appearance of an object, such as its shape and size. For example, the more attractive the produce, the higher the demand. Therefore, shipping standards that determine the grade (quality) of produce are set based on appearance criteria such as color, size, shape, and the presence or absence of blemishes. Produced produce is then sorted into grades according to the shipping standards and traded. However, because the quantity of produce produced is enormous, manual sorting can be a heavy burden. Furthermore, sorting produce is sometimes done by a large number of people, and the grades of the sorted produce can be inconsistent. For this reason, in recent years, technology has been proposed that uses images of the produce to help determine the grade of the produce.

JP 2013-169156 A

Detecting the shape of an object using an image requires evaluating the entire object, which imposes a huge processing load and does not always allow for accurate quality evaluation of the object based on its shape. The present invention aims to provide an information processing device that can accurately evaluate the quality of an object based on its appearance using photographed images of the object.

The information processing device according to the present invention is characterized by having an acquisition means for acquiring an image of an object to be evaluated, a detection means for detecting the contour of the object to be evaluated and a rectangular area circumscribing the contour of the object to be evaluated from the image, an index calculation means for deriving the area ratio between the area within the contour of the object to the area of the rectangular area, and a determination means for making a determination regarding the quality evaluation of the object to be evaluated based on the derived area ratio and outputting the determination result.

The present invention makes it possible to accurately evaluate the quality of an object based on its appearance using photographed images of the object.

FIG. 1 is a diagram illustrating an example of the configuration of an information processing system. FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device. FIG. 3 is a diagram illustrating an example of a functional configuration of an information processing apparatus. FIG. 4 is a flowchart illustrating an example of processing by the information processing device. FIG. 5A is a diagram illustrating an example of photographing an object. FIG. 5B is a diagram illustrating an example of photographing an object. FIG. 6 is a diagram illustrating an example of feature information related to shape. FIG. 7A is a diagram illustrating an example of the evaluation index. FIG. 7B is a diagram illustrating an example of the evaluation index. FIG. 7C is a diagram illustrating an example of the evaluation index. FIG. 8 is a diagram illustrating an example of output of the determination result. FIG. 9 is a diagram illustrating an example of a mask image. FIG. 10 is a flowchart illustrating an example of a result change process. FIG. 11 is a diagram illustrating an example of the evaluation index. FIG. 12A is a diagram illustrating an example of the evaluation index. FIG. 12B is a diagram illustrating an example of the evaluation index. FIG. 13A is a diagram illustrating an example of output of a determination result. FIG. 13B is a diagram illustrating an example of output of the determination result.

The following describes an embodiment of the present invention with reference to the drawings.

First Embodiment
FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment of the present invention. The information processing system according to this embodiment is an information processing system that supports evaluation work on an object to be evaluated (hereinafter also referred to as an "object"), and uses a photographed image of the object to evaluate the quality of the object based on the appearance of the photographed object. The information processing system according to this embodiment includes an information processing device 100, an imaging device 110, and a display device 120. The information processing device 100 is communicatively connected to each of the imaging device 110 and the display device 120. The information processing device 100, the imaging device 110, and the display device 120 may be connected by wire or wirelessly. Furthermore, the information processing device 100, the imaging device 110, and the display device 120 may be communicatively connected to each other via a network.

The information processing device 100 acquires an image of the object (target) 130 to be evaluated from the imaging device 110, and performs a quality evaluation of the target using the acquired image. The information processing device 100 detects characteristic information about the target's appearance from the image of the target acquired from the imaging device 110, and evaluates the quality of the target based on the detected characteristic information. The characteristic information about the target's appearance includes, for example, information about the target's color, size, shape, etc. The characteristic information about the target's appearance may also include information about the presence or absence of defects on the target, the surface condition, color, luster, etc.

The imaging device 110 is an imaging device such as a camera, and captures an image of the object (target) 130 to be evaluated. The information processing system in this embodiment may be provided with any number of imaging devices 110; for example, multiple imaging devices 110 may be provided to capture images of the target 130 from multiple directions. Note that the imaging device 110 is not limited to devices with only imaging functions such as cameras, but may also be devices with imaging functions such as smartphones and tablet terminals.

The display device 120 is a display device for displaying the results of quality evaluation by the information processing device 100, etc. For example, the display device 120 is a display that displays display information on a screen, or a projector that projects display information. Furthermore, for example, the display device 120 may be smart glasses (AR glasses). Furthermore, for example, the functions of the imaging device 110 and the display device 120 may be realized using a smartphone, tablet terminal, etc., and the results of quality evaluation, etc. may be superimposed on an image of the object 130.

FIG. 2 is a diagram showing an example of the hardware configuration of the information processing device 100 in this embodiment. The information processing device 100 has a CPU 201, ROM 202, RAM 203, auxiliary storage device 204, output device 205, input device 206, and network I/F 207. The CPU 201, ROM 202, RAM 203, auxiliary storage device 204, output device 205, input device 206, and network I/F 207 are communicatively connected via a system bus 208.

CPU (Central Processing Unit) 201 is a central processing unit that controls various operations of information processing device 100. For example, CPU 201 may control the operation of the entire information processing device 100. ROM (Read Only Memory) 202 stores control programs, boot programs, etc. that can be executed by CPU 201. RAM (Random Access Memory) 203 is the main memory of CPU 201 and is used as a work area or temporary storage area for expanding various programs.

The auxiliary storage device 204 stores various data, programs, etc. The auxiliary storage device 204 is realized by a storage device that can temporarily or permanently store various data, such as a non-volatile memory such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

The output device 205 is a device that outputs various types of information and is used to present various types of information to the user. For example, the output device 205 is realized by a display device such as a display. The output device 205 may present information to the user by displaying various types of display information. As another example, the output device 205 may be realized by an audio output device that outputs sounds such as voice and electronic sounds. In this case, the output device 205 may present information to the user by outputting sounds such as voice and electronic sounds. Furthermore, the device used as the output device 205 may be changed as appropriate depending on the medium used to present information to the user.

The input device 206 is used to receive various instructions from the user. For example, the input device 206 may include an input device such as a mouse, keyboard, or touch panel. As another example, the input device 206 may include a sound collection device such as a microphone, and collect the voice spoken by the user. In this case, the collected voice may be subjected to various analytical processes such as acoustic analysis and natural language processing, and the content of this voice may be recognized as an instruction from the user. Furthermore, the device used as the input device 206 may be changed as appropriate depending on the method for recognizing instructions from the user. Furthermore, multiple types of devices may be used as the input device 206.

Network I/F 207 is used for communication with external devices over a network. The device used as network I/F 207 may be changed as appropriate depending on the type of communication path and the communication method used.

The CPU 201 loads a program stored in ROM 202 or auxiliary storage device 204 into RAM 203 and executes the program, thereby realizing the functions and processes of the information processing device described below. The program for the information processing device 100 may be provided to the information processing device 100 by a recording medium such as a CD-ROM, or may be downloaded via a network, for example. When the program for the information processing device 100 is provided by a recording medium, the program recorded on the recording medium is installed in the auxiliary storage device 204 by inserting the recording medium into a specified drive device.

The configuration shown in FIG. 2 is merely an example and does not necessarily limit the hardware configuration of the information processing device 100 in this embodiment. As a specific example, some components such as the output device 205 and input device 206 may not be included. As another example, components may be added as appropriate depending on the functions realized by the information processing device 100.

FIG. 3 is a diagram showing an example of the functional configuration of the information processing device 100 in this embodiment. The information processing device 100 has an acquisition unit 301, an evaluation processing unit 302, a control unit 306, an input/output control unit 307, and a storage unit 308.

The acquisition unit 301 acquires an image of the object (target) to be evaluated from the imaging device 110. In addition to the image of the target, the acquisition unit 301 may also acquire point cloud data about the target obtained by a distance measurement sensor such as LiDAR, or other information about the target obtained by a weighing scale or saccharometer. For example, by acquiring point cloud data about the target, it is possible to obtain information about the unevenness of the target's surface.

The evaluation processing unit 302 evaluates the quality of the object based on the images and information acquired by the acquisition unit 301. For example, the evaluation processing unit 302 detects feature information about the object from the image of the object acquired by the acquisition unit 301, and evaluates the quality of the object based on the detected feature information. The evaluation processing unit 302 has a detection unit 303, an index calculation unit 304, and a determination unit 305.

The detection unit 303 detects feature information about the object from the images, information, etc. acquired by the acquisition unit 301. For example, the detection unit 303 uses a trained model obtained in advance through machine learning to detect feature information about the appearance of the object, such as its color, size, and shape, from images of the object acquired by the acquisition unit 301. Furthermore, for example, the detection unit 303 may detect information such as the presence or absence of flaws in the object and the condition of its surface as feature information from the images, information, etc. acquired by the acquisition unit 301.

The index calculation unit 304 calculates an evaluation index used to evaluate the quality of the object based on the feature information about the object detected by the detection unit 303. The index calculation unit 304 calculates, for example, an evaluation index for evaluating the degree of deformation or bending of the object based on the feature information detected by the detection unit 303.

The determination unit 305 makes a determination regarding the quality evaluation of the object based on the characteristic information detected by the detection unit 303 and the evaluation index obtained by the index calculation unit 304, and outputs the determination result. The determination unit 305, for example, compares the detected characteristic information or the obtained evaluation index with a set threshold value to make a determination regarding the quality of the object. For example, if the object is an agricultural product, the determination unit 305 determines the grade in terms of shipping standards, etc.

The control unit 306 is responsible for controlling each component of the information processing device 100. The input/output control unit 307 performs various processes related to presenting various information to the user via the display device 120, etc., and accepting information input (e.g., instructions, etc.) from the user via an input device, etc. For example, the input/output control unit 307 may perform processes related to presenting a UI (User Interface) and accepting input via the UI. This enables the information processing device 100 to recognize instructions from the user and present the results of processing in accordance with those instructions to the user.

The storage unit 308 stores various data and the like used when performing processing in the information processing device 100. The storage unit 308 stores, for example, a trained model that has undergone machine learning and is used to infer feature information performed by the detection unit 303, and thresholds used for quality evaluation performed by the determination unit 305.

FIG. 4 is a flowchart showing an example of processing by the information processing device 100 in this embodiment. Note that the following describes an example of processing by the information processing device 100, using as an example a case where the object (target) to be evaluated is a sweet potato, an agricultural product, and the grade is determined.

In step S401, the acquisition unit 301 acquires an image of a sweet potato, which is the object (target) to be evaluated, from the imaging device 110. In addition to the image of the sweet potato, which is the target, in step S401, the acquisition unit 301 may also acquire point cloud data indicating the surface of the sweet potato obtained by a distance sensor or the like, or other information obtained by a weighing scale, saccharometer, or the like. In this embodiment, the image of the sweet potato, which is the target, is captured as shown in Figures 5A and 5B.

Figures 5A and 5B are diagrams illustrating an example of photographing a sweet potato, which is an object of interest. Figure 5A shows a state from above of a platform 502 on which the sweet potato 501, which is an object of interest, is placed. Figure 5B shows a cross section indicated by line I-I in Figure 5A. In Figures 5A and 5B, the sweet potato 501 is placed on a platform 502 made of a transparent material. The platform 502 is formed in a V-shape, and when the sweet potato 501 is placed on it, it can be stably stationary at the bottom of the platform 502. In the example shown in Figures 5A and 5B, three imaging devices 503, 504, and 505 capable of photographing the sweet potato 501 placed on the platform 502 are arranged, and one sweet potato 501 is photographed from three different directions at the same time. The three imaging devices 503, 504, and 505 are preferably positioned on the same plane of the cross section of the table 502, offset by approximately 120 degrees, so that the shape of the target object (sweet potato 501) can be detected with high accuracy. By positioning the imaging devices 503, 504, and 505 in this manner, it is possible to capture the entire target object, sweet potato 501, including its back and sides. Furthermore, imaging devices 506 and 507 may be further positioned on the front and back sides of the table 502 (in the normal direction of the plane on which the imaging devices 503-505 are positioned) so that the front and back surfaces of the target object, sweet potato 501, can be captured.

In addition to the examples shown in Figures 5A and 5B, for example, a track such as a rail may be installed in a circular or spherical shape surrounding the object, and the imaging device may be moved along the track to capture images of the object from a desired direction using a single imaging device. Furthermore, by replacing some imaging devices with components such as mirrors that capture images of the object and arranging them so that they are within the field of view of other imaging devices, the number of imaging devices may be reduced without reducing the number of directions from which the object can be captured. Images from multiple directions may also be captured by photographing the object by rolling it down a slope or by the user holding it directly in their hand and rotating it, and the side that was photographed may also be stored.

In step S402, the detection unit 303 of the evaluation processing unit 302 detects feature information about the object from the image and information acquired in step S401. For example, the detection unit 303 uses a trained model that has undergone machine learning to detect feature information about the appearance of the sweet potato, which is the object, from the image acquired in step S401 of the sweet potato.

Figure 6 is a diagram illustrating an example of feature information regarding the external shape of a sweet potato detected by the detection unit 303. The detection unit 303 uses a trained model that has undergone machine learning to detect the contour 601 of the object (sweet potato) from an image 600 of the object (sweet potato). The detection unit 303 also derives a rectangular area 602 that circumscribes (includes) the contour 601 of the object (sweet potato) based on the detected contour 601 of the object (sweet potato). Hereinafter, the rectangular area 602 that circumscribes the contour 601 of the object is also referred to as the "circumscribed rectangle." Note that the circumscribed rectangle 602 is the rectangular area with the smallest area circumscribing the contour 601 of the object (sweet potato). The detection unit 303 also derives a center line 603 of the contour 601 based on the detected contour 601 of the object (sweet potato). The detection unit 303 derives the center line 603, for example, by taking the midpoint of two intersections between the contour 601 and a line parallel to the short side of the circumscribing rectangle 602 as the center of the contour. In this way, the detection unit 303 detects the contour 601, the circumscribing rectangle 602, and the contour center line 603 as feature information related to the external shape for each direction in which the sweet potato, which is the target object, is photographed.

The detection unit 303 may also detect information about the appearance of the target object, such as blemishes, diseases, wrinkles, and discoloration, as feature information from the image of the target object, the sweet potato, acquired in step S401.

In step S403, the index calculation unit 304 of the evaluation processing unit 302 derives an evaluation index to be used in evaluating the quality of the object based on the feature information about the object detected in step S402. In the first embodiment, the index calculation unit 304 calculates, for example, the area ratio between the outline of the object and a circumscribing rectangle for each direction in which the object (sweet potato) was photographed based on the feature information detected in step S402 as an evaluation index for evaluating the degree of deformation of the object (sweet potato). The index calculation unit 304 determines, for example, the ratio between the number of pixels within the outline of the object and the number of pixels within the circumscribing rectangle as the area ratio between the outline of the object and the circumscribing rectangle. The evaluation index calculated in step S403 in this first embodiment will be described with reference to Figures 7A to 7C.

7A to 7C are diagrams illustrating examples of evaluation indices calculated in step S403 in the first embodiment. As shown in FIG. 7A, when the difference between the area 701 within the object's outline and the area 702 of the circumscribing rectangle is small and the area ratio is close to 1, the difference in shape between the object's outline and the circumscribing rectangle is small, and the degree of deformation of the object is small. Note that in FIG. 7A, 713 is the center line of the object's outline. On the other hand, as shown in FIG. 7B, when the center line 713 of the object's outline is significantly curved and the degree of deformation of the object is large, the difference between the area 711 within the object's outline and the area 712 of the circumscribing rectangle becomes large, and the area ratio becomes a value significantly different from 1. Furthermore, as shown in FIG. 7C, when the center line 723 of the object's outline is slightly curved but the degree of deformation of the object is large, the difference between the area 721 within the object's outline and the area 722 of the circumscribing rectangle becomes large, and the area ratio becomes a value significantly different from 1. In this way, the area ratio between the outline of the object and the circumscribing rectangle is calculated as an evaluation index, and the degree of deviation of the area ratio from 1 can be evaluated to assess the degree of deformation of the object.

Furthermore, the index calculation unit 304 may derive, as an evaluation index for each flaw or disease in the object, the area ratio between the area of the object estimated to be flawed or diseased and the total area of the object based on the feature information detected by the detection unit 303.

In step S404, the judgment unit 305 of the evaluation processing unit 302 judges the grade of the target sweet potato based on the evaluation index derived in step S403 and the feature information detected in step S402. The judgment unit 305 evaluates the degree of deformation of the target sweet potato and judges the grade, for example, by determining whether the area ratio between the target outline and circumscribing rectangle derived as the evaluation index in step S403 is within a predetermined range. At this time, the judgment unit 305 judges the grade of the target sweet potato based on the worst value (area ratio farthest from 1) among the area ratios between the target outline and circumscribing rectangle derived for each direction in which the target sweet potato was photographed.

Furthermore, for example, the determination unit 305 may determine the grade of the sweet potato object based on whether the area ratio of the area estimated to be flawed or diseased in the object, derived as the evaluation index in step S403, to the total area of the object (area estimated to be flawed or diseased / total area of the object) exceeds a set threshold. Note that the determination unit 305 may determine that an object estimated to have a flaw or disease is of a grade that corresponds to processing or disposal, regardless of the area ratio. Furthermore, for example, the grade of the sweet potato object may be determined based on feature information regarding wrinkles or discoloration on the object detected in step S402, or, for example, a trained model may be used to classify images of the sweet potato object and determine the grade.

In step S405, the determination unit 305 outputs the determination result from step S404. The determination result output from the determination unit 305 is displayed on the display device 120, for example, via the input/output control unit 307. Figure 8 is a diagram illustrating an example of the output of the determination result, and displays information 804 related to the determination result, such as grade, length, and thickness, along with the outline 801, circumscribed rectangle 802, and center line 803 of the outline of the object detected as feature information in step S402. For example, the color of the displayed outline 801 of the object may be changed depending on the grade of the determination result. Note that, based on the determination result output from the determination unit 305, the sweet potatoes, which are the objects, may be sorted by grade using a sorting device (not shown) that can switch transportation routes, etc.

Here, to enable confirmation of the judgment results in real time, the judgment results output from the judgment unit 305 may be displayed using projection mapping. In this case, if the display position of the judgment results is misaligned with the position of the sweet potato (the target object), the image of the sweet potato projected and displayed misaligned will be captured by the imaging device, and repeated capture will result in multiple images being displayed that are misaligned and overlapping. This can be avoided, for example, by displaying an image as shown in Figure 9, in which a black mask is applied to all but the object's outline 801, circumscribing rectangle 802, outline center line 803, and information about the judgment result 804 shown in Figure 8. Note that even if the entire image other than the object's outline 801, circumscribing rectangle 802, outline center line 803, and information about the judgment result 804 shown in Figure 8 is not black-masked, the same effect can be achieved by applying black masking to at least the area within the circumscribing rectangle 802 (excluding the object's outline 801, circumscribing rectangle 802, and outline center line 803). Additionally, multiple overlapping images can be avoided by displaying and capturing a projection image that includes so-called AR markers, calculating the amount of correction based on the position of the AR markers in the captured image, and correcting the projection image.

Furthermore, when displaying the judgment results using projection mapping, the display of the circumscribing rectangle 802 of the object may be changed depending on the judgment result, as shown in FIG. 13A. FIG. 13A is a diagram illustrating an example of the output of the judgment result. Note that while FIG. 13A illustrates the circumscribing rectangle of the object and information related to the judgment result, the object's outline and the center line of the outline may also be displayed, as in the example described above. For example, if the object is judged to be a first-class product (object 1301 in the illustrated example), the circumscribing rectangle 802 may be displayed in green (circumscribing rectangle 1311 shown by a solid line in the figure); if the object is judged to be a second-class product (object 1302 in the illustrated example), the circumscribing rectangle 802 may be displayed in yellow (circumscribing rectangle 1312 shown by a dashed line in the figure); and if the object is judged to be a B-class product (object 1303 in the illustrated example), the circumscribing rectangle 802 may be displayed in red (circumscribing rectangle 1313 shown by a dashed line in the figure). Although the embodiment in which the color of the circumscribing rectangle 802 is changed according to the judgment result has been described above, the present invention is not limited to this. For example, the color of the circumscribing rectangle may be changed according to the size, length, thickness, etc. of the object. For example, if the judgment result is a first-class product and the length is equal to or greater than a predetermined length, the color of the circumscribing rectangle 802 may be displayed in green. If the judgment result is a first-class product and the length is within a predetermined length range, the color of the circumscribing rectangle 802 may be displayed in yellow. If the judgment result is a second-class product or the length is less than the predetermined length, the color of the circumscribing rectangle 802 may be displayed in light blue. If the judgment result is a third-class product, the color of the circumscribing rectangle 802 may be displayed in pink. In this way, by changing and displaying the color of the circumscribing rectangle 802 according to the judgment result, the size, length, thickness, etc. of the object, or a combination thereof, the user may be able to intuitively recognize the grade of the object, thereby improving the efficiency of sorting the objects by grade. Furthermore, as shown in FIG. 13B, the circumscribing rectangle 802 of the object may be displayed in a different color, and a legend 1321 of the color of the circumscribing rectangle 802 may be displayed using projection mapping.

Note that if the user determines that the judgment result by the judgment unit 305 is inappropriate, the information processing device 100 may accept input from the user and change the judgment result. Figure 10 is a flowchart showing an example of the judgment result change process in the information processing device 100. By performing the judgment result change process described below, it becomes possible for the user to automatically adjust the threshold value in the quality (grade) evaluation simply by inputting a change to the result, without having to adjust detailed parameters, etc.

In step S1001, the control unit 306 determines whether the user has input a change to the results. If the control unit 306 determines that the user has input a change to the results (YES), the process of step S1002 is executed. On the other hand, if the control unit 306 determines that the user has not input a change to the results (NO), the change process shown in FIG. 10 is terminated.

In step S1002, the control unit 306 changes the determination result regarding the object in accordance with the user's input for changing the result.
In step S1003, the control unit 306 adjusts and updates the threshold value in the quality (grade) evaluation based on the determination result changed by the user, and ends the change processing shown in FIG.

According to the first embodiment, the information processing device 100 derives the area ratio between the area within the contour of the object and the area of the circumscribing rectangle as an evaluation index for evaluating the degree of deformation of the object, based on feature information detected from a photographed image of the object. The information processing device 100 then evaluates the degree of deformation of the object by determining whether the area ratio between the area within the contour of the object and the area of the circumscribing rectangle, derived as the evaluation index, is within a predetermined range, thereby making a determination regarding the quality evaluation of the object. This makes it possible to accurately evaluate the quality of an object based on its appearance using a photographed image of the object, using processing with a relatively low processing load.

Second Embodiment
In the first embodiment, in step S403 of Fig. 4, the index calculation unit 304 derives an evaluation index for evaluating the degree of deformation of the sweet potato, which is the object. In the second embodiment, the index calculation unit 304 derives an evaluation index for evaluating the curvature of the sweet potato, which is the object, instead of an evaluation index for evaluating the degree of deformation of the sweet potato, which is the object. The second embodiment is the same as the first embodiment described above except that the index calculation unit 304 derives an evaluation index for evaluating the curvature of the sweet potato, which is the object. Therefore, a description of these aspects will be omitted, and the following describes the derivation of the evaluation index in step S403 of Fig. 4.

In the second embodiment, in step S403 of FIG. 4, the index calculation unit 304 derives an evaluation index for evaluating the curvature of the sweet potato, which is the object, for each direction in which the sweet potato, which is the object, is photographed, instead of an evaluation index for evaluating the degree of deformation of the sweet potato, which is the object. The evaluation index calculated in step S403 in the second embodiment will be described with reference to FIG. 11.

Figure 11 is a diagram illustrating an example of an evaluation index for evaluating the curvature of the sweet potato, which is the object, calculated in step S403 in the second embodiment. In Figure 11, 1101 to 1103 are feature information detected in step S402, with 1101 being the contour of the object, 1102 being the circumscribing rectangle, and 1103 being the center line of the contour. Based on the feature information detected in step S402, the index calculation unit 304 calculates, as an evaluation index for evaluating the curvature of the sweet potato, the area of the region enclosed by the center line 1103 of the object's contour and the straight line 1104 connecting both ends of the contour center line 1103 (points 1103A and 1103B). If the curvature of the object is small, the area of the region enclosed by the center line 1103 of the object's contour and the straight line 1104 connecting both ends is small. On the other hand, if the curvature of the object is large, the area of the region enclosed by the center line 1103 of the object's outline and the straight line 1104 connecting both ends of the line will be large. Therefore, the area of the region enclosed by the center line 1103 of the object's outline and the straight line 1104 connecting both ends of the line will be calculated as an evaluation index, and the curvature of the object can be evaluated by evaluating its size.

In the second embodiment, in step S404 following step S403, the determination unit 305 evaluates the curvature of the sweet potato object and determines its grade based on whether the area of the region enclosed by the center line 1103 of the object's outline, derived as the evaluation index, and the straight lines 1104 connecting both ends of the line, exceeds a set threshold value. At this time, the determination unit 305 determines the grade of the sweet potato object based on the maximum value of the area of the region enclosed by the center line 1103 of the object's outline, derived for each direction in which the sweet potato object was photographed, and the straight lines 1104 connecting both ends of the line.

According to the second embodiment, the information processing device 100 derives the area of the region enclosed by the center line of the object's outline and the lines connecting both ends of the outline as an evaluation index for evaluating the curvature of the object, based on feature information detected from a photographed image of the object. The information processing device 100 then evaluates the curvature of the sweet potato object, which is the object, depending on whether the area of the region enclosed by the center line of the object's outline and the lines connecting both ends of the outline, derived as the evaluation index, exceeds a set threshold, and makes a judgment regarding the quality evaluation of the object. This makes it possible to directly evaluate the curvature of the object and accurately evaluate the quality of the object based on its appearance using a photographed image of the object.

In the above description, the area of the region enclosed by the center line 1103 of the object's contour detected as feature information and the straight line 1104 connecting both ends of the center line 1103 is calculated as the evaluation index for evaluating the curvature of the sweet potato object, but this is not limited to this. For example, the index calculation unit 304 may calculate the area of the region enclosed by the center line 1203 of the object's contour 1201 and the straight line 1204 that divides the circumscribing rectangle 1202 into two in the longitudinal direction, as shown in FIG. 12A, as the evaluation index for evaluating the curvature of the sweet potato object. Also, for example, as shown in FIG. 12B, the circumscribing rectangle 1212 for the object's contour 1211 is divided into multiple regions, approximately 20 to 30 in number. The index calculation unit 304 may then calculate the maximum value of the angle 1215 between the center line 1213 of the contour 1211 of the object and a line 1214 parallel to the long side of the circumscribing rectangle 1212 as an evaluation index for evaluating the curvature of the sweet potato object, in which case local curvature can be detected and evaluated.

It is also possible to combine the first and second embodiments described above. That is, the index calculation unit 304 may calculate both an evaluation index for evaluating the degree of deformation of the object and an evaluation index for evaluating the curvature of the object based on the feature information detected by the detection unit 303, and use these evaluation indexes to evaluate the quality of the object. By using two different evaluation indexes in this way to evaluate the quality of the object, it is possible to reduce the occurrence of errors in the quality evaluation.

Furthermore, in each of the above-described embodiments, an example has been described in which the object (target) to be evaluated is a sweet potato, an agricultural product, and the grade is determined. However, this is not limited to this, and the method can also be applied to the quality evaluation of other agricultural products, industrial products, daily necessities, etc. For other agricultural products, it can be applied to the quality evaluation of root vegetables and vegetables such as eggplant, cucumber, carrot, and burdock. Furthermore, for example, it can be applied to the quality evaluation of the shape of other items, such as handicrafts and food products such as bread, which are determined based on appearance.

<Other embodiments>
The present invention can also be realized by executing the following process. That is, software (programs) that realize the functions of the above-described embodiments are supplied to a system or device via a network or various recording media. The computer (or CPU, MPU, etc.) of the system or device then reads and executes the programs. Furthermore, computer-readable recording media on which the programs are recorded and computer program products such as the programs can also be applied as embodiments of the present invention. Examples of recording media that can be used include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs.

It should be noted that the above embodiments are merely examples of specific embodiments of the present invention, and should not be construed as limiting the technical scope of the present invention. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

The present invention makes it possible to accurately evaluate the quality of an object based on its appearance using photographed images of the object.

Claims (21)

an acquisition means for acquiring an image of an object to be evaluated;
a detection means for detecting a contour of the evaluation object and a rectangular area circumscribing the contour of the evaluation object from the image;
an index calculation means for deriving an area ratio between the area within the contour of the evaluation object and the area of the rectangular region;
An information processing apparatus comprising: a determining means for determining a quality evaluation of the evaluation object based on the derived area ratio, and outputting a result of the determination. The determination means
determining a degree of deformation of the evaluation object based on the derived area ratio;
The information processing apparatus according to claim 1 , wherein a quality evaluation of the evaluation object is determined based on the determined degree of deformation of the evaluation object. The information processing device described in claim 2, characterized in that the determination means determines the degree of deformation of the evaluation object depending on whether the derived area ratio is within a predetermined range. The detection means further detects a center line of a contour of the evaluation object,
The index calculation means further derives a first area of a region surrounded by a center line of the contour of the evaluation object and a straight line connecting both ends of the center line,
The information processing device according to any one of claims 1 to 3, characterized in that the judgment means makes a judgment regarding the quality evaluation of the object to be evaluated based on the derived area ratio and the derived first area. The detection means further detects a center line of a contour of the evaluation object,
The index calculation means further derives a first area of a region surrounded by a center line of the contour of the evaluation object and a line that divides the rectangular region into two in a longitudinal direction,
The information processing device according to any one of claims 1 to 3, characterized in that the judgment means makes a judgment regarding the quality evaluation of the object to be evaluated based on the derived area ratio and the derived first area. The determination means
determining a degree of deformation of the evaluation object based on the derived area ratio;
determining a bending of the evaluation object based on the derived first area;
6. The information processing apparatus according to claim 4, wherein a quality evaluation of the evaluation object is determined based on the determined degree of deformation and curvature of the evaluation object. The detection means further detects a center line of a contour of the evaluation object,
the index calculation means divides the rectangular area into a plurality of areas, and derives, for each divided area, an angle formed between a straight line parallel to a long side of the rectangular area and a center line of a contour of the evaluation object;
An information processing device as described in any one of claims 1 to 3, characterized in that the judgment means makes a judgment regarding the quality evaluation of the object to be evaluated based on the derived area ratio and the angle between a straight line parallel to the long side of the derived rectangular area and the center line of the contour of the object to be evaluated. The information processing device described in any one of claims 1 to 7 further comprises a change means for accepting user input regarding a change to the judgment result, and changing the judgment result regarding the evaluation object based on the input, thereby changing the threshold value used in the quality evaluation by the judgment means. The acquisition means acquires images of the evaluation object taken from a plurality of directions,
9. The information processing apparatus according to claim 1, wherein the index calculation means derives the area ratio for each of the plurality of directions. The information processing device described in claim 9, characterized in that the determination means makes a determination regarding the quality evaluation of the evaluation object based on the worst value of the area ratios derived for each of the multiple directions. The information processing device described in any one of claims 1 to 10, wherein the evaluation object is an agricultural product. The detection means further detects information about at least one of defects, diseases, wrinkles, and discoloration in the crop from the image;
12. The information processing apparatus according to claim 11, wherein the determining means further determines the quality of the agricultural product based on information about at least one of blemishes, diseases, wrinkles, and discoloration detected in the agricultural product. An information processing device according to any one of claims 1 to 12;
an imaging means for capturing an image of the evaluation object;
and a display unit for displaying the determination result. The information processing system described in claim 13, characterized in that it has multiple imaging means and captures images of the object to be evaluated from multiple directions. The information processing system described in claim 13 or 14, characterized in that the display means displays the determination results using projection mapping. The information processing system described in claim 15, characterized in that the display means displays the determination result by masking an area other than the area showing the determination result. The information processing system described in claim 15 or 16, characterized in that the display means displays a rectangle corresponding to the object based on the rectangular area detected by the detection means, and changes the color of the rectangle depending on the judgment result of the quality evaluation of the object. The information processing system described in claim 17, characterized in that the display means changes the color of the rectangle depending on at least one of the size, length, and thickness of the object in addition to the determination result. An information processing method executed by an information processing device,
an acquisition step of acquiring an image of an object to be evaluated;
a detection step of detecting a contour of the evaluation object and a rectangular area circumscribing the contour of the evaluation object from the image;
an index calculation step of deriving an area ratio between the area within the contour of the evaluation object and the area of the rectangular region;
and a determination step of making a determination regarding the quality evaluation of the evaluation object based on the derived area ratio and outputting a determination result. The computer of the information processing device
an acquisition step of acquiring an image of an object to be evaluated;
a detection step of detecting a contour of the evaluation object and a rectangular area circumscribing the contour of the evaluation object from the image;
an index calculation step of deriving an area ratio between an area within the contour of the evaluation object and an area of the rectangular region;
and a determination step of making a determination regarding the quality evaluation of the evaluation object based on the derived area ratio and outputting the determination result. The computer of the information processing device
an acquisition step of acquiring an image of an object to be evaluated;
a detection step of detecting a contour of the evaluation object and a rectangular area circumscribing the contour of the evaluation object from the image;
an index calculation step of deriving an area ratio between an area within the contour of the evaluation object and an area of the rectangular region;
A computer-readable recording medium having a program recorded thereon that executes a judgment step of making a judgment regarding the quality evaluation of the object to be evaluated based on the derived area ratio and outputting the judgment result.