JP2023013867A - Information processing system and information processing method - Google Patents

Abstract

An object of the present invention is to suppress variation in measurement of an object. An information processing system includes a control device that controls processing executed on one or more objects, and an imaging process that images the one or more objects multiple times at predetermined time intervals. an image processing device for obtaining a plurality of measurement values relating to the one or more objects by performing processing on a plurality of image data obtained by the above; identification information relating to the one or more objects; a storage device for storing values and imaging dates and times for each imaging of the one or more objects; analysis of the plurality of measured values based on the identification information, the plurality of measured values, and the imaging dates and times; and a display device for displaying variation values relating to the plurality of measured values calculated by executing the process. [Selection drawing] Fig. 1

Description

The present invention relates to an information processing system and an information processing method.

Image data and measurement data acquired by an image processing system and various sensor data acquired by an equipment control system are synchronized and stored in a storage on a network (see, for example, Patent Document 1).

Japanese Patent Application Laid-Open No. 2021-22296

In the production line, parts, semi-finished products, etc. (hereinafter collectively referred to as "workpieces"), which are the objects to be processed, are measured by an image processing system, and based on the measurement data, the work is judged to be good or defective. making judgments. Depending on the parameter settings for measuring the workpiece, the mounting position of various sensors in the facility control system, and the environment, variations in workpiece measurement may occur, and it may not be possible to correctly determine whether the workpiece is good or bad.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique capable of suppressing variation in measurement of an object.

An information processing system according to one aspect of the present invention includes a control device that controls processing to be performed on one or more objects, and an imaging system that images the one or more objects a plurality of times at predetermined time intervals. an image processing device that performs processing on a plurality of image data obtained by performing the processing to acquire a plurality of measurement values regarding the one or more objects; identification information regarding the one or more objects; a storage device for storing a plurality of measured values and an imaging date and time for each imaging of the one or more objects; and the plurality of measurements based on the identification information, the plurality of measured values, and the imaging date and time. and a display device that displays variation values related to the plurality of measured values calculated by performing analysis processing on the values.

The user can recognize whether or not the variation value is greater than or equal to the threshold value by visually recognizing the variation value for the plurality of measured values. If the fluctuation value is greater than or equal to the threshold, the variation in measurement of the object can be reduced by changing the parameters for measuring the object or by changing the mounting positions of various sensors in the equipment that processes the object. can be suppressed.

In the above information processing system, the processing on the plurality of image data executed by the image processing device includes a plurality of measurement processing corresponding to a plurality of measurement items, and the image processing device performs the measurement on the plurality of measurement items. The plurality of corresponding measured values may be obtained, and the display device may display the variation value for each of the plurality of measured items.

The information processing system may include a plurality of facilities that perform processing on the one or more objects, and the content and number of the plurality of measurement items may differ for each of the plurality of facilities.

In the above information processing system, the display device displays a plurality of the variation values,
A method of displaying a variation value equal to or greater than a threshold among the plurality of variation values may be different from a method of displaying a variation value less than the threshold among the plurality of variation values.

In the above information processing system, the display device may arrange and display the plurality of variation values in descending order of the plurality of variation values.

In the above information processing system, the imaging process includes imaging the one or more objects a plurality of times at the predetermined time intervals, and after a predetermined time has passed, the one or more objects are photographed for the predetermined time. It may also include a process of imaging multiple times at intervals.

In the above information processing system, the storage device may store a user's comment on the variation value, and the display device may display the variation value and the comment.

In the above information processing system, the storage device may store a user's comment on the variation value, and the display device may display the variation value and the comment for each of the plurality of measurement items.

In the above information processing system, the control device calculates the variation value by executing the analysis process on the plurality of measured values based on the identification information, the plurality of measured values, and the imaging date and time. may

In the above information processing system, the image processing device calculates the variation value by performing the analysis process on the plurality of measured values based on the identification information, the plurality of measured values, and the imaging date and time. You may

The information processing system includes an information processing device that calculates the variation value by executing the analysis process on the plurality of measured values based on the identification information, the plurality of measured values, and the imaging date and time. may

An information processing method according to one aspect of the present invention is an information processing method in a system including a control device, an image processing device, a storage device, and a display device, wherein the control device controls one or more objects and the image processing device processes a plurality of image data obtained by performing an imaging process in which the one or more objects are imaged a plurality of times at predetermined time intervals. obtaining a plurality of measured values for the one or more objects; and the storage device stores identification information for the one or more objects, the plurality of measured values, and the one or more and the display device executes analysis processing for the plurality of measured values based on the identification information, the plurality of measured values, and the imaging date and time. and displaying variation values relating to the plurality of measured values calculated by performing the calculation.

ADVANTAGE OF THE INVENTION According to this invention, the technique which can suppress the dispersion|variation in the measurement of a target object can be provided.

1 is a block diagram showing the overall configuration of an information processing system according to this embodiment; FIG. It is a figure which shows the structural example of a facility control system. It is a figure which shows the structural example of an image processing system. FIG. 2 is a diagram showing a configuration example of a storage; FIG. It is a figure which shows the structural example of an information processing apparatus. 4 is a diagram showing an example structure of data stored in a storage by each of the facility control system and the image processing system; FIG. 4 is a flow chart showing operation procedures related to notification of imaging timing and generation of sensor data in the equipment control system. 4 is a flowchart showing an operation procedure related to transmission of sensor data in the equipment control system; 4 is a flow chart showing an operation procedure related to transmission of image data and measurement data in the image processing system; FIG. 4 is a diagram for explaining data that can be referenced from a storage by an information processing device; 4 is a flow chart showing an operation procedure related to display of measurement items in the information processing apparatus; FIG. 10 is a diagram showing an example of a case in which a plurality of measurement items are rearranged in descending order of variation values; FIG. 4 is a diagram for explaining data that can be referenced from a storage by an information processing device;

<Application example>
First, an example of a scene to which the present invention is applied will be described with reference to FIG. FIG. 1 is a block diagram showing the overall configuration of an information processing system (information management system) 1 according to this embodiment. The information processing system 1 includes a facility control system (control device) 10 , an image processing system (image processing device) 20 , and a storage (storage device) 30 . The facility control system (control device) 10 controls processing performed by the facility 12 such as a robot or a manufacturing device on one or a plurality of workpieces (objects to be processed) 82 continuously conveyed by the conveyor 80 . The image processing system 20 is arranged in association with the equipment control system 10, and performs processing on a plurality of image data obtained by executing an imaging process of capturing images of the work 82 a plurality of times at predetermined time intervals to obtain a plurality of image data related to the work 82. Get the measurement value of The storage 30 is communicably connected to each of the facility control system 10 and the image processing system 20 via the network 50, and stores identification information about the work 82, a plurality of measured values, and the date and time of each image of the work 82. Remember. The information processing system 1 may include multiple facility control systems 10 , multiple image processing devices 20 , and multiple facilities 12 . A single facility control system 10 may control each process performed by multiple facilities 12 . For example, the processing shown in FIG. 1 is performed during a production preparation stage, a temporary suspension of operation of the production line, or the like.

The facility control system 10 notifies the image processing system 20 of the imaging timing of the work 82, transmits identification information including the individual identification information of the work 82, and performs processing on the work 82 in response to the notified imaging timing. The first data (sensor data) in which the information about the work 82 and the individual identification information of the work 82 are linked is stored (recorded) in the storage 30 . The individual identification information of the work 82 is information for identifying the individual of the work 82 . The image processing system 20 captures an image of the work 82 with the camera 70 in response to the image capturing timing notified from the equipment control system 10, and performs measurement processing on one or a plurality of image data obtained by the image capture to obtain the work 82. to obtain the measurement data of Then, the image processing system 20 transfers the second data (image data, measurement data, imaging date and time) linking the measurement data of the workpiece 82, the individual identification information of the workpiece 82, and the imaging date and time of the workpiece 82 to the storage 30. Memorize (record). The imaging date and time of the work 82 is the date and time when the work 82 was imaged by the camera 70 corresponding to the imaging timing notified from the facility control system 10 . The date and time has information of year, month, day, hour, minute, and second. Seconds in the date and time include mm seconds.

The identification information includes at least individual identification information of the workpiece 82 . The identification information includes at least one of information specifying a process (process information) for processing the workpiece 82, information specifying the facility 12 that processes the workpiece 82 (equipment information), imaging timing information, and the like. may contain. The imaging timing information is typically information of the time when the imaging timing is notified.

The information about the processing on the workpiece 82 is, for example, a physical quantity or character string detectable by a sensor regarding the processing performed by the equipment 12, examples of which include torque, flow rate, pressure, length, and the like. Further, the measurement data of the work 82 includes at least one of measurement values and character strings obtained by performing image recognition processing based on image data, for example. , distance, color, information indicating the presence or absence of defects, and the like.

The information processing device 40 performs analysis processing such as measurement system analysis on a plurality of measured values based on the identification information, the measurement data including the plurality of measured values, and the imaging date and time of the workpiece 82. A variation value is calculated for a plurality of measurements. The information processing device 40 calculates a variation value for each of a plurality of measurement items for the workpiece 82 . The measurement items include, for example, the length (height, width) and area of the workpiece 82, the length (height, width) of members of the workpiece 82, the area, the distance between members, and the like. The information processing device 40 may calculate variation values for all measurement items for the workpiece 82 by executing analysis processing on a plurality of measurement values. When the information processing system 1 includes a plurality of facilities 12 , the information processing device 40 calculates variation values for a plurality of measurement items for the work 82 for each facility 12 . The contents and number of the plurality of measurement items for the work 82 are different for each of the plurality of facilities 12 . The display device of the information processing device 40 displays variation values for a plurality of measured values. The display device of the information processing device 40 may display variation values for each of a plurality of measurement items for the workpiece 82 .

In the information processing system 1, variation values for a plurality of measured values are displayed. When the variation values are displayed for each of the plurality of measurement items for the work 82, the user can grasp the variation values for each of the plurality of measurement items for the work 82. FIG. The user can recognize whether or not the variation value is greater than or equal to the threshold value by visually recognizing the variation value for the plurality of measured values. If the fluctuation value is equal to or greater than the threshold, the variation in the measurement of the workpiece 82, which is the object to be processed, can be reduced by changing the parameters for measuring the workpiece 82 or by changing the mounting positions of various sensors in the equipment 12. can be suppressed.

(Configuration example)
Next, an example of the functional configuration of the information processing system according to this embodiment will be described with reference to FIG. An information processing system 1 according to the present embodiment is used to process or manage information used for analysis of defects in manufacturing processes of various products. The information processing system 1 includes a facility control system 10 , an image processing system 20 , a storage 30 , an information processing device 40 and a network 50 . The facility control system 10, the image processing system 20, the storage 30, and the information processing device 40 are each connected to a network 50 and configured to be able to communicate information therebetween. Network 50 is, for example, a local area network.

1 shows one facility control system 10 and one image processing system 20 for convenience of explanation, one or more facility control systems 10 and one or more A plurality of image processing systems 20 are provided in association with each other.

The equipment control system 10 controls equipment 12 such as robots and manufacturing equipment in order to execute processing such as predetermined processing in each process, and obtains detection values of various sensors, which are information during processing. Data (hereinafter referred to as “sensor data”) is stored in the storage 30 .
Sensor data is set according to the content of the processing in the target process, and examples include torque values, speeds, positions, lengths such as height and width, flow values, and pressure values. is mentioned.

The image processing system 20 performs predetermined image processing on image data obtained by capturing an image of the work 82 with the camera 70 when processing such as predetermined processing in each process is executed. Get measurement data. The image processing system 20 stores image data and measurement data acquired from the image data in the storage 30 . Examples of the work 82 include parts, semi-finished products, etc., which are objects to be processed. The measurement data can be obtained from the image data of the work 82 captured when the predetermined processing such as processing in each process is executed. mentioned. Predetermined image processing for image data executed by image processing system 20 includes a plurality of measurement processes corresponding to a plurality of measurement items for work 82 . The image processing system 20 acquires a plurality of measurement values corresponding to the plurality of measurement items by executing a plurality of measurement processes corresponding to the plurality of measurement items for the workpiece 82 . As described above, the contents and number of the plurality of measurement items for the work 82 are different for each of the plurality of facilities 12 .

The storage 30 stores various data transmitted from the equipment control system 10 and the image processing system 20 via the network 50 .

The information processing device 40 is configured by using, for example, a general/general-purpose personal computer, and is used to refer to various data stored in the storage 30 and perform failure analysis and the like. The information processing device 40 performs measurement system analysis based on the measurement data.

As shown in FIG. 1, in the information processing system 1 of the present embodiment, the timing of imaging a workpiece 82 (hereinafter referred to as "imaging timing") when performing processing such as processing in each process is controlled by equipment. The system 10 notifies the image processing system 20 . Further, individual identification information, which is information for identifying (specifying) the workpiece 82 corresponding to the notification of the imaging timing, is transmitted from the equipment control system 10 to the image processing system 20 . Then, the facility control system 10 associates (adds) the individual identification information to the sensor data acquired corresponding to the imaging timing, and stores the sensor data in the storage 30 . In addition, the image processing system 20 associates (adds) the individual identification information and the imaging date and time of the work 82 to the image data (may be plural) and the measurement data acquired corresponding to the imaging timing, and stores them in the storage 30. Remember. When performing analysis in the information processing device 40, by using individual identification information as common information, sensor data, image data, and measurement data for each imaging timing can be uniquely specified, extracted, and used. Further details will be described below.

FIG. 2 is a diagram showing a configuration example of the facility control system 10. As shown in FIG. The equipment control system 10 includes a processor 101, a system controller 102, an I/O controller 103, a RAM 104, a storage unit 105, an input interface (IF) 107, an output interface (IF) 108, and a communication interface (IF ) 109 and a memory card interface (IF) 110 . Each of these components can communicate data with each other via the system controller 102 .

The processor 101 controls the entire facility control system 10 by executing various programs including the operating system and the facility control program 106 stored in the storage unit 105 in a predetermined order. The system controller 102 is connected to the processor 101, I/O controller 103, RAM 104, and storage unit 105 via a bus, and exchanges data with each component.

The I/O controller 103 is connected to each of the RAM 104, storage unit 105, input IF 107, output IF 108, communication IF 109, and memory card IF 110, and controls data exchange. RAM 104 provides a work area for storing data necessary for program execution in processor 101 . The storage unit 105 stores programs to be executed by the processor 101 in a nonvolatile manner, and is configured using a hard disk drive, for example. The equipment control program 106 is stored in the storage unit 105 .

The input IF 107 mediates data exchange between various sensors 60 provided in the equipment 12 and the processor 101 . Various sensors that can be used in the equipment 12 are conceivable as the sensors 60, and examples thereof include a pressure sensor, a flow rate sensor, a torque sensor, a speed sensor, and a position sensor. The output IF 108 is provided in the facility 12 and mediates data exchange between various servos 61 to be controlled by the facility control system 10 and the processor 101 . The communication IF 109 mediates data communication via the network 50 between the processor 101 and the image processing system 20 and storage 30 . The memory card IF 110 writes data to and reads data from the removable memory card 63 .

FIG. 3 is a diagram showing a configuration example of the image processing system 20. As shown in FIG. The image processing system 20 includes a main processor 201, an image processor 202, an I/O controller 203, a RAM 204, a storage unit 205, a camera interface (IF) 207, a communication interface (IF) 208, and a memory card. An interface (IF) 209 is included. Each of these components is in data communication with each other.

The main processor 201 controls the entire image processing system 20 by executing various programs including an operating system and an image processing program 206 stored in the storage unit 205 in a predetermined order. The image processor 202 acquires measurement data regarding the work 82 by performing image processing based on images obtained from each camera 70 via the camera IF 207 .

The I/O controller 203 is connected to each of the RAM 204, storage unit 205, camera IF 207, communication IF 208, and memory card IF 209, and controls data exchange. A RAM 204 provides a work area for storing data necessary for program execution in the main processor 201 . The storage unit 205 stores programs to be executed by the main processor 201 in a non-volatile manner, and is configured using, for example, a hard disk drive. An image processing program 206 is stored in the storage unit 205 .

The camera IF 207 mediates data exchange between each camera 70 capturing an image of the work 82 processed by the equipment 12 and the main processor 201 and the image processor 202 . A plurality of cameras 70 may be installed in order to image the state of the work 82 from a plurality of directions. The communication IF 208 mediates data communication via the network 50 between the main processor 201 and the facility control system 10 and storage 30 . The memory card IF 209 writes data to and reads data from the removable memory card 71 .

The image processing system 20 executes imaging processing for imaging one or more workpieces 82 multiple times at predetermined time intervals. The image processing system 20 acquires measurement data including a plurality of measurement values regarding the work 82 by executing processing on a plurality of image data obtained by executing imaging processing. In the imaging process executed by the image processing system 20, one or more workpieces 82 are imaged multiple times at predetermined time intervals, and after a predetermined period of time has elapsed, one or more workpieces 82 are captured at predetermined time intervals. It includes a process of capturing images twice. Any value of 0.1 mm seconds or more and 10 mm seconds or less can be set as the predetermined time interval, but it is not limited to these values. As the predetermined time, an arbitrary value of 1 hour or more and 10 hours or less can be set, but it is not limited to these values.

FIG. 4 is a diagram showing a configuration example of the storage 30. As shown in FIG. The storage 30 stores at least identification information including individual identification information of the workpiece 82 , measurement data including a plurality of measured values, and the imaging date and time of each imaging of the workpiece 82 . The storage 30 includes a main processor 301 , an I/O controller 302 , a RAM 303 , a communication interface (IF) 304 and a storage section 305 . Each of these components is in data communication with each other.

The main processor 301 controls the entire storage 30 by executing various programs including an operating system and control programs stored in the storage unit 305 in a predetermined order. The I/O controller 302 is connected to each of the RAM 303, communication IF 304, and storage unit 305, and controls data exchange. A RAM 303 provides a work area for storing data necessary for program execution in the main processor 301 .

The communication IF 304 mediates data communication via the network 50 between the main processor 301 and the equipment control system 10 and the image processing system 20 . A storage unit 305 stores programs executed by the main processor 301 and various data in a non-volatile manner, and is configured using a hard disk drive, for example. The storage unit 305 stores image data 306 and measurement data 307 transmitted from the image processing system 20 and sensor data 308 transmitted from the equipment control system 10 .

FIG. 5 is a diagram showing a configuration example of the information processing device 40. As shown in FIG. The information processing apparatus 40 is configured using, for example, a general-purpose personal computer, and includes a main processor 401, an I/O controller 402, a RAM 403, a communication interface (IF) 404, a storage unit 405, and a display interface. (IF) 407 and a display device 408 . Each of these components is in data communication with each other.

The main processor 401 executes various programs including the operating system and the reference/analysis program 406 stored in the storage unit 405 in a predetermined order. As a result, the main processor 401 performs processing for referencing and analyzing the image data 306 , measurement data 307 and sensor data 308 stored in the storage 30 . Reference and analysis based on each data are performed according to input using an input unit (for example, keyboard, mouse, etc.) not shown, and the results of reference and analysis based on each data are displayed on the screen of the display device 408. .

The I/O controller 402 is connected to each of the RAM 403, communication IF 404, and storage unit 405, and controls data exchange. A RAM 403 provides a work area for storing data necessary for program execution in the main processor 401 . Communication IF 404 mediates data communication via network 50 between main processor 401 and storage 30 .

A storage unit 405 stores programs to be executed by the main processor 401 and various data in a non-volatile manner, and is configured using a hard disk drive, for example. The display IF 407 is connected to the display device 408 and exchanges data with the display device 408 . The display device 408 is composed of, for example, a liquid crystal display, and displays various information on the screen. The display device 408 may be detachably attached to the information processing device 40 or may be integrally attached to the information processing device 40 .

FIG. 6 is a diagram showing an example structure of data stored in the storage 30 by the equipment control system 10 and the image processing system 20, respectively. Here, data stored in the storage 30 by the facility control system 10 is called "facility control system information", and data stored in the storage 30 by the image processing system 20 is called "image processing system information".

The facility control system information includes "linkage information" that is used for association with the image processing system information, and "sensor data (sensor information)" obtained from each sensor. The equipment control system information is stored in the storage 30 from the equipment control system 10 in a general-purpose data format file in which each piece of information is separated by a delimiter (delimiter/code), for example. As a general-purpose data format in which each piece of information is separated by a delimiter, there is a CSV (Comma Separated Value) format. By using such a data format file, the number of items can be made variable and database maintenance can be eliminated.

The linking information includes individual identification information (lot number), individual identification information (serial number in lot number), process information, equipment information, product number information, and imaging timing information. The individual identification information (lot number) is information for identifying each work 82 to be processed such as processing, and is, for example, a number assigned to the lot to which the work 82 belongs. The individual identification information (serial number in lot number) is information for identifying each work 82 to be processed such as processing. number is used. Depending on the object of the individual, if only one individual identification information is sufficient, either the individual identification information (lot number) or the individual identification information (serial number in lot number) can be omitted.

The process information is information for identifying each process for performing processing such as machining on the workpiece 82 . The facility information is information for identifying the facility used when performing processing such as processing on the workpiece 82 . Note that the process information can be omitted when the process information can be covered by the equipment information. The product number information is information such as a number given to each product corresponding to the workpiece 82 . The imaging timing information is information specifying the date and time when the equipment control system 10 notifies the image processing system 20 of the imaging timing.

Among the information stored in the storage 30, at least one of individual identification information (lot number), individual identification information (serial number in lot number), process information, equipment information, and imaging timing information is used as "identification information". be able to.

The sensor data includes sensor information item master allocation information, one or more sensor values (numerical values), one or more sensor values (character strings), and other subordinate information. The sensor information item master allocation information is allocation information such as the number, name, and unit of sensor values. The sensor value (numerical value) is represented by numerical data among the detection results obtained by each sensor 60 . A sensor value (character string) is represented by character string data among detection results obtained by each sensor 60 . Other subordinate information is arbitrary information such as information associated with each sensor value described above, for example.

The image processing system information includes "association information" which is information used for association with facility control system information, imaging date and time, and "measurement data (measurement information)" obtained by image processing. The image processing system information is stored in the storage 30 from the image processing system 20 in a general-purpose data format file in which each piece of information is separated by a delimiter (delimiter/code), for example. As a general-purpose data format in which each piece of information is separated by a delimiter, there is a CSV (Comma Separated Value) format. By using such a data format file, the number of items can be made variable and database maintenance can be eliminated.

The linking information includes individual identification information (lot number), individual identification information (serial number in lot number), process information, facility information, product number information, and imaging timing information. Each of these pieces of information is handed over from the equipment control system 10 to the image processing system 20 along with the notification of the imaging timing. Therefore, a detailed description of the content of each piece of information is omitted. The imaging date and time is the date and time when the work 82 was imaged by the camera 70 corresponding to the imaging timing notified from the facility control system 10 to the image processing system 20 .

The measurement data includes an imaging camera number, an imaging sequence, measurement data management master allocation information, one or more measured values (numerical values), one or more measured values (character strings), and other dependent information. The imaging camera number is a number for identifying each camera 70 . The imaging order is information specifying the imaging order of the images used for the measurement. The measurement data management master allocation information is allocation information such as the number of measured values, names, and units. The measured value (numerical value) is represented by numerical data among the measured values obtained by image processing based on the image obtained by each camera 70 . The measured value (character string) is represented by character string data among the measured values obtained by image processing based on the image obtained by each camera 70 . Other subordinate information is arbitrary information such as information associated with each measurement value described above, for example.

FIG. 7 is a flow chart showing an operation procedure related to notification of imaging timing and generation of sensor data in the equipment control system 10 . Note that the flowchart shown in FIG. 7 is an example, and the order of each step may be changed as long as there is no contradiction or inconsistency in the processing results, and other processing steps (not shown) may be included ( The same applies to other flowcharts described below).

The processor 101 of the facility control system 10 determines whether or not to end the process when the process loop starts when the power is turned on (step S11). For example, when an operation to turn off the power is performed, or an operation to temporarily suspend the processing is performed, the processor 101 determines to end the processing (step S11; YES), and performs a series of processing. exit the loop. If it is not necessary to end the process (step S11; NO), the processor 101 performs predetermined facility control (step S12). Specifically, the processor 101 gives operation instructions to each servo 61 and the like.

Next, the processor 101 determines whether or not the current time is the imaging timing for acquiring the image of the workpiece 82 (step S13). The imaging timing is set in advance, and can be set in various ways, for example, the timing at which detection by each sensor 60 is performed during processing such as processing, the timing at which processing is completed, and the like. If the current time is the imaging timing (step S13; YES), the processor 101 stores sensor data including detection results (numerical values, character strings) by each sensor 60 in the RAM 104 (step S14). Next, the processor 101 notifies the image processing system 20 of the identification information including the individual identification information (step S15), and notifies the image processing system 20 of the imaging timing (step S16). The identification information includes individual identification information (lot number), individual identification information (serial number in lot number), process information, facility information, product number information, and imaging timing information.

Next, the processor 101 generates a sensor data file (step S17). Here, a file in which the identification information and the sensor data as described with reference to FIG. For example, a file in a general data format such as CSV format may be generated. Next, the processor 101 stores the file transmission request generated in step S17 in the RAM 104 (step S18). After that, the process returns to step S11.

FIG. 8 is a flow chart showing an operational procedure for transmitting sensor data in the equipment control system. The processor 101 of the facility control system 10 determines whether or not to end the process when the process loop is started by power-on or the like (step S21). For example, when an operation to turn off the power is performed, or when an operation to temporarily suspend the processing is performed, the processor 101 determines to end the processing (step S21; YES), and the series of processing is performed. exit the loop.

If it is not necessary to end the process (step S21; NO), the processor 101 waits for a file transmission request (step S22). This standby state is maintained while there is no file transmission request in RAM 104 (step S23; NO). If there is a file transmission request in RAM 104 (step S23; YES), processor 101 acquires the sensor data file from memory card 63 (step S24) and transmits this file to storage 30 (step S25). As a result, the sensor data file is stored in the storage 30 .

Next, the processor 101 deletes the sensor data file stored in the memory card 63 (step S26), and confirms the file transmission request in the RAM 104 (step S27). After that, the process returns to step S21.

FIG. 9 is a flowchart showing an operation procedure for transmitting image data and measurement data in the image processing system 20. As shown in FIG. The main processor 201 of the image processing system 20 determines whether or not to end the processing when the processing loop starts when the power is turned on (step S31). For example, when an operation to turn off the power is performed, or an operation to temporarily suspend the processing is performed, the main processor 201 determines to end the processing (step S31; YES), and performs a series of operations. End the processing loop.

If it is not necessary to end the process (step S31; NO), the main processor 201 waits for notification of the imaging timing (step S32). This standby state is maintained while the facility control system 10 does not notify the imaging timing (imaging request) (step S33; NO). When the facility control system 10 notifies the imaging timing (imaging request) (step S33; YES), the main processor 201 acquires the image data (camera image) and the imaging date and time from each camera 70, and the image processing processor Image data is input to 202 (step S34). The image processor 202 performs predetermined measurement processing and determination processing based on the input image data (step S35).

Next, the main processor 201 generates a file (image data file) containing the image data acquired from each camera 70 and the identification information acquired from the facility control system 10, and transmits it to the storage 30 (step S36). The main processor 201 generates a file (measurement data file) including the measurement data such as the measurement values generated by the process of step S35, the identification information acquired from the equipment control system 10, and the imaging date and time, and transmits the file to the storage 30. (Step S37). After that, the process returns to step S31.

FIG. 10 is a diagram for explaining data that can be referenced from the storage 30 by the information processing device 40. As shown in FIG. As shown in FIG. 10, the information processing device 40 accesses the storage 30 to obtain image processing system information 502 including measurement data and equipment control system information including sensor data based on linking information 501 including identification information. 503 can be uniquely identified and referenced, and various data analyzes can be performed as desired.

FIG. 11 is a flowchart showing an operation procedure for displaying measurement items in the information processing device 40. As shown in FIG. The main processor 401 of the information processing device 40 identifies a measurement target (step S41). Specifically, the main processor 401 refers to the storage 30 and specifies information for calculating a value (fluctuation value) relating to fluctuations (variation) of a plurality of measured values included in the measurement data. The variation value may be a gauge R&R value (GRR value) in Measurement Systems Analysis (MSA). The variability values include repeatability variability (instrument variability) values, part-to-part variability values, and reproducibility variability (operator variability) values.

The main processor 401 acquires from the storage 30 a plurality of measurement values (measurement data) for calculating the repeatability variation value. A plurality of measurement values for calculating the value of repeatability variation are a plurality of measurement values obtained when the same workpiece 82 is imaged a plurality of times. These multiple measurement values are measurement values obtained from each of multiple images (image data) obtained when the same workpiece 82 is imaged multiple times. For example, when imaging the same work 82 three times, the main processor 401 acquires three measured values from the storage 30 as multiple measured values for calculating the repeatability variation value. The image processing system 20 may image the same workpiece 82 multiple times with one camera 70 , or may image the same workpiece 82 once or multiple times with multiple cameras 70 . The image processing system 20 may continuously image the same workpiece 82 multiple times.

A plurality of measured values for calculating the value of repeatability variability satisfy all the items listed in Condition 1 below.
(Condition 1)
- The individual identification information corresponding to each measured value is the same.
- The process information corresponding to each measured value is the same.
- Equipment information corresponding to each measured value is the same.
- The interval between the imaging dates and times corresponding to each measurement value is short.
The main processor 401 acquires from the storage 30 a plurality of measured values that satisfy all the items listed in Condition 1. The main processor 401 may determine that the interval between the imaging dates and times is short if the interval between the imaging dates and times is equal to or less than a first predetermined time (for example, 10 milliseconds). If the interval between the imaging dates corresponding to each measurement value is short, each measurement value is a measurement value obtained when the work 82 is imaged multiple times in the same process. The interval between the imaging dates and times corresponding to the values is short." is listed.

The main processor 401 acquires from the storage 30 a plurality of measurement values (measurement data) for calculating inter-part variation values. The plurality of measurement values for calculating the inter-part variation value are the plurality of measurement values obtained when the plurality of workpieces 82 are imaged a plurality of times. These plurality of measured values are measured values obtained from each of a plurality of images (image data) obtained when a plurality of workpieces 82 are imaged a plurality of times. For example, when 10 workpieces 82 are imaged 3 times, the main processor 401 acquires 30 measurement values from the storage 30 as a plurality of measurement values for calculating inter-part variation values. The image processing system 20 may image a plurality of workpieces 82 multiple times with a single camera 70 , or may image a plurality of workpieces 82 multiple times with a plurality of cameras 70 . The image processing system 20 may continuously image each of the plurality of works 82 a plurality of times.

A plurality of measured values for calculating the value of part-to-part variation satisfy all items listed in Condition 2-1 or all items listed in Condition 2-2 below.
(Condition 2-1)
- The individual identification information corresponding to each measured value is the same.
- The process information corresponding to each measured value is the same.
- Equipment information corresponding to each measured value is the same.
- The interval between the imaging dates and times corresponding to each measurement value is short.
(Condition 2-2)
・Individual identification information corresponding to each measured value is different.
- The process information corresponding to each measured value is the same.
- Equipment information corresponding to each measured value is the same.
- The interval between the imaging dates and times corresponding to each measurement value is short.

The main processor 401 acquires from the storage 30 a plurality of measurement values that satisfy all items listed in Condition 2-1 or all items listed in Condition 2-2. The main processor 401 may determine that the interval between the imaging dates and times is short if the interval between the imaging dates and times is equal to or less than a first predetermined time (for example, 10 milliseconds).

The main processor 401 acquires from the storage 30 a plurality of measurement values (measurement data) for calculating reproducibility variation values. A plurality of measurement values for calculating the value of reproducibility variation are a plurality of measurement values obtained when a plurality of workpieces 82 are imaged a plurality of times when the same process is performed a plurality of times. The plurality of measurement values are measurement values obtained from each of a plurality of images (image data) when the plurality of workpieces 82 are imaged a plurality of times when the same process is performed a plurality of times. For example, when ten workpieces 82 are imaged three times and the same process is performed three times, the main processor 401 stores 90 measurement values in the storage 30 as a plurality of measurement values for calculating reproducibility variation values. Get from The image processing system 20 may image a plurality of workpieces 82 multiple times with a single camera 70 , or may image a plurality of workpieces 82 multiple times with a plurality of cameras 70 . The image processing system 20 may continuously image each of the plurality of works 82 a plurality of times.

A plurality of measured values for calculating reproducibility variation values satisfy all items listed in Condition 3-1 or all items listed in Condition 3-2 below.
(Condition 3-1)
- The individual identification information corresponding to each measured value is the same.
- The process information corresponding to each measured value is the same.
- Equipment information corresponding to each measured value is the same.
- The plurality of measured values include measured values with short intervals between imaging dates and times corresponding to each measured value.
- The plurality of measured values include measured values with long intervals between imaging dates and times corresponding to each measured value.
(Condition 3-2)
・Individual identification information corresponding to each measured value is different.
- The process information corresponding to each measured value is the same.
- Equipment information corresponding to each measured value is the same.
- The plurality of measured values include measured values with short intervals between imaging dates and times corresponding to each measured value.
- The plurality of measured values include measured values with long intervals between imaging dates and times corresponding to each measured value.

The main processor 401 acquires from the storage 30 a plurality of measurement values that satisfy all items listed in Condition 3-1 or all items listed in Condition 3-2. The main processor 401 may determine that the interval between the imaging dates and times is short if the interval between the imaging dates and times is equal to or less than a first predetermined time (for example, 10 milliseconds). The main processor 401 may determine that the interval between the imaging dates and times is long if the interval between the imaging dates and times is equal to or longer than a second predetermined time (eg, 3 hours). When the interval between the imaging dates corresponding to each measurement value is short, each of these measurement values is a measurement value obtained when the work 82 is imaged multiple times in the same process, so condition 3-1 and condition 3-2 includes the item "The plurality of measured values include measured values with short intervals between the imaging dates corresponding to each measured value." When the interval between the imaging dates corresponding to each measurement value is long, each measurement value is a measurement value obtained when the work 82 is imaged multiple times in each process when the same process is performed multiple times. Therefore, the condition 3-1 and the condition 3-2 are set such that "the plurality of measured values include measured values with long intervals between the imaging dates and times corresponding to each measured value.
” is mentioned.

Next, the main processor 401 acquires a plurality of measurement items for the workpiece 82 from the storage 30 (step S42). A plurality of measurement items for the work 82 are stored in advance in the storage 30 . The main processor 401 may acquire all measurement items for the work 82 from the storage 30 . The main processor 401, when the processing loop is started with the acquisition of a plurality of measurement items for the workpiece 82, determines whether the processing of steps S44 and S45, which will be described later, has been completed for all of the acquired plurality of measurement items. (Step S43).

If the processing of steps S44 and S45 has not been completed for any of the acquired measurement items (step S43; NO), the main processor 401 calculates a variation value based on the measurement data (step S44). . More specifically, the main processor 401 calculates the variation values based on the multiple measurement values acquired from the storage 30 for the multiple measurement items acquired by the process of step S42. The main processor 401 calculates the value of repeatability variation based on a plurality of measured values (see Condition 1) for calculating the value of repeatability variation. The main processor 401 calculates inter-part variation values based on a plurality of measured values (see conditions 2-1 and 2-2) for calculating inter-part variation values. The main processor 401 calculates the reproducibility variation value based on a plurality of measured values (see conditions 3-1 and 3-2) for calculating the reproducibility variation value.

If the processing of steps S44 and S45 has been completed for all of the acquired measurement items (step S43; YES), the main processor 401 terminates the series of processing loops. Next, the main processor 401 displays the variation values for each of the plurality of measurement items on the screen of the display device 408 (step S44). The main processor 401 may display the variation values for each measurement item on the screen of the display device 408 for all the measurement items for the workpiece 82 . The main processor 401 may make a list of the variation values for each of the plurality of measurement items and display it on the screen of the display device 408 .

The display device 408 may display a plurality of variation values, and may display the variation values equal to or greater than the threshold among the plurality of variation values in a different manner from the display method for the variation values less than the threshold among the plurality of variation values. . The display device 408 may arrange and display a plurality of variation values in descending order. When displaying the variation values for each of the plurality of measurement items on the screen of the display device 408, the main processor 401 may rearrange the plurality of measurement items in descending order of the variation values. When displaying the variation values for each of the plurality of measurement items on the screen of the display device 408, the main processor 401 extracts the measurement items whose variation values are equal to or greater than the threshold among the plurality of measurement items, and extracts the measurement items with the extracted measurement items. , the fluctuation values of the extracted measurement items may be displayed. The display of measurement items and variation values on the screen of the display device 408 can be selectively or arbitrarily changed by input (designation of display options) using the input unit of the information processing device 40 .

FIG. 12 is a diagram showing an example of rearranging a plurality of measurement items in descending order of variation values. In FIG. 12, the vertical axis indicates the variation value (GRR value), and the horizontal axis indicates the measurement item. In FIG. 12 , the measurement items with large GRR values are rearranged and displayed in order from the left. Further, in FIG. 12, when the GRR value is equal to or greater than the first threshold value (0.3), when the GRR value is less than the first threshold value (0.3) and equal to or greater than the second threshold value (0.1), The pattern of the bar graph showing the GRR value is changed depending on whether the GRR value is less than the second threshold value (0.1). As shown in FIG. 12, the larger the GRR value, the longer the bar graph showing the GRR value, and the smaller the GRR value, the shorter the bar graph showing the GRR value. In FIG. 12, the accuracy in the measurement items (pin 01_length, pin 01_tip area) corresponding to the GRR value equal to or greater than the first threshold value (0.3) is low. By changing the display of the measurement items with low accuracy and the display of the measurement items whose accuracy satisfies the standard, the user who maintains and manages the equipment 12 and the user who measures the work 82 can easily understand the measurement items with low accuracy. can be emphasized and presented.

For example, adding a pattern to the bar graph showing the GRR value when the GRR value is equal to or greater than the first threshold value (0.3), and the bar graph showing the GRR value when the GRR value is less than the first threshold value (0.3) It is also possible not to add a pattern to . Depending on the magnitude of the GRR value, the color of the characters of the measurement items displayed on the screen of the display device 408 may be changed, or the color of the bar graph displayed on the screen of the display device 408 may be changed. good. If the GRR value is greater than or equal to the first threshold value (0.3), the measurement items are displayed in red characters. may be added. If the GRR value is less than the first threshold value (0.3) and is greater than or equal to the second threshold value (0.1), the measurement item is displayed in yellow characters, or if the GRR value is less than the first threshold value (0.3) Yellow may be added to the bar graph showing the GRR value when it is equal to or greater than the second threshold value (0.1). If the GRR value is less than the second threshold (0.1), the measurement item is displayed in black characters, or if the GRR value is less than the second threshold (0.1), the bar graph showing the GRR value is displayed in black. may be added.

The variability values displayed on the screen of the display device 408 may include at least one of repeatability variability values, part-to-part variability values, and reproducibility variability values. The variability value displayed on the screen of the display device 408 may be the repeatability variability value. The variation value displayed on the screen of the display device 408 may be the part-to-part variation value. The variation value displayed on the screen of the display device 408 may be the reproducibility variation value. The variable value displayed on the screen of the display device 408 may be the GRR value.

Further, when displaying the variation values for each of the plurality of measurement items on the screen of the display device 408, the main processor 401 may rearrange the plurality of measurement items in numerical order. Furthermore, the main processor 401 may narrow down a plurality of measurement items to display the variation values for one or more specific measurement items on the screen of the display device 408 . For example, if some of the plurality of variation values are equal to or greater than the first threshold (0.3), the variation values equal to or greater than the first threshold (0.3) are displayed on the screen of the display device 408 along with the measurement items. , a variation value less than the first threshold (0.3) may be hidden.

By emphasizing and presenting measurement items with low accuracy to the user who maintains and manages the equipment 12 , it is possible to prompt the user to review the maintenance of the equipment 12 . By emphasizing and presenting measurement items with low accuracy to the user who is measuring the workpiece 82, it is possible to prompt the user to review the measurement of the workpiece 82. FIG. The user can check the measurement items with low accuracy and evaluate the measurement items with low accuracy. For example, if the measurement items with low accuracy are items related to various sensors in the equipment 12, the user can improve the measurement items with low accuracy by changing and adjusting the mounting positions of the various sensors in the equipment 12. . For example, if the measurement item with low accuracy is an item related to the method of measuring the workpiece 82, the user can improve the measurement item with low accuracy by changing/adjusting the measurement parameters. A measurement parameter is a parameter for measuring the workpiece 82 . When the measurement item is the area of the work 82, the measurement parameter is how to determine the reference point for measuring the area of the work 82, and the like.

The storage 30 may store the user's comments on the variation values, and the display device 408 may display the variation values and the user's comments on the variation values. The display device 408 may display a variation value for each of the plurality of measurement items and a user's comment on the variation value for each of the plurality of measurement items. For example, user confirmation results and evaluation results for measurement items with low accuracy can be stored in the storage 30 as comment information. The user inputs using the input unit of the information processing device 40 , and the information processing device 40 accesses the storage 30 , so that the user's confirmation results and evaluation results for low-accuracy measurement items are stored in the storage 30 .

FIG. 13 is a diagram for explaining data that can be referenced from the storage 30 by the information processing device 40. As shown in FIG. As shown in FIG. 13 , process information, facility information, measurement item number, registration date and time, review method comment, and user comment information (measurement item review method) 601 including registrants may be stored in the storage 30 . Further, as shown in FIG. 13 , process information, equipment information, measurement item number, registration date and time, review history comment, and user comment information (measurement item review history) 602 including registrants may be stored in the storage 30 . The user comment information may include comments and messages such as "measurement item 01 does not need to be reviewed due to reason 1" and "review of facility A". The maintenance screen of the display device 408 can be used to add, update, or delete each item of the user comment information 601 and 602 . The main processor 401 may store user comment information in the storage 30 in association with a specific measurement item among the plurality of measurement items. When the variation values for each of the plurality of measurement items are displayed on the screen of the display device 408, the user may designate one or more of the plurality of measurement items. The main processor 401 may display user comment information associated with the designated measurement item on the screen of the display device 408 .

[Support function for reviewing measurement items]
For at least one of the facility unit and the measurement item unit, the storage 30 may pre-store, as recommended information (guide information), items that may induce variation (variation) in the measured values. When the user inputs using the input unit of the information processing device 40 and the information processing device 40 accesses the storage 30 , the storage 30 stores items that may induce fluctuations in the measured values. Matters that may induce fluctuations in measured values include edge detection gradation level, presence or absence of preprocessing (filter expansion and contraction, binarization level), model taking method, correlation value level, horizontal expansion, etc. Related item names can be listed.

The main processor 401 may store, in the storage 30, as the recommendation information, contents that include personal elements such as setting of measurement parameters that are specific to an inspection program when it is created. The main processor 401 may store the recommendation information in the storage 30 in association with a specific measurement item among the plurality of measurement items. When the variation values for each of the plurality of measurement items are displayed on the screen of the display device 408, the user may designate one or more of the plurality of measurement items. The main processor 401 may display recommended information associated with the designated measurement item on the screen of the display device 408 .

Although an example in which the information processing device 40 performs measurement system analysis has been described above, the present embodiment is not limited to this example. One or both of the facility control system 10 and the image processing system 20 may perform the measurement system analysis and calculate the variation value. The equipment control system 10 may have the configuration and functions of the information processing device 40 . The image processing system 20 may have the configuration and functions of the information processing device 40 . Facility control system 10 may obtain information and data necessary to perform measurement system analysis from one or both of image processing system 20 and information processor 40 . Image processing system 20 may obtain information and data necessary to perform measurement system analysis from one or both of facility control system 10 and information processor 40 . One or both of the facility control system 10 and the image processing system 20 may be provided with a display device, and one or both of the display device of the facility control system 10 and the display device of the image processing system 20 display fluctuation values, measurement items, and the like. may be displayed.

According to the present embodiment, it becomes easy to specify measurement items with low accuracy, and it is possible to provide a standard for creating an inspection program that is individualized. According to the present embodiment, by improving measurement items with low accuracy in the production preparation stage, it is possible to perform quality determination at a certain level even in the production stage. According to the present embodiment, it is possible to give a determination that subsequent maintenance man-hours may be reduced for measurement items with high accuracy. According to this embodiment, even if there is a difference in the number of measurement items managed by each equipment 12, there is no need to change the system. can do As a result, it is possible to construct a system with an intensive operation base and low-spec hardware, thereby reducing equipment investment. According to this embodiment, since the information is stored in the storage 30, it is possible to refer to the history of the past information at the production preparation stage, at the timing of regular maintenance after operation, etc., so the transition of deterioration due to aging can be confirmed. It is also possible to According to this embodiment, under the condition that the length, width, area, etc. can be precisely collected in advance, it is possible to apply the linearity evaluation after collecting the information.

<Appendix>
a controller (10) for controlling a process performed on one or more objects (82);
Acquiring a plurality of measurement values relating to the one or more objects by performing processing on a plurality of image data obtained by performing an imaging process of imaging the one or more objects a plurality of times at predetermined time intervals. an image processing device (20);
a storage device (30) for storing identification information relating to the one or more objects, the plurality of measured values, and an imaging date and time for each imaging of the one or more objects;
A display device ( 408) and
comprising
Information processing system (1).

1: Information processing system 10: Facility control system (control device)
12: Equipment 20: Image processing system (image processing device)
30: Storage (storage device)
40: Information processing device 50: Network 70: Camera 80: Conveyor 82: Work 408: Display device

Claims (12)

a control device for controlling the processing performed on one or more objects;
Acquiring a plurality of measurement values relating to the one or more objects by performing processing on a plurality of image data obtained by performing an imaging process of imaging the one or more objects a plurality of times at predetermined time intervals. an image processing device;
a storage device that stores identification information about the one or more objects, the plurality of measured values, and an imaging date and time for each imaging of the one or more objects;
a display device for displaying variation values relating to the plurality of measured values calculated by performing analysis processing on the plurality of measured values based on the identification information, the plurality of measured values, and the imaging date and time; ,
comprising
Information processing system. the processing on the plurality of image data executed by the image processing device includes a plurality of measurement processing corresponding to a plurality of measurement items;
The image processing device acquires the plurality of measured values corresponding to the plurality of measurement items,
The display device displays the variation value for each of the plurality of measurement items.
The information processing system according to claim 1. A plurality of facilities for performing processing on the one or more objects,
The content and number of the plurality of measurement items are different for each of the plurality of facilities,
The information processing system according to claim 2. The display device displays a plurality of the variation values, and uses a different display method for variation values equal to or greater than a threshold among the plurality of variation values and a display method for variation values less than the threshold among the plurality of variation values. let
The information processing system according to any one of claims 1 to 3. The display device arranges and displays the plurality of variation values in descending order of the plurality of variation values.
The information processing system according to any one of claims 1 to 4. In the imaging process, the one or more objects are imaged a plurality of times at the predetermined time intervals after a predetermined time has elapsed after the one or more objects are imaged a plurality of times at the predetermined time intervals. including processing,
The information processing system according to any one of claims 1 to 5. The storage device stores a user's comment on the variation value,
The display device displays the variation value and the comment.
The information processing system according to any one of claims 1 to 6. The storage device stores a user's comment on the variation value,
The display device displays the variation value and the comment for each of the plurality of measurement items.
The information processing system according to claim 2. The control device calculates the variation value by executing the analysis process for the plurality of measured values based on the identification information, the plurality of measured values, and the imaging date and time.
The information processing system according to any one of claims 1 to 8. The image processing device calculates the variation value by executing the analysis process on the plurality of measured values based on the identification information, the plurality of measured values, and the imaging date and time.
The information processing system according to any one of claims 1 to 8. An information processing device that calculates the variation value by performing the analysis process on the plurality of measured values based on the identification information, the plurality of measured values, and the imaging date and time,
The information processing system according to any one of claims 1 to 8. An information processing method in a system including a control device, an image processing device, a storage device, and a display device,
the controller controlling a process performed on one or more objects;
The image processing device performs processing on a plurality of image data obtained by executing imaging processing for capturing images of the one or more objects a plurality of times at predetermined time intervals, and performs processing on a plurality of image data related to the one or more objects. obtaining a measurement of
a step in which the storage device stores identification information about the one or more objects, the plurality of measured values, and an imaging date and time for each imaging of the one or more objects;
The display device displays a variation value regarding the plurality of measured values calculated by performing analysis processing on the plurality of measured values based on the identification information, the plurality of measured values, and the imaging date and time. a step to display;
including,
Information processing methods.

Images