Disclosure of Invention
The invention provides a cloth inspection processing method, a device, equipment, a system and a medium, which are used for solving the problems of higher cloth inspection cost and lower efficiency in the prior art.
In a first aspect, the present invention provides a method for processing a cloth inspection, applied to a server, the method comprising:
In the cloth inspection process, receiving a cloth image to be processed sent by a cloth inspection machine;
inputting the cloth image into a pre-acquired cloth detection model, and determining the flaw position and flaw type in the cloth image, wherein the cloth detection model is used for identifying the cloth image to determine the position and the flaw type of the cloth flaw;
Marking the cloth image according to the flaw positions and flaw types in the cloth image to obtain a marked cloth image;
Sending the marked cloth image to terminal equipment of a user;
After receiving a cloth checking stopping instruction sent by the terminal equipment, sending the cloth checking stopping instruction to the cloth inspecting machine, wherein the cloth checking stopping instruction is an instruction which is generated by the terminal equipment in response to the operation of the user and used for controlling the cloth inspecting machine to stop a rotating shaft.
In a specific embodiment, before the cloth image to be processed sent by the cloth inspection machine is received in the cloth inspection process, the method further includes:
and receiving a cloth starting instruction sent by the terminal equipment, and sending the cloth starting instruction to the cloth inspecting machine, wherein the cloth starting instruction is used for controlling the cloth inspecting machine to start a rotating shaft and start cloth inspection.
In one embodiment, the method further comprises:
Obtaining a pre-calibrated cloth image set, wherein the cloth image set comprises a plurality of calibrated cloth images marked with flaw positions and flaw types of cloth;
And carrying out model training by adopting a convolution network model according to the cloth image set to obtain the cloth detection model.
In one embodiment, the method further comprises:
calculating coordinate information of the at least one flaw according to the flaw marking message;
And sending a marking instruction to the cloth inspection machine according to the coordinate information of the at least one flaw, wherein the marking instruction comprises the coordinate information of the at least one flaw, and the marking instruction is used for controlling the cloth inspection machine to mark the at least one flaw on cloth.
In one embodiment, the method further comprises:
receiving cloth inspection requirement information sent by a factory cloth inspection platform, wherein the cloth inspection requirement information comprises the required grades of cloth inspection users and the number of the cloth inspection users;
According to the pre-acquired rating information of each user person and the cloth inspection requirement information, at least one target cloth inspection user meeting the cloth inspection requirement information is acquired from currently selectable cloth inspection users;
And sending a verification task to terminal equipment corresponding to the at least one target verification user.
In a specific embodiment, the rating information of each cloth inspection user comprises the rating of the cloth inspection user, the video playing speed, the missing report rate and the false report rate corresponding to the rating, and the rating information is obtained according to cloth inspection data of the cloth inspection user in the cloth inspection training process.
In a second aspect, the present invention provides a method for processing a cloth inspection, applied to a terminal device, where the method includes:
In the cloth inspection process, receiving a marked cloth image sent by a server, wherein the marked cloth image comprises flaw positions and flaw types automatically detected and marked by the server;
displaying the marked cloth image on a graphical user interface;
and responding to the operation of a user on a cloth checking stopping button on the graphical user interface, sending a cloth checking stopping instruction to the server, wherein the cloth checking stopping instruction is used for controlling the instruction of the cloth inspecting machine to stop the rotating shaft.
In a specific embodiment, before the step of receiving the marked cloth image sent by the server in the cloth inspection process, the method further includes:
And responding to the operation of a start cloth inspection button on the graphical user interface by a user, and sending a start cloth inspection instruction to the server, wherein the start cloth inspection instruction is used for controlling the cloth inspection machine to start a rotating shaft and start cloth inspection.
In one embodiment, the method further comprises:
And sending flaw marking information to the server in response to the position and type of at least one flaw manually marked on the graphical user interface by the user, wherein the flaw marking information comprises the position and type of the at least one flaw.
In one embodiment, the method further comprises:
and receiving the cloth verification task sent by the server, and displaying the cloth verification task on the graphical user interface.
In one embodiment, the method further comprises:
In the process of verifying and training, responding to the operation of the user on a graphical user interface, and acquiring verifying and training data;
And sending the verification data to the server, wherein the verification data is used for acquiring the rating information of the user, and the rating information comprises the rating of the user, the video playing speed, the missing report rate and the false report rate corresponding to the rating.
In a third aspect, the present invention provides a cloth inspection processing apparatus comprising:
The receiving module is used for receiving the cloth image to be processed sent by the cloth inspection machine in the cloth inspection process;
The detection module is used for inputting the cloth image into a pre-acquired cloth detection model, determining the flaw position and flaw type in the cloth image, and identifying the cloth image by the cloth detection model to determine the position and the flaw type of the cloth flaw;
The processing module is used for marking the cloth image according to the flaw position and flaw type in the cloth image to obtain a marked cloth image;
The sending module is used for sending the marked cloth image to terminal equipment of a user;
The sending module is further used for sending the cloth checking stopping instruction to the cloth inspecting machine after receiving the cloth checking stopping instruction sent by the terminal equipment, wherein the cloth checking stopping instruction is an instruction which is generated by the terminal equipment in response to the operation of the user and used for controlling the cloth inspecting machine to stop the rotating shaft.
In a fourth aspect, the present invention provides a cloth inspection processing apparatus comprising:
The receiving module is used for receiving the marked cloth image sent by the server in the cloth inspection process, wherein the marked cloth image comprises flaw positions and flaw types automatically detected and marked by the server;
the display module is used for displaying the marked cloth image on a graphical user interface;
And the sending module is used for responding to the operation of a user on the cloth checking stopping button on the graphical user interface and sending a cloth checking stopping instruction to the server, wherein the cloth checking stopping instruction is used for controlling the instruction of the cloth inspecting machine to stop the rotating shaft.
In a fifth aspect, the present invention provides a server comprising:
a processor, a memory, a communication interface;
the memory is used for storing executable instructions of the processor;
wherein the processor is configured to perform the cloth inspection processing method of any of the first aspects via execution of the executable instructions.
In a sixth aspect, the present invention provides a terminal device, including:
A processor, a memory, a display, and a communication interface;
the memory is used for storing executable instructions of the processor;
wherein the processor is configured to perform the cloth inspection processing method of any of the second aspects via execution of the executable instructions.
In a seventh aspect, the present invention provides a cloth inspection processing system comprising:
The system comprises a server, terminal equipment, a switch and a cloth inspection machine;
The server is used for executing the cloth inspection processing method in any one of the first aspect;
The terminal device is configured to execute the cloth inspection processing method in any one of the second aspects;
The switch is used for transmitting network communication signals;
The cloth inspection machine is used for receiving the cloth inspection starting instruction, the cloth inspection stopping instruction and the marking instruction sent by the server, controlling a rotating shaft on the cloth inspection machine to start rotating according to the cloth inspection starting instruction, controlling the rotating shaft on the cloth inspection machine to stop rotating according to the cloth inspection stopping instruction, and controlling a coding machine on the cloth inspection machine to mark flaws on cloth according to the marking instruction;
the cloth inspection machine is further used for sending the cloth image to be processed to the server.
In an eighth aspect, the present invention provides a readable storage medium having stored thereon a computer program which, when executed by a processor, implements the cloth inspection processing method of any one of the first to second aspects.
According to the cloth inspection processing method, the device, the equipment, the system and the medium, the type and the position of the flaw in the cloth image are identified by inputting the cloth image sent by the cloth inspection machine into the cloth detection model, the flaw is marked in the cloth image and then sent to the terminal equipment of a user, the user determines the position and the type of the marked flaw and then sends an instruction for stopping cloth inspection to the server, and then the server sends the instruction for stopping cloth inspection to the cloth inspection machine, so that the cloth inspection machine can stop the rotating shaft. And then the user sends a marking message to the server, and the server processes the cloth inspection message to obtain flaw coordinates and sends a marking instruction to the cloth inspection machine, and the cloth inspection machine marks the cloth according to the flaw coordinates. According to the cloth inspection device, cloth inspection is achieved through manual work and cloth inspection models, cloth inspection cost is effectively reduced, and cloth inspection efficiency is improved.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which are made by a person skilled in the art based on the embodiments of the invention in light of the present disclosure, are intended to be within the scope of the invention.
The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In daily life, the cloth is widely applied, and clothes, used bedsheets and the like worn by people cannot be separated from the cloth, and strict requirements are also provided for the quality of the cloth, so that manufacturers can detect the cloth after the cloth is produced.
Because the cloth quality inspection industry generally uses manpower to carry out cloth inspection operation on a cloth inspection machine, the detected breadth is at least more than 1.5 meters, the cloth is faster to run, usually 20-50 meters per minute, the cloth is operated on the cloth inspection machine for a long time, the human eyes are easy to be tired, so that subtle and unobvious defects are easy to appear, the detection omission and the false detection are easy to occur, in addition, the cloth inspection standard is different, and the study and the judgment are completely carried out by the subjective experience of people. At present, the industry also has an intelligent cloth inspection machine, but the machine still cannot completely replace people to finish cloth inspection work in face of the diversity of cloth and flaws. In addition, currently, the laboratory-setting users are generally recruited and trained by factories, so that the yield in a short period is easily affected by order change, personnel flow and the like, and the maximization of the yield and the value cannot be realized. Training before the test user goes on duty depends entirely on the experienced master-aged dictation quality inspection experience, and can not realize the standardization of flaws, so that quality inspection results among people are different, and training time and training cost are high. Thus, the prior art results in higher cloth inspection costs and lower cloth inspection efficiency.
The inventor finds that in the process of researching a related system for inspecting cloth, a cloth image can be acquired through a camera, the cloth image is input into a cloth detection model, the flaw type and the flaw position are displayed to a user after being obtained, the user can inspect and mark the flaw type and the flaw position obtained by the cloth detection model, and the cloth detection model can be reinforced trained by using the flaw data of the manual mark, so that the recognition accuracy of the cloth detection model is higher. In addition, the training of staff can be realized by collecting the cloth inspection video in advance, the to-be-trained staff can select the playing speed to watch the video and mark the flaw position, the to-be-trained staff is rated according to the playing speed, the missing report rate and the false report rate, the cloth inspection users can be scheduled according to the requirements of factories after the training is finished, recruitment and training tasks of quality inspection staff are not required to be maintained in time and cost of the factories, manpower allocation is flexible, the manpower shortage of the factories can be quickly supplemented when the generation demand is large, and the cloth inspection operation can be simultaneously carried out by multiple people when the requirement on the quality of products is high, so that the cloth inspection task can be completed in a guarantee period. Based on the inventive concept, a treatment scheme of the cloth inspection in the invention is designed.
Before the cloth inspection operation, the cloth inspection user needs to be trained and scheduled, and the training and scheduling method of the cloth inspection user provided by the invention is described below.
The training of the cloth inspection user specifically comprises the following steps:
In the process of verifying and training, the terminal equipment responds to the operation of a user on a graphical user interface to obtain verifying and training data.
Before training the user, the staff needs to input the collected multiple verification videos and the corresponding accuracy into the server so that the user can acquire the videos for watching. The cloth inspection video contains standard samples of various cloths and flaws, the positions and types of the flaws on the picture are recorded in advance, and the positions and types of the flaws are used as learning prompts or reference answers. The cloth inspection video can be obtained from a network or recorded when the cloth inspection method provided by the invention is used, the cloth inspection video is not particularly limited, and the cloth inspection video can be obtained according to actual conditions.
The server screens the cloth inspection video input by the staff according to whether the cloth inspection accuracy reaches a preset value, and numbers and stores the cloth inspection video after screening. The server screens out videos with the accuracy rate of more than or equal to 95% for numbering and storing, and can ensure the accuracy of calculating the false alarm rate and the false alarm rate of the cloth inspection user. The preset value is set by a worker, the size of the preset value is not limited specifically, and the preset value can be set according to actual conditions.
In the step, after the verification video is stored in the server, the verification user can be trained, the first priori verification user needs to input an account password on the terminal to log in the system, and the terminal sends the account password to the server for verification. The user can be ensured to operate by logging in by using the account number password, and the account number can be corresponding to the class after training of the user.
After the verification of the server is passed, an instruction for allowing login is sent to the terminal, and the verification user can further operate.
The method comprises the steps that a cloth inspection user selects the number and the video playing speed of a cloth inspection video on a user graphical interface on a terminal, the terminal sends the cloth inspection video number and the video playing speed to a server, the server searches the corresponding cloth inspection video according to the number and then sends the corresponding cloth inspection video to the terminal, and the cloth inspection user can watch the cloth inspection video on the terminal by using the playing speed selected by the user and marks flaws. Illustratively, the user is unfamiliar with the cloth inspection at the beginning, the cloth inspection speed can be selected to be 10 meters per minute, and the actual video playing speed is reduced to 0.5 times of the speed. After the people become familiar step by step, the normal speed can be adjusted to 20 meters per minute, and the video is 1 time speed. While the video is being played, the cloth inspection user marks directly on the screen, for example, when a flaw is found, the screen is stopped, and the type and the position of the flaw are marked. Thus, the terminal device can acquire the verification data.
The terminal equipment sends the verification data to the server, wherein the verification data is used for acquiring rating information of the user, and the rating information comprises the rating of the user, video playing speed, missing report rate and false report rate corresponding to the rating. And the server acquires the rating information of the user according to the verification data.
In the step, after the cloth inspection user marks the type and the position of the flaw on the terminal, the terminal sends cloth inspection data to the server, wherein the cloth inspection data comprises the type and the position of the flaw, the server can calculate the missing report rate and the false report rate of the cloth inspection user, then the cloth inspection user is rated according to the video playing speed, the missing report rate and the false report rate, and after the rating, the mapping relation between the account number and the grade of the cloth inspection user is established and stored, so that the cloth inspection user can be searched according to the grade when personnel dispatch. For example, the correspondence between the video playing speed, the missing report rate, and the false report rate and the level of the user for verification may be as shown in table 1:
TABLE 1
It should be noted that, table 1 only illustrates, in an exemplary manner, the correspondence between the video playing speed, the missing report rate, the false report rate, and the level of the user for checking the cloth, and is not limited to this correspondence, and in the actual application process, the correspondence may be set according to the actual needs.
The rating information of each cloth inspection user comprises the rating of the cloth inspection user, the video playing speed, the missing report rate and the false report rate corresponding to the rating, and the rating information is obtained according to the cloth inspection data of the cloth inspection user in the cloth inspection training process.
After the training of the cloth inspection user is finished, when a production enterprise needs to inspect the cloth of the cloth inspection user, the cloth inspection user can be scheduled.
The scheduling of the verification user specifically comprises the following steps:
the terminal equipment sends the cloth inspection requirement information to the server through a factory cloth inspection platform on the terminal equipment, and the server receives the cloth inspection requirement information sent by the factory cloth inspection platform, wherein the cloth inspection requirement information comprises the required rating of the cloth inspection users and the number of the cloth inspection users.
In the step, when a manufacturer needs to use the cloth inspection users trained by the scheme, the cloth inspection users can be selected through the terminal, the cloth inspection requirement information is sent to the server through the factory cloth inspection platform, and then the server can select the cloth inspection users according to the cloth inspection requirement information.
For example, during cloth detection, images of the same cloth can be sent to a plurality of cloth inspection users, flaw identification and inspection are independently performed on line, and the quality of the cloth can be guaranteed as the number of people is increased. And the manufacturer can schedule personnel according to the order of the factory, for example, the factory needs 10 cloth inspection users according to the month of the order, the month of february needs 20 cloth inspection users, and the quantity, qualification, missing report and false report rate of different cloth inspection users can be divided according to the quality level of the cloth. For example, the correspondence between the cloth inspection user, the false alarm rate, and the false alarm rate and the cloth quality level may be as shown in table 2:
TABLE 2
| Quality grade of cloth |
Cloth inspection user |
Rate of missing report |
False alarm rate |
| Special grade |
4 Persons (advanced) |
1% |
1% |
| Top-quality products |
2 Persons (above middle grade) |
10% |
10% |
| In general |
1 Person (above primary) |
20% |
20% |
It should be noted that, table 1 only illustrates the correspondence between the cloth inspection user, the missing report rate, the false report rate, and the cloth quality level in an exemplary manner, and the correspondence is not limited, and in the actual application process, the correspondence may be set according to the actual needs.
And the server acquires at least one target cloth verification user meeting the cloth verification requirement information from the currently selectable cloth verification users according to the pre-acquired rating information of each user person and the cloth verification requirement information.
In the step, after the server acquires the cloth inspection requirement information, the cloth inspection users can be selected according to the ratings of the cloth inspection users and the number of the cloth inspection users in the cloth inspection requirement information, and the rating information of the cloth inspection users is stored in the server after the training process of the cloth inspection users is completed, so that the target cloth inspection users meeting the cloth inspection requirement can be found according to the rating information.
It should be noted that when there is only one rating information stored in the server, the number of target verification users selected according to the verification requirement is only one, and when there is a plurality of rating information stored in the server, the number of target verification users can be selected according to the verification requirement.
And the server sends the verification task to the terminal equipment corresponding to the at least one target verification user. And the terminal equipment receives the cloth verification task sent by the server and displays the cloth verification task on the graphical user interface.
In the step, after the server selects the target cloth inspection user, the cloth inspection task can be sent to the terminal corresponding to the cloth inspection user, so that the terminal equipment displays the cloth inspection task on a user graphical interface and displays the cloth inspection task to the cloth inspection user, and the cloth inspection user performs cloth inspection according to the cloth inspection task.
After training and scheduling of the cloth inspection user are completed, the manufacturer can let the cloth inspection user participate in cloth inspection. The cloth inspection processing method provided by the invention is described below.
An exemplary application scenario of the cloth inspection processing method provided by the present invention is shown in fig. 1, where the application scenario includes a terminal device 11, a switch 12, a switch 13, a server 14, at least one industrial camera (2 industrial cameras are shown in fig. 1, and are respectively an industrial camera 15 and an industrial camera 16), an encoder 17, a coding machine 18, a cloth 19, a light source 20, a controller 21, and a rotating shaft 22. Wherein at least one industrial camera (2 industrial cameras are shown in the first figure, namely an industrial camera 15 and an industrial camera 16 respectively) and a light source 20 form an optical acquisition system, and the optical acquisition system, an encoder 17, a coding machine 18, cloth 19, a controller 21 and a rotating shaft 22 form a cloth inspecting machine.
The terminal device 11 may transmit a start cloth inspection instruction, a stop cloth inspection instruction, and flaw marking information to the server through the switch 12 and the switch 13, and may receive a marked cloth image from the server through the switch 12 and the switch 13, for example.
The switch 12 and the switch 13 may be connected through a network to transmit network communication signals, and the terminal device 12 and the server 14 may implement data transmission through the switch 12 and the switch 13.
The server 14 can receive the start cloth inspection instruction, the stop cloth inspection instruction and the flaw marking information from the terminal through the switch 12 and the switch 13, can send the marked cloth image through the switch 12 and the switch 13, can receive the cloth image to be processed sent by the industrial camera 15 in the cloth inspection machine, can receive the cloth length signal sent by the encoder 17 in the cloth inspection machine, can send the start cloth inspection instruction and the stop cloth inspection instruction to the controller 21, and can also send the marking instruction to the coding machine 18.
The industrial camera 15 may acquire an image of the cloth to be processed and send the image to be processed to the server 14.
The encoder 17 can collect the cloth length information and send a cloth length signal to the server 14;
The code printer 18 includes a one-dimensional moving platform and an automatic label attaching device, and is capable of receiving a marking instruction from the server 14 and attaching labels to the cloth 19 according to the marking instruction.
The light source 20 can illuminate the cloth 19 to increase the brightness of the cloth 19.
The controller 21 comprises a programmable logic control (Programmable Logic Controller, abbreviated as PLC) system of the cloth inspecting machine, and comprises starting and stopping control of the rotating shaft of the cloth inspecting machine. The controller 21 may receive the start cloth inspection instruction and the stop cloth inspection instruction from the server 14, and control the rotation and stop of the rotation shaft 22 according to the start cloth inspection instruction and the stop cloth inspection instruction.
The rotating shaft 22 can be controlled by the controller 21 and can also drive the cloth 19 to move and stop.
It should be noted that, fig. 1 is only a schematic diagram of an application scenario provided by the embodiment of the present invention, and the embodiment of the present invention does not limit the actual form and specific number of the various devices included in fig. 1, nor limit the positional relationship and interaction manner between the devices in fig. 1, and in the specific application of the scheme, the embodiment may be set according to the actual requirement.
It should be understood that the terminal device is a device capable of running corresponding operation software of the electronic device, and may be a computer, or may be another intelligent terminal such as a smart phone.
The technical scheme of the invention is described in detail through specific embodiments. It should be noted that the following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
Fig. 2a is a schematic flow chart of an embodiment of a cloth inspection processing method provided by the present invention, as shown in fig. 2a, the cloth inspection processing method specifically includes the following steps:
And S201, responding to the operation of a start verification button on a graphical user interface by a user, and sending a start verification instruction to a server.
In this step, when the user needs to perform cloth inspection, the user needs to click a cloth inspection starting button on a user graphical interface of the terminal device, and the terminal device sends a cloth inspection starting instruction to the server, wherein the cloth inspection starting instruction is used for controlling the cloth inspection machine to start the rotating shaft and start cloth inspection.
S202, sending a cloth inspection starting instruction to the cloth inspection machine.
In the step, after receiving a cloth starting instruction sent by the terminal equipment, the server sends the cloth starting instruction to the cloth inspecting machine, and after receiving the cloth starting instruction, the cloth inspecting machine starts the rotating shaft to start cloth inspecting.
After the cloth inspection machine receives a cloth inspection starting instruction sent by the server, the rotating shaft of the cloth inspection machine is controlled by the controller to start rotating, so that the cloth to be processed is driven to move, at the moment, the industrial camera on the cloth inspection machine starts working, the cloth to be processed is shot, and the shot cloth image to be processed is sent to the server. Meanwhile, an encoder on the cloth inspecting machine starts to work, cloth length information is collected, a cloth length signal is sent to the server, the cloth length signal contains the cloth length information, and the cloth length information is used for calculating coordinate information of flaws by the server. Fig. 2b is a schematic diagram of an encoder according to the present invention, and as shown in fig. 2b, the encoder may obtain the cloth length information by rotating a roller thereon.
It should be noted that, the shooting of the cloth to be processed by the industrial camera may be video shooting or continuous image shooting, and the embodiment of the invention does not limit the specific shooting mode of the industrial camera and can be set according to actual situations.
The to-be-processed cloth image shot by the industrial camera can be each frame of image in a video shot by the industrial camera to be processed cloth, can be continuous images shot by the industrial camera, can be images shot by the industrial camera at a certain time interval, can be connected to cover all the cloth, and the method for obtaining the to-be-processed cloth image from the shot content of the industrial camera is not particularly limited and can be set according to actual conditions.
S203, receiving the cloth image to be processed sent by the cloth inspection machine.
S204, inputting the cloth image into a pre-acquired cloth detection model, and determining the flaw position and flaw type in the cloth image.
In the above steps, after receiving the cloth image to be processed and the cloth length signal sent by the cloth inspection machine, the server inputs the cloth image to be processed into a cloth detection model obtained in advance, so as to obtain the flaw position and flaw type in the cloth image, and after determining the flaw position and flaw type, the server can mark the flaw position and flaw type.
The cloth detection model is an image recognition model which is obtained by performing model training by using a cloth image set with calibration flaws and a convolution network before cloth inspection operation and is used for recognizing cloth images to determine positions and types of the cloth flaws.
Exemplary, fig. 2c is a diagram of a defect type provided by the present invention, which is hook filament, hole, warp and heterochromatic from left to right, and the embodiment of the present invention does not limit the specific type of defect.
And S205, marking the cloth image according to the flaw position and flaw type in the cloth image to obtain a marked cloth image.
S206, sending the marked cloth image to the terminal equipment of the user.
In the above steps, after determining the flaw position and flaw type in the cloth image, the server can mark the flaw so that after sending to the terminal equipment, the user can check the flaw, after finishing marking, the cloth image and the cloth length information can be sent to the terminal equipment, and the cloth length information received by the terminal can be displayed on the user graphical interface for reminding the user.
When there is no flaw in the cloth image, the image does not need to be marked, and the image at this time is also referred to as a marked image.
Fig. 2d is an exemplary view of a marked cloth image provided by the present invention, as shown in fig. 2d, with 5 roving-type flaws and one stain-type flaws marked and their corresponding positions. It should be noted that fig. 2d is only an example of a piece of cloth image after marking, and the marking method and the type of the flaw are not specifically limited in the embodiment of the present invention, and may be selected according to practical situations.
S207, responding to the operation of a user on a cloth checking stopping button on the graphical user interface, and sending a cloth checking stopping instruction to the server.
In the step, after the terminal equipment receives the marked cloth image sent by the server, the marked cloth image is displayed to a user through a user graphical interface, the user can judge whether the type or the position of the marked flaw is correct or not and whether the flaw is missing or not, if the marked flaw is not found in the marked cloth image and the flaw is missing, the cloth inspection is not required to be stopped, if the marked flaw is found in the marked cloth image or the flaw is missing, the user needs to click a cloth inspection stopping button on the user graphical interface, the terminal equipment sends a cloth inspection stopping instruction to the server, so that the server sends a cloth inspection stopping instruction to the cloth inspection machine, and the cloth inspection machine stops a rotating shaft.
And S208, sending a cloth inspection stopping instruction to the cloth inspection machine.
After receiving the cloth inspection stopping instruction sent by the terminal equipment, the server sends the cloth inspection stopping instruction to the cloth inspection machine, and the cloth inspection machine stops the rotating shaft after receiving the cloth inspection stopping instruction.
After the cloth inspection machine receives the cloth inspection stopping instruction sent by the server, the rotation of the rotating shaft of the cloth inspection machine is controlled by the controller to stop, so that cloth stops moving.
According to the cloth inspection processing method provided by the embodiment of the invention, the cloth inspection machine is started by manual remote control, the cloth inspection machine sends the shot cloth image to be processed to the server, the server inputs the image into the cloth inspection model to obtain the flaw position and the flaw type, marks the flaw position and the flaw type, then sends the marked image to the terminal equipment, a user can check whether the mark is correct or not and whether the mark is missing or not, and then the user sends a cloth inspection stopping instruction to the server through the terminal, and the server sends the cloth inspection stopping instruction to the cloth inspection machine to realize remote control of cloth inspection machine to stop cloth inspection. According to the invention, the manual and cloth detection model is used for jointly inspecting cloth, and the cloth can be inspected remotely by manual work, so that the cloth inspection cost is effectively reduced, and the cloth inspection efficiency is improved.
Fig. 3a is a schematic flow chart of a second embodiment of the cloth inspection processing method provided by the present invention, as shown in fig. 3a, on the basis of the foregoing embodiment, after the server sends a cloth inspection stopping instruction to the cloth inspection machine, the cloth inspection processing method further includes the following steps:
and S301, transmitting flaw marking information to a server in response to the position and the type of at least one flaw manually marked by a user on the graphical user interface.
In the step, after the cloth inspection machine stops rotating shafts, a user can further mark the marked cloth images through a user graphic interface, mark the positions and types of flaws, and the terminal sends flaw information to the server so that the server can calculate flaw coordinate information according to the flaw marking information.
In the following cases, the user needs to perform a further marking operation on the marked cloth image.
In the first case, the positions and types of the flaws marked in the marked cloth image are correct and no flaws are missed, at this time, the further marking operation of the user is a confirmation marking operation, and the terminal equipment can send flaw marking information to the server according to the confirmation marking operation.
And secondly, marking the correct flaw position and type by a user in a further marking operation of manually marking the flaw position or type in the marked cloth image, and performing a marking confirmation operation.
And thirdly, marking the positions and types of flaws in the marked cloth image correctly but missing flaws, wherein the further marking operation of a user is to manually mark the missing flaw positions and types, and the marking operation is confirmed.
And fourthly, marking the correct flaw position and type and the missing flaw position and type in the marked cloth image by a user, and performing a confirmation marking operation.
And fifthly, marking the cloth image without marked flaws but with missing flaws, wherein the further marking operation of the user is to manually mark the missing flaw positions and types, and the marking operation is confirmed.
Fig. 3b is an exemplary schematic diagram of a user marking a flaw, as shown in fig. 3b, where a user marks a rectangular frame at a flaw position, and fig. 3b is merely an example of a piece of cloth image after marking, and the embodiment of the present invention does not specifically limit a flaw marking method and flaw types, and may be selected according to practical situations.
S302, calculating coordinate information of at least one flaw according to the flaw marking message.
In this step, after the server receives the flaw marking information from the terminal device, the coordinate information of the flaw can be calculated according to the flaw marking information and the cloth length signal sent by the encoder on the cloth inspecting machine, and then the cloth inspecting machine can mark the flaw according to the coordinate information of the flaw.
The following illustrates the calculation of coordinate information of flaws:
Before cloth inspection starts, the scale factors of an industrial camera are required to be acquired, and the abscissa of the flaw on the cloth can be calculated according to the pixels of the flaw in the image and the scale factors. For example, fig. 3c is a schematic diagram of a scale factor calculation provided in the present invention, as shown in fig. 3c, a piece of A4 paper may be stuck on a piece of cloth, and an image is obtained by photographing the piece of A4 paper with an industrial camera, and by using parameters of the image, we can obtain the width of the pixel occupied by the A4 paper, which may be set to w. While the true physical size of A4 paper is known to be 297 mm. Thus, the scale factor of the camera acquisition system is s=297/w, S representing the scale factor.
For example, the location of the flaw in the flaw mark information is determined by a rectangular frame, so that the center pixel k of the rectangular frame can be calculated, and then the transverse coordinate x=k×s of the flaw on the cloth can be calculated, and meanwhile, the length signal is queried to obtain the length of the cloth where the flaw is located, namely, the longitudinal coordinate value Y. After the position coordinates of the flaw are obtained, the position coordinates XY of the flaw are stored in a server.
The above examples are merely examples of a method for obtaining a scale factor, a method for determining a location of a flaw, and a method for calculating coordinate information of a flaw, and the present invention is not limited specifically to the method for obtaining a scale factor, the method for determining a location of a flaw, the method for calculating coordinate information of a flaw, and the reverse direction, and may be selected according to practical situations.
S303, sending a marking instruction to the cloth inspection machine according to the coordinate information of at least one flaw.
In the step, after the server calculates the coordinate information of the flaw, a marking instruction can be sent to the cloth inspecting machine, the marking instruction is used for controlling the cloth inspecting machine to mark the flaw on the cloth, and the marking instruction comprises the coordinate and the type of the flaw.
After the cloth inspection machine receives the marking instruction, a coding machine on the cloth inspection machine finds the position of the flaw according to the horizontal and vertical coordinates of the flaw, and marks the position of the flaw.
According to the cloth inspection processing method provided by the embodiment of the invention, the flaws determined by the cloth detection model are further confirmed, errors in the flaws determined by the cloth detection model are modified, missing flaws are marked, flaw marking information is sent to the server, the server can calculate coordinate information of the flaws, the server sends the coordinate information to the cloth inspection machine, and the cloth inspection machine can mark the cloth according to the coordinate information.
Fig. 3d is a schematic diagram of the relationship between two cloth inspection modes provided by the invention, as shown in fig. 3d, the invention adopts a mode of combining manual cloth inspection and cloth inspection of a cloth inspection model to perform cloth inspection, and the manual cloth inspection is taken as a main mode and the cloth inspection model is taken as an auxiliary mode, so that the accuracy rate of cloth inspection is increased, the cost of cloth inspection is reduced, and the cloth inspection efficiency is improved.
Fig. 4a is a schematic flow chart of a third embodiment of a cloth inspection processing method provided by the present invention, as shown in fig. 4a, on the basis of the above embodiment, before sending a cloth inspection start instruction to a server in response to an operation of a cloth inspection start button on a graphical user interface by a user, the cloth inspection processing method further includes the following steps:
S401, acquiring a pre-calibrated cloth image set, wherein the cloth image set comprises a plurality of calibrated cloth images marked with flaw positions and flaw types of cloth.
In the cloth inspection process, a cloth detection model which is acquired in advance is used for inspecting cloth, so that the cloth detection model is acquired before cloth inspection, and the cloth detection model is obtained by training according to a cloth image set which is calibrated in advance and a convolution network model.
In the step, before model training is performed, a pre-calibrated cloth image set is required to be acquired, the cloth image set comprises a plurality of calibrated cloth images marked with flaw positions and flaw types of cloth, and the convolutional neural network can perform learning training according to the flaw positions and flaw types in the calibrated cloth images.
It should be noted that, the pre-calibrated cloth image may be a cloth image with positions and types of flaws marked in the above embodiment, or may be obtained from a network, and the embodiment of the present invention does not limit the obtaining manner of the pre-calibrated cloth image, and may be selected according to practical situations.
Fig. 4b is a schematic view of a pre-calibrated cloth image provided by the present invention, and as shown in fig. 4b, flaws of the flaw types of holes and stains and flaw positions are marked in the image. It should be noted that fig. 4b is only an example of a pre-calibrated cloth image, and the embodiment of the present invention does not specifically limit the calibrated cloth image, and may be selected according to practical situations.
And S402, performing model training by adopting a convolution network model according to the cloth image set to obtain a cloth detection model.
In the step, after a cloth image set is obtained, a cloth detection model can be obtained by training a convolutional network model, and the position and the type of the cloth flaw can be identified and determined by inputting the cloth image into the cloth detection model.
The training process of the cloth inspection model is illustrated below:
The cloth image set contains at least 100 marked cloth images of each flaw, and before model training is carried out, the marked cloth images are required to be converted into marked files.
By way of example, the following is an example of a markup document:
The code is an example of a labeling document, in which a cloth image is labeled with a width of 800 pixels and a height of 600 pixels, two flaws exist, the flaw type is a hole, the minimum abscissa of a rectangular frame where one flaw is located is 353 pixels, the maximum ordinate is 439 pixels, the minimum abscissa of a rectangular frame where the other flaw is located is 383 pixels, and the maximum ordinate is 599 pixels.
It should be noted that, the above examples are only examples of the annotation file, and the embodiment of the invention does not limit the annotation file, and can be set according to actual situations.
The convolutional network model is selected as a YOLO network, the labeling file is imported into the YOLO network for iterative training, the training is stopped until the convergence drop rate is lower than 0.001 at least 2000 times, a trained cloth detection model is obtained, and the position and the type of the flaw can be obtained by inputting the cloth image into the cloth detection model. It should be noted that, the embodiment of the present invention does not specifically limit the convolutional network model and the training process, and may be selected according to the actual situation.
It should be noted that the above training process is only an example of the training process of the cloth detection model, and the embodiment of the invention does not specifically limit the training process of the cloth detection model, and can be set according to actual situations.
According to the cloth inspection processing method provided by the embodiment of the invention, the cloth detection model is obtained through training the pre-calibrated cloth image and the convolution network model, and then the cloth inspection is performed through the cloth detection model, so that the cloth inspection cost can be reduced, and the cloth inspection efficiency can be improved.
The following are examples of the apparatus of the present invention that may be used to perform the method embodiments of the present invention. For details not disclosed in the embodiments of the apparatus of the present invention, please refer to the embodiments of the method of the present invention.
Fig. 5 is a schematic structural diagram of a first embodiment of a cloth inspection processing apparatus according to the present invention, and as shown in fig. 5, the cloth inspection processing apparatus 50 includes:
A receiving module 51, configured to receive a cloth image to be processed sent by a cloth inspection machine in a cloth inspection process;
The detection module 52 is configured to input the cloth image into a pre-acquired cloth detection model, determine a flaw position and a flaw type in the cloth image, and identify the cloth image by the cloth detection model to determine the position and the type of the cloth flaw;
the processing module 53 is configured to mark the cloth image according to the flaw position and the flaw type in the cloth image, so as to obtain a marked cloth image;
A sending module 54, configured to send the marked cloth image to a terminal device of a user;
the sending module 54 is further configured to send a stop cloth inspection instruction to the cloth inspection machine after receiving the stop cloth inspection instruction sent by the terminal device, where the stop cloth inspection instruction is an instruction generated by the terminal device in response to the operation of the user and used to control the cloth inspection machine to stop the rotating shaft.
Further, the receiving module 51 is further configured to receive a start-verification instruction sent by the terminal device.
Further, the sending module 54 is further configured to send a start cloth inspection instruction to the cloth inspection machine.
Further, the receiving module 51 is further configured to obtain a pre-calibrated cloth image set, where the cloth image set includes a plurality of calibrated cloth images of flaw positions and flaw types of marked cloth.
Further, the processing module 53 is further configured to perform model training by using a convolutional network model according to the cloth image set, so as to obtain the cloth detection model.
Further, the receiving module 51 is further configured to receive a flaw marking message sent by the terminal device, where the flaw marking message includes a location and a type of at least one flaw that is manually marked by the user.
Further, the processing module 53 is further configured to calculate coordinate information of the at least one flaw according to the flaw marking message.
Further, the sending module 54 is further configured to send a marking instruction to the cloth inspection machine according to the coordinate information of the at least one flaw, where the marking instruction includes the coordinate information of the at least one flaw, and the marking instruction is used to control the cloth inspection machine to mark the at least one flaw on the cloth.
Further, the receiving module 51 is further configured to receive the verification requirement information sent by the factory verification platform, where the verification requirement information includes a rating of a required verification user and a number of verification users.
Further, the processing module 53 is further configured to obtain, according to the pre-obtained rating information of each user person and the verification requirement information, at least one target verification user satisfying the verification requirement information from currently selectable verification users.
Further, the processing module 53 is further configured to send a verification task to a terminal device corresponding to the at least one target verification user.
In one possible design of the embodiment of the invention, the rating information of each cloth inspection user comprises the rating of the cloth inspection user, and the video playing speed, the missing report rate and the false report rate corresponding to the rating, wherein the rating information is obtained according to the cloth inspection data of the cloth inspection user in the cloth inspection training process.
The cloth inspection processing device is used for executing the technical scheme of the server in any method embodiment, the implementation principle and the technical effect are similar, the cloth inspection processing device is connected with the terminal equipment of a user and the cloth inspection machine through the server, manual remote cloth inspection can be achieved, cloth inspection is conducted in the server by using a cloth detection model, the cloth inspection cost is effectively reduced, and the cloth inspection efficiency is improved.
Fig. 6a is a schematic structural diagram of a second embodiment of a cloth inspection processing apparatus according to the present invention, and as shown in fig. 6a, the cloth inspection processing apparatus 60 includes:
a receiving module 61, configured to receive a marked piece goods image sent by a server in a cloth inspection process, where the marked piece goods image includes a flaw position and a flaw type that are automatically detected and marked by the server;
A display module 62 for displaying the marked cloth image on a graphical user interface;
And the sending module 63 is used for responding to the operation of a user on the cloth checking stopping button on the graphical user interface and sending a cloth checking stopping instruction to the server, wherein the cloth checking stopping instruction is used for controlling the instruction of the cloth inspecting machine to stop the rotating shaft.
Further, the sending module 63 is further configured to:
And responding to the operation of a start cloth inspection button on the graphical user interface by a user, and sending a start cloth inspection instruction to the server, wherein the start cloth inspection instruction is used for controlling the cloth inspection machine to start a rotating shaft and start cloth inspection.
Further, the sending module 63 is further configured to:
And sending flaw marking information to the server in response to the position and type of at least one flaw manually marked on the graphical user interface by the user, wherein the flaw marking information comprises the position and type of the at least one flaw.
Further, the receiving module 61 is further configured to receive a verification task sent by the server, and display the verification task on the graphical user interface.
Fig. 6b is a schematic structural diagram of a third embodiment of the cloth inspection processing apparatus according to the present invention, and as shown in fig. 6b, the cloth inspection processing apparatus 60 further includes:
and the processing module 64 is used for responding to the operation of the user on the graphical user interface in the cloth inspection training process to acquire cloth inspection data.
Further, the sending module 63 is further configured to send the verification data to the server, where the verification data is used to obtain rating information of the user, and the rating information includes a rating of the user, and a video playing speed, a missing report rate and a false report rate corresponding to the rating.
The cloth inspection processing device is used for executing the technical scheme of the user's terminal equipment in any method embodiment, the implementation principle and the technical effect are similar, the user can inspect cloth through the terminal equipment, and the flaw position and the flaw type determined by the cloth detection model can be inspected, so that the cloth inspection cost is effectively reduced, and the cloth inspection efficiency is improved.
Fig. 7 is a schematic structural diagram of a server according to the present invention. As shown in fig. 7, the server 70 includes:
a processor 71, a memory 72, and a communication interface 73;
the memory 72 is for storing executable instructions of the processor 71;
Wherein the processor 71 is configured to execute the solution of the server in any of the method embodiments described above via execution of the executable instructions.
Alternatively, the memory 72 may be separate or integrated with the processor 71.
Alternatively, when the memory 72 is a device independent from the processor 71, the server 70 may further include:
and a bus for connecting the devices.
The implementation principle and technical effects of the technical solution on the server side in the embodiment of any of the foregoing methods are similar, and are not described herein.
Fig. 8 is a schematic structural diagram of a terminal device according to the present invention. As shown in fig. 8, the service terminal apparatus 80 includes:
A processor 81, a memory 82, a display 83 and a communication interface 84;
the memory 82 is used for storing executable instructions of the processor;
Wherein the processor 81 is configured to execute the technical solution on the terminal device side in any of the foregoing embodiments by executing the executable instructions.
Alternatively, the memory 82 may be separate or integrated with the processor 81.
Optionally, when the memory 82 is a device independent from the processor 81, the terminal device 80 may further include:
and a bus for connecting the devices.
The technical scheme of the terminal device for executing any of the foregoing method embodiments is similar to the implementation principle and technical effect, and will not be described herein.
The embodiment of the invention also provides a readable storage medium, on which a computer program is stored, which when executed by a processor, implements the technical scheme provided by any one of the foregoing embodiments on the server or terminal device side.
The embodiment of the invention also provides a computer program product, which comprises a computer program, wherein the computer program is used for realizing the technical scheme of the server or the terminal equipment side provided by any one of the method embodiments when being executed by a processor.
Those of ordinary skill in the art will appreciate that all or a portion of the steps of implementing the various method embodiments described above may be implemented by hardware associated with program instructions. The foregoing program may be stored in a computer readable storage medium. The program, when executed, performs the steps comprising the method embodiments described above, and the storage medium described above includes various media capable of storing program code, such as ROM, RAM, magnetic or optical disk.
It should be noted that the above embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that the technical solution described in the above embodiments may be modified or some or all of the technical features may be equivalently replaced, and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiments of the present invention.