JP2018180631A - Information processing apparatus, information processing method, program, and shipping inspection system - Google Patents

Abstract

Even if a deviation occurs in the process of assembling parts, it is possible to prevent a decrease in product search accuracy at the time of shipment. An information processing apparatus according to an embodiment of the present invention is manufactured with registration image generation means for generating an image of a product at the time of shipment as a registration image based on part information and post-process information included in product order information. A band detection unit that detects a band from a captured image of a product, and when a band is detected by the band detection unit, an image search is performed on the registered image for each of the whole image and the component image generated from the captured image. Thus, the product specifying means for specifying the product is provided, and the component image includes a band image and a content image obtained by removing the band image from the entire image. [Selection] Figure 8

Description

  The present invention relates to an information processing apparatus, an information processing method, a program, and a shipping inspection system mainly used for shipping inspection.

  Conventionally, in the commercial printing industry, various products are manufactured by combining an image forming apparatus and a post-processing apparatus. For example, by applying post-processing (post-processing) such as bookbinding and cutting to printed materials on which printing processing has been performed by an image forming apparatus, products in various formats such as books, catalogs, flyers, tickets, and business cards are produced There is. Then, when shipping the manufactured product, it is carried out that the product is specified at the time of shipping, and the delivery note corresponding to the specified product is printed on demand.

  Here, if product identification information (for example, a bar code or the like) is printed on the product at the time of shipping, the product can be identified by reading the identification information. In addition, for products such as flyers, tickets, business cards, etc., for which identification information is difficult to assign due to design or space, an image obtained by imaging the produced product and converting the imaged image from print data The product can be identified by comparing with.

  However, in the method of imaging a product and specifying the product, when the print data is directly converted into an image and compared with the converted image, if the physical layout of the product is changed by post-processing, Both images may not match, and the search accuracy may be degraded. Therefore, in order to prevent such a decrease in search accuracy, according to Patent Document 1, print data is converted into an image in consideration of a physical layout, and an image obtained by converting the image and an image obtained by capturing a product An image forming apparatus is disclosed that identifies the product by comparing.

JP, 2015-148900, A

  However, in the case of a product manufactured by assembling a part such as a band or a jacket, if a shift occurs in the process of assembling the part, the image of the product will be the image even if the image of print data is converted in consideration of the physical layout. It may not match the image of the print data. That is, in the image forming apparatus described in Patent Document 1, if a shift occurs in the process of assembling the parts, the search accuracy may be reduced.

  The present invention has been made in view of the above-described conventional problems, and an object thereof is to prevent a decrease in product search accuracy at the time of shipment even when a deviation occurs in the process of assembling parts. It is.

  In order to achieve the above object, an information processing apparatus according to the present invention generates registered image generation means for generating an image of the product at the time of shipment as a registered image based on part information and post-process information included in order information of the product Band detection means for detecting a band from the captured image of the manufactured product, and when the band is detected by the band detection means, the registration is performed for each of the entire image and the part image generated from the captured image Product specifying means for specifying the product by executing an image search for an image, and the part image includes a band image and a content image obtained by removing the band image from the entire image It features.

  According to the present invention, even when misalignment occurs in the process of assembling parts, it is possible to prevent a reduction in the product search accuracy at the time of shipment.

BRIEF DESCRIPTION OF THE DRAWINGS It is a figure which shows the whole structure of the shipping inspection system which concerns on embodiment of this invention. It is a figure showing hardware constitutions of an information processor concerning an embodiment of the present invention. It is a figure showing software composition of an information processor concerning an embodiment of the present invention. It is a figure showing an example of order information registered into order information storage part of an information processor concerning an embodiment of the present invention. It is a figure which shows an example of the information memorize | stored in image DB of the information processing apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the registration image produced | generated by the image generation part of the information processing apparatus which concerns on embodiment of this invention. It is a flowchart which shows the flow of the process at the time of order registration in the information processing apparatus which concerns on embodiment of this invention. It is a flow chart which shows a flow of processing at the time of shipment inspection in an information processor concerning an embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Example of configuration of shipping inspection system)
FIG. 1 is a diagram showing an entire configuration of a shipping inspection system according to an embodiment of the present invention. The entire configuration of the shipping inspection system shown in FIG. 1 is shown to facilitate understanding of the present invention, and the configuration is not limited to this. As shown in FIG. 1, the shipping inspection system is configured, for example, by connecting an information processing apparatus 101, an image forming apparatus 102, and a post-processing apparatus 103 to a network 100 such as the Internet.

  The information processing apparatus 101 executes various programs, for example, a job management program for managing production of a printed matter, a Web server, and a program such as a shipping inspection program and a bar code reading program used at the time of shipping. Further, in the information processing apparatus 101, a printer driver program or the like for controlling the image forming apparatus 102 is installed.

  The image forming apparatus 102 analyzes print data including print data transmitted from the information processing apparatus 101 or the like, converts each page into a dot image, and executes print processing. The printed matter printed by the image forming apparatus 102 is handed to the post-processing apparatus 103 by hand-carried by the print operator, or conveyed to the post-processing apparatus 103 by a belt conveyor (not shown).

  The post-processing apparatus 103 performs post-processing such as binding processing, punching processing, and folding processing on the printed matter printed by the image forming apparatus 102 based on a job (that is, an instruction) transmitted from the information processing apparatus 101. Run. In FIG. 1, the post-processing apparatus 103 is not physically connected to the image forming apparatus 102, but can communicate with the information processing apparatus 101 and the image forming apparatus 102 via the network 100. Although the information processing apparatus 101, the image forming apparatus 102, and the post-processing apparatus 103 are illustrated as a single apparatus in FIG. 1, each can be configured by a plurality of apparatuses.

(Example of software configuration of information processing apparatus)
FIG. 2 is a diagram showing a hardware configuration of the information processing apparatus 101 according to the embodiment of the present invention. The information processing apparatus 101 can be configured by a general personal computer.

  In FIG. 2, a CPU 200 is a central processing unit that controls the entire information processing apparatus 101. The CPU 200 executes programs stored in a program ROM in the ROM 202 and programs such as an OS (Operation System) and application programs loaded from an external memory 210 such as a hard disk to the RAM 201. That is, when the CPU 200 executes various programs stored in a readable storage medium, the CPU 200 functions as each processing unit that executes the processing of each flowchart to be described later.

  A random access memory (RAM) 201 is a main memory of the CPU 200 and functions as a work area or the like. A ROM (Read Only Memory) 202 includes a program ROM and a data ROM, and stores a program executed by the CPU 200. The keyboard controller 203 controls operation input from the keyboard 208, a barcode reader (not shown), and a pointing device (mouse, touch pad, touch panel, track ball, etc.).

  The display controller 204 controls the display of the display 209. The disk controller 205 controls data access to the external memory 210 such as a hard disk (HD) or flexible disk (FD) that stores various data. The camera controller 206 controls imaging data input from the camera 211. A network controller (NC) 207 is connected to the network and executes communication control processing with other devices connected to the network.

(Example of software configuration of information processing apparatus)
FIG. 3 is a diagram showing the software configuration of the information processing apparatus 101 according to the embodiment of the present invention, and FIG. 3 shows each function of the information processing apparatus 101. The control unit 300 controls execution of the functional blocks 301 to 309 of the information processing apparatus 101.

  The order information management unit 301 controls the process of registering or changing the order information shown in FIG. 4 described later in the information processing apparatus 101. Specifically, the order information management unit 301 analyzes the order information received from the upper layer application such as MIS or the server one line by one line, and passes the analysis result for each line via the control unit 300. It is stored in the information storage unit 302. The analysis result is composed of a file path of print data, part information, post-process (post-process) information, and the like.

  The order information storage unit 302 stores the analysis result of the order information. The order information storage unit 302 may store the information in the RAM 201 or may save the file in the external memory 210. Alternatively, it may be stored in a data storage program such as a database.

  The image generation unit 303 acquires order information from the order information storage unit 302, and executes editing and composition of print data of the component based on the component information and post-process information to register the registered image to be registered in the image DB 305. Generate For example, when a product is produced by cutting printed printed matter like a ticket, the print data is cut out as a product based on the information in the cutting process, thereby generating a registered image according to the product. Also, in the case of a product such as a wireless binding and binding including a band and a jacket, print data of the band and jacket are cut out based on the information of the cut crease process, and a cover is printed based on the information of the wireless binding process and the three-way cutting process Cut out the data. Then, the print data of the cut-out band, the jacket, and the cover is synthesized based on the information of assembly to generate an entire image. Furthermore, the band image and the content image are generated by dividing the entire image into the band portion and the content portion not including the band.

  The image registration unit 304 delivers the registered image generated by the image generation unit 303 to the feature amount extraction unit 309 to acquire the feature amount of the registered image. Then, when acquiring the feature amount of the registered image, the image registration unit 304 registers the registered image, the feature amount of the registered image, and the OrderID for identifying the product associated with the registered image in the image DB 305. That is, the image registration unit 304 corresponds to the storage control unit of the present invention.

  The image DB 305 stores the registered image registered by the image registration unit 304, the feature amount of the registered image, and the OrderID associated with the registered image for identifying the product. For example, in the case of registering the entire image, the band image, and the content image, three images are registered in association with the same OrderID, together with their feature quantities for each. The imaging unit 306 generates a captured image by controlling start / stop of imaging of a printed material in the camera 211. The captured image can be generated in a file format such as Bitmap or JPEG, and can be stored as image data in the RAM 201.

  The band detection unit 307 detects whether or not a band is included in the product in the image of the product captured by the imaging unit 306. For example, the band detection unit 307 performs edge detection processing on an image of a product captured by the imaging unit 306 and acquires an area of the product, and at the same time, an edge parallel to and the same length as an edge indicating an outer frame of the product It is determined that there is a band (ie, a band is detected) when Alternatively, if a list of only band images is received from the images registered in the image DB 305, and the band image included in the list is used as a template image, the captured image is scanned for a target, and there is a matching region When it is judged, it is detected that there is a band.

  First, when the band detection unit 307 detects that a band is included in the captured product, the search unit 308 detects the entire product image, the band image, and the content image from the image of the product captured by the imaging unit 306. Generate Next, the search unit 308 delivers the generated entire image, band image, and content image of the product to the feature extraction unit 309 to obtain the feature extracted from the images. Then, the search unit 308 executes an image search for each image using the acquired feature amount, and specifies a product based on the information registered in the image DB 305. That is, the search unit 308 corresponds to the product specifying means of the present invention.

  When no band is detected in the imaged product, the search unit 308 first generates an entire product image from the image of the product imaged by the imaging unit 306 and delivers it to the feature quantity extraction unit 309. Then, the feature quantity extracted from the whole image is acquired. Next, the search unit 308 executes an image search on the entire image using the acquired feature amount, and specifies a product based on the information registered in the image DB 305.

  The order information management unit 301 performs shipping completion registration of the product by receiving the information of the product searched (specified) by the search unit 308. In addition, after identifying the product, the search unit 308 deletes the information linked to the identified product from the image DB 305. A feature amount extraction unit 309 extracts feature amounts from the image.

(Example of order information registered in information processing apparatus)
FIG. 4 is a view showing an example of order information registered in the order information storage unit 302 of the information processing apparatus 101 according to the embodiment of the present invention. In FIG. 4, column 400 stores OrderIDs that identify each of the products that make up the order information. In addition, in column 401, the product name corresponding to each order, in column 402, the part information that constitutes the product, in column 403, post-process information necessary for producing the product, and in column 404, the part that constitutes the product Each file path of print data is stored.

(Example of information stored in image DB)
FIG. 5 is a diagram showing an example of information stored in the image DB 305 of the information processing apparatus according to the embodiment of the present invention. The column 500 stores an OrderID to be associated with the registered image registered in the order information storage unit 302. A row 501 stores the file path of the registered image registered by the image registration unit 304, and a row 502 stores the feature amount of the registered image extracted by the feature amount extraction unit 309.

(Example of generation of registered image)
FIG. 6 is a view showing an example of the registered image generated by the image generation unit 303 based on the order information stored in the order information storage unit 302 of the information processing apparatus 101 according to the embodiment of the present invention. For example, in the case of a product including a band such as a photo book, the registered image 605 of the entire product from the cover 601, the jacket 602, and the band 603, the registered image 606 of the content in which the band portion of the registered image 605 of the entire product is masked, and the band Is generated. On the other hand, in the case of a product such as a ticket or the like consisting only of the text 604, only the entire registered image 608 is generated.

(Order information registration flow in the information processing apparatus 101)
FIG. 7 is a flow of processing from reading order information to storing the order information and further registering a registered image generated based on the order information in the information processing apparatus 101 according to the embodiment of the present invention Is a flowchart showing The flowchart illustrated in FIG. 7 is realized, for example, by the CPU 200 reading a program stored in the ROM 202 into the RAM 201 and executing the program.

  When the information processing apparatus 101 receives an order (ie, receives the order information), the order information management unit 301 analyzes the order information one line by one line, and the analysis result for each line is stored in the order information storage unit 302. It memorizes (S701). Here, as described above, the analysis result is the information of the product constituting the order information, and is, for example, the file path of the print data, the part information, the post process information, and the like.

  When the analysis result of the order information is stored in the order information storage unit 302, the image generation unit 303 acquires component information from the analysis result and determines whether or not the component includes a band (S702). Then, when it is determined by the image generation unit 303 that a band is included in the part, the information processing apparatus 101 proceeds to step S703, and the image generation unit 303 determines that the part is not included in the part. And the process proceeds to step S704.

  If it is determined that the part includes a band, the image generation unit 303 generates an image by editing the print data of the band based on the post-process information, and adds the image to the target image list for registered image generation (S703) ). For example, in the case where a band cut-crease process is included as a post-process, the band print data is cut out based on the information of the cut-crease process to generate a band image.

  Next, the image generation unit 303 acquires component information from the analysis result of the order information stored in the order information storage unit 302, and determines whether a jacket is included in the component (S704). Then, when the image generation unit 303 determines that the part includes a jacket, the information processing apparatus 101 proceeds to step S705, and the image generation unit 303 determines that the part does not include a jacket. And the process proceeds to step S706.

  If it is determined that the part includes a jacket, the image generation unit 303 generates an image by editing print data of the jacket based on the post-process information, and adds the image to the target image list for registered image generation (S705) ). For example, in the case where the cut-crease process of the jacket is included as a post-process, the print data of the jacket is cut out based on the information of the cut-crease process to generate an image.

  Next, the image generation unit 303 acquires post-process information from the analysis result of the order information stored in the order information storage unit 302, and determines whether the product is bound by wireless binding (S706). If the image generation unit 303 determines that the product is not bound by wireless binding, the process proceeds to step S 707, and the image generation unit 303 determines that the product is bound by wireless binding. The process then proceeds to step S 708.

  If it is determined that the product is not bound by perfect binding, the image generation unit 303 generates an image by editing the body print data based on the post-process information, and adds the image to the target image list for registered image generation (S707). ). For example, when the saddle-stitch binding process is included as a post-process, an image is generated by dividing the first page of the text print data into right and left and cutting each of the divided pages as a front cover and a back cover.

  In addition, when it is determined that the product is bound by wireless binding, the image generation unit 303 generates an image by editing the print data of the front cover based on the post-process information, and the target image list for registration image generation is generated. Add (S708). For example, when a perfect binding process is included as a post-process, the first page of print data of the cover is divided into right and left excluding the spine width, and the divided pages are cut out as a cover and a back cover. Generate

  Next, the image generation unit 303 determines whether a band image is included in the target image list for registered image generation (S709). If the information processing apparatus 101 determines that the band image is not included in the target image list for registered image generation, the process proceeds to step S710, and the band image is included in the target image list for registered image generation. If it is determined, the process proceeds to step S711.

  If it is determined that the part does not contain a band, the image generation unit 303 generates a registered image of the entire product by combining the part images included in the target image list based on the assembly information (S710). For example, when the product is bound by perfect binding and includes a text, a cover, and a jacket, the image generation unit 303 generates an image of the product at the time of shipment based on the generated jacket image.

  If the image generation unit 303 determines that the product includes a band, it uses the partial image included in the target image list based on the assembly information to register the registered image of the band, the registered image of the content, and the entire registered image of the product. It generates (S711). Specifically, the image generation unit 303 generates a registered image of the entire product by combining the component images included in the target image list, and further, the image portion of the band from the entire registered image of the product The registered image of the masked content and the registered image of the band are generated together.

  The feature amount extraction unit 309 extracts a feature amount from the registered image generated by the image generation unit 303 (S712). The image registration unit 304 associates the registered image and the feature amount with the OrderID and registers them in the image DB 305 (S713). Specifically, the image registration unit 304 associates the entire image, the content image, the band image, and their feature amounts with a predetermined OrderID and registers them.

(Flow at the time of shipment inspection in the information processing device)
In FIG. 8, in the information processing apparatus 101 according to the embodiment of the present invention, after imaging a product at the time of shipping inspection, the product is identified using an image of the imaged product, and the shipping completion process of the product is further performed. It is a flow chart until execution. The flowchart illustrated in FIG. 8 is realized, for example, by the CPU 200 reading out a program stored in the ROM 202 to the RAM 201 and executing the program.

  The imaging unit 306 controls so as to perform imaging of a printed matter (that is, a product), and generates captured image data (S801). The band detection unit 307 determines whether or not a band is included in the product from the generated captured image data (S802). When it is determined by the band detection unit 307 that the product does not contain a band (that is, when no band is detected), the information processing apparatus 101 causes the process to proceed to step S803, and the product includes a band. If it is determined that it is determined, the process proceeds to step S804.

  When the band is not detected by the band detection unit 307, the search unit 308 performs an image search on the image registered in the image DB 305, using the entire image of the imaged product as an input, and specifies the product. The search unit 308 searches for a similar image, for example, using the feature amount extracted from the entire image of the imaged product by the feature amount extraction unit 309. More specifically, the search unit 308 calculates the similarity for all the images included in the image DB 305. Then, the search unit 308 identifies an image having the highest calculated similarity as a matching product.

  In step S804, when the band is detected by the band detection unit 307, the search unit 308 performs an image search on the image registered in the image DB 305, with the entire image of the captured product, the content image, and the band image as an input. Run. The search unit 308 searches, for example, the similar image using the feature amount extracted from each of the captured whole image, content image, and band image of the product by the feature amount extraction unit 309. More specifically, by performing similar image search using feature amounts for each of the entire image, content image, and band image, the similarity to all the images included in the image DB 305 of the entire image, content image, and band image Each degree is calculated. Then, the calculated similarity is weighted for each of the entire image, the content image, and the band image and added up, and a product with the highest added similarity is specified as the imaged product.

  In addition, in order to specify this product, when registering an image in the image DB 305, the type of the image is divided and registered for each whole image, content image and band image, and the type of whole image, content image and band image at the time of image search It is also possible to perform a search on images that match. Also, as a matter of course, the search can be performed on all the images without dividing the types of the images. When the order information management unit 301 receives the OrderID of the product specified by the search unit 308, the order information management unit 301 executes shipping completion processing of the corresponding product (S805).

  As described above, according to the information processing apparatus of the present invention, even if a shift occurs in the process of assembling parts, it is possible based on the feature amounts of the entire image and the partial image (content image, band image) Since each image is compared individually, it is possible to prevent a decrease in search accuracy.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. Can also be realized. It can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

Claims (10)

Registered image generation means for generating an image of the product at the time of shipment as a registered image based on parts information and post-process information included in order information of the product;
Band detection means for detecting a band from a captured image of the manufactured product;
A product specification for specifying the product by performing an image search for the registered image with respect to each of the entire image and the component image generated from the captured image when the band is detected by the band detection unit Equipped with means and
An information processing apparatus characterized in that the component image includes a band image and a content image obtained by removing the band image from the entire image.   The product specifying unit specifies the product by executing an image search for the registered image with respect to the entire image generated from the captured image when the band is not detected by the band detection unit. An information processing apparatus according to claim 1, characterized in that.   The said registration image generation means produces | generates the whole image of the said product, a band image, and a content image as said registration image, when a band is contained in the said components information. Information processing device.   The information processing apparatus according to claim 1, wherein the registered image generation unit generates an entire image of the product as the registered image when no band is included in the component information. Feature quantity extraction means for extracting a feature quantity of an image from the registered image and the captured image;
5. A storage control means for controlling so as to store the registered image and the feature amount extracted from the registered image by the feature amount extracting means in a storage medium. An information processing apparatus according to item 1.   The information processing apparatus according to claim 5, wherein the product specifying unit executes an image search by comparing a feature amount of the registered image with a feature amount of the captured image. When a band is detected by the band detection means,
The product specifying means specifies the product from the result of weighting each of the similarities calculated by comparing the feature amounts of the entire image, the content image, and the band image. An information processing apparatus according to Item 6. A registered image generation step of generating an image of the product at the time of shipment as a registered image based on the part information and the post process information included in the order information of the product by the registered image generation means;
A band detection step of detecting a band from a captured image of the manufactured product by the band detection means;
When the band is detected in the band detection step by the product specifying unit, the product search is performed on the registered image for each of the entire image and the component image generated from the captured image. Product identification steps to identify
The information processing method characterized in that the component image includes a band image and a content image obtained by removing the band image from the entire image.   A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 7. Registered image generation means for generating an image of the product at the time of shipment as a registered image based on parts information and post-process information included in order information of the product;
Band detection means for detecting a band from a captured image of the manufactured product;
A product specification for specifying the product by performing an image search for the registered image with respect to each of the entire image and the component image generated from the captured image when the band is detected by the band detection unit An information processing apparatus comprising: product specifying means including: a band image; and a content image obtained by removing the band image from the entire image.
An image forming apparatus that executes print processing based on print data of the order information;
A post-processing device that performs post-processing on the printed matter printed by the image forming apparatus based on post-process information of the order information;
A shipment inspection system comprising: an imaging device for imaging a product on which post-processing has been performed by the post-processing device;

Images