JP2018160184A - Information processing apparatus and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing device and a program capable of standardizing and improving efficiency of appraisal work. SOLUTION: A POS terminal 1 has a three-dimensional shape and a surface of an article from a distance image including an image of the article obtained by imaging the article to be evaluated and distance information indicating a distance in the depth direction of the article. The generation means for generating a three-dimensional model representing the state of the article, the acquisition means for acquiring the identification information for identifying the article from the captured image of the article, the identification information of each article, and the reference state of the article. From the storage unit that stores the represented three-dimensional model in association with each other, the specific means for specifying the article corresponding to the identification information acquired by the acquisition means, the three-dimensional model of the article specified by the specific means, and the acquisition means It is provided with a detection means for comparing the acquired three-dimensional model and detecting a difference in appearance between the two three-dimensional models, and an appraisal means for outputting the difference detected by the detection means as an appraisal result. [Selection diagram] Fig. 3

Description

  Embodiments described herein relate generally to an information processing apparatus and a program.

  Conventionally, in stores such as second-hand goods purchase shops and recycle shops, salesclerks appraise the purchase price by judging the type and condition of goods brought in by customers. Such appraisal (assessment) work requires specialized knowledge and experience, and there may be individual differences in judgment criteria depending on the store clerk. In particular, visual inspection is an important factor in pricing, so there is a need for leveling and more efficient appraisal operations.

  On the other hand, conventionally, an appearance state of an article is inspected based on a captured image obtained by imaging the article. For example, in the field of trading cards, a technique for identifying a trading card from data representing the state of the surface of the trading card read by a scanner or the like has been proposed.

  However, articles that are subject to appraisal in stores are not limited to flat objects such as trading cards, but also include three-dimensional objects, and are therefore difficult to use for the appraisal work described above.

  The problem to be solved by the embodiment of the present invention is to provide an information processing apparatus and program capable of leveling and improving the efficiency of appraisal work.

  The information processing apparatus according to the embodiment uses a three-dimensional shape of the article from a distance image including a captured image of the article obtained by imaging the article to be verified and distance information indicating a distance in the depth direction of the article. Generating means for generating a three-dimensional model representing the state of the surface; acquisition means for acquiring identification information for identifying the article from the captured image of the article; identification information for each article; From the storage unit that stores the three-dimensional model representing the state to be associated with each other, a specifying unit that specifies an article corresponding to the identification information acquired by the acquiring unit, and the tertiary of the article specified by the specifying unit The original model and the three-dimensional model acquired by the acquisition unit are compared, a detection unit that detects a difference in appearance between the two-dimensional models, and a difference detected by the detection unit is output as an appraisal result. It includes an appraisal means.

FIG. 1 is a block diagram illustrating an example of a hardware configuration of a POS terminal according to the embodiment. FIG. 2 is a diagram showing an example of the data configuration of the reference information file shown in FIG. FIG. 3 is a block diagram illustrating an example of a functional configuration of the POS terminal according to the embodiment. FIG. 4 is a flowchart illustrating an example of data processing executed by the POS terminal according to the embodiment.

  Hereinafter, an information processing apparatus and a program according to an embodiment will be described with reference to the drawings. In the present embodiment, an example in which the information processing apparatus and the program are applied to a POS (Point Of Sale) terminal installed in a store such as an old book store or a recycle shop that purchases books will be described. However, the present embodiment is limited to this embodiment. Shall not be.

  FIG. 1 is a block diagram illustrating an example of a hardware configuration of the POS terminal 1 according to the present embodiment. The POS terminal 1 includes a control unit 10 that functions as a computer of the POS terminal 1. The control unit 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, and a RAM (Random Access Memory) 13 which are examples of processors.

  An operation unit 14, a display unit 15, and a storage unit 16 are connected to the CPU 11 via various input / output circuits. These are controlled by the CPU 11.

  The operation unit 14 includes an input device such as a keyboard and a pointing device such as a mouse, and receives an operation input from a store clerk serving as an operator. The display unit 15 includes a display device such as a liquid crystal display, and displays various types of information under the control of the CPU 11.

  The storage unit 16 is an example of a storage unit. The storage unit 16 includes a storage device such as a hard disk drive (HDD) or a solid state drive (SSD), and stores various programs executed by the CPU 11 and various setting information. For example, the storage unit 16 stores, as an example of setting information, a reference information file F1 that stores reference information relating to book assessment (appraisal).

  FIG. 2 is a diagram illustrating an example of a data configuration of the reference information file F1. As shown in FIG. 2, the reference information file F <b> 1 is associated with identification information that can identify each book, and includes various information such as the book name of the book, the name of the publisher that publishes the book, the reference price, and the reference state. Store information.

  The identification information is an identification code such as an ISBN code or a book JAN code. In addition, for a book or the like that does not have an identification code, information that can identify the book, such as an image representing a cover of the book (for example, a texture image described later), may be stored. In addition, when using a texture image, it is good also as a structure which uses the texture image memorize | stored in the reference | standard state mentioned later as identification information.

  The reference price is information indicating the amount of money that serves as a reference for the book when the purchase price is assessed. As the reference price, for example, a fixed price of the book or an arbitrary price such as a market price can be set. In addition, it is preferable to set the amount of money according to the state of the book set to the reference state as the reference price.

  The reference state is information representing a state that becomes a reference of a book when appraising the book. In the present embodiment, a reference state of a book that is a three-dimensional object is expressed using a three-dimensional model such as 3D-CAD (Computer Aided Design) or 3D-CG (Computer Graphics). The three-dimensional model is data formed by pasting a two-dimensional texture image representing the state of the surface on each surface of the three-dimensional shape information representing the three-dimensional shape (cuboid shape) of the book. The three-dimensional model in the reference state is preferably created based on a book in a good state with no pain or defect such as a new article or a beautiful article.

  Note that the reference information file F1 is not limited to the data configuration of FIG. 2, and may store other information. The reference information file F1 is preferably held in an updatable state. For example, the reference information file F1 may be provided periodically or as needed from a server device (not shown) that performs maintenance of the reference information file F1.

  Returning to FIG. 1, the communication unit 17, the imaging unit 18, the printer 19, and the like are connected to the CPU 11 through various input / output circuits.

  The communication unit 17 is a communication interface that can be connected to a network such as a LAN (Local Area Network) or a WAN (Wide Area Network). The communication unit 17 performs data communication with a server device (not shown) via, for example, a network.

  The imaging unit 18 is an imaging device such as an RGB-D camera or a stereo camera that can capture an image of a subject and also record a distance (Depth) of the subject in the depth direction. The imaging unit 18 images a book to be verified (assessed) as a subject, and outputs a distance image including an image (captured image) of the book and distance information indicating a distance in the depth direction.

  In addition, the imaging part 18 shall be provided in the position and form which can image a book. For example, the imaging unit 18 may be provided in a state of being supported by a stand such as a tripod so that a book can be imaged from above. Further, for example, the imaging unit 18 may be a hand-held imaging device that can be captured by a store clerk in hand.

  The printer 19 is a printing device such as a thermal printer. Under the control of the CPU 11, the printer 19 prints transaction details relating to the purchase of a book on a predetermined sheet.

  Next, the functional configuration of the POS terminal 1 will be described. FIG. 3 is a block diagram illustrating an example of a functional configuration of the POS terminal 1.

  As shown in FIG. 3, the control unit 10 (CPU 11) of the POS terminal 1 cooperates with the program stored in the storage unit 16, thereby acquiring the capture unit 101, the shape information acquisition unit 102, and the texture image acquisition unit 103. The combination unit 104, the identification unit 105, the state determination unit 106, the assessment unit 107, the display control unit 108, the purchase processing unit 109, and the like are realized as functional units.

  The capturing unit 101 outputs an imaging-on signal to the imaging unit 18 and causes the imaging unit 18 to start an imaging operation. The capturing unit 101 captures the distance image captured by the image capturing unit 18 and stored in the RAM 13.

  The shape information acquisition unit 102 acquires shape information representing the three-dimensional shape of the subject (book) from the distance image captured by the capture unit 101. Specifically, the shape information acquisition unit 102 acquires shape information representing the three-dimensional shape of the book that is the subject based on the distance information included in the distance image. The shape information acquisition method is not particularly limited, and for example, a known technique such as a three-dimensional scanner or RealSense (registered trademark) can be used. For example, the shape information acquisition unit 102 may detect a rectangular parallelepiped shape from the distance image and acquire the rectangular parallelepiped shape as shape information of the book.

  The texture image acquisition unit 103 acquires a texture image representing the surface state of the subject (book) from the distance image captured by the capture unit 101. Specifically, the shape information acquisition unit 102 extracts an image portion representing a book surface as a texture image from the captured image included in the distance image. Note that a texture image extraction method is not particularly limited, and a known technique such as edge detection can be used. For example, the shape information acquisition unit 102 may extract an image portion of a rectangle or a trapezoid area as a texture image of a book in cooperation with the shape information acquisition unit 102 or the like.

  The combining unit 104 is an example of a generation unit. The synthesizing unit 104 synthesizes the shape information acquired by the shape information acquiring unit 102 and the texture image acquired by the texture image acquiring unit 103 to generate a three-dimensional model of the book that is the subject. Specifically, the synthesis unit 104 generates a three-dimensional model representing the appearance of the book by pasting the texture image acquired for the surface to each surface of the three-dimensional shape represented by the shape information. Further, the synthesis unit 104 generates a three-dimensional model of a rectangular parallelepiped shape that conforms to the actual shape and size of the book by correcting distortion (such as trapezoidal distortion) in the shape information and texture information. Note that a known technique such as a three-dimensional scanner can be used as a method for generating a three-dimensional model and a method for correcting distortion.

  Note that when a three-dimensional model is generated, the book is imaged from a plurality of directions such as the front and back surfaces and the side surfaces of the book. Therefore, the synthesizing unit 104 may display a guide screen (not shown) that guides the position and direction for imaging the book on the display unit 15 in cooperation with the display control unit 108. Then, when the shape information and the texture image are acquired for each guided direction, the synthesis unit 104 starts generating a three-dimensional model using the shape information and the texture image.

  The identification unit 105 is an example of an acquisition unit and an identification unit. The identifying unit 105 identifies the book that is the subject based on the identification information of each book registered in the reference information file F1 and the identification information included in the texture image acquired by the texture image acquiring unit 103.

  For example, the specifying unit 105 acquires a book identification code or a book name from the texture image acquired by the texture image acquisition unit 103 as a search key. And the specific | specification part 105 searches the reference | standard information file F1 for the book which contains the acquired search key in identification information or a book name, and specifies an applicable book. Note that a known technique such as OCR (Optical Character Recognition) can be used to acquire (read) the identification code and the book name.

  Further, for example, the specifying unit 105 acquires a feature amount that represents a feature on the appearance of the texture image from the texture image acquired by the texture image acquisition unit 103. Then, the specifying unit 105 compares the acquired feature amount with the feature amount of the texture image of each book in the reference information file F1, and specifies the book based on the similarity. It should be noted that known techniques such as pattern matching and general object recognition can be used for feature amount extraction and similarity determination. When this specifying method is adopted, the feature amount of the texture image (for example, cover) of each book may be stored in advance in the reference information file F1.

  The state determination unit 106 is an example of a detection unit and an appraisal unit. The state determination unit 106 compares the three-dimensional model of the book generated by the combining unit 104 with the reference state (three-dimensional model) of the book specified by the specifying unit 105, and the difference in appearance between the three-dimensional models. Detect points. Moreover, the state determination part 106 outputs the detected difference as an appraisal result of the book of appraisal object.

  In addition, when comparing the three-dimensional models, the state determination unit 106 compares each of the same side and the same surface of the book, such as comparing the covers. In addition, the state determination unit 106 may perform normalization so that the size of any one side or the size of any one surface is the same in order to directly compare the sizes of the three-dimensional models. .

  For example, the state determination unit 106 detects the size difference (difference) by comparing the sizes of the sides or the surfaces constituting both the three-dimensional models. Then, the state determination unit 106 outputs an appraisal result in which the detected numerical value, the detected difference type (size), and the part information indicating the side or the surface where the difference is detected are associated with each other.

  In addition, for example, the state determination unit 106 detects a difference in the state of the surface of the book represented by both the three-dimensional models by comparing the texture images of the surfaces constituting both the three-dimensional models. Specifically, the state determination unit 106 detects scratches, tears, dirt, sunburn, etc. present in the book to be verified. Then, the state determination unit 106 identifies the type of the detected difference (scratches, dirt, etc.), the part information indicating the part where the difference is detected, and the index value quantifying the degree of difference. Output the result.

  It should be noted that the method of discriminating the type of difference is not particularly limited, and various methods such as pattern recognition can be employed. For example, the state determination unit 106 compares the feature information that stores appearance features (shape, color, etc.) such as scratches, tears, dirt, and sunburn with the appearance features of the location where the difference is detected. Thus, the type of difference may be determined. Further, the state determination unit 106 may normalize the color tones representing both the three-dimensional models, and may simultaneously capture a reference color as a reference such as a color chart when generating the reference state and the three-dimensional model. In this case, the synthesis unit 104 may correct the tone of the generated three-dimensional model to a predetermined tone based on the reference color included in the texture image. Thereby, since the color tone between both three-dimensional models can be matched, for example, color differences such as sunburn can be detected with high accuracy.

  Further, for example, the state determination unit 106 compares the three-dimensional models to detect a difference between the uneven shapes of the books represented by the three-dimensional models. Specifically, the state determination part 106 detects the dent which exists in the surface of the book of appraisal object. Then, the state determination unit 106 outputs an appraisal result in which the detected difference type (dent), the part information indicating the part where the difference is detected, and the index value quantifying the degree of the difference are output. To do.

  The quantification of the degree of difference is performed based on a predetermined standard (hereinafter referred to as a quantification standard). The quantification standard can be set for each type of difference, for example. In this embodiment, the quantification standard is such that the degree of scratches, the depth of dents, the size of dirt, the amount of change in color, etc. is large, that is, the greater the difference from the reference state, the larger the index value. Is set to be derived.

  The difference in size may be a value as it is as an index value, or may be converted into an index value by the same method as described above. The quantification standard may be common to all books or may be set individually for each book. In the latter case, the quantification reference may be set (stored) in association with the identification information of each book stored in the reference information file F1, for example.

  The assessment unit 107 is an example of assessment means. The assessment unit 107 assesses the purchase price of the book that has become the subject based on the reference price of the book identified by the identification unit 105 and the appraisal result of the state determination unit 106.

  Specifically, the assessment unit 107 calculates an amount to be reduced from the reference price (hereinafter referred to as an adjustment amount) based on an index value of each difference included in the appraisal result. Then, the assessment unit 107 assesses (calculates) the purchase amount of the book by subtracting the adjustment amount from the reference price of the book identified by the identification unit 105. Then, the assessment unit 107 registers (stores) in the RAM 13 or the storage unit 16 purchase data that associates the identification information or book name of the assessed book with the purchase price of the book.

  Here, the calculation method of the adjustment amount is not particularly limited, and various methods can be employed. For example, the assessment unit 107 may calculate the adjustment amount for each type of difference based on a data table in which each index value is associated with the amount of reduction corresponding to the index value. In this case, the assessment unit 107 identifies the amount of reduction corresponding to each index value included in the appraisal result, and uses the sum of those amounts as the adjustment amount. Note that the data table may be shared by all types of differences, or may be prepared for each type of differences.

  For example, when the index value itself represents the amount of money to be reduced, the assessment unit 107 adds the index value included in the state information and sets the total value as the adjustment amount.

  The display control unit 108 causes the display unit 15 to display the purchase amount calculated by the assessment unit 107. Specifically, the display control unit 108 displays, on the display unit 15, the book name, the reference price, and the like of the book that has been assessed based on the purchase data registered by the specifying unit 105 in association with the purchase amount. To do. In addition, when a plurality of purchase data is registered, the display control unit 108 displays a purchase price and the like for each purchase data, and displays the total amount of the purchase price on the display unit 15. Thereby, the store clerk can easily confirm the purchase price of the book brought in by the customer. Note that the display control unit 108 may display the appraisal result of the state determination unit 106 on the display unit 15 in association with the corresponding book.

  The purchase processing unit 109 executes purchase processing for purchasing a book with the purchase price assessed by the assessment unit 107 when execution of the purchase processing is instructed via the operation unit 14 or the like. In the purchase process, the purchase processing unit 109 displays a message on the display unit 15 for instructing to give the customer the money for the purchase amount in cooperation with the display control unit 108. Further, when an operation for instructing the completion of money transfer is performed, the purchase processing unit 109 records the purchase data and the contents of the money transfer as one commercial transaction. Further, the purchase processing unit 109 causes the printer 19 to issue a sheet on which a breakdown of the purchase process is printed as a receipt.

  Hereinafter, the operation of the POS terminal 1 will be described with reference to FIG. FIG. 4 is a flowchart illustrating an example of data processing executed by the POS terminal 1.

  First, the capturing unit 101 outputs an imaging on signal to the imaging unit 18 to start an imaging operation by the imaging unit 18 (step S11). Accordingly, the capturing unit 101 captures the distance image captured by the image capturing unit 18 and stored in the RAM 13 (step S12). At this time, the store clerk images the book that is the subject of the appraisal (assessment) with the imaging unit 18.

  Subsequently, the shape information acquisition unit 102 and the texture image acquisition unit 103 acquire shape information and a texture image from the distance image acquired by the acquisition unit 101 (step S13). Specifically, the shape information acquisition unit 102 acquires shape information of the book imaged by the imaging unit 18 based on the distance image acquired by the acquisition unit 101. Further, the texture image acquisition unit 103 acquires the texture image of the book imaged by the imaging unit 18 based on the distance image captured by the capturing unit 101.

  Subsequently, the synthesizing unit 104 generates a three-dimensional model of the book imaged by the imaging unit 18 by synthesizing the shape information acquired in step S13 and the texture image (step S14). Further, the specifying unit 105 specifies the book imaged by the imaging unit 18 from among the books registered in the reference information file F1 based on the identification code acquired from the texture image (step S15).

  Subsequently, the state determination unit 106 compares the three-dimensional model of the book generated in step S14 with the reference state (three-dimensional model) of the book specified in step S15, and the book imaged by the imaging unit 18 Is determined (verified) (step S16).

  Further, the assessment unit 107 assesses the purchase price of the book imaged by the imaging unit 18 based on the reference price of the book specified in step S15 and the appraisal result in step S16 (step S17). Next, the display control unit 108 causes the display unit 15 to display the assessment result of step S17 (step S18).

  Subsequently, the control unit 10 determines whether or not the end of the assessment has been instructed via the operation unit 14 or the like (step S19). Here, when the reading of the next book is instructed (step S19; No), the control unit 10 returns the process to step S12, thereby starting appraisal and assessment of the next book.

  On the other hand, when the end of the assessment is instructed in step S19 (step S19; Yes), the purchase processing unit 109 executes a purchase process for the assessed book (step S20). Then, the purchase processing unit 109 issues a receipt on which a breakdown of the purchase process is printed (step S21), and the process ends.

  As described above, the POS terminal 1 represents the state of the book imaged by the imaging unit 18 as a three-dimensional model, and compares the difference with the pre-stored three-dimensional model of the reference state as an appraisal result. The output POS terminal 1 assesses the purchase price of the book imaged by the imaging unit 18 based on the appraisal result. Thereby, since the POS terminal 1 can automatically perform appraisal of a book that is a three-dimensional object and appraisal of the purchase price, it is possible to achieve leveling and efficiency of appraisal work.

  As mentioned above, although embodiment of this invention was described, the said embodiment was shown as an example and is not intending limiting the range of invention. The above embodiment can be implemented in various other forms, and various omissions, replacements, changes, additions, and the like can be made without departing from the scope of the invention. Moreover, the said embodiment and its deformation | transformation are included in the range of the invention, the summary, and the invention described in the claim, and its equal range.

  For example, in the above embodiment, the reference information file F1 is configured to be included in the POS terminal 1. However, the present invention is not limited to this. For example, it is good also as a structure with which other apparatuses, such as a server apparatus and a cloud system connected with the POS terminal 1 by the network, are equipped.

  Moreover, although the said embodiment detected the point inferior to a reference state as a difference with a reference state, and demonstrated the example subtracted from a reference price based on the appraisal result, it shall not be restricted to this. Specifically, as a difference from the reference state, a point superior to the reference state may be detected, and the price may be increased to the reference price based on the appraisal result. In this case, the purchase price considering both characteristics can be assessed by properly using the sign (+, −) of the index value according to the characteristic of the difference to be detected.

  Moreover, in the said embodiment, although the object of appraisal (assessment) was made into the book, not only this but another article | item may be made into object.

  In addition, the program executed by the information processing apparatus of the above embodiment is provided by being incorporated in advance in a storage medium (ROM or storage unit) included in the information processing apparatus, but is not limited to this and can be installed. Alternatively, an executable file may be recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk). . Furthermore, the storage medium is not limited to a medium independent of a computer or an embedded system, but also includes a storage medium in which a program transmitted via a LAN or the Internet is downloaded and stored or temporarily stored.

  Further, the program executed by the information processing apparatus of the above embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. It may be configured to be provided or distributed via.

DESCRIPTION OF SYMBOLS 1 POS terminal 10 Control part 18 Image pick-up part 101 Capture part 102 Shape information acquisition part 103 Texture image acquisition part 104 Composition part 105 Specification part 106 State determination part 107 Assessment part 108 Display control part 109 Purchase process part F1 Reference | standard information file

JP 2005-259101 A

Claims (6)

A three-dimensional representation of the three-dimensional shape and surface state of the article from a distance image including a captured image of the article obtained by imaging the article to be verified and distance information indicating the distance in the depth direction of the article. Generating means for generating a model;
Acquisition means for acquiring identification information for identifying the article from the captured image of the article;
A specifying means for specifying an article corresponding to the identification information acquired by the acquisition means from a storage unit that stores the identification information of each article in association with a three-dimensional model representing a reference state of the article; ,
Detecting means for comparing the three-dimensional model of the article specified by the specifying means with the three-dimensional model acquired by the acquiring means and detecting a difference in appearance between the three-dimensional models;
Appraisal means for outputting the difference detected by the detection means as an appraisal result;
An information processing apparatus comprising: The storage unit stores a reference amount of the article in association with the identification information of each article,
The information processing apparatus according to claim 1, further comprising an assessment unit that assesses the purchase price of the appraisal target article based on the amount of the article specified by the specifying unit and the appraisal result of the appraisal unit.   The information processing apparatus according to claim 1, wherein the detection unit detects a difference in size of the article represented by the three-dimensional model.   The information processing apparatus according to claim 1, wherein the detection unit detects a difference in a surface state of the article represented by the three-dimensional model.   The information processing apparatus according to any one of claims 1 to 4, wherein the detection unit detects a difference in the uneven shape of the surface of the article represented by the three-dimensional model. The computer of the information processing device
A three-dimensional representation of the three-dimensional shape and surface state of the article from a distance image including a captured image of the article obtained by imaging the article to be verified and distance information indicating the distance in the depth direction of the article. Generating means for generating a model;
Acquisition means for acquiring identification information for identifying the article from the captured image of the article;
A specifying means for specifying an article corresponding to the identification information acquired by the acquisition means from a storage unit that stores the identification information of each article in association with a three-dimensional model representing a reference state of the article; ,
Detecting means for comparing the three-dimensional model of the article specified by the specifying means with the three-dimensional model acquired by the acquiring means and detecting a difference in appearance between the three-dimensional models;
Appraisal means for outputting the difference detected by the detection means as an appraisal result;
Program to make it function.

Images