JP7728610B2 - Product sales data processing system - Google Patents

Description

The present invention relates to a product sales data processing system .

A system is known in which customers can register products themselves using a mobile device equipped with a camera (see, for example, Patent Document 1).

Japanese Patent Application Laid-Open No. 2018-147252

However, various problems can arise when customers scan barcodes to register products themselves.

The present invention was made in light of these circumstances, and aims to provide an appropriate solution to problems that arise when customers register products themselves.

One aspect of the present invention that solves the above-mentioned problems is a merchandise sales data processing system that registers merchandise using a mobile terminal operated by a customer in a store, and includes: a reading means for reading merchandise codes; an acquisition means for acquiring merchandise information corresponding to the merchandise code read by the reading means; a detection means for detecting a reading failure indicating that a customer has reserved a merchandise whose merchandise code has not been read by the reading means; and a display means for displaying candidate merchandise in a selectable manner when the detection means detects the reading failure, and the acquisition means acquires merchandise information for the candidate merchandise selected by the customer's selection operation.

1 is a diagram illustrating an example of the overall configuration of a product sales data processing system according to an embodiment of the present invention. FIG. 1 is a diagram showing an example of the state near the entrance of a store that has introduced a product sales data processing system according to this embodiment. 10A and 10B are diagrams illustrating examples of how a mobile terminal is set in a shopping cart according to the present embodiment. FIG. 1 is a diagram illustrating a settlement device according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of the configuration of a mobile terminal according to the present embodiment. 1 is a diagram illustrating an example of the configuration of a transaction management device according to an embodiment of the present invention. 3A to 3C are diagrams illustrating examples of customer information, store information, and cart information according to the present embodiment. FIG. 2 is an explanatory diagram illustrating classification of reserved products according to the embodiment. A sequence diagram showing an example of processing procedures executed by a mobile terminal, a settlement device, and a transaction management device according to this embodiment in response to the accounting of one transaction in self-service operation. 10A to 10C are diagrams showing examples of a start-up screen, a product registration initial screen, a product registration update screen, and a code display screen according to the present embodiment. 10A to 10C are diagrams illustrating an example of a registered product presentation screen, a product registration screen, and a guide image according to the present embodiment. 10 is a flowchart illustrating an example of an operation of the mobile terminal according to the present embodiment. 10 is a flowchart illustrating an example of an operation of the mobile terminal according to the present embodiment. 10 is a flowchart illustrating an example of an operation of the mobile terminal according to the present embodiment. 10 is a flowchart illustrating an example of the operation of the transaction management device according to the present embodiment. 10A and 10B are diagrams showing an example of a reread instruction screen and a preset key screen according to the embodiment; 10A and 10B are diagrams showing an example of a reread instruction screen and a preset key screen according to the embodiment; FIG. 10 is a diagram showing an example of a settlement screen according to the present embodiment. FIG. 10 is a diagram showing an example of a reserved product processing screen according to the embodiment. FIG. 10 is a diagram showing an example of a reserved product processing screen according to the embodiment. FIG. 10 is a diagram showing an example of an enlarged captured image area according to the embodiment. FIG. 10 is a diagram showing an example of a reserved product processing screen according to the embodiment. FIG. 10 is a diagram showing an example of an enlarged captured image area according to the embodiment. 10 is a flowchart illustrating an example of a processing procedure executed by the settlement device according to the present embodiment in relation to displaying a reserved product processing screen. FIG. 10 is a diagram showing an example of a captured image area according to a first modified example of the embodiment. FIG. 10 is a diagram showing an example of a settlement screen according to a second modified example of the present embodiment.

<Embodiment>
[Example of overall configuration of product sales data processing system]
FIG. 1 shows an example of the overall configuration of a product sales data processing system 1 according to this embodiment.
The product sales data processing system 1 shown in the figure includes a store controller 10, a transaction management device 20 (cloud server), a registration and settlement device 30, a settlement device 40, a transaction status management device 50 (e.g., a personal computer), and a mobile terminal 60. The product sales data processing system 1 shown in the figure can be introduced into various stores (supermarkets, convenience stores, drug stores, home centers, electronics retailers, etc.).

Note that, although one registered settlement device 30 is illustrated in the figure, the number of registered settlement devices 30 installed in one store may be two or more. Also, although two settlement devices 40 are illustrated in the figure, the number of settlement devices 40 in one store may be one or three or more. Also, although one transaction status management device 50 is illustrated in the figure, the number of transaction status management devices 50 in one store may be two or more. Also, there may be two or more store controllers 10 in one store.

The product sales data processing system 1 in the figure is capable of handling both a store clerk-assisted operation in which a store clerk uses a registration and settlement device 30 to register products, and a self-service operation in which a customer uses a mobile terminal 60 to register products and then uses a settlement device 40 to settle the payment.
In the following explanation, self-operation will be mainly described.

The store controller 10, registration and settlement device 30, settlement device 40, and transaction status management device 50 are installed inside the store. The store controller 10, registration and settlement device 30, settlement device 40, and transaction status management device 50 are connected to each other so that they can communicate with each other via a LAN 19 (which may be wired or wireless). The store controller 10 is capable of communicating with the transaction management device 20.

The store controller 10 is a device that mainly manages various devices installed in the store. For example, the store controller 10 has the ability to communicate with external devices, and receives product files (product master data) from external devices (such as a headquarters server (not shown) or the transaction management device 20) and supplies them to the registered settlement device 30, etc., or receives transaction information from the settlement device 40, etc. and sends it to an external device (such as the transaction management device 20, etc.).

The transaction management device 20 is a server that provides various services. For example, the transaction management device 20 manages various information corresponding to self-service operations (e.g., transaction information (cart information), etc.).

The registration and settlement device 30 is a device that performs registration and settlement processes. The registration and settlement device 30 is used by a store clerk (the registration and settlement processes are performed by the store clerk).

The settlement device 40 is a device that executes settlement processing. The settlement device 40 is used by customers. That is, the settlement device 40 executes settlement processing in response to operations by the customers.
In addition, if the payment includes items whose processing is temporarily suspended for confirmation by a store clerk (hereinafter referred to as "suspended items"; details will be given later), a store clerk will be called and the clerk (or customer) will operate the payment device 40 to confirm the suspended items.

The transaction status management device 50 is a device that mainly manages (or monitors) various devices within the LAN 19. The transaction status management device 50 manages the operating status of other devices (e.g., the registration settlement device 30, the settlement device 40, etc.), displays various information about the other devices, and outputs various commands to the other devices.

The mobile terminal 60 is a terminal used by customers in a self-service environment. In this embodiment, the mobile terminal 60 is used while attached to a shopping cart 70 in a self-service environment.

In addition to communication and imaging functions (camera), the mobile terminal 60 also has a recognition function that recognizes various codes (barcodes, two-dimensional codes, etc.). For example, the mobile terminal 60 recognizes barcodes attached to products. In other words, the mobile terminal 60 reads product identification information (product codes). The mobile terminal 60 may also extract feature points within images captured by the imaging function (images acquired as through images) and recognize objects (various codes).

The mobile terminal 60 is a smartphone or the like owned by the customer. The mobile terminal 60 may also be owned by the store and loaned to the customer when they shop. While two mobile terminals 60 are shown in the figure, the number of mobile terminals 60 used for product registration in the store varies depending on the number of customers.

A shopping application with product registration functionality compatible with the self-service operation of this embodiment is installed on the mobile terminal 60.

Figure 2 shows an example of the area near the entrance of a store that has installed the product sales data processing system 1 of this embodiment. As shown in Figure 2, rental shopping carts 70, rental shopping baskets 80, and sales shopping bags 90 are prepared near the entrance (or the entrance if the entrance and exit are separate). When a customer visits the store, they place their shopping basket 80 in the shopping cart 70, place their shopping bag 90 in the shopping basket 80, and proceed inside the store. At this time, if the customer wishes to do their own shopping, they place their mobile terminal 60 in the shopping cart 70.

The customer may bring the shopping bag 90 they purchased this time as their own bag the next time they visit the store, or they may bring a different bag than the shopping bag 90 as their own bag. They may also purchase a plastic shopping bag.

Although not shown in the figure, the shopping cart 70 is equipped with a bag for reserved items (described below) in addition to the shopping basket 80.

[Example of setting a mobile terminal in a shopping cart]
An example of how the mobile terminal 60 is set in the shopping cart 70 will be described with reference to Figure 3. Figure 3(A) is a side view of the area of the shopping cart 70 where the mobile terminal 60 is set. Figure 3(B) is a top view of the shopping cart 70.

As shown in FIG. 3(A), a holder 71 for setting the mobile terminal 60 is attached to the shopping cart 70. The customer sets the mobile terminal 60 in the holder 71. When the mobile terminal 60 is set in the holder 71, it is ready to capture an image of the entire shopping bag 90 or personal bag using the imaging unit 606 (see FIG. 4(B)), as shown in FIG. 3(B).

[About the payment device]
Figure 4 is a diagram illustrating the settlement device 40. Figure 4(A) is a diagram showing an example of the appearance of the settlement device 40 (a front view seen from the customer side). Figure 4(B) is a diagram showing an example of the configuration of the settlement device 40. In Figures 4(A) and 4(B), the same parts are given the same reference numerals.

An example configuration of the settlement device 40 shown in Figure 4(B) will be described below with reference to Figure 4(A). The settlement device 40 comprises a CPU 401, ROM 402, RAM 403, hard disk 404, display unit 405, scanner unit 406, card payment unit 408, change dispenser 409, printing unit 413, audio output unit 414, communication unit 415, and imaging unit 416 (camera). These can communicate with each other via a bus. The settlement device 40 may also be provided with a key operation unit and an alarm unit (for example, a lamp on a sign pole, etc.).

The CPU 401 is a central processing unit that controls the operation of the settlement device 40 by reading and executing programs stored in the ROM 402 .
The ROM 402 is a read-only memory that stores various types of information used by the CPU 401, including programs.

RAM 403 is a read/write memory that stores various information as a main storage device. For example, RAM 403 stores information read from ROM 402 or hard disk 404, information obtained from external sources, information generated during processing, etc.

The hard disk 404 stores various information as an auxiliary storage device. For example, the hard disk 404 may store programs executed by the CPU 401 instead of the ROM 402. Furthermore, instead of the RAM 403, the hard disk 404 may store information read from the ROM 402, information obtained from the outside, information generated during processing, etc. Note that a storage device such as an SSD (Solid State Drive) may be provided instead of the hard disk 404.

The display unit 405 is a touch display that displays various information and accepts various inputs.

The scanner unit 406 optically reads various information. For example, the scanner unit 406 scans a code (barcode, two-dimensional code, etc.) displayed on the display unit of the mobile terminal 60 and reads the information necessary for the settlement process. The scanner unit 406 may also scan a code printed on a receipt (registered trademark) issued by another device (for example, a registered settlement device 30) and read the information necessary for the settlement process. The scanner unit 406 may also scan a code attached to a product and read the product code. The scanner unit 406 may also scan a code attached to a store clerk's name tag and read the store clerk code.

The card payment unit 408 is a payment mechanism that accepts various cards (credit cards, prepaid cards such as transportation cards, cash cards for debit payments, point cards for point payments, etc.). The card payment unit 408 is equipped with a card recognition unit (reading unit), a display unit, and an operation unit, but it is sufficient if it is equipped with at least a card recognition unit. The card recognition unit may also recognize various cards (e.g., point cards, membership cards, shareholder cards, etc.) for the purpose of granting benefits, discounts, etc.

The change dispenser 409 (cash settlement unit) is a cash settlement mechanism that has an input port for banknotes and coins and an output port for banknotes and coins. It calculates the amount inserted into the input port, calculates the change amount, which is the difference between the input amount and the purchase amount, and outputs the change from the output port. When banknotes or coins are inserted into the input port, they are detected by a sensor (detecting that they have been inserted, the number of coins by denomination, etc.).

The printing unit 413 prints and issues various media (receipts, bills, etc.).
The audio output unit 414 outputs audio. For example, the audio output unit 414 outputs audio guidance and the like.
The communication unit 415 transmits and receives information to and from other devices (store controller 10, registered settlement device 30, other settlement devices 40, transaction status management device 50, etc.).

The imaging unit 416 is configured to capture, for example, the hands of a customer operating the change dispenser.

[Regarding the registration settlement device]
The registration and settlement device 30 may be configured to have a display unit and a scanner unit on each side (the clerk side and the customer side), and a payment unit (card payment unit, cash payment unit) on one side (the customer side). The registration and settlement device 30 configured in this manner may be a device that can switch between multiple operating modes (such as a mode in which the registration process is performed by a clerk and the payment process is performed by a customer, a mode in which the registration process and the payment process are performed by a customer, or a mode in which the registration process is performed by both a clerk and a customer and the payment process is performed by a customer) depending on, for example, the operating status. Furthermore, if the payment process is performed by a customer, the registration and settlement device 30 can be used for self-service operation. That is, a customer can perform the registration process on a mobile terminal 60 and then have the registration and settlement device 30 perform the payment process to complete a transaction.
However, the following description will be given as an example of a case where the settlement device 40 is used for self-service operation.

[Example of mobile terminal configuration]
5 shows an example of the configuration of the mobile terminal 60. The mobile terminal 60 in the figure includes a CPU 601, a storage unit 602, a RAM 603, a display unit 604, an operation unit 605, an imaging unit 606, and a network-compatible communication unit 607.

The CPU 601 is a central processing unit that controls the operation of the mobile terminal 60 by reading and executing programs stored in the storage unit 602 .
The storage unit 602 is an auxiliary storage device for the CPU 601 and stores various types of information used by the CPU 601, including programs.
The RAM 603 is the main storage device of the CPU 601 .

The display unit 604 displays an image in accordance with the display control of the CPU 601. The display unit 604 may be configured as a touch panel that allows operations to be performed on the screen.
The operation unit 605 collectively indicates various operators provided on the mobile terminal 60 and various input devices connected to the mobile terminal 60 .

The imaging unit 606 captures images under the control of the CPU 401. In addition, the imaging unit 606 of this embodiment can also function as a component that reads two-dimensional codes in a code reader that processes two-dimensional codes.

The network-compatible communication unit 607 is a component that communicates wirelessly via a network. The network-compatible communication unit 607 enables the mobile terminal 60 to communicate with the transaction management device 20.

[Example of configuration of transaction management device]
6 shows an example of the configuration of transaction management device 20. Transaction management device 20 in the figure includes CPU 201, storage unit 202, RAM 203, and network-compatible communication unit 204.
The CPU 201 is a central processing unit that controls the operation of the transaction management device 20 by reading and executing programs stored in the memory unit 202 .
The storage unit 202 is an auxiliary storage device for the CPU 201 and stores various types of information used by the CPU 201, including programs.
The RAM 203 is the main storage device of the CPU 201 .
The network-compatible communication unit 204 is a part that performs communication via a network.

The memory unit 202 of the transaction management device 20 stores customer information, store information, and cart information.

[Information stored in the transaction management device]
The customer information stored in the memory unit 202 is information for managing individual customers. The transaction management device 20 generates customer information when a customer is registered (it may be considered that a customer has been registered when their customer information is stored). The transaction management device 20 also updates the customer information as appropriate based on cart information, etc. The transaction management device 20 may, for example, refer to the cart information at a predetermined time every day and update the customer information.

7A shows an example of customer information stored in the memory unit 202 of the transaction management device 20. The customer information in this figure includes customer identification information, customer name, customer registration date, product cancellation information, customer rank, number of points, etc.
The customer identification information is identification information that uniquely identifies a customer.
As the customer identification information (or instead of the customer identification information), for example, application identification information that is uniquely (e.g., sequentially) assigned to each installed shopping application may be used. Also, as the customer identification information (or instead of the customer identification information), terminal identification information unique to the mobile terminal 60 on which the shopping application is installed may be used. The terminal identification information may be, for example, a telephone number assigned to the customer who is the user of the mobile terminal 60, or hardware identification information uniquely assigned to the hardware of the mobile terminal 60, such as a MAC (Media Access Control) address.
The customer name is the customer's name or nickname, etc. The customer registration date is the date and time the customer was registered. The product cancellation information is information regarding cancellation of a product after it has been registered. The customer rank is a rank based on the customer's purchase history. When generating customer information for a new customer, the customer identification information, customer name, and customer registration date are generated, but since this is before the start of the actual transaction (product registration), other information (product cancellation information, etc.) is not generated.

The transaction management device 20 generates and stores customer identification information, for example, when a customer is registered (for example, when the mobile terminal 60 downloads or installs a shopping application from an external device (for example, a predetermined server that provides all applications, the transaction management device 20) to use the shopping service provided by the transaction management device 20). Furthermore, the transaction management device 20 acquires, for example, information entered into the name field of a registration form (input form) using the mobile terminal 60 during customer registration, and stores this as the customer name. Furthermore, the transaction management device 20 acquires, for example, the current date and time during customer registration, and stores this as the customer registration date.

In addition to or instead of storing customer information in its own memory, the transaction management device 20 may store some or all of the customer information in another device (such as a file server accessible by the transaction management device 20).

7(B) shows an example of store information stored in the memory unit 202 of the transaction management device 20. The store information in the figure includes store identification information, store name (branch name), store identification information 1, and store identification information 2.
The store identification information is identification information that uniquely identifies a store. The store identification information shown in Fig. 7B is composed of a store (trade name) or company code and a branch code.
The store name is the name of the store. The store name shown in Fig. 7B is composed of the store (trade name) or company name and the branch name.
The store identification information 1 is information of a two-dimensional code (QR code (registered trademark) or the like) for identifying the store where the transaction takes place (the store where the product is bought and sold).
The store identification information 2 is store location information (GPS information) for identifying the store where the transaction is to be made.
In the example shown in FIG. 7B, the store identification information and the store specifying information 1 are different, but the store identification information and the store specifying information 1 may be the same.

The transaction management device 20 may acquire and store store information from an external source (such as a server (not shown) at a headquarters that oversees each store). Furthermore, instead of or in addition to storing store information in its own memory, the transaction management device 20 may store some or all of the store information in another device (such as a file server accessible by the transaction management device 20).

Cart information is information used to manage individual transactions. The transaction management device 20 generates cart information at the start of a transaction. The transaction management device 20 also updates the cart information as the transaction progresses (each time a product is registered). (The storage of a product in the cart information can be interpreted as the product being registered.)

FIG. 7C is an example of cart information stored in the memory unit 202 of the transaction management device 20.
The cart information in the figure includes cart identification information, transaction start date and time, transaction end date and time, customer identification information, registered product information, reserved product information, and the like.

The cart identification information is identification information that uniquely identifies the cart information, and includes, for example, store identification information, a date, and a serial number (for example, a serial number for each store and date).
The transaction start date and time is the date and time when the transaction started. The transaction start date and time may be, for example, the date and time when the cart information was created. The transaction start date and time may also be the date and time when the first item was registered (the date and time when the registered product information (registered product 1) in FIG. 7C was stored). The date and time when the cart information was created and the date and time when the first item was registered may both be stored separately.

The transaction end date and time is the end date and time of the transaction. For example, the transaction start date and time may be the settlement date and time. The customer identification information is customer identification information that identifies the customer of the transaction. When cart information is generated, cart identification information, transaction start date and time, and customer identification information are generated, but since this is before the actual start of the transaction (product registration), other information (transaction end date and time, etc.) is not generated. The settlement date and time may be the settlement start date and time or the settlement end date and time. Both the settlement start date and time and the settlement end date and time may be stored separately as the transaction end date and time.

Registered product information (total) is information that is updated each time a product is registered. Registered product information (total) includes the number of items (number of products), estimated subtotal amount (for example, the subtotal amount before discounts, etc. are reflected), subtotal amount, etc. Registered product information (registered product 1) is the registration information for one product item. Registered product information (registered product 2) is the registration information for two products. Note that in the example shown in Figure 7 (C), registered product information (registered product 3) to registered product information (registered product 5) are not shown. Registered product information (registered product N; N is an integer) includes the product code, product name (product name), price, etc.

Registered product information (N) may include the registration date and time of the Nth item. In other words, the transaction management device 20 may store the registration date and time of the registered product as registered product information. The registration date and time of each product may also be used as information to determine whether or not to apply a limited-time offer or other service, and the effect of applying it.

The reserved item information (total) is updated each time a reserved item is registered. The reserved item information (total) includes the number of reserved items (items) of all types and the number of reserved items (items) of each type. In this embodiment, reserved items are classified into multiple types, and each classification is managed by classification number (reserved major classification number, reserved minor classification number) (details are explained in Figure 8). In the example of Figure 7 (C), the reserved item information (total) "Total number of items: 3" indicates that the number of items in all types (total of reserved items with reserved major classification numbers "1" to "4") is 3, "NON-FILE: 2" indicates that the number of reserved items classified as NON-FILE (total of reserved items with reserved major classification number "1") is 2, and "Reading failed: 1" indicates that the number of reserved items classified as read failed (total of reserved items with reserved major classification number "2") is 1.

A NON-FILE means that the barcode or product code was successfully scanned at the store (the product code was read), but the product code has not been stored (registered) in the product file. In other words, reserved products classified as NON-FILE are reserved products whose product code was successfully scanned, but whose product code is not stored in the product file.

An NG reading means that the product code was not successfully read (scanned) at the store. In other words, reserved products classified as NG reading are reserved products whose product code was not successfully read.

Reserved Item Information (Reserved Item 1) is information about reserved item 1. Reserved Item Information (Reserved Item 2) is information about reserved item 2. Reserved Item Information (Reserved Item 3) is information about reserved item 3.

The reserved product information (reserved product N; N is an integer) includes a reserved major classification number (described below). For example, if the Nth item is a reserved product classified as a NON-FILE with a reserved major classification number of "1," the reserved product information (reserved product N) includes the reserved major classification number "1 (NON-FILE)" and the product code. Furthermore, if the Nth item is a reserved product classified as a read-NG with a reserved major classification number of "2," the reserved product information (reserved product N) includes the reserved major classification number "2 (read-NG)" and image data (image captured by the mobile terminal 60). Note that the reserved product information for reserved products classified as a NON-FILE may include image data in addition to the reserved major classification number "1 (NON-FILE)" and the product code.

Note that the information shown in FIG. 7 is an example of information stored by the transaction management device 20, and the transaction management device 20 may also store other information (e.g., product files). Information about one product in a product file may include, for example, a product code (product identifier), product name, price, etc.

Figure 8 is an explanatory diagram explaining the classification of reserved items according to this embodiment. As shown in Figure 8, in this embodiment, reserved items are classified into major categories (classified by reserved major category numbers) and minor categories (classified by reserved minor category numbers). Note that devices within the product sales data processing system 1 may be able to reference information such as that shown in Figure 8 (such as a file showing the relationship between reserved major category numbers and reserved minor category numbers).

As shown in Figure 8, in this embodiment, reserved items are classified into four major categories, with reserved major category numbers "1" to "4."

Reserved items classified under reserve major category number "1" are NON-FILE reserved items. As mentioned above, NON-FILE reserved items are reserved items whose product code has been successfully scanned, but the product code is not stored in the product file. In other words, if a product code has been successfully scanned, but the product code has not been stored in the product file, the reserved item is classified under reserve major category number "1." Reserved items classified under reserve major category number "1" have an unknown price, so confirmation by a store clerk is required at checkout.

In Figure 8, "NON-FILE" in "1 (NON-FILE)" is the category name of the reserved item (major reserved category name). The same applies to "Reading NG" in "2 (Reading NG)," "Specific Item" in "3 (Specific Item)," and "Cancelled Item" in "4 (Cancelled Item)," which will be described later.

In this embodiment, the reserved minor category number "11" exists as a subordinate (lower) minor category of the reserved major category number "1." Because there is only one reserved minor category, reserved minor category number "11," under the reserved major category number "1," all reserved products classified under reserved major category number "1" are also classified under reserved minor category number "11."

In Figure 8, "NON-FILE" in "11 (NON-FILE)" is the category name (subcategory name) of the reserved product. The same applies to "Pseudo scan (coating, etc.)" in "21 (Pseudo scan (coating, etc.)")" and "Pseudo scan (short time)" in "22 (Pseudo scan (short time))," which will be described later.

In Figure 8, "Read operation" indicates that a read operation was performed, "Read operation not performed" indicates that a read operation was not performed, "Read success" indicates that the read operation was successful, "Read success" indicates that the read operation was unsuccessful, and "Read success" indicates that the read operation was unsuccessful, and "Read success" indicates that the read operation was not performed, which means that the problem is not related to the success or failure of the read.

Pending items classified under the pending major classification number "2" are those for which the reading was not successful. As mentioned above, pending items for which the reading of the product code was not successful are those for which the reading of the product code was not successful. In other words, if a pending item is placed on hold because the reading of the product code was not successful, the pending item is classified under the pending major classification number "2." Pending items classified under the pending major classification number "2" require confirmation by a store clerk at the time of payment, as their price is unknown and there is a risk of fraud.

In this embodiment, the major category with reservation major category number "2" has reservation minor category numbers "21," "22," "23," and "24" as subordinate categories. Reservation minor category number "21" is a reserved product with a "pseudo scan (coated, etc.)" reservation. Reservation minor category number "22" is a reserved product with a "pseudo scan (short time)" reservation. Reservation minor category number "23" is a reserved product with a "bad barcode." Reservation minor category number "23" is a reserved product with a "non-scanned input" reservation.

Pending items with pending subcategory number "21" (pseudo scan (covered, etc.)) are, for example, pending items for which a reading operation was performed (the item was held over), i.e., the item was placed within the imaging range of the imaging unit 606 (more specifically, the barcode readable range), but the barcode of the item was not captured (captured within the readable range) (because only parts other than the barcode of the item were captured within the readable range), so reading failed; pending items for which the barcode of the item was captured but was not readable because part of the barcode was covered, for example by a finger; or pending items for which the barcode of the item was captured but was not readable because the imaging angle of the barcode was inappropriate.

A reserved product with reservation subcategory number "22" (pseudo scan (short time)) is, for example, a reserved product whose barcode was imaged but could not be read because the barcode moved (including passing by) too quickly to be read.

Reserved products with the reserved major classification number "23" (defective barcode) are, for example, reserved products whose barcodes have been photographed but cannot be read due to wrinkles, blurs, dirt, reflections, etc. in the printed portion of the barcode.

A reserved item (entered without scanning) with the reserved major classification number "24" is, for example, a reserved item that could not be read because it was placed in the shopping cart 80 without undergoing a scanning operation (without the item being held over the reader).

Pending products with reserve subcategory numbers "21" through "24" (i.e., pending products with reserve major category number "2") may be referred to as unreadable products. Furthermore, situations resulting in pending products with reserve subcategory numbers "21" through "24" may also be referred to as unreadable situations (or simply "unreadable").

Note that the reserved items with reserved minor category number "21" (pseudo scan (covering, etc.)), reserved items with reserved minor category number "22" (pseudo scan (short time)), and reserved items with reserved major category number "23" (bad barcode) all have in common that they are reserved items for which a reading operation was attempted but the reading was unsuccessful. However, the reserved items with reserved minor category number "21" (pseudo scan (covering, etc.)) and reserved minor category number "22" (pseudo scan (short time)) are thought to have less of an intention to successfully read than the reserved items with reserved major category number "23" (bad barcode). In other words, the reserved items with reserved minor category number "21" (pseudo scan (covering, etc.)) and reserved minor category number "22" (pseudo scan (short time)) can also be said to be pseudo-reading operations (fake reading operations), and are therefore referred to as pseudo-scanned (fake scan) reserved items.

Pending items classified under the reserve major classification number "3" are reserved items of specific products (described below). Pending items of specific products have been successfully registered, but because they are specific products, they require the assistance of a store clerk when checking out. In other words, when a pending item is placed on hold because it is a specific product, it is classified under the reserve major classification number "3." Pending items classified under the reserve major classification number "3" are specific products, and therefore require confirmation by a store clerk when checking out.

In this embodiment, the major category with reservation major category number "3" has reservation minor category numbers "31," "32," and "33" as subordinate categories. Reservation minor category number "31" is a reserved product for "age verification products." Reservation minor category number "32" is a reserved product for "pharmaceuticals." Reservation minor category number "33" is a reserved product for "security tags."

Reserved items with reservation subcategory number "31" (age verification items) are reserved items such as alcohol and tobacco, which require a store clerk to verify the customer's age before selling them.

"Reserved products (pharmaceuticals) with reserved subcategory number 32" are reserved products that require a licensed pharmacist or other person to explain the product to the customer.

Reserved items with reservation subcategory number "33" (items with security tags) are, for example, reserved items that have security tags attached and require a store clerk to remove the security tags from the items when they are sold.

Reserved items classified under the reserve major category number "4" are canceled items. Cancelled items are reserved items that were successfully registered, but whose registration was cancelled after registration. In other words, if a product becomes reserved because its registration was cancelled after it was registered, the reserved item is classified under the reserve major category number "4." Reserved items classified under the reserve major category number "4" are cancelled items (i.e., items that will not be purchased) and will not be taken out of the store, so confirmation by a store clerk is required when checking out.

In this embodiment, the major category with reservation major category number "4" has a subordinate subcategory with reservation minor category number "41." Because the major category with reservation major category number "4" has only one subordinate subcategory with reservation minor category number "41," all reserved items classified under reservation major category number "4" are also classified under reservation minor category number "41."

The above explanation assumes that a reading operation has been performed on reserved items (age verification items) with reserved subcategory number "31." However, if a reading operation is not performed, the item will be classified as a reserved item (non-scanned) with reserved subcategory number "24." The same applies to reserved items (pharmaceuticals) with reserved subcategory number "32," reserved items (items with security tags) with reserved subcategory number "33," and reserved items (cancelled items) with reserved subcategory number "41."

In addition, the above explanation assumes that a reading operation was performed and successful for reserved items (age verification items) with reserved subcategory number "31." However, if a reading operation is performed and the reading fails, the item will be classified as either a reserved item with reserved subcategory number "21" (pseudo scan (covering, etc.)), a reserved item with reserved subcategory number "22" (pseudo scan (short time)), or a reserved item with reserved subcategory number "23" (bad barcode). The same applies to reserved items with reserved subcategory number "32" (pharmaceuticals), reserved items with reserved subcategory number "33" (items with security tags), and reserved items with reserved subcategory number "41" (cancelled items).

Note that, as shown in Figure 8, the reserved major category number and the reserved minor category number are in a higher-order and lower-order relationship, but one or both may be used depending on the situation. For example, in the cart information shown in Figure 7 (C), an example was explained in which the reserved major category number is stored in the reserved product information, but the reserved minor category number may be stored instead of or in addition to the reserved major category number.

In addition to or instead of storing the cart information in its own memory, the transaction management device 20 may store some or all of the cart information in another device (such as a file server accessible by the transaction management device 20).

[Example of basic processing procedure for self-operation]
The sequence diagram in Figure 9 shows an example of the processing procedure executed by the mobile terminal 60, the settlement device 40, and the transaction management device 20 in response to the accounting of one transaction in a self-service operation. Specifically, the processing in Figure 9 shows an example of the processing of the mobile terminal 60, the settlement device 40, and the transaction management device 20 from when a customer registers a product to be purchased from products displayed in the store until the settlement of the registered product is completed.

Step S1: The mobile terminal 60 acquires information identifying the store (store identification information). For example, a two-dimensional code identifying the store may be displayed near the entrance of the store (the two-dimensional code may be output on a display screen, a medium with the two-dimensional code printed on it may be attached, etc.), and a customer visiting the store may scan (read) the two-dimensional code with the mobile terminal 60, thereby allowing the mobile terminal 60 to acquire the store identification information. When a customer visiting the store launches a shopping application, a start-up screen may be displayed instructing the customer to scan the two-dimensional code. Alternatively, when a customer visiting the store scans the two-dimensional code with the mobile terminal 60, the shopping application may be launched, and a start-up screen may be displayed informing the customer that they are currently connected to the transaction management device 20.

Figure 10 (A) shows an example of the shopping application startup screen displayed on the display unit 604 of the mobile terminal 60. The startup screen in this figure displays the store name. This store name is displayed based on the store-specific information acquired in step S1. When a customer wants to start registering products for shopping, they operate the transaction start button BT10 located on the startup screen.

In response to the operation of the transaction start instruction button BT10, the mobile terminal 60 sends a transaction start request including the acquired store identification information and customer identification information to the transaction management device 20. The destination information (the address of the transaction management device 20) may be acquired during customer registration (when a shopping application is downloaded or installed on the mobile terminal 60) and stored in the memory of the mobile terminal 60. Note that when scanning a two-dimensional code, the destination information may also be converted into a two-dimensional code in addition to the store identification information, and by scanning the two-dimensional code with the mobile terminal 60, the mobile terminal 60 may acquire the destination information along with the store identification information.

Step S2: The transaction management device 20, which has received customer identification information and store identification information as a transaction start request from the mobile terminal 60, generates cart information for the transaction. For example, the transaction management device 20 acquires store identification information from the store identification information received from the mobile terminal 60, acquires the current date, issues (assigns) a serial number, combines the store identification information, the current date, and the serial number, and stores the combined information as cart identification information in the cart information. The transaction management device 20 also stores the current date and time as the transaction start date and time (generation date and time) in the cart information. The transaction management device 20 also stores the customer identification information received from the mobile terminal 60 as customer identification information in the cart information.

Step S3: The transaction management device 20, which has generated the cart information for the transaction, generates product registration initial screen information (screen information for the product registration screen, which is the initial screen) and transmits it to the mobile terminal 60. Specifically, the transaction management device 20, for example, generates product registration initial screen information and transmits the generated product registration initial screen information to the mobile terminal 60 together with cart identification information.

Step S4: The mobile terminal 60, which has received the cart identification information and product registration initial screen information from the transaction management device 20, stores the cart identification information and displays the registration screen on the display unit. Specifically, the mobile terminal 60 displays, for example, the product registration initial screen as shown in FIG. 10 (B).

Step S5: The customer operates the mobile terminal 60 to scan the barcode attached to the product and read the product code. Note that in the figure, it is assumed that the barcode was successfully scanned. Steps S5 to S9 are repeated each time a barcode attached to a product is scanned.

The mobile terminal 60 that acquires the barcode transmits the cart identification information and product registration information, including the product code obtained by scanning, to the transaction management device 20.

Step S6: Upon receiving the product registration information (cart identification information and product code) from the mobile terminal 60, the transaction management device 20 identifies the cart information for the transaction from the cart identification information included in the received product registration information.

Step S7: The transaction management device 20 updates the product data in the identified cart information. Specifically, if the received product registration information includes a product code for item N, the transaction management device 20 stores the product code as the product code of the registered product information (registered product N) in the identified cart information, obtains the product name and price corresponding to the product code from the product information, and stores them as the product and price of the registered product information (registered product N). The transaction management device 20 also updates the registered product information (total) in the identified cart information.

Step S8: Having updated the product data in the cart information, the transaction management device 20 generates product registration update screen information (screen information for the product registration screen, which is the update screen to which the registered product has been added) and transmits it to the mobile device 60. Specifically, the transaction management device 20 generates product registration update screen information such that the mobile device 60 displays a product registration update screen to which the list item for the currently registered product has been added in the registered product list, as shown in FIG. 10 (C), for example, and transmits the generated product registration update screen information to the mobile device 60 together with the cart identification information.

The product registration screen (product registration update screen) shown in FIG. 10(C) is displayed on the mobile device 60 after "XXX bread" has been registered as the third product item. In other words, when "XXX yogurt" is stored in the cart information as the first item, the transaction management device 20 generates product registration update screen information that displays "XXX yogurt" on the mobile device 60 and transmits the generated product registration update screen information to the mobile device 60 along with the cart identification information. When "XXX chocolate" is stored in the cart as the second item, the transaction management device 20 generates product registration update screen information that displays "XXX yogurt" and "XXX chocolate" on the mobile device 60 and transmits the generated product registration update screen information to the mobile device 60 along with the cart identification information. When "XXX bread" is stored in the cart information as the third item, the transaction management device 20 generates product registration update screen information that displays "XXX yogurt", "XXX chocolate", and "XXX bread" on the mobile device 60 as shown in FIG. 10(C). The generated product registration update screen information is transmitted to the mobile device 60 along with the cart identification information.

In response to the registration of one product, a registered product presentation screen WD1 may be displayed, presenting the newly registered product, as shown in FIG. 11(A), for example. Furthermore, in response to the product being available for eat-in, the registered product presentation screen WD1 in the same figure displays a "Yes" button BT1 and a "No" button BT2 to confirm with the customer whether or not they wish to eat or drink in the restaurant. In this case, the price of the product will be calculated so that the consumption tax rate for the product is different when the "Yes" button BT1 is operated and when the "No" button BT2 is operated.

Step S9: The mobile terminal 60, which has received the cart identification information and product registration update screen information from the transaction management device 20, adds the product to the registration screen. Specifically, the mobile terminal 60 displays a product registration update screen such as that shown in FIG. 10(C). As described above, the product registration screen (product registration update screen) shown in FIG. 10(C) is displayed on the mobile terminal 60 after "XX bread" has been registered as the third product item.

Step S10: The mobile terminal 60 accepts a payment instruction operation as a customer operation. For example, it accepts a touch of the "Proceed to Checkout" button shown in Figure 10 (C).

Step S11: The mobile terminal 60 that has accepted the payment transition instruction operation generates a two-dimensional code. That is, the mobile terminal 60 converts information required to execute the payment process for the items purchased by the mobile terminal 60 (e.g., cart identification information) into a two-dimensional code. The mobile terminal 60 that has generated the two-dimensional code displays the generated two-dimensional code on its display unit. For example, it displays a code display screen on the display unit that displays the two-dimensional code as shown in Figure 10 (D).

Step S12: The settlement device 40 scans (reads) the two-dimensional code displayed on the display unit of the mobile terminal 60. For example, the settlement device 40 scans the two-dimensional code displayed on the display unit of the mobile terminal 60 that the customer has pointed within the recognition range of the scanner unit 406.

Step S13: The settlement device 40 reads the two-dimensional code displayed on the display of the mobile terminal 60 and requests the transaction management device 20 to calculate the subtotal. For example, the settlement device 40 sends a calculation request (subtotal calculation request information) requesting the calculation of the subtotal to the transaction management device 20 along with the cart identification information obtained from the two-dimensional code.

Step S14: Having received the cart identification information and the sub-calculation output request information from the mobile terminal 60, the transaction management device 20 identifies the cart information for the transaction from the cart identification information.

Step S15: Having identified the cart, the transaction management device 20 calculates the subtotal amount using information such as prices in the registered product information included in the cart information corresponding to the identified cart information.

Step S16: After calculating the subtotal, the transaction management device 20 updates the cart information (storing the subtotal (calculated subtotal)) and sends the subtotal information indicating the calculated subtotal along with the cart identification information to the settlement device 40.

Step S17: Having received the cart identification information and subtotal information from the transaction management device 20, the settlement device 40 displays the subtotal amount (a settlement screen including the subtotal amount) on the display unit 405.

Step S18: The settlement device 40, which has displayed the subtotal on the display unit 405, executes the payment (settlement). Specifically, the settlement device 40 accepts the selection of the payment type. In the case of cash, it accepts the deposit, calculates the amount of change, and, if there is change, dispenses the change and issues a receipt. Furthermore, when the settlement is complete, the settlement device 40 sends settlement completion information along with the cart information to the transaction management device 20, and the transaction management device 20 stores the transaction completion date and time (settlement date and time) for that cart.

[About reading failure situations]
9 shows the process corresponding to the case where the reading of the product code is successful in step S5. However, during the product code reading process corresponding to step S5, the portable terminal 60 may encounter a situation where it fails to read any of the reserved sub-category numbers "21" to "24" in FIG. 8 (a reading failure situation).

As a specific example, Figure 11 (B) shows a situation in which a reserved product with reserved subcategory number "21" (pseudo scan (covered, etc.)) may occur, using a captured image of the product registration screen. The figure shows the product registration screen displayed on the mobile terminal 60. The product registration screen in the figure displays a captured image of a customer attempting to have the mobile terminal 60 read the product's barcode, with the barcode portion obscured by their finger.

In the example shown in Figure 11 (B), the portable terminal 60 recognizes the presence of a barcode in the captured image, but fails to read the barcode (the code cannot be extracted). In this case, the portable terminal 60 executes a reading failure process (described below) to deal with the reading failure situation.

Details will be provided later, but in the reading failure process, a retry (read) is performed on the product that has become unreadable, and if the retry does not resolve the unreadable situation, a preset key is displayed. Products that have become unreadable can be registered via a preset key. In other words, unreadable situations can be resolved by retrying or using a preset key. In many cases, unreadable situations can be resolved by retrying or using a preset key, but if the unreadable situation is not resolved, the product will be placed on hold.

If the unscanned item is not resolved (i.e., if the item becomes a pending item), a guide image WD2 may be displayed, for example, as shown in FIG. 11(C), to encourage the user to place the pending item in a bag for pending items. Specifically, the guide image WD2 in the same figure displays a message saying, "If scanning is not possible, please place the item in a dedicated bag." Note that the form of the guide image WD2 is not particularly limited. The message displayed in the guide image WD2 may be, for example, "Scanning failed" or "The item has been determined to be unscannable."
The guide image WD2 in the figure is displayed so as to be superimposed on the product registration screen. A confirmation button BT3 is arranged in the guide image WD2 in the figure. In this case, the guide image WD2 is erased in response to an operation on the confirmation button BT3. Alternatively, the guide image WD2 may be erased in response to a certain period of time having elapsed since the start of display. After the guide image WD2 is erased, the screen may return to the product registration screen of, for example, FIG. 10(B) or FIG. 10(C).

Furthermore, when the mobile terminal 60 places a misreadable product on hold, it may transmit captured images corresponding to the misreading situation to the transaction management device 20. The transmitted captured images may be extracted from the buffered captured images, for example, by buffering the captured images captured by the imaging unit 606 for a certain period of time in order to read the product code. Furthermore, the transmitted captured images may be, for example, a predetermined number of still images, or moving images for a predetermined period of time depending on the occurrence of the misreading situation.

Note that while the product registration screen displays a registered product list as shown in FIG. 10(C), the mobile device 60 may display the products as list items regardless of the type of reserved product (i.e., for all reserved products in all subcategories). When displaying reserved products, information indicating that the products are reserved products (such as an icon or button indicating that the products are reserved, a classification number (either a major reserved classification number or a minor reserved classification number, or both), or a classification name (either a major reserved classification name or a minor reserved classification name, or both)) may be displayed. When displaying reserved products, information indicating the type of reserved product (such as a classification number (either a major reserved classification number or a minor reserved classification number, or both), or a classification name (either a major reserved classification name or a minor reserved classification name, or both), or an icon or button corresponding to the classification number or classification name) may be displayed. Note that the mobile device 60 may not display some reserved products (reserved products with a specific minor reserved classification number) as list items, but may display other reserved products as list items.

Figures 12 to 14 are flowcharts showing an example of the operation of the mobile terminal 60 according to this embodiment. Figure 15 is a flowchart showing an example of the operation of the transaction management device 20 according to this embodiment. The flowcharts in Figures 12 to 15 correspond to the processing of steps S5 to S9 in Figure 9. Note that the flowcharts in Figures 12 to 15 omit processing related to reserved items (specific items) classified under reservation major classification number "3" and reserved items (cancelled items) classified under reservation major classification number "4."

Although omitted from the flowcharts of Figures 12 to 14, the mobile terminal 60 is constantly capturing images using the imaging unit 606 to read barcodes and recognize the placement of items in the shopping cart 80.

[Explanation of the flowchart in FIG. 12]
Step S100: The portable terminal 60 determines whether the customer has started reading a barcode. For example, if the portable terminal 60 recognizes a barcode-like object in the captured image (through image), it determines that the customer has started reading a barcode. Note that the storage unit 602 of the portable terminal 60 (or another device accessible to the portable terminal 60, such as the storage unit 202 of the transaction management device 20) may store several images of barcode-like objects as comparison images, and the portable terminal 60 may determine that it has recognized a barcode-like object (i.e., that the customer has started reading a barcode) if it recognizes an image similar to one of the comparison images in the captured image. If the reading operation has started (Step S100: YES), the process proceeds to Step S101. If the reading operation has not started (Step S100: NO), the process proceeds to Step S104.

Step S101: The mobile terminal 60 determines whether reading of the barcode has been completed (successful). In other words, the mobile terminal 60 determines whether the product code has been obtained from the barcode. If reading has been completed (Step S101: YES), proceed to Step S110. If reading has not been completed (Step S101: NO), proceed to Step S102.

Step S102: The mobile terminal 60 determines whether or not it has detected the addition of an item to the shopping basket 80. That is, the mobile terminal 60 set in the manner illustrated in FIG. 3 captures an image of the contents of the shopping basket 80 placed in the shopping cart 70, and can detect the addition of an item to the shopping basket 80 based on changes in the captured image. If it detects the addition of an item (Step S102: YES), it proceeds to Step S120. If it does not detect the addition of an item (Step S102: NO), it proceeds to Step S103.

Step S103: The portable terminal 60 determines whether a predetermined time has elapsed since the start of the reading operation. If the predetermined time has elapsed (Step S103: YES), proceed to Step S120. If the predetermined time has not elapsed (Step S103: NO), return to Step S101.

Step S104: As in step S102, the mobile terminal 60 determines whether or not it has detected the addition of an item to the shopping cart 80. If it has detected the addition of an item (step S104: YES), it proceeds to step S120. If it has not detected the addition of an item (step S104: NO), this flowchart ends.

Step S110: Following step S101 (YES), the mobile terminal 60 executes the product registration process (described below). This flowchart then ends.

Step S120: Following step S102 (YES), step S103 (YES), and step S104 (YES), the portable terminal 60 executes the reading failure process (described below). This flowchart then ends.

[Explanation of the flowchart in FIG. 13]
The flowchart in FIG. 13 is an example of the product registration process (S110) in FIG.
Step S111: Mobile terminal 60 transmits product registration information to transaction management device 20. The product registration information includes the product code obtained from the barcode read in step S101 and cart identification information. Then, the process proceeds to step S112.

Step S112: The mobile terminal 60 receives the registration completion notification sent from the transaction management device 20 in response to the transmission of the product registration information in step S111. Then, proceed to step S113.

Step S113: The mobile terminal 60 updates the product registration screen, for example, as shown in Figure 10 (C).

[Explanation of the flowchart in FIG. 14]
The flowchart in FIG. 14 is an example of the reading failure process (S120) in FIG.
Step S121: The portable terminal 60 determines whether the value of the reread operation instruction flag is "1." The reread operation instruction flag is information indicating whether a reread operation (a re-read operation) has been instructed. A value of "0" for the reread operation instruction flag indicates that a reread operation has not yet been instructed for the transaction, and a value of "1" for the reread operation instruction flag indicates that a reread operation has already been instructed for the transaction. If the value of the reread operation instruction flag is "1," i.e., if a reread operation has already been instructed for the transaction (step S121 (YES)), the portable terminal 60 proceeds to step S124. If the value of the reread operation instruction flag is "0," i.e., if a reread operation has not yet been instructed for the transaction (step S121 (NO)), the portable terminal 60 proceeds to step S122. The re-read operation instruction completed flag may be stored in the memory unit 602 of the portable terminal 60 (or in another device that can be accessed by the portable terminal 60, for example, the memory unit 202 of the transaction management device 20). When the re-read operation instruction completed flag is stored in another device, the re-read operation instruction completed flag for each portable terminal 60 (or customer) may be stored in the other device so that each can be identified.

Step S122: The mobile terminal 60 sets the value of the re-read operation instruction completion flag to "1". Then, proceed to step S123.

Step S123: The portable terminal 60 displays a message prompting the user to perform a reading operation. Then, the process returns to step S100 in FIG. 12.

Step S124: The portable terminal 60 sets the value of the re-read operation instruction completion flag to "0". Then, proceed to step S125.

Step S125: The mobile terminal 60 displays preset keys on the display unit 604. The customer can operate the preset keys to register products. Details of the preset keys will be described later. Next, proceed to step S126.

Step S126: The mobile terminal 60 determines whether a preset key has been operated. In other words, the mobile terminal 60 determines whether a preset key has been operated to register a product. If a preset key has been operated (Step S126: YES), the process returns to Step S110 (product registration process) in Figures 12 and 13. If a preset key has not been operated (Step S126: NO), the process proceeds to Step S127.

Step S127: The mobile terminal 60 determines whether a predetermined time has elapsed since the display of the preset keys in step S125 (start of display). If the predetermined time has elapsed (step S127: YES), proceed to step S130. If the predetermined time has not elapsed (step S127: NO), return to step S126.

Step S130: The mobile terminal 60 determines the classification number (e.g., the reserved sub-classification number) of the reserved product. For example, the mobile terminal 60 may determine the reserved sub-classification number as follows:

[Determining the classification number (reserved minor classification number) for reserved products]
(1) When the process transitions from step S102 (YES) in FIG. 12 to the reading failure process (step S120), the reserved subcategory number is determined to be "22." In other words, since the insertion of a product was detected before the predetermined time elapsed, the reserved product is determined to be a reserved product with reserved subcategory number "22" (pseudo scan (short time)).
(2) When the process transitions from step S103 (YES) in Fig. 12 to the reading failure process (step S120) and it is determined that the operation is a pseudo scan, the reserved sub-category number is determined to be "21." In other words, the reading operation is permitted until a predetermined time has elapsed, and if it is determined that the operation is a pseudo scan, the reserved product is determined to be a reserved product with reserved sub-category number "21" (pseudo scan (coated, etc.)).
(3) When the process transitions from step S103 (YES) in Fig. 12 to the reading failure process (step S120) and it is not determined that the operation is a pseudo scan, the reserved sub-category number is determined to be "23." In other words, the reading operation is permitted until a predetermined time has elapsed, and if it is determined that the operation is not a pseudo scan, the reserved product is determined to be a reserved product (bad barcode) with reserved sub-category number "23."
(4) When the process transitions from step S104 (YES) in Fig. 12 to the reading failure process (step S120), the reserved subcategory number is determined to be "24." In other words, since the insertion of a product was detected without recognizing the start of the reading operation, the reserved product is determined to be a reserved product (non-scan-inserted) with the reserved subcategory number "24."

Further explanation will be given regarding the reserved item with reserved subcategory number "21" (pseudo scan (coated, etc.)) and the reserved item with reserved subcategory number "23" (bad barcode). Depending on whether the customer's reading operation up until the specified time has elapsed is a pseudo scan (specifically, a pseudo scan of coating, etc.), the mobile terminal 60 determines whether the reserved item is a reserved item with reserved subcategory number "21" (pseudo scan (coated, etc.)) or a reserved item with reserved subcategory number "23" (bad barcode). Whether or not the item is a pseudo scan may be determined, for example, as follows:

The portable terminal 60 may determine whether or not the scan is a pseudo scan based on a plurality of captured images taken until a predetermined time has elapsed (until step S103 (YES) is reached).
(Judgment method 1)
The mobile terminal 60 determines that a pseudo scan has occurred if, in the multiple images captured within a predetermined time period, at least a portion of the barcode is not captured (for example, if a portion is covered by a finger or the like), or if the barcode is within the imaging range of the imaging unit 606 but at an inappropriate angle; however, if the entire barcode is captured and the angle of the barcode is appropriate, it determines that a pseudo scan has not occurred.
(Judgment method 2)
Instead of determination method 1, the portable terminal 60 may determine that a pseudo scan is occurring if the object has not changed (for example, the angle or position of the object has changed by more than a predetermined amount) in the multiple captured images taken over a predetermined period of time, even if the entire barcode is captured and the angle of the barcode is appropriate (i.e., there is no evidence of an attempt to read it). In other words, the portable terminal 60 may determine that a pseudo scan is not occurring if the entire barcode is captured over a plurality of captured images taken over a predetermined period of time, the angle of the barcode is appropriate, and the object has changed in the multiple captured images (i.e., there is evidence of an attempt to read it), and may determine that a pseudo scan is occurring otherwise.

The portable terminal 60 may determine (distinguish) how the transition to the reading failure processing occurred in the processing of step S130 (i.e., whether the transition to the reading failure processing occurred from step S102 (YES) of FIG. 12, whether the transition to the reading failure processing occurred from step S103 (YES) of FIG. 12, or whether the transition to the reading failure processing occurred from step S104 (YES) of FIG. 12) as follows. As an example, the portable terminal 60 may set a different flag (with a value of 1) for each of the cases where step S102 (YES) of FIG. 12 occurs, step S103 (YES) of FIG. 12, and step S104 (YES) of FIG. 12, and may determine how the transition to the reading failure processing occurred based on which flag is set in the processing of step S130.

In addition, the process for determining the reserved sub-category number when transitioning from step S102 (YES) in Figure 12 to reading failure processing, the process for determining the reserved sub-category number when transitioning from step S103 (YES) in Figure 12 to reading failure processing, and the process for determining the reserved sub-category number when transitioning from step S104 (YES) in Figure 12 to reading failure processing may simply be different processes (i.e., by branching the processing flow), and the corresponding reserved sub-category number may be determined (assigned) in each process. In other words, when transitioning to reading failure processing from step S102 (YES) in FIG. 12, the reserved subcategory number may be determined as "22" in the reserved subcategory number determination process (classification number determination process A), when transitioning to reading failure processing from step S103 (YES) in FIG. 12, the reserved subcategory number may be determined as "23" or "21" in the reserved subcategory number determination process (classification number determination process C) when transitioning to reading failure processing from step S104 (YES) in FIG. 12, the reserved subcategory number may be determined as "24".

It is also possible that the transition to the reading error processing before (first time) step S123 in FIG. 14 is different from the transition to the reading error processing after (second time) step S123 in FIG. 14 is different. If the first transition to the reading error processing is different from the second transition to the reading error processing, the second (i.e., newer) transition may be used to determine the reserved sub-category number. As an example, if the first transition to the reading error processing is from step S104 (YES) in FIG. 12, and the second transition to the reading error processing is from step S102 (YES) in FIG. 12 (the first time is non-scanning and the second time is short-term), the second transition (the transition to the reading error processing from step S102 (YES) in FIG. 12) may be used, and the reserved sub-category number may be determined to be "22."

Step S131: Following step S130, the portable terminal 60 acquires (extracts) an image corresponding to the currently occurring reading failure from among the images captured by the imaging unit 606. Then, the process proceeds to step S132.

The captured images acquired by the portable terminal 60 in step S131 will now be described. The captured images acquired by the portable terminal 60 are intended to allow the store clerk to visually recognize the reading failure that has occurred. For this reason, the format of the captured images acquired by the portable terminal 60 (video, still image), the number of captured images, the time and period for which the captured images are acquired, etc. may vary depending on the reading failure situation.

[Images acquired for reserved products with reserved sub-category numbers "21" to "23"]
In the case of reserved items with reservation subcategory numbers "21" to "23" (i.e., when a barcode reading operation has been performed), the mobile terminal 60 acquires captured images that allow the barcode reading status to be recognized. In other words, the mobile terminal 60 is constantly capturing images using the imaging unit 606 to recognize the barcode reading operation and the addition of items to the shopping cart 80 as described above, and temporarily stores these images, and in step S131 extracts captured images that allow the barcode reading status to be recognized. Note that in the case of reserved items with reservation subcategory numbers "21" to "23" (i.e., when a barcode reading operation has been performed), the mobile terminal 60 may acquire (extract) captured images that allow the addition of items to the shopping cart 80 to be recognized (e.g., images before and after addition), in addition to captured images that allow the barcode reading status to be recognized.

[Image captured for reserved product with reserved subclassification number "24"]
In the case of a reserved product with reservation subcategory number "24" (i.e., when the barcode has not been read), the mobile terminal 60 acquires captured images (images before and after the product has been added, etc.) that allow recognition of the product being added to the shopping cart 80. In other words, the mobile terminal 60 is constantly capturing images using the imaging unit 606 in order to recognize the barcode reading operation and the addition of the product to the shopping cart 80 as described above, and temporarily stores these images, and in step S131 extracts captured images (images before and after the product has been added, etc.) that allow recognition of the product being added to the shopping cart 80.

Step S132: The mobile terminal 60 transmits unsuccessful reading information to the transaction management device 20. The unsuccessful reading information includes the captured image acquired in step S131, the reserved subcategory number determined in step S130, and cart identification information. Then, proceed to step S133.

Step S133: The mobile terminal 60 receives the registration completion notification sent from the transaction management device 20 in response to the transmission of the reading failure information in step S132. Then, proceed to step S134.

Step S134: The mobile terminal 60 may, for example, update the product registration screen and display the guide image WD2 superimposed thereon, as illustrated in FIG. 11(C). This flowchart then ends.

As shown in Figures 12 to 14, after displaying a message prompting the user to try the barcode reading operation again, if the mobile terminal 60 recognizes the barcode reading operation but does not complete (succeed) the barcode reading normally (step S102 (YES), step S103 (YES)), it displays a preset key (step S125) in the reading failure process (step S120), and if the product is not registered using the preset key within a predetermined time (step S127 (YES)), it processes the product as a reserved product with a reserved sub-category number "21" to "23".

Furthermore, as shown in Figures 12 to 14, if the mobile terminal 60 detects the insertion of a product without recognizing the barcode reading operation after displaying a message prompting another reading operation (step S104 (YES)), it displays a preset key (step S125) in the reading failure process (step S120), and if the product is not registered using the preset key within a predetermined time (step S127 (YES)), it processes the product as a reserved product with the reserved subcategory number "24".

In the flowchart shown in Figure 14, if the first reading is unsuccessful, a second (retry) reading is performed, and if the second reading is also unsuccessful, a preset key is displayed. However, if the first reading is unsuccessful, a preset key may be displayed without a second reading. For example, whether or not a second reading is performed may depend on the reserved sub-category number. As an example, in the case of a non-scan input where the reserved sub-category number is "24," a preset key may be displayed immediately without a second (retry) reading, based on or independent of operation.

[Explanation of the flowchart in FIG. 15]
Step S201: Transaction management device 20 determines whether or not it has received the product registration information sent by mobile terminal 60 in step S111 of Figure 13. If it has received the product registration information (step S201: YES), it proceeds to step S202. If it has not received the product registration information (step S201: NO), it proceeds to step S211.

Step S202: The transaction management device 20 determines whether the product has been identified. That is, the transaction management device 20 determines whether the product code included in the product registration information is stored in the product file (product master). If the product has been identified (Step S202: YES), proceed to Step S203. If the product has not been identified (Step S202: NO), proceed to Step S213.

Step S203: The transaction management device 20 identifies the cart information. That is, the transaction management device 20 identifies the cart information corresponding to the cart identification information included in the product registration information. Then, the process proceeds to step S204.

Step S204: The transaction management device 20 updates the cart information identified in step S203. That is, the transaction management device 20 updates the registered product information (total, registered product N) in the cart information. As a result, one product is registered in the cart information. Then, proceed to step S205.

Step S205: The transaction management device 20 sends a registration completion notification to the mobile terminal 60. The mobile terminal 60 receives the registration completion notification in step S112 of FIG. 13. This flowchart then ends.

Step S211: The transaction management device 20 determines whether or not it has received the unreadable information sent by the mobile terminal 60 in step S132 of FIG. 14. If it has received unreadable information (step S211: YES), it proceeds to step S213. If it has not received unreadable information (step S211: NO), this flowchart ends.

Step S213: The transaction management device 20 identifies the cart information. That is, the transaction management device 20 identifies the cart information corresponding to the cart identification information included in the unread information. Then, the process proceeds to step S214.

Step S214: The transaction management device 20 updates the cart information identified in step S213. That is, the transaction management device 20 updates the pending product information (total, pending products N) in the cart information. As a result, one product (pending product) is registered in the cart information. Then, the process proceeds to step S215. The transaction management device 20 can refer to information showing the relationship between the pending major classification number and the pending minor classification number (for example, information such as that shown in Figure 8), and recognizes the pending major classification number that is higher than the pending minor classification number included in the unreadable information.

Step S215: The transaction management device 20 sends a registration completion notification to the mobile terminal 60. The mobile terminal 60 receives the registration completion notification in step S133 of FIG. 14. This flowchart then ends.

If the transaction management device 20 is unable to identify the product (step S202: NO), it will register the reserved product (NON-FILE) with reserved major classification number "1" when updating the cart information (step S214).

In addition, when updating the cart information, the transaction management device 20 also registers reserved items (specific items) classified under the reserved major category number "3," i.e., reserved items with reserved minor category number "31" (age verification items), reserved items with reserved minor category number "32" (pharmaceuticals), and reserved items with reserved minor category number "33" (anti-theft tag items). For example, the mobile terminal 60 may transmit product registration information for age verification items (pharmaceuticals and anti-theft tag items as well) in the same way as for other items that do not fall under the category of specific items (step S111 in FIG. 13), and the transaction management device 20 may register the reserved items (specific items) in the processing of step S204 (or in the processing of step S214).

In addition, when updating the cart information, the transaction management device 20 also registers reserved items (cancelled items) classified as reservation major classification number "4." For example, during the cancellation process (cancellation process; not shown), the mobile terminal 60 may transmit product registration information (product registration information including information indicating cancellation; also referred to as registration cancellation information), and the transaction management device 20 may register the reserved items (cancelled items) during the process of step S204 (or the process of step S214).

Figures 16 and 17 show examples of the re-read instruction screen and preset key screen (small preset key screen) according to this embodiment. The display of preset keys will be explained using Figures 16 and 17.

Figure 16 shows a situation in which a customer K has purchased one "XX potato" and two "XX tea" and then purchases one "XX chocolate."

Figure 16(A) shows a scene in which one "XX potato" and two "XX tea" have been registered (all successfully read). In Figure 16(A), the mobile terminal 60 is capturing an image of the inside of the shopping cart 80. The captured image in Figure 16(A) shows a "XX potato" object (symbol a) and a "XX tea" object (symbol b). If, from the scene in Figure 16(A), customer K reads the barcode for "XX chocolate" and adds the "XX chocolate" to the shopping cart 80 without successfully reading it (if step S102 (YES) or step S103 (YES) in Figure 12), the image captured immediately after the addition will be as shown in Figure 16(B). The captured image in Figure 16(B) shows a "XX potato" object (symbol a), a "XX tea" object (symbol b), and a "XX chocolate" object (symbol c).

The mobile terminal 60 then displays a re-read instruction screen WD31 as shown in FIG. 16(C) (step S123 in FIG. 14). The re-read instruction screen WD31 in FIG. 16(C) displays a message indicating that registration failed (symbol d) and prompting the user to perform a read operation (symbol e). The re-read instruction screen WD31 in FIG. 16(C) also displays an OK button BT51 that can be operated to perform a read operation again in accordance with the message, and a Do Not Purchase button BT53 that can be operated to cancel the purchase of the product (XX Chocolate).

If the OK button BT51 is operated on the reread instruction screen WD31, the reread instruction screen WD31 is closed and the process returns to step S100 in FIG. 12, as per the flowchart shown in FIG. 14.

If the "Do not purchase" button BT53 is pressed on the rescan instruction screen WD31, the rescan instruction screen WD31 is closed, and although omitted from the flowchart shown in Figure 14, a cancellation process (cancellation process, not shown) is executed for the product, and the process ends.

If the second reading operation is unsuccessful, as with the first attempt, the mobile terminal 60 displays the preset key screen WD41 as shown in FIG. 16 (D) (step S125 in FIG. 14).

The preset key screen WD41 in FIG. 16(D) displays a message (symbol d) indicating that registration (reading) failed and a message (symbol f) prompting the user to select a product using a button (preset key). The preset key screen WD41 in FIG. 16(D) also displays multiple preset keys BT60 and a preset key display switch button BT61 (a button to display the next multiple preset keys BT60 and a button to display the previous multiple preset keys BT60; either one is displayed in the case of the first or last). The preset key screen WD41 in FIG. 16(D) also displays a "Can't Select" button BT62 that is operated when the user intends to purchase the product (XX chocolate) but is unable to select the product (XX chocolate) using a preset key for reasons such as not being able to find a preset key corresponding to the product (XX chocolate), and a "Can't Purchase" button BT63 that is operated to cancel the purchase of the product (XX chocolate).

When any of the preset keys BT60 on the preset key screen WD41 is operated, the preset key screen WD41 is cleared, and the process returns to step S110 (product registration process) in FIGS. 12 and 13, as shown in the flowchart in FIG. 14. This registers the product corresponding to the operated preset key BT60. Specifically, in this example, the preset key BT60 corresponding to XX chocolate is operated on the preset key screen WD41. The mobile terminal 60 acquires the product code for XX chocolate based on the operation of the preset key BT60 corresponding to XX chocolate. When the mobile terminal 60 acquires the product code for XX chocolate based on the operation of the preset key BT60 corresponding to XX chocolate, the mobile terminal 60 executes the product registration process (step S110), just as when reading the barcode attached to XX chocolate is successful (i.e., when the product code for XX chocolate is acquired by reading the barcode), and XX chocolate is registered.

When the display switch button BT61 is operated on the preset key screen WD41, the preset keys BT60 displayed on the preset key screen WD41 are switched. In other words, the mobile terminal 60 switches the preset keys BT60 displayed on the preset key screen WD41 based on the operation of the display switch button BT61.

If the non-selectable button BT62 on the preset key screen WD41 is operated, the preset key screen WD41 is cleared, and although omitted from the flowchart shown in FIG. 14, the process proceeds to step S130. In other words, just as when a predetermined time has elapsed (step S127 (YES)), the product (XX chocolate) is processed as a reserved product.

When the "Do not purchase" button BT63 on the preset key screen WD41 is operated, the preset key screen WD41 is cleared, and although this is omitted from the flowchart shown in Figure 14, a cancellation process (not shown) is executed for the product, and the process ends.

Figure 17 shows a situation in which a customer K has purchased one "XX potato" and two "XX tea" and then purchases one "apple." Note that the "apple" does not have a barcode attached.

Figure 17(A), like Figure 16(A), shows a scene in which one "XX potato" and two "XX tea" have been registered (all successfully read). If customer K adds an "apple" to shopping cart 80 from the scene in Figure 17(A) (step S104 (YES) in Figure 12), the image captured immediately after the addition will look like Figure 17(B). The image captured in Figure 17(B) contains an object for "XX potato" (symbol a), an object for "XX tea" (symbol b), and an object for "apple" (symbol g).

Then, the mobile terminal 60 displays a reread instruction screen WD32 as shown in FIG. 17(C) (step S123 in FIG. 14). The reread instruction screen WD32 in FIG. 17(C) displays a message indicating that registration failed (symbol d) and prompting the user to perform a read operation (symbol e). Note that for loose products such as fresh produce, a label with a barcode may be placed on the display stand, allowing the user to scan the label and enter the quantity in the app. If such a label is placed, a message prompting the user to read the barcode on the label may be displayed. The reread instruction screen WD32 in FIG. 17(C) also displays an OK button BT51 to be operated to perform a read operation in accordance with the message, a No Barcode button BT52 to be operated if the product (apple) does not have a barcode, and a Don't Purchase button BT53 to be operated to cancel the purchase of the product (apple).

When comparing the reread instruction screen WD31 in Figure 16(C) with the reread instruction screen WD32 in Figure 17(C), Figure 16 shows a scene where the barcode reading failed (although the presence of the barcode was confirmed), and therefore the No Barcode button BT52 that is present on the reread instruction screen WD32 in Figure 17(C) is not located on the reread instruction screen WD31 in Figure 16(C).

If the OK button BT51 is operated on the reread instruction screen WD32, the reread instruction screen WD32 is closed, and the process returns to step S100 in FIG. 12, as per the flowchart shown in FIG. 14. In the example of FIG. 17, the apple does not have a barcode attached, so customer K does not operate the OK button BT51. However, if the apple had a barcode attached, customer K might operate the OK button BT51.

If the No Barcode button BT52 is operated on the reread instruction screen WD32, the reread instruction screen WD32 is cleared, and although this is omitted from the flowchart shown in Figure 14, the process proceeds to step S124 in Figure 14. In other words, if the product does not have a barcode, even if the process returns to step S100 in Figure 12 and a reading operation is performed, the reading will not be successful, so the process simply proceeds to step S124 in Figure 14.

If the "Do not purchase" button BT53 is pressed on the rescan instruction screen WD32, the rescan instruction screen WD32 is closed, and although omitted from the flowchart shown in Figure 14, a cancellation process (cancellation process, not shown) is executed for the product, and the process ends.

Based on the operation of the No Barcode button BT52, the portable terminal 60 displays the preset key screen WD42 as shown in FIG. 17(D) (step S125 in FIG. 14). Note that even if the OK button BT51 is operated in a case where the apple has a barcode attached, and the re-reading operation produces the same result as the first time, the portable terminal 60 also displays the preset key screen WD42 as shown in FIG. 17(D) (step S125 in FIG. 14).

The preset key screen WD42 in FIG. 17(D) displays a message (symbol d) indicating that registration (reading) failed and a message (symbol f) prompting the user to select a product using a button (preset key). The preset key screen WD42 in FIG. 17(D) also displays multiple preset keys BT60 (preset key BT60a corresponding to apples, preset key BT60b corresponding to pears), and a preset key display switch button BT61. The preset key screen WD42 in FIG. 17(D) also displays a "Can't Select" button BT62 that is operated when the user intends to purchase the product (apples) but is unable to select the product (apples) using a preset key for reasons such as not being able to find the preset key corresponding to the product (apples), and a "Can't Purchase" button BT63 that is operated to cancel the purchase of the product (apples).

For example, if the preset key BT60a corresponding to "apple" is operated on the preset key screen WD42, the preset key screen WD42 is cleared, and the process returns to step S110 (product registration process) in FIGS. 12 and 13, as shown in the flowchart in FIG. 14. This registers the "apple" corresponding to the operated preset key BT60a.

When the display switch button BT61 is operated on the preset key screen WD42, the preset keys BT60 displayed on the preset key screen WD42 are switched. In other words, the mobile terminal 60 switches the preset keys BT60 displayed on the preset key screen WD42 based on the operation of the display switch button BT61.

If the non-selectable button BT62 on the preset key screen WD42 is operated, the preset key screen WD42 is cleared, and although omitted from the flowchart shown in FIG. 14, the process proceeds to step S130. In other words, the product (apple) is processed as a reserved product, just as when a predetermined time has elapsed (step S127 (YES)).

When the "Do not purchase" button BT63 on the preset key screen WD42 is operated, the preset key screen WD42 is cleared, and although this is omitted from the flowchart shown in Figure 14, a cancellation process (cancellation process, not shown) is executed for the product, and the process ends.

[Preset keys to display]
The preset keys displayed on the preset key screen WD41 (WD42) will be described below. The portable terminal 60 may display the preset keys in any one of the following display modes 1 to 4, for example.

(Display mode 1)
For example, the mobile terminal 60 displays preset keys for all products. Specifically, as shown in the preset key screen WD41 of Fig. 16(D), the mobile terminal 60 displays preset keys for some of the products, and displays other preset keys in response to the operation of the display switching button BT61.
In display mode 1, the number of preset keys to be displayed is extremely large. Therefore, there is a possibility that the customer will not be able to select a preset that corresponds to the product they wish to purchase. In such a case, the customer may operate, for example, the button BT62 that cannot be selected in order to proceed with the process.

(Display mode 2)
The mobile terminal 60 displays preset keys for specific products. Specifically, as shown in the preset key screen WD42 of FIG. 17(D), the mobile terminal 60 displays preset keys for specific products and does not display preset keys for other products. The specific products may be products sold individually, such as fresh produce (whether or not they have a barcode). In other words, the specific products may be products sold in a predetermined manner (for example, sold individually). Furthermore, the specific products may be products without barcodes (whether or not they are sold individually).
In display mode 2, the preset keys displayed are those corresponding to the specific product whether a specific product is registered or a product other than the specific product. In other words, when a specific product is registered, the preset keys corresponding to the product to be purchased (the specific product) are displayed, but when a product other than the specific product is registered, the preset keys corresponding to the product to be purchased (the product other than the specific product) are not displayed. Therefore, in the latter case, the customer operates, for example, the unselectable button BT62. Note that a button for displaying preset keys for products other than the specific product may be provided, and in the latter case, the preset keys for products other than the specific product may be displayed by operating the button for displaying preset keys for products other than the specific product.

(Display mode 3)
The mobile terminal 60 displays preset keys corresponding to the location within the store (the location of the mobile terminal 60 itself, or the location of the customer using the mobile terminal 60). Specifically, the mobile terminal 60 captures (identifies, estimates) the location within the store of each customer (mobile terminal 60) using GPS, wireless LAN, a wireless tag, a beacon, or the like, and displays preset keys corresponding to the captured location. For example, if the captured location is in the alcohol section, preset keys related to alcohol may be displayed, and if the captured location is in the daily necessities section, preset keys related to daily necessities may be displayed. Note that the preset keys corresponding to the captured location may be displayed preferentially (the preset keys corresponding to the captured location may be displayed on the first page, and other preset keys may be displayed depending on the operation).

(Display mode 4)
The mobile terminal 60 may determine the preset key to be displayed based on image recognition. That is, the mobile terminal 60 may display a preset key narrowed down from a large number of preset keys based on image recognition. For example, if the product is round and red, an apple or the like may be displayed as shown in the preset key screen WD42 of FIG. 17(D).

Note that display modes 1 to 4 may be combined as appropriate. Also, which of display modes 1 to 3 is adopted may vary depending on the customer.

[After payment transfer instruction]
When the customer has finished registering all of the products to be purchased, he or she performs a settlement instruction operation and has the settlement device 40 read the two-dimensional code displayed on the mobile terminal 60. After reading the two-dimensional code, the settlement device 40 requests the transaction management device 20 to calculate the subtotal, as shown in step S13 of Figure 9. Here, in this embodiment, when requesting calculation of the subtotal, the settlement device 40 requests settlement information to be used in the settlement process using the settlement screen.

In response to a request for settlement information, the transaction management device 20 identifies the relevant cart information and generates settlement information based on the identified cart information. The settlement information includes various information in the cart information (e.g., some or all of the registered product information, pending product information, cancellation information, etc.). For example, the settlement information includes image data (captured images) related to the pending product information. The transaction management device 20 transmits the generated settlement information to the settlement device 40. Upon receiving the settlement information, the settlement device 40 displays a settlement screen on the display unit 405 that reflects the contents of the received settlement information.

FIG. 18 shows an example of a settlement screen displayed on the display unit 405 of the settlement device 40.
The checkout screen in the figure includes a registered product list area AR3, which displays a list of products registered by the mobile terminal 60.

If the registered items include reserved items, the list items for the reserved items are given priority and arranged from the top of the left column. In the example shown in the figure, since the registered items include six reserved items, a reserved item list area AR31 containing list items for the six reserved items from the top left is formed in the registered item list area AR3. The list items in the reserved item list area AR31 have different aspects, such as background color, from the list items for other items.

The list items in the registered product list area AR3 show the product name, registered quantity, price according to the registered quantity, etc. of the corresponding product. In addition, for example, the list item corresponding to a reserved product may show the reserved product's classification number (reserved major classification number, reserved minor classification number, or both) and classification name (reserved major classification name, reserved minor classification name, or both).

A call button BT31 is located in each list item for each reserved item in the reserved item list area AR31. Each call button BT31 displays the name of the reserved subcategory to which the corresponding item falls.

The checkout screen also includes a customer monitor area AR32. The customer monitor area AR32 is an area in which images captured by the imaging unit 416 are displayed as monitor images in real time. The customer monitor area AR32 displays in real time the state of the customer's hands as they operate the checkout device 40 to settle their bill. By the checkout device 40 displaying such images in the customer monitor area AR32, customers become aware that they are being photographed, which has the effect of deterring customers from committing fraud when making payments. The checkout device 40 may also be configured to store (save) the images used to display in the customer monitor area AR32.

Here, as shown in the figure, if the pending item list area AR31 is displayed in the registered item list area AR3 on the checkout screen, a store clerk must perform an operation to process the pending items. When the checkout screen with the pending item list area AR31 displayed in this manner is displayed, the checkout device 40 executes a call to the store clerk to have the store clerk perform an operation to process the pending items. To call the clerk, for example, the checkout device 40 may display a display such as a signpost in a predetermined manner, or send a notification of the clerk call to a terminal device carried or monitored by the clerk. When the clerk responds to the call to the checkout device 40, the clerk performs an operation to process the pending items corresponding to each list item shown in the pending item list area AR31 on the displayed checkout screen.

To process reserved item 1, the clerk operates the call button BT31 for the list item of the reserved item to be processed. In this case, the clerk operates the call button BT31 for the list item in the first row corresponding to canned beer in the reserved item list area AR31. In response to operating the call button BT31, a reserved item processing screen is displayed on the checkout screen to process the reserved item corresponding to the list item including the operated call button BT31. Note that the entire list item of the reserved item to be processed may be touch operable.

For example, if a store clerk operates the call button BT31 for the first list item corresponding to a can of beer, which is an age verification item, the pending item processing screen for the can of beer will be displayed on the checkout screen.

Figure 19 shows an example of the reserved item processing screen AR4 for canned beer, an age verification item. The reserved item processing screen AR4 includes a product information area AR41, an input operation area AR42, a register button BT41, and a cancel button BT42.

The product information area AR41 is an area that displays information about the corresponding reserved product. The product information area AR41 in the same figure displays the product name, product code, price, registered quantity, etc. In the case of age verification products, the content displayed in the product information area AR41 reflects the information about the corresponding product (reserved product information) included in the payment information.

The input operation area AR42 is an area where a software keyboard is located for entering product information in the product information area AR41. The input operation area AR42 can be switched between a keyboard with a key layout suitable for numeric input and a keyboard with a key layout suitable for character input, depending on the operation. The input operation area AR42 also has a preset call key labeled, for example, "Preset." Operating the preset call key displays a preset key screen on which preset keys corresponding to each product are located. By operating the preset keys located on the preset key screen (preset key area), the store clerk can enter information about the product corresponding to the operated preset key into the product information area AR41.

The registration button BT41 is a button used to cancel the pending registration of the corresponding product and register it as a purchase target for the customer. The cancel button BT42 is a button used to instruct the cancellation of the registration of a product whose registration is on hold.

For example, after confirming the customer's age, if there are no problems, the store clerk operates the registration button BT41 to confirm the registration of the canned beer. This causes the settlement device 40 to cancel the pending registration status of the canned beer and process it to include it in the registered products. Alternatively, if the store clerk determines that the canned beer should not be registered, he or she operates the cancel button BT42 to cancel the registration of the canned beer and collects the canned beer product.

In addition, for items that have not been read properly, the call button BT31 in the list item in the pending item list area AR31 will display, for example, "No scan." When the store clerk processes an item that has not been read properly, they operate the call button BT31 that displays "No scan" in the list item for the item that has not been read properly.

For example, in the case of a reserved item classified as "NON-FILE," no product information is displayed in the product information area AR41 in the initial display state of the reserved item processing screen AR4. In this case, the store clerk can register the item by entering a numeric string as the product code or a character string as the product name in the product information area AR41 using numeric and character input operations in the input operation area AR42. The product information registered in this manner is sent to the transaction management device 20. The transaction management device 20 synchronizes the cart information by updating the information for the corresponding product in the cart information (reserved item information) from the "NON-FILE" state so that the information entered in the product information area AR41 is reflected.

The reserved item processing screen may also display the classification number (reserved major classification number, reserved minor classification number, or both) and classification name (reserved major classification name, reserved minor classification name, or both).

Figure 20 shows an example of the pending product processing screen AR4 for processing unreadable products, displayed in response to pressing the call button BT31 marked "No Scan." In the initial display state of the pending product processing screen AR4 corresponding to an unreadable product, no information about the product has yet been acquired, and therefore, as shown in the figure, no information about the product is displayed in the product information area AR41.

In addition, the pending product processing screen AR4 corresponding to an incompletely read product has a product information area AR41, an input operation area AR42, a registration button BT41, a cancel button BT42, and a captured image area AR43.

In the captured image area AR43, a captured image corresponding to the occurrence of the corresponding reading failure situation is displayed. This captured image was acquired by the mobile terminal 60 in step S131 of FIG. 14.

Specifically, the image in image area AR43 in the figure shows a reading failure situation classified as a non-scan insertion. Two images G11 and G12 are arranged in image area AR43 in the figure. Image G11 is an image showing the state before an item is added to the shopping cart 80. Image G12 is an image taken after image G11, showing the state after an item has been added to the shopping cart 80.

By looking at the captured image in captured image area AR43 in the figure, the store clerk can see that the unreadable product to be processed was added to the shopping cart 80 without having its barcode read. Furthermore, in captured image G12, a frame FL is placed in the area of the added product. This allows the store clerk to easily identify the added product without it being registered. The identification of the added product and the placement of the frame FL may be performed by either the mobile terminal 60 or the checkout device 40.

If the store clerk wants to check the captured image in the captured image area AR43 in more detail, they can perform an image enlargement operation, for example, by tapping any position in the captured image area AR43. In response to the captured image enlargement operation, the settlement device 40 places an enlarged captured image area AR431 on the reserved item processing screen AR4, as shown in FIG. 21. The enlarged captured image area AR431 can be erased (closed) in accordance with a specified operation.

Referring back to Figure 20 for the explanation, the store clerk will confirm with the customer whether or not they wish to purchase the unreadable product. If they do not wish to purchase, the store clerk presses the cancel button BT42 and collects the unreadable product. If they do wish to purchase, the store clerk operates the input operation area AR42 to enter product information in the product information area AR41, and then operates the register button BT41 to register the product.

Furthermore, the pending product processing screen AR4 in Figure 22 shows an example of a case where the unreadable product is classified as a "pseudo scan." In the captured image area AR43 in the same figure, one captured image is displayed. The captured image in captured image area AR43 displayed in this way shows that when the unreadable situation occurred, the barcode was captured with part of it obscured by a finger. By looking at this captured image in captured image area AR43, the store clerk can see that the unreadable product to be processed is classified as a pseudo scan.

In this case, too, by performing an operation on the captured image area AR43, the enlarged captured image area AR431 can be displayed on the reserved item processing screen AR4, as shown in FIG. 23.

In this case, the store clerk will also confirm with the customer whether they wish to purchase the unreadable product, and will register or cancel the product depending on whether they wish to purchase it.

The pseudo scan also includes a situation where a product is held over the imaging range of the imaging unit 606, but the barcode is not included in the imaging range. In this case, the captured image area AR43 may be configured to display an image in which part of the product is included in the imaging range, but the barcode is not included in the imaging range.

The pseudo scan also includes a situation in which a product passes through the imaging range of the imaging unit 606 at a speed that makes it impossible to read the product code. In this case, the captured image area AR43 may display a still image captured as the product passes through the imaging range, or a video image captured as the product passes through the imaging range.

With reference to the flowchart in Figure 24, an example of the processing procedure executed by the settlement device 40 in relation to the display of the reserved item processing screen will be described. Note that at the start of the flowchart in Figure 24, the settlement device 40 is assumed to be displaying the settlement screen (Figure 18).

Step S301: While the settlement screen is displayed, the settlement device 40 determines whether an operation to specify a reserved item to be processed (reserved item specification operation) has been performed. If a reserved item specification operation has been performed (Step S301: YES), proceed to Step S302. If a reserved item specification operation has not been performed (Step S301: NO), return to Step S301.

Step S302: The settlement device 40 determines whether the reserved item specified by the reserved item specification operation is an incompletely readable item (a reserved item with a reserved sub-category number "21" to "24"). If it is an incompletely readable item (Step S302: YES), proceed to Step S303. If it is not an incompletely readable item (Step S302: NO), proceed to Step S304.

Step S303: The settlement device 40 generates a reserved item processing screen including an image capture area AR43 for the reserved item designated for processing (reserved items with reserved sub-category numbers "21" to "24" corresponding to unreadable items). The settlement device 40 may obtain the captured image in the image capture area AR43 from the reserved item information for the corresponding item in the settlement information. The settlement device 40 displays the generated reserved item processing screen. This flowchart then ends.

Step S304: The settlement device 40 generates a reserved item processing screen that does not include the captured image area AR43 for the reserved items designated for processing (reserved items with reserved sub-category numbers "11," "31" to "33," and "41" that do not correspond to misreadable items). The settlement device 40 displays the generated reserved item processing screen. This flowchart then ends.

<Modification>
A modification of this embodiment will now be described.

[First Modification]
In the above embodiment, one or two captured images (FIGS. 20 and 22) are displayed in the captured image area AR43 when the reserved product is an incompletely read product. However, the number of captured images displayed in the captured image area AR43 may be three or more.

Figure 25 shows an example in which three captured images G11, G12, and G13 are arranged in the captured image area AR43 of the reserved item processing screen AR4. The reserved item processing screen AR4 in this figure shows an example in which the unreadable item to be processed is classified as a non-scanned item.

The captured images G11, G12, and G13 in the figure show the following. In the captured image area AR43, the captured images G11, G12, and G13 are arranged in this order from left to right. The arrangement order of the captured images G11, G12, and G13 corresponds to the time lapse in which the images were captured. The first captured image G11 was captured when the shopping cart 80 was filled with images before the unreadable product to be processed was added. The second captured image G12 was captured when the captured image G11 changed into the next, third captured image G13. The third captured image G13 was captured when the unreadable product to be processed was added to the shopping cart 80. Note that the second captured image G12 may be extracted from captured images containing the product to be added, or may be captured at a predetermined time, such as exactly midway between the times when the captured images G11 and G13 were captured. Additionally, in the captured image G13, a frame FL is placed over the product that has been added to the shopping cart 80.

[Second Modification]
In the above embodiment, the display screen of the display unit 405 of the settlement device 40 is horizontally long. However, the display screen of the display unit 405 of the settlement device 40 may be vertically long.

Figure 26 (A) shows an example of a checkout screen displayed on the display unit 405 with a vertically long display surface corresponding to this modified example. In this figure, parts that are the same as those in Figure 18 are given the same reference numerals. As can be seen by comparing this figure with Figure 18, the information presented on the checkout screen is the same, but the layout of the information presented on the checkout screen is different due to the vertically long display surface size. However, the checkout screen in Figure 26 (A) also has a pending product list area AR31 in the upper left corner of the registered product list area AR3, which includes list items for pending products, and is different in background color, etc. from list items other than pending products, just like the checkout screen in Figure 18.

In this modified example, for example, the reserved item list area AR31 in FIG. 26(A) can be enlarged and displayed as an enlarged reserved item list area AR311 as shown in FIG. 26(B). The operation to display the enlarged reserved item list area AR311 may be an operation of touching any position within the reserved item list area AR31. Alternatively, it may be an operation of touching the reserved item count area AR33 located on the checkout screen in FIG. 26(A).

Displaying the expanded pending item list area AR311 in this way makes it easier for store clerks to see the list items of pending items and understand what pending items are available. Furthermore, for example, depending on the placement of the display unit 405 on the payment device 40, the display unit 405 may be positioned high, making it difficult to operate the upper part of the display surface of the display unit 405, especially when the display surface of the display unit 405 is vertically long. Therefore, in the example of Figure 26 (B), the expanded pending item list area AR311 is placed on the lower part of the display surface. This makes it easier for store clerks to operate the expanded pending item list area AR311.

When a store clerk processes a pending item, they operate the call button BT31 located on the list item of the pending item to be processed among the list items in the expanded pending item list area AR311 displayed as shown in Figure 26 (B). In response to the operation of the call button BT31, the pending item processing screen AR4 is displayed in place of the expanded pending item list area AR311, as shown in Figure 26 (C). The pending item processing screen AR4 displayed in this way processes the pending item indicated by the list item of the operated call button BT31.

Furthermore, if the pending product to be processed is an unreadable product, the settlement device 40 displays the captured image area AR43 on the same screen as the pending product processing screen AR4, as shown in the same Figure 26 (C). Note that this figure shows an example in which one image is displayed in the captured image area AR43. However, even in the case of a vertically long screen like this, multiple predetermined captured images may be displayed in the captured image area AR43 depending on the reading failure status (according to the pending minor classification number), as shown in Figures 20 and 25.

In addition, in the examples shown in Figure 20 and elsewhere, the captured image area AR43 was located within the pending item processing screen AR4. In contrast, in Figure 26 (C), due to the vertically long display surface of the display unit 405, the captured image area AR43 is not included in the pending item processing screen AR4. This allows the captured image area AR43 to be enlarged and placed at a larger size than when it is placed within the pending item processing screen AR4, making the captured image easier to see. Also, as shown in Figure 26 (C), on the vertically long display surface, the captured image area AR43 is placed at the top and the pending item processing screen AR4 is placed at the bottom. This makes it easier for store clerks to operate the pending item processing screen AR4.

[Third Modification]
In the above embodiment, the captured image area AR43 is displayed on the pending product processing screen for processing pending products for which the barcode has not been read, and the captured image area AR43 is not displayed on the pending product processing screen for pending products other than the pending product for which the barcode has not been read. However, in this modified example, the captured image area AR43 may also be displayed on the pending product processing screen for pending products other than the pending product for which the barcode has not been read. In this case, the mobile terminal 60 may be configured to transmit product registration information including the captured image for registered products for which the barcode has been successfully read.

[Fourth Modification]
The captured image area AR43 may be displayed as an independent screen different from the reserved item processing screen, for example, depending on the store clerk's operation on the payment device 40.

[Fifth Modification]
When product registration is initiated by operating the transaction start instruction button BT10 on the startup screen of FIG. 10(A), the mobile terminal 60 may display a warning screen informing the customer in advance that "images taken at the time of product registration may be displayed on the checkout device." Displaying such a warning screen may, for example, prevent fraudulent product registration. Alternatively, instead of the warning screen, the mobile terminal may display a consent screen that asks the customer to agree to the "images taken at the time of product registration may be displayed on the checkout device." The consent screen may include buttons, such as a "Yes" button and a "No" button, that accept a confirmation of intent. If the "Yes" button is then operated, the customer may be treated as a non-fraudulent customer, and the cancellation may be confirmed without registering, for example, a canceled product as a pending product.

[Sixth Modification]
In the above embodiment, a situation where the barcode is not successfully read is treated as a product registration error. In this modified example, in addition to a situation where the barcode is not successfully read, a situation where, for example, the barcode is successfully read but the product cannot be added to the shopping cart 80 may also be treated as a product registration error. Note that, for example, rice over a certain weight or six cans of canned drinks are often placed at the bottom of the cart. Therefore, for such products, even if it is detected that they cannot be added to the shopping cart 80, they may not be treated as a product registration error.

[Seventh Modification]
For example, due to a communication error or a communication speed limit set on the mobile terminal 60, it may be impossible to send the captured image to the transaction management device 20 when sending the unreadable product information in step S132 of FIG. 14 . In such a case, the transaction management device 20 may receive the unreadable product information including the reserved subcategory number but not the captured image. Alternatively, the mobile terminal 60 may resend the unreadable product information without the captured image, allowing the transaction management device 20 to receive the unreadable product information including the reserved subcategory number but not the captured image. In this way, even in the case of a communication error or a communication speed limit set on the mobile terminal 60, the transaction management device 20 may be able to register at least the reserved subcategory number for the unreadable product in the cart information whenever possible.
As a result, by using the settlement information based on the cart information, the settlement device 40 can display, for example, a pending product processing screen for the unreadable product as shown in Fig. 20, although the captured image is not displayed in the captured image area AR43. In this case, a message that can inform the store clerk that the unreadable product is present, such as "Are there any products that you forgot to scan?", may be displayed in the blank area of the pending product processing screen where the captured image area AR43 is not displayed.

[Eighth Modification]
The occurrence of a reading failure may be displayed on a terminal (store terminal) operating in the store other than the settlement device 40, such as the transaction status management device 50. For example, the transaction status management device 50, which is one of the store terminals, can manage the status of each transaction in the store by communicating with the transaction management device 20, for example. When a pending item is registered, the transaction status management device 50 may acquire information identifying the customer to whom the pending item corresponds, and product information and price information for the target product at the time the pending item is registered, and display the acquired information. Then, for example, if a store clerk performing management tasks using the transaction status management device 50 confirms that there is no problem with the displayed product, the clerk may be able to cancel the registration of the product as a pending item and re-register it as a regular item.

[Ninth Modification]
The imaging device used for product registration is not particularly limited as long as it is capable of reading product barcodes, is capable of capturing images of the contents of the shopping cart 80, etc., and is capable of outputting captured images to the mobile terminal 60, etc. Such an imaging device may be capable of capturing images of the inside of a customer's own bag, rather than the shopping cart 80 provided by the store, for example.

[Tenth Modification]
The mobile terminal 60 may be held by a customer and used to register products without being placed in the shopping cart 70. In this case, it is sufficient that the placement of a product in the shopping cart 70 is detected. For example, a device (terminal) equipped with an imaging unit capable of detecting the placement of a product in the shopping basket 80 (shopping bag 90) may be installed, or a device equipped with a weighing unit capable of detecting the placement of a product in the shopping basket 80 (shopping bag 90) may be installed. Even in a configuration in which the placement of a product in the shopping cart 70 is not detected (a configuration in which no device other than the mobile terminal 60 is used), the mobile terminal 60 cannot transition to the reading failure process from step S102 (YES) in FIG. 12, which is related to the placement of a product, but can transition to the reading failure process from step S103 (YES) in FIG. 12, which is not related to the placement of a product. In other words, it can at least recognize the reserved product with the reserved subcategory number "21" (pseudo scan (coated, etc.)) and the reserved product with the reserved subcategory number "23" (bad barcode).

[Eleventh Modification]
In the above embodiment, the preset key is not displayed in the first reading failure situation, but is displayed in the second reading failure situation ( FIG. 14 ). However, the preset key may be displayed in the first reading failure situation. For example, steps S121 to S124 in FIG. 14 may be omitted. Alternatively, the preset key may not be displayed in the first reading failure situation (N (where N is an integer greater than or equal to 2)) but may be displayed in the (N+1)th reading failure situation. In this case, instead of a flag indicating whether or not a reading has been performed after a reading failure situation (reread operation instruction completion flag), a counter may be used to count the number of readings performed after a reading failure situation. The number of times (0 or more) until the preset key is displayed may vary depending on the customer. For example, past purchasing history, the number of occurrences of reading failure situations, etc. may be stored for each customer, and the number of times (0 or more) until the preset key is displayed may vary depending on the past purchasing history, the number of occurrences of reading failure situations, etc. Furthermore, instead of or in addition to the customer, the number of times (0 or more) until the preset key is displayed may be varied depending on the type of product.

[Twelfth Modification]
In the above embodiment, no particular distinction is made between products registered by reading a barcode and products registered using a preset key. However, products registered by reading a barcode and products registered using a preset key may be stored and output (displayed, etc.) so as to be distinguishable from each other. For example, in the cart information (registered product N), products registered by reading a barcode and products registered using a preset key may be stored so as to be distinguishable from each other. For example, information indicating that a product registered using a preset key was registered using a preset key may be stored for each product. In the cart information (registered product information (total)), products registered by reading a barcode and products registered using a preset key may be stored so as to be distinguishable from each other. For example, a breakdown (number of items, estimated subtotal price, etc.) of products registered using a preset key may be stored. Furthermore, the mobile terminal 60 may display products registered by reading a barcode and products registered using a preset key in a registered product list such as that shown in FIG. 10(C), so as to be distinguishable from each other. For example, the mobile terminal 60 may display information in the registered product list indicating that a product registered using a preset key was registered using a preset key. Furthermore, the settlement device 40 may distinguish between products registered by reading a barcode and products registered using a preset key on a settlement screen such as that shown in FIG. 18 . For example, the settlement device 40 may display information on the settlement screen indicating that a product registered using a preset key was registered using a preset key.

[Thirteenth Modification]
In the above embodiment, the mobile terminal 60 detects a read failure state indicating that a customer has secured (added to the shopping cart 80 (shopping bag 90)) a product whose product code has not been read. However, instead of the mobile terminal 60, another device (e.g., the transaction status management device 50, the store controller 10, the transaction management device 20, etc.) may detect the read failure state. The transaction status management device 50 (similarly applicable to the store controller 10 and the transaction management device 20) may detect the read failure state, for example, as follows: The mobile terminal 60 notifies the transaction status management device 50 at the respective times that it has read the product code and that the product has been added to the shopping cart 80. If the mobile terminal 60 notifies the transaction status management device 50 that the product code has been read but not the transaction status management device 50 notifies the transaction status management device 50 that the product has been added to the shopping cart 80, the transaction status management device 50 determines that a read failure state has occurred (i.e., detects the read failure state). When the transaction status management device 50 (similar to the store controller 10 and transaction management device 20) detects a reading failure, it notifies the mobile terminal 60 that a reading failure has occurred. When the mobile terminal 60 is notified by the transaction status management device 50 (similar to the store controller 10 and transaction management device 20) that a reading failure has occurred, it executes a reading failure process (FIG. 14).

[Fourteenth Modification]
In the above embodiment, the imaging unit 606 of the mobile terminal 60 is used to read the product code and determine whether a product has been inserted, but it is not necessary to use the imaging unit 606 for both. For example, a device equipped with an imaging means capable of detecting the insertion of a product into the shopping basket 80 (shopping bag 90) may be installed (for example, installed on each shopping cart 70, installed on the ceiling of the store, etc.). In other words, the imaging unit 606 of the mobile terminal 60 may be used to read the product code, and a device installed on the shopping cart 70 or on the ceiling of the store may be used to detect the insertion of the product.

[Fifteenth Modification]
In the above embodiment, the insertion of a product is determined by the imaging unit 606 (captured image), but it does not have to be determined by the captured image. For example, instead of or in addition to the imaging means, a weighing device (for example, installed on each shopping cart 70) that can detect the insertion of a product into the shopping basket 80 (shopping bag 90) may be used.

[16th Modification]
In the above embodiment, in the case of a misreading, a message prompting the user to read the item is displayed regardless of the reserved subcategory number "21" to "24," and a preset key is displayed if the item is misread again. However, the control may be different depending on the classification. For example, for a certain reserved subcategory number (e.g., reserved subcategory number "24"), a preset key may be displayed without displaying a message prompting the user to read the item, while for other reserved subcategory numbers (e.g., reserved subcategory numbers "21" to "23"), a message prompting the user to read the item may be displayed and then the preset key may be displayed. Furthermore, for a certain reserved subcategory number (e.g., reserved subcategory number "24"), a preset key may be displayed (with or without a message prompting the user to read the item), while for other reserved subcategory numbers (e.g., reserved subcategory numbers "21" to "23"), the preset key may be displayed without displaying the preset key (with or without a message prompting the user to read the item). This allows processing to be performed according to the situation (misreading situation). In the above embodiment (when different processes are performed for each classification of reserved items), the process of determining the reserved subcategory number of the reserved item (step S130 in FIG. 14) may be performed earlier (for example, at the beginning of the reading failure process (S120)). Note that although the "certain reserved subcategory number" is the reserved subcategory number "24" and the "other reserved subcategory numbers" are the reserved subcategory numbers "21" to "23," the combination of the "certain reserved subcategory number" and the "other reserved subcategory numbers" is not limited to this. Furthermore, instead of two combinations such as "certain reserved subcategory number" and "other reserved subcategory numbers," three to five combinations are also possible.

[Seventeenth Modification]
In the above embodiment, the mobile terminal 60 displays the preset keys (candidate products), but other devices (e.g., the settlement device 40, the registered settlement device 30, etc.) may display the preset keys (candidate products) instead of or in addition to the mobile terminal 60. For example, the mobile terminal 60 may not display the preset keys, and the settlement device 40 may display the preset keys (candidate products) for each of the reserved products (reserved products with reservation subcategory numbers "21" to "24"). Also, for example, the mobile terminal 60 may display the preset keys (candidate products), and if registration using the preset keys is not performed (i.e., the reserved products are), the settlement device 40 may display the preset keys (candidate products) for each of the reserved products (reserved products with reservation subcategory numbers "21" to "24").

[18th Modification]
Note that all or part of the functions (input/output, storage, processing (including judgment)) of each device described above (store controller 10, transaction management device 20, registration and settlement device 30, settlement device 40, transaction status management device 50, mobile terminal 60, etc.) may be realized by a device other than the device described as the entity that executes the function. For example, as the processing of the mobile terminal 60, at least part of the processing described using the left column of the sequence diagram of FIG. 9 and at least part of the processing described using the flowcharts of FIGS. 12 to 15 may be executed by the transaction management device 20, the store controller 10, or the settlement device 40 instead of the mobile terminal 60. Furthermore, as the processing of the settlement device 40, at least part of the processing described using the middle column of the sequence diagram of FIG. 9 may be executed by the transaction management device 20, the store controller 10, or the mobile terminal 60 instead of the settlement device 40.
In relation to the above, the mobile terminal 60 may function as a so-called thin client, specializing in an input/output interface, for the present shopping service (e.g., product registration described in the sequence diagram of FIG. 9 and the flowcharts of FIGS. 12 to 15). For example, instead of the mobile terminal 60 executing a shopping application (no installation on the mobile terminal 60 is required), an external device (such as the transaction management device 20 or another cloud server) may execute the shopping application. In other words, the mobile terminal 60 may accept various inputs (customer operations, detection by a device such as a camera), transmit input information (operation information, image information, etc.) to an external device, receive processing results of the external device based on the input information (e.g., updated screen information, device control information), and perform various outputs (display on a display unit, control of a device). Another device (such as the settlement device 40) may also function as a thin client instead of or in addition to the mobile terminal 60.

<Summary of the embodiment>
[Technical Field]
The present invention relates to a product sales data processing system, a mobile terminal, and a program.
[Background technology]
A system is known in which customers themselves register products using a mobile terminal equipped with a camera (see, for example, Patent Document 1).
[Prior art documents]
[Patent Documents]
[Patent Document 1] JP 2018-147252 A [Summary of the Invention]
[Problem to be solved by the invention]
However, various problems can arise when reading barcodes when customers register products themselves. The present invention has been made in light of these circumstances, and aims to provide a system that can appropriately address the problems that arise when customers register products themselves.

[Means for solving the problems]
(A1) One aspect of this embodiment that solves the above-mentioned problems is a merchandise sales data processing system (1) that registers merchandise using a mobile terminal (e.g., mobile terminal 60) operated by a customer of a store, and that includes: a reading means that reads merchandise codes; an acquisition means that acquires merchandise information corresponding to the merchandise code read by the reading means; a detection means (e.g., processing of step S102 in Figure 12) that detects a reading failure situation that indicates that a customer has reserved a merchandise whose merchandise code has not been read by the reading means (e.g., that a merchandise whose merchandise code has not been read has been placed in a shopping cart 80 (shopping bag 90)); and a display means that, when the detection means detects the reading failure situation, displays candidate merchandise in a selectable manner (e.g., displays in Figures 16(D) and 17(D)), and the acquisition means acquires merchandise information of the candidate merchandise selected by the customer's selection operation.

The above configuration makes it possible to appropriately deal with problems that arise when customers register products themselves. For example, it is possible to appropriately deal with various problems, such as problems related to the state of the barcode (including whether or not the barcode is present), and problems such as customers misreading the barcode or fraud. Examples of misreading or fraud by customers include customers who think they have read the barcode of a product but are actually unable to do so, or customers who maliciously put products into their shopping cart without having the barcode read.
(a) For example, in a system where customers register products themselves, it is preferable to affix barcodes to loosely sold products such as fresh produce, just like other products. This is because customers can register loosely sold products by scanning the barcodes attached to the products, just like other products. However, depending on the shape of the product, the affixed barcode may become wrinkled, making it difficult to read. Furthermore, the task of affixing barcodes to loosely sold products is itself a burden for the store.
According to the above configuration, candidate products are displayed if reading of the barcode fails due to, for example, wrinkles. Furthermore, according to the above configuration, candidate products are also displayed if reading of the barcode fails because, for example, no barcode is attached. Therefore, even if the barcode attached to a product sold individually is wrinkled, it does not cause a problem, and furthermore, the work of attaching a barcode to a product sold individually can be omitted, eliminating the burden involved.
(b) Furthermore, if a customer is unfamiliar with barcode reading or if the customer's reading action is improper, the reading may fail, and an unread product may be placed in the cart. Also, a customer may accidentally forget to read the barcode, and an unread product may be placed in the cart. Furthermore, there may be customers who intentionally fail to read the barcode or who place products in the cart without scanning it.
According to the above configuration, no matter what the situation, if the barcode reading fails or if a product is added to the basket without being read, the candidate products are displayed. Therefore, it is possible to prevent or deter the removal of unregistered products (so-called basket slip-out), regardless of whether they are done so intentionally or maliciously.
(c) In addition, products that are not registered at the product registration stage in the store will be processed at the checkout stage as so-called reserved products, but if there are a lot of reserved products, the number of operations required for the reserved products at the checkout stage will increase, which will be a burden.
With this configuration, if the barcode reading fails or if a product is added to the cart without being read, candidate products are displayed and the selected reserved product is registered. This reduces the number of reserved products at the registration stage, reduces the number of operations at the checkout stage, and reduces the burden.

(A2) One aspect of this embodiment is the product sales data processing system described in (A1), wherein the display means, when the detection means detects the reading failure state, displays information (for example, the displays shown in Figures 16(C) and 17(C)) prompting the user to reread the product code before displaying the candidate products; when the reading means successfully rereads the product code, the candidate products are not displayed; and when the reading means fails to reread the product code, the candidate products are displayed.

The above configuration allows for the display of candidate products in an optimal manner. In other words, with the above configuration, candidate products are not displayed when a reading failure is detected the first time, but are displayed when a reading failure is detected again. Therefore, for example, if a reading failure occurs by chance, the customer is prompted to re-scan and, if possible, to register using the original (official) registration method (this also serves to inform the customer), and if the re-scanning does not work, the customer can be allowed to register from the candidate products as needed.

(A3) One aspect of this embodiment is a product sales data processing system as described in (A1) or (A2), which includes a reserved product determination means that, if no selection operation of a candidate product is performed after the detection of the reading failure state, determines that the product that caused the reading failure state (e.g., the reading failure state) is a reserved product for which payment is suspended at the time of checkout (the process is temporarily suspended for confirmation by a store clerk, etc. at the time of checkout), and if a selection operation of a candidate product is performed after the detection of the reading failure state, the reserved product determination means determines that the product that caused the reading failure state is not the reserved product.

With the above configuration, it is possible to appropriately determine (sort) whether or not to place an incompletely read product on hold, depending on whether or not a candidate product has been selected.

(A4) One aspect of this embodiment is the product sales data processing system described in (A3), wherein the acquisition means, if a selection operation for a candidate product is performed after the detection of the reading failure state, acquires product information for the candidate product selected by the selection operation as product information for the product that caused the reading failure state; and, if no selection operation for a candidate product is performed after the detection of the reading failure state, acquires information (e.g., a captured image) used to detect the reading failure state as information about the pending product.

According to the above configuration, it is possible to acquire information according to whether or not a selection operation has been performed on a candidate product. In other words, if a selection operation on a candidate product has been performed, the selection operation is considered to be correct and product information on the candidate product is acquired, and if a selection operation on a candidate product has not been performed, captured images or the like can be acquired as information on the reserved product.
In addition, if no selection operation of a candidate product is performed after the detection of a reading failure situation, information regarding the reading failure situation (e.g., a reserved subcategory number, etc.) may be obtained instead of or in addition to the information used to detect the reading failure situation.

(A5) One aspect of this embodiment is a product sales data processing system described in any one of (A1) to (A4), in which, when the detection means detects the reading failure state, products sold in a predetermined sales method (e.g., sold individually) are displayed as selectable candidate products.

The above configuration makes it possible to display suitable candidate products. For example, with the above configuration, if a product sold through a specific method is prone to reading errors, suitable products (products prone to reading errors) can be displayed as candidate products.

(A6) One aspect of this embodiment is a product sales data processing system described in any one of (A1) to (A4), in which, when the detection means detects the reading failure state, products corresponding to the customer's location within the store are displayed as selectable candidate products.

The above configuration makes it possible to display suitable candidate products. For example, with the above configuration, if a reading failure is detected near a certain display shelf, suitable products (products displayed on that display shelf) can be displayed as candidate products.

(A7) One aspect of this embodiment is a product sales data processing system described in any one of (A1) to (A6), which includes an imaging means for capturing an image of a reserved product that has been reserved by a customer, and the imaging means is capable of capturing an image of the reserved product as well as the reading operation in which the customer reads the product code, the reading means reads the product code from the captured image of the reading operation, and the detection means detects the reading failure situation based on a change in the captured image of the reserved product.

The above configuration can be easily achieved using an imaging device.

(A8) One aspect of this embodiment is a mobile terminal operated by a customer in a store, comprising: a reading means for reading a product code; an acquisition means for acquiring product information corresponding to the product code read by the reading means; a detection means for detecting a reading failure state indicating that a product whose product code has not been read by the reading means has been reserved by the customer; and a display means for displaying candidate products in a selectable manner when the detection means detects the reading failure state, wherein the acquisition means acquires product information for the candidate product selected by the customer's selection operation.

The above configuration can achieve the same effect as (A1).

(A9) One aspect of this embodiment is a program for causing a mobile terminal operated by a customer in a store to function as a computer, the program causing the computer to function as a reading means for reading product codes, an acquisition means for acquiring product information corresponding to the product code read by the reading means, a detection means for detecting a reading failure state indicating that a product whose product code has not been read by the reading means has been reserved by the customer, and a display means for displaying candidate products in a selectable manner when the detection means detects the reading failure state, and the acquisition means acquires product information for the candidate product selected by the customer's selection operation.

The above configuration can achieve the same effect as (A1).

(B1) One aspect of this embodiment that solves the above-mentioned problems is a merchandise sales data processing system (1) in which merchandise is registered using a mobile terminal (e.g., mobile terminal 60) and a settlement device (40) performs settlement processing corresponding to the merchandise registered using the mobile terminal, the merchandise sales data processing system comprising: an imaging means (e.g., imaging unit 606) that is capable of capturing images of the status of merchandise registration by a customer; a detection means that detects a predetermined reading failure status in which merchandise registration (reading) is deemed unsuccessful while the merchandise is being registered in accordance with a transaction; and a display control means that, when settlement is performed for the merchandise registered in accordance with the transaction at the settlement device, displays an image (e.g., an image in image capture area AR43) corresponding to the reading failure status detected by the detection means, based on the image captured by the imaging means.

With the above configuration, if a reading error occurs while a customer is registering a product using the mobile terminal 60, an image of the reading error can be displayed. This makes it possible to, for example, prevent reading error situations (e.g., prevent fraudulent product registration activities) and appropriately deal with products that are subject to reading error situations, thereby enabling appropriate responses to problems that arise when customers register products themselves.

(B2) One aspect of this embodiment is the product sales data processing system described in (B1), in which the display control means classifies a reading failure situation as one in which a product is inserted into a product insertion section (e.g., shopping cart 80) without its product information being read (non-scan insertion), by displaying multiple captured images corresponding to a predetermined period before and after the product is inserted into the product insertion section.

With the above configuration, when the classification of the reading failure situation is non-scanned input (for a reserved item with the reserved subcategory number "24"), it is possible to display an image of the contents of the shopping cart 80 captured from before the target item is inserted until it is inserted. This allows the user to accurately grasp the state transition of the contents of the shopping cart 80 when the item is inserted without being scanned.

(B3) One aspect of this embodiment is a product sales data processing system as described in (B2), which displays a display showing parts of a predetermined captured image from the plurality of captured images that differ from the other captured images.

With the above configuration, a frame FL showing the additionally inserted product can be placed in the image of the additionally inserted product among the multiple images displayed when the classification of the reading failure status is non-scan insertion. This allows the additionally inserted product to be immediately identified.

(B4) One aspect of this embodiment is a product sales data processing system described in any one of (B1) to (B3), wherein the display control means displays captured images with different imaging subjects depending on the classification of the reading failure situation detected by the detection means (reserved minor classification number "21" to "24").

With the above configuration, by displaying a captured image of the target according to the classification of the reading failure situation, it is possible to display an appropriate captured image as evidence according to the reading failure situation.

(B5) One aspect of this embodiment is a product sales data processing system described in any one of (B1) to (B4), wherein the display control means displays an image captured when an operation to read the product information was being performed in the case of a reading failure status classified as an unsuccessful reading of the product information (in the case of a reserved product with a reserved sub-category number of "21" to "24").

With the above configuration, for example, in the case of a false scan or a defective barcode, it is possible to display an image of the product that was within the imaging range of the imaging unit 606 when a reading failure was determined. This makes it possible to identify the product whose barcode should have been read in the case of a false scan or a defective barcode.

(B6) One aspect of this embodiment is a product sales data processing system described in any one of (B1) to (B5), in which the display control means displays a screen (e.g., a pending product processing screen AR4) for processing pending products identified as unreadable products at the checkout device, along with an image captured corresponding to the unreadable product.

With the above configuration, if the pending item is an unreadable item, the captured image area AR43 can be displayed along with the pending item processing screen AR4, which is displayed so that the store clerk can process the pending item. This allows the store clerk to properly process the pending item as an unreadable item.

(B7) One aspect of this embodiment is a mobile terminal (e.g., mobile terminal 60) capable of registering products, which includes an imaging means capable of capturing images of the status of product registration by a customer, a detection means for detecting a predetermined reading failure status in which product registration (reading) is deemed unsuccessful while the product is being registered for a transaction, and a display control means for displaying product information for the registered product and an image captured by the imaging means (e.g., a product registration screen), and displaying information about the detected reading failure status (e.g., a guide image WD2) in response to the detection of the reading failure status by the detection means.

With the above configuration, when a reading failure occurs, a display related to the reading failure is displayed on the mobile terminal 60. This allows the customer to understand that a reading failure has occurred. Furthermore, this has the effect of deterring customers from attempting fraud.

In addition, programs for implementing the functions of the above-mentioned store controller 10, transaction management device 20, registration and settlement device 30, settlement device 40, transaction status management device 50, mobile terminal 60, etc. may be recorded on a computer-readable recording medium, and the programs recorded on the recording medium may be loaded into a computer system and executed to perform processing for each of the above-mentioned devices. Here, "loading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The term "computer system" here includes hardware such as an operating system and peripheral devices. Furthermore, a "computer system" may include multiple computer devices connected via a network, including communication lines such as the Internet, WAN, LAN, and dedicated lines. Furthermore, "computer-readable recording medium" refers to portable media such as floppy disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. Thus, recording media storing programs may be non-transitory recording media such as CD-ROMs. Recording media also include internal or external recording media accessible from a distribution server for distributing the program. The program code stored on the distribution server's recording medium may be different from the program code in a format executable on the terminal device. In other words, the format stored on the distribution server is not important as long as it can be downloaded from the distribution server and installed in a format executable on the terminal device. The program may be divided into multiple parts, downloaded at different times, and then combined on the terminal device, or each part may be distributed by a different distribution server. Furthermore, the term "computer-readable recording medium" also includes a storage medium that stores a program for a certain period of time, such as volatile memory (RAM) within a computer system that acts as a server or client when a program is transmitted over a network. The program may also be a program that realizes part of the functions described above. Furthermore, the program may be a so-called differential file (differential program) that can realize the functions described above in combination with a program already stored on the computer system.

1. Product sales data processing system, 10. Store controller, 20. Transaction management device, 40. Payment device, 50. Transaction status management device, 60. Mobile terminal, 70. Shopping cart, 80. Shopping basket

Claims (4)

A product sales data processing system for registering products using a mobile terminal operated by a customer of a store,
reading means for reading a product code;
an acquisition means for acquiring product information corresponding to the product code read by the reading means;
a detection means for detecting a reading failure state as a state in which the reading means has failed to read the product code;
a display means for displaying a message to instruct the customer to read the product code when the detection means detects the reading failure state , and for displaying products of a predetermined sales method as candidate products so that the customer can select the product code if the product code is not read by the reading means after the message is displayed ;
a determination means for determining a classification of the unreadable commodity in which the unreadable situation has occurred, when a predetermined time has elapsed since the display means displayed the commodity of a predetermined sales method as a selectable candidate commodity;
Equipped with
A product sales data processing system. The display means
The product sales data processing system of claim 1, characterized in that when a message is displayed instructing the customer to read a product code, and when products with a specified sales method are displayed as candidate products that can be selected, an operation to cancel the purchase of the product is displayed so that it can be accepted. The detection means
If a customer acquires a product whose product code has not been read by the reading means before a predetermined time has elapsed since the start of the reading operation, or if the product code has not been read by the reading means before a predetermined time has elapsed since the start of the reading operation, this is detected as a reading failure situation;
The determining means
If the reading failure state detected by the detection means is a reading failure state in which a customer acquires a product whose product code has not been read by the reading means before a predetermined time has elapsed since the start of the reading operation, the reading failure state is classified as a pseudo-scan product,
A product sales data processing system as described in claim 1 or claim 2, characterized in that if the reading failure situation detected by the detection means is a reading failure situation in which the product code is not read by the reading means before a predetermined time has elapsed since the reading operation began, the reading failure product is classified as a product with a defective barcode or a false scan. The detection means
If the product is secured by the customer without the reading operation being performed, this is detected as a reading failure situation;
The determining means
A product sales data processing system as described in any one of claims 1 to 3, characterized in that if the reading failure situation detected by the detection means is a reading failure situation in which the product is secured by a customer without a reading operation being performed, the reading failure product is classified as a non-scanned product.