WO2022113653A1 - Designed article recognition information providing system, designed article recognition information providing method, and designed article recognition information providing program - Google Patents

Abstract

Provided is a designed article recognition information providing system that enables a customer to easily acquire information content in a short time. A designed article recognition information providing system 1 is provided with: a control unit 30 that executes steps S11, S12 for acquiring a captured image in which at least a part of a designed article is included in one captured image and imaging position information indicating an imaging position at which the captured image is captured, step S14 for extracting designed article data having a predetermined position relationship range between the imaging position of the captured image and designed article position data, and step S16 for extracting designed article data having designed article image data matching with or similar to said designed article image data in terms of a design and/or an entire image; and a display unit 44 for a search terminal 4 and the control unit 30, for executing step S17 for displaying a home page associated in advance with the designed article data extracted by a match designed article data extraction means or at least one data item included in said designed article data.

Description

Design body recognition information provision system, design body recognition information provision method, and design body recognition information provision program

The present invention relates to a design body recognition information providing system that performs image recognition of an captured design body image and provides specific information, a design body recognition information providing method, and a design body recognition information providing program.

Conventionally, when trying to obtain information such as the route to the store and the products or services handled by the store that provides a specific product or service, the corresponding store opened on the Internet using a terminal owned by the customer. The method of accessing the homepage of is used.

Various methods have been proposed to simplify the procedure for accessing such a homepage.

For example, a method has been proposed in which a customer is provided with store information corresponding to the two-dimensional code by reading the two-dimensional code storing the tag identification information using a terminal owned by the customer. (See Patent Document 1)

In addition, the direction, angle, or direction of the image is specified, and the object (for example, an external structure that is a building that becomes a landmark) reflected in the captured image data is identified by image recognition and related to the object. A method of presenting information to be done is also proposed. (See Patent Document 2)

However, when a customer searches for a store using a terminal, it takes time and effort to set conditions such as search words, and if the search words are not sufficiently narrowed down, many searches are performed. There was a problem that it took time and effort to search the homepage of the relevant store from the results. In addition, the method of using a two-dimensional code requires creating a two-dimensional code that stores information in advance and writing it on the advertising signboard inside or outside the store, which has the problem of affecting the beauty of the appearance. rice field. Further, the method of simply using image recognition for a captured image has a problem that it can be easily affected by changes in the shooting position, direction, distance, and surrounding conditions.

Japanese Unexamined Patent Publication No. 2014-44442 JP-A-2009-251774

In view of the above problems, it is an object of the present invention to provide a design body recognition information providing system that allows a customer to easily acquire information contents in a short time.

INDUSTRIAL APPLICABILITY The present invention includes design body image data which is a design body image in which at least a part of a design body having a design representing a character or a figure is imaged, and design body position data indicating an installation position where the design body is installed. A storage means for storing design body data associated with information content-related data for displaying information content, and a captured image for acquiring captured image data in which at least a part of the design body is contained in one captured image. The acquisition means, the imaging position information acquisition means for acquiring the imaging position information indicating the imaging position where the captured image is captured, and the installation position and the imaging position of the design body position data are within a predetermined positional relationship range. The design and / or the entire image in the captured image of the design body data extracted by the in-range design body data extraction means for extracting the design body data and the in-range design body data extraction means match or are similar to each other. The matching design body data extracting means for extracting the design body data having the design body image data and the information content based on the information content-related data possessed by the design body data extracted by the matching design body data extracting means are displayed. It is characterized by being a design body recognition information providing system equipped with an information display means.

According to the present invention, it is possible to provide a design body recognition information providing system that allows a customer to easily acquire information contents in a short time.

System configuration diagram of the design body recognition information provision system. Data structure diagram of design body data. Screen configuration diagram of the design body data registration screen. Screen configuration diagram of the design data search condition registration screen. Screen configuration diagram of the judgment body image data list screen. Screen configuration diagram of the judgment result log screen. Screen configuration diagram of the design body search screen and the search result screen. Design body data registration flow diagram by design body data registration program. Design body data extraction flow diagram by design body data extraction program.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing a system configuration of the design body recognition information providing system 1 in this embodiment. As shown in FIG. 1, the design body recognition information providing system 1 mainly stores a design body registration terminal 2 used by a design body owner having at least a character or a design, and stores data and manages the stored data. It is provided with a management server 3 for performing the above and a search terminal 4 used by a customer who desires to search for a design body. The design registration terminal 2, the management server 3, and the search terminal 4 are connected to the Internet 5. The management server 3 and the search terminal 4 function as computers.

The design registration terminal 2 is composed of a control unit 20 which is composed of a CPU, a ROM, a RAM, and the like to execute various calculations and control operations as hardware elements, and is composed of a hard disk, a flash memory, and the like, and allows reading and writing of information. The storage unit 21 is composed of a LAN board, a WiFi unit, etc., and is connected to the Internet 5 by wire or wirelessly to transmit and receive data, and a touch panel, keyboard, mouse, press button, or a plurality of these. It includes an input unit 23 that is configured to receive input by contact operation, and a display unit 24 that is configured to include a liquid crystal display, an organic EL display, or the like and displays images such as characters and figures.

The management server 3 is composed of a CPU, a ROM, a RAM, and the like as hardware elements to execute various calculations and control operations, and a hard disk, a flash memory, and the like, and is a storage that allows information read / write. It is composed of a unit 31, a communication unit 32 that is composed of a LAN board, a WiFi unit, etc. and is connected to the Internet 5 by wire or wirelessly to send and receive data, and a touch panel, a keyboard, a mouse, a press button, or a plurality of these. It includes an input unit 33 that accepts input by a contact operation, and a display unit 34 that is composed of a liquid crystal display, an organic EL display, or the like and displays images such as characters and figures.

The storage unit 31 stores a design body data registration program 310 executed by the control unit 30, a design body data extraction program 311 and a design body database 312 that stores the design body data.

The search terminal 4 is composed of a CPU, a ROM, a RAM, and the like as hardware elements to execute various calculations and control operations, and a hard disk, a flash memory, and the like, and is a storage that allows information read / write. The unit 41 is composed of a LAN board, a WiFi unit, or the like, and is connected to the Internet 5 by wire or wirelessly to transmit / receive data. It includes an input unit 43 that accepts input, and a display unit 44 that is composed of a liquid crystal display, an organic EL display, or the like and displays images such as characters and figures.

FIG. 2 is a data structure diagram of the design body data 3121. As shown in FIG. 2, the design body data 3121 stored in the design body database 312 includes design body name data, store name data, telephone number data, address data, and design body position data (position data of the installation position of the design body). ), Display ready Takumi image data, Judgment ready Takumi image data, Judgment condition data, Judgment character string data (collection of judgment character data), Character judgment threshold data, Image judgment threshold data, WEB page setting data, and target range Have data.

The design body name data stores the name of the design body.
The store name data stores the name of the store where the design body is installed.
The telephone number data stores the telephone number of the store where the design body is installed.
The address data stores the address of the store where the design body is installed.

The design body position data can be said to store the latitude and longitude of the position where the design body is installed.
The design body image data ready for display stores an image of the design body for confirmation by a user or the like. Here, the design body is a design body that expresses characters, figures, and patterns by strokes, and refers to a design body that is used as a so-called signboard (especially a mark that is regarded as an identification mark). Specifically, various design bodies representing characters, figures and patterns such as metal signs, LED signs, fluorescent light signs, and printed signs are applicable. Since such a design body is created so as to be clearly visible to a person, the recognition accuracy can be improved by setting this design body as a target for image recognition.
The design body image data prepared for determination stores an image of the design body for determination by image recognition. This image may be one or more.

The determination condition data stores AND, which is a match when both the character string determination and the image determination match, and OR, which is a match when either the character string determination or the image determination matches. It should be noted that it may be a condition that only one of the character string determination and the image determination is used, if either one matches, the match is made, or if both match, the match is made.
The determination character string data stores a character string used for character determination, and is data obtained by summarizing the determination character data for each character for each character string.
The character determination threshold data stores a threshold value for determining a match for character determination.
The image determination threshold data stores a threshold value for determining a match for image determination. This threshold value can be a threshold value for determining that the design and / or the entire image in the captured image is a match.

The WEB page setting data stores the address (URL) of the WEB page.
The target range data stores a range on a map indicating in which range the imaging means (camera) for searching the design body should be present for hitting. This setting range is one closed area on two dimensions indicated by latitude and longitude, and is set as an appropriate area such as a circle, a quadrangle, an arbitrary polygon, or an area surrounded by a line. In addition, although it is configured to set the target range for each design body, it is not limited to this, even if the map of the earth is divided into multiple area groups and which area group belongs to is stored in the target range data. good. In this case, the area group is preset by appropriately dividing the area on the earth within the range of latitude and longitude. The division is not limited to latitude and longitude, and may be divided by address such as prefecture or municipality.

FIG. 3 is a screen configuration diagram of the design body data registration screen 201 displayed on the display unit 24 of the design body registration terminal 2.
The design body data registration screen 201 has a screen call tab 202 for reading various screens, a display design body image setting unit 203 for accepting designation of a design body image to be displayed on the search terminal 4, and a design body for receiving input of a design body name. The name input unit 204, the store name input unit 205 that accepts the input of the store name displayed by the design body, the WEB page setting unit 206 that accepts the setting of the URL that can access the homepage of the store displayed by the design body, and the WEB page. The link name input unit 207 that accepts the input of the notation when displaying the URL link set in the setting unit 206 on the search terminal 4, and the telephone number input unit that accepts the input of the telephone number that is the contact information of the store displayed by the design body. 208, an address input unit 209 that accepts input of the location address of the store displayed by the design body, a peripheral map 210 showing a map around the store, a store location pin 211 indicating the location of the store in the peripheral map 210, and a store location. The target range 212 indicating a predetermined range from the pin 211, the latitude / longitude input unit 213 that accepts input in the form of latitude and longitude for the location of the store displayed by the design, and the search position range of the searcher who searches the store. It is provided with a target range setting unit 214 that accepts the setting of.

When displaying the peripheral map 210, the position where many images are taken, the position where the design body is set, the position where the number of errors occurs frequently, or a plurality of these are displayed with different types of marks. In this case, the position where the image was taken, the position where the design body is set, and the position where the number of errors occur frequently are stored in the storage unit 31 of the management server 3, and are displayed on the peripheral map 210. If there is a corresponding item, the mark may be displayed. Different types of marks may be of different types, for example, due to differences in color, shape, or both. In addition, height information such as the number of floors in the building and whether it is above ground or underground may be displayed. Thereby, it is possible to refer to whether the target range 212 is widened or narrowed.

The design body data registration screen 201 of this embodiment is configured to be published on the WEB page and to accept each design body data input via the Internet 5. The registrant who registers the design body data uses the design body registration terminal 2 to access the WEB page displaying the design body data registration screen 201 through the Internet 5. At this time, the registrant acquires an account for using the design body data registration function in advance, or creates an account and logs in when he / she first accesses the WEB page displaying the design body data registration screen 201. The logged-in registrant can call the design body data registration screen 201 from the screen call tab 202.

The screen call tab 202 causes the display unit 24 of the design body registration terminal 2 to display various screens according to the input and selection items received by accepting the input and selection. The various screens are, for example, a design body data registration screen 201, a registered design body data list display screen, a user information screen which is registrant information, and a determination when image recognition is performed. This is the result list screen.

The display design body image setting unit 203 accepts the setting of the display design body image to be displayed on the search terminal 4 as the extraction result. As a method of setting the display design body image, for example, it is set by reading the image stored in the storage unit 21 of the design body registration terminal 2. The display design body image is preferably an image that can confirm the appearance of the design body and the store in which the design body is installed.

The design body name input unit 204 accepts the input of the design body name which is the name of the design body data 3121 registered by the registrant.

The store name input unit 205 accepts the input of the store name to be displayed on the search terminal 4 as the extraction result. That is, it is preferable that the store name set here is a store name that allows the searcher to easily identify the store at a glance. In addition, if you own multiple branches such as stores that are chained all over the country, you can accept the input of the branch name where the design you are trying to register is installed in addition to the store name.

The WEB page setting unit 206 accepts the setting of the URL link for guiding to a specific WEB page to be displayed on the search terminal 4 as the extraction result. That is, the URL link set here is the access guidance link to the homepage managed by the store received by the store name input unit 205 or the access guidance link to the distribution site page that distributes the store introduction video. This is a URL link to lead to a WEB page related to the store.

When displaying the WEB page setting unit 206 on the search terminal 4, the link name input unit 207 accepts the input of the link name which is the display name when the WEB page setting unit 206 is replaced with the character string of the URL and displayed as a hyperlink.

The surrounding map 210 displays the peripheral map 210 of the store. At this time, the setting of the store location is accepted by using a means for moving a point on the screen such as dragging the designation of the store location with respect to the store location pin 211. Further, the target range 212 indicates a predetermined range from the store location pin 211 set by the target range setting unit 214 in a circle centered on the store location pin 211. The store location pin 211 may be automatically set based on the address input by the address input unit 209.

The latitude / longitude input unit 213 accepts input in latitude and longitude for the position of the design body. As for the latitude and longitude received by the latitude / longitude input unit 213, it is preferable to accept input up to the sixth decimal place because the position of the design body (location of the store) is made clearer. The latitude / longitude input unit 213 automatically inputs the latitude / longitude of the address when the address is input by the address input unit 209.

The target range setting unit 214 sets the position range of the search terminal 4 for the registered design body to be the search target (that is, the position range of the search terminal 4 capable of searching the design body) at the position (latitude) of the design body. Accept by setting the radius distance from (longitude). The target range is set as the radius of the circle centered on the position of the design body. The accepted target range is reflected as the target range 212 on the surrounding map 210.

The position range setting by the target range setting unit 214 is not limited to the setting based on the radial distance, and may be set by the registrant by designating one closed area of an arbitrary shape on the two-dimensional map. good. In this case, every time the registrant specifies a position on the 2D map, a straight line is drawn from the position specified immediately before to the position specified this time, and when the position specified first is specified again. , The area drawn so as to connect the points with a line is closed, and one target range 212 is completed. The line connecting the points is preferably a straight line, but may be a curved line. The control unit 20 of the design body registration terminal 2 that executes the process of performing this setting functions as a target range setting means.

FIG. 4 is a screen configuration diagram of the design body data search condition registration screen 221 displayed on the display unit 24 of the design body registration terminal 2.
The design data search condition registration screen 221 has a judgment condition selection unit 222 that accepts selection of whether the judgment condition is AND or OR, and a use that accepts a setting of a target to be used for recognition from characters and / or images. Judgment setting unit 223, judgment character string input unit 224 that accepts input of character string used when making judgment by character, and design body data being input as a result of judgment by character are required to be extracted. A character determination threshold input unit 225 that accepts a matching degree threshold input, and an image determination threshold input unit 226 that accepts a matching degree threshold input necessary for extracting design data being input as a result of image determination. , Remarks input unit 227 that accepts the input of special matters about the design body data being input, and save button 228 that saves the items input and set on the design body data registration screen 201 and the design body data search condition registration screen 221. , The design body data registration screen 201 and the cancel button 229 for canceling the items input and set on the design body data search condition registration screen 221 are provided.

The determination condition selection unit 222 determines that the image determination result and the character determination result are both extraction targets based on one captured image (captured image data) (when they are determined to match). Accepts the selection of whether to extract the design body data (AND) or to extract the design body data when either one is the extraction target (OR).

The usage determination setting unit 223 accepts selection from image recognition and / or character recognition for items used for determination. The image recognition means, for example, collating one captured image (captured image data) with the registered determination ready design body image 237, and if the degree of matching is equal to or higher than a certain level, the determination ready design body image 237 is used. It is a recognition method to extract the associated design body data. Character recognition is, for example, identifying characters included in one captured image, collating them with a registered judgment character string, and performing design body data associated with a judgment character string having a certain degree of matching or more. It is a recognition method to extract.

The judgment character string input unit 224 accepts one or more input of a character string used for character judgment. In the design body data in which a plurality of character strings are registered, the design body data is the extraction target when it is determined to be the extraction target in any one of the plurality of character strings. That is, there are many opportunities to be extracted.

The character determination threshold input unit 225 has a degree of coincidence between the character string included in the captured image and the determination character 246, which is a threshold for determining the design body data being input to be the determination target, from 0 to 100 in percentage units. Accepts numerical input of. That is, when the character string recognized to be included in the captured image and the character string received by the determination character string input unit 224 are collated, if the degree of matching is equal to or higher than the numerical value input here, the design being input is being input. Body data is extracted.

The image judgment threshold input unit 226 is a numerical value of 0 or more and 100 or less in the unit of percentage with respect to the degree of coincidence between the captured image and the design preparation image 237, which is a threshold for determining the design body data being input. Accept input. That is, when the captured image and the determination ready design body image 237 are collated, if the degree of matching is equal to or higher than the numerical value input here, the design body data being input is extracted. It should be noted that the image determination threshold value input unit 226 can be set to a plurality of units, such as two. In this case, a plurality of types of image recognition engines for image determination may be mounted, and the image determination threshold value 249a may be stored as many as the number of the types. Further, in this case, it is advisable to add to the determination condition selection unit 222 the selection of which image recognition engine to use or whether to use a plurality of image recognition engines. In this case, depending on the design body, an image recognition engine suitable for image recognition of the design body can be used. At the time of actual search, image recognition is performed by all image recognition engines, and if it exceeds the threshold in the image recognition engine or character recognition set for each design body, it is judged as a match and extracted, resulting in a time lag. It can be determined and extracted as a match by a determination method suitable for the design body.

Remarks Input unit 227 accepts input of memos related to the design body data being input. The input contents of the accepted remarks are displayed when the design body data is displayed in detail.

Upon receiving the press of the save button 228, the design body registration terminal 2 associates the input data of each input item into the design body data 3141 and sends it to the management server 3.

FIG. 5 is a screen configuration diagram of the determination ready design body image data list screen 231 displayed on the display unit 24 of the design body registration terminal 2.
The judgment-prepared design body image data list screen 231 is displayed by accepting an input for displaying a list of judgment-ready design body images 237 registered for one design body data from the registered design body data list screen. .. The determination-prepared takumi image data list screen 231 includes a determination-prepared takumi image data addition unit 232, a determination-prepared takumi image data deletion unit 233, and a determination-ready takumi image data detailed display unit 234.

Judgment preparation Takumi image data addition unit 232 accepts the setting of the judgment preparation Takumi image 237 used when performing character determination or image determination. The set determination-ready design body image data is stored as design body data 3121 in association with each data. As a method of setting the determination body image 237, for example, it is set by reading the image stored in the storage unit 21 of the design body registration terminal 2. At this time, it is preferable to register a plurality of determination-prepared design body images 237 obtained by photographing the design body from various angles as the determination-prepared design body image data. Further, in the determination ready design body image 237, it is preferable that the design representing the character or the figure included in the design body is contained in the image by 50% or more, and it is more preferable that all the designs are contained in the image. In the image determination, it is preferable to compare all the image data prepared for the determination with the image captured by the user and adopt the one having the highest degree of matching as the determination result.

The judgment-prepared design body image data detail display unit 234 includes a selection check unit 235 as an input item, an image ID 236 as a display item, a judgment-prepared design body image 237 for displaying the design body image, and a date when the design body image is registered. It has a determination preparation image registration date 238.

The judgment-prepared design body image data deletion unit 233 deletes the judgment-ready design body image data checked by the selection check unit 235 by being selected by clicking or the like.

FIG. 6 is a screen configuration diagram of the determination result log screen 241.
The judgment result log screen 241 displays on the map the judgment result log selection check unit 242 that accepts the selection of the judgment result log, the captured photograph 243 that is the determined photograph, and the position (latitude / longitude or address) where the photograph was taken. The image pickup position display 244, the judgment time 245, which is the time required for the judgment, the judgment character 246 recognized as a character included in the photograph and used for the judgment, the judgment result 247, and the extracted design body data. It includes a design body data 248, an image recognition result 249 showing a detailed determination result of image recognition, and a character recognition result 250 showing a detailed determination result of character recognition.

The determination character 246 displays a character string included in the captured photograph 243 used for the determination, that is, a character string recognized by character recognition (OCR). In addition, it displays in detail which part of the captured photograph 243 the characters recognized by accepting the detailed display link access are captured. In this example, two character strings, the first character string "DAIKAN" and the second character string "AIKAN", are recognized. When a plurality of character strings are recognized in this way, whether or not they match the registered character strings is compared with all the character strings, and the highest degree of matching is used as the determination result.

The determination result 247 has a comprehensive determination result 247a, an image determination result 247b, and a character determination result 247c.

The image determination result 247b displays the final image determination result based on the image recognition result 249. That is, if the numerical value of the degree of coincidence with the judgment-prepared design body image 237 of the captured photograph 243 is higher than the numerical value of the image judgment threshold value 249a input by the image judgment threshold value input unit 226, it is an extraction target and is displayed as "OK".

The character determination result 247c displays the final character determination result based on the character recognition result 250. That is, if the numerical value of the degree of matching of the determination character 246 with respect to the determination character string input unit 224 is higher than the numerical value of the character determination threshold input by the character determination threshold input unit 225, it is an extraction target and is displayed as "OK".

The comprehensive determination result 247a displays the overall determination result based on the image determination result 247b, the character determination result 247c, and the determination condition selection unit 222. That is, when the AND search is selected by the determination condition selection unit 222, "OK" is displayed when it is determined as the extraction target in both the image determination result 247b and the character determination result 247c. On the other hand, when the OR search is selected by the determination condition selection unit 222, "OK" is displayed when it is determined to be the extraction target in either the image determination result 247b or the character determination result 247c.

The extracted design body data 248 displays the name of the extracted design body data. In addition, the name of the design body data is a link for displaying the detailed screen of the design body data. The registrant can confirm what kind of setting the extracted design body data is by accessing the link embedded in the name of the design body data.

The image recognition result 249 is an image judgment numerical value which is a numerical value of the degree of coincidence between the image judgment threshold value 249a for displaying the numerical value of the image judgment threshold value input by the image judgment threshold value input unit 226 and the judgment ready takumi image 237 of the captured photograph 243. It has 249b.

The character recognition result 250 is a character judgment numerical value which is a numerical value of the degree of coincidence between the character judgment threshold 250a for displaying the numerical value of the character judgment threshold input by the character judgment threshold input unit 225 and the judgment character string input unit 224 of the judgment character 246. It has 250b and.

FIG. 7A is a screen configuration diagram of the design body search screen 400 displayed on the display unit 44 of the search terminal 4.
The design body search screen 400 includes an imaging unit activation button 401 for activating the imaging unit 45 included in the search terminal 4 and an imaging unit activation icon 402.

The image pickup unit activation button 401 has a sentence 401a prompting the searcher to take a picture of the design body. When the searcher presses the image pickup unit start button 401, the search terminal 4 activates the image pickup unit 45 composed of a built-in camera or the like to shift to the image pickup mode. The captured image is transmitted to the management server 3 by the search terminal 4.

The image pickup unit activation icon 402 has the same function as the image pickup unit activation button 401 except that it does not have the sentence 401a. That is, when the searcher presses the image pickup unit activation icon 402, the search terminal 4 activates the image pickup unit 45 composed of the built-in camera or the like and shifts to the image pickup mode.

FIG. 7B is a screen configuration diagram of the extracted design result display screen 410 displayed on the display unit 44 of the search terminal 4.
The extracted design body result display screen 410 has the extracted design body data number display unit 411 showing the number of the extracted design body data 3121 as a result of the image determination and / or the character determination, and the items of the design body data 3121 to be extracted. It is provided with a design body data display unit 412 for displaying the above, and a re-shooting button 413 for activating the image pickup unit 45 included in the search terminal 4.

The design body data display unit 412 has display items such as a display design body image set by the display design body image setting unit 203, a store name and a branch name input by the store name input unit 205, and a telephone number input unit 208. It has a telephone number input in, an address input in the address input unit 209, and a link name input in the link name input unit 207 in which the URL link input in the WEB page setting unit 206 is embedded. This link name (URL link) is information content-related data, and the home page on which the link name is selected and displayed is the information content. In addition, the WEB page (information content) in the link name (URL link) may be directly displayed without displaying the extracted design result display screen 410.

When there are a plurality of extracted design body data 3121s, the design body data display unit 412 is displayed vertically side by side from the upper part of the display unit 44 of the search terminal 4 for each design body data 3121 for the plurality of design body data 3121s. .. At this time, it is preferable that the design body data 3121 having the position data closer to the position where the search terminal 4 has taken the image is displayed on the upper part of the display unit 44.

FIG. 7C is a screen configuration diagram of the design body search result screen 420 displayed on the display unit 44 of the search terminal 4.
The design body search result screen 420 has a sentence 421 indicating that the design body data 3121 to be extracted as a result of the search is not registered.

FIG. 8 is a flowchart at the time of design body data registration by the design body data registration program 310 executed by the control unit 30 of the management server 3.

The control unit 30 is set by the design body data 3121 input and set on the design body data registration screen 201 and the design body data search condition registration screen 221 from the design body registration terminal 2, and the judgment body image data addition unit 232. Acquire the determined ready-to-design body image data (step S1).

Subsequently, the control unit 30 sets the target range 212 for the design body position data possessed by the acquired design body data 3121 (step S2). The setting of the target range 212 can be set by specifying a radius and setting the area within the radius from the design body position, manually setting the area on the map by the registrant, or already registered. It can be set by an appropriate method such as calling and duplicating the target range 212. The setting of the target range 212 is performed for each design body.
The setting of the target range 212 is not limited to this, and the map is divided into a plurality of area groups, and the area of the area group is set as the target range 212 by designating which area group the design body exists in. May be. In this case, the area group can be a group divided into a plurality of areas on the map on the earth within a predetermined range. As a method for grouping an area group, for example, a method of dividing by a range of latitude and longitude, or a method of dividing by an area such as a city or a ward can be used.

Subsequently, the control unit 30 associates the design body data 3121 with the data of the target range 212 assigned in step S2 and stores it in the design body database 312 (step S3).

FIG. 9 is a flowchart of the design body data extraction flow by the design body data extraction program 311 executed by the control unit 30 of the management server 3.

The control unit 30 acquires a captured image containing an image of the design body from the search terminal 4 (step S11). In the image acquired at this time, it is preferable that the design representing the characters or figures included in the design body is contained in the image by 50% or more, and it is more preferable that all the designs are contained in the image.

The control unit 30 acquires the image pickup position information indicating the image pickup position of the search terminal 4 when the search terminal 4 acquires the image capture image (step S12). This imaging position is a position near the position information (position information of the installation position) where the imaged design body is installed.

The control unit 30 determines whether or not the image pickup position information acquired in step S12 is within the target range 212 of each design body (step S13). That is, whether or not the installation position of the design body and the imaging position are within the predetermined positional relationship range, whether or not the imaging position is located within the target range 212 set for each design body, or for each design body. Determines if the imaging position is located within the selected area group.

The control unit 30 extracts the design body data 3121 having a shooting range determined in step S13 that the imaging position information is within the target range 212 of the design body (step S14). That is, the design body data 3121 of the design body installed within a certain distance range from the imaging position is extracted. This fixed distance range is wider than the imaging distance when an attempt is made to capture the design body with a camera (imaging means) of a mobile terminal such as a smartphone so as to capture the design body at a certain size, and the design has the same shape or appearance. It is set narrower than the distance where the body is installed (for example, narrower than the shortest distance between multiple stores of the same brand where the unified signboard is installed).

The control unit 30 recognizes the character string included in the captured image acquired in step S11 by character recognition (OCR). After that, the determination ready design body image 237 included in the design body data extracted in step S14 is collated with the captured image, and the image matching degree is calculated. Further, the determination character string data included in the design body data extracted in step S14 is collated with the character string included in the recognized captured image, and the character string matching degree is calculated (step S15). An appropriate image matching program can be used to calculate the image matching degree, and an appropriate method such as calculating the matching degree by matching the scales and angles of both images as a monochrome binary image can be used. ..

The control unit 30 compares the character string matching degree and the image matching degree calculated in step S15 with the character judgment threshold and the image judgment threshold associated with the judgment preparation design body image 237 used for the calculation, and converts the character string matching degree and the image matching degree into the design body data 3121. When both matching degrees or one matching degree is a numerical value higher than the threshold value based on the associated determination condition data, the corresponding design body data is extracted as matching design body data (step S16).

The control unit 30 transmits all of the design body data 3121 extracted in step S16 to the search terminal 4 and displays it on the display unit 44 in the screen configuration of the extracted design body result display screen 410 (step S17). As a result, the URL link can be displayed, and the information content can be displayed by selecting the URL link.

When the captured image is newly acquired from the search terminal 4, the control unit 30 re-executes the captured image obtained from step S12 to step S17 (step S18).

With the above configuration, it is possible to provide the design body recognition information providing system 1 in which the customer can easily acquire the information content in a short time.
The control unit 30 acquires the design body data 3121 including the design body image data prepared for determination and the position data for the design body to be installed, and further, the captured image containing the design body from the search terminal 4 and the position information of the imaged position. And, from the design body data 3121 having the position data within the predetermined distance range from the position information of the imaging position, the design body data 3121 having the matching or similar judgment body image data by the character judgment or the image judgment is obtained. Extract. With this configuration, compared to the case where collation is performed using only character determination or image determination, the control unit 30 does not need to collate all the registered determination-prepared design body images 237, and the captured image is acquired. The time to extract the design body data can be shortened. In addition, compared to the case where collation is performed using only location information, it is less likely to extract another store that exists in the same shopping mall building, and the store information desired by the searcher is provided more reliably. can. Further, the searcher can confirm the design body data 3121 only by acquiring the captured image containing the design body by using the image pickup unit 45 of the search terminal 4, which is easier than the conventional information providing method. You can get information on. In addition, even if a brand with multiple stores registers the same signboard as a design body in all stores, the stores can be properly distinguished by the position of the design body, and the image recognition can be used correctly without being confused with other brands. Since it is recognized, it is possible to appropriately respond to the provision of different information content for each store within the brand.

Further, the control unit 30 acquires the determination character string data as the design body data 3121, recognizes the character string included in the captured image acquired from the search terminal 4, collates it with the determination character string data, and the character strings match or match. Design body data 3121 having similar determination character string data is extracted. With this configuration, the target design body data 3121 can be extracted more reliably than the determination by image recognition, and the searcher can more reliably obtain information about the captured design body.

Further, the design body data search condition registration screen 221 has a determination condition selection unit 222 that accepts selection of whether the determination condition is AND or OR, and a usage determination that accepts selection of the determination condition to be used from image determination and character determination. It has a setting unit 223. With this configuration, the registrant of the design body data 3121 can set the condition for extracting the design body data 3121 according to the search condition desired to be extracted according to the design of the design body to be registered. As a result, it is possible to appropriately recognize various design bodies by setting judgment conditions for increasing the recognition accuracy for each design body by making full use of character judgment and image judgment.

Further, the design body data 3121 is within the range including the installation position where the design body is installed, and within what range the position captured by the captured image is from the installation position, the search can be hit (extracted). It has a target range data indicating a target range 212 that determines (whether it is possible). Then, the control unit 30 extracts all the design body data 3121 having the target range 212 including the image pickup position of the acquired image, and executes character determination and image determination on the extracted design body data 3121. With this configuration, it is possible to limit the image data of the judgment-prepared body to be subjected to the character judgment and the image judgment in advance, and it is possible to shorten the time required for the judgment. This target range 212 is set to a narrow area in the central part of the city where there are many design bodies, and the target range 212 is set wide in the suburbs where there are few design bodies, or by setting an arbitrary range for each design body, which target range is used. Even if it is 212, it is possible to quickly and surely recognize the design body and display the information content with an appropriate amount of data. When the target range 212 is an area group, the design body data 3121 may have area group data indicating which of the preset area groups the design body belongs to.

Further, the design body data 3121 has threshold data indicating a threshold of similarity which is a judgment standard in image judgment and character judgment, and the control unit 30 collates the judgment preparation design body image 237 with the captured image and images. The degree of matching is calculated, and the character string matching degree is calculated by collating the judgment character string data included in the design body data with the character string included in the recognized image. With this configuration, the registrant can determine in detail the boundary line from which the design body data 3121 is to be extracted.

Further, the extracted design body result display screen 410 includes a URL link for accessing the WEB page related to the design body preset in the WEB page setting unit 206 as a display item of the design body data display unit 412. With this configuration, the searcher can obtain more detailed information about the store in which the design body displayed as the extraction result is installed by accessing the WEB page of the URL link destination if desired.

Further, since the character string threshold and the image threshold can be set for each design body and the determination condition can be set for each design body, it is possible to perform recognition of various design bodies in a form suitable for the design body. ..

The storage means in the present invention corresponds to the design body database 312 possessed by the storage unit 31 of the management server 3, and the same applies hereinafter.
The captured image acquisition means corresponds to the control unit 30 that executes step S11, and corresponds to the control unit 30.
The imaging position information acquisition means corresponds to the control unit 30 that executes step S12, and corresponds to the control unit 30.
The in-range design body data extraction means corresponds to the control unit 30 that executes step S14, and corresponds to the control unit 30.
The matching design body data extraction means corresponds to the control unit 30 that executes step S16, and corresponds to the control unit 30.
The information display means corresponds to the control unit 30 that executes step S17 and the display unit 44 of the search terminal 4.
The character recognition means corresponds to the control unit 30 that executes step S15, and corresponds to the control unit 30.
The position dividing means corresponds to the control unit 30 that executes step S2, but
The present invention is not limited to this embodiment and can be various other embodiments.

It should be noted that the present invention is not limited to the configuration of the above-described embodiment, and many embodiments can be obtained.
For example, in this embodiment, the control unit 30 of the management server 3 performs character determination and image determination, but the control unit 30 connects to the determination server connected to the Internet 5 with determination conditions, determination image data, and determination. The captured image may be transmitted, the determination server may perform each determination, and the determination result may be transmitted to the management server 3. Even with such a configuration, the same effect as that of the above-described embodiment can be obtained.

Further, in this embodiment, the extraction design body result display screen 410 including the design body data display unit 412 is displayed as the extraction result, but the WEB page relating to the design body preset by the WEB page setting unit 206 is displayed. It may be directly accessed and displayed.

Further, the design body search screen 400 (see FIG. 7A) may display a map 210 around the current location. In this case, it is advisable to indicate the position where the design body is installed on the surrounding map. As a result, the user can recognize the presence or absence of the design body on the map, and it can be used for the opportunity of shooting, the determination of the shooting direction, and the like, and it is also possible to encourage the user to search for the design body and encourage the use. When the position of the design body is displayed on the surrounding map, height information such as the number of floors of the building and the height above ground or underground may be displayed so as to be recognizable. In this case, even if the design body is not found indoors, it is possible to recognize at which height (floor) the design body exists, and it is possible to move to that height (floor) and take an image of the design body.

Further, the design body recognition information providing system 1 may provide the search terminal 4 with a maintenance mode. In the maintenance mode, the search terminal 4 may be configured to be able to transmit the captured image captured by the imaging unit 45, the position information, and an appropriate message to the management server 3. As a result, for example, when a part of the LED sign as a design body does not light up, it is possible to transmit an image captured by capturing the LED sign and a repair request message. As a result, it can be used for maintenance of the design body. Further, it may be provided with a function of notifying the management server 3 of an abnormality such as a part of the design body shown in the captured image being turned off. In this case, the necessity of maintenance such as repair can be grasped from the captured image captured by the user.

The present invention can be used in the industry of a design body recognition information providing system that performs image recognition of an captured design body image and provides specific information.

1 ... Design body recognition information providing system 2 ... Design body registration terminal 3 ... Management server 4 ... Search terminal 5 ... Internet 30 ... Control unit 31 ... Storage unit 44 ... Display unit 45 ... Imaging unit 312 ... Design body database 3121: Design body Data 201 ... Design body data registration screen 221 ... Design body data search condition registration screen 231 ... Judgment ready Takumi image data list screen 237 ... Judgment ready Takumi image 241 ... Judgment result log screen

Claims (6)

Displaying design body image data which is a design body image obtained by capturing at least a part of a design body having a design representing a character or a figure, design body position data indicating an installation position where the design body is installed, and information content. Information for storing design data associated with content-related data, and a storage means for storing
A captured image acquisition means for acquiring captured image data in which at least a part of the design body is contained in one captured image, and
An imaging position information acquisition means for acquiring imaging position information indicating an imaging position in which the captured image is captured, and an imaging position information acquisition means.
An in-range design body data extraction means for extracting the design body data in which the installation position of the design body position data and the imaging position are within a predetermined positional relationship range, and
A matching design body that extracts the design body data having the design body image data in which the design and / or the whole image in the captured image matches or is similar to the design body data extracted by the in-range design body data extraction means. Data extraction means and
A design body recognition information providing system including an information display means for displaying the information content based on the information content-related data possessed by the design body data extracted by the matching design body data extraction means. The design body data has threshold data indicating a threshold value of similarity which is a determination criterion for the extraction by the matching design body data extraction means.
The design body recognition information providing system according to claim 1, wherein the matching design body data extraction means extracts the design body data having the design body image data having a degree of similarity higher than the threshold value to the captured image. The positional relationship range is a target range including the installation position.
It has a target range setting means for setting the target range as one closed area on the map.
The design body recognition information according to claim 1 or 2, wherein the target range setting means is configured to set the target range by connecting points in order by specifying arbitrary points on a map in order. Offering system. The design body has at least one design representing a character, and the design body has at least one design.
The design body data has determination character data indicating the character, and has
The captured image is captured by containing at least a part of the characters.
A character recognition means for recognizing the character in the captured image is provided.
The design body recognition information according to claim 1, 2 or 3, wherein the matching design body data extracting means extracts the design body data having the determination character data matching or similar to the character recognized by the character recognition means. system. Displaying design body image data which is a design body image obtained by capturing at least a part of a design body having a design representing a character or a figure, design body position data indicating an installation position where the design body is installed, and information content. The design body data associated with the information content related data is stored by the storage means, and
The captured image data in which at least a part of the design body is contained in one captured image is acquired by the captured image acquisition means, and the image is acquired.
The imaging position information indicating the imaging position where the captured image is captured is acquired by the imaging position information acquisition means, and the image is acquired.
The design body data having the design body position data in which the installation position of the design body position data is within a predetermined distance range from the imaging position is extracted by the in-range design body data extraction means.
Of the design body data extracted by the in-range design body data extraction means, the design body data having the design body image data in which the design and / or the whole image in the captured image matches or is similar is extracted. Extracted by means,
A method for providing design body recognition information for displaying the information content based on the information content-related data possessed by the design body data extracted by the matching design body data extraction means by the information display means. Computer,
Displaying design body image data which is a design body image obtained by capturing at least a part of a design body having a design representing a character or a figure, design body position data indicating an installation position where the design body is installed, and information content. Information for storing design data associated with content-related data, and a storage means for storing
A captured image acquisition means for acquiring captured image data in which at least a part of the design body is contained in one captured image, and
An imaging position information acquisition means for acquiring imaging position information indicating an imaging position in which the captured image is captured, and an imaging position information acquisition means.
An in-range design body data extraction means for extracting the design body data having the design body position data in which the installation position of the design body position data is within a predetermined distance range from the imaging position.
A matching design body that extracts the design body data having the design body image data in which the design and / or the whole image in the captured image matches or is similar to the design body data extracted by the in-range design body data extraction means. Data extraction means and
A design body recognition information providing program that functions as an information display means for displaying the information content based on the information content-related data possessed by the design body data extracted by the matching design body data extraction means.

Images